VRAM is the single number that decides which AI models you can run. Here's the simple rule of thumb, a size-by-size table, and a calculator to check any model against your card.
When people ask what they need to run AI at home, the answer almost always comes down to one number: VRAM — the memory on your graphics card. It's the single biggest factor in which models you can run, because a model has to fit in memory to run at a usable speed. The good news is the rule is simple once you know it, and you don't need to guess — there's a free calculator that checks any model against your card. Here's how much VRAM you actually need, and why.
The rule of thumb
Here's the mental model. Model size is measured in parameters (billions, the 'B' in names like Llama-8B). To run one, its weights must fit in VRAM. At 4-bit quantization — the standard, near-lossless way to run local models — each billion parameters takes roughly 0.5–0.6GB. So an 8-billion-parameter model needs about 5GB for weights, plus overhead for context and the app, landing near the 8GB mark. Double the parameters, double the memory. That single relationship — half the parameter count in GB, plus headroom — lets you eyeball any model. Quantization is the lever that makes this practical: the same model at full 16-bit precision would need four times the memory.
VRAM needed by model size (4-bit)
7–8B
Model size
~8GB
VRAM to run
RTX 3060/4060
14B
Model size
~16GB
VRAM to run
RTX 4070 Ti/5070 Ti
32B
Model size
~24GB
VRAM to run
RTX 3090/4090/5090
70B
Model size
~48GB
VRAM to run
2× 24GB or A6000
Model size
VRAM to run
Example card
7–8B
~8GB
RTX 3060/4060
14B
~16GB
RTX 4070 Ti/5070 Ti
32B
~24GB
RTX 3090/4090/5090
70B
~48GB
2× 24GB or A6000
VRAM is the gatekeeper — a model has to fit in memory to run at a usable speed. · Unsplash
Don't forget context
The weights are the biggest piece, but not the only one. The context — how much text the model is holding in a conversation — also uses VRAM, and long contexts add up. A short chat needs little extra; a very long document or a lengthy back-and-forth can add several gigabytes. That's why it's smart to leave headroom rather than pick a model that fits exactly. The practical implication: if a model just fits at a short context, it may run out of memory on a long one. The VRAM calculator accounts for context, so you can see the real requirement for your use, not just the weights.
Quick answers
How much VRAM do I need to run an AI model?
At 4-bit quantization — the standard way to run local models — a model needs roughly half its parameter count in gigabytes, plus overhead. So an 8-billion-parameter model needs about 8GB of VRAM, a 14B needs ~16GB, a 32B needs ~24GB, and a 70B needs ~48GB (usually two cards). Context length adds more on top. As a quick rule: 8GB runs small models, 16GB runs mid-size, and 24GB runs the biggest that fit a single consumer card.
Can I run AI models without a lot of VRAM?
Yes, within limits. Small models (7–8B) run on 8GB cards, and even smaller ones (1–4B) run on very little. You can also run bigger models partly on system RAM (CPU offloading), though that's much slower. Quantization helps enormously — a 4-bit model uses a quarter of the memory of full precision. And Apple Silicon Macs use unified memory, so a Mac with lots of RAM can run large models. But for fast performance, the model needs to fit in VRAM.
Does context length affect VRAM usage?
Yes. Beyond the model's weights, the context — the text the model holds in a conversation — also consumes VRAM, and long contexts can add several gigabytes. A short chat needs little extra; a long document or extended back-and-forth needs much more. That's why you should leave headroom rather than pick a model that fits exactly at a short context. Our VRAM calculator factors in context length so you see the real requirement for your actual use.
That's the whole rule: at 4-bit, half the parameter count in GB plus headroom, and mind your context. Check any specific model against your card in the VRAM calculator, then match it to the best model for your card and the beginner's setup guide.
