aliteq.

Which Gemma, Llama, Mistral or gpt-oss model can you run on an 8 to 24GB GPU?

Google, Meta, Mistral and OpenAI all ship open models that fit a gaming card, if you pick the right size and know each family's trap. Which one fits 8, 12, 16 and 24GB, the quant to use, the Ollama tag to type, and what changes on a Mac, from our VRAM engine and the model cards.

Aliteq TeamPublished 8 min read
Can you run gpt-oss-20b? 16 GB runs it, next to a screenshot of the llama.cpp guide's minimum memory table and a card reading 15.5 GB for gpt-oss-20b at 32K context, read Nov 3, 2026

Some links on this page are referral links: we may earn a commission, at no cost to you.

Share

Which Gemma, Llama, Mistral or gpt-oss model fits your GPU? Pick by VRAM. On our VRAM engine at Q4_K_M with 16K context, an 8GB card runs Gemma 4 E4B (about 6 GB) or Llama 3.1 8B (about 7 GB), 12GB runs Gemma 4 12B (about 8 GB) or Phi-4 (about 12 GB, tight), 16GB runs gpt-oss-20b, which OpenAI says runs within 16GB, and 24GB runs Gemma 4 31B (about 20 GB), Mistral Small 24B or gpt-oss-20b with its full 128K context. Each family has one trap, mostly in Ollama's default tags, covered below.

Part of our guide to which local AI model fits your GPU's VRAM. The sibling guides cover every Qwen size and the giant open models that need server memory.

Gemma 4 E4B

8GB

Q4_K_M, about 6 GB

Gemma 4 12B

12GB

Q4_K_M, about 8 GB

gpt-oss-20b

16GB

15.5 GB at 32K (llama.cpp)

Gemma 4 31B

24GB

Q4_K_M, about 20 GB

Which model fits each VRAM tier?

Memory is our VRAM engine's estimate at Q4_K_M (gpt-oss at its own MXFP4) with 16K tokens of context, from each model's best-GPU page on Nov 17, 2026. For other quants and context lengths, run the VRAM calculator.

Gemma 4 E2B

Maker
Google
Memory at Q4_K_M, 16K
about 4 GB
Smallest card that fits
6 to 8GB
Ollama tag
gemma4:e2b

Llama 3.2 3B

Maker
Meta
Memory at Q4_K_M, 16K
about 4 GB
Smallest card that fits
6 to 8GB
Ollama tag
llama3.2:3b

Gemma 4 E4B

Maker
Google
Memory at Q4_K_M, 16K
about 6 GB
Smallest card that fits
8GB
Ollama tag
gemma4 (the default)

Llama 3.1 8B

Maker
Meta
Memory at Q4_K_M, 16K
about 7 GB
Smallest card that fits
8GB, tight
Ollama tag
llama3.1:8b

Gemma 4 12B

Maker
Google
Memory at Q4_K_M, 16K
about 8 GB
Smallest card that fits
12GB
Ollama tag
gemma4:12b

Gemma 3 12B

Maker
Google
Memory at Q4_K_M, 16K
about 9 GB
Smallest card that fits
12GB
Ollama tag
gemma3:12b

Phi-4 14B

Maker
Microsoft
Memory at Q4_K_M, 16K
about 12 GB
Smallest card that fits
12GB tight, 16GB comfortable
Ollama tag
phi4:14b

gpt-oss-20b

Maker
OpenAI
Memory at Q4_K_M, 16K
about 14 GB (MXFP4)
Smallest card that fits
16GB
Ollama tag
gpt-oss:20b (14 GB)

Gemma 4 26B-A4B

Maker
Google
Memory at Q4_K_M, 16K
about 16 GB
Smallest card that fits
24GB, or 16GB with Google's 14.4 GB QAT file
Ollama tag
gemma4:26b

Mistral Small 24B

Maker
Mistral
Memory at Q4_K_M, 16K
about 17 GB (a ~14 GB file)
Smallest card that fits
24GB, 16GB tight
Ollama tag
mistral-small

Gemma 3 27B

Maker
Google
Memory at Q4_K_M, 16K
about 18 GB
Smallest card that fits
24GB
Ollama tag
gemma3:27b

Gemma 4 31B

Maker
Google
Memory at Q4_K_M, 16K
about 20 GB
Smallest card that fits
24GB
Ollama tag
gemma4:31b

Llama's bigger models aren't on this list for a reason: Llama 3.3 70B needs about 46 GB and Llama 4 Scout about 63 GB, which is giant-model territory.

Which one should you run on your GPU?

8GB, Gemma 4 E4B or a small Llama

Gemma 4 E4B fits at Q6_K with 8K context in about 7.0 GB on our engine, with a little more room than the 8B-class models, and it's what Ollama's bare gemma4 tag pulls. Llama 3.1 8B needs about 7 GB at Q4_K_M with 16K context, so it fits but leaves little headroom. Gemma 3 4B is the lighter option and runs on 8GB laptops too. gpt-oss-20b doesn't fit 8GB on its own, but the llama.cpp guide shows about 42 tokens a second on an 8GB RTX 4060 Laptop GPU with --n-cpu-moe keeping part of the model in system RAM. More 8GB picks in the best local LLM for 8GB of VRAM.

12GB, Gemma 4 12B or Phi-4

Gemma 4 12B is the pick: about 7.7 GB at Q4_K_M with 8K context, and Q6_K with 32K context (about 10.4 GB) still fits a 12GB card like the RTX 3060 12GB. Its sliding-window layers keep long chats cheap. Phi-4 (Microsoft, MIT licence) needs about 12 GB at Q4_K_M, a tight fit, and it shows its late-2024 age with a 16K context. Mistral Nemo 12B, run as ollama run mistral-nemo, is the Mistral option for this tier. See the best local LLM for 12GB.

16GB, gpt-oss-20b

This is the tier gpt-oss-20b was made for. OpenAI's model card says MXFP4 makes it "run within 16GB of memory", and the llama.cpp guide lists 14.9 GB at 8K context, 15.5 GB at 32K and 17.9 GB at the full 128K. Ollama's gpt-oss:20b is a 14 GB download. It ships in OpenAI's own 4-bit format, so there's no point hunting for an 8-bit build. On an RTX 5060 Ti 16GB the alternatives are Gemma 4 12B at Q8_0 (about 12.7 GB) or Gemma 4 26B-A4B using Google's 14.4 GB QAT Q4_0 file. See can you run gpt-oss-20b on your GPU and the best local LLM for 16GB.

24GB, Gemma 4 31B, Gemma 4 26B-A4B or Mistral Small 24B

A 24GB RTX 3090 or RTX 4090 opens up the bigger dense models:

  • Gemma 4 31B at Q4_K_M, about 19.1 GB with 8K context, the strongest Gemma that fits.
  • Gemma 4 26B-A4B at Q6_K, a mixture-of-experts model with about 3.8B active parameters, so it answers faster than the 31B.
  • Mistral Small 24B (v3.2), a dense 24B model with a ~14 GB Q4_K_M file. Community benchmarks compiled by Markaicode put an RTX 4090 at about 38 tokens a second in llama.cpp at Q4_K_M. Q8 needs about 25 GB, so it doesn't fit.
  • gpt-oss-20b with its full 128K context, 17.9 GB per the llama.cpp guide.
  • Gemma 3 27B, about 18 GB at Q4, if you're already set up on Gemma 3.

For text and code only, Qwen3.8 27B is the other strong 24GB choice; we compare all of them in the best model for a 24GB card.

On a Mac

A Mac's GPU shares one pool of memory with macOS. The llama.cpp guide says gpt-oss-20b "can run efficiently on Macs with at least 16GB RAM", but a 16GB Mac needs --n-cpu-moe 12 and leaves only 1 to 2 GiB for everything else at 8K to 32K context. A 24GB Mac fits it fully. For the runtime, LM Studio is the app, Ollama the one-command option, mlx-lm the Python route and llama.cpp the one with every setting. See gpt-oss-20b on a 16GB Mac mini and MLX or llama.cpp on a Mac.

Each family's trap, and how to avoid it

  1. Gemma 4 in Ollama. The bare gemma4 tag is the small E4B. Type gemma4:12b, gemma4:26b or gemma4:31b for the bigger ones. Audio input works only on E2B, E4B and 12B, and not through Ollama.
  2. Short default context. Ollama's docs say cards under 24 GiB get a 4K context by default. For documents or code, raise it with /set parameter num_ctx 32768 inside a session, then check ollama ps.
  3. gpt-oss quants. It ships in MXFP4, so smaller GGUF quants don't shrink it much. Fit it by context length instead.
  4. Mistral sizes. ollama run mistral is the 7B, mistral-nemo the 12B and mistral-small the 24B. Several Mistral models are Apache 2.0, so commercial use is allowed. See how to run Mistral locally.
  5. Phi-4's context. It's 16K and full attention, so long chats cost more memory than on newer models. See Phi-4 locally.

Should you rent a 24GB card first?

If you're on 8 to 16GB and wondering whether Gemma 4 31B or Mistral Small 24B is worth an upgrade, an hour on a rented 24GB card answers it. On our price feed (Nov 17, 2026) a 24GB RTX 3090 started at $0.12 an hour on Vast.ai and an RTX 4090 at $0.27; Runpod listed them at $0.50 and $0.89. Prices move hourly; today's are on our cloud GPU page, and rent vs buy shows when owning wins.

Every guide on Gemma, Llama, Mistral and gpt-oss

gpt-oss-20b

Gemma

Mistral

Llama, Phi and Mac runtimes

Related guides: What's the best local LLM for a 32GB RTX 5090, Got an Intel Arc Card? Here's Every Local AI Model It Fits (2026), Is the DGX Spark worth it in November 2026.

This guide is one part of our Local AI section; start with how to run a local AI model on your own PC in 2026 — a genuinely simple beginner's guide for the full picture.

Gemma, Llama, Mistral and gpt-oss locally, quick answers

What is the best open model for a 16GB GPU?
gpt-oss-20b. OpenAI's card says it runs within 16GB of memory, the llama.cpp guide lists 15.5 GB at 32K context, and Ollama's gpt-oss:20b is a 14 GB download. Gemma 4 12B at Q8_0 is the dense alternative.
Which Gemma 4 size fits my GPU?
E4B on 8GB, 12B on 12GB at Q4 or 16GB at Q8, 26B-A4B on 24GB (16GB only with Google's 14.4 GB QAT file), and 31B on 24GB at Q4_K_M, about 19.1 GB with 8K context. All five sizes are Apache-2.0.
Can I run gpt-oss-20b on an 8GB GPU?
Yes, slower. llama.cpp's --n-cpu-moe setting keeps part of the model in system RAM; a community result in the llama.cpp guide shows about 42 tokens a second on an 8GB RTX 4060 Laptop GPU.
How much VRAM does Mistral Small 24B need?
Its Q4_K_M file is about 14 GB, and our engine puts it at about 17 GB with 16K context, so a 24GB card is the comfortable fit and 16GB is tight. Q8 needs about 25 GB.
Which Llama runs on an 8GB GPU?
Llama 3.1 8B at Q4_K_M, about 7 GB with 16K context on our engine, or Llama 3.2 3B at about 4 GB. Llama 3.3 70B and Llama 4 need 46 GB or more.
Why does Ollama give me the wrong Gemma or a short context?
The bare gemma4 tag pulls the small E4B, so name the size, for example gemma4:31b. And Ollama's docs say cards under 24 GiB get a 4K context by default; raise num_ctx for long documents or code.

Found this useful? Share it

Share

Aliteq

Aliteq Team

Aliteq started in 2019 as a PC hardware shop and is now a US-based publication. We build the VRAM and cost calculators on this site, track live cloud-GPU prices, and write guides with AI-assisted drafting that we check against primary sources before publishing.

Work out the hardware

GPUs in this article

Specs, the AI models each one runs, and where to buy or rent it.

Email list

Local AI, only the changes that matter

A new model your GPU can actually run, a GPU price move worth acting on, or a fix that saves you an evening. Nothing in between.

Written and sent by hand. One-click unsubscribe in every email. Emails may contain referral links.

Keep reading