8GB gets E4B, 12GB gets 12B, 16GB gets 12B at Q8 (or 26B-A4B with Google's own 4-bit file), and 24GB gets 26B-A4B or 31B. The VRAM for each size and quant, the exact Ollama tag (the bare one pulls a small model), the 4K-context default that catches Ollama users on cards under 24GB, and when Qwen3.8 27B is the better pick.
Some links on this page are referral links: we may earn a commission, at no cost to you.
Share
Updated Nov 3, 2026: every spec, Ollama tag and file size below was re-read on Google's and Ollama's own pages today. New in this update: Ollama's 4K context default on cards under 24 GiB, Google's own llama.cpp commands, and QAT files for all five sizes. This guide replaced our three separate 12B, 26B-A4B and 31B pages on Nov 1, 2026.
Which Gemma 4 fits your GPU? Pick by VRAM. On an 8GB card run Gemma 4 E4B. On 12GB run Gemma 4 12B at Q4, and on 16GB run the same 12B at near-lossless Q8_0. On 24GB choose between Gemma 4 26B-A4B, a mixture-of-experts model that reads only 3.8B parameters per token and so answers quickly, and Gemma 4 31B, the dense one that scores highest on Google's own benchmarks. Two traps catch almost everyone: ollama run gemma4 pulls the small E4B, and Ollama gives cards under 24 GiB only 4K tokens of context unless you raise it.
Which Gemma 4 size fits my GPU?
The table below is the answer for each card size. VRAM is from our VRAM engine at 8K tokens of context: the weights at each quant's measured bits per weight, the KV cache in fp16 and about 0.8 GB of runtime overhead. "Tight" means over 90% of the card.
E2B
Weights in Google's checkpoint
5.1B (2.3B effective)
Q4_K_M, 8K
~3.7 GB
Q8_0, 8K
~5.9 GB
Card it fits
4 to 6GB, laptops
Pick it for
Offline helpers, quick drafts, phones
E4B
Weights in Google's checkpoint
8.0B (4.5B effective)
Q4_K_M, 8K
~5.4 GB
Q8_0, 8K
~8.8 GB
Card it fits
8GB
Pick it for
The best Gemma for an 8GB card
12B
Weights in Google's checkpoint
11.96B, dense
Q4_K_M, 8K
~7.7 GB
Q8_0, 8K
~12.7 GB
Card it fits
12GB at Q4, 16GB at Q8
Pick it for
Text, images and audio on a mid-range card
26B-A4B
Weights in Google's checkpoint
25.8B, 3.8B active (MoE)
Q4_K_M, 8K
~15.5 GB (tight on 16GB)
Q8_0, 8K
~26.5 GB
Card it fits
24GB; 16GB with the QAT file
Pick it for
Fast replies, long chats, writing
31B
Weights in Google's checkpoint
31.3B, dense
Q4_K_M, 8K
~19.1 GB
Q8_0, 8K
~32.4 GB
Card it fits
24GB at Q4, 32GB for Q6 and up
Pick it for
The strongest Gemma on one card
Weights in Google's checkpoint
Q4_K_M, 8K
Q8_0, 8K
Card it fits
Pick it for
E2B
5.1B (2.3B effective)
~3.7 GB
~5.9 GB
4 to 6GB, laptops
Offline helpers, quick drafts, phones
E4B
8.0B (4.5B effective)
~5.4 GB
~8.8 GB
8GB
The best Gemma for an 8GB card
12B
11.96B, dense
~7.7 GB
~12.7 GB
12GB at Q4, 16GB at Q8
Text, images and audio on a mid-range card
26B-A4B
25.8B, 3.8B active (MoE)
~15.5 GB (tight on 16GB)
~26.5 GB
24GB; 16GB with the QAT file
Fast replies, long chats, writing
31B
31.3B, dense
~19.1 GB
~32.4 GB
24GB at Q4, 32GB for Q6 and up
The strongest Gemma on one card
Parameter counts are the safetensors totals on Google's Hugging Face repos (re-read Nov 3, 2026), vision and audio parts included, and they are the counts our VRAM engine uses. Try your own quant and context length in the VRAM calculator; each size also has a page with live rental prices: E4B, 12B, 26B-A4B, 31B.
Google's own size table on the Gemma 4 12B model card (huggingface.co/google/gemma-4-12B), read Nov 3, 2026. The 31B figure here is the language model alone; the repo total with the vision encoder is 31.3B. · Screenshot of huggingface.co by aliteq
Should I pick Gemma 4 or Qwen3.8 27B?
Pick Qwen3.8 27B if you only need text and code and want the strongest answers that fit 16 to 24GB; pick Gemma 4 for audio, speed, small hardware or its writing style. In November 2026 Gemma 4 is no longer the model enthusiasts reach for first. When a r/LocalLLaMA thread ranked models for 10GB to 16GB of VRAM, the top reply put Qwen 3.8 27B first. Qwen's model card describes a dense 27B vision-language model that takes images and video, under Apache-2.0; on a 24GB card it runs at Q5_K_M with 32K of context on our numbers, and on 16GB only at a 3-bit quant (full VRAM table). For a small card with a lot of system RAM, people talk about Qwen3.8 Flash Next, a mixture-of-experts model Qwen lists at "125B with 6B activated" (plus a 51B n-gram embedding), which offloading engines run partly from RAM.
So who should still pick Gemma 4?
You want one licence you don't have to read twice. Every Gemma 4 size is Apache-2.0 on its Hugging Face card. Qwen3.8 27B is Apache-2.0 too, but Qwen3.8 Flash Next uses a custom "qwen-community-1.0" licence.
You need audio, or a model that fits small hardware. Google's card says audio is supported on E2B, E4B and 12B, with clips up to 30 seconds. Qwen3.8 27B takes text, images and video. Gemma 4 also spans phone to 24GB card in one family.
You want speed on 16 to 24GB. 26B-A4B reads 3.8B parameters per token; a dense 27B reads all 27B. On the same card the MoE answers faster. Qwen3.6 35B-A3B is the other fast MoE in this class (24GB guide).
Writing and companion apps. On Reddit, people building companion and roleplay apps and doing creative writing name Gemma 4 26B for its writing style. That is taste, not a benchmark, but it comes up often.
Tooling that just works. Ollama, llama.cpp and LM Studio all run it, and Google publishes its own quantization-aware (QAT) Q4_0 GGUF files for all five sizes, so you don't have to trust a stranger's quant.
Gemma 5 has not been announced. Google's release notes on gemma.ai (read Nov 3, 2026) list Gemma 4 12B Unified on June 3, 2026 as the last Gemma 4 release, and the only Gemma release after it is EmbeddingGemma 2 on October 6, 2026. Reddit threads about Gemma 5 (one, two) are wish lists, not leaks.
What are the Gemma 4 sizes, and what's the difference?
Gemma 4 family (Google's model cards and config.json, re-read Nov 3, 2026)
E2B
Active per token
2.3B effective
Layers (sliding / full)
35, 512-token window
Context
128K
Input
Text, image, audio
E4B
Active per token
4.5B effective
Layers (sliding / full)
42, 512-token window
Context
128K
Input
Text, image, audio
12B
Active per token
All 11.96B (dense)
Layers (sliding / full)
48 (40 / 8)
Context
256K
Input
Text, image, audio
26B-A4B
Active per token
3.8B (8 of 128 experts + 1 shared)
Layers (sliding / full)
30 (25 / 5)
Context
256K
Input
Text, image
31B
Active per token
All 31.3B (dense)
Layers (sliding / full)
60 (50 / 10)
Context
256K
Input
Text, image
Active per token
Layers (sliding / full)
Context
Input
E2B
2.3B effective
35, 512-token window
128K
Text, image, audio
E4B
4.5B effective
42, 512-token window
128K
Text, image, audio
12B
All 11.96B (dense)
48 (40 / 8)
256K
Text, image, audio
26B-A4B
3.8B (8 of 128 experts + 1 shared)
30 (25 / 5)
256K
Text, image
31B
All 31.3B (dense)
60 (50 / 10)
256K
Text, image
Google's release notes date the family to March 31, 2026 (E2B, E4B, 26B-A4B and 31B) and June 3, 2026 (12B, which Google calls "12B Unified" because it has no separate vision or audio encoder). On April 16, 2026 Google added multi-token prediction (MTP) versions of E2B, E4B, 26B-A4B and 31B. The "E" in E2B and E4B stands for effective parameters: per-layer embedding tables add billions of parameters that are only looked up, so the effective count is what does the work, but the whole file still has to load.
On Google's own instruction-tuned benchmarks the order is what you would expect: MMLU Pro 85.2% for 31B, 82.6% for 26B-A4B and 77.2% for 12B; LiveCodeBench v6 80.0%, 77.1% and 72.0%; GPQA Diamond 84.3%, 82.3% and 78.8%. Those are the vendor's numbers, not ours. They show 26B-A4B sitting close to 31B for a fraction of the per-token compute.
Does long context fit too?
On the 12B and 26B-A4B, yes, almost for free; on the 31B it costs real VRAM. Five of every six layers in those three models use a sliding window of 1,024 tokens, so their part of the KV cache stops growing once the conversation passes that length. Only the full-attention layers keep a cache that grows with every token, and the 31B's 10 full-attention layers carry more key/value heads:
Q4_K_M VRAM as context grows (our VRAM engine)
Gemma 4 12B
8K
~7.7 GB
32K
~8.0 GB
128K
~9.5 GB
Gemma 4 26B-A4B
8K
~15.5 GB
32K
~16.0 GB
128K
~17.9 GB
Gemma 4 31B
8K
~19.1 GB
32K
~20.9 GB
128K
~28.4 GB
8K
32K
128K
Gemma 4 12B
~7.7 GB
~8.0 GB
~9.5 GB
Gemma 4 26B-A4B
~15.5 GB
~16.0 GB
~17.9 GB
Gemma 4 31B
~19.1 GB
~20.9 GB
~28.4 GB
So the 256K context is genuinely usable on the 12B and 26B-A4B, while on a 24GB card the 31B at Q4_K_M gets tight past about 40K tokens and stops fitting past about 70K. Our old 31B page said its context "barely adds" to VRAM; on our own numbers that was wrong for the 31B, and this guide corrects it.
Which Gemma 4 runs on an 8GB card or a laptop?
E4B. On our numbers it fits at Q6_K with 8K of context in about 7.0 GB, or at Q4_K_M in about 5.4 GB with room to spare, and Google's card lists text, image and audio input. E2B is smaller still (about 3.7 GB at Q4_K_M) and is built for phones and laptops. Google's QAT Q4_0 file for E4B is 5.2 GB plus a 1.0 GB vision file on Hugging Face (Nov 3, 2026).
Keep expectations honest on the vision side. In an r/ollama thread, a user who tried Gemma 4 E2B for offline bird identification reported that it "confidently got them wrong." Small multimodal models describe a photo well and fail on fine-grained identification, and they do it confidently. Use them to caption, sort and read, not to identify a species or a part number you will act on.
For the wider 8GB picture, including Qwen3.5 9B, see the 8GB guide.
Is Gemma 4 12B right for a 12GB or 16GB card?
Yes: it is the size most people should start with. Gemma 4 12B is a dense model with 11.96B parameters, a 256K context and, per Google's card, text, image and audio input.
16GB card: Q8_0, effectively lossless, with 32K of context. Our 16GB guide compares it with gpt-oss-20b and a 3-bit Qwen3.8 27B.
8GB card: not a clean fit. Ollama's gemma4:12b-it-q4_K_M is an 8.0 GB download, so it spills into system RAM. Google's QAT Q4_0 file is 7.0 GB (plus a 175 MB vision file) and can squeeze in with short context; E4B is the more comfortable choice.
Need audio? Not through Ollama. Google's own Ollama page says: "Ollama publishes Gemma 4 as text and image only. For audio input, use Transformers or llama.cpp." Ollama's tag list matches: every gemma4 tag, E4B and 12B included, says "Text, Image" (Nov 3, 2026).
Will Gemma 4 26B-A4B fit my card?
On 24GB, comfortably; on 16GB, only with Google's QAT file. 26B-A4B is a mixture-of-experts model: 25.8B parameters in the checkpoint, of which 3.8B are active per token (8 of 128 experts plus one shared expert, per Google's card). The active count sets the speed; the total sets the memory. All of it has to load, into VRAM or partly into system RAM.
Q4_K_M
8K context
~15.5 GB
32K context
~16.0 GB
Q6_K
8K context
~20.6 GB
32K context
~21.1 GB
Q8_0
8K context
~26.5 GB
32K context
~26.9 GB
BF16
8K context
~49.0 GB
32K context
~49.5 GB
8K context
32K context
Q4_K_M
~15.5 GB
~16.0 GB
Q6_K
~20.6 GB
~21.1 GB
Q8_0
~26.5 GB
~26.9 GB
BF16
~49.0 GB
~49.5 GB
24GB card (RTX 3090, RTX 4090): the natural home. Q6_K with 32K context fits, and long context stays cheap.
16GB card: our engine puts Q4_K_M right at the limit, and Ollama's own gemma4:26b-a4b-it-q4_K_M is an 18 GB download, so it will not fit on the card. The realistic route is Google's QAT Q4_0 file: 14.4 GB for the language model plus 1.2 GB for the vision projector. Text-only it leaves a little room for context; with images it is very tight. The other route is llama.cpp's --n-cpu-moe, which keeps some layers' experts in system RAM at a cost in speed (how it works).
12GB or less: only with heavy expert offload. On 12GB, the 12B is the better fit.
Google's own 4-bit QAT file for 26B-A4B is 14.4 GB, plus a 1.19 GB vision projector (huggingface.co/google/gemma-4-26B-A4B-it-qat-q4_0-gguf, read Nov 3, 2026). That is the file that makes a 16GB card possible. · Screenshot of huggingface.co by aliteq
Yes, at Q4_K_M with up to about 40K of context. Gemma 4 31B is the largest and strongest Gemma 4: 31.3B parameters in the checkpoint (Google's card lists 30.7B for the language model and about 550M for the vision encoder), dense, text and image input, 256K context.
Q4_K_M
8K context
~19.1 GB
32K context
~20.9 GB
Q4_0
8K context
~18.0 GB
32K context
~19.8 GB
Q6_K
8K context
~25.3 GB
32K context
~27.2 GB
Q8_0
8K context
~32.4 GB
32K context
~34.2 GB
8K context
32K context
Q4_K_M
~19.1 GB
~20.9 GB
Q4_0
~18.0 GB
~19.8 GB
Q6_K
~25.3 GB
~27.2 GB
Q8_0
~32.4 GB
~34.2 GB
24GB card: Q4_K_M with up to about 40K of context comfortably; it gets tight beyond that and stops fitting past about 70K. Google's QAT Q4_0 file (17.7 GB plus a 1.2 GB vision projector) leaves a little more room. For very long documents, 26B-A4B or a rented bigger card is the better tool.
32GB card (RTX 5090 class): Q5_K_M with 32K context (~24.0 GB) or Q6_K with 8K to 32K (~25.3 to 27.2 GB). See the 32GB guide.
16GB card: no. Use 26B-A4B or 12B; forcing a dense 31B through heavy offload is slow.
Being dense, the 31B reads every parameter for every token, so it generates more slowly than 26B-A4B on the same card. Choose it when answer quality matters more than speed, for example long-form analysis or describing complex images. The ranked card list is on best GPU for Gemma 4 31B.
Renting instead of buying? On our price feed (checked Nov 17, 2026), an on-demand RTX 4090 (24GB) started at $0.27 an hour on Vast.ai (median $0.49), and Runpod listed it at $0.89. Prices move every hour, so check today's cloud GPU prices before you rent, and see the cheapest RTX 4090 rentals for how the hosts compare.
How do I run Gemma 4 in Ollama, llama.cpp or LM Studio?
Ollama: name the size. On Ollama's gemma4 page, gemma4:e4b carries the "latest" label, so the bare gemma4 tag is the same build as E4B. Type ollama run gemma4 on a 24GB card and you get a model sized for 8GB:
ollama run gemma4:e4b # 8GB cards
ollama run gemma4:12b # 12-16GB cards
ollama run gemma4:26b # the 26B-A4B MoE (same build as gemma4:26b-a4b)
ollama run gemma4:31b # 24GB cards
Ollama's gemma4 tags, read Nov 3, 2026 (ollama.com/library/gemma4/tags). The outlined row: e4b is the one labelled latest, and it has the same ID as gemma4:latest. · Screenshot of ollama.com by aliteq
Ollama: raise the context. Ollama's docs say it sets the default context from your VRAM, not from the model: "< 24 GiB VRAM: 4k context", 24 to 48 GiB gets 32k, and 48 GiB or more gets 256k. So on an 8GB, 12GB or 16GB card, Gemma 4's 128K or 256K window is cut to 4K tokens until you change it, and a long chat or document quietly loses its beginning. Ollama recommends at least 64000 tokens for agents and coding tools. Raise it with OLLAMA_CONTEXT_LENGTH=32768 ollama serve, or per request with "options": {"num_ctx": 32768}, and check the bigger context still fits with the VRAM calculator.
Ollama's own default-context rule (docs.ollama.com/context-length), read Nov 3, 2026. · Screenshot of docs.ollama.com by aliteq
The plain size tags show a size range and don't say which quant they pull. To know exactly what you get, use an explicit tag. Download sizes as Ollama lists them on Nov 3, 2026:
gemma4:e4b (= gemma4)
Download
6.6 to 9.5 GB
gemma4:e4b-it-qat
Download
6.1 GB
gemma4:12b-it-qat
Download
7.2 GB
gemma4:12b-it-q4_K_M
Download
8.0 GB
gemma4:12b-it-q8_0
Download
13 GB
gemma4:26b-a4b-it-qat
Download
16 GB
gemma4:26b-a4b-it-q4_K_M
Download
18 GB
gemma4:26b-a4b-it-q8_0
Download
28 GB
gemma4:31b-it-qat
Download
19 GB
gemma4:31b-it-q4_K_M
Download
20 GB
gemma4:31b-it-q8_0
Download
34 GB
Download
gemma4:e4b (= gemma4)
6.6 to 9.5 GB
gemma4:e4b-it-qat
6.1 GB
gemma4:12b-it-qat
7.2 GB
gemma4:12b-it-q4_K_M
8.0 GB
gemma4:12b-it-q8_0
13 GB
gemma4:26b-a4b-it-qat
16 GB
gemma4:26b-a4b-it-q4_K_M
18 GB
gemma4:26b-a4b-it-q8_0
28 GB
gemma4:31b-it-qat
19 GB
gemma4:31b-it-q4_K_M
20 GB
gemma4:31b-it-q8_0
34 GB
Ollama's download sizes are in decimal GB, while our VRAM tables count GiB, so the two differ by a few percent. The gemma4:cloud and gemma4:31b-cloud tags run on Ollama's servers, not your GPU. A thread on r/ollama (read Nov 1, 2026) reports that Ollama's free cloud models move to pay-as-you-go on Nov 9, 2026, while local models stay free.
llama.cpp. Google's own llama.cpp page calls it "the most capable local runtime for Gemma 4" because image, audio and video input all work there, and for 4-bit it says to "prefer Google's QAT weights". Its command shape, with the 26B-A4B QAT repo swapped in:
llama server -hf google/gemma-4-26B-A4B-it-qat-q4_0-gguf
The QAT repos exist for every size (google/gemma-4-{E2B,E4B,12B,26B-A4B,31B}-it-qat-q4_0-gguf), and -hf downloads the multimodal projector too. llama server opens a browser chat at localhost:8080 and an OpenAI-compatible API at localhost:8080/v1; Google's page also notes that "Multimodal input consumes context quickly", so set -c 32768 or more when you send images. Google says the QAT build preserves "similar quality to bfloat16" while cutting memory.
LM Studio runs the same GGUF files; search for the google/gemma-4-…-qat-q4_0-gguf repos to get Google's own quant rather than a re-upload.
"8GB: E4B. 12GB: 12B at Q4_K_M. 16GB: 12B at Q8_0, or 26B-A4B using Google's QAT Q4_0 file. 24GB: 26B-A4B at Q6_K for speed or 31B at Q4_K_M for the best answers. 32GB: 31B at Q5_K_M or Q6_K. Checked on our VRAM engine and Google's model cards on Nov 3, 2026."
Why does 'ollama run gemma4' give me a small model?
The bare gemma4 tag is the same build as gemma4:e4b, a 4.5B-effective model. Run gemma4:12b, gemma4:26b or gemma4:31b to get the bigger sizes.
Why does Gemma 4 in Ollama forget the start of a long chat?
Ollama's docs say it gives cards with under 24 GiB of VRAM a 4K context by default, whatever the model supports. Raise it with OLLAMA_CONTEXT_LENGTH or num_ctx, and check the extra context still fits your card.
How much VRAM does Gemma 4 31B need?
About 19.1 GB at Q4_K_M with 8K context and 20.9 GB with 32K, on our VRAM engine. That fits a 24GB card; Q6_K and Q8_0 need a 32GB card or more.
Does Gemma 4 26B-A4B fit a 16GB card?
Only just. Ollama's Q4_K_M build is an 18 GB download and won't fit. Google's QAT Q4_0 file is 14.4 GB plus a 1.2 GB vision projector, which fits text-only with a little context, or you can offload some experts to system RAM.
Can Gemma 4 take audio input?
E2B, E4B and 12B can, per Google's model card, with clips up to 30 seconds. Ollama publishes Gemma 4 as text and image only, so for audio Google points to Transformers or llama.cpp.
Is Gemma 4 better than Qwen3.8 27B?
For text and code only on 16 to 24GB, Qwen3.8 27B is the common pick on Reddit in November 2026. Gemma 4 is the better choice for audio input, small hardware, the fast 26B-A4B MoE, and writing-style uses where users prefer its tone.
Is Gemma 4 free for commercial use?
Google's Hugging Face cards list every Gemma 4 size under Apache-2.0.
Is there a Gemma 5?
Not as of Nov 3, 2026. Google's release notes list nothing for Gemma 5; anything you read about it is speculation.
If you are still choosing hardware, start with the best GPU for local AI, then come back here for the Gemma that fits it.