Which MoE model runs best on a 12GB or 16GB GPU, and how much RAM do you need?

Which MoE model runs best on a 12GB or 16GB GPU? The one your system RAM can hold. With 16 to 32 GB of RAM, run Qwen3.6 35B-A3B, Qwen3 30B-A3B or gpt-oss 20B. With 48 GB, Qwen3.8 Flash Next at a 2-bit quant is the biggest model that fits. With 64 GB, gpt-oss 120B. The card barely changes the answer: on either size, the always-used parts of every model below fit, and the rest is expert weight in RAM. Files and flags re-checked on Hugging Face and the llama.cpp docs on November 3, 2026.

Our llama.cpp offloading guides answer "how do I use the flag". The question that comes before it is the buying one: which model, and how much RAM? That is arithmetic on published files, and this page does it per model. For the mechanics of the flag, read what --n-cpu-moe actually does.

Which MoE model fits, by how much RAM you have

Pick the model by your system RAM first, then set the flag for your card. At 32K tokens of context, a 12 or 16 GB card holds the always-used parts of every model below. The experts that do not fit go to RAM, and that RAM bill runs from zero to 76 GiB.

"RAM to have" is the expert weight in RAM plus 8 GB for your operating system and apps, rounded up to a common kit size (16, 32, 48, 64, 96 GB). That 8 GB is our allowance, not a llama.cpp number; a browser with 40 tabs needs more. For Flash Next it assumes the n-gram table stays on disk (see below); if it ends up resident, add about 27 GiB. The Q4_K_M rows for GLM-4.5-Air (68.0 GiB file, 96 GB of RAM) and Qwen3 235B (132.4 GiB, 192 GB of RAM) are in the model notes further down.

How do you work out the RAM a MoE model needs?

Every MoE model splits into two piles. Expert weights are the big feed-forward stacks that --n-cpu-moe moves. Everything else (attention, router, shared experts, embeddings) stays on the GPU. Both piles were read straight from each GGUF file, then the context cache and overhead were added with our VRAM engine.

The llama.cpp server README (read Nov 3, 2026) describes the flag as keeping "the Mixture of Experts (MoE) weights of the first N layers in the CPU". Its sibling --cpu-moe keeps "all Mixture of Experts (MoE) weights in the CPU". So the math is: put the fixed part on the card, fill what is left with whole layers of experts, and send the rest to RAM.

GPU floor   = non-expert weights + KV cache + 0.78 GiB overhead
KV cache    = 2 x attention layers x KV heads x head dim x tokens x 2 bytes
budget      = 0.9 x card memory            (12 GB -> 10.8, 16 GB -> 14.4)
layers kept = floor((budget - GPU floor) / expert GiB per layer)
--n-cpu-moe = layers with experts - layers kept
RAM weights = --n-cpu-moe x expert GiB per layer

gpt-oss 120B on 16 GB at 32K:
  floor  = 2.14 + 2.25 + 0.78 = 5.17 GiB
  kept   = floor((14.4 - 5.17) / 1.58) = 5 of 36 layers
  N      = 36 - 5 = 31, RAM weights = 31 x 1.58 = 49.0 GiB

All of this is DERIVED. The 90 percent budget is our engine's line between "comfortable" and "tight", because a desktop and a browser also use the card. For gpt-oss the cache figure counts every layer, though half of them use a 128-token sliding window; that overstates its cache and errs toward "won't fit". For the hybrid Qwen3.6, Qwen3-Next and Qwen3.8 Flash Next, only the full-attention layers keep a cache (10, 12 and 12 respectively), per their configs. If your card or context is different, the VRAM calculator runs the same engine for any size.

Model by model: which one should you pick?

The A3B models are the easy wins on 16 to 32 GB of RAM. Flash Next is the reason to buy 48 GB, and gpt-oss 120B the reason to buy 64 GB. GLM-4.5-Air and Qwen3 235B fit on paper, but they lean hard on system RAM. Here is each one, with the file it was sized from.

gpt-oss 20B: no tricks needed on 16 GB

OpenAI's model card says gpt-oss 20B has 21B parameters with 3.6B active, and that it can "run within 16GB of memory" (read Nov 3, 2026). The ggml-org MXFP4 file is 11.3 GiB. On a 16 GB card it fits whole with 32K of context. On a 12 GB card, --n-cpu-moe 7 moves about 2.8 GiB of experts to RAM. See its cost to run page for the full ladder, and can you run gpt-oss 20B locally for setup.

Qwen3.6 35B-A3B and Qwen3 30B-A3B: the 12 GB sweet spot

Both route a few small experts per token: 3B activated of 35B for Qwen3.6, and 3.3B of 30.5B for Qwen3 2507, per their model cards (read Nov 3, 2026). On a 12 GB card each puts about 11 GiB of experts in RAM, so 32 GB of RAM is the comfortable buy. On a 16 GB card that drops to about 7 to 8 GiB, and 16 GB of RAM is enough if little else is open.

Qwen3.6 has a cheap cache: only 10 of its 40 layers keep one, so 32K tokens costs 0.6 GiB. That matters because its model card recommends a context of "at least 128K tokens to preserve thinking capabilities". At 128K our math gives --n-cpu-moe 29 on 12 GB and 21 on 16 GB, with 32 GB of RAM either way. Qwen3 30B-A3B cannot do that: its 128K cache alone is 12 GiB.

Qwen3.6 35B-A3B is still what many 16 GB owners pick for agent speed. One r/ollama user moved from a dense 27B to a 35B-A3B coder fine-tune because the dense model gave 10 to 20 tokens/s on an RTX 4070 Ti Super (thread read Nov 1, 2026). If quality matters more than speed and you have no spare RAM, the dense Qwen3.8 27B is the usual Reddit pick for 10 to 16 GB; our 16GB guide has its quants.

Qwen3.8 Flash Next: the biggest model 48 GB of RAM can hold

Per its model card (read Nov 3, 2026), Flash Next has "125B with 6B activated, plus 51B n-gram embedding and 4B MTP". Its config has 48 layers, 512 experts per layer with 10 routed and 1 shared active per token, and full attention on every fourth layer (2 KV heads, head dimension 256). The licence is qwen-community-1.0, a custom licence. The card says the n-gram embedding is "more amenable to offloading than Mixture-of-Experts (MoE)".

The files that fit desktop RAM are 2-bit and 3-bit. The ISTA-DASLab IQ2_XS build ships in two shards: 39.2 GB (36.5 GiB) of model weights and a 28.8 GB (26.8 GiB) n-gram table. ISTA-DASLab's own table says shard 1 "must be resident" and shard 2 "can stay on disk".

Strata's model notes put the expert part of the IQ2_XS first shard at 35.5 GB (33.1 GiB), so about 3.45 GiB is everything else. That gives:

Qwen3.8 Flash Next, ISTA-DASLab IQ2_XS, 32K context (derived, GiB)
KV cache    = 2 x 12 full-attention layers x 2 KV heads x 256 x 32,768 x 2 bytes = 0.75
GPU floor   = 3.45 non-expert + 0.75 cache + 0.78 overhead = 4.98
per layer   = 33.06 GiB of experts / 48 layers = 0.69
12 GB card: kept = floor((10.8 - 4.98) / 0.69) = 8  ->  --n-cpu-moe 40, 27.6 GiB in RAM
16 GB card: kept = floor((14.4 - 4.98) / 0.69) = 13 ->  --n-cpu-moe 35, 24.1 GiB in RAM

Add the 8 GB allowance and both cards land on a 48 GB kit. The IQ3_XXS build (43.8 GiB first shard, 43 GB or 40.0 GiB of experts per Strata) works out to --n-cpu-moe 42 on 12 GB and 38 on 16 GB, with 35.0 and 31.7 GiB in RAM. That fits 48 GB by our allowance, with little room left on a 12 GB card. Strata's own rule is stricter, because it keeps every expert in RAM: the model fits when "RAM is at least the experts plus about 10 GB", and it says "With 48 GB, Q2_0 and IQ2_XS fit" while 64 GB fits every size. Unsloth's builds are bigger: UD-Q2_K_XL totals 73.5 GiB and UD-Q4_K_XL 103.7 GiB across their shards (file listing read Nov 3, 2026).

The n-gram table is what makes this model unusual. llama.cpp's --lazy-mode on reads such tensors "from disk on demand instead of keeping them resident (requires mmap)", and its default, auto, already does that for tensors larger than 4 GiB (server README, read Nov 3, 2026). The ISTA-DASLab card's example uses -lm mmap --lazy-mode on, which it says "removes 28.8 GB from the resident footprint". Our numbers assume that. They leave out the small sparse-attention indexer cache and the fixed linear-attention state, so treat the floor as a lower bound. The cost-to-run page for Flash Next shows its memory per quant, and how to run Qwen3.8 Flash Next locally walks through the setup.

What people report running is often less RAM than this, with an engine that reads experts from the SSD. These are their reports, read on Reddit on November 1, 2026, not our tests:

• RTX 3060 12 GB + 16 GB DDR4, IQ2_XS (the 68 GB GSQ-RCO build), Betelgeuse engine: 22 tokens/s sustained, per its author on r/LocalLLaMA. That is far below the RAM our all-experts-in-RAM arithmetic needs, so it depends on that engine's own memory handling.
• 8 GB VRAM + 32 GB RAM, Q2, Strata: up to 900 tokens/s prompt processing and 30 tokens/s generation, per a popular r/LocalLLaMA post. Strata splits the model across VRAM, RAM and SSD; its docs say reading experts from the SSD "works and is much slower".
• 8 GB VRAM + 24 GB RAM, about 3.5 bits per weight (Flash-Next Coder), llama.cpp: most experts in RAM, n-gram table on disk, per this thread, where a critic pointed out that roughly 70 tokens/s of prompt processing means minutes to first token in an agent harness.

The quality caveat is real. A 2-bit quant of anything is contested: in one r/LocalLLM thread a reply argued that no model is great below Q4, while replies in another reported a Q3_S Flash Next beating Qwen3.8 27B at FP8. ISTA-DASLab's own notes report its IQ3 builds close to the full model on its tests; those are the quantizer's numbers. In another thread, the top reply blamed many bad first impressions on greedy decoding. Use the sampling the model card lists (thinking mode: temperature 1.0, top_p 0.95, top_k 20), and test on your own work.

Qwen3-Next 80B-A3B: the 48 GB option at Q4

Qwen3-Next has "80B in total and 3B activated" per its model card, with 512 small experts and 10 active per token. The unsloth Q4_K_M file is 45.2 GiB, and about 33 to 36 GiB of it lands in RAM. Budget 48 GB of RAM. With the same RAM, Flash Next at a 2-bit or 3-bit quant is the newer, bigger alternative; Qwen3-Next lets you stay at Q4. Its tracker page is here.

gpt-oss 120B: the reason to buy 64 GB

The model card lists "117B parameters with 5.1B active parameters" (read Nov 3, 2026). The MXFP4 file is 59.0 GiB, of which 56.9 GiB is experts. On a 16 GB card, five layers of experts stay on the GPU and 49 GiB goes to RAM. On 12 GB it is three layers and 52 GiB. Either way, 64 GB of RAM covers it, and 32 GB does not. Our gpt-oss 120B hardware requirements page covers the other ways to run it.

GLM-4.5-Air: fits, but every layer of experts goes to RAM

GLM-4.5-Air has 106 billion total parameters and 12 billion active, per Z.ai's card. Its cache is the heaviest here at 184 KB per token, so the always-on-GPU part is already 11.3 GiB at 32K. On a 16 GB card only about two expert layers would fit beside it, so --cpu-moe is the sensible start. On a 12 GB card, --cpu-moe plus about 29K of context is the ceiling. The IQ4_XS file (56.3 GiB) needs 64 GB of RAM; Q4_K_M (68.0 GiB) needs 96 GB.

One counting quirk: GLM's layer 0 is dense, with no experts, and the file carries an extra next-token-prediction layer at the end. So --n-cpu-moe 1 frees nothing on this model.

Qwen3 235B-A22B: only with 96 GB, and only at 2 bits

Qwen3 235B has "235B in total and 22B activated". Even the unsloth UD-Q2_K_XL file is 82.7 GiB, which needs 96 GB of RAM. Q4_K_M is 132.4 GiB and needs 192 GB. Our engine's note on 2-bit quants is blunt: heavy degradation, usually worse than a smaller model at Q4. On these cards, gpt-oss 120B or Flash Next is the more practical choice.

Where do the gigabytes sit on a 16 GB card?

The GPU side is nearly constant. What changes is the pile in system RAM. That is why "which MoE model fits 16 GB" is really "how much RAM is in the box". Flash Next at IQ2_XS fits the pattern: about 14.0 GiB on the card at --n-cpu-moe 35 and 24.1 GiB of experts in RAM, with the n-gram table on disk.

Why do the A3B models feel faster?

Each new token only reads the experts the router picks. So the RAM traffic per token is the expert weight in RAM times the share of experts used. The A3B models read under 1 GiB per token on a 16 GB card. Flash Next at IQ2_XS reads about 0.47 GiB, because it picks 10 of 512. GLM-4.5-Air reads 3.2 GiB, and Qwen3 235B 4.7 GiB.

GiB read per token = experts per token / experts per layer x GiB of experts in RAM

Qwen3.8 Flash Next IQ2_XS, 16 GB: 10 / 512 x 24.1  = 0.47
gpt-oss 120B, 16 GB card:          4 / 128 x 48.98 = 1.53
Qwen3.6 35B-A3B, 16 GB:            8 / 256 x 7.74  = 0.24
GLM-4.5-Air IQ4_XS:                8 / 128 x 51.52 = 3.22

Your RAM's bandwidth turns that number into speed, so this page does not print a tokens-per-second figure for your machine. The ratio is the useful part. At the same RAM speed, GLM-4.5-Air moves about twice the expert data per token of gpt-oss 120B, and about 13 times that of Qwen3.6. For the bandwidth arithmetic and measured llama-bench results, see our --n-cpu-moe explainer.

Do you still need to set --n-cpu-moe by hand?

Often not. The llama.cpp server README (read Nov 3, 2026) lists --fit with "default: 'on'", described as "whether to adjust unset arguments to fit in device memory". So the table above is mostly your RAM shopping list and a sanity check. Set the flag by hand when you want control.

# Qwen3.8 Flash Next IQ2_XS on a 16 GB card, 32K context, n-gram table on disk
llama-server -m IQ2_XS/Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf \
  -c 32768 -ngl 99 --n-cpu-moe 35 -lm mmap --lazy-mode on

# gpt-oss 120B on a 16 GB card, 32K context (starting point from the table)
llama-server -m gpt-oss-120b-MXFP4.gguf -c 32768 -ngl 99 --n-cpu-moe 31

# GLM-4.5-Air on a 12 GB card: every expert in RAM, shorter context
llama-server -m GLM-4.5-Air-IQ4_XS-00001-of-00002.gguf -c 24576 -ngl 99 --cpu-moe

The --n-cpu-moe explainer shows the -ot regex route when whole layers are too coarse. Tune in this order:

Using a desktop app instead? LM Studio's slider takes the same number, and Ollama places experts automatically if you leave num_gpu unset.

What these numbers do not tell you

They do not tell you speed or quality. No benchmarks and no evals were run, so there is no claim here about which model answers better. They also assume one user and an fp16 cache. An 8-bit cache (-ctk q8_0 -ctv q8_0) halves the cache term. That frees room for more expert layers on models with a big cache, such as Qwen3 30B-A3B and GLM-4.5-Air, and very little on the hybrid Qwen models.

They come from one publisher's quants per model. Other GGUF builds mix tensor types differently and shift the split by a few percent. Qwen3.6's image input adds a separate 0.84 GiB projector file, and Flash Next's is 0.85 GiB on the ISTA-DASLab repo. If your card is not 12 or 16 GB, run the same model in the VRAM calculator. If you are still choosing a dense model, our picks for 12 GB and 16 GB cards cover those.