how to run a 30B model on a 12GB GPU: the one llama.cpp flag that makes it possible

--n-cpu-moe is the setting that lets a modest graphics card punch far above its VRAM. It offloads the parts of a Mixture-of-Experts model you use… Aliteq Lena Fischer · AI & Local Compute Editor

--n-cpu-moe N moves N layers' worth of MoE expert weights from the GPU to system RAM, freeing VRAM for everything else.

It only works on Mixture-of-Experts models (Qwen3-30B-A3B, gpt-oss, etc.) — experts are big and used infrequently, so offloading them costs the least speed per GB freed.

Higher N = less VRAM used, but slower — each token that hits an offloaded expert pays a PCIe transfer cost.

Good starting point: --n-cpu-moe 32 on a 12GB card; lower it until you're using most of your VRAM without overflowing.

Tune by: reducing --ctx-size or raising --n-cpu-moe in small steps until the server starts cleanly.

--n-cpu-moe counts from the highest-numbered layers downward, not from layer zero. That matters because many models put their dense (non-expert) layers at the very start — so offloading "the first…

The counting quirk that trips everyone up

Read the full story

Aliteq how to run a 30B model on a 12GB GPU: the one llama.cpp flag that makes it possible Read the full story on Aliteq