how to run a 30B model on a 12GB GPU: the one llama.cpp flag that makes it possible

--n-cpu-moe is the setting that lets a modest graphics card punch far above its VRAM. It offloads the parts of a Mixture-of-Experts model you use… Aliteq Tensor · Local AI & Automation Editor

--n-cpu-moe N moves N layers' worth of MoE expert weights from the GPU to system RAM, freeing VRAM for everything else.

It only works on Mixture-of-Experts models (Qwen3-30B-A3B, gpt-oss, etc.) — experts are big and used infrequently, so offloading them costs the least speed per GB freed.

Higher N = less VRAM used, but slower — each token that hits an offloaded expert pays a PCIe transfer cost.

Good starting point: --n-cpu-moe 32 on a 12GB card; lower it until you're using most of your VRAM without overflowing.

Tune by: reducing --ctx-size or raising --n-cpu-moe in small steps until the server starts cleanly.

Which layers does it pick? Per the llama-server README (llama.cpp v0.5.0, accessed 25 Sep 2026), --n-cpu-moe N keeps "the Mixture of Experts (MoE) weights of the first N layers in the CPU" —…

The counting quirk that trips everyone up

Read the full story

Aliteq how to run a 30B model on a 12GB GPU: the one llama.cpp flag that makes it possible Read the full story on Aliteq