n-cpu-moe

4 articles · newest first

AI

Which MoE model runs best on a 12GB or 16GB GPU, and how much RAM do you need?

On a 12GB or 16GB card, your system RAM decides which mixture-of-experts model you can run. Nine files from gpt-oss 20B to Qwen3.8 Flash Next, split into what stays on the card and what moves to RAM, with the --n-cpu-moe starting value for each. Re-checked Nov 3, 2026.

Aliteq Team · Updated Nov 3, 2026 · 11 min

AI

gpt-oss-20b on an 8GB or 12GB GPU: yes, with --n-cpu-moe

Everyone says 16GB minimum. That's only true if the whole model has to live on the GPU. llama.cpp's own guide runs it on an 8GB RTX 2060. Here are the commands, and the starting values for your card.

Aliteq Team · Sep 25, 2026 · 7 min

AI

how much system RAM do you actually need for local AI? more than you think, if you offload

Everyone obsesses over VRAM, but system RAM matters too — especially the moment you try to run a model bigger than your GPU. Here's the real number for every kind of local-AI setup.

Aliteq Team · Updated Aug 5, 2026 · 9 min

AI

how to run a 30B model on a 12GB GPU: the one llama.cpp flag that makes it possible

--n-cpu-moe is the setting that lets a modest graphics card punch far above its VRAM. It offloads the parts of a Mixture-of-Experts model you use least to system RAM — and here's exactly how to tune it for your card.

Aliteq Team · Updated Oct 28, 2026 · 9 min