n-cpu-moe

4 articles · newest first

Low-angle photo of a row of blank matte memory modules on a motherboard in the foreground, with a plain graphics card blurred behind them under indigo and coral light

Best MoE Models for a 12GB or 16GB GPU: The --n-cpu-moe Number and the RAM Each Needs

Seven mixture-of-experts models, from gpt-oss 20B to Qwen3 235B, split into what stays on your card and what moves to system RAM. The card barely changes. The RAM you need goes from 16 GB to 96 GB.

Aliteq Team · Oct 4 · 10 min

Flat illustration of a person splitting an AI brain puzzle piece between a small graphics card and a row of memory sticks

gpt-oss-20b on an 8GB or 12GB GPU: yes, with --n-cpu-moe

Everyone says 16GB minimum. That's only true if the whole model has to live on the GPU. llama.cpp's own guide runs it on an 8GB RTX 2060. Here are the commands, and the starting values for your card.

Aliteq Team · Sep 25 · 7 min

how much system RAM do you actually need for local AI? more than you think, if you offload

Everyone obsesses over VRAM, but system RAM matters too — especially the moment you try to run a model bigger than your GPU. Here's the real number for every kind of local-AI setup.

Aliteq Team · Jul 28 · 9 min

An AMD Ryzen CPU seated in a motherboard next to DDR4 memory slots

how to run a 30B model on a 12GB GPU: the one llama.cpp flag that makes it possible

--n-cpu-moe is the setting that lets a modest graphics card punch far above its VRAM. It offloads the parts of a Mixture-of-Experts model you use least to system RAM — and here's exactly how to tune it for your card.

Aliteq Team · Jul 27 · 9 min