cpu offload

4 articles · newest first

Line-art illustration on a dark background next to the headline: One LM Studio switch Put MoE expert weights on the CPU. A card reads Setting: MoE on CPU (LM Studio).

LM Studio's 'force MoE weights onto CPU' slider: fit a 30B model or Qwen3.8 Flash Next

It started as a toggle and became a layer slider in LM Studio 0.4.0. Where it lives, what it moves, the number to set on an 8, 12 or 16GB card, whether LM Studio runs Qwen3.8 Flash Next, and why Mac MLX models don't have the setting.

Aliteq Team · Updated Nov 1, 2026 · 8 min

Line-art illustration on a dark background next to the headline: Ollama and MoE No --n-cpu-moe. Here is what it does. A card reads Common mistake: Defaults (context + quant).

Ollama has no --n-cpu-moe switch. What it does with MoE experts, and Qwen3.8 Flash Next

The GitHub request to keep MoE experts in RAM is still open after 15 months. But since Ollama 0.30, llama.cpp's own fit step can park them there for you, as long as you leave num_gpu alone. Plus what Ollama's Qwen3.8 Flash Next tags need, and why Reddit can tell when you're running Ollama.

Aliteq Team · Updated Nov 1, 2026 · 8 min

how much system RAM do you actually need for local AI? more than you think, if you offload

Everyone obsesses over VRAM, but system RAM matters too — especially the moment you try to run a model bigger than your GPU. Here's the real number for every kind of local-AI setup.

Aliteq Team · Updated Aug 5, 2026 · 9 min

An AMD Ryzen CPU seated in a motherboard next to DDR4 memory slots

how to run a 30B model on a 12GB GPU: the one llama.cpp flag that makes it possible

--n-cpu-moe is the setting that lets a modest graphics card punch far above its VRAM. It offloads the parts of a Mixture-of-Experts model you use least to system RAM — and here's exactly how to tune it for your card.

Aliteq Team · Updated Oct 28, 2026 · 9 min