Llama 4 Quantized: GGUF Sizes & Memory, Explained Honestly

Quantization is the only reason Scout runs at home — but MoE means you shrink all 109B, not the 17B active. The honest GGUF memory ladder, the… Aliteq Lena Fischer · AI & Local Compute Editor

You quantize all 109B. MoE keeps every expert in memory, so a quant's size scales with the full 109B — not the 17B active. This is why Scout's files are large even at 4-bit.

The quality-usable target is Q4_K_M (~62GB). It's the widely-agreed sweet spot of quality vs size — but for a 109B model that still means a 64GB+ card or unified memory.

Q5/Q6 (~75–90GB) buy a little quality for a lot more memory — usually not worth it locally unless you have the room to spare.

Below Q4 the quality falls off — Q3/Q2 (~40–53GB) run, but you feel it, especially on the long-context and reasoning tasks that would justify Scout.

Unsloth's dynamic ~1.78-bit (~33GB) is the clever exception: it protects the most important weights, so it holds up better than a naive 2-bit and fits a 32GB card.

Q4_K_M is the default I'd recommend to anyone with the memory for it — it's the level the community has settled on as the best balance for most models, Scout included. If you can't fit ~62GB, skip…

How low I'd actually go

Read the full story

Aliteq Llama 4 Quantized: GGUF Sizes & Memory, Explained Honestly Read the full story on Aliteq