Quantization is the only reason Scout runs at home — but MoE means you shrink all 109B, not the 17B active. The honest GGUF memory ladder, the Q4_K_M sweet spot, and how low you can safely go.
Quantization is the only reason Llama 4 Scout is a home model at all, so it's worth getting right instead of just grabbing whatever file downloads fastest. The job is simple to state: shrink the model's weights from 16-bit down to something smaller so they fit your memory, while giving up as little quality as possible. The trick with Scout is that it's a 109B mixture-of-experts model, so the numbers are bigger and less forgiving than people expect — you're quantizing the whole herd, not the 17B that runs per token. Here's the honest GGUF memory ladder and how low I'd actually go.
Quantization shrinks the weights to fit your memory — but with MoE you're shrinking all 109B. Illustration by Aliteq. · Illustration by Aliteq / generated with Higgsfield
The GGUF memory ladder for Scout
GGUF is the file format llama.cpp uses, and its quant levels (Q8, Q6_K, Q5_K_M, Q4_K_M, and so on) trade bits-per-weight for size and quality. These sizes are approximate — they shift a little with the exact quant and don't include the KV cache — but the shape is what matters:
Llama 4 Scout weight size by GGUF quant (approx, 109B)
Q8_0 (~8-bit)~116 GB
near-lossless, needs serious memory
Q6_K (~6.5-bit)~89 GB
marginal gain over Q4 for the size
Q5_K_M (~5.5-bit)~75 GB
if you have the room
Q4_K_M (~4.8-bit)~62–65 GB
the quality/size sweet spot
Q3_K_M (~3.9-bit)~53 GB
quality starts to show
Dynamic ~1.78-bit~33 GB
fits a 32GB card; surprisingly usable
Why MoE makes this less forgiving
With a normal dense model, quantization size tracks the number you see on the box. With MoE it tracks the total, which is much bigger than the model "feels" at inference. Scout runs like a 17B (fast, cheap per token) but quantizes like a 109B (large files, big memory). That mismatch is where people get burned: they read "17B active," download a quant expecting a 10GB file, and get a 60GB+ one. Set your expectations by the total parameter count, always.
Quick answers
What quant should I use for Llama 4 Scout?
Q4_K_M if you have the memory (~62GB, so a 64GB+ card or unified memory) — it's the community-agreed quality/size sweet spot. If you can't fit that, use Unsloth's dynamic ~1.78-bit (~33GB), which holds up far better than a flat 2-bit and fits a 32GB card.
Why is a 4-bit Llama 4 file still so big?
Because it's mixture-of-experts: all 109B parameters get quantized and loaded, even though only 17B compute per token. A 4-bit quant of 109B is ~62GB — set your expectations by the total parameter count, not the active one.
Is 2-bit quantization usable?
A naive flat 2-bit generally isn't — the quality drop is severe. But Unsloth's dynamic quantization (~1.78-bit average) is different: it keeps the most important weights at higher precision, so it stays surprisingly usable while fitting a 32GB GPU. Prefer it over any flat sub-3-bit quant.
Does quantization hurt the long context?
Aggressive quantization can degrade the fidelity of long-context recall, which matters because context is Scout's main selling point. If you care about that, stay at Q4_K_M or above and don't skimp on KV-cache memory.
