gpt-oss-20b on an 8GB or 12GB GPU: yes, with --n-cpu-moe

The short answer

Yes, with one llama.cpp flag. gpt-oss-20b doesn't fit fully in 8GB or 12GB. llama.cpp's own guide puts the full model at about 14.9GB at 8K context, and our VRAM engine puts it at about 13 GiB. But…

Fits fully? No: ~13 GiB (our engine, Q4_K_M, 8K) / 14.9GB (llama.cpp's table, 8K)

Always on the GPU: ~1.8B non-expert params ≈ 1.0 GiB, plus KV cache and buffers

8GB (e.g. RTX 2060): --n-cpu-moe 16 at 32K, llama.cpp's own example

12GB (e.g. RTX 3060): start around --n-cpu-moe 6 at 16K, then tune

Windows: "it loaded" doesn't mean "it fits"

llama.cpp's guide warns that on Windows "it is possible to allocate more VRAM than available, and the result will be slow swapping to RAM and very bad performance." If generation is unexpectedly…