16GB GPU? Which local AI models fit now, from gpt-oss-20b to Qwen3.8 27B (Nov 2026)

On a 16GB card like the RTX 5060 Ti, 4060 Ti or 5070 Ti, you now have two good answers. For speed and long context, gpt-oss-20b at Q4_K_M fits with 32K tokens of context in about 14.1 GB on our VRAM numbers. For the most capable model that still fits, Qwen3.8 27B runs at Q3_K_M with 8K of context (about 13.9 GB) or at IQ3_XS with 32K (about 13.5 GB), a lower quant than you would pick on a 24GB card. Gemma 4 12B at Q8_0 is the comfortable, near-lossless option with vision. What 16GB still cannot do is hold a 27B to 32B model at Q4 or better.

What fits a 16GB card, on our numbers

"Tight" means over 90% of the card, so anything else on the GPU can push it over. The table shows the quant we would actually run; the VRAM calculator shows every quant at any context length. For the other tiers, see the full VRAM ladder.

Qwen3.8 27B on 16GB: the trade

Qwen3.8 27B is a dense 27B model with vision input, a 262,144-token native context and an Apache-2.0 licence, per Qwen's model card. Only 16 of its 64 layers keep a growing KV cache, about 64 MB per 1,000 tokens at fp16 on our numbers, so long context is cheap for its size. The weights are the problem: at Q4_K_M they alone are about 15.7 GB, which leaves nothing for context on a 16GB card. So on 16GB it is a 3-bit model. Full numbers: cost to run Qwen3.8 27B.

It is also what 16GB owners are reaching for. In a r/LocalLLaMA thread ranking every model for 10GB to 16GB of VRAM, the top reply put Qwen 3.8 27B first, though other replies flagged errors in the original ranking. Our view: if you want the best answers and can live with short to medium context and slower generation (it is dense, so all 27B parameters are read per token), run it. If you want quick replies and long documents, gpt-oss-20b is the better daily driver.

RAM matters now: MoE models past 16GB

Mixture-of-experts models read only a few experts per token, so llama.cpp's --n-cpu-moe flag (or an engine such as Strata) can keep the busy layers on the GPU and push the rest to system RAM. On 16GB that is the sensible way to run Qwen3.6 35B-A3B at Q4 instead of squeezing it to Q2_K. Our MoE offloading guide shows the settings.

The same idea reaches much bigger models if you have the RAM. Qwen3.8 Flash Next (125B parameters, 6B active, per its model card) runs in Strata, whose model notes say it fits when RAM is at least the expert weights plus about 10 GB: 48GB of RAM for the 2-bit packs, 64GB for every size. A bigger graphics card makes it faster but, per those notes, does not lower the RAM needed.

Know the costs before you try:

• Low quants. The packs that fit desktop RAM are 2-bit and 3-bit.
• Slower, and less predictable. Anything that lives in system RAM is read over PCIe for every token that needs it. If RAM runs short, Strata reads experts from the SSD, which its docs call "much slower".
• Young software. Strata is a 0.x project; Reddit users report greedy sampling by default and quality changes between versions. Pin a version, check the temperature against Qwen's card, and keep llama.cpp as your stable fallback.
• Context still costs VRAM. Offloading experts frees the card for the KV cache, but very long context on a hybrid model still needs memory.

Getting the most from 16GB

• Pick by job. gpt-oss-20b for fast replies and long documents, Qwen3.8 27B for the hardest prompts, Gemma 4 12B when you need images or headroom.
• Watch the context setting. A 128K window you never use still reserves memory in some apps. Set it to what you need.
• If it crawls, it spilled. A model that loads but runs much slower than expected has part of itself or its cache in system RAM.

16GB is a solid tier: gpt-oss-20b, Gemma 4 12B and a 3-bit Qwen3.8 27B all run on the card. If you find yourself always wanting Qwen3.8 27B at a better quant, that is the case for a 24GB card; for a smaller card, see the 12GB guide.