12GB GPU? The best local LLM for an RTX 3060 or 4070, updated for November 2026

On a 12GB card like the RTX 3060 12GB or RTX 4070, the best all-round local model in November 2026 is Gemma 4 12B at Q6_K: on our VRAM numbers it uses about 10.0 GB with 8K tokens of context and about 10.4 GB with 32K, so it fits comfortably, handles images, and keeps a long chat on the card. Qwen3.5 9B at near-lossless Q8_0 is the other easy fit. The new Qwen3.8 27B does squeeze onto 12GB, but only at IQ3_XS with 8K of context (about 12.0 GB, with no margin), so treat it as a stretch, not a default.

What fits a 12GB card, on our numbers

"Tight" means over 90% of the card. It loads, but anything else on the GPU can push it over. The table shows the quant we would actually run; for some models a bigger quant technically fits at 8K with no margin. The VRAM calculator shows every quant for any context length.

Why Gemma 4 12B, not Qwen3-14B

For two years the 12GB answer was "a 14B model at Q4". The problem is context. Qwen3-14B keeps a full KV cache on all 40 layers, about 160 MB per 1,000 tokens at fp16 on our numbers, so by 32K it no longer fits at Q4_K_M. Gemma 4 12B keeps a growing cache on only 8 of its 48 layers (the rest use a 1,024-token sliding window), about 16 MB per 1,000 tokens. That is why it holds 32K at Q6_K with room to spare. Qwen3-14B is still a good model; it just suits short chats on this card.

Qwen3.8 27B on 12GB: possible, with limits

Qwen3.8 27B is a dense 27B model with vision input, a 262,144-token native context and an Apache-2.0 licence, per Qwen's model card. Its weights at IQ3_XS come to about 10.7 GB on our numbers, so on 12GB it fits only with short context and nothing else on the GPU. Full numbers: cost to run Qwen3.8 27B.

People are pushing it further. A r/LocalLLM user reported running a Q3-XXS GGUF of Qwen3.8 27B on 12GB of VRAM and 16GB of RAM with 150K of context at 11 tokens per second, using a q4_0 KV cache. By our arithmetic, 150K tokens of KV cache at roughly 4 bits is still 2 to 3 GB on top of the weights, so part of that setup has to sit in system RAM. It works, but you are trading quality (a 3-bit quant and a 4-bit cache) and speed for context.

RAM matters now: bigger MoE models on 12GB

Mixture-of-experts models use only a few experts per token, so engines can keep the busy parts on the GPU and the rest in system RAM. On a 12GB card that is how you reach models such as Qwen3.6 35B-A3B or Qwen3.8 Flash Next. How far you get depends on your RAM, and our guide to MoE models on 12GB and 16GB cards works through it with llama.cpp.

The Strata engine publishes its own test table. On an RTX 3060 12GB with 46GB of RAM it lists DeepSeek V4.1 Flash 1017 at a 2.00-bit quant generating 9 to 11 tokens per second, measured on Strata 0.2.5 per its README. That is the vendor's number, at a very low quant, on a model whose download is roughly 239 to 274 GB. It shows what is possible, not what is comfortable.

The trade-offs are the same everywhere: low quants, more setup, and slower replies whenever part of the model lives in RAM or on the SSD.

Getting the most from 12GB

• Pick the model by context, not just size. If you want long chats or documents, a hybrid-attention model (Gemma 4, Qwen3.5) keeps far more context in the same memory.
• Stay at Q4_K_M or above for daily use. Drop to a 3-bit quant only to fit a model that is worth it, like Qwen3.8 27B.
• If it crawls, it spilled. A model that loads but runs far slower than expected has part of itself or its KV cache in system RAM. Shorten the context or drop a quant.
• Compare tiers. The VRAM ladder puts 8GB, 12GB, 16GB and 24GB side by side.

On 12GB, start with Gemma 4 12B at Q6_K, try Qwen3.8 27B when you want more depth on short prompts, and compare the 8GB and 16GB guides if you are deciding on a card. The RTX 3060 12GB is still the budget card for this tier.