A 24 GB card runs Qwen3.8 27B at Q4 or Q5, and 32 GB runs Q6 with room to spare. 16 GB works too, at a 3-bit quant with a squeezed cache. Here is the math, what Reddit owners report, and what to check when it codes badly.
Some links on this page are referral links: we may earn a commission, at no cost to you.
Share
Updated Nov 1, 2026: re-checked the config, added how 16 GB owners run it, what to check when it codes badly, and 27B vs Flash Next.
Qwen3.8 27B needs about 18.5 GB of VRAM at Q4_K_M with 32K tokens of context, so a 24 GB card runs it well and a 16 GB card only at a 3-bit quant. It is a dense model with 27.78 billion parameters, and in October 2026 it became the default answer to "what should I run on my card" on Reddit. In a r/LocalLLaMA thread ranking every model for 10 to 16 GB of VRAM, the top reply was simply "1 Qwen 3.8 27b", though other replies flagged errors in the original list. The question that follows is always the same: will it fit my card, and at what quant?
The VRAM figures below are arithmetic on the published config, checked against real file sizes. The Aliteq Team has not loaded this model on any card; speed and quality reports are Reddit users' own, attributed and linked. For the live numbers on any context length, see cost to run Qwen3.8 27B and the VRAM calculator.
What the model is
Qwen3.8 27B is a dense, 64-layer model with a vision encoder, so it reads images and video as well as text. The weights file holds 27,781,427,952 parameters. Its model card lists a native context of 262,144 tokens, extendable further with YaRN scaling, which the card says to enable only when needed.
One detail drives every number below. The config sets full_attention_interval to 4: three linear-attention layers, then one full-attention layer, sixteen times over. Only those 16 full-attention layers keep a key-value cache, the memory of what you already said. The other 48 keep a small fixed state instead.
Qwen3.8 27B, from config.json (re-read Nov 1, 2026)
Parameters (all BF16 weights)
Value
27,781,427,952
Layers
Value
64 (16 full attention, 48 linear)
Full-attention KV heads x head dimension
Value
4 x 256
Linear-attention heads
Value
16 key, 48 value, 128 dim each
Native context
Value
262,144 tokens
Input
Value
Text, image, video
Licence
Value
Apache-2.0
Value
Parameters (all BF16 weights)
27,781,427,952
Layers
64 (16 full attention, 48 linear)
Full-attention KV heads x head dimension
4 x 256
Linear-attention heads
16 key, 48 value, 128 dim each
Native context
262,144 tokens
Input
Text, image, video
Licence
Apache-2.0
How we work out the VRAM
The answer is three terms added together: weights, cache and overhead. Weights are parameters times bits per weight, divided by 8. The cache is 2 x full-attention layers x KV heads x head dimension x tokens x bytes per value. Overhead is a flat 800 MiB for the runtime.
For Qwen3.8 27B the fp16 cache works out to 64 KiB per token: 2 x 16 x 4 x 256 x 2 bytes. That is 2 GB at 32K tokens and 8 GB at 128K. An 8-bit cache roughly halves both.
weights = 27.78e9 x bits_per_weight / 8
KV cache = 2 x 16 x 4 x 256 x tokens x 2 bytes (= 64 KiB per token at fp16)
total = weights + KV cache + 0.78 GB overhead
Q4_K_M at 32K: 15.69 + 2.00 + 0.78 = 18.5 GB
The linear-attention layers add a fixed state we leave out. From the config it is 48 layers x 48 value heads x 128 x 128 values, about 0.14 GB at 32-bit, and it does not grow with context. The bits-per-weight figures are the engine's effective values for llama.cpp quants, which run higher than the names suggest: Q4_K_M is 4.85, Q5_K_M 5.68, Q6_K 6.56, Q8_0 8.5.
VRAM by quant
Weights are most of the bill at every quant. At 32K tokens the fp16 cache is a flat 2 GB, so the quant you pick is the decision.
Derived with the aliteq VRAM engine, 32K context, fp16 cache. Not measured. · aliteq research
Total VRAM by quant and context (GB, computed, fp16 cache)
IQ3_XS
Weights
10.7
8K tokens
12.0
32K tokens
13.5
128K tokens
19.5
256K tokens
27.5
Q3_K_M
Weights
12.6
8K tokens
13.9
32K tokens
15.4
128K tokens
21.4
256K tokens
29.4
Q4_K_M
Weights
15.7
8K tokens
17.0
32K tokens
18.5
128K tokens
24.5
256K tokens
32.5
Q5_K_M
Weights
18.4
8K tokens
19.7
32K tokens
21.2
128K tokens
27.2
256K tokens
35.2
Q6_K
Weights
21.2
8K tokens
22.5
32K tokens
24.0
128K tokens
30.0
256K tokens
38.0
Q8_0
Weights
27.5
8K tokens
28.8
32K tokens
30.3
128K tokens
36.3
256K tokens
44.3
BF16 (full)
Weights
51.7
8K tokens
53.0
32K tokens
54.5
128K tokens
60.5
256K tokens
68.5
Weights
8K tokens
32K tokens
128K tokens
256K tokens
IQ3_XS
10.7
12.0
13.5
19.5
27.5
Q3_K_M
12.6
13.9
15.4
21.4
29.4
Q4_K_M
15.7
17.0
18.5
24.5
32.5
Q5_K_M
18.4
19.7
21.2
27.2
35.2
Q6_K
21.2
22.5
24.0
30.0
38.0
Q8_0
27.5
28.8
30.3
36.3
44.3
BF16 (full)
51.7
53.0
54.5
60.5
68.5
Q4_K_M is the usual starting point: our engine rates it the best quality per gigabyte for most people, and Q8_0 is effectively lossless when memory is not the limit. The 3-bit rows exist for one reason, the 16 GB card, covered below.
Does the math match real files?
Close, and on the safe side. The engine's weight figures sit within a few percent of the file sizes in one community GGUF set (Unsloth, whose files now carry a UD- prefix; re-read November 1, 2026). Other publishers' quants will differ by a few percent.
Engine estimate vs real file size (GB, weights only)
Q4_K_M
Real file
15.33
Engine
15.69
Gap
+2.3%
Q5_K_M
Real file
18.41
Engine
18.37
Gap
-0.2%
Q6_K
Real file
20.47
Engine
21.22
Gap
+3.7%
Q8_0
Real file
27.05
Engine
27.49
Gap
+1.6%
BF16 (official safetensors)
Real file
51.75
Engine
51.75
Gap
0%
Real file
Engine
Gap
Q4_K_M
15.33
15.69
+2.3%
Q5_K_M
18.41
18.37
-0.2%
Q6_K
20.47
21.22
+3.7%
Q8_0
27.05
27.49
+1.6%
BF16 (official safetensors)
51.75
51.75
0%
Two caveats. The image input uses a separate file: the vision projector is about 0.87 GB in that GGUF set, so budget about a gigabyte more if you send pictures. And Ollama's own qwen3.8 listing was 18 GB when we read it on 3 October, bigger than the 16.8 GB (15.69 GiB) we compute for Q4 weights. Its page does not say why, so size your card from Ollama's number if you pull it there.
Which GPU holds it
Pick the card by memory, then by how much context you need. The scorecard shows the longest context that keeps the model under 90 percent of the card at Q4_K_M and above, the point where a desktop or browser stops causing out-of-memory errors.
Q4_K_M and above, fp16 cache, under 90 percent of VRAM. 16 GB cards need the 3-bit quants in the next section. Derived from the formula, not measured. · aliteq research
24 GB. The sweet spot. The RTX 3090, RTX 4090 and RX 7900 XTX all have 24 GB. Q4_K_M leaves room for about 82K tokens, enough for a long chat or a mid-size codebase. Q5_K_M gets about 39K. Q6_K only fits tight, at about 32K. For how it compares with the other models a 24 GB card holds, see the best local LLM for 24 GB.
32 GB. The comfortable fit. The RTX 5090 holds Q6_K with about 108K tokens, or Q4_K_M with about 197K. Q8_0 fits, but with only about 8K tokens of room, so it is a weights-only quant here.
48 GB. Everything, including the model's full native context. An RTX A6000 or L40S runs Q8_0 with about 238K tokens.
The 16 GB reality: 3-bit, a squeezed cache, and a free desktop
At Q4_K_M the weights alone are 15.7 GB, so a 16 GB card cannot hold Q4 at all. At 3-bit it can, and that is how most 16 GB owners run it. Our 16 GB model guide puts it at Q3_K_M with 8K of context (about 13.9 GB) or IQ3_XS with 32K (about 13.5 GB) with a standard fp16 cache.
Owners push it much further. One r/LocalLLaMA user reported running Qwen3.8 27B on a 16 GB RTX 5070 Ti at about 110K tokens of context and about 55 tokens a second, using an 11.8 GB custom quant the poster labelled 4.2 bits per weight. The key step was moving the display to the CPU's integrated graphics: the desktop (Xorg) had been holding 600 to 800 MB of VRAM.
Two replies are worth as much as the post:
The quant. A reply pointed out that the "4.2bpw" label is misleading: the file is mostly IQ3, and pushing the model's linear-attention (SSM) layers that low damages them. Our arithmetic agrees on the size: 11.8 GB over 27.78 billion parameters is about 3.4 to 3.7 bits per weight on average (depending on whether the poster meant decimal or binary gigabytes), not 4.2. The reply suggested Unsloth's Q3_K_XL instead, which is a slightly bigger file (12.2 GB in our units, 13.1 billion bytes, on Hugging Face on 1 Nov). It is a custom build, so we describe it here rather than recommend it.
The cache. Another commenter suggested a q5_0 or q5_1 KV cache. That is the lever that makes 110K possible at all.
Here is why, on our formula. At fp16, 110K tokens of cache is 6.9 GB, which no 3-bit file plus runtime squeezes under 16 GB. llama.cpp's quantized cache types shrink it: q8_0 stores 8.5 bits per value, q5_1 6, q5_0 5.5.
Unsloth UD-Q3_K_XL (12.24 GB file) on a 16 GB card, computed
fp16 cache
Most context that fits 16 GB (at 100%)
47K
Total at 110K tokens
19.9 GB, no
q8_0 cache
Most context that fits 16 GB (at 100%)
89K
Total at 110K tokens
16.7 GB, no
q5_1 cache
Most context that fits 16 GB (at 100%)
126K
Total at 110K tokens
15.6 GB, very tight
q5_0 cache
Most context that fits 16 GB (at 100%)
138K
Total at 110K tokens
15.4 GB, very tight
Most context that fits 16 GB (at 100%)
Total at 110K tokens
fp16 cache
47K
19.9 GB, no
q8_0 cache
89K
16.7 GB, no
q5_1 cache
126K
15.6 GB, very tight
q5_0 cache
138K
15.4 GB, very tight
So 110K on 16 GB means a 3-bit file, a 5-bit cache and nothing else on the card, which is exactly why freeing the 600 to 800 MB the desktop held made the difference. The poster did not say which cache type they used in what we read, and the 0.78 GB overhead is our engine's middle estimate, so treat the table as a guide, not a guarantee. In llama.cpp the cache types are set with --cache-type-k and --cache-type-v (allowed types include q8_0, q5_1 and q5_0, per the llama.cpp server README); flash attention (--flash-attn) defaults to auto in current builds.
On speed, reports vary with the setup. In an r/ollama thread, one user reported 10 to 20 tokens a second for 27B dense on a 16 GB RTX 4070 Ti Super (no quant given) and moved to a 35B-A3B MoE coder fine-tune for agent speed; a reply said they would rather have 20 tokens a second of good output than 120 of worse. The gap with the 55 tokens a second above is plausible: a dense model reads every weight for every token, so a smaller file that sits fully on the card generates faster than a bigger one that spills to system RAM. We have not measured either.
"It's unusable for coding": what to check first
The loudest complaint about this model is a r/LocalLLM post from a small software house: an RTX 3090 with 32 GB of RAM, Qwen3.8 27B at 128K context, and a verdict of "practically unusable for coding". The replies are a checklist. The top one (629 votes) was "I can already tell you're running ollama". The next asked for the quant, the temperature, the harness and the engine settings. Others said it works well even at Q4 once configured, and that small models need smaller, explicit tasks.
The thread and its top reply on r/LocalLLM, captured Nov 1, 2026. · Reddit
Turned into steps, in the order that costs least to check:
Engine. The replies pushed llama.cpp, vLLM or SGLang over a default Ollama install. Qwen's model card recommends SGLang, vLLM and TokenSpeed for production and high throughput.
Context actually set. A 128K setting on a 24 GB card at Q4_K_M is about 24.5 GB by our formula with an fp16 cache, over the card. Either the engine is offloading (slow) or it silently set a shorter window. Check what it really allocated.
Quant. Q4_K_M or better on 24 GB. Do not judge the model at 3-bit and blame the model.
Sampling. Qwen's card gives temperature 1.0, top_p 0.95, top_k 20 for thinking mode, and temperature 0.7, top_p 0.80, top_k 20 with presence_penalty 1.5 for non-thinking mode. Greedy decoding or an engine's generic defaults are not that.
Harness. The coding agent's prompts and tool format matter as much as the model. Try a second harness before giving up.
Task size. Smaller, explicit tasks, as the replies said. A 27B model is not a frontier model handed a whole repo.
27B dense or Flash Next?
The other Qwen everyone is talking about is Qwen3.8 Flash Next, a mixture-of-experts model that its model card puts at 125B parameters with 6B active, plus a 51B n-gram embedding and 4B for multi-token prediction. It runs on small cards by keeping most of the model in system RAM, and its licence is a custom "qwen-community-1.0", not Apache-2.0. Our guide to running Qwen3.8 Flash Next locally covers the hardware.
Owners disagree on which is better, and both threads are worth reading:
For Flash Next. Top replies in the first thread disagreed strongly: one reported Flash Next at a Q3_S quant beating 27B at FP8, another called a mixed Q4 to Q8 build "utterly dominating". In the second, others said Flash Next had been "leagues above 27B" for them, and the top reply (268 votes) blamed many bad experiences on the Strata engine defaulting to greedy decoding instead of Qwen's recommended sampling.
The fair reading: sampling and engine settings decide these comparisons as much as the models do. Pick 27B if the whole model fits on your card and you want predictable, fully on-GPU behaviour. Try Flash Next if you have plenty of system RAM and a small card, and set the sampling by hand.
How to run it, and when to rent
The model card says Qwen3.8 works with Transformers, vLLM, SGLang and TokenSpeed. For a desk-side setup, Ollama listed it as ollama run qwen3.8, which was the 27B at 18 GB with a 256K window when we read the tags on 3 Oct. Unlike some other families, the bare tag here is the big model, not a small default. If you want to serve it to your apps, see how to run local AI as an API server.
If you do not own a 24 or 32 GB card, renting one costs less than you might think. On Vast.ai on Nov 17, 2026, a 24 GB RTX 4090 started at $0.27 an hour (median $0.49) and a 32 GB RTX 5090 at $0.38; Runpod listed the same two cards at $0.89 and $1.39 an hour. Our live cloud GPU prices show today's figures, and the cheapest RTX 4090 rental guide compares the hosts. Whether renting beats buying depends on your hours, which we cover in rent vs buy.
They do not tell you speed. Tokens per second depends on your engine, your card's memory bandwidth and the quant; the speeds above are Reddit users' reports, not our measurements. They do not tell you quality either. We did not run benchmarks, so we make no claim about how Qwen3.8 27B compares with other models beyond what the linked users report.
They also assume a single user. Several users at once multiply the cache by the number of live conversations. And the engine's overhead is a middle estimate, so leave headroom rather than aiming for 100 percent.
The image that ran with an earlier version of this guide.
About 18.5 GB at Q4_K_M and 30.3 GB at Q8_0 with 32K tokens of context, and 54.5 GB at full BF16. Those are our formula's results, within 2 to 4 percent of real GGUF file sizes.
Can I run Qwen3.8 27B on a 24 GB GPU?
Yes at Q4_K_M or Q5_K_M. Q4_K_M leaves room for about 82K tokens of context and Q5_K_M for about 39K. Q6_K only fits tight, and Q8_0 does not fit.
Can a 16 GB card run it?
Yes, at a 3-bit quant only. With an fp16 cache, Q3_K_M fits with about 8K tokens and IQ3_XS with about 32K. A quantized q5 KV cache and a desktop moved to integrated graphics let one RTX 5070 Ti owner report about 110K. Q4 and above do not fit.
Why is the context cache so small?
Only 16 of the model's 64 layers use full attention and keep a cache. The other 48 use linear attention with a small fixed state. That gives 64 KiB per token at fp16, about 8 GB at 128K tokens.
Why does Qwen3.8 27B code badly for me?
In the biggest complaint thread, replies pointed to the engine, the quant, the sampling settings, the coding harness and oversized tasks. Check that the context you set really fits, use Qwen's recommended sampling and a Q4 or better quant, and give it smaller tasks.
Should I run 27B or Qwen3.8 Flash Next?
27B if it fits fully on your card and you want predictable behaviour. Flash Next if you have a small card and lots of system RAM. Owners disagree on which is better, and sampling settings explain many of the bad reports on both sides.
Does Qwen3.8 27B take images?
Yes. The model card describes it as a native vision-language model. The image input adds a separate vision file of about 0.9 GB in the GGUF set we checked.