Qwen3.8 27B Hardware Requirements: VRAM by Quant, and Why Context Costs So Little

Qwen3.8 27B is a dense model with 27.78 billion parameters, and it is built to be run at home. The question I get is always the same: will it fit my card? I like questions I can answer with a spreadsheet, so here is the spreadsheet.

I read the model's config on Hugging Face on 3 October 2026 and ran the numbers through our VRAM engine. I have not loaded this model on any card. Everything below is arithmetic on published specs, checked against real file sizes. For the same math on other models, see our cost to run tool and the best GPU for local AI guide.

What the model is

Qwen3.8 27B is a dense, 64-layer model with a vision encoder, so it reads images and video as well as text. The weights file holds 27,781,427,952 parameters. Its model card lists a native context of 262,144 tokens.

One detail drives every number below. The model card describes the layout as 16 blocks of three linear-attention layers followed by one full-attention layer. Only those 16 full-attention layers keep a key-value cache, the memory of what you already said. The other 48 keep a small fixed state instead.

How we work out the VRAM

The answer is three terms added together: weights, cache and overhead. Weights are parameters times bits per weight, divided by 8. The cache is 2 x full-attention layers x KV heads x head dimension x tokens x 2 bytes. Overhead is a flat 800 MiB for the runtime.

For Qwen3.8 27B the cache works out to 64 KiB per token: 2 x 16 x 4 x 256 x 2 bytes. That is 2 GB at 32K tokens and 8 GB at 128K. An 8-bit cache halves both.

weights  = 27.78e9 x bits_per_weight / 8
KV cache = 2 x 16 x 4 x 256 x tokens x 2 bytes   (= 64 KiB per token)
total    = weights + KV cache + 0.78 GB overhead

Q4_K_M at 32K: 15.69 + 2.00 + 0.78 = 18.5 GB

The linear-attention layers add a fixed state we leave out. By our estimate it is about 0.14 GB, and a safe allowance is under 0.5 GB. The bits-per-weight figures are the engine's effective values for llama.cpp quants, which run higher than the names suggest: Q4_K_M is 4.85, Q5_K_M 5.68, Q6_K 6.56, Q8_0 8.5.

VRAM by quant

Weights are most of the bill at every quant. At 32K tokens the cache is a flat 2 GB, so the quant you pick is the decision.

Q4_K_M is the usual starting point. Our engine's note on it is that it gives the best quality per gigabyte for most people, and Q8_0 is effectively lossless when memory is not the limit. I would drop below Q4 only to make a bigger model fit, never to squeeze this one onto a smaller card.

Does the math match real files?

Close, and on the safe side. I compared the engine's weight figures with file sizes on Hugging Face from one community GGUF set (Unsloth, read 3 October 2026). Other publishers' quants will differ by a few percent.

Two caveats. The image input uses a separate file: the vision projector is 0.87 GB in that GGUF set, so budget about a gigabyte more if you send pictures. And Ollama's own qwen3.8 listing is 18 GB, which is bigger than the 16.8 GB (15.69 GiB) we compute for Q4 weights. Its page does not say why, so size your card from Ollama's number if you pull it there.

Which GPU holds it

Pick the card by memory, then by how much context you need. The table below shows the longest context that keeps the model under 90 percent of the card, the point where a desktop or browser stops causing out-of-memory errors.

16 GB. No. The Q4_K_M weights alone are 15.7 GB, and the runtime needs more on top. A 16 GB card such as the RTX 5080 or 4080 SUPER fits a smaller model, not this one.

24 GB. Yes, with limits. The RTX 3090, RTX 4090 and RX 7900 XTX all have 24 GB. Q4_K_M leaves room for about 82K tokens, enough for a long chat or a mid-size codebase. Q5_K_M gets about 39K. Q6_K only fits tight, at about 32K.

32 GB. The comfortable fit. The RTX 5090 holds Q6_K with about 108K tokens, or Q4_K_M with about 197K. Q8_0 fits, but with only about 8K tokens of room, so it is a weights-only quant here.

48 GB. Everything, including the model's full native context. An RTX A6000 or L40S runs Q8_0 with about 238K tokens.

Why long context costs so little here

Most dense models are far hungrier. The cache grows with every token, and each full-attention layer adds to it. Qwen3 32B has 64 such layers with 8 KV heads each, so 128K tokens costs it 32 GB. Qwen3.8 27B costs 8.

That is a quarter of Qwen3 32B's cache, and less than the 8B's. The reason is the layout in the config: three of every four layers use linear attention, which keeps a small fixed state instead of a growing cache. If a tool treats all 64 layers as full attention, it will report 32 GB at 128K and tell you the model does not fit when it does. We count only the 16 layers that keep a cache.

The same shape applies to the older Qwen3.6 27B: the config has the same layers, KV heads and exact parameter count, so its cost-to-run page and best GPU page give the same VRAM math today. We have not added Qwen3.8 itself to the tracker yet.

How to run it, and when to rent

The model card says Qwen3.8 works with Transformers, vLLM, SGLang and TokenSpeed, and recommends the last three for production and high throughput. For a desk-side setup, Ollama lists it as ollama run qwen3.8, which is the 27B at 18 GB with a 256K window. Unlike some other families, the bare tag here is the big model, not a small default. If you want to serve it to your apps, see how to run local AI as an API server.

If you do not own a 24 or 32 GB card, renting one costs less than you might think. On 3 October 2026 our tracker showed an RTX 4090 (24 GB) at $0.34 an hour and an RTX 5090 (32 GB) at $0.69 an hour on Runpod, on demand. Whether renting beats buying depends on your hours, which we cover in rent vs buy.

Referral link

No 32 GB card? Rent one

An RTX 5090 with 32 GB listed at $0.69 an hour on Runpod on demand, and an RTX 4090 with 24 GB at $0.34 an hour (our tracker, 3 Oct 2026). Prices move, so check the live figure before you rent.

See live 24 and 32 GB GPU pricesCompare live prices first →

Referral link — we may earn a commission at no cost to you. Prices on our compare page are the provider's live figures, cheapest first; this never changes the ranking.

What these numbers do not tell you

They do not tell you speed. Tokens per second depends on your engine, your card's memory bandwidth and the quant, and we did not measure any of it. They do not tell you quality either. We did not run benchmarks, so we make no claim about how Qwen3.8 27B compares with other models.

They also assume an fp16 cache and a single user. An 8-bit cache halves the cache column. Several users at once multiply it by the number of live conversations. And the engine's overhead is a middle estimate, so leave headroom rather than aiming for 100 percent.