How much VRAM Qwen3.8-Flash-Next (MoE) needs, and what it costs to run
Some links on this page are referral links: we may earn a commission, at no cost to you.
Trying to decide what to buy? The best GPU for Qwen3.8-Flash-Next (MoE) →
Cheapest way to rent it right now
$0.27/hr
NVIDIA DGX Spark (GB10 Grace Blackwell) · Q5_K_M · on-demand · ≈ $65/mo at 8h/day
Live price captured Thu, 12 Nov 2026 22:20:09 GMT.It never changes which card is cheapest.
Memory needed, by quantisation
At 16k context. Weights + KV cache + ~0.8 GB overhead. Memory is in binary GB (1,024³ bytes), the unit GPU memory is sold in; Hugging Face and Ollama list downloads in decimal GB, about 7% more.
| Quant | Weights | KV cache | Total | Quality |
|---|---|---|---|---|
| F16 | 335.3 GB | 0.4 GB | 336.4 GB | Full precision. Reference quality, twice the size of Q8 for no practical gain in most chat use. |
| Q8_0 | 178.1 GB | 0.4 GB | 179.3 GB | Effectively lossless. Use when VRAM is not the constraint. |
| Q6_K | 137.5 GB | 0.4 GB | 138.6 GB | Very close to Q8 at meaningfully less memory. A safe high-quality pick. |
| Q5_K_M | 119.0 GB | 0.4 GB | 120.2 GB | Small, hard-to-notice quality loss. Good balance. |
| Q5_0 | 116.1 GB | 0.4 GB | 117.2 GB | Older-style 5-bit. Q5_K_M is usually the better pick at the same size. |
| Q4_K_M | 101.6 GB | 0.4 GB | 102.8 GB | The community default. Best quality-per-gigabyte for most people. |
| Q4_0 | 95.3 GB | 0.4 GB | 96.5 GB | Older-style 4-bit, measurably worse than Q4_K_M at a similar size. Avoid unless required. |
| IQ4_XS | 89.1 GB | 0.4 GB | 90.2 GB | Newer 4-bit, smaller than Q4_K_M with comparable quality. Needs a recent llama.cpp. |
| Q3_K_M | 81.9 GB | 0.4 GB | 83.1 GB | Noticeable degradation. Use to fit a larger model that would otherwise not run. |
| Q2_K | 70.2 GB | 0.4 GB | 71.4 GB | Heavy degradation. Almost always better to run a smaller model at Q4_K_M instead. |
| IQ3_XS | 69.2 GB | 0.4 GB | 70.3 GB | Aggressive. Usually better than a smaller model at Q4, but test before trusting it. |
Cards that can run it
Best quantisation each card fits at 16k context, with live rental price where we track one.
| GPU | VRAM | Best fit | Uses | ~tok/s | Rent from |
|---|---|---|---|---|---|
| Apple M5 Ultra | 512 GB | F16 | 336 GB | — | — |
| Apple M3 Ultra | 512 GB | F16 | 336 GB | — | — |
| Apple M2 Ultra | 192 GB | Q8_0tight | 179 GB | — | — |
| AMD Ryzen AI Max+ 395 (Strix Halo, 128 GB) | 128 GB | Q5_K_Mtight | 120 GB | — | — |
| Apple M4 Max (16-core CPU / 40-core GPU) | 128 GB | Q5_K_Mtight | 120 GB | — | — |
| NVIDIA DGX Spark (GB10 Grace Blackwell) | 128 GB | Q5_K_Mtight | 120 GB | — | $0.269/hrVast.ai |
| Apple M5 Max (18-core CPU / 40-core GPU) | 128 GB | Q5_K_Mtight | 120 GB | — | — |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 96 GB | IQ4_XStight | 90 GB | — | $0.467/hrVast.ai |
| NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition | 96 GB | IQ4_XStight | 90 GB | — | $0.295/hrVast.ai |
| Apple M2 Max | 96 GB | IQ4_XStight | 90 GB | — | — |
How these numbers are produced
- Model shape comes from the model's own config.json — 12 layers, 24 attention heads, 2 KV heads.
- Weight size is parameters × effective bits-per-weight. Those constants are checked against real published quantised file sizes — median error 0.7% across 40 measurements.
- KV cache is 2 × layers × KV-heads × head-dim × context × 2 bytes. Using KV-heads rather than attention heads is what makes this correct for grouped-query attention; treating a GQA model as multi-head overstates the cache by up to 8×.
- Tokens/sec is an estimate, not a benchmark. Generation is memory-bandwidth-bound, so this is bandwidth ÷ weight-bytes derated to 75%. Real throughput depends on your runtime, batch size and kernels. We don't run our own hardware tests — see our editorial policy.
- This is a mixture-of-experts model. Generation reads only the experts active for each token (10 of 512), so throughput is scored on ~6.0B active parameters, not the full 180B. Memory is the opposite: every expert must still be resident in VRAM, so the figures above use the full weight set.Hybrid MoE with vision encoder: 48 layers, full attention every 4th (full_attention_interval 4) = 12 full-attention (QSA) + 36 Gated DeltaNet linear layers; layers = the 12 KV layers. Model card: 125B params with 6B activated, plus 51B n-gram embedding (Qwen says it is amenable to offloading) and 4B MTP; params = safetensors total (~180B), so the VRAM figure assumes everything resident. 10 routed + 1 shared expert of 512. Licence qwen-community-1.0 (not Apache). HF config.json + model card read 28 Oct 2026.
- Some memory-bandwidth figures are not yet independently verified.Where that's the case we show no tokens/sec at all rather than a number we can't stand behind.
- Rental prices are pulled hourly from provider APIs (last updated Thu, 12 Nov 2026 22:20:09 GMT). Spot/interruptible pricing can change or vanish without notice.
Try it interactively
Drag the context slider and watch the KV cache fill the card
Worth reading before you buy
What setup runs Qwen3.8 Flash Next locally on your 8GB to 24GB GPU?
A 125B model on a gaming card sounds wrong. Flash Next uses only 6B parameters per token, so the experts can live in system RAM and the n-gram table on your SSD. What each card size needs, which quants people run, and where the quality argument stands.
Qwen3.8-Max open weights (2.4T-A95B): what it takes to run, and the 27B you can actually run
Alibaba put its Max-class model on Hugging Face as Qwen3.8-2.4T-A95B. The download is 2.5 TB in FP8 and the smallest community quant is still 397 GB. Here is the file-size math, the license, and the sibling that fits a 24 GB card.
How much VRAM do you need?
About 0.57 GB per billion parameters at 4-bit, plus room for context: the rule behind the numbers on this page, with worked examples.
The best GPU for local AI right now
The card with enough VRAM for your model at the lowest price, budget by budget.