aliteq.

Qwen3.8 27B NVFP4, how much VRAM it needs and which GPU runs it

The NVFP4 build of Qwen3.8 27B is about 22 GB of files and runs fastest on a Blackwell card like the RTX 5090. What the nvidia and unsloth builds are, the VRAM math, the vLLM command, and measured speeds from other people's tests.

Aliteq TeamPublished 10 min read
Line-art illustration on a dark background with a headline about Qwen3.8 27B VRAM by quant

Some links on this page are referral links: we may earn a commission, at no cost to you.

Share

Qwen3.8 27B NVFP4 is the 4-bit NVFP4 build of Alibaba's dense 27B model. NVIDIA's version (nvidia/Qwen3.8-27B-NVFP4) is 21.9 GB of files and needs about 22 to 25 GB of VRAM with 32K to 128K tokens of context, so a 32GB RTX 5090 runs it, and so do the RTX PRO 6000 Blackwell and DGX Spark. Its speed advantage needs a Blackwell GPU. On an RTX 4090 or 3090 vLLM can load it but runs the math in 16-bit, so a Q4_K_M GGUF is usually the better choice there.

Checked November 17, 2026: file sizes from the Hugging Face API, commands from the model cards and docs, speeds from the tests linked in each row. The Aliteq Team has not run this model on our own card; every speed here is someone else's measurement, attributed.

What the NVFP4 build is

NVFP4 is NVIDIA's 4-bit floating-point format: 4-bit values in blocks of 16 that share an FP8 scale. Blackwell GPUs run it directly on their Tensor Cores. It is NVIDIA's tighter cousin of the open MXFP4 format that gpt-oss ships in. Qwen itself ships Qwen3.8 27B in BF16 and FP8; the two NVFP4 builds people search for come from NVIDIA and from Unsloth, and they are not the same file.

nvidia/Qwen3.8-27B-NVFP4

Files
21.9 GB (3 files)
Made with
NVIDIA Model Optimizer 0.48.0, posted September 2026
What is 4-bit
MLP layers and output head in NVFP4; self-attention and linear-attention layers in FP8

unsloth/Qwen3.8-27B-NVFP4

Files
23.4 GB (2 files)
Made with
Unsloth Dynamic V3.0 (preview)
What is 4-bit
Most layers W4A4; layers Unsloth judges important stay FP8 or BF16, including the output head

Qwen/Qwen3.8-27B-FP8 (for comparison)

Files
30.9 GB
Made with
Qwen
What is 4-bit
Nothing; FP8 weights

Qwen/Qwen3.8-27B (original)

Files
55.6 GB
Made with
Qwen
What is 4-bit
Nothing; BF16 weights

Sizes are the safetensors files listed by the Hugging Face API on November 17, 2026, in decimal GB. NVIDIA's card lists context up to 262K tokens, the model's native length. NVIDIA's card reports scores close to the original: GPQA Diamond 88.01 vs 88.92 for BF16, Terminal-Bench 74.02 vs 75.56, MMMU-Pro 74.86 vs 75.14, and slightly higher on AA-LCR (73.38 vs 72.63) and SciCode (48.41 vs 47.93). Unsloth's model card gives no accuracy figures for its build.

Which one? NVIDIA's is the smaller file and the one most benchmarks below use. Unsloth's guide offers a multi-token-prediction (MTP) option for faster replies; its SGLang support is limited to version 0.5.19, per its model card.

How much VRAM Qwen3.8 27B NVFP4 needs

The math is the same as on our Qwen3.8 27B VRAM page: weights, plus the KV cache, plus about 0.78 GB of runtime overhead. Only 16 of the model's 64 layers keep a KV cache, so context is cheap: 64 KiB per token at 16-bit, half that with the 8-bit cache NVIDIA's model card uses (--kv-cache-dtype fp8_e4m3).

NVFP4 (nvidia), 8-bit cache

Weights
20.4
32K tokens
22.2
128K tokens
25.2
262K tokens
29.2

NVFP4 (nvidia), 16-bit cache

Weights
20.4
32K tokens
23.2
128K tokens
29.2
262K tokens
37.2

FP8 (Qwen), 8-bit cache

Weights
28.8
32K tokens
30.5
128K tokens
33.5
262K tokens
37.5

Q4_K_M GGUF, 16-bit cache

Weights
15.7
32K tokens
18.5
128K tokens
24.5
262K tokens
32.5

All figures in GB of 1,024³ bytes (the unit nvidia-smi shows), computed by us from the file sizes and the config, not measured. The NVFP4 weight figure includes the vision encoder; ComputingForGeeks saw vLLM load 19.05 GiB of text-only weights on an RTX 5090. Leave 5 to 10 percent spare, and check any other context length on cost to run Qwen3.8 27B.

What the table says: a 32GB card holds the NVFP4 build with room for long context, while the FP8 build barely fits at 32K and not beyond. That is the practical reason to pick NVFP4 on a 5090.

Which GPU runs it

[RTX 5090](/gpus/rtx-5090)

Memory
32 GB
Runs NVFP4 natively?
Yes (Blackwell)
Verdict for this model
The best single consumer card; fits even the full 262K context with an 8-bit cache by our math, with little to spare

[RTX PRO 6000 Blackwell](/gpus/rtx-pro-6000-blackwell-workstation)

Memory
96 GB
Runs NVFP4 natively?
Yes
Verdict for this model
Full 262K context with room to serve several users at once

[DGX Spark](/gpus/dgx-spark)

Memory
128 GB unified
Runs NVFP4 natively?
Yes (GB10)
Verdict for this model
Fits easily; slower memory than a 5090, but the most-tested NVFP4 box on NVIDIA's forum

RTX 5080, 5070 Ti

Memory
16 GB
Runs NVFP4 natively?
Yes
Verdict for this model
No: the 20.4 GB of weights don't fit

[RTX 4090](/gpus/rtx-4090)

Memory
24 GB
Runs NVFP4 natively?
No (Ada)
Verdict for this model
vLLM can load it with a Marlin weight-only kernel (16-bit math), leaving little room for context; use Q4_K_M instead

RTX 3090

Memory
24 GB
Runs NVFP4 natively?
No (Ampere)
Verdict for this model
Same as the 4090; Q4_K_M GGUF is the better fit

Mac (Apple silicon)

Memory
unified
Runs NVFP4 natively?
No FP4 hardware
Verdict for this model
Ollama's MLX engine runs the qwen3.8:27b-nvfp4 tag

On the RTX 4090 and 3090: vLLM's docs list its Marlin FP4 kernel for Turing, Ampere, Ada and Hopper and say Marlin "always runs with 16-bit activations", so an NVFP4 checkpoint runs as W4A16 there. You keep the small weights but lose the FP4 speed, and 20.4 GB of weights on a 24 GB card leaves only a few GB for the cache. The 18 GB Q4_K_M GGUF in llama.cpp or Ollama fits better. Want the full list of cards for this model? See the best GPU for Qwen3.8 27B. No GPU at all? A Ryzen AI Max machine with 128GB runs the GGUF build.

How to run Qwen3.8 27B NVFP4

On a Blackwell GPU with a recent vLLM (Unsloth's NVFP4 docs ask for 0.25.0 or newer; NVIDIA's card uses the vllm/vllm-openai:nightly Docker image):

# NVIDIA's build on one 32GB card (NVIDIA's card uses --tensor-parallel-size 4 and --max-model-len 262144 on a GB300)
vllm serve nvidia/Qwen3.8-27B-NVFP4 \
  --kv-cache-dtype fp8_e4m3 \
  --max-model-len 32768

# Unsloth's build, as in Unsloth's Qwen3.8 guide
vllm serve unsloth/Qwen3.8-27B-NVFP4

# Unsloth's build with multi-token prediction (faster single replies, per Unsloth)
vllm serve unsloth/Qwen3.8-27B-NVFP4 --speculative-config '{"method": "mtp", "num_speculative_tokens": 2}'

# SGLang, from NVIDIA's model card
sglang serve --model-path nvidia/Qwen3.8-27B-NVFP4 --kv-cache-dtype fp8_e4m3 --mem-fraction-static 0.85

The 32K context in the first command is our pick for a 32GB card; raise it if the table above says you have room. Each server opens an OpenAI-compatible API on port 8000 (vLLM) that chat apps and scripts can use. Unsloth notes MTP needs about 1 to 2 GB of extra VRAM.

Ollama: ollama run qwen3.8:27b-nvfp4 pulls an 18GB tag that Ollama's library labels MLX, its Apple silicon engine. A 27b-dflash2-nvfp4 tag (19GB) is listed too. On an NVIDIA card, Ollama's default qwen3.8:27b tag (Q4_K_M, 18GB) is the usual pick; we found no Ollama note that the nvfp4 tags use Blackwell's FP4 Tensor Cores.

How fast is it

Three independent tests on one model. Different hardware and settings, so read each row on its own.

ComputingForGeeks, Oct 13, 2026

Setup
RTX 5090, nvidia build, 4K context
Result
One chat: 68.5 tokens/s (vLLM 0.31.0), 73.0 (TensorRT-LLM 1.3.0rc29). 32 chats at once: 1,195 tokens/s (vLLM) and 1,361 (TensorRT-LLM with an FP8 cache)

NVIDIA forum, user shahizat, Aug 15, 2026

Setup
DGX Spark, vLLM 0.27.1, 16 requests at once
Result
Unsloth NVFP4 vs Qwen FP8: 132.1 vs 99.5 output tokens/s decode-heavy, 134.4 vs 104.4 balanced, 87.9 vs 65.6 prompt-heavy (about 30 to 34% faster)

Unsloth's Qwen3.8 guide, read Nov 17, 2026

Setup
1x B200, vLLM
Result
NVFP4 vs BF16: 133.7 vs 89.8 total tokens/s at batch 1 (1.49x), 4,407 vs 3,049 at batch 64 (1.45x)

So on one RTX 5090, expect roughly 70 tokens a second for a single chat by those tests, far more in total if you serve many requests at once. The forum test is the cleanest NVFP4-vs-FP8 comparison: same box, same flags, NVFP4 about a third faster and 7.5 GB smaller.

No Blackwell card? Rent one by the hour

If you have a 24GB card or less, renting an RTX 5090 for an evening is the cheap way to try the NVFP4 build at full speed. On Vast.ai, Nov 17, 2026, RTX 5090 offers started at $0.38 an hour (median $0.71); Runpod listed the RTX 5090 at $1.39 an hour. Live prices for every card are on our cloud GPU price page.

Planning to serve it to a team, or want a managed setup with templates? Runpod runs the same card in its own datacenters.

Buying instead? What an RTX 5090 costs today and RTX PRO 6000 prices track the two cards that run this build best, and is DGX Spark worth it covers the 128GB option.

Related: what quantization is, how to quantize a model to GGUF.

This guide is part of our How much VRAM do you need to run an AI model guide. Related guides: Run a 120B MoE on an 8-24GB GPU with llama.cpp --n-cpu-moe, Ollama has no --n-cpu-moe switch. Here's what it does with MoE experts, Out of memory at long context.

This guide is part of our Which local AI model fits my GPU's VRAM? The 8GB to 32GB chart, model by model guide and our Which Qwen model can you run locally? Every size matched to your GPU guide.

Qwen3.8 27B NVFP4 questions

How much VRAM does Qwen3.8 27B NVFP4 need?
NVIDIA's build has 20.4 GB (1,024³ bytes) of weights. With an 8-bit KV cache it needs about 22.2 GB at 32K tokens and 25.2 GB at 128K by our formula, so a 32GB card is the practical minimum for useful context.
Can an RTX 5090 run Qwen3.8 27B NVFP4?
Yes. The RTX 5090 is Blackwell with 32GB, so it runs NVFP4 natively and holds the model with long context. ComputingForGeeks measured 68.5 tokens/s for one chat in vLLM and 73.0 in TensorRT-LLM (October 2026).
Can an RTX 4090 run Qwen3.8 27B NVFP4?
It can load it, but not fast. The 4090 has no FP4 hardware, so vLLM runs NVFP4 with its Marlin kernel at 16-bit math, and 20.4 GB of weights leaves little of its 24GB for context. The Q4_K_M GGUF (18GB in Ollama) is the better choice on a 4090.
What is the difference between nvidia/Qwen3.8-27B-NVFP4 and unsloth/Qwen3.8-27B-NVFP4?
NVIDIA's build (21.9 GB) is made with Model Optimizer, with NVFP4 on the MLP layers and output head and FP8 on attention. Unsloth's (23.4 GB) uses its Dynamic recipe, keeping more layers in FP8 or BF16, and offers multi-token prediction. NVIDIA publishes accuracy scores close to BF16; Unsloth's card gives none.
Is there an Ollama tag for Qwen3.8 27B NVFP4?
Yes, qwen3.8:27b-nvfp4 (18GB) and qwen3.8:27b-dflash2-nvfp4 (19GB), both labelled MLX in Ollama's library on November 17, 2026. They are built for Ollama's MLX engine on Apple silicon.
Is NVFP4 faster than FP8 for Qwen3.8 27B?
In one DGX Spark test on NVIDIA's forum (August 2026, vLLM 0.27.1, 16 requests at once), the Unsloth NVFP4 build produced about 30 to 34% more tokens per second than Qwen's FP8 build, and it is 7.5 GB smaller.

For the format itself, how NVFP4 compares with FP8 and GGUF, and how to make your own NVFP4 checkpoint, read what NVFP4 is. For other models that fit a 32GB card, see the best local LLMs for 32GB of VRAM.

Found this useful? Share it

Share

Sources

What this guide was researched from. How we check facts and compute numbers: methodology.

Aliteq

Aliteq Team

Aliteq started in 2019 as a PC hardware shop and is now a US-based publication. We build the VRAM and cost calculators on this site, track live cloud-GPU prices, and write guides with AI-assisted drafting that we check against primary sources before publishing.

Work out the hardware

GPUs in this article

Specs, the AI models each one runs, and where to buy or rent it.

Email list

Local AI, only the changes that matter

A new model your GPU can actually run, a GPU price move worth acting on, or a fix that saves you an evening. Nothing in between.

Written and sent by hand. One-click unsubscribe in every email. Emails may contain referral links.

Keep reading