The NVFP4 build of Qwen3.8 27B is about 22 GB of files and runs fastest on a Blackwell card like the RTX 5090. What the nvidia and unsloth builds are, the VRAM math, the vLLM command, and measured speeds from other people's tests.
Some links on this page are referral links: we may earn a commission, at no cost to you.
Share
Qwen3.8 27B NVFP4 is the 4-bit NVFP4 build of Alibaba's dense 27B model. NVIDIA's version (nvidia/Qwen3.8-27B-NVFP4) is 21.9 GB of files and needs about 22 to 25 GB of VRAM with 32K to 128K tokens of context, so a 32GB RTX 5090 runs it, and so do the RTX PRO 6000 Blackwell and DGX Spark. Its speed advantage needs a Blackwell GPU. On an RTX 4090 or 3090 vLLM can load it but runs the math in 16-bit, so a Q4_K_M GGUF is usually the better choice there.
Checked November 17, 2026: file sizes from the Hugging Face API, commands from the model cards and docs, speeds from the tests linked in each row. The Aliteq Team has not run this model on our own card; every speed here is someone else's measurement, attributed.
What the NVFP4 build is
NVFP4 is NVIDIA's 4-bit floating-point format: 4-bit values in blocks of 16 that share an FP8 scale. Blackwell GPUs run it directly on their Tensor Cores. It is NVIDIA's tighter cousin of the open MXFP4 format that gpt-oss ships in. Qwen itself ships Qwen3.8 27B in BF16 and FP8; the two NVFP4 builds people search for come from NVIDIA and from Unsloth, and they are not the same file.
nvidia/Qwen3.8-27B-NVFP4
Files
21.9 GB (3 files)
Made with
NVIDIA Model Optimizer 0.48.0, posted September 2026
What is 4-bit
MLP layers and output head in NVFP4; self-attention and linear-attention layers in FP8
unsloth/Qwen3.8-27B-NVFP4
Files
23.4 GB (2 files)
Made with
Unsloth Dynamic V3.0 (preview)
What is 4-bit
Most layers W4A4; layers Unsloth judges important stay FP8 or BF16, including the output head
Qwen/Qwen3.8-27B-FP8 (for comparison)
Files
30.9 GB
Made with
Qwen
What is 4-bit
Nothing; FP8 weights
Qwen/Qwen3.8-27B (original)
Files
55.6 GB
Made with
Qwen
What is 4-bit
Nothing; BF16 weights
Files
Made with
What is 4-bit
nvidia/Qwen3.8-27B-NVFP4
21.9 GB (3 files)
NVIDIA Model Optimizer 0.48.0, posted September 2026
MLP layers and output head in NVFP4; self-attention and linear-attention layers in FP8
unsloth/Qwen3.8-27B-NVFP4
23.4 GB (2 files)
Unsloth Dynamic V3.0 (preview)
Most layers W4A4; layers Unsloth judges important stay FP8 or BF16, including the output head
Qwen/Qwen3.8-27B-FP8 (for comparison)
30.9 GB
Qwen
Nothing; FP8 weights
Qwen/Qwen3.8-27B (original)
55.6 GB
Qwen
Nothing; BF16 weights
Sizes are the safetensors files listed by the Hugging Face API on November 17, 2026, in decimal GB. NVIDIA's card lists context up to 262K tokens, the model's native length. NVIDIA's card reports scores close to the original: GPQA Diamond 88.01 vs 88.92 for BF16, Terminal-Bench 74.02 vs 75.56, MMMU-Pro 74.86 vs 75.14, and slightly higher on AA-LCR (73.38 vs 72.63) and SciCode (48.41 vs 47.93). Unsloth's model card gives no accuracy figures for its build.
Which one? NVIDIA's is the smaller file and the one most benchmarks below use. Unsloth's guide offers a multi-token-prediction (MTP) option for faster replies; its SGLang support is limited to version 0.5.19, per its model card.
How much VRAM Qwen3.8 27B NVFP4 needs
The math is the same as on our Qwen3.8 27B VRAM page: weights, plus the KV cache, plus about 0.78 GB of runtime overhead. Only 16 of the model's 64 layers keep a KV cache, so context is cheap: 64 KiB per token at 16-bit, half that with the 8-bit cache NVIDIA's model card uses (--kv-cache-dtype fp8_e4m3).
NVFP4 (nvidia), 8-bit cache
Weights
20.4
32K tokens
22.2
128K tokens
25.2
262K tokens
29.2
NVFP4 (nvidia), 16-bit cache
Weights
20.4
32K tokens
23.2
128K tokens
29.2
262K tokens
37.2
FP8 (Qwen), 8-bit cache
Weights
28.8
32K tokens
30.5
128K tokens
33.5
262K tokens
37.5
Q4_K_M GGUF, 16-bit cache
Weights
15.7
32K tokens
18.5
128K tokens
24.5
262K tokens
32.5
Weights
32K tokens
128K tokens
262K tokens
NVFP4 (nvidia), 8-bit cache
20.4
22.2
25.2
29.2
NVFP4 (nvidia), 16-bit cache
20.4
23.2
29.2
37.2
FP8 (Qwen), 8-bit cache
28.8
30.5
33.5
37.5
Q4_K_M GGUF, 16-bit cache
15.7
18.5
24.5
32.5
All figures in GB of 1,024³ bytes (the unit nvidia-smi shows), computed by us from the file sizes and the config, not measured. The NVFP4 weight figure includes the vision encoder; ComputingForGeeks saw vLLM load 19.05 GiB of text-only weights on an RTX 5090. Leave 5 to 10 percent spare, and check any other context length on cost to run Qwen3.8 27B.
What the table says: a 32GB card holds the NVFP4 build with room for long context, while the FP8 build barely fits at 32K and not beyond. That is the practical reason to pick NVFP4 on a 5090.
Which GPU runs it
[RTX 5090](/gpus/rtx-5090)
Memory
32 GB
Runs NVFP4 natively?
Yes (Blackwell)
Verdict for this model
The best single consumer card; fits even the full 262K context with an 8-bit cache by our math, with little to spare
[RTX PRO 6000 Blackwell](/gpus/rtx-pro-6000-blackwell-workstation)
Memory
96 GB
Runs NVFP4 natively?
Yes
Verdict for this model
Full 262K context with room to serve several users at once
[DGX Spark](/gpus/dgx-spark)
Memory
128 GB unified
Runs NVFP4 natively?
Yes (GB10)
Verdict for this model
Fits easily; slower memory than a 5090, but the most-tested NVFP4 box on NVIDIA's forum
RTX 5080, 5070 Ti
Memory
16 GB
Runs NVFP4 natively?
Yes
Verdict for this model
No: the 20.4 GB of weights don't fit
[RTX 4090](/gpus/rtx-4090)
Memory
24 GB
Runs NVFP4 natively?
No (Ada)
Verdict for this model
vLLM can load it with a Marlin weight-only kernel (16-bit math), leaving little room for context; use Q4_K_M instead
RTX 3090
Memory
24 GB
Runs NVFP4 natively?
No (Ampere)
Verdict for this model
Same as the 4090; Q4_K_M GGUF is the better fit
Mac (Apple silicon)
Memory
unified
Runs NVFP4 natively?
No FP4 hardware
Verdict for this model
Ollama's MLX engine runs the qwen3.8:27b-nvfp4 tag
Memory
Runs NVFP4 natively?
Verdict for this model
[RTX 5090](/gpus/rtx-5090)
32 GB
Yes (Blackwell)
The best single consumer card; fits even the full 262K context with an 8-bit cache by our math, with little to spare
[RTX PRO 6000 Blackwell](/gpus/rtx-pro-6000-blackwell-workstation)
96 GB
Yes
Full 262K context with room to serve several users at once
[DGX Spark](/gpus/dgx-spark)
128 GB unified
Yes (GB10)
Fits easily; slower memory than a 5090, but the most-tested NVFP4 box on NVIDIA's forum
RTX 5080, 5070 Ti
16 GB
Yes
No: the 20.4 GB of weights don't fit
[RTX 4090](/gpus/rtx-4090)
24 GB
No (Ada)
vLLM can load it with a Marlin weight-only kernel (16-bit math), leaving little room for context; use Q4_K_M instead
RTX 3090
24 GB
No (Ampere)
Same as the 4090; Q4_K_M GGUF is the better fit
Mac (Apple silicon)
unified
No FP4 hardware
Ollama's MLX engine runs the qwen3.8:27b-nvfp4 tag
On the RTX 4090 and 3090: vLLM's docs list its Marlin FP4 kernel for Turing, Ampere, Ada and Hopper and say Marlin "always runs with 16-bit activations", so an NVFP4 checkpoint runs as W4A16 there. You keep the small weights but lose the FP4 speed, and 20.4 GB of weights on a 24 GB card leaves only a few GB for the cache. The 18 GB Q4_K_M GGUF in llama.cpp or Ollama fits better. Want the full list of cards for this model? See the best GPU for Qwen3.8 27B. No GPU at all? A Ryzen AI Max machine with 128GB runs the GGUF build.
How to run Qwen3.8 27B NVFP4
On a Blackwell GPU with a recent vLLM (Unsloth's NVFP4 docs ask for 0.25.0 or newer; NVIDIA's card uses the vllm/vllm-openai:nightly Docker image):
# NVIDIA's build on one 32GB card (NVIDIA's card uses --tensor-parallel-size 4 and --max-model-len 262144 on a GB300)
vllm serve nvidia/Qwen3.8-27B-NVFP4 \
--kv-cache-dtype fp8_e4m3 \
--max-model-len 32768
# Unsloth's build, as in Unsloth's Qwen3.8 guide
vllm serve unsloth/Qwen3.8-27B-NVFP4
# Unsloth's build with multi-token prediction (faster single replies, per Unsloth)
vllm serve unsloth/Qwen3.8-27B-NVFP4 --speculative-config '{"method": "mtp", "num_speculative_tokens": 2}'
# SGLang, from NVIDIA's model card
sglang serve --model-path nvidia/Qwen3.8-27B-NVFP4 --kv-cache-dtype fp8_e4m3 --mem-fraction-static 0.85
The 32K context in the first command is our pick for a 32GB card; raise it if the table above says you have room. Each server opens an OpenAI-compatible API on port 8000 (vLLM) that chat apps and scripts can use. Unsloth notes MTP needs about 1 to 2 GB of extra VRAM.
Ollama:ollama run qwen3.8:27b-nvfp4 pulls an 18GB tag that Ollama's library labels MLX, its Apple silicon engine. A 27b-dflash2-nvfp4 tag (19GB) is listed too. On an NVIDIA card, Ollama's default qwen3.8:27b tag (Q4_K_M, 18GB) is the usual pick; we found no Ollama note that the nvfp4 tags use Blackwell's FP4 Tensor Cores.
How fast is it
Three independent tests on one model. Different hardware and settings, so read each row on its own.
ComputingForGeeks, Oct 13, 2026
Setup
RTX 5090, nvidia build, 4K context
Result
One chat: 68.5 tokens/s (vLLM 0.31.0), 73.0 (TensorRT-LLM 1.3.0rc29). 32 chats at once: 1,195 tokens/s (vLLM) and 1,361 (TensorRT-LLM with an FP8 cache)
NVIDIA forum, user shahizat, Aug 15, 2026
Setup
DGX Spark, vLLM 0.27.1, 16 requests at once
Result
Unsloth NVFP4 vs Qwen FP8: 132.1 vs 99.5 output tokens/s decode-heavy, 134.4 vs 104.4 balanced, 87.9 vs 65.6 prompt-heavy (about 30 to 34% faster)
Unsloth's Qwen3.8 guide, read Nov 17, 2026
Setup
1x B200, vLLM
Result
NVFP4 vs BF16: 133.7 vs 89.8 total tokens/s at batch 1 (1.49x), 4,407 vs 3,049 at batch 64 (1.45x)
Setup
Result
ComputingForGeeks, Oct 13, 2026
RTX 5090, nvidia build, 4K context
One chat: 68.5 tokens/s (vLLM 0.31.0), 73.0 (TensorRT-LLM 1.3.0rc29). 32 chats at once: 1,195 tokens/s (vLLM) and 1,361 (TensorRT-LLM with an FP8 cache)
NVIDIA forum, user shahizat, Aug 15, 2026
DGX Spark, vLLM 0.27.1, 16 requests at once
Unsloth NVFP4 vs Qwen FP8: 132.1 vs 99.5 output tokens/s decode-heavy, 134.4 vs 104.4 balanced, 87.9 vs 65.6 prompt-heavy (about 30 to 34% faster)
Unsloth's Qwen3.8 guide, read Nov 17, 2026
1x B200, vLLM
NVFP4 vs BF16: 133.7 vs 89.8 total tokens/s at batch 1 (1.49x), 4,407 vs 3,049 at batch 64 (1.45x)
So on one RTX 5090, expect roughly 70 tokens a second for a single chat by those tests, far more in total if you serve many requests at once. The forum test is the cleanest NVFP4-vs-FP8 comparison: same box, same flags, NVFP4 about a third faster and 7.5 GB smaller.
No Blackwell card? Rent one by the hour
If you have a 24GB card or less, renting an RTX 5090 for an evening is the cheap way to try the NVFP4 build at full speed. On Vast.ai, Nov 17, 2026, RTX 5090 offers started at $0.38 an hour (median $0.71); Runpod listed the RTX 5090 at $1.39 an hour. Live prices for every card are on our cloud GPU price page.
Planning to serve it to a team, or want a managed setup with templates? Runpod runs the same card in its own datacenters.
NVIDIA's build has 20.4 GB (1,024³ bytes) of weights. With an 8-bit KV cache it needs about 22.2 GB at 32K tokens and 25.2 GB at 128K by our formula, so a 32GB card is the practical minimum for useful context.
Can an RTX 5090 run Qwen3.8 27B NVFP4?
Yes. The RTX 5090 is Blackwell with 32GB, so it runs NVFP4 natively and holds the model with long context. ComputingForGeeks measured 68.5 tokens/s for one chat in vLLM and 73.0 in TensorRT-LLM (October 2026).
Can an RTX 4090 run Qwen3.8 27B NVFP4?
It can load it, but not fast. The 4090 has no FP4 hardware, so vLLM runs NVFP4 with its Marlin kernel at 16-bit math, and 20.4 GB of weights leaves little of its 24GB for context. The Q4_K_M GGUF (18GB in Ollama) is the better choice on a 4090.
What is the difference between nvidia/Qwen3.8-27B-NVFP4 and unsloth/Qwen3.8-27B-NVFP4?
NVIDIA's build (21.9 GB) is made with Model Optimizer, with NVFP4 on the MLP layers and output head and FP8 on attention. Unsloth's (23.4 GB) uses its Dynamic recipe, keeping more layers in FP8 or BF16, and offers multi-token prediction. NVIDIA publishes accuracy scores close to BF16; Unsloth's card gives none.
Is there an Ollama tag for Qwen3.8 27B NVFP4?
Yes, qwen3.8:27b-nvfp4 (18GB) and qwen3.8:27b-dflash2-nvfp4 (19GB), both labelled MLX in Ollama's library on November 17, 2026. They are built for Ollama's MLX engine on Apple silicon.
Is NVFP4 faster than FP8 for Qwen3.8 27B?
In one DGX Spark test on NVIDIA's forum (August 2026, vLLM 0.27.1, 16 requests at once), the Unsloth NVFP4 build produced about 30 to 34% more tokens per second than Qwen's FP8 build, and it is 7.5 GB smaller.
For the format itself, how NVFP4 compares with FP8 and GGUF, and how to make your own NVFP4 checkpoint, read what NVFP4 is. For other models that fit a 32GB card, see the best local LLMs for 32GB of VRAM.