NVFP4 is Nvidia's 4-bit trick that gets FP8 accuracy at 3x the speed — but only on Blackwell

The new 4-bit format baked into RTX 50-series and Blackwell chips runs AI models at up to 3x the inference throughput of FP8, at nearly the same… Aliteq Lena Fischer · AI & Local Compute Editor

NVFP4 is a 4-bit floating-point format introduced with NVIDIA Blackwell GPUs (including SM120 desktop parts). It achieves accuracy close to FP8 while delivering up to ~3x the inference throughput…

4-bit precision (E2M1: 1 sign, 2 exponent, 1 mantissa bit) — a quarter the size of 16-bit weights.

Near-FP8 accuracy despite being half the bits, thanks to fine-grained scaling.

Up to ~3x FP8 inference throughput on Blackwell, with ~50% less memory traffic per element.

The trick: values are grouped into blocks of 16 sharing a high-precision FP8 scale, plus a per-tensor FP32 scale — so tiny 4-bit numbers stay accurate.

Blackwell-only: Tensor Cores must natively support NVFP4. RTX 50-series and datacenter Blackwell can; earlier GPUs can't accelerate it.

Read the full story

Aliteq NVFP4 is Nvidia's 4-bit trick that gets FP8 accuracy at 3x the speed — but only on Blackwell Read the full story on Aliteq