What it costs to run Phi-4 14B
14.7B params16k max contextmit
Memory needed, by quantisation
At 16k context. Weights + KV cache + ~0.8 GB overhead.
| Quant | Weights | KV cache | Total | Quality |
|---|---|---|---|---|
| F16 | 27.3 GB | 3.1 GB | 31.2 GB | Full precision. Reference quality, twice the size of Q8 for no practical gain in most chat use. |
| Q8_0 | 14.5 GB | 3.1 GB | 18.4 GB | Effectively lossless. Use when VRAM is not the constraint. |
| Q6_K | 11.2 GB | 3.1 GB | 15.1 GB | Very close to Q8 at meaningfully less memory. A safe high-quality pick. |
| Q5_K_M | 9.7 GB | 3.1 GB | 13.6 GB | Small, hard-to-notice quality loss. Good balance. |
| Q5_0 | 9.5 GB | 3.1 GB | 13.4 GB | Older-style 5-bit. Q5_K_M is usually the better pick at the same size. |
| Q4_K_M | 8.3 GB | 3.1 GB | 12.2 GB | The community default. Best quality-per-gigabyte for most people. |
| Q4_0 | 7.8 GB | 3.1 GB | 11.7 GB | Older-style 4-bit, measurably worse than Q4_K_M at a similar size. Avoid unless required. |
| IQ4_XS | 7.3 GB | 3.1 GB | 11.2 GB | Newer 4-bit, smaller than Q4_K_M with comparable quality. Needs a recent llama.cpp. |
| Q3_K_M | 6.7 GB | 3.1 GB | 10.6 GB | Noticeable degradation. Use to fit a larger model that would otherwise not run. |
| Q2_K | 5.7 GB | 3.1 GB | 9.6 GB | Heavy degradation. Almost always better to run a smaller model at Q4_K_M instead. |
| IQ3_XS | 5.6 GB | 3.1 GB | 9.5 GB | Aggressive. Usually better than a smaller model at Q4, but test before trusting it. |
Cards that can run it
Best quantisation each card fits at 16k context, with live rental price where we track one.
| GPU | VRAM | Best fit | Uses | ~tok/s | Rent from |
|---|---|---|---|---|---|
| Apple M3 Ultra | 512 GB | F16 | 31 GB | ~21 | — |
| Apple M2 Ultra | 192 GB | F16 | 31 GB | ~21 | — |
| Apple M4 Max (16-core CPU / 40-core GPU) | 128 GB | F16 | 31 GB | ~14 | — |
| AMD Ryzen AI Max+ 395 (Strix Halo, 128 GB) | 128 GB | F16 | 31 GB | ~7 | — |
| NVIDIA DGX Spark (GB10 Grace Blackwell) | 128 GB | F16 | 31 GB | ~7 | — |
| NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition | 96 GB | F16 | 31 GB | ~46 | $1.690/hr |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 96 GB | F16 | 31 GB | ~46 | — |
| Apple M2 Max | 96 GB | F16 | 31 GB | ~10 | — |
| Apple M4 Pro | 64 GB | F16 | 31 GB | ~7 | — |
| NVIDIA RTX 6000 Ada Generation | 48 GB | F16 | 31 GB | ~25 | $0.428/hr |
| NVIDIA RTX A6000 | 48 GB | F16 | 31 GB | ~20 | $0.330/hr |
| AMD Radeon PRO W7900 | 48 GB | F16 | 31 GB | ~22 | — |
| Apple M4 Max (14-core CPU / 32-core GPU) | 36 GB | F16 | 31 GB | ~10 | — |
| NVIDIA RTX 5000 Ada Generation | 32 GB | F16tight | 31 GB | ~15 | $0.490/hr |
| NVIDIA GeForce RTX 5090 | 32 GB | F16tight | 31 GB | ~46 | $0.058/hr |
| NVIDIA RTX A5000 | 24 GB | Q8_0 | 18 GB | ~37 | $0.160/hr |
| NVIDIA GeForce RTX 4090 | 24 GB | Q8_0 | 18 GB | ~49 | $0.136/hr |
| NVIDIA GeForce RTX 3090 Ti | 24 GB | Q8_0 | 18 GB | ~49 | $0.116/hr |
| AMD Radeon RX 7900 XTX | 24 GB | Q8_0 | 18 GB | ~46 | — |
| NVIDIA GeForce RTX 3090 | 24 GB | Q8_0 | 18 GB | ~45 | — |
| AMD Radeon RX 7900 XT | 20 GB | Q8_0tight | 18 GB | ~39 | — |
| NVIDIA GeForce RTX 5080 | 16 GB | Q6_Ktight | 15 GB | ~60 | $0.190/hr |
| NVIDIA RTX A4000 | 16 GB | Q6_Ktight | 15 GB | ~28 | $0.170/hr |
| NVIDIA GeForce RTX 5060 Ti 16GB | 16 GB | Q6_Ktight | 15 GB | ~28 | $0.083/hr |
| NVIDIA GeForce RTX 5070 Ti | 16 GB | Q6_Ktight | 15 GB | ~56 | $0.096/hr |
| NVIDIA GeForce RTX 4080 SUPER | 16 GB | Q6_Ktight | 15 GB | ~46 | $0.134/hr |
| NVIDIA GeForce RTX 4080 | 16 GB | Q6_Ktight | 15 GB | ~45 | — |
| NVIDIA GeForce RTX 4070 Ti SUPER | 16 GB | Q6_Ktight | 15 GB | ~42 | $0.190/hr |
| NVIDIA GeForce RTX 4060 Ti 16GB | 16 GB | Q6_Ktight | 15 GB | ~18 | — |
| NVIDIA GeForce RTX 4070 SUPER | 12 GB | Q4_0tight | 12 GB | ~45 | $0.082/hr |
| NVIDIA GeForce RTX 4070 | 12 GB | Q4_0tight | 12 GB | ~45 | $0.094/hr |
| NVIDIA GeForce RTX 4070 Ti | 12 GB | Q4_0tight | 12 GB | ~45 | — |
| NVIDIA GeForce RTX 5070 | 12 GB | Q4_0tight | 12 GB | ~60 | — |
| NVIDIA GeForce RTX 3080 Ti | 12 GB | Q4_0tight | 12 GB | ~82 | $0.062/hr |
| NVIDIA GeForce RTX 3080 12GB | 12 GB | Q4_0tight | 12 GB | ~82 | $0.062/hr |
| NVIDIA GeForce RTX 3060 12GB | 12 GB | Q4_0tight | 12 GB | ~32 | — |
| NVIDIA GeForce RTX 3080 10GB | 10 GB | Q2_Ktight | 10 GB | ~93 | — |
How these numbers are produced
- Model shape comes from the model's own config.json — 40 layers, 40 attention heads, 10 KV heads.
- Weight size is parameters × effective bits-per-weight. Those constants are checked against real published quantised file sizes — median error 0.7% across 40 measurements.
- KV cache is 2 × layers × KV-heads × head-dim × context × 2 bytes. Using KV-heads rather than attention heads is what makes this correct for grouped-query attention; treating a GQA model as multi-head overstates the cache by up to 8×.
- Tokens/sec is an estimate, not a benchmark. Generation is memory-bandwidth-bound, so this is bandwidth ÷ weight-bytes derated to 75%. Real throughput depends on your runtime, batch size and kernels. We don't run our own hardware tests — see our editorial policy.
- Rental prices are pulled hourly from provider APIs (last updated Tue, 21 Jul 2026 15:20:05 GMT). Spot/interruptible pricing can change or vanish without notice.