How much VRAM gpt-oss 120B (MoE) needs, and what it costs to run
Some links on this page are referral links: we may earn a commission, at no cost to you.
Trying to decide what to buy? The best GPU for gpt-oss 120B (MoE) ↓ · Got a card already? See what your PC can run →
Cheapest way to rent it right now
$0.24/hr
NVIDIA DGX Spark (GB10 Grace Blackwell) · MXFP4 · on-demand · ≈ $58/mo at 8h/day
Live price captured Thu, 19 Nov 2026 17:20:08 GMT.It never changes which card is cheapest.
Just want to use it? Per-token API
$0.05in·$0.25out
per 1M tokens · OpenAI GPT OSS 120B on Novita AI · read Oct 29, 2026
No GPU to manage: you pay per token, which suits occasional use. Your prompts run on Novita's servers, not your own machine.
Price as listed on Novita's pricing page on Oct 29, 2026; it may have changed.
Rent a GPU or pay per token?
A per-token API charges for every token, so it is cheapest when you use gpt-oss 120B (MoE) a little. A rented GPU charges by the hour whether it is busy or idle, so it only pays off once you push enough tokens through it. Set your daily volume and see where the line falls for this model, using the cheapest live rental that fits and that we have a memory-bandwidth figure for (DGX Spark (GB10 Grace Blackwell), MXFP4, $0.240/hr on-demand) and Novita AI's published rate.
- Pay per token · Novita AI$3.00/mo
$0.10/day · $0.10 per 1M tokens blended at 3:1
- Rent DGX Spark (GB10 Grace Blackwell) · Vast.ai$9.05/mo
$0.30/day · 1.3 GPU-hours needed
At 1M tokens/day the API is cheaper. The API is cheaper at every volume: at our estimated speed, renting the DGX Spark (GB10 Grace Blackwell) works out to about $0.30 per 1M tokens against the API's $0.10.
At a glance · 3:1 input:output
| Tokens/day | API /mo | Rent /mo |
|---|---|---|
| 100K | $0.30 | $7.21 |
| 1M | $3.00 | $9.05 |
| 10M | $30.00 | $90.54 |
| 100M | $300 | $905 |
Cheaper option in green. 30-day month. Break-even: none, the API stays cheaper.
How we work it out. API cost = input tokens × $0.05 + output tokens × $0.25 per 1M. Rent cost = the hourly price × the GPU time needed, with at least 1 hour billed per day (spin-up, model load and idle time cost money too). GPU time assumes the DGX Spark (GB10 Grace Blackwell) writes ~72 output tokens/sec for gpt-oss 120B (MoE) at MXFP4 (our estimate: its 273 GB/s memory bandwidth from the spec sheet, not yet confirmed by a second source ÷ the bytes of weights read per token, × 75% efficiency, one request at a time) and reads prompt tokens 10× faster than it writes. Above 24 GPU-hours a day you would need more than one card.
These are estimates: the tokens/sec figure is our model, not a benchmark, and batching many requests (vLLM, SGLang) can raise a rented card's throughput a lot, which moves the break-even lower. Rental price captured Sat, 21 Nov 2026 08:20:09 GMT. API price as listed by Novita AI on Oct 29, 2026; API prices change.
Memory needed, by quantisation
At 16k context. Weights + KV cache + ~0.8 GB overhead. gpt-oss 120B (MoE) ships in MXFP4, so its own release leads and no bigger copy is listed: it would only store the same numbers in more bytes. Memory is in binary GB (1,024³ bytes), the unit GPU memory is sold in; Hugging Face and Ollama list downloads in decimal GB, about 7% more.
| Quant | Weights | KV cache | Total | Quality |
|---|---|---|---|---|
| MXFP4official | 60.8 GB | 1.1 GB | 62.7 GB | The model's own release (MXFP4, a 65.2 GB download): the best quality there is. A bigger copy only stores the same numbers in more bytes. Its 4-bit GGUF builds keep the 4-bit experts as they are, so they come out about this size too. |
Cards that can run it
Best quantisation each card fits at 16k context, with live rental price where we track one.
| GPU | VRAM | Best fit | Uses | ~tok/s | Rent from |
|---|---|---|---|---|---|
| Apple M3 Ultra | 512 GB | MXFP4 | 63 GB | — | — |
| Apple M5 Ultra | 512 GB | MXFP4 | 63 GB | — | — |
| Apple M2 Ultra | 192 GB | MXFP4 | 63 GB | — | — |
| Apple M5 Max (18-core CPU / 40-core GPU) | 128 GB | MXFP4 | 63 GB | — | — |
| NVIDIA DGX Spark (GB10 Grace Blackwell) | 128 GB | MXFP4 | 63 GB | — | $0.240/hrVast.ai |
| Apple M4 Max (16-core CPU / 40-core GPU) | 128 GB | MXFP4 | 63 GB | — | — |
| AMD Ryzen AI Max+ 395 (Strix Halo, 128 GB) | 128 GB | MXFP4 | 63 GB | — | — |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 96 GB | MXFP4 | 63 GB | — | $0.533/hrVast.ai |
| Apple M2 Max | 96 GB | MXFP4 | 63 GB | — | — |
| NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition | 96 GB | MXFP4 | 63 GB | — | $0.333/hrVast.ai |
| Apple M4 Pro | 64 GB | MXFP4tight | 63 GB | — | — |
The best GPU for gpt-oss 120B (MoE)
Ranked from live rental prices and computed VRAM fit, not opinion. Every card is judged at its own MXFP4 release with a 8K context, so the comparison is fair; a card that could only run a crushed version isn't listed.
Best value
NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition
Best throughput per dollar among cards that fit.
Cheapest that runs it
NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition
Lowest hourly rental that fits it — $0.333/hr, MXFP4.
No compromise
—
Most headroom and highest throughput.
Every card that runs it, ranked
Sorted by tokens/sec per rental dollar. Throughput is a bandwidth-derived estimate.
| GPU | VRAM | VRAM used | ~tok/s | Cheapest rental | tok/s per $ |
|---|---|---|---|---|---|
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 96 GB | 62 GB65% | — | $0.533 | — |
| NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition | 96 GB | 62 GB65% | — | $0.333 | — |
The value column ranks renting, because that's what we can price precisely (prices captured Thu, 19 Nov 2026 17:20:08 UTC). Buying a card? The fit and throughput columns are what you need; we don't publish a purchase-price ranking because we don't hold verified street prices. Tokens/sec is an estimate, not a benchmark. Got a card already? See everything your PC can run.
gpt-oss-120bHow these numbers are produced
- Model shape comes from the model's own config.json — 36 layers, 64 attention heads, 8 KV heads.
- Weight size is parameters × effective bits-per-weight. Those constants are checked against real published quantised file sizes — median error 0.7% across 40 measurements.
- KV cache is 2 × layers × KV-heads × head-dim × context × 2 bytes. Using KV-heads rather than attention heads is what makes this correct for grouped-query attention; treating a GQA model as multi-head overstates the cache by up to 8×.
- Tokens/sec is an estimate, not a benchmark. Generation is memory-bandwidth-bound, so this is bandwidth ÷ weight-bytes derated to 75%. Real throughput depends on your runtime, batch size and kernels. We don't run our own hardware tests — see our editorial policy.
- This is a mixture-of-experts model. Generation reads only the experts active for each token (4 of 128), so throughput is scored on ~5.1B active parameters, not the full 117B. Memory is the opposite: every expert must still be resident in VRAM, so the figures above use the full weight set.Active = 5.1B, vendor figure: OpenAI model card (huggingface.co/openai/gpt-oss-120b, read 29 Oct 2026): "117B parameters with 5.1B active parameters". params = HF safetensors total. Also set in workers/prices/src/models.ts seeds so the daily ingest keeps it.
- Some memory-bandwidth figures are not yet independently verified.Where that's the case we show no tokens/sec at all rather than a number we can't stand behind.
- Rental prices are pulled hourly from provider APIs (last updated Thu, 19 Nov 2026 17:20:08 GMT). Spot/interruptible pricing can change or vanish without notice.
Try it interactively
Drag the context slider and watch the KV cache fill the card
Running gpt-oss 120B (MoE): common questions
How much VRAM do you need to run gpt-oss 120B (MoE)?
Can an RTX 4090 run gpt-oss 120B (MoE)?
Can an RTX 3090 run gpt-oss 120B (MoE)?
Can an RTX 4060 Ti 16GB run gpt-oss 120B (MoE)?
What's the cheapest way to run gpt-oss 120B (MoE)?
Worth reading before you buy
gpt-oss-120b needs about 64GB of memory, and a smaller quant won't save you
OpenAI's 117B open model ships its experts in 4-bit MXFP4, so the files are about 63 GB whichever quant you pick. Here is how much VRAM and RAM it needs at 8K to 128K context, which GPUs and machines hold it, and when a 24 GB card plus RAM is enough.
Which Gemma, Llama, Mistral or gpt-oss model can you run on an 8 to 24GB GPU?
Google, Meta, Mistral and OpenAI all ship open models that fit a gaming card, if you pick the right size and know each family's trap. Which one fits 8, 12, 16 and 24GB, the quant to use, the Ollama tag to type, and what changes on a Mac, from our VRAM engine and the model cards.
Cheapest A100 80GB rental
Live A100 80GB prices cheapest-first, when the A100 beats an H100 on price-per-job, and the 70B-class workloads it's the sweet spot for.
Cheapest H100 rental
Live H100 prices across Vast and Runpod with the capture date, SXM vs PCIe, and the per-job maths for 70B–120B models.