OpenAI's open-weight 120B model sounds impossible for consumer hardware. But it's a Mixture-of-Experts model, and that changes the math completely. Here's what it really takes to run it.
'120 billion parameters' sounds like a model that needs a rack of H100s, but gpt-oss-120b is far more approachable than the number implies — because of one architectural fact. It's a Mixture-of-Experts model: 120B parameters total, but only around 5 billion active per token. And it ships natively in MXFP4, which puts the full weights at roughly 60GB. That combination — MoE efficiency plus 4-bit weights — means you don't need a datacenter. A 64GB+ unified-memory machine runs it, and even a 24GB GPU can with the MoE offload trick. Here's the real requirement.
Why MoE changes the math
The key to gpt-oss-120b is separating two things people conflate: memory and compute. All 120 billion parameters have to be loaded into memory — that's the ~60GB requirement, and it doesn't shrink. But only ~5 billion of them run for any given token, so the compute per token is like running a 5B model. That's why it's fast if you can fit it, and why it fits places a 120B dense model never could. A 64GB unified-memory machine holds the weights and runs it comfortably; a 24GB GPU can't hold 60GB, so you offload the experts to system RAM, where the MoE design keeps the speed penalty small. Same principle that makes Qwen3-Coder-30B so efficient, scaled up.
Does it fit? — gpt-oss-120b (MXFP4) needs ≈60GB
gpt-oss-120b (MXFP4) needs ≈60 GB
RTX 4090 (24GB)24 GBover 36 GB
RTX 5090 (32GB)32 GBover 28 GB
Strix Halo (96GB)96 GBfits
Mac Studio (128GB)128 GBfits
A 24GB GPU can't hold it (offload required), but a 64GB+ unified-memory machine or big multi-GPU rig clears it.
120B parameters in memory, ~5B active per token — MoE is what makes a model this size run on a mini-PC instead of a rack. · Unsplash
What to actually run it on
The cleanest home for gpt-oss-120b is a large unified-memory machine — a 96GB Strix Halo or a 64GB+ Mac holds the ~60GB weights with room for context and runs it well on efficient hardware. Failing that, a dual-GPU rig with enough combined VRAM, or a single 24GB card leaning on --n-cpu-moe offload to push experts into system RAM. What you don't need is a datacenter — the whole point of the MoE design is that a 120B model becomes a machine you can actually own.
Quick answers
Can I run gpt-oss-120b on a single 24GB GPU?
Yes, but only with offloading. The ~60GB of weights won't fit in 24GB of VRAM, so you use llama.cpp's --n-cpu-moe to push most of the MoE experts to system RAM. Because only ~5B parameters are active per token, the speed penalty is smaller than offloading a dense model would incur. You'll need plenty of system RAM (64GB+), and it'll be slower than running it fully in memory, but it works.
How much memory does gpt-oss-120b need?
About 60GB to hold all 120B parameters in MXFP4 (the format it ships in). That's the hard requirement — all parameters must be loaded even though only ~5B run per token. So the ideal machine has 64GB+ of fast memory: a large unified-memory system like a Strix Halo or Mac, or a multi-GPU setup. On less memory you offload the experts to system RAM, trading speed for the ability to run it at all.
Is gpt-oss-120b better than smaller models I can run more easily?
It's more capable than a 30B model, but whether it's worth the extra hardware depends on your needs. For many tasks, a 30B-class model on a single 24GB card is faster and easier and good enough. gpt-oss-120b makes sense if you want frontier-adjacent capability locally and have (or will build) a 64GB+ memory machine. Try a smaller model first; reach for 120B when you've hit its limits.
gpt-oss-120b proves the MoE point: a 120B model that runs on a mini-PC, because memory (not compute) is the constraint and 4-bit weights keep it to ~60GB. Best on a 96GB Strix Halo or big Mac, or a 24GB GPU with offload. Size it in the VRAM calculator.
