OpenAI released open-weight models you can run yourself. The 20B fits a 16GB card thanks to clever quantization; the 120B needs a serious rig. Here's how to run gpt-oss locally.
With Ollama, in one command: `ollama run gpt-oss:20b`. gpt-oss is OpenAI's family of open-weight models — yes, from the ChatGPT company — that you can download and run entirely on your own hardware. The smaller gpt-oss-20b fits in about 16GB of VRAM, which is remarkable for a 20B model, thanks to a Mixture-of-Experts design and aggressive MXFP4 quantization. The larger gpt-oss-120b needs 60-80GB, so it's for serious rigs. These are built for reasoning, agentic tasks, and developer use. Here's how to run them and what you need.
The setup, and the VRAM reality
Running gpt-oss-20b is genuinely easy — install Ollama and run ollama run gpt-oss:20b. The interesting part is why a 20-billion-parameter model fits on a 16GB card. Two things: it's a Mixture-of-Experts (MoE) model, so only a fraction of its parameters are active per token, and it ships in MXFP4 — a 4-bit microscaling format (~4.25 bits per weight) that OpenAI designed for efficiency. Together, those pack gpt-oss-20b down to roughly 12-13GB on disk, fitting comfortably in 16GB of VRAM. If you're short, you can offload to CPU (slower). The big sibling, gpt-oss-120b, is a different story — it wants 60-80GB of memory, so you're looking at a multi-GPU setup or a big-unified-memory machine; see our gpt-oss-120b hardware breakdown for that. And a nice 2026 bonus: llama.cpp and Ollama gained ~35% faster generation with native MXFP4 handling, and Blackwell GPUs (like the RTX 5090) run this 4-bit math directly in hardware.
gpt-oss models and what they need
gpt-oss-20b
Model
~16GB VRAM
VRAM / memory
Desktop GPUs — the accessible one
gpt-oss-120b
Model
60-80GB memory
VRAM / memory
Workstation / multi-GPU
Model
VRAM / memory
For
gpt-oss-20b
~16GB VRAM
Desktop GPUs — the accessible one
gpt-oss-120b
60-80GB memory
Workstation / multi-GPU
gpt-oss-20b fits a 16GB card thanks to its MoE design and OpenAI's MXFP4 4-bit format. · Unsplash
Is it worth running?
Honestly, it depends on what else you'd run. gpt-oss-20b is a strong, reasoning-focused model that's genuinely useful, and there's something appealing about running OpenAI's own open weights privately. But it competes with excellent alternatives at similar sizes — Qwen3, Gemma 3, and Mistral — so it's one good option among several, not an automatic winner; try it against those and keep whichever you prefer for your tasks. Where gpt-oss is especially interesting is agentic and developer workflows, which it was tuned for, and it slots neatly into a local OpenAI-compatible API server since it comes from the company that defined that API. My take: if you have a 16GB card, gpt-oss-20b is well worth adding to your rotation. The 120b is only worth the hardware investment if you specifically need its capability and have the VRAM to spare.
Quick answers
How do I run gpt-oss locally?
Install Ollama (free, from ollama.com), then run 'ollama run gpt-oss:20b' to download and chat with OpenAI's 20B open-weight model. It runs privately and offline on your own hardware. The 20B model needs about 16GB of VRAM; if you're short, you can offload to CPU (slower). The larger gpt-oss-120b requires 60-80GB of memory, so it needs a workstation or multi-GPU setup. gpt-oss also works through a local OpenAI-compatible API, since OpenAI designed it to.
How much VRAM does gpt-oss need?
gpt-oss-20b needs about 16GB of VRAM, which is impressive for a 20B model — it fits because of its Mixture-of-Experts design (only some parameters active per token) and MXFP4 4-bit quantization, packing it to roughly 12-13GB on disk. The larger gpt-oss-120b needs 60-80GB of memory, requiring a multi-GPU rig or a large unified-memory machine. If you're short on VRAM for the 20B, CPU offloading works but is slower. A 16GB graphics card is the practical target for the accessible gpt-oss-20b.
Is gpt-oss better than other local models?
gpt-oss-20b is a strong, reasoning-focused model, and running OpenAI's own open weights locally is appealing, but it's one good option among several rather than a clear winner. At similar sizes, Qwen3, Gemma 3, and Mistral are all excellent, so it's worth testing gpt-oss against them for your specific tasks. gpt-oss stands out for agentic and developer workflows it was tuned for, and it fits neatly into a local OpenAI-compatible API. If you have a 16GB card, it's well worth adding to your model rotation.
gpt-oss brings OpenAI's open weights to your machine — the 20B fits a 16GB card and is worth a spot in your rotation. Compare with Qwen3 and Mistral, serve it via a local API, and for the 120B see the hardware requirements. Source: OpenAI and Ollama.
