Local AI software
Ollama, answered: what runs on your GPU, how to manage models, and what to compare it with
Ollama is a free, open-source app that downloads open AI models and runs them on your own computer, on Windows, macOS and Linux. How far it goes depends on one number: your graphics card's memory (VRAM). The table shows the biggest model each common VRAM size runs at good quality, worked out by our VRAM engine from each model's published architecture, with the cheapest card to rent if yours is too small. Below it is every guide we have on Ollama, from managing models to using it with coding agents.
Official site: ollama.com · checked 2026-11-07
What runs on your GPU
The biggest model we track that fits each VRAM size at Q4_K_M quality or better with an 8K context, from our VRAM engine, and the cheapest card with at least that much memory to rent right now.
| VRAM | Biggest model that fits | Cheapest card to rent | Rent |
|---|---|---|---|
| 8 GB | Gemma 4 12B | RTX 3060 from $0.033/hr | Vast.ai |
| 12 GB | DeepSeek R1 Distill 14B | RTX 3060 from $0.033/hr | Vast.ai |
| 16 GB | Gemma 4 26B-A4B (MoE) | L4 from $0.045/hr | Vast.ai |
| 24 GB | Qwen3.6 35B-A3B (MoE) | L4 from $0.045/hr | Vast.ai |
| 32 GB | Mixtral 8x7B (MoE) | RTX A6000 from $0.107/hr | Vast.ai |
| 48 GB | Qwen3-Next 80B-A3B (MoE) | RTX A6000 from $0.107/hr | Vast.ai |
| 96 GB | Qwen3.5 122B-A10B (MoE) | RTX PRO 6000 Blackwell Max-Q Workstation Edition from $0.443/hr | Vast.ai |
Model fits are our VRAM engine's estimates, not benchmarks. Prices are the lowest in our hourly snapshot (1h ago); rent buttons are referral links. Check a model and context of your own in the VRAM calculator.
Set it up and manage models
- How do I delete, list, update and move Ollama models?Local models pile up fast and each one is gigabytes. Here are the exact Ollama commands to remove one model or all of them, see what you have, stop what's running, update, and move the whole store to another drive.
- How do I run Qwen (Qwen3.8 27B, Flash Next) in Ollama?Qwen3.8 27B is the default pick for a 16 to 24GB card, and Flash Next runs on smaller cards with lots of RAM. One command gets you started. Three default settings decide whether it is any good.
- Which model does 'ollama run qwen3.5' actually give me?Type ollama run qwen3.5 and this is the model you get — the 9B. What it is, how little VRAM it needs, and the cheapest card that runs it well.
Use it with coding agents
- Claude Code, Codex or OpenCode on a local model?All three coding agents can run on your own GPU through Ollama. The catch is 64k of context, which Ollama doesn't give you by default. Which agent to pick, and what your card can actually hold.
- How do I use Ollama with Claude Code?One command, or three environment variables. Plus the 64k context setting Ollama's defaults don't give you, the VRAM that costs, and what Ollama's Anthropic API doesn't support.
- How do I use Ollama with Codex CLI?ollama launch codex, the --oss flag, or a permanent profile, plus the 64k context Codex needs and what gpt-oss 20B vs 120B costs in VRAM at that length.
- How do I use Ollama with OpenCode?ollama launch opencode or one opencode.json block, plus the 64k context OpenCode requires, why tool calls fail without it, and what fits on your GPU at that length.
- Which local model runs OpenClaw or Hermes on my GPU?Ollama's default gemma4 is the small E4B and qwen3.5 is the 9B, not what most people assume. What each tag really downloads, what the bigger models need at the 64K context agents use, and what to pick for 12, 16 and 24GB cards.
Ollama vs the others
- Ollama or LM Studio?Both are free and run the same open models. Pick LM Studio to browse and tune models in an app, Ollama for a model your other tools call. Checked Nov 1, 2026.
- Ollama, llama.cpp, LM Studio or vLLM?One person on one machine, or a team hitting one server? That decides it. Then licence, hardware, ports, security defaults, and where Strata fits. Read Nov 1, 2026.
- Open WebUI or LM Studio?LM Studio is a desktop app that finds, downloads and runs models for you. Open WebUI is a web platform you host for yourself, your family or a team. Here's when each one wins, what changed this week, and the five-minute setup that uses both.
- On a Mac: MLX or llama.cpp?Four ways to run a local model on Apple silicon: mlx-lm, llama.cpp, Ollama and LM Studio. I read each project's docs to sort out who each is for, which model files they take, and the one sysctl the MLX README tells you to try when a model runs slow.
When a model doesn't fit
- Which models fit my GPU's VRAM?Your VRAM picks the model before you do. What fits on 8, 12, 16, 24 and 32GB cards, worked out on our VRAM engine and checked against each model card and Ollama's download sizes on Nov 6, 2026, plus why NVIDIA's "24GB" Nemotron wants a 32GB card in llama.cpp.
- How much VRAM does a model need?The rule by model size, the card sizes it maps to, and the real Ollama download sizes we checked it against on Nov 3, 2026. Why context costs some models far more than others, why MoE models still need room for every parameter, and what to do when your card is too small.
- Out of memory at long context: shrink the KV cacheThe model fits. Then you paste in a long document and the GPU runs out of memory. The culprit is the KV cache, it has a formula you can do on a napkin, and two settings cut it by roughly half or three quarters.
- What does Ollama do with MoE experts that don't fit?The GitHub request to keep MoE experts in RAM is still open after 15 months. But since Ollama 0.30, llama.cpp's own fit step can park them there for you, as long as you leave num_gpu alone. Plus what Ollama's Qwen3.8 Flash Next tags need, and why Reddit can tell when you're running Ollama.
- LM Studio's 'MoE weights onto CPU' sliderIt started as a toggle and became a layer slider in LM Studio 0.4.0. Where it lives, what it moves, the number to set on an 8, 12 or 16GB card, whether LM Studio runs Qwen3.8 Flash Next, and why Mac MLX models don't have the setting.
- Rent a cloud GPU or buy one?Divide the card's price by the hourly rental rate. On Nov 3, 2026 a $1,000 RTX 3090 paid for itself after about 2,000 hours against Runpod ($0.50 an hour) and about 6,200 hours against a typical Vast.ai rental ($0.16). Add electricity and the gap widens. Here's the math, by GPU and by hours a day.
Get the GPU