How much VRAM Inkling 2 Mini (MoE) needs, and what it costs to run
Some links on this page are referral links: we may earn a commission, at no cost to you.
Trying to decide what to buy? The best GPU for Inkling 2 Mini (MoE) ↓ · Got a card already? See what your PC can run →
Cheapest way to rent it right now
$0.04/hr
NVIDIA GeForce RTX 3060 12GB · Q2_K · spot · ≈ $9/mo at 8h/day
Live price captured Thu, 19 Nov 2026 17:20:08 GMT.It never changes which card is cheapest.
Memory needed, by quantisation
At 16k context. Weights + KV cache + ~0.8 GB overhead. Memory is in binary GB (1,024³ bytes), the unit GPU memory is sold in; Hugging Face and Ollama list downloads in decimal GB, about 7% more.
| Quant | Weights | KV cache | Total | Quality |
|---|---|---|---|---|
| F16 | 49.2 GB | 0.5 GB | 50.4 GB | Full precision. Reference quality, twice the size of Q8 for no practical gain in most chat use. |
| Q8_0 | 26.1 GB | 0.5 GB | 27.4 GB | Effectively lossless. Use when VRAM is not the constraint. |
| Q6_K | 20.2 GB | 0.5 GB | 21.4 GB | Very close to Q8 at meaningfully less memory. A safe high-quality pick. |
| Q5_K_M | 17.4 GB | 0.5 GB | 18.7 GB | Small, hard-to-notice quality loss. Good balance. |
| Q5_0 | 17.0 GB | 0.5 GB | 18.3 GB | Older-style 5-bit. Q5_K_M is usually the better pick at the same size. |
| Q4_K_M | 14.9 GB | 0.5 GB | 16.2 GB | The community default. Best quality-per-gigabyte for most people. |
| Q4_0 | 14.0 GB | 0.5 GB | 15.3 GB | Older-style 4-bit, measurably worse than Q4_K_M at a similar size. Avoid unless required. |
| IQ4_XS | 13.1 GB | 0.5 GB | 14.3 GB | Newer 4-bit, smaller than Q4_K_M with comparable quality. Needs a recent llama.cpp. |
| Q3_K_M | 12.0 GB | 0.5 GB | 13.3 GB | Noticeable degradation. Use to fit a larger model that would otherwise not run. |
| Q2_K | 10.3 GB | 0.5 GB | 11.6 GB | Heavy degradation. Almost always better to run a smaller model at Q4_K_M instead. |
| IQ3_XS | 10.1 GB | 0.5 GB | 11.4 GB | Aggressive. Usually better than a smaller model at Q4, but test before trusting it. |
Cards that can run it
Best quantisation each card fits at 16k context, with live rental price where we track one.
The best GPU for Inkling 2 Mini (MoE)
Ranked from live rental prices and computed VRAM fit, not opinion. Every card is judged at Q4_K_M with a 8K context, so the comparison is fair; a card that could only run a crushed version isn't listed.
Best value
NVIDIA GeForce RTX 3090
Best throughput per dollar among cards that fit.
Cheapest that runs it
NVIDIA GeForce RTX 3090
Lowest hourly rental that fits it — $0.067/hr, Q4_K_M.
No compromise
—
Most headroom and highest throughput.
Every card that runs it, ranked
Sorted by tokens/sec per rental dollar. Throughput is a bandwidth-derived estimate.
| GPU | VRAM | VRAM used | ~tok/s | Cheapest rental | tok/s per $ |
|---|---|---|---|---|---|
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 96 GB | 16 GB17% | — | $0.467 | — |
| NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition | 96 GB | 16 GB17% | — | $0.333 | — |
| NVIDIA RTX 6000 Ada Generation | 48 GB | 16 GB33% | — | $0.223 | — |
| NVIDIA RTX PRO 5000 Blackwell | 48 GB | 16 GB33% | — | $0.202 | — |
| NVIDIA RTX A6000 | 48 GB | 16 GB33% | — | $0.200 | — |
| AMD Radeon PRO W7900 | 48 GB | 16 GB33% | — | — | — |
| NVIDIA RTX PRO 4500 Blackwell | 32 GB | 16 GB50% | — | $0.200 | — |
| NVIDIA GeForce RTX 5090 | 32 GB | 16 GB50% | — | $0.550 | — |
| NVIDIA RTX 5000 Ada Generation | 32 GB | 16 GB50% | — | $0.265 | — |
| NVIDIA GeForce RTX 3090 Ti | 24 GB | 16 GB66% | — | $0.144 | — |
| AMD Radeon RX 7900 XTX | 24 GB | 16 GB66% | — | — | — |
| NVIDIA RTX PRO 4000 Blackwell | 24 GB | 16 GB66% | — | $0.100 | — |
The value column ranks renting, because that's what we can price precisely (prices captured Thu, 19 Nov 2026 17:20:08 UTC). Buying a card? The fit and throughput columns are what you need; we don't publish a purchase-price ranking because we don't hold verified street prices. Tokens/sec is an estimate, not a benchmark. Got a card already? See everything your PC can run.
inkling-2-miniHow these numbers are produced
- Model shape comes from the model's own config.json — 8 layers, 24 attention heads, 8 KV heads.
- Weight size is parameters × effective bits-per-weight. Those constants are checked against real published quantised file sizes — median error 0.7% across 40 measurements.
- KV cache is 2 × layers × KV-heads × head-dim × context × 2 bytes. Using KV-heads rather than attention heads is what makes this correct for grouped-query attention; treating a GQA model as multi-head overstates the cache by up to 8×.
- Tokens/sec is an estimate, not a benchmark. Generation is memory-bandwidth-bound, so this is bandwidth ÷ weight-bytes derated to 75%. Real throughput depends on your runtime, batch size and kernels. We don't run our own hardware tests — see our editorial policy.
- This is a mixture-of-experts model. Generation reads only the experts active for each token (8 of 128), so throughput is scored on ~3.4B active parameters, not the full 26B. Memory is the opposite: every expert must still be resident in VRAM, so the figures above use the full weight set.Looped hybrid transformer: 24 stored layers, and the middle 12 (layers 6 to 17) run three times with shared weights, so every token goes through 48 layer passes and each pass computes its own keys and values. Attention mixes local and global layers: 20 of the 24 layers use a 1,024-token sliding window, only layers 5, 11, 17 and 23 are global. Global passes = layer 5 once + layers 11 and 17 three times each + layer 23 once = 8, so the KV figures here count 8 full-context caches (8 KV heads x 128); the 40 sliding-window passes add a fixed ~0.16 GB at any context. A plain 24-layer model with the same pattern would hold 4, so the loop doubles the KV cache. Model card: 26.4B total, 3.4B active (5.4B effective with the three passes), 128 experts with 8 active, up to 1M context, text and image input. The official checkpoint is mixed NVFP4 + MXFP8 at about 20.2 GB; GGUF builds go below that (Unsloth UD-Q2_K_XL 9.1 GB, UD-Q4_K_M 16.6 GB), so this page keeps the full quant ladder. Params = the HF safetensors total. HF config.json and model card read Nov 21, 2026.
- Some memory-bandwidth figures are not yet independently verified.Where that's the case we show no tokens/sec at all rather than a number we can't stand behind.
- Rental prices are pulled hourly from provider APIs (last updated Thu, 19 Nov 2026 17:20:08 GMT). Spot/interruptible pricing can change or vanish without notice.
Try it interactively
Drag the context slider and watch the KV cache fill the card
Running Inkling 2 Mini (MoE): common questions
How much VRAM do you need to run Inkling 2 Mini (MoE)?
Can an RTX 4090 run Inkling 2 Mini (MoE)?
Can an RTX 3090 run Inkling 2 Mini (MoE)?
Can an RTX 4060 Ti 16GB run Inkling 2 Mini (MoE)?
What's the cheapest way to run Inkling 2 Mini (MoE)?
Worth reading before you buy
Can you run Inkling 2 Mini locally? Yes, even on a 12 GB GPU, with one long-context catch
Thinking Machines' Inkling 2 Mini is a 26.4B looped MoE that fits a 12 GB card at 2-bit and a 24 GB card at 4-bit. What each GPU needs, the speeds people posted from an 8 GB RTX 4060 to an RTX 5090, and why its memory climbs faster at long context than a 26B model should. Checked Nov 21, 2026.
How much VRAM do you need?
About 0.57 GB per billion parameters at 4-bit, plus room for context: the rule behind the numbers on this page, with worked examples.
The best GPU for local AI right now
The card with enough VRAM for your model at the lowest price, budget by budget.
Rent a cloud GPU or buy one?
Rent, unless you'll pass about 2,000 hours of use: the break-even and the costs people forget on both sides.