aliteq.

Type to search every guide, tool and GPU page.

How much VRAM Inkling 2 Small (MoE) needs, and what it costs to run

Some links on this page are referral links: we may earn a commission, at no cost to you.

Trying to decide what to buy? The best GPU for Inkling 2 Small (MoE) ↓ · Got a card already? See what your PC can run →

594.1B paramsMoE · 384 experts, 8 active1024k max contextapache-2.0

Memory needed, by quantisation

At 16k context. Weights + KV cache + ~0.8 GB overhead. Inkling 2 Small (MoE) ships in NVFP4, so its own release leads and no bigger copy is listed: it would only store the same numbers in more bytes. Memory is in binary GB (1,024³ bytes), the unit GPU memory is sold in; Hugging Face and Ollama list downloads in decimal GB, about 7% more.

QuantWeightsKV cacheTotalQuality
NVFP4official366.5 GB0.5 GB367.7 GBThe model's own release (NVFP4, a 393.5 GB download): the best quality there is. A bigger copy only stores the same numbers in more bytes. Its 4-bit GGUF builds keep the 4-bit experts as they are, so they come out about this size too.

Cards that can run it

Best quantisation each card fits at 16k context, with live rental price where we track one.

GPUVRAMBest fitUses~tok/sRent from
Apple M3 Ultra512 GBNVFP4368 GB——
Apple M5 Ultra512 GBNVFP4368 GB——

The best GPU for Inkling 2 Small (MoE)

Ranked from live rental prices and computed VRAM fit, not opinion. Every card is judged at its own NVFP4 release with a 8K context, so the comparison is fair; a card that could only run a crushed version isn't listed.

Best value

—

Best throughput per dollar among cards that fit.

Cheapest that runs it

—

Lowest-cost card that still fits the model.

No compromise

—

Most headroom and highest throughput.

The value column ranks renting, because that's what we can price precisely (prices captured Thu, 19 Nov 2026 17:20:08 UTC). Buying a card? The fit and throughput columns are what you need; we don't publish a purchase-price ranking because we don't hold verified street prices. Tokens/sec is an estimate, not a benchmark. Got a card already? See everything your PC can run.

inkling-2-small

How these numbers are produced

  • Model shape comes from the model's own config.json — 8 layers, 48 attention heads, 8 KV heads.
  • Weight size is parameters × effective bits-per-weight. Those constants are checked against real published quantised file sizes — median error 0.7% across 40 measurements.
  • KV cache is 2 × layers × KV-heads × head-dim × context × 2 bytes. Using KV-heads rather than attention heads is what makes this correct for grouped-query attention; treating a GQA model as multi-head overstates the cache by up to 8×.
  • Tokens/sec is an estimate, not a benchmark. Generation is memory-bandwidth-bound, so this is bandwidth ÷ weight-bytes derated to 75%. Real throughput depends on your runtime, batch size and kernels. We don't run our own hardware tests — see our editorial policy.
  • This is a mixture-of-experts model. Generation reads only the experts active for each token (8 of 384), so throughput is scored on ~17.0B active parameters, not the full 594B. Memory is the opposite: every expert must still be resident in VRAM, so the figures above use the full weight set.Hybrid local/global attention, 48 layers (no loop): 40 are 1,024-token sliding-window (config local_layer_ids), 8 are global (layers 5, 11, ..., 47), so `layers` = the 8 global KV layers (8 KV heads x 128); the sliding-window layers add a fixed ~0.16 GB. Card: 594B total, 17B active, 384 experts top-8, up to 1M context, text + image in. Pretrained, midtrained and post-trained in NVFP4 (card: the native precision); the download is the NVFP4 checkpoint, 24 shards + the MTP file (393.5 GB). An MXFP8 copy exists (Inkling-2-Small-MXFP8, card says 800 GB+ of VRAM). params = HF safetensors total. HF repo created Nov 4, 2026 (card states no date). HF config.json + card read Nov 21, 2026.
  • Some memory-bandwidth figures are not yet independently verified.Where that's the case we show no tokens/sec at all rather than a number we can't stand behind.
  • Rental prices are pulled hourly from provider APIs (last updated Thu, 19 Nov 2026 17:20:08 GMT). Spot/interruptible pricing can change or vanish without notice.

Try it interactively

Drag the context slider and watch the KV cache fill the card

Running Inkling 2 Small (MoE): common questions

How much VRAM do you need to run Inkling 2 Small (MoE)?
In its own NVFP4 release and at 8k context, Inkling 2 Small (MoE) needs about 367.5 GB — 366.5 GB of weights plus 0.3 GB of KV cache and ~0.8 GB overhead. So you want a card with at least that much free memory; the KV cache grows if you use longer context.
Can an RTX 4090 run Inkling 2 Small (MoE)?
Not in its own NVFP4 release — Inkling 2 Small (MoE) needs more than the card's 24 GB at that quality and 8k context. Its NVFP4 release is already the small build (its 4-bit GGUFs are about the same size), so you'd need a bigger card, two cards, or to spill part of it into system RAM, which is much slower.
Can an RTX 3090 run Inkling 2 Small (MoE)?
Not in its own NVFP4 release — Inkling 2 Small (MoE) needs more than the card's 24 GB at that quality and 8k context. Its NVFP4 release is already the small build (its 4-bit GGUFs are about the same size), so you'd need a bigger card, two cards, or to spill part of it into system RAM, which is much slower.
Can an RTX 4060 Ti 16GB run Inkling 2 Small (MoE)?
Not in its own NVFP4 release — Inkling 2 Small (MoE) needs more than the card's 16 GB at that quality and 8k context. Its NVFP4 release is already the small build (its 4-bit GGUFs are about the same size), so you'd need a bigger card, two cards, or to spill part of it into system RAM, which is much slower.

Worth reading before you buy

Other models