mistral small 24b

1 article · newest first

Flat illustration of a person running a local AI chat assistant on a laptop with a single graphics card, on a teal background

run Mistral Small 24B locally in 2026: VRAM, which GPU, and how fast

Mistral Small 24B fits a single 24GB card at Q4 (~14GB), running ~85 tok/s on a 4090. VRAM by quant, which GPU, and the cheapest way to run it — own or rent.

Lena Fischer · 2h ago · 7 min