run locally

9 articles · newest first

Isometric diorama of a code editor window and a graphics card together on a desk with a small padlock, near-black background, indigo-violet with a coral accent

Qwen2.5-Coder-32B locally: a private Copilot on a 24GB card

Qwen2.5-Coder-32B still runs a serious private coding assistant on a single 24GB card, with a 32K native context (128K with YaRN). The VRAM math, the tight-but-real fit, and how it stands against the newer Qwen3-Coder.

Lena Fischer · 2h ago · 7 min

Realistic painterly illustration of a small compact glowing processor on a dark surface, deep indigo-violet with a warm coral highlight

Phi-4 locally: a small model that's still handy, with one dated catch

Microsoft's Phi-4 is a compact 14B that runs on a mid-range card — but at 16K context and a late-2024 release, it shows its age next to newer 14B-class models. The VRAM math and an honest verdict.

Lena Fischer · 2h ago · 7 min

Editorial infographic of a solid dense 27B model block fitting neatly inside a 24GB graphics card, near-black background, indigo-violet with a coral accent

Qwen3.6 27B locally: the all-dense 24GB Qwen

Qwen3.6's dense 27B is the no-fuss 24GB-card option — every parameter fires, nothing to offload, a real 256K context. The VRAM math, the GPU, and how it differs from the 35B-A3B MoE.

Lena Fischer · 2h ago · 7 min

Isometric diorama of a huge grid of expert blocks overflowing off a single graphics card into several system-RAM sticks, near-black background, indigo-violet with a coral accent

Qwen3-Next 80B-A3B locally: an 80B at home, if you can hold it

Qwen3-Next 80B-A3B activates only 3B parameters per token — so it's fast. But 80B of weights still have to live somewhere. The honest VRAM math, the offload trick, and what it really takes.

Lena Fischer · 2h ago · 7 min

Editorial infographic of a mixture-of-experts model: a few active expert blocks on a GPU and the rest resting in system memory, near-black background, indigo-violet with a coral accent

Gemma 4 26B-A4B locally: the Gemma that runs fast on 16GB

Gemma 4's mixture-of-experts model holds about 26B parameters but activates only 3.8B — so it's quick, and it fits a 16GB card at Q4. The VRAM math, the GPU, and the run command.

Lena Fischer · 2h ago · 7 min

Realistic painterly illustration of a large glowing processor die radiating soft light on a dark surface, deep indigo-violet with a warm coral highlight

Gemma 4 31B locally: the 24GB-card Gemma

Gemma 4 31B is the big dense Gemma — the one that asks for a 24GB card and rewards it. The VRAM per quant, the GPU that fits, and how to run it without pulling the wrong tag.

Lena Fischer · 2h ago · 7 min

Isometric diorama of a grid of many small expert blocks with only a few lit up feeding a token, near-black background, indigo-violet with a coral accent

Qwen3.6 35B-A3B locally: a 35B that thinks like a 3B

Qwen3.6's default is a mixture-of-experts model: 35B on disk, 3B active per token. That makes it fast — but the VRAM math has a twist worth understanding before you buy a card.

Lena Fischer · 2h ago · 7 min

Editorial infographic of a compact AI model badge flowing into a mid-range GPU that comfortably fits it, near-black background, indigo-violet with a coral accent

Qwen3.5 9B locally: the model 'ollama run qwen3.5' actually gives you

Type ollama run qwen3.5 and this is the model you get — the 9B. What it is, how little VRAM it needs, and the cheapest card that runs it well.

Lena Fischer · 2h ago · 7 min

Isometric diorama of a friendly glowing AI chip seated on a mid-range graphics card, near-black background, indigo-violet with a coral accent

Gemma 4 12B locally: the VRAM, the GPU, and one Ollama gotcha

Google's Gemma 4 12B is the multimodal open model that fits a mid-range card — if you pull the right Ollama tag. The honest VRAM math, the cheapest GPU that runs it, and how to start.

Lena Fischer · 2h ago · 7 min