llama.cpp

4 articles · newest first

A multi-GPU workstation built for running large language models locally

The cheapest way to run Llama 70B isn't the one you're about to build

"Cheapest" flips completely on how often you run it and how many people hit it at once — and in Europe, the electricity bill alone can cost most of what renting does.

Lena Fischer · 3m ago · 10 min

Close-up of computer hardware used for local AI inference

The MoE offload trick that makes your model 5× faster — and why it probably won't work for you

Adding CPU offload should make inference slower. Sometimes it makes it five times faster. Both are true, and the reason matters more than the trick.

Lena Fischer · 1h ago · 9 min

Two graphics cards installed in a desktop PC

Hardware

Two 3090s or one 5090? The answer isn't the one everyone gives

The dual-3090 build doesn't give you double the bandwidth for chat — and the reason has nothing to do with the cards. What the measurements actually say.

Ravi Malhotra · 2h ago · 11 min

Two graphics cards side by side, illustrating GPU VRAM tier comparison

everyone asks which local AI model is 'best.' wrong question — your VRAM already picked it

Llama 4 Scout, Qwen3.6-27B, GPT-OSS-20B, Phi-4-14B — four totally different 'best' models, and which one is yours comes down to a single number: how much VRAM your GPU actually has.

Lena Fischer · 3d ago · 6 min