The cheapest way to run Llama 70B isn't the one you're about to build

The short answer

For one person at 16k context, two used RTX 3090s running llama.cpp is the well-trodden cheapest path. For concurrent serving, a single 80 GB card running vLLM is a different product, not a faster…

A 70B at Q4 needs ~40 GB of weights before the KV cache, so any real setup is a two-card build or one datacenter card.

In Denmark, electricity for a 2×3090 rig running 24/7 is roughly €205/month — about two-thirds of what it costs to just rent the same two cards.

Three config defaults silently wreck a 16k deployment: llama.cpp's context budget is shared across users, Ollama defaults to 4k, and multi-GPU without NVLink wants pipeline, not tensor, parallelism.

Interruptible/spot rental is fine for batch jobs and disqualifying for a served endpoint — a higher bidder kills your process with no warning.

Two answers, because it's two different machines

Single user, occasional use: two used 3090s on llama.cpp, or just rent by the hour — at low duty cycle, renting almost always wins and you skip the maintenance entirely. A real served endpoint with…