Two 3090s or one 5090? The answer isn't the one everyone gives

The advice you'll get in most threads is that two used RTX 3090s beat one RTX 5090 for local LLM work: 48 GB against 32 GB, and roughly double the memory bandwidth on paper. The capacity half of that is true and it matters. The bandwidth half is wrong in the case most people are actually buying for, and it's wrong for a reason that has nothing to do with the cards.

The bandwidth arithmetic everyone starts with

Token generation is memory-bandwidth-bound. Producing one token means reading the whole active weight set out of VRAM once, so throughput tracks bandwidth far more closely than it tracks compute. That's why the back-of-envelope comparison looks so decisive:

Why the second card doesn't make it faster

llama.cpp's default multi-GPU strategy is -sm layer, and its own multi-GPU documentation describes it plainly: each GPU holds a contiguous slice of layers, and the KV cache for a layer lives on the GPU that owns it. That is pipeline parallelism. The docs say it "processes tokens sequentially through the pipeline" and "requires many tokens to scale well".

Read that again with a chat session in mind. You send one prompt. Token one goes through GPU 0's layers, then GPU 1's layers, then comes out. While GPU 0 works, GPU 1 waits. While GPU 1 works, GPU 0 waits. The bandwidth is not summed — it is alternated. Two 3090s in the default configuration give you approximately one 3090's decode latency, with 48 GB of space to put things in.

There is a mode that genuinely aggregates bandwidth. -sm tensor splits individual weight tensors across both cards so they read in parallel, and llama.cpp's docs say it "prioritizes fast token generation". It also comes with a list of conditions that undo much of the reason you bought the second card:

What the measurements actually show

The llama.cpp project maintains a community CUDA scoreboard where contributors submit llama-bench runs on identical models. It's the cleanest apples-to-apples data available, because the model, quantisation and harness are pinned. We don't run our own benchmarks — see our editorial policy — so everything below is somebody else's measurement, named.

Two things fall out of that table, and they point in different directions.

Decode is bandwidth, almost exactly. The 5090's memory bandwidth is 1.915× the 3090's (1,792 ÷ 936). The measured decode ratio is 1.83×, or 95.8% of the prediction. That is a striking result: for a 7B model at 4-bit, throughput is almost entirely explained by how fast the card can read memory. It's also a useful check on the model we use across this site — our per-model VRAM and throughput figures are computed from bandwidth on exactly this assumption, and here it lands within 5% of somebody else's independent measurement.

Prefill is compute, and it isn't close. At 2.89×, the 5090 pulls far ahead of what bandwidth alone would suggest, because processing a prompt is a big parallel matrix operation rather than a serial read. If your work is long prompts — RAG over documents, whole-codebase context, anything where you paste a lot in — that gap matters more than the decode number, and it's the strongest single argument for the 5090.

The case for two 3090s, made properly

Everything above is an argument about speed. Capacity is a different axis, and on that axis the dual build wins something the 5090 cannot offer at any speed: 48 GB versus 32 GB is a categorical difference, not a quantitative one.

A 70B model at Q4_K_M needs roughly 40 GB of weights before you add any KV cache. It fits in 48 GB. It does not fit in 32 GB, at any quantisation worth running. If your actual requirement is "run a 70B locally", the 5090 doesn't lose on points — it's disqualified. You can check the arithmetic for any model on our cost-to-run pages, which compute weights, KV cache and overhead separately so you can see which term is the one squeezing you.

There's a second case, and it's the one the pipeline-parallel finding actually favours. llama.cpp's docs say layer split "maximizes batch throughput". The penalty only bites at batch size 1. If you're serving several concurrent users, or batching a job, the pipeline fills up and both cards work at once. A dual-3090 box is a serving machine, not a low-latency chat machine — and if that's what you're building, the folklore is right for reasons other than the ones usually given.

The things that bite after you've bought

Both routes have a software problem in 2026, and they're mirror images of each other.

What we can't tell you

Price. We could not obtain current used-3090 street prices or real 5090 retail availability from a source we'd be willing to cite, so this article doesn't contain a price comparison — and you should be a little suspicious of any that does without saying where the numbers came from. Used GPU pricing moves weekly, varies enormously by region, and the figures that circulate in forum posts are usually months stale.

This matters because price could invert the entire recommendation. If used 3090s have fallen far enough in your market, the dual build could cost meaningfully less than one 5090 for 1.5× the VRAM, and that would outweigh a good deal of the software friction above. Check completed sales in your own region, then come back to the decision framework rather than the other way round.

So which one

If you want to check any of this against a specific model rather than the 7B used in the benchmarks above, our cost-to-run pages compute the memory requirement and expected throughput per card from each model's own configuration file, and show the working.