NVIDIA spent three months optimising the DGX Spark. Token generation got slower.

Between November 2025 and February 2026, llama.cpp got three months of CUDA optimisation work. Georgi Gerganov benchmarked a DGX Spark on both builds, same machine, same harness, same models. Prompt processing got up to 27% faster.

Token generation got slower. Not flat — down 3 to 4% on every single model tested.

That is not a regression anyone needs to fix. It is a memory bus demonstrating that it is a memory bus, and it tells you more about whether you should buy this machine than any spec sheet does.

Two numbers, and the one that gets advertised

Generating text with an LLM is two different jobs wearing one name.

Prefill is reading your prompt. It processes many tokens at once — a big parallel matrix operation — so it scales with compute. Decode is writing the reply, one token at a time, and each token requires reading the model's active weights out of memory. That scales with memory bandwidth and essentially nothing else.

NVIDIA's product page leads with 1 PFLOP of FP4 compute. That's a prefill number. The same page lists memory bandwidth at 273 GB/s, and that's the decode number — it's just further down and much less exciting. For context, an RTX 5090 has 1,792 GB/s and an M2 Ultra has 800.

The accidental experiment

Gerganov keeps benchmark results for the DGX Spark in the llama.cpp repository, and the file has been updated twice: build 6989 in November 2025, build 7941 in February 2026. Diffing those two commits gives you something rare — a controlled before-and-after on identical hardware with three months of software work in between.

Three months of kernel work bought a fifth to a quarter more prefill throughput. It bought nothing at all for generation, because generation was never waiting on the kernels. It was waiting on the bus, and the bus doesn't get software updates.

Against a Mac, it depends entirely on the model

The obvious comparison is a Mac Studio, which is the other way to buy a lot of unified memory. The llama.cpp benchmark set covers both, so the harness is comparable. The result flips depending on what you run — and this is the part the coverage generally misses.

On a dense model, it's not close. Qwen2-7B at Q8_0: the M2 Ultra does 79.68 t/s, the Spark does 29.43. That's a 2.71× gap against a bandwidth ratio of 2.93× (800 ÷ 273). Both machines are running at roughly 80–90% of what their memory bus allows, which means the ranking isn't about architecture or drivers or CUDA. It's about which one has the wider pipe.

On MoE models the picture changes, because only a fraction of the weights are read per token. On gpt-oss-120B the Mac's lead collapses from 2.71× to 1.46×.

And on prefill the Spark wins outright, increasingly so as context grows:

That last row is the strongest argument anyone has made for this machine, and I haven't seen it in a single review. At 32k of context on a MoE model, the Spark processes prompts nearly four times faster than the Mac — and wins on generation too (32.65 vs 21.81 t/s). If your work is agentic — long tool-call chains, big retrieved contexts, repeated re-reading of a large prompt — you spend most of your time in prefill, and that is the workload this machine is genuinely built for.

The 200-billion-parameter claim

NVIDIA says the Spark handles inference on models up to 200 billion parameters. That's true in the sense that the weights fit in 128 GB. It is worth knowing what "handles" means in practice.

LMSYS measured Llama-3.1-70B at FP8 in their DGX Spark review: 803 tokens/sec prefill, 2.7 tokens/sec decode. Roughly 70 GB of weights divided by 273 GB/s gives a theoretical ceiling around 3.9 t/s, so 2.7 is about 69% of the maximum the hardware can physically do. Nothing is broken. That is the machine.

You can check this arithmetic for any model on our cost-to-run pages, which separate the weight requirement from the throughput estimate for exactly this reason — fitting and running fast are different questions, and the answer is frequently yes to one and no to the other.

Who should actually buy one

One caveat on price, stated plainly because I can't verify it properly: the Spark was announced around $2,999 at CES 2025 and is widely reported at roughly $4,699 in 2026, attributed to memory costs. I could only find that in aggregator coverage, not on an NVIDIA page, so treat the exact figure as unconfirmed — but a rise of that magnitude clearly matters to a value case that was already narrow. Check current pricing before deciding.