moe

2 articles · newest first

NVIDIA spent three months optimising the DGX Spark. Token generation got slower.

Same machine, same benchmarks, three months apart. Prefill improved 27%. Generation regressed on every model — and that tells you exactly who should buy one.

Lena Fischer · 1h ago · 10 min

Close-up of computer hardware used for local AI inference

The MoE offload trick that makes your model 5× faster — and why it probably won't work for you

Adding CPU offload should make inference slower. Sometimes it makes it five times faster. Both are true, and the reason matters more than the trick.

Lena Fischer · 2h ago · 9 min