Nemotron 3.5 Lightning is a 30-billion-parameter model with only about 3 billion active per token — Nvidia says that's why it runs up to 4x faster than comparable open models on a single consumer GPU.
Nvidia just shipped a 30-billion-parameter AI model that only wakes up about 3 billion of those parameters for any given token — and that's the whole trick behind why it reportedly runs up to four times faster than comparable open models while fitting on a single consumer GPU. Nemotron 3.5 Lightning landed August 11, and it's Nvidia's first open-weight model release since CEO Jensen Huang used his first-ever post on X, three weeks earlier, to argue that the world needs open AI models just as much as closed ones.
3 billion active out of 30 billion — what that means for your GPU
A dense 30B model activates every one of its 30 billion parameters on every single token — that's the compute and memory-bandwidth bill you pay whether you need it or not. Nemotron 3.5 Lightning is a hybrid architecture that interleaves Mamba-2 layers, mixture-of-experts layers, and a handful of attention layers, and it routes each token through only a fraction of the total network — about 3 billion parameters' worth, hence 'A3B.' The rest of the weights sit dormant on that pass. That's why Nvidia also ships an NVFP4-quantized build sized specifically to fit inside consumer-card VRAM budgets rather than requiring a workstation card just to load it.
Nemotron 3.5 Lightning at a glance
Total parameters
Spec
30B
Active parameters per token
Spec
~3B (A3B)
Architecture
Spec
Hybrid Mamba-2 + MoE + attention
Measured throughput
Spec
~293 tokens/sec
Vs. Qwen3.6 35B
Spec
30% faster at comparable accuracy
Target hardware
Spec
Single RTX GPU, DGX Spark/Station, Jetson
Spec
Value
Total parameters
30B
Active parameters per token
~3B (A3B)
Architecture
Hybrid Mamba-2 + MoE + attention
Measured throughput
~293 tokens/sec
Vs. Qwen3.6 35B
30% faster at comparable accuracy
Target hardware
Single RTX GPU, DGX Spark/Station, Jetson
The Jensen Huang about-face, in one paragraph
On July 24, Huang posted on X for the first time ever, backing an open letter signed by roughly 25 companies — Nvidia, Microsoft, Meta, Palantir, Hugging Face, Andreessen Horowitz, Perplexity and IBM among them — arguing against choking off open-weight AI models just as Washington was debating restrictions on Chinese open models like Moonshot's Kimi K3. His own words: 'Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. The world needs both frontier closed models and frontier open models.' Less than three weeks later, Nvidia backed that stance with an actual release.
Only a fraction of Nemotron 3.5 Lightning's 30 billion parameters activate per token — the rest stay dormant on each pass. · Unsplash
How the active-parameter math compares
Parameters actually engaged per token
Nemotron 3.5 Lightning (MoE)~3B active
A dense 30B model30B active
A dense 70B model70B active
That gap is the entire reason a 30B-class model can hit ~293 tokens/sec on a single card instead of crawling. If you're already comparing this against gpt-oss and Qwen3 for local coding work, or working out what VRAM a 20B-class model actually needs, the active-parameter count — not the headline total — is the number to anchor on. It's available now on Hugging Face, OpenRouter, and build.nvidia.com; if you're new to running quantized local models, our quantization primer covers what NVFP4 actually trades away versus full precision.
Who should actually try this
7/ 10
Verdict
Verdict
If you're already running local models for agentic or coding workloads on a single RTX card and 30B-class dense models have felt sluggish, Nemotron 3.5 Lightning is worth downloading this week. It's free, it's genuinely fast in both Nvidia's and third-party benchmarks, and NeMo Switchyard means you don't have to commit to it exclusively — you can route the easy tasks to it and keep a frontier model in reserve for the hard ones.
Best for: Local-AI users with a single modern RTX GPU running agentic or coding workloads
Can Nemotron 3.5 Lightning run on a laptop GPU?
Nvidia specifically built it to run on a single RTX GPU, including RTX laptops, not just DGX-class hardware — keeping active parameters down to about 3B is what makes that possible.
Is it a general chatbot, or something narrower?
Narrower. Nvidia positions it for agentic and multi-agent workloads — tool use, coding, long-context tasks — rather than as a general conversational assistant.
What's NeMo Switchyard?
An open-source routing library Nvidia released alongside the model. It sends each request to whichever model — Nemotron, another open model, or a proprietary frontier model — actually fits the task, which Nvidia says can cut total task cost to roughly a third of running Anthropic's Opus 4.8 alone.
Why is Nvidia releasing open models at all — isn't that Meta and OpenAI's territory?
Because open models still need GPUs to run. Jensen Huang has been explicit that open AI 'enables sovereignty' and accelerates diffusion — and broader diffusion of models people can actually afford to run locally is, not coincidentally, also Nvidia's business.
How does this compare to Meta's Muse Glimmer?
Both are open-weight models built to run outside a data center, but from rival hardware ecosystems — see our AMD vs Nvidia real-world benchmark of Muse Glimmer for the comparison.
This almost certainly isn't Nvidia's last open release now that Huang has staked out the position publicly — expect Nemotron 3.5 Lightning to be the opening move, not the whole strategy. For anyone building a local-AI rig around a single RTX card, it's the first model this month actually worth re-downloading your shortlist for.
