You don't need a discrete graphics card to get GPU-accelerated local AI. A modern AMD Radeon iGPU shares your system RAM as VRAM and runs 7B models faster than a CPU — here's what to expect.
Yes — and modern integrated GPUs are far better at it than most people expect. You don't need a discrete graphics card: a recent AMD Radeon iGPU shares your system RAM as VRAM and gives you real GPU acceleration. The Radeon 780M runs 7B models at 12-20 tokens per second (and 3B models at a snappy 30-50 tok/s), while the newer Radeon 890M hits 18-25 tok/s on 7B and — with enough RAM — can even handle 30B+ models at usable speeds. The trick is that these iGPUs use your DDR5 system memory as VRAM, so a mini PC or laptop with 32-64GB of RAM effectively hands the GPU a huge memory pool. Ollama auto-offloads to the iGPU, so it mostly just works. Here's what's realistic.
Why the iGPU trick works — it's all bandwidth
Here's the key insight: LLM inference is almost entirely memory-bandwidth-bound. The bottleneck isn't raw compute — it's how fast the chip can read the model's weights from memory each token. That's why a modern integrated GPU punches above its weight: AMD's Radeon 780M and 890M share the same DDR5 system RAM with the CPU, and because they're a GPU, they read it with GPU parallelism. Ollama (and llama.cpp with Vulkan) automatically offloads model layers to the iGPU, so you get real acceleration without a graphics card — and since the iGPU can address your whole pool of RAM, a box with 64GB of DDR5 gives it far more usable 'VRAM' than a $1,000 discrete card. The flip side of being bandwidth-bound: faster RAM directly means more tokens/second — DDR5-8000 roughly doubles the throughput of slower memory, which is why the best iGPU AI boxes pair the chip with the fastest RAM they can. So the mini PCs built around these chips aren't a gimmick; they're exploiting exactly the thing that makes local AI fast.
Integrated GPU local-AI speed (real numbers)
Radeon 780M
iGPU
~12-20 tok/s
7B model (Q4)
3B at ~30-50 tok/s
Radeon 890M
iGPU
~18-25 tok/s
7B model (Q4)
30B+ with enough RAM
Older/Intel iGPU
iGPU
Slower — CPU-class
7B model (Q4)
Small models only
iGPU
7B model (Q4)
Also runs
Radeon 780M
~12-20 tok/s
3B at ~30-50 tok/s
Radeon 890M
~18-25 tok/s
30B+ with enough RAM
Older/Intel iGPU
Slower — CPU-class
Small models only
Modern AMD iGPUs share DDR5 system RAM as VRAM — so a 64GB box hands the GPU a huge memory pool. · Unsplash
Should you rely on an iGPU for AI?
It depends what you want. An integrated GPU is a genuinely good choice if you want GPU-accelerated local AI without buying a graphics card — a modern Ryzen mini PC or laptop with a Radeon 780M/890M and 32-64GB of fast DDR5 runs 7-14B models at usable speeds, sips power, and stays quiet. For a private assistant, a home RAG box, or a coder who mostly runs mid-size models, it's a sweet spot between CPU-only (slower) and a discrete GPU (faster but pricier and power-hungry). Stick with a discrete GPU if you want the fastest possible responses, run big models constantly, or need long-context throughput — a dedicated card with its own fast GDDR memory will still beat an iGPU, which is limited by shared DDR5 bandwidth. And if you want the extreme version of the iGPU idea — a huge unified-memory pool — that's exactly what the Strix Halo mini PCs deliver. The bottom line: don't dismiss integrated graphics for local AI anymore. On modern AMD chips, with enough fast RAM, an iGPU is a legitimately capable, efficient way to run models locally.
Quick answers
Can you run a local LLM on integrated graphics?
Yes, and modern AMD integrated GPUs do it well. A Radeon 780M runs 7B models at roughly 12-20 tokens per second and 3B models at 30-50 tok/s, while the newer Radeon 890M reaches 18-25 tok/s on 7B and can handle 30B+ models with enough RAM. The reason it works is that these iGPUs share your DDR5 system memory as VRAM, so a machine with 32-64GB of RAM gives the GPU a large memory pool. Ollama automatically offloads model layers to the iGPU. It's noticeably faster than CPU-only inference and much cheaper than a discrete graphics card, making it a strong middle option.
What is the best integrated GPU for local AI?
The AMD Radeon 890M is currently the most capable integrated GPU for local AI. Found in chips like the Ryzen AI 9 HX 370, it runs 7B models at about 18-25 tokens per second and, paired with enough system RAM, can run 30B+ models at usable speeds. The Radeon 780M is a strong step down but still good, handling 7B models at 12-20 tok/s and 3B models at 30-50 tok/s. Both benefit hugely from fast DDR5 memory, because LLM inference is bandwidth-bound — faster RAM directly increases tokens per second. Intel and older integrated GPUs are much slower and suited only to small models.
Do you need a graphics card to run AI, or is an iGPU enough?
For many people, a modern integrated GPU is enough. An AMD Radeon 780M or 890M with 32-64GB of fast DDR5 runs 7-14B models at usable speeds, uses little power, and stays quiet — ideal for a private assistant, a home RAG box, or mid-size coding models. You'll still want a discrete GPU if you need the fastest responses, run large models constantly, or work with long context, since a dedicated card's fast GDDR memory outpaces an iGPU limited by shared DDR5 bandwidth. But the old assumption that integrated graphics can't do AI is outdated: on modern AMD chips with enough fast RAM, an iGPU is genuinely capable.
