everyone says AMD means fighting ROCm for local AI. on llama.cpp, the thing you're told to install is the slower one

The short answer

The RX 7900 XTX gives you 24GB of VRAM for well under a used RTX 3090's price and runs small models fast — a llama.cpp scoreboard clocks its Vulkan backend at 191 t/s on Llama 2 7B (Q4_0). The catch…

24GB at a discount: the same capacity as a 3090/4090, usually cheaper — enough for 32B-class models at Q4, the local-AI sweet spot.

Surprise finding: on llama.cpp's Vulkan backend the 7900 XTX hits ~191 t/s on a 7B; an open issue reports ROCm ~20–30% *slower* on the same card. For inference, Vulkan is both easier and faster.

The real limitation is software breadth, not chat speed: LoRA/QLoRA tooling, Stable Diffusion optimizations and serving stacks target CUDA first, sometimes only.

Buy it if you mostly run local LLM inference via llama.cpp/Ollama and want max VRAM-per-dollar. Skip it if fine-tuning or image generation is central — that's still NVIDIA's lane.

It has no dedicated matrix/tensor cores, which is why it trails a 4090 on compute-bound work even when it matches on memory.