lora

2 articles · newest first

Can you fine-tune an LLM on your own GPU, and what does it cost?

Yes. With QLoRA, an 8B model needs as little as 6 GB of VRAM and a 14B model 8.5 GB (Unsloth's minimums, read Nov 3, 2026), so a 12GB RTX 3060 or a 24GB RTX 4090 can do it. The software is free; the cost is electricity, about 6 to 13 cents an hour for a whole PC at the US average rate. Here is what fits on which card, and when renting is cheaper.

Aliteq Team · Updated Nov 3, 2026 · 8 min

Thumbnail: Fine-tune an LLM, rent the smallest card that fits. QLoRA needs 6 GB for 8B and 41 GB for 70B. A card reads RTX 4090, Vast median, $0.47/hr, read Nov 3, 2026, with the Vast.ai and Runpod wordmarks.

Which cloud GPU should you rent to fine-tune an LLM, and what does a run cost?

Pick the card by the VRAM your fine-tune needs, not by the biggest name. On Nov 3, 2026 a 24GB RTX 4090 rented for a $0.47/hr median on Vast.ai ($0.89 on Runpod), enough for QLoRA up to about 14B; a 70B QLoRA needs an 80GB A100 from about $0.95/hr. Here is the card for each job and the math for a run.

Aliteq Team · Updated Nov 3, 2026 · 9 min