Will a model fit on my GPU? (VRAM)

For local inference, memory goes to three things, and we compute each one separately from the model's own config.json (layers, attention heads, key-value heads, head size, parameter count):

  • Weights= parameters × effective bits per weight for the quantisation you pick. We use the real effective size of each GGUF quant type, not its nominal name: a "4-bit" file is about 4.8 bits per weight once block scales and higher-precision tensors are counted.
  • KV cache = 2 × layers × KV heads × head size × context length × 2 bytes. Using key-value heads (not attention heads) is what makes this right for grouped-query attention models.
  • Overhead: a fixed allowance of about 0.8 GB for the driver context, activations and the runtime.

How we checked it: the bits-per-weight table is validated against 40 real published quantised file sizes (Llama 3.1 8B, Llama 3.3 70B, Qwen2.5 7B, Mistral 7B and Gemma 2 9B, as listed on ollama.com). Median error: 0.7%. Known limits: the 2-bit Q2_K quant is over-estimated on large models (we err towards "won't fit"), and models with mixed attention (some Qwen3.5/3.6 and Gemma 4 layers use linear or sliding-window attention) are modelled by their full-attention layers, with the other layers' small fixed cost noted on the model's page.

Mixture-of-experts models need memory for every expert, so their VRAM uses the full parameter count even though each token only reads the active experts.

How fast will it run? (tokens per second)

Generating text is limited by memory bandwidth, so our speed figure is the card's memory bandwidth divided by the bytes read per token (the active weights), derated to 75%. It is an estimate, not a benchmark. Your runtime, batch size and kernels change the real number. Where we haven't confirmed a card's bandwidth from a second source, we show no speed at all rather than a number we can't stand behind. We don't run our own hardware tests; measured results quoted in articles are attributed to whoever measured them.

What does it cost to rent? (cloud GPU prices)

  • Prices come from the public pricing APIs of Vast.ai and Runpod, captured every hour.
  • We show the lowest listed hourly price per GPU and the time it was captured. Spot and interruptible prices can change or disappear without notice.
  • Only two providers are tracked. Other clouds may be cheaper; we say so rather than imply full coverage.
  • Rental links are referral links, marked on the page. They never change the price you pay or which card we list first: the order is cheapest first.

How articles are researched

  • Each guide lists the sources it was built from under Sources, preferring primary ones: model cards and configs, vendor spec sheets, official pricing pages, changelogs and papers.
  • Articles are published by the Aliteq Team. We use AI tools to help draft, then check figures against those primary sources before publishing, and we show when a page was last updated.
  • Hardware performance claims always name who measured them.
  • Corrections are made in place with an updated date. See the editorial policy.

Use the tools