Run OpenClaw or Hermes on a local model: which fits your GPU

Ollama's pages for OpenClaw and Hermes Agent each recommend a few local models with a rough VRAM figure. What they don't spell out is which size of each model a bare name like gemma4 downloads, and that turns out to matter a lot. Here's what each default tag actually is, what the bigger versions need at the long context agents use, and what to pick for your card. If you haven't chosen between the two agents yet, start with OpenClaw vs Hermes.

What the default model names actually download

From the qwen3.5, gemma4 and qwen3.6 tag pages. The surprise is gemma4: the default is the small E4B model, not the 26B. If you want the bigger Gemma, you have to ask for gemma4:26b (19GB) by name. Likewise qwen3.5 gives you the 9B (sized on our engine at best GPU for Qwen3.5 9B) unless you type qwen3.5:35b or qwen3.5:27b. You can always check what you've got with ollama show <model>.

Set the context first

Ollama's context-length page says tasks like agents and coding tools "should be set to at least 64000 tokens," but its defaults are 4K on cards under 24 GiB and 32K on 24–48 GiB. An agent that forgets the start of the conversation is usually a context problem, not a model problem. Raise it before you judge any model:

# Ollama docs (context-length), accessed 25 Sep 2026
OLLAMA_CONTEXT_LENGTH=64000 ollama serve

# then launch with a specific tag (flag per Ollama's OpenClaw page)
ollama launch openclaw --model gemma4:26b
ollama launch hermes            # pick the model in the selector; change later with: hermes model

What fits your card at 64K context

Why the hybrid models do so well at long context: per their Hugging Face configs, Qwen3.5/3.6 use full attention on only 10 of 40 layers (the 35B-A3B), and Gemma 4 26B-A4B on only 5 of 30. The rest use linear or short sliding-window attention with a small fixed memory cost. So their memory barely grows as context gets longer. Compare Qwen3-Coder 30B, a regular-attention model, at ≈24.0 GiB for the same 64K. Per-card tables: Qwen3.6 35B-A3B, Gemma 4 26B-A4B, Gemma 4 12B, Qwen3.6 27B.

Referral link

Card too small for the model you want? Rent a 24GB one

On Vast.ai's spot market, 24GB cards like the RTX 3090 had listings from about $0.12/hr (lowest live listing, 25 Sep 2026). That fits Gemma 4 26B comfortably and Qwen3.6 35B at 64K tightly. Run Ollama on the rented machine and point the agent at it. Spot machines can be reclaimed, so keep the agent's state somewhere safe.

See live 24GB GPU pricesCompare live prices first →

Referral link — we may earn a commission at no cost to you. Prices on our compare page are the provider's live figures, cheapest first; this never changes the ranking.

My pick, by card

• 12GB: Gemma 4 12B (gemma4:12b), about 8.8 GiB at 64K. It leaves room, and it's a real step up from the tiny default.
• 16GB: gpt-oss 20B (gpt-oss:20b) at a snug 15.6 GiB, or drop context to 32K for headroom. The default gemma4 (E4B) also works here.
• 24GB: Gemma 4 26B-A4B (gemma4:26b) is the comfortable choice at ≈17.2 GiB. Qwen3.6 35B-A3B, Hermes's recommendation, fits but tightly (≈22.6 GiB).
• 32GB+: anything above, including Gemma 4 31B.
• Whatever you pick: the model must support tool calling. Ollama's agent pages point to its tools-capable models list. Then read the security page before you connect a chat app.