Tencent's Hy4 Flash is a 220B mixture-of-experts model with about 12B active per token. What it needs, the speeds owners measured, why llama.cpp isn't ready yet, and what renting a GPU for it costs. Checked Nov 22, 2026.
Yes, you can run Hy4 Flash locally, but it lives in system RAM, not on your GPU. Tencent's Hy4 Flash is a 220B mixture-of-experts model that uses about 12B parameters per token. The easiest way in is Strata, a free open-source engine whose smallest Hy4 pack runs on 64 GB of RAM plus an 8 GB GPU and takes 130 GB of disk. A 4-bit GGUF is 122 to 168 GB, but llama.cpp support is still an unmerged pull request. One owner measured about 40 tokens a second on an RTX 5090 with 128 GB of RAM. Checked November 22, 2026.
What is Hy4 Flash?
Hy4 Flash is an open-weights model from Tencent, published on Hugging Face around November 17, 2026 under the Tencent Hy License, with a separate FP8 version and a base model. The model card describes a 155.5B-parameter backbone, 63.5B parameters of "Engram" lookup tables, a small vision encoder and a built-in 0.85B drafter for speculative decoding, about 220.3B in total. Each token runs about 11.3B text parameters, which Tencent rounds to 12B: 16 of 512 experts plus one shared expert. The context window is 1,048,576 tokens.
The attention design is why its memory cache stays small. Only 12 of its 42 layers keep the full context, and those pick the 2,048 most relevant tokens; the other 30 look at a 512-token window. Tencent says the cache is 1/24 the size of Hy3's. Our Hy4 Flash cost-to-run page works it out at about 0.2 GB for a typical context, so almost all of the memory goes on the weights.
How much RAM and VRAM does Hy4 Flash need?
130 GB of disk and 64 GB of RAM plus an 8 GB GPU at the smallest, and 120 to 170 GB of memory to hold a 4-bit file. The official BF16 release is 440.5 GB and the FP8 one 287 GB; Tencent's own vLLM and SGLang examples run the FP8 checkpoint across 8 GPUs, which is a data-center setup. People at home use one of two routes.
Strata packs split the model: the experts that run most sit on the GPU and the rest stay in system RAM. Every pack also needs the 67.6 GB Engram file.
Hy4 Flash Strata packs (strata-engine, read Nov 22, 2026)
SQ-64
System RAM
64 GB
GPU
8 to 16 GB
Download
129.6 GB
Drift from full model (KL, lower is better)
0.232
SQ-96
System RAM
96 GB
GPU
8 or 12 GB
Download
157.2 GB
Drift from full model (KL, lower is better)
0.136
SQ-128
System RAM
128 GB or more
GPU
8 or 12 GB
Download
181.2 GB
Drift from full model (KL, lower is better)
0.088
System RAM
GPU
Download
Drift from full model (KL, lower is better)
SQ-64
64 GB
8 to 16 GB
129.6 GB
0.232
SQ-96
96 GB
8 or 12 GB
157.2 GB
0.136
SQ-128
128 GB or more
8 or 12 GB
181.2 GB
0.088
GGUF files are what llama.cpp-based tools load. Sizes from the two biggest repos:
Hy4 Flash GGUF size by quant (GB)
IQ2_XXS (2-bit)83.1
Xylolabs
IQ3_XXS (3-bit)100.5
Xylolabs
IQ4_XS (4-bit)122.6
Xylolabs
Q3_K (3-bit)155.4
AngelSlim (Tencent)
Q4_K_M (4-bit)161.2
Xylolabs
Q4_K_M (4-bit)168.4
AngelSlim (Tencent)
Q8_0 (8-bit)228.9
Xylolabs
A rule of thumb from these sizes: you need the file size in RAM plus VRAM, plus a few GB for the system. So 128 GB of total memory runs the 2-bit and 3-bit files, and a 4-bit file wants 192 GB, such as two 96 GB cards or a 128 GB Mac with a 3-bit file. For what quantization does to quality, Strata's KL numbers above are a fair guide: the 64 GB pack drifts almost three times as far from the full model as the 128 GB one.
How fast is Hy4 Flash on my PC?
These are runs owners posted in the first days after release. They are their numbers, not ours, on different engines and settings.
Hy4 Flash speeds people measured (Nov 2026, decode tok/s)
RTX 5090 + 128 GB RAM
Engine and file
Strata SQ-64
Speed
about 40 (mixed benchmark)
Who
@jun1228909
RTX 5060 Ti 16 GB + 80 GB RAM
Engine and file
Strata SQ-64, WSL2
Speed
about half of Qwen 3.8 Flash Next
Who
@yutacar
RTX 5060 Ti 16 GB + 64 GB RAM
Engine and file
Strata SQ-64
Speed
11 (short prompt)
Who
@kero_fire
M5 Max, 128 GB
Engine and file
GGUF Q4_K_M / Q3_K
Speed
48 at 4K, 44 at 32K / 50
Who
AngelSlim (Tencent)
Ryzen AI Max+ 395, 124 GB
Engine and file
GGUF Q3_K, kyojin engine
Speed
27 to 28 (738 prefill)
Who
@zaxbysauce
Engine and file
Speed
Who
RTX 5090 + 128 GB RAM
Strata SQ-64
about 40 (mixed benchmark)
@jun1228909
RTX 5060 Ti 16 GB + 80 GB RAM
Strata SQ-64, WSL2
about half of Qwen 3.8 Flash Next
@yutacar
RTX 5060 Ti 16 GB + 64 GB RAM
Strata SQ-64
11 (short prompt)
@kero_fire
M5 Max, 128 GB
GGUF Q4_K_M / Q3_K
48 at 4K, 44 at 32K / 50
AngelSlim (Tencent)
Ryzen AI Max+ 395, 124 GB
GGUF Q3_K, kyojin engine
27 to 28 (738 prefill)
@zaxbysauce
RTX 5090 with 128 GB of RAM:@jun1228909 tried all three Strata packs. SQ-64 ran at a median of about 40 tokens a second on his own mixed benchmark of Japanese and English explanation, JSON, code and copying input back. Strata's model page claims 90 to 111 on an RTX 5090, and he notes his conditions differ; Strata's developer replied that Hy4 gets more optimization in the next update. All three packs wrote a working single-file HTML app on all three of his coding tries (the first SQ-128 try was slightly broken), which he says is rarer with Qwen models.
16 GB RTX 5060 Ti with 80 GB of RAM:@yutacar ran SQ-64 under WSL2. On 5 coding tasks it matched Qwen 3.8 Flash Next for quality but took about twice as long, 97 minutes in total against 50.
16 GB RTX 5060 Ti with only 64 GB of RAM:@kero_fire got 11 tokens a second from SQ-64 on a short prompt, against 35 for Inkling 2 Mini on the same machine. So the 64 GB pack does run on a 64 GB machine, just slowly.
If you get 1 to 2 tokens a second: another owner reported 1.4 on SQ-64 (his RAM and Strata version weren't stated). Check that the model isn't spilling from RAM to disk (close other apps, watch memory use) and update Strata, since new versions keep adding speedups.
M5 Max MacBook Pro with 128 GB: Tencent's own AngelSlim GGUF card lists 48.1 tokens a second at 4-bit with a 4K context, 45.2 at 16K and 44.2 at 32K, and about 50 with the 3-bit Q3_K file.
Strix Halo (Ryzen AI Max+ 395):@zaxbysauce ran the 3-bit Q3_K file on the kyojin engine at 26.8 to 28.3 tokens a second decode and 738 prefill. He also fed it a 996,000-token prompt: it found the needle, after 1,475 seconds of reading, and then generated at 0.12 tokens a second. The 1M context works, but it isn't practical at home.
Strata's developer lists faster numbers with its built-in drafter on: up to 240 tokens a second on an RTX PRO 6000, 69 to 110 on an M5 Max, 62 on an AMD R9700, 44 on an RTX 5080 16 GB with 64 GB of RAM and 43.9 on a DGX Spark. Treat these as the developer's best case until owners match them.
How fast is Hy4 Flash on a rented GPU, and what does it cost?
A 96 GB RTX PRO 6000 is the sweet spot: from $0.64, typical $1.47 an hour on Vast, and Strata's developer reports up to 240 tokens a second on one. It holds the 2-bit 83 GB file entirely on the GPU, or runs Strata with the host's RAM. To put a 4-bit file fully on one card you need an H200 (141 GB) with the 122.6 GB IQ4_XS file.
Renting a GPU for Hy4 Flash (prices Nov 22, 2026, on demand)
RTX 5090 32 GB + 128 GB host RAM
Card
Strata SQ-128 or SQ-96
What fits
Vast from $0.27, typical $0.67; Runpod $1.39
RTX PRO 6000 96 GB
Card
2-bit GGUF on the GPU, or Strata
What fits
Vast from $0.64, typical $1.47 (Max-Q); Runpod $1.64 (MaxQ), $2.69
H200 141 GB
Card
4-bit IQ4_XS GGUF fully on the GPU
What fits
Vast from $3.00, typical $4.00 (NVL); Runpod $3.79 (NVL)
Card
What fits
Price per hour
RTX 5090 32 GB + 128 GB host RAM
Strata SQ-128 or SQ-96
Vast from $0.27, typical $0.67; Runpod $1.39
RTX PRO 6000 96 GB
2-bit GGUF on the GPU, or Strata
Vast from $0.64, typical $1.47 (Max-Q); Runpod $1.64 (MaxQ), $2.69
H200 141 GB
4-bit IQ4_XS GGUF fully on the GPU
Vast from $3.00, typical $4.00 (NVL); Runpod $3.79 (NVL)
"Typical" is the middle listing on Vast, the number to budget with; the cheapest listing comes and goes within hours. On Vast, filter for at least 128 GB of system RAM if you want Strata's bigger packs, because host RAM varies from machine to machine. Our cloud GPU price tracker and the Hy4 Flash cost-to-run page show the live prices.
Live price check
RTX PRO 6000 rental prices right now
Good time to rentToday's typical on-demand price is lower than on 80% of the last 20 days.
cheapest daypriciest day
Now vs usualThe typical on-demand price is $1.47/hr now, 7% below its median since Nov 2 of $1.58/hr.
Lowest since Nov 2Lowest typical on-demand price since Nov 2: $1.45/hr on Nov 6 (Lium).
Last 7 daysOver the last 7 days the typical on-demand price is falling, down 7%.
Cheapest right nowCheapest on-demand right now: Vast.ai at $1.07/hr, then Lium at $1.70/hr.
Provider spreadThe same card costs 2.6× more on Runpod ($2.79/hr) than on Vast.ai ($1.07/hr), so shop around.
InterruptibleInterruptible (spot) offers start at $0.467/hr, 56% less than on-demand, but can be stopped mid-job.
Computed from our live prices as of . Compared like for like: the typical on-demand price is the median listing at the cheapest provider, today and on each past day. Vast.ai, Runpod, Lium and Novita, updated every hour, nothing typed by hand.
Or skip the hardware. Nous Portal lists Hy4 Flash with a free tier ($0 per million tokens, read November 22) and a paid one at $0.04 per million input tokens and $0.14 per million output. If you only want to try the model, that's the cheapest way; run it locally when you need privacy or long jobs.
How do I run Hy4 Flash?
Easiest: Strata. Install Strata v0.2.18 or later (Windows, Linux, Mac, DGX Spark) and pick the pack for your RAM: SQ-64, SQ-96 or SQ-128.
With GGUF: build llama.cpp from pull request 32517 until it merges, then load a file that fits your RAM plus VRAM from the table above.
On a server: Tencent publishes vLLM (vllm/vllm-openai:hy4-flash) and SGLang (lmsysorg/sglang:hy4-flash) images, set up for 8 GPUs with the FP8 checkpoint.
Use the recommended sampling: temperature 0.9, top-p 1.0. Thinking is on by default; set the reasoning effort to "no_think" for quick answers.
Keep the KV cache at fp16. It's small, and a quantized cache can hurt this model.
Should I buy hardware for Hy4 Flash?
Only if you already planned a big-memory machine. The cheapest way to run it well at home is a desktop with 128 GB of DDR5 and any recent 8 to 16 GB GPU, or a 128 GB Mac or Strix Halo box; a 64 GB machine runs the SQ-64 pack with a visible quality drop. With RAM prices high this year (our rent-vs-buy numbers), renting a 96 GB card for a few hours is the cheaper test. If your goal is a fast coding model on one GPU, a smaller model like Inkling 2 Mini fits a 24 GB card; best GPU for local AI compares the cards.
Hy4 Flash questions
Can Hy4 Flash run on 64 GB of RAM?
Yes. Strata's SQ-64 pack targets 64 GB of RAM plus an 8 to 16 GB GPU and is a 130 GB download. It's the lowest-quality pack (KL divergence 0.232 against 0.088 for SQ-128), and one owner found it matched Qwen 3.8 Flash Next on coding tasks at about half the speed.
How much VRAM does Hy4 Flash need?
Little on its own if you use Strata: 8 to 16 GB, with the rest in system RAM. To hold a GGUF fully on GPUs you need 83 GB for 2-bit (one 96 GB RTX PRO 6000), 123 GB for IQ4_XS (one 141 GB H200) or 161 to 168 GB for Q4_K_M.
Does llama.cpp, LM Studio or Ollama support Hy4 Flash?
Not in a release yet (Nov 22, 2026). llama.cpp support is pull request 32517, still open, so GGUF files need a build from that branch, and Ollama has no library entry. Strata, vLLM and SGLang run it today.
How fast is Hy4 Flash on an RTX 5090?
About 40 tokens a second with Strata's SQ-64 pack and 128 GB of RAM on one owner's mixed benchmark (Nov 22, 2026). Strata's own page claims 90 to 111.
How much does it cost to rent a GPU for Hy4 Flash?
A 96 GB RTX PRO 6000 costs from $0.64, typical $1.47 an hour on Vast, or $1.64 to $2.69 on Runpod (Nov 22, 2026). An RTX 5090 host with 128 GB of RAM for Strata is from $0.27, typical $0.67 on Vast.
Is there a free way to try Hy4 Flash?
Nous Portal lists a free tier for tencent/hy4-flash at $0 per million tokens (read Nov 22, 2026); how long it stays free isn't stated.