awq

GGUF vs AWQ vs EXL3 (and EXL2): which quantization format should you use?

Those letters after a model's name decide what hardware it runs on and which app can load it. What a GGUF file actually is, how it compares with AWQ and ExLlama's EXL3, and why EXL2 and AutoAWQ are now in maintenance limbo.

Aliteq Team · Aug 1 · 9 min