inference

3 articles · newest first

Macro photo of two identical blank black chips side by side on a dark circuit board with gold contact pads, lit in indigo-violet light

Slow Local LLM? Speculative Decoding Explained for llama.cpp and vLLM

A small model guesses the next few words and the big one checks them all at once. Same answers, fewer slow steps. What the original paper measured, what llama.cpp and vLLM actually let you switch on, and when it won't help.

Aliteq Team · 2h ago · 9 min

Four small black computer towers with warm-lit vents in a row on a dark desk, a taller server unit with a red status light blurred behind them

Ollama vs llama.cpp vs LM Studio vs vLLM: Pick by Who's Using It

One person on one machine, or a team hitting one server? That single question decides it. Then licence, GPU and Mac support, and the API your app talks to, all read from each project's own docs.

Aliteq Team · Oct 4 · 9 min

Flat illustration of a person toggling a serverless GPU on and off — pay only while it runs — on a teal background

RunPod Serverless pricing, explained: why it looks pricier but usually isn't

Serverless per-hour rates look higher than a pod — but they bill per second and scale to zero. The whole decision is one number: how busy your GPU actually is.

Aliteq Team · Sep 23 · 7 min