vllm

5 articles · newest first

Four small black computer towers with warm-lit vents in a row on a dark desk, a taller server unit with a red status light blurred behind them

Ollama vs llama.cpp vs LM Studio vs vLLM: Pick by Who's Using It

One person on one machine, or a team hitting one server? That single question decides it. Then licence, GPU and Mac support, and the API your app talks to, all read from each project's own docs.

Tensor · 5d ago · 9 min

A developer's laptop screen showing a terminal window running a local AI model

I compared Ollama, vLLM and LM Studio — only one of them survives a second user

at one person typing, all three are basically tied. add a second and the gap turns into a cliff

Tensor · Aug 22 · 7 min

A server rack with network cabling in green light

how to turn your local AI into an API — a private, free drop-in for the OpenAI API

Point your apps, scripts, and coding tools at a model on your own machine instead of a paid cloud API. It's an OpenAI-compatible endpoint, it's free, and it's one command to start.

Tensor · Aug 1 · 10 min

Code representing a local AI inference engine

vLLM, Ollama or llama.cpp? the three ways to run local AI, and which one is actually for you

They all run local models, but they're built for completely different people. One's for easy desktop use, one's for flexibility, one's for serving at scale. Here's how to pick without wasting a weekend.

Tensor · Jul 28 · 9 min

A multi-GPU workstation built for running large language models locally

The cheapest way to run Llama 70B isn't the one you're about to build

"Cheapest" flips completely on how often you run it and how many people hit it at once — and in Europe, the electricity bill alone can cost most of what renting does.

Tensor · Jul 21 · 10 min