vllm

2 articles · newest first

A server rack with network cabling in green light

how to turn your local AI into an API — a private, free drop-in for the OpenAI API

Point your apps, scripts, and coding tools at a model on your own machine instead of a paid cloud API. It's an OpenAI-compatible endpoint, it's free, and it's one command to start.

Lena Fischer · Aug 1 · 10 min

Code representing a local AI inference engine

vLLM, Ollama or llama.cpp? the three ways to run local AI, and which one is actually for you

They all run local models, but they're built for completely different people. One's for easy desktop use, one's for flexibility, one's for serving at scale. Here's how to pick without wasting a weekend.

Lena Fischer · Jul 28 · 9 min