someone got a real ai talking on a chip that costs less than your coffee

A developer who goes by Slava S. just got a real language model — not a chatbot gimmick, an actual 28.9-million-parameter neural network — running entirely on an ESP32-S3, the same $8 microcontroller you'd find inside a smart plug or a DIY weather station. It writes short stories, on-device, with no cloud call and no Wi-Fi required, printing text at just under 10 tokens a second onto a tiny I2C display. The project, published to GitHub as esp32-ai in late July 2026, isn't a toy demo pretending to think. It's a genuine architecture trick borrowed from Google's Gemma, and it's roughly 100 times bigger than the last language model anyone successfully ran on hardware this small.

How you cram an AI into 512 kilobytes

Here's the actual problem this project solves: a language model's size mostly lives in its embedding table, the lookup structure that turns each token into a vector the network can process. For a 28.9-million-parameter model, that table alone is around 25 million parameters, and there's no way that fits in 512KB of fast SRAM alongside everything else the chip needs to do. The old approach was to shrink the whole model until it fit in RAM, which is how you end up with a 260,000-parameter toy that can barely string a sentence together. Slava's approach instead borrows Gemma 3n's Per-Layer Embeddings idea: keep the table in slow 16MB flash memory, memory-mapped so it reads like RAM, and pull only the handful of rows, about six of them, roughly 450 bytes, that a given token actually needs. The dense processing core, a much smaller ~560K parameters, is the only part that has to live in fast memory the whole time.

None of this makes the ESP32 a replacement for anything you'd use day to day. The model was trained purely on TinyStories, a dataset built specifically for tiny models, so it can only generate short, simple narratives — ask it a real question and it will just keep telling you a story instead. That's the honest ceiling here. But the technique underneath it isn't a toy: Per-Layer Embeddings is the same idea Google ships in production Gemma models, just applied at a scale nobody had bothered to test on a chip this cheap before.

How each word actually gets generated

Why this actually matters beyond the demo

The interesting part isn't that a microcontroller can now write toy stories — it's what the memory trick implies for everything one size class up. The same logic that makes 25 million parameters fit in flash instead of RAM is exactly why phones and integrated graphics chips are getting noticeably better at running real local models without dedicated VRAM. If you've read about running a local model without a GPU at all or wondered what a token actually is once you strip away the marketing, this project is that same architecture pushed to its most extreme, cheapest possible endpoint — and it still works.

My honest take: this won't replace your Raspberry Pi local-AI setup, and it isn't supposed to. What it proves is that the industry's obsession with ever-bigger models has a mirror image nobody pays enough attention to — models getting radically more efficient at the bottom end, not just bigger at the top. If you've thought about what running AI actually costs in electricity at GPU scale, keep an eye on this end of the market instead. Efficiency tricks like this one tend to climb up the stack, not stay stuck at the bottom.