A developer who goes by slvDev just got a real language model running entirely on an ESP32-S3 — a microcontroller that costs about $10 (some reports put it closer to $8, depending on the vendor) and is better known for smart plugs and video doorbells than artificial intelligence. No Wi-Fi call, no cloud API, no GPU anywhere near it. The chip generates original short stories at about 9.5 tokens per second, printing them straight to a wired display, and per Tom's Hardware, the model behind it has 28.9 million parameters — roughly 100 times bigger than anything previously demonstrated on a chip this small.
The trick: keep the big part of the model somewhere you don't have to load it
A standard transformer keeps one embedding table and folds most of its capacity into dense weight matrices that all have to sit in memory at once, ready to be multiplied against whatever comes through next. That's exactly what a microcontroller with half a megabyte of SRAM can't do. Per-Layer Embeddings, the technique Google built for its Gemma models, splits things differently: a small, dense compute core stays in fast memory, while a much larger embedding table — one that gives every token its own learned vector at every layer — sits in flash storage and gets looked up a few hundred bytes at a time, exactly when it's needed.
A token arrives at the model.
2
The chip pulls roughly 450 bytes of that token's embedding slice out of the 16MB flash table — not the whole table, just that token's row for that layer.
3
That slice combines with the tiny ~3.9-million-parameter compute core, which runs 4-bit quantized in the 8MB of PSRAM.
4
The next token prints to the screen, and the chip repeats the lookup — about 9.5 times a second.
~$8–10
Chip cost
512 KB
SRAM
8 MB
PSRAM
16 MB
Flash storage
28.9 million
Total parameters
9.5 tok/sec
Speed
It doesn't know anything, can't answer a question, and can't hold a conversation. It just writes toddler-length stories, forever, on a chip built for doorbells — and that's exactly the point.
Why this matters beyond doorbells
My honest take: this won't replace anything you're running through Ollama on a real GPU, and it isn't trying to. It can't. What it demonstrates is that the industry's obsession with bigger GPUs and more VRAM is only half the story — the other half is architecture, and a hobbyist with a cheap microcontroller just made that case more convincingly than most research papers do.
The ESP32-S3 was designed for smart-home devices, not AI — which is exactly why running a language model on it is notable. · Unsplash
What it can't do — and what to actually watch next
Be clear about the limits: this model can't answer a question, follow an instruction, or recall a fact outside its training stories. It's a proof of concept for an architecture idea, not a product. But proof-of-concept is exactly how most of the local-AI trends this site covers started — running DeepSeek on a consumer GPU looked like a stunt not long before it became routine. The thing worth watching isn't whether your smart speaker gets a chatbot. It's whether a chip vendor takes this same trick one size class up — a wearable or a basic smart-home hub running a genuinely useful few-hundred-million-parameter model, not just short stories.
Quick answers
What is Per-Layer Embeddings?
A technique from Google's Gemma models that stores each layer's embedding vectors in a separate table, so most of a model's raw parameter count can live in flash or run on CPU rather than needing to sit in accelerator memory all at once.
Can I run a real chatbot on an ESP32?
Not with this project. It's trained only on short children's stories and can't answer questions or follow instructions — it's a demonstration of an architecture, not a usable assistant.
Is the code available?
Yes — slvDev published the full project on GitHub under an MIT license, including the training pipeline and the ESP32-S3 firmware.
Does this matter if I never touch a microcontroller?
Yes, indirectly. The same Per-Layer Embeddings idea is already in Gemma 3n and Gemma 4, which is why those models run on phones with far less accelerator memory than their parameter counts would suggest.
Nobody's shipping a doorbell with opinions any time soon. But the direction here is worth sitting with: for years, running a bigger model meant buying a bigger GPU. This is a small, very specific counter-example — proof that with the right split between what has to be fast and what can afford to be slow, the floor for 'a language model can run here' keeps dropping. The full project is open source on GitHub if you want to try it yourself.
