someone got a real ai talking on a chip that costs less than your coffee

A hobbyist just proved a genuine language model can run entirely on an $8 microcontroller — no cloud, no GPU, no internet connection required. Aliteq Ravi Malhotra · Hardware Editor

The model has 28.9 million parameters and runs on an ESP32-S3 board that costs about $8, with 512KB of SRAM, 8MB of PSRAM, and 16MB of flash storage.

It generates text at roughly 9.88 tokens per second — genuinely readable speed for a chip this cheap.

The trick is Google's Per-Layer Embeddings technique from Gemma 3n: instead of one giant embedding table in fast memory, only about 450 bytes per token get pulled from slow flash storage.

It was trained on the TinyStories dataset, so it only writes simple children's-style stories — it can't answer questions, follow instructions, or write code.

The previous record for an LLM on hardware this constrained was a 260,000-parameter model. This one has about 100 times more parameters.

The whole project is open source. The reusable architecture lives in src/, and the training and quantization code that reproduces the published numbers is in research/tinystories/ on the GitHub…

Worth trying yourself

Read the full story

Aliteq someone got a real ai talking on a chip that costs less than your coffee Read the full story on Aliteq