What is scaling? What happens when a hundred people show up

Ten users, no problem. Ten thousand at once is a different app. That gap is scaling.

Scaling is handling more load without falling over, either by using a bigger machine or, more often, by adding more machines.

Wikipedia: "Scalability is the property of a system to handle a growing amount of work," which "may be done by adding resources to the system."

There are two directions. Up: a bigger, stronger machine (vertical). Out: more machines sharing the load behind a load balancer (horizontal). Most modern apps scale out, running many copies of themselves.

What breaks first is usually the database, or a slow bit of code that was fine for ten people and melts for ten thousand. We hit exactly this: some of our hardware pages once did work that grew with the amount of data and began timing out under load, until we rewrote it to stay linear. "It worked yesterday" can just mean "it wasn't busy yesterday."

The cloud makes adding machines easy, on demand. But more machines don't fix a slow query; sometimes the answer is to fix the code, not to add to the count. Caching (two lessons back) often buys more than raw horsepower.

Try it with the rules off. In the sandbox on launch day, push the users up and watch errors climb. Then add servers and switch on caching and watch them fall, and notice that caching often helps more than adding machines does.

Next in when it breaks: Rate limits, explained.