How to Build an App That Scales to a Million Users
· 6 min read · Mona Technologies
"Will it scale?" is usually the wrong question to ask before launch, because at ten users almost anything scales. The right question is which specific parts of your system have a ceiling, where that ceiling sits, and whether you'll notice it coming or find out when your app goes down during your best week of press coverage.
The database is where it actually breaks first
Founders picture scaling failures as 'the servers fell over,' but in practice the first thing to buckle is almost always the database, specifically the number of connections it can hold open at once. PostgreSQL, one of the most widely used databases, ships with a default limit of 100 concurrent connections, and its own documentation is blunt about the fix: if you need more than 200 connections, you should be using connection pooling rather than raising the limit indefinitely, because each open connection consumes real server memory.
This matters because a common growth-stage mistake is letting every server, every background job, and every serverless function open its own direct database connection. That works fine at low traffic. At real traffic, a burst of concurrent requests can exhaust the connection limit in seconds, and the failure looks like 'random timeouts' rather than an obvious crash, which makes it painful to diagnose under pressure.
- Ask your development team directly: what happens to the database when 500 requests arrive in the same second?
- Confirm a connection pooler (like PgBouncer, or your cloud provider's managed pooling) sits between your application and the database before launch, not after an incident.
- Separate read-heavy traffic (browsing, search) from write-heavy traffic (checkout, sign-up) early, since they scale differently and fail differently.
Autoscaling isn't a switch you flip once
Cloud providers market autoscaling as if it's a checkbox: turn it on, add servers when busy, remove them when quiet. AWS's own description of the mechanism is closer to reality - it continually monitors your application and adjusts capacity based on scaling plans you configure, choosing between optimizing for performance, optimizing for cost, or balancing the two. That configuration work is the part that gets skipped.
The practical risk is a system that scales up fine but scales up too slowly. New server instances take time to boot, install dependencies, and warm up caches - often 60 to 90 seconds in real deployments. If your traffic can double in 30 seconds (a viral post, a TV mention, a flash sale), autoscaling that reacts correctly but too late still produces an outage. Ask what your scaling trigger is (CPU load? request queue length?) and what your minimum server count is during quiet hours, so cold starts aren't happening during your first traffic spike of the day.
Static and cacheable content should almost never touch your servers
The cheapest scaling win available to most apps is serving content that doesn't change per-user - product pages, blog posts, marketing pages, category listings - from a cache close to the visitor instead of regenerating it from scratch on every request. Next.js's Incremental Static Regeneration model is a good illustration of the pattern generally: pages are built once, served instantly from cache to every visitor, and only regenerated in the background on a schedule or when the underlying data actually changes, rather than on every single page load.
Applied correctly, this means a product page viewed 50,000 times a day costs roughly the same in server load as one viewed 50 times, because the database and application server are doing the work once, not 50,000 times. Apps that don't do this end up paying for enormous compute capacity just to recompute identical output over and over, which is an expensive way to fail to scale.
Decide now what breaks gracefully and what breaks completely
No system scales infinitely on every dimension at once, and pretending otherwise leads to generic 'just add more servers' advice that doesn't hold up. The useful exercise is deciding, in advance, which features are allowed to degrade under load and which absolutely cannot. Search suggestions can fall back to a simpler version under heavy load. Payment processing cannot silently drop requests - it has to queue, retry, or clearly fail, never lose the transaction.
- List your 5-10 core user actions and rank them by what happens if that feature is slow versus if it's simply down.
- For anything involving money or inventory, insist on idempotency - a repeated request (from a retry, a double-click, a network blip) should never double-charge or double-ship.
- Put rate limits on expensive operations (search, report generation, AI features) before you need them, not after one user's script takes the whole app down.
The short version
Scaling to a million users is rarely one big architectural decision - it's a handful of specific, boring choices made correctly ahead of time: pool your database connections, tune autoscaling to react before you need it rather than after, cache anything that doesn't change per visitor, and decide in advance which parts of the app are allowed to bend instead of break. Ask your development team about these four things specifically; vague reassurance that 'it's built to scale' is not an answer.