First-principles thinking: reason from the ground up, not from habit

Most of our everyday decisions are reasoning by analogy: others do it this way, the last project did it this way, so we do too. It's fast and usually good enough. But it also copies the limits of whoever came before.

First-principles thinking goes the other way: strip a problem down until only things you know to be true are left, then build the answer back up from there, as if nobody had solved it before.

What a first principle is

The idea is old. Aristotle called a first principle the first basis from which a thing is known: something that can't be deduced from anything else. In mathematics those are axioms, in physics they're laws, and in everyday work they're usually:

• Physical limits: the speed of light, bandwidth, memory, the hours in a day.
• Measured data: real numbers, not impressions.
• What users actually need: what they need to get done, not the feature they ask for.

Everything else, like "this framework is the standard", "a website needs a slider" or "every project needs microservices", is an assumption. Assumptions can be right, but they have to be checked.

A famous example: the price of batteries

Elon Musk often tells this one when he talks about first principles. As he tells it, around 2012 battery packs for electric cars cost about 600 US dollars per kWh, and everyone assumed they always would. Instead of comparing with market prices, he asked a different question: what is a battery made of? Cobalt, nickel, aluminum, carbon, some polymers and a steel can. Buying exactly those materials on the London Metal Exchange came to only about 80 dollars per kWh.

The gap between 80 and 600 isn't a law of nature. It's the cost of the way batteries were being made, so it could be made differently.

Three steps

1. Write down your assumptions. Spell out what you're taking for granted. This step alone often shows the problem, because "everyone does it" isn't a reason.
2. Break them down until you hit facts. For each assumption, ask "is this certainly true, and why?" until you reach a physical limit, a measured number or a real need.
3. Rebuild from those facts. Set the old answer aside and ask: given only what's certainly true, what's the simplest way to reach the goal?

Example: serving a million users

A very common requirement: "the system has to handle a million users." The usual reflex is to reach for multithreading: a million people need lots of threads, the more the better. To be concrete, say the backend is Python with FastAPI, run by uvicorn, a very common choice. Let's go through the three steps.

1. Write down your assumptions

• A million users means a million people using it at the same time.
• Each request needs its own thread, so we need a lot of threads.
• More threads means more work gets done.

2. Break them down until you hit facts

A million users isn't a million requests at once. "A million users" is usually the number of accounts; far fewer visit on any given day. Say all million do visit every day: a day has 86,400 seconds, and each person sends a few dozen requests to the application server (images, CSS and JavaScript come from a CDN). How many requests are being handled at any moment follows Little's law: work in progress = arrivals per second × time each one takes.

A request's time has two very different parts. Time using the CPU (running code, rendering HTML) and time waiting on I/O: waiting for data to move over the network or the disk, like waiting on the database or another API. Work that is mostly the first is called CPU-bound; mostly the second, I/O-bound. Only the first needs the CPU; while waiting on I/O, the CPU is idle.

The number of CPU cores is a physical limit. A 4-core machine runs 4 things at a time. If 1,000 threads all need the CPU, 996 of them still wait in line, plus the cost of switching between them. Python adds one more fact: the default CPython build has a GIL, which lets only one thread run Python code at a time, so threads don't help with the CPU part at all: one uvicorn process can use only about one core, however many threads it has. (Since Python 3.14 there's an officially supported build without the GIL, but you have to opt in.)

Put those facts into a calculation:

import math

SECONDS_PER_DAY = 86_400


def capacity(daily_users: int, requests_per_user: int, peak_factor: float, cpu_ms: float, wait_ms: float) -> None:
    peak_rps = daily_users * requests_per_user / SECONDS_PER_DAY * peak_factor
    in_flight = peak_rps * (cpu_ms + wait_ms) / 1000  # Little's law: L = λ × W
    busy_cores = peak_rps * cpu_ms / 1000  # only CPU time needs a core; waiting doesn't
    print(f"peak requests per second : {peak_rps:6.0f}")
    print(f"requests in flight       : {in_flight:6.0f}")
    print(f"CPU cores kept busy      : {busy_cores:6.1f}")
    print(f"4-core servers at 70%    : {math.ceil(busy_cores / (4 * 0.7)):6d}")


# 1 million users a day, 30 requests each to the app server (static files come from a CDN),
# peak 10× the daily average to leave room for bursts,
# each request: 5 ms of CPU, 45 ms waiting on the database and other services
capacity(1_000_000, 30, 10, cpu_ms=5, wait_ms=45)

peak requests per second :   3472
requests in flight       :    174
CPU cores kept busy      :   17.4
4-core servers at 70%    :      7

The number to look at is 174, not a million: even at a peak generously set at 10× the average, only about 174 requests are in progress. And the real limit is 17 CPU cores, which no arrangement of threads can change. The 70% isn't arbitrary: as the CPU gets close to 100%, the time requests spend waiting in line shoots up, so you always leave headroom.

3. Rebuild from those facts

Waiting on I/O decides how many requests are waiting at once, and so how to hold them: a thread per request, or a coroutine per request. CPU work decides how many processes and machines you need.

• The I/O wait: each uvicorn process has one event loop running on one thread. An endpoint written as async def is a coroutine running in that loop: at an await (waiting on the database, on an API) the coroutine pauses and hands the loop to another request, and is woken up when the data arrives. A waiting request is just a coroutine costing a few KB of memory, holding no thread (non-blocking I/O). Endpoints written as plain def are sent by FastAPI to a thread pool (40 threads by default), and each request holds a thread for the whole wait: the thread is blocked and can't do anything else (blocking I/O). With 174 requests in progress across 7 machines, about 25 per machine, either way is enough. The difference only shows when waits get long: if every request waits 10 seconds on an outside API, say an AI model, the same calculation gives about 35,000 requests waiting. A 40-thread pool chokes at once, while the async loop copes, as long as nothing inside async def calls a synchronous library (like requests or a synchronous database driver), because one such call blocks the whole loop.
• The CPU work: one process uses only one core, so the number of processes has to match the number of cores. There are two ways: run uvicorn --workers 4 on each 4-core machine (4 processes, each with its own loop, accepting connections on the same port), or run one process per pod that requests 1 core, and add pods. To handle more load, add processes, not threads. To go faster, cut the CPU time of each request (caching, removing wasted work). And don't do CPU-heavy work like resizing images right inside async def: while it runs, the loop stands still and every other request in that process waits. (What happens underneath, from the GIL to the kernel, is in How Python runs concurrent code.)
• Measure before you believe: the 30 requests, 5 ms and 45 ms above are assumptions. Replace them with real numbers from your logs. If a request actually takes 50 ms of CPU, you need ten times the machines, and that's the thing to optimize.

Scale out, and only where the CPU is actually needed. Since the limit is cores, the way to handle more load is to run more machines side by side behind a load balancer: 7 pods with 4 cores each, plus at least one spare so a single failed pod doesn't overload the rest. For that, the app has to be stateless: sessions, carts and uploaded files live in the database, Redis or object storage, not in one process's memory, so any request can run in any process. That holds even with just 4 workers on one machine, because uvicorn processes don't share memory.

But "5 ms of CPU on average" is also an assumption worth breaking down. Measure by request type and you'll often see a split like this: 90% of requests are page views at 2 ms of CPU each; 10% are image uploads that get resized, at 32 ms each. The average is still 5 ms, but:

10% of requests use 64% of the CPU. If everything lives in one service, every time users upload images at once, the site gets slower for everyone, and each time you scale you have to copy the whole app just for one feature. Split image processing into its own service (a worker taking jobs from a queue) and:

• Each service scales on its own load: web pods by request rate, worker pods by queue length.
• Spikes in the heavy part don't slow down the light part.
• Work that doesn't have to finish right away waits in line. A thumbnail ready a few seconds later is fine, so short bursts only make the queue longer for a while. Workers can be sized for the busiest hour's average instead of the busiest second, which means fewer pods than the 4 in the table.

The next bottleneck is the database. Adding app pods doesn't make the database faster, and each new pod opens more connections to it. The same calculation, applied to queries per second and time per query, tells you what the database needs: better indexes, caching, or read replicas.

Don't split the other way around: breaking an app into a dozen small services "to do microservices properly" before any measurement shows which part is heavy only adds network latency and more to operate.

The problem only really changes when a million people keep connections open at once, like a chat app on WebSockets. Then the work in progress really is a million, and the deciding fact is the cost of each connection. A thread per connection hits operating-system limits early: every thread needs its own stack mapping, and Linux by default allows each process only about 65,000 memory mappings. With a model lighter than threads, the Phoenix (Elixir) team held 2 million WebSocket connections on one machine with 40 cores and 128 GB of RAM in 2015. Same three steps, different facts, different conclusion.

Questions to ask yourself

• What am I believing without having checked it?
• If I started from zero, would I do it exactly this way?
• Is this limit physical, or just how it's usually done?
• What's the real number? Have I measured it?
• What do users need to get done, as opposed to what they say they want?

When not to use it

First-principles thinking takes time and effort. You don't need it for everything: naming a variable, picking a date library or configuring a linter is fine done the usual way. Save it for decisions that are expensive and hard to undo: system architecture, pricing, core data structures, or problems where every familiar approach has already failed.

And don't confuse "reasoning from the ground up" with "ignoring experience". Other people's experience is valuable data. The goal is to understand why they did it, so you know whether it still holds for your problem.

In short

Analogy helps you move fast along a road that already exists. First principles help you see whether that road is needed at all. Next time you hear "we've always done it this way", ask one more question: "Why?", and keep asking until you hit something that's certainly true.