This series takes a computer apart layer by layer, starting with the lowest one: numbers. The goal of this first post sounds simple: follow the calculation 1 + 1 from a line of code down to electricity flowing through a few hundred transistors, and understand why a chip can do it billions of times a second.
The route: everything is bits, how negative numbers are written, why 0.1 + 0.2 isn't 0.3, from switches to logic gates, building an adder out of nothing but NAND gates, what the clock is and why it limits speed, measuring the CPU's clock with addition itself, reading the add machine instruction bit by bit, then seeing how much more 1 + 1 costs in Python.
Every number in this post was measured on a real machine, with Python 3.12 and the clang compiler. The machine code uses the arm64 instruction set (the kind of CPU in most phones and many laptops); on x86 CPUs the instructions have different names, but the ideas are exactly the same.
Everything is bits
Inside the chip, each "number" is a group of wires, each wire in one of two states: high voltage or low voltage. Call them 1 and 0; each state is a bit. Eight bits make a byte, enough for 256 different values.
The number 2 in bits is 10: one 2, no 1s. In memory, a 64-bit integer takes 8 bytes:
print(bin(2)) # 0b10
print((2).to_bytes(8, "little").hex(" ")) # 02 00 00 00 00 00 00 00
The byte holding the smallest part comes first ("little-endian", the way almost every common CPU today stores numbers). Nothing deep here; it's just a convention hardware and software have to agree on.
Negative numbers: two's complement
No wire carries a "minus sign". The way almost every CPU writes negative numbers is called two's complement: with 8 bits, −1 is 11111111 and −2 is 11111110.
print((-1).to_bytes(1, "little", signed=True).hex()) # ff
print(f"{-2 & 0xff:08b}") # 11111110
The reason for this odd notation: with two's complement, adding negative numbers uses exactly the same circuit as adding positive ones. 11111111 (−1) plus 00000001 (1) gives 1 00000000; the ninth bit falls off, leaving 00000000 = 0. Correct. The CPU needs no separate circuit for negatives.
The price is a limit. A signed 8-bit number only holds −128 to 127. Add 1 to 127:
import ctypes
print(ctypes.c_int8(127 + 1).value) # -128
01111111 + 1 = 10000000, which in two's complement is −128. This is overflow, and it has caused more than a few real software failures. Python avoids it with a trick: Python integers have no fixed size. The number 1 takes 28 bytes (mostly the object header, as Inside CPython showed), while 2**64 takes 36 bytes; however big a number gets, Python allocates more room for it.
Floats: why 0.1 + 0.2 ≠ 0.3
print(0.1 + 0.2) # 0.30000000000000004
print(0.1 + 0.2 == 0.3) # False
Floating-point numbers in a computer follow the IEEE 754 standard: 64 bits split into three parts, like scientific notation (1.6 × 2⁻⁴):
0.1 = 0 01111111011 1001100110011001100110011001100110011001100110011010
↑ ↑ ↑
│ │ mantissa (52 bits): the fraction part of 1.6 (the leading 1 is implied)
│ exponent (11 bits): −4
sign (1 bit): positive
The problem is in the mantissa. 1/10 in binary is a repeating fraction, 0.0001100110011…, just as 1/3 = 0.333… in decimal. 52 bits have to stop somewhere, so the number the machine actually stores isn't 0.1:
from decimal import Decimal
print(Decimal(0.1)) # 0.1000000000000000055511151231257827021181583404541015625
print(Decimal(0.1 + 0.2)) # 0.3000000000000000444089209850062616169452667236328125
Each number is off by a tiny amount, around the 17th–18th digit after the decimal point, and the two errors add up to just enough to miss the number closest to 0.3. This isn't a Python bug; every language using IEEE 754 floats gives the same result. The practical consequences: don't compare floats with ==, and don't use floats for money (use integer cents, or Decimal).
From switches to logic gates
The basic building block of a chip is the transistor: a switch controlled by electricity. The voltage on one terminal decides whether current flows between the other two. A modern chip has tens of billions of them.
A few transistors together make a logic gate. The most important one is NAND ("not and"): the output is 0 if and only if both inputs are 1. In the common chip technology (CMOS), a NAND gate takes 4 transistors. What's special about NAND is that every logic circuit can be built from NANDs. No need to take that on faith; write it out:
COUNT = {"nand": 0}
def nand(a: int, b: int) -> int:
"""The only gate we allow ourselves: 0 only when both inputs are 1."""
COUNT["nand"] += 1
return 0 if (a and b) else 1
def not_(a): return nand(a, a)
def and_(a, b): return not_(nand(a, b))
def or_(a, b): return nand(not_(a), not_(b))
def xor(a, b):
n = nand(a, b)
return nand(nand(a, n), nand(b, n))
- NOT is a NAND with both inputs the same.
- AND is the NOT of a NAND.
- OR: "a or b" equals "not (not a and not b)".
- XOR (1 when the inputs differ) is built from 4 NANDs.
Building an adder
Adding two binary numbers works just like adding decimal numbers by hand: add each column from the right, carrying into the next column. Each column has three bits in (a's bit, b's bit, and the carry from the previous column) and two bits out (the sum bit and the carry into the next column). The circuit that does this is called a full adder:
def full_adder(a: int, b: int, carry_in: int) -> tuple[int, int]:
"""Add three bits: returns (sum bit, carry out)."""
s1 = xor(a, b)
total = xor(s1, carry_in)
carry_out = or_(and_(a, b), and_(s1, carry_in))
return total, carry_out
def add(x: int, y: int, bits: int = 8, trace: bool = False) -> int:
"""Ripple-carry adder: one full adder per bit, the carry passed along like a bucket brigade."""
carry, out = 0, 0
for i in range(bits):
a, b = (x >> i) & 1, (y >> i) & 1
s, new_carry = full_adder(a, b, carry)
if trace:
print(f" bit {i}: a={a} b={b} carry_in={carry} -> sum={s} carry_out={new_carry}")
out |= s << i
carry = new_carry
return out # the final carry falls off the end: that's overflow
Line up 8 full adders, each one's carry wired into the next, and you have an 8-bit adder. Now 1 + 1:
1 + 1 on an 8-bit adder:
bit 0: a=1 b=1 carry_in=0 -> sum=0 carry_out=1
bit 1: a=0 b=0 carry_in=1 -> sum=1 carry_out=0
bit 2: a=0 b=0 carry_in=0 -> sum=0 carry_out=0
...
bit 7: a=0 b=0 carry_in=0 -> sum=0 carry_out=0
result 00000010 = 2
That's the whole calculation: in column 0, 1 + 1 = 0 carry 1; in column 1, 0 + 0 + carry 1 = 1. The result is 00000010, which is 2. Nothing here "understands" numbers; there are only NAND gates switching on and off.
A few numbers about the circuit we just built:
| NAND gates | |
|---|---|
| One full adder (built as above) | 15 |
| 8-bit adder | 120 |
| 64-bit adder | 960 |
(Built more cleverly, a full adder needs only 9 NAND gates; the version above is easier to read.) Testing every pair of 8-bit numbers, all 65,536 sums are correct (modulo 256). And 255 + 1 gives 0: the last carry falls off the left end, exactly the overflow from earlier, now seen with your own eyes.
The clock: the chip's heartbeat
The adder above doesn't produce its result instantly. Each gate needs a tiny amount of time for its output to follow its inputs, and in this "bucket brigade" adder, the carry has to pass through every column in turn: column 63 only has the right answer after the carry has run through the 63 columns before it.
So how does the CPU know when the result is "done"? It doesn't. Instead, the whole chip runs to a clock: a signal oscillating steadily between 0 and 1, usually derived from a quartz crystal and multiplied up to a high frequency by a PLL circuit on the chip. On each rising edge of the clock, the registers (the smallest storage cells, right inside the CPU core) latch whatever value is at their inputs. Between two latches, current flows through the logic gates, and has to settle before the next rising edge.
clock: _|‾‾|__|‾‾|__|‾‾|__|‾‾|_
↑ ↑ ↑ ↑ each rising edge: the registers latch new values
└─────┘
one cycle: the logic has to settle within this window
A 4.5 GHz clock means 4.5 billion cycles a second, each about 0.22 nanoseconds. In that time, light in a vacuum travels only about 6.6 cm. Everything in one cycle has to happen in the time light takes to cross the width of a phone.
This explains two things:
- Why real CPUs don't use a "bucket brigade" adder. Letting the carry run through 64 columns is too slow. Adders in real CPUs work out the carries for groups of columns in parallel (carry-lookahead and its variants), so the number of gate levels a signal goes through grows only with log₂ of the number of bits: about 6 levels of carry computation for 64 bits instead of 64. More gates, in exchange for finishing within one cycle.
- Why clocks can't just keep getting faster. The shorter the cycle, the less time the logic has to settle, and every switch costs energy and makes heat. Since the mid-2000s, clock speeds have stalled at a few GHz; CPUs get faster mostly by doing more work per cycle (next section) and having more cores.
Measuring the clock with addition
There's a way to measure the clock without asking the operating system: tell the CPU to do a chain of dependent additions, each needing the result of the one before (x = x + 1, then x = x + 1 again…). On modern CPUs, an integer add has a latency of exactly one cycle, so additions per second equals cycles per second.
Written in C with inline assembly, so the compiler can't "cleverly" fold 100 additions into one multiplication:
// Estimate the clock: a chain of dependent adds runs at one add per cycle.
#include <stdio.h>
#include <stdint.h>
#include <time.h>
#define ADD10 "add %0, %0, %1\n\t" "add %0, %0, %1\n\t" "add %0, %0, %1\n\t" "add %0, %0, %1\n\t" "add %0, %0, %1\n\t" \
"add %0, %0, %1\n\t" "add %0, %0, %1\n\t" "add %0, %0, %1\n\t" "add %0, %0, %1\n\t" "add %0, %0, %1\n\t"
static double now(void) { struct timespec t; clock_gettime(CLOCK_MONOTONIC, &t); return t.tv_sec + t.tv_nsec * 1e-9; }
int main(void) {
const long loops = 20000000; // x 100 adds = 2 billion adds
uint64_t x = 0, one = 1;
double best = 1e9;
for (int run = 0; run < 3; run++) {
double t = now();
for (long i = 0; i < loops; i++)
__asm__ volatile(ADD10 ADD10 ADD10 ADD10 ADD10 ADD10 ADD10 ADD10 ADD10 ADD10 : "+r"(x) : "r"(one));
t = now() - t; if (t < best) best = t;
}
printf("dependent: %.2f billion adds/s (x=%llu)\n", loops * 100 / best / 1e9, (unsigned long long)x);
}
Add a second version, also 100 additions per loop but spread over four independent variables (a, b, c, d), so the additions don't wait for each other. The results (best of 3 runs, 2 billion additions each):
| Additions per second | Meaning | |
|---|---|---|
| Dependent | 4.57 billion | ≈ the clock: about 4.6 GHz |
| Four independent chains | 13.26 billion | ≈ 2.9 additions per cycle |
The first number is this core's clock while running flat out (modern CPUs change their clock with load and temperature, so they run slower when idle). The second is more interesting: a CPU core has several adders (several ALUs) working in parallel, and when instructions don't depend on each other, it notices on its own and runs them in the same cycle. The additions are the same; the only difference is whether one instruction has to wait for another. That's the subject of the next post: how a CPU finds parallel work in a program written to run one line at a time.
1 + 1 in machine code
A C function that adds two numbers:
int add(int a, int b) { return a + b; }
compiles to exactly one instruction, plus the return:
0: 0b000020 add w0, w1, w0
4: d65f03c0 ret
The add instruction is the four bytes 0b000020. Written out bit by bit, each group of bits has a meaning:
0b000020 = 0 00 01011 00 0 00000 000000 00001 00000
│ │ │ │ │ │ │ │ └ Rd = 0: write the result to register w0
│ │ │ │ │ │ │ └ Rn = 1: first operand from w1
│ │ │ │ │ │ └ shift the second operand by 0 bits
│ │ │ │ │ └ Rm = 0: second operand from w0
│ │ │ └──┴ shift type = 00, bit 21 = 0
│ │ └ 01011: the "add/subtract two registers" instruction group
│ └ 00: add (not subtract), don't update the flags
└ 0: 32-bit numbers
By the arm64 calling convention, a is in register w0 and b in w1, and the return value must also be in w0. So add w0, w1, w0 does exactly w0 = b + a.
When the CPU meets these four bytes, this is what happens:
- Fetch: read 4 bytes from memory (really from a cache, next post).
- Decode: the wires carrying bits 24–28 (
01011) switch on the part of the circuit meant for addition; bits 5–9 and 16–20 select the two registers to read. - Execute: the two registers' values flow into the adder, the same adder as above, just wider and faster.
- Write back: on the next rising edge of the clock, the result is latched into register
w0.
Those four steps don't happen in one cycle. The CPU splits them into a pipeline: while this instruction is executing, the next one is being decoded, and the one after that is being fetched. Each instruction takes several cycles to go through the pipeline, but every cycle a new one comes out at the other end. That's how an addition has a "latency of 1 cycle" despite having a dozen steps.
And in Python?
import dis
dis.dis(compile("x + y", "", "eval"))
LOAD_NAME 0 (x)
LOAD_NAME 1 (y)
BINARY_OP 0 (+)
RETURN_VALUE
BINARY_OP isn't the CPU's add instruction. It's a stretch of C code inside the CPython interpreter: check the types of both objects, find the int type's add function, read the values out of the two objects, add them, then create (or reuse) a result object. Timing it:
x + y in Python 3.12 |
Time | About how many cycles |
|---|---|---|
1 + 1 |
4.1 ns | ~19 |
0.1 + 0.2 |
6.0 ns | ~27 |
10**30 + 10**30 |
10.0 ns | ~46 |
(Timed with timeit, including the cost of the timing loop itself.) The hardware's add takes 1 cycle; 1 + 1 in Python takes about 19 cycles, because most of the work is managing objects, not adding. For big numbers, Python adds 30 bits at a time, the way we add digit by digit on paper. That's no reason to drop Python; 4 nanoseconds is still very fast. But it explains why number-crunching libraries like NumPy or PyTorch (part 2 of the LLM series) pack millions of operations into a single call down into C: to pay the "object management" cost once for a whole array, instead of once per number.
Summary
| Concept | What it really is |
|---|---|
| Bit | One wire at high or low voltage |
| Two's complement | A way of writing negatives so they share the adder with positives |
| Overflow | The last carry falls off: 127 + 1 = −128 with 8 bits |
| IEEE 754 | Sign + exponent + mantissa; 0.1 repeats forever in binary, so it gets rounded |
| Transistor and NAND | An electrically controlled switch; 4 transistors make a NAND; every circuit can be built from NANDs |
| Adder | One full adder per bit; 1 + 1 = 0 carry 1, then 0 + 0 + carry 1 = 1 |
| Clock | The latching beat of the registers; logic must settle within one cycle (~0.22 ns at 4.5 GHz) |
| Measuring the clock | A dependent chain of adds: ~4.6 billion/s; independent: ~2.9 per cycle |
| Machine instruction | add w0, w1, w0 is 4 bytes; groups of bits choose the operation and the registers |
| Python | BINARY_OP ≈ 19 cycles for 1 + 1: mostly managing objects |
The next post goes up one floor: what a CPU does to run almost 3 instructions per cycle from a program written one line at a time, why it has to guess which way an if will go, and why reading one number from RAM can be slower than a hundred additions.