How a Code Compiler Works

Your processor can’t read a single word of code. A compiler chops your text into tokens, builds a grammar tree, checks it, does your math ahead of time, and hands the chip a handful of bytes.

Step 01 of 09

1 · Words in, numbers out

Every app you use started as text someone typed. Your processor can’t read a word of it. It only runs numbers. This box is a compiler. A page of code goes in the top, and four bytes come out the front: 8D 47 06 C3. Here’s how it gets from one to the other.

Step 02 of 09

2 · Chop the text into tokens

To the compiler, your code arrives as a row of characters, no different from a shopping list. The lexer reads them in order and groups them into tokens: keywords, names, numbers and symbols. Spaces mean nothing to it, and the comment “// combo” is thrown away. The colours in your code editor come from this same step.

Step 03 of 09

3 · Build the grammar tree

Next, the parser fits the tokens into a tree, the way you might diagram a sentence. “points + bonus * 2” becomes a + with points on one side and a * on the other. Multiply binds tighter than add, so the * sits lower in the tree and gets worked out first. Brackets and semicolons only mark where things start and stop. Once the tree is built, they’re gone.

Step 04 of 09

4 · Check that it makes sense

Now the compiler checks the meaning. Every name has to lead back to where it was declared: bonus on line 3 traces to line 2, and points to the function’s input. Every operation has to fit its types: an int times an int gives an int. Misspell bonus and the trace finds nothing. That’s a compile error, and the build stops right here.

Step 05 of 09

5 · Flatten it into tiny steps

The tree is then flattened into a list of tiny steps called the intermediate representation, or IR. Each line does exactly one thing, and every in-between result gets a name of its own, t1 to t4. The list is built from the bottom of the tree up, so the multiply lands before the add. Every later stage works on this list.

Step 06 of 09

6 · Do the math now, delete the rest

Here the compiler rewrites your program. bonus is always 3, so the optimizer swaps in the 3. And 3 × 2 is always 6, so it does that multiply once, right now, instead of every time the program runs. Steps whose results nobody needs anymore are deleted. Six steps become two: add 6 to points, then return.

Step 07 of 09

7 · Pick real instructions

Now the compiler writes for one particular chip, here an x86-64. Registers are a few storage slots built into the processor itself. On Linux and macOS, a function’s first number arrives in the register rdi, and the answer has to leave in eax. The compiler picks LEA, an instruction made for address math, because it can add 6 to rdi and drop the result in eax in a single move.

Step 08 of 09

8 · Encode it as bytes

Last, the assembler turns each instruction into bytes. 8D means LEA. 47 says “into eax, from rdi plus a small number”. 06 is the 6 itself, baked right into the code. C3 means return. That’s 32 bits, each a lamp that is on or off. This is the whole function, and it is all the processor will ever see.

Step 09 of 09

9 · Four lines in. Four bytes out.

Every build runs this pipeline over every function in your project: chop, build the tree, check, flatten, optimize, translate, encode. The compiler did arithmetic you wrote and threw out steps you asked for. The four bytes still do exactly what your four lines said.