Opening the black box of large language models: tokens, one generation step, why the first token is slow, training, serving many users, from model to application, and the harness around today's agents (skills, subagents, MCP, hooks), with experiments you can run.
A language model reads numbers, not text. From the UTF-8 bytes of Vietnamese to a hand-written BPE tokenizer, inside the tokenizers of GPT-4, GPT-4o, Qwen and DeepSeek, then measuring how many tokens Vietnamese costs and what that does to price, speed and streaming.
Rewriting Qwen2.5 in about 70 lines of PyTorch: embedding, attention, RoPE, MLP, then loading the real weights and getting the same tokens as transformers. Plus temperature, top-p, repetition penalty, and why the generation loop slows down.
Why the first token is slow and the rest come at a steady pace, what the KV cache does, and why, when generating text, the speed limit is memory bandwidth rather than computing power. Adding a KV cache to the hand-written GPT, measuring it, and trying 16-bit numbers.
Where do a model's weights come from? Training a 3-million-parameter GPT on Vietnamese Wikipedia in one minute on a CPU: data, tokenizer, the loss, the training loop, watching text take shape, and measuring the model as a compression algorithm.
How one copy of a model serves many people: batching gives 16 users nearly the price of one, continuous batching lets free slots fill at once, the KV cache decides how many users fit in memory, and what quantizing from 32 down to 8, 4, 3 and 2 bits costs.
An LLM application is text wrapped around a model. Letting Qwen call tools, writing the agent loop, then answering questions about this site's own posts with BM25-based RAG, including where a small model gets it wrong and what each technique costs.
The 20-line agent loop from part 6 and a tool like Claude Code differ in what surrounds it: the harness. Writing an MCP server from the 2026-07-28 spec, measuring how much context skills, subagents and tool definitions cost, and using hooks as a real fence.