Thinking Large Language Models
Text resisted machine learning for a long time, and the reason is dependency at a distance. Change one word in a paragraph and the meaning of everything around it can invert; two sentences a hundred pages apart can be the only two that matter to each other. A model of language has to hold all of that at once. What follows is my compressed account of how one gets built, walking the lifecycle from a crawl of the web to the model you actually talk to, assembled mostly from Andrej Karpathy’s deep dive into LLMs.
Pre-training
Pre-training begins by downloading and processing the internet. FineWeb is the closest public analogue to what a frontier lab holds internally: a filtered distillation of Common Crawl. The pipeline is mostly a sequence of deletions: URL filtering to drop the parts of the web you have no interest in modelling, text extraction to keep prose and discard markup and navigation, language filtering to decide how many languages the model speaks and in what proportion, and PII removal to strip private data. What survives is not the internet but a curated slice of it, chosen for the widest possible diversity of high-quality documents.
Networks read numbers, not text, so the next problem is representation. Raw bits work but make the sequences absurdly long; group them eight at a time and you get bytes, 256 symbols for a quarter of the length. Byte pair encoding continues that trade by finding the pair of symbols that co-occurs most often, minting a new symbol for it, and repeating, buying shorter sequences with a larger vocabulary. GPT-4 stops at 100,277 tokens, and tiktokenizer shows where the splits land. A lot of model strangeness starts here: tokenization is case- and whitespace-sensitive, so hello world, Hello World and hello world are three different things, and a model that has never seen a character has no good reason to be able to spell or count one.
Training
Training samples random windows of tokens up to a maximum length, which becomes the context window, and asks the network to predict the next token. The output is a probability distribution over the entire vocabulary, and training nudges the weights up for the token that actually followed and down for everything else; this visualization walks a single pass end to end. Inference is the same machinery run forward: sample from the distribution, append, repeat. It is a biased coin flip, which is why the same prompt gives different answers. The result is a base model, an internet text token simulator. Give it the opening of a famous Wikipedia article and it recites from memory before drifting, because high-quality sources are seen many times in pre-training; give it something it has never seen and you can watch it guess. Hyperbolic is a convenient place to poke at open base models. Even at this stage they show in-context learning: a few examples in the prompt change behaviour without touching a weight.
What makes distant dependencies tractable is self-attention. Each token is projected into three vectors: a query for what it is looking for, a key for what it advertises to others, and a value for the information it contributes when matched. Weights come from query-key similarity and mix the values accordingly, so a pronoun can reach back to its noun in one step instead of relaying through everything in between. Attention on its own is order-agnostic, so position has to be injected separately, classically with positional encodings built from sine waves at varying frequencies. The architecture is still being sharpened; multi-token prediction roughly doubles inference speed without giving up accuracy.
Post-training
Post-training is computationally cheap by comparison, and it is where the model gets a personality and a rulebook. The mechanism is unchanged, still next-token prediction, and only the formatting differs: conversations are serialised into the same stream with special tokens marking the turns, as in <|im_start|>user<|im_sep|>What is 2+2?<|im_end|>. Supervised fine-tuning then imitates human-written exchanges, which is why the dataset needs breadth across question answering, summarization, brainstorming and refusal. OpenAssistant’s oasst1 is the readable human-written example, UltraChat the synthetic one (mapped here, if you want to see its coverage), and the Tülu 3 mixture a current production-scale one. When you talk to an assistant, you are talking to a statistical imitation of the labelers who wrote that data.
Reinforcement learning
If pre-training is reading the textbook and supervised fine-tuning is studying worked examples, reinforcement learning is doing the practice problems. A math question may have four correct derivations, and nobody knows which token sequence works best for the model, so rather than prescribing one we let it attempt many and reinforce whatever reaches the verified answer. DeepSeek R1 made that recipe public. This is also why RL can beat imitation: supervised fine-tuning is capped by human performance, while a model that can check its own attempts is not. AlphaGo’s move 37 is the canonical illustration.
All of which depends on being able to verify. For unverifiable tasks, such as whether a joke is funny, fine-tuning from human preferences puts a reward model between the human and the LLM: people rank a handful of outputs, a network learns to reproduce those rankings, and RL then optimises against the network instead of the person. It scales because ranking is easier than producing, the discriminator-generator gap. But a reward model is a lossy simulation of a human, and RL is very good at finding its seams; run it long enough and something like the the the the scores brilliantly. Verifiable rewards are far harder to game, which is part of why RL is not yet standard practice beyond them.
Hallucinations
Hallucinations are confidence learned by imitation. Fine-tuning data is full of assured answers, so the model produces one even where it has nothing to draw on. One fix is to train on its own boundary: interrogate it across a topic, find where it fails, and add examples whose correct response is that it does not know. That is roughly what Llama 3 did, wiring the internal signal for unknown to an admission of ignorance. The better fix is not to rely on memory at all, emitting a search token so the answer arrives in the context window instead. Knowledge in the weights is vague recollection; knowledge in the context is working memory, and working memory is the more trustworthy of the two. Asking the same question repeatedly and watching which facts move is a cheap detector, easiest in something like the inference playground.
Related, and underrated: models think in tokens. Each token costs roughly the same computation, so an answer stated in the first token had almost none behind it. That is why training data should put the reasoning before the result, and why telling a model to skip the explanation and give you the answer is asking it to be wrong; the explanation is for the model, not for you. When arithmetic or counting is involved, asking for code and letting a tool run it beats letting the model do it in its head. Even so, treat them as swiss cheese: fluent across genuinely hard problems, then falling through a hole on whether 9.11 is bigger than 9.9.
Inference
Two knobs matter at inference. Temperature sets how sharply you sample: low is deterministic and repetitive, high is creative and less reliable. Reasoning effort is the other, and it is usually misread as a model switch when it is really a budget: the same reasoning-capable model is given more room to decompose the problem, try approaches, check itself and verify before answering. Non-reasoning models answer almost immediately from what they know, which is the right trade for writing, summarization and straightforward code; the extra thinking budget earns its latency and cost on algorithm design, debugging, mathematics and long multi-step work.
Fine-tuning and compression
Fine-tuning teaches a trained model a new skill or unlocks a behaviour; making it deployable is a separate problem, and largely one of precision. As 32-bit floats a 7B model is 28 GB; half precision halves it, INT8 stores an integer plus a scale factor and reaches 7 GB, INT4 3.5 GB, with the error per value growing from roughly 1e-7 to 1e-1 along the way. Quantization can be symmetric, with the weight distribution centred on zero, or asymmetric, carrying a zero point alongside a scale of (x_max − x_min) / (q_max − q_min). It can be applied after training with a calibration pass, or during it, so the model learns to tolerate the coarser grid.
LoRA attacks the other cost, that full fine-tuning updates every weight. It freezes the original matrix and learns the update as a product of two thin ones, W′ = W + BA, with B of shape d×r and A of r×k for a rank r far smaller than either dimension. On a nine-parameter 3×3 layer, rank 1 means six trainable numbers instead of nine; at real scale the ratio is orders of magnitude, and rank is the dial between adaptation capacity and cost. QLoRA stacks the two ideas, training adapters on top of a frozen 4-bit base, which is what puts fine-tuning a 7B model on one consumer GPU. Compress further and 1-bit models claim a Pareto improvement rather than a trade.
Choosing a model
In production the harder question is which model to run, and the answer is not the cheapest model but the cheapest cost per successful task: input and output tokens, plus retries, tool calls, latency and the cost of being wrong. Output tokens run several times the price of input tokens and dominate latency; input tokens are individually cheap but explode in retrieval and agentic systems, where a naive N-step loop grows roughly quadratically. So the useful discipline is to ask, for each task, what the weakest model is that does it reliably (routing and classification rarely need what ambiguous synthesis does), and to answer that on a small eval set of your own rather than a public benchmark. Prompt caching, output length caps, query-based routing and batch APIs are the levers that apply everywhere, and token cuts that quietly cost accuracy are a false economy. The same restraint applies one level up: reach for classical ML when the task is learning a pattern from well-defined features, and for an LLM when it is understanding, transforming or reasoning over unstructured input.
This lifecycle has been stable for a few years while every stage inside it keeps moving. Simon Willison’s year-in-review posts (2023, 2024, 2025) are the best compressed history I know of, and the arena leaderboard and smol news are where I check what changed this week.