Fancy Autocomplete: How LLMs Work

The World's Most Overqualified Autocomplete

Here's the secret at the heart of every large language model, and it sounds almost insultingly simple:

An LLM predicts the next word. That's it. That's the trick. Given everything written so far, it asks: "What token most plausibly comes next?" Then it appends that token and asks again. And again. One token at a time, until an entire essay, poem, or apology email to your landlord falls out.

People call this "fancy autocomplete," usually as a dunk. But let's give the phrase the respect it deserves, because the fancy part is doing heroic amounts of work.

Your Keyboard vs an LLM

Your phone keyboard also predicts the next word. Type "see you" and it offers "later," "soon," "tomorrow." It's working from a shallow bag of statistics: which words tend to follow which other words.

An LLM plays the same game with a radically deeper understanding of context. Ask your keyboard to continue "The mitochondria is the..." and it might manage "best." An LLM completes it with "powerhouse of the cell," then explains cellular respiration, then makes a meme reference about it, because it has absorbed patterns from a colossal amount of text, and it weighs the entire conversation, not just the last two words.

Why Prediction Starts Looking Like Understanding

Here's the mind-bending part. To predict the next word really well, shallow statistics stop being enough.

To finish the sentence "The trophy didn't fit in the suitcase because it was too...", you have to figure out whether "it" means the trophy (too big) or the suitcase (too small). That's not word statistics. That's reasoning about objects and space.

To predict the last line of a murder mystery, you'd need to track clues. To complete a buggy program with the fixed version, you need something that functions like knowledge of programming. Prediction at massive scale quietly smuggles in something that behaves like understanding, because the best way to predict text about the world is to model the patterns of the world that produced it.

One Token at a Time

Watch a chatbot type out its answer word by word. That's not a loading animation for vibes. It's literally how generation works. Predict a token, append it, feed the whole thing back in, predict the next. Every word the model writes becomes part of the context for the following word.

Watch Prediction Happen

Here's a single sentence being generated the way a model actually does it, one prediction at a time, with the leading candidates at each step.

So far: "The cat sat on the"
Candidates: mat (72%) | couch (11%) | roof (6%) | keyboard (4%)
Picked: "mat"
So far: "The cat sat on the mat"
Candidates: . (48%) | and (21%) | , (14%) | because (5%)
Picked: "and"
So far: "The cat sat on the mat and"
Candidates: purred (31%) | stared (22%) | refused (12%)
Picked: "refused"
So far: "The cat sat on the mat and refused"
Candidates: to (89%) | all (4%) | ...
Picked: "to" ... and so on, token by token.

Notice the model didn't have to pick the top candidate every time. It grabbed "refused" over "purred." How boldly a model strays from the safest pick is a dial you can actually turn. It's called temperature. Try it:

At temperature 0, the model always grabs the most likely token, the same answer every time. Crank it up and the model samples from the less-likely candidates too: more surprising, more creative, occasionally unhinged. Same prediction engine, different appetite for risk.

Respect the Autocomplete

So: is "fancy autocomplete" fair? Technically accurate, spiritually incomplete. Yes, the mechanism is next-token prediction. But prediction at this scale produces translation, coding, tutoring, and jokes that occasionally land. The simple mechanism is exactly what makes the results so surprising.

Quick Quiz