How LLMs Generate Text: Transformers, Attention and Next-Token Prediction

A common objection goes: "If it only predicts the next word, how can it write working code?" The honest answer is that the phrase hides an enormous amount of structure. Predicting the next token well, across a huge range of text, forces a model to capture grammar, facts, formats and patterns of reasoning. This lesson builds up from a version of next-token prediction small enough to run in your head.

One step: predict the next token

A language model is a function that takes the tokens so far and returns a probability for every token in its vocabulary being next. That is all. Text is produced by running the function, choosing one token, appending it to the input and running it again.

The smallest possible version is a bigram model: count which word follows which in a text, and read probabilities off the counts. It looks only at the previous word. A large model replaces the count table with a neural network that looks at the entire context.

from collections import Counter, defaultdict

corpus = (
    "the keeper lit the lamp at dusk . "
    "the keeper climbed the stairs . "
    "the lamp was bright . "
    "the sailor saw the lamp ."
).split()

# Count which word follows which: a bigram model, the simplest next-token predictor.
follows = defaultdict(Counter)
for prev, nxt in zip(corpus, corpus[1:]):
    follows[prev][nxt] += 1

def next_token_probs(prev):
    counts = follows[prev]
    total = sum(counts.values())
    return {tok: round(n / total, 3) for tok, n in counts.most_common()}

print("after 'the':", next_token_probs("the"))
print("after 'lamp':", next_token_probs("lamp"))

# Generate greedily: always take the most likely next token.
tok, out = "the", ["the"]
for _ in range(8):
    tok = max(follows[tok], key=follows[tok].get)
    out.append(tok)
print(" ".join(out))

after 'the': {'lamp': 0.429, 'keeper': 0.286, 'stairs': 0.143, 'sailor': 0.143}
after 'lamp': {'at': 0.333, 'was': 0.333, '.': 0.333}
the lamp at dusk . the lamp at dusk

The last line is a classic failure of greedy decoding: always taking the single most likely token walks straight into a loop, because the most likely word after "the" is "lamp", and after "lamp" the first tie wins. A bigram has almost no memory, so it cannot tell it has said this before. Big models see far more context, which makes loops rarer, but greedy choices can still produce dull, repetitive text. Lesson three covers how sampling avoids that.

What happens inside the big version

A transformer is the neural-network design behind most current language models. Roughly, each token is turned into a vector of numbers, passed through a stack of identical blocks, and the final result is turned into one score per vocabulary token. Softmax converts scores into probabilities.

  text -> tokens -> vectors (one per token)
                          |
               +----------v-----------+
               |  block, repeated N x |
               |   1. attention       |
               |   2. feed-forward    |
               +----------+-----------+
                          |
          one score per vocabulary token
                          |
                softmax -> probabilities
                          |
             pick one token (sampling)
                          |
        append it to the input and repeat

Attention in miniature

Inside each block, attention lets every token look at the others and pull in information from the ones that matter. Take the sentence "the keeper lit the lamp so it glowed". To process "it", the model needs to work out that "it" refers to the lamp, not the keeper. Attention does this by comparing a query from "it" with a key from each earlier token, using a dot product, and turning the scores into weights that sum to one. The values of the heavily weighted tokens are then blended into the representation of "it".

The vectors below are made up by hand so you can follow the arithmetic. They are not weights from any real model; in a real one, training discovers numbers that behave this way.

import math

def softmax(scores):
    peak = max(scores)                       # subtract the max for numerical stability
    exps = [math.exp(s - peak) for s in scores]
    total = sum(exps)
    return [e / total for e in exps]

def dot(a, b):
    return sum(x * y for x, y in zip(a, b))

# Sentence: "the keeper lit the lamp so it glowed"
# We look at the token "it" and ask how much it should attend to each earlier token.
# The 3-number vectors are hand-made, purely to show the arithmetic. A real model
# learns thousands of numbers per token during training.
tokens = ["the", "keeper", "lit", "the", "lamp", "so", "it"]
keys = [
    [0.1, 0.0, 0.1],   # the
    [0.9, 0.0, 0.2],   # keeper  (a person)
    [0.1, 0.6, 0.0],   # lit     (an action)
    [0.1, 0.0, 0.1],   # the
    [0.3, 0.1, 1.0],   # lamp    (a thing that can glow)
    [0.0, 0.2, 0.1],   # so
    [0.2, 0.1, 0.4],   # it
]
query_for_it = [0.2, 0.1, 1.5]               # "it" is looking for a thing, not a person

scores = [dot(query_for_it, k) for k in keys]
weights = softmax(scores)
for tok, w in zip(tokens, weights):
    print(f"{tok:7} {w:.2f} {'#' * round(w * 40)}")
print("sum of weights:", round(sum(weights), 6))

the     0.09 ####
keeper  0.12 #####
lit     0.08 ###
the     0.09 ####
lamp    0.37 ###############
so      0.09 ####
it      0.15 ######
sum of weights: 1.0

"lamp" gets the largest weight because its key lines up with what the query for "it" is looking for. Real models do this with many attention heads in parallel, each learning to track a different kind of relationship (who did what, which noun a pronoun points to, which bracket is still open). Because every token is compared with every earlier token, the work in basic attention grows with the square of the input length, which is one reason very long contexts are expensive.

What training gave the model, and what it did not

In pretraining, the model reads a very large amount of text and is repeatedly asked to predict the next token, adjusting its internal numbers (the weights) to do better. Afterwards, further training steps (instruction tuning and learning from human preferences, among others) shape it to follow requests and respond helpfully. What is left is a set of weights, not a database. There is no table of facts to look up, which explains a lot of behaviour.

Common mistakes and the symptom each one causes

• Treating the model as a search engine or database. Symptom: confident answers about obscure facts that turn out to be invented. Fix: supply the source text, or add a retrieval step.
• Expecting identical outputs. Symptom: a test passes on Monday and fails on Tuesday with the same prompt. Fix: tolerate variation, lower randomness where the API allows it and test on many cases.
• Assuming recent knowledge. Symptom: outdated answers about current versions, prices or events. Fix: give it current data in the prompt.
• Believing the model's self-explanation. Symptom: a tidy justification that does not match what actually drove the answer. Fix: verify claims independently.
• Imagining a loop of lookups. Symptom: wrong debugging assumptions, such as searching for the missing row that "the model must have fetched". Fix: remember it only reads the tokens in front of it.

How this shows up in real projects and interviews

Day to day you will rarely touch attention directly, but the mental model drives design choices: why you pass documents in the prompt, why you validate output, why latency tracks output length (each token needs one more pass through the network). Interview questions in this area are usually conceptual: "explain a transformer at a high level", "what does attention do", "why is the context window limited". A good answer gives the generate-append-repeat loop, the idea of query, key and weight, and the quadratic cost of basic attention, and avoids claiming more than you know about any specific model's internals. Next up: the sampling step, and where hallucination comes from.