Skip to main content
PProfileMatch
Read Play
← Path

Large Language Models · How LLMs Generate Text: Transformers, Attention and Next-Token Prediction

0 XP
  1. 1 current stop: learning
  2. 2 upcoming
  3. 3 upcoming
  4. 4 upcoming
  5. 5 upcoming

Finish after stop 5

Stop 1 of 5 · Starting line: the tipping scales

Trick to keep in mind+3 XP

Not one answer, a whole scoreboard: every token gets a probability.

At each step the network does not name a winner. It produces a probability for every token in its vocabulary. A separate selection step then picks one of them, appends it to the text and the loop runs again. Picking and scoring are two different jobs.

Read it once, then a question tests whether the trick stuck.