Module 3 · Lesson 8
Large language models generate text by repeatedly predicting what token should come next. The architecture that made this approach scale extraordinarily well is the transformer.
What changed with transformers?
Before transformers, language systems already used recurrent neural networks, LSTMs, sequence-to-sequence models, and earlier forms of attention. They were not simply translating language with rigid word-by-word rules.
The transformer architecture, introduced in 2017, made attention the central mechanism and made training across sequence positions much more parallelizable than recurrent approaches.
Self-attention connects information across the context
For each token representation, the model computes values commonly described as queries, keys, and values. Attention scores determine how strongly one position should incorporate information from other positions.
Consider:
The animal did not cross the street because it was too tired.
When processing “it,” attention can assign strong relevance to “animal.” This is not human understanding in a literal sense. It is a learned mathematical relationship that helps the network build a context-sensitive representation.
Training can be parallel; generation is sequential
This distinction is easy to miss.
During training, a transformer can process many token positions in parallel because the correct sequence is already available. During autoregressive generation, however, the model still produces tokens sequentially:
Prompt
↓
predict token 1
↓
append token 1
↓
predict token 2
↓
append token 2
↓
continue...So it is more accurate to say transformers enable highly parallel computation during training, while autoregressive text generation still unfolds one token after another.
The context window sets a working boundary
The context window is the amount of tokenized information the model can consider in a single interaction. Larger context windows let a model work with more material at once, but a larger window does not guarantee that every detail will be used equally well.
Does next-token prediction create reasoning?
Next-token prediction is the training objective behind many language models, but useful capabilities do not come from that phrase alone. They arise from the combination of large-scale representation learning, architecture, data, optimization, post-training, and sometimes external tools or retrieval.
The transformer is powerful because attention lets the model build rich relationships across context while the training objective gives it an enormous number of opportunities to learn from language.
Demystifying AI series