Attention
How an AI decides which other words matter most when understanding each word.
In everyday terms
In "The trophy didn't fit in the suitcase because it was too big", attention helps link "it" to "trophy".
For professionals
Each token forms query, key and value vectors. Similarity of queries and keys weights a sum of values. Multiple heads capture different relations.
Think of it like…
A highlighter: for every word, the AI highlights the other words it needs to make sense of it.
You've already seen it
The 2017 paper title "Attention Is All You Need".
Myth vs reality
Myth: Attention means the AI is paying attention like a person.
Reality: It's a weighting calculation, not awareness.
Quick check
In "Sara gave Tom her book", attention helps the model link "her" to…
- Tom
- Sara
- book
- gave
Show answer
Sara: "Her" refers to Sara.