Attention, Structurally

Scores, Masks, Softmax

• Attention Scores
• The Causal Mask
• Softmax
• Value-Weighting Walkthrough
• Assigned Weights Are Not Softmax
• Weighting the Values
• The Attention Grid
• What Attention Is and Is Not
• Where Softmax Is a Real Number