Words as Vectors

Similarity by Dot Product

Using dot product as the score gives an exact argmax. The nearest word to cat by this rule is selected from displayed integer dots.

highlighted = computed this step

Similarity by dot product

A larger dot product means the vectors point more in the same direction under this exact rule. Here the query word is cat.

larger u⋅v⇒more aligned by this rule\text{larger }u\cdot v\Rightarrow\text{more aligned by this rule}
Nearest by dot productThe largest exact dot selects the nearest word.Nearest by dot productThe largest exact dot selects the nearest word.exact dot productstie-break: lexicographically smallest word among max dot productsvectors:cat=(3,1); kitten=(2,1); car=(1,3)pairdotcat·kitten7cat·car6kitten·car5nearest to cat by dot=kitten (dot=7)

Nearest to cat

Against cat, kitten has dot 7 and car has dot 6. Since 7 is larger than 6, the nearest word by dot is kitten.

argmax⁡{7,6}=kitten\operatorname{argmax}\{7,6\}=\text{kitten}
Nearest by dot productThe largest exact dot selects the nearest word.Nearest by dot productThe largest exact dot selects the nearest word.exact dot productstie-break: lexicographically smallest word among max dot productsvectors:cat=(3,1); kitten=(2,1); car=(1,3)pairdotcat·kitten7cat·car6kitten·car5nearest to cat by dot=kitten (dot=7)

Summary

If two candidates tied, the table would choose the lexicographically smallest word. No tie occurs here, so the exact argmax is kitten.

tie-break: lexicographic; here 7>6\text{tie-break: lexicographic; here }7>6
Nearest by dot productThe largest exact dot selects the nearest word.Nearest by dot productThe largest exact dot selects the nearest word.exact dot productstie-break: lexicographically smallest word among max dot productsvectors:cat=(3,1); kitten=(2,1); car=(1,3)pairdotcat·kitten7cat·car6kitten·car5nearest to cat by dot=kitten (dot=7)