A tiny corpus can make two adjacent pairs tie. This lesson shows the stated lexicographic tie-break as an exact deterministic rule.

highlighted = computed this step

A tiny tie corpus

Use a tiny corpus: word ab has frequency 1 and symbols a b; word ac has frequency 1 and symbols a c. Each word contributes one adjacent pair.

ab:a b,ac:a cab:a\ b,\quad ac:a\ c
When pair counts tieThe lexicographic tie-break is visible on a tiny corpus.When pair counts tieThe lexicographic tie-break is visible on a tiny corpus.lexicographic tie-breaktie-break: lexicographically smallest pair among max countstiny tie corpusab freq=1 symbols=a bac freq=1 symbols=a cpair counts(a,b)=1(a,c)=1tied max pairs: (a,b), (a,c)chosen merge(a,b)->ab count=1(a,b) comes before (a,c)one toy corpus and one stated deterministic tie policy; not a claim about learned meaning; not a claim about better tokenization; not a claim about every tokenizer implementation; not a probabilistic claim

The pair counts tie

The pair (a,b) has count 1, and the pair (a,c) also has count 1. Both are maximum counts, so the stated tie policy must decide.

(a,b)=1,(a,c)=1(a,b)=1,\quad (a,c)=1
When pair counts tieThe lexicographic tie-break is visible on a tiny corpus.When pair counts tieThe lexicographic tie-break is visible on a tiny corpus.lexicographic tie-breaktie-break: lexicographically smallest pair among max countstiny tie corpusab freq=1 symbols=a bac freq=1 symbols=a cpair counts(a,b)=1(a,c)=1tied max pairs: (a,b), (a,c)chosen merge(a,b)->ab count=1(a,b) comes before (a,c)one toy corpus and one stated deterministic tie policy; not a claim about learned meaning; not a claim about better tokenization; not a claim about every tokenizer implementation; not a probabilistic claim

Lexicographic tie-break

The stated tie-break chooses the lexicographically smallest pair among max counts. Since (a,b) comes before (a,c), the chosen merge is (a,b) to ab with count 1. This is one toy corpus and one deterministic tie policy; it is not a claim about learned meaning, better tokenization, every tokenizer implementation, or probabilistic behavior.

(a,b)ab,count=1(a,b)\rightarrow ab,\quad \text{count}=1
When pair counts tieThe lexicographic tie-break is visible on a tiny corpus.When pair counts tieThe lexicographic tie-break is visible on a tiny corpus.lexicographic tie-breaktie-break: lexicographically smallest pair among max countstiny tie corpusab freq=1 symbols=a bac freq=1 symbols=a cpair counts(a,b)=1(a,c)=1tied max pairs: (a,b), (a,c)chosen merge(a,b)->ab count=1(a,b) comes before (a,c)one toy corpus and one stated deterministic tie policy; not a claim about learned meaning; not a claim about better tokenization; not a claim about every tokenizer implementation; not a probabilistic claim