Greedy, Top-k, and the Tree

Greedy Decoding

Greedy decoding takes the largest logit at each step. The rule is deterministic because the tie-break is fixed.

highlighted = computed this step

Greedy decoding

Greedy decoding picks the largest displayed logit at each step. Equal logits use the lowest vocab index as the deterministic tie-break.

argmax⁡ with lowest-index tie-break\operatorname{argmax}\text{ with lowest-index tie-break}
Greedy selectionsEach step chooses the exact argmax.Greedy selectionsEach step chooses the exact argmax.exact greedy decode stepdisplayed integer logits are the source; ties break by lowest vocab indexstep 1 logits: a=3, b=1, c=2tokenlogitranka31b13c22ranked order: a > c > bgreedy argmax: agreedy selection uses exact ordering; the branch list is not shown in this viewexact greedy decode stepdisplayed integer logits are the source; ties break by lowest vocab indexstep 2 logits: a=0, b=4, c=1tokenlogitranka03b41c12ranked order: b > c > agreedy argmax: bgreedy selection uses exact ordering; the branch list is not shown in this view

Two exact choices

The first step chooses a from logit 3, and the second step chooses b from logit 4.

step one→a,step two→b\text{step one}\rightarrow a,\quad \text{step two}\rightarrow b
Greedy selectionsEach step chooses the exact argmax.Greedy selectionsEach step chooses the exact argmax.exact greedy decode stepdisplayed integer logits are the source; ties break by lowest vocab indexstep 1 logits: a=3, b=1, c=2tokenlogitranka31b13c22ranked order: a > c > bgreedy argmax: agreedy selection uses exact ordering; the branch list is not shown in this viewexact greedy decode stepdisplayed integer logits are the source; ties break by lowest vocab indexstep 2 logits: a=0, b=4, c=1tokenlogitranka03b41c12ranked order: b > c > agreedy argmax: bgreedy selection uses exact ordering; the branch list is not shown in this view

Summary

The greedy output is a, b. The output comes from exact integer ordering, not from a probability calculation.

greedy decode=a,b\text{greedy decode}=a,b
Greedy selectionsEach step chooses the exact argmax.Greedy selectionsEach step chooses the exact argmax.exact greedy decode stepdisplayed integer logits are the source; ties break by lowest vocab indexstep 1 logits: a=3, b=1, c=2tokenlogitranka31b13c22ranked order: a > c > bgreedy argmax: agreedy selection uses exact ordering; the branch list is not shown in this viewexact greedy decode stepdisplayed integer logits are the source; ties break by lowest vocab indexstep 2 logits: a=0, b=4, c=1tokenlogitranka03b41c12ranked order: b > c > agreedy argmax: bgreedy selection uses exact ordering; the branch list is not shown in this view