Decoding

Greedy, Top-k, and the Tree

• From Logits to Tokens
• Logit Gaps Before Probabilities
• Greedy Decoding
• Top-k
• Top-p Takes a Prefix
• Probabilities
• The Decode Tree
• What Decoding Is and Is Not