Greedy, Top-k, and the Tree

Probabilities

The Named Boundary

Softmax converts logits into probability weights with exponentials. The expression is named instead of decimalized, while argmax and top-k stay exact.

highlighted = computed this step

Probabilities: the named boundary

Softmax converts logits into probability weights using exponentials. The expression is named in the diagram and never pinned as a decimal. This is where decoding crosses from exact ordering into named probability mass.

softmax⁡(ℓ)i=eℓi/∑jeℓj\operatorname{softmax}(\ell)_i=e^{\ell_i}/\sum_j e^{\ell_j}
Softmax named boundarySelection is exact; softmax probabilities are named.Softmax named boundarySelection is exact; softmax probabilities are named.softmax named-boundary tableexponentials and normalized probabilities stay named; no decimal probability is pinnedstep 1 denominator: e^3+e^1+e^2tokenlogite^logitnormalizeda3e^3e^3/(e^3+e^1+e^2)b1e^1e^1/(e^3+e^1+e^2)c2e^2e^2/(e^3+e^1+e^2)shared denominator is derived once from the exact logits

Selection stays exact

Argmax and top-k use integer ordering. Only the probability mass crosses the named softmax boundary. That means the selected token can be exact even though the probability expression remains symbolic.

ordering exact; softmax named\text{ordering exact; softmax named}
Softmax named boundarySelection is exact; softmax probabilities are named.Softmax named boundarySelection is exact; softmax probabilities are named.softmax named-boundary tableexponentials and normalized probabilities stay named; no decimal probability is pinnedstep 1 denominator: e^3+e^1+e^2tokenlogite^logitnormalizeda3e^3e^3/(e^3+e^1+e^2)b1e^1e^1/(e^3+e^1+e^2)c2e^2e^2/(e^3+e^1+e^2)shared denominator is derived once from the exact logits

Summary

Sampling depends on probability mass, so it stays outside this exact page. The separate top-p page uses assigned exact weights only. This softmax page shows no decimal probability weights.

sampling deferred; softmax named\text{sampling deferred; softmax named}
Softmax named boundarySelection is exact; softmax probabilities are named.Softmax named boundarySelection is exact; softmax probabilities are named.softmax named-boundary tableexponentials and normalized probabilities stay named; no decimal probability is pinnedstep 1 denominator: e^3+e^1+e^2tokenlogite^logitnormalizeda3e^3e^3/(e^3+e^1+e^2)b1e^1e^1/(e^3+e^1+e^2)c2e^2e^2/(e^3+e^1+e^2)shared denominator is derived once from the exact logits