Cross-Entropy and the Softmax Boundary

Cross-entropy usually consumes probabilities produced by softmax. This book keeps softmax named and starts from a displayed exact probability vector.

highlighted = computed this step

Probabilities come from softmax

In the attention and decoding books, softmax was already the named boundary that turns logits into probabilities. Cross-entropy starts after that boundary: it consumes a pinned probability distribution and applies a named log to the selected entry.

\text{softmax named}\rightarrow p\rightarrow -\log(p_k)\text{ named}

The log-loss link

The logistic scoring book used the same structure: choose the class term, then name the logarithm instead of pinning a decimal. Cross-entropy is the multi-class version of that same honesty split.

\text{selected probability}\rightarrow\text{named log loss}

Summary

This book does not compute softmax as exact. It assumes the probability vector is the displayed exact input, then names the logarithm where the next boundary appears.

\text{probabilities displayed; log named}