Cross-entropy compares a one-hot target with a probability distribution. The inputs in this book are exact rational probabilities; the logarithm will be named.
highlighted = computed this step
A distribution and a target
Cross-entropy compares a probability distribution with a one-hot target. Here the model distribution is p=(1/4,1/2,1/4), and the target is y=(0,1,0). All probabilities are exact rationals and sum to 1.
p=(1/4,1/2,1/4),y=(0,1,0)
The loss formula
The formula is H equals minus the sum of y_i times log of p_i. The sum is structural: each class contributes its target entry times the log term. The displayed table recomputes which term remains.
H=−i∑yilog(pi)
Summary
The exact part is the pinned distribution and one-hot target. The logarithm is not evaluated; it will be named at the boundary instead of decimalized.