Average Over a Tiny Batch - Cross-Entropy, Exactly

A batch loss averages row losses. This lesson keeps each selected probability exact, keeps each log loss named, and averages the named expressions symbolically.

highlighted = computed this step

Two rows in the batch

The tiny batch has row 1 with target B and selected probability 1/2. Row 2 has target A and selected probability 3/4. Each selected probability comes from that row's displayed probability vector.

p_{\text{true,row }1}=1/2,\quad p_{\text{true,row }2}=3/4

Keep each log named

Row 1 contributes -log(1/2). Row 2 contributes -log(3/4). The table names those two losses and does not evaluate either logarithm.

H_1=-\log(1/2),\quad H_2=-\log(3/4)

Average the symbols

The batch loss is the exact average of the two named losses shown in the table. This is only the displayed two-row batch average; no log decimal or broader claim is added.

H_{\text{batch}}=\frac{-\log(1/2)+-\log(3/4)}{2}