Cross-Entropy, Exactly

Named Log Loss

• What Cross-Entropy Measures
• The One-Hot Target Selects One Term
• Soft Targets Weight Terms
• The Log Is the Named Boundary
• Lower Probability, Higher Loss
• Compare Two Candidate Distributions
• Average Over a Tiny Batch
• Cross-Entropy and the Softmax Boundary
• What Cross-Entropy Is and Is Not