Logistic Classification

Log Loss (Small Example)

Compute log loss = −(1/n)Σ(yᵢ·ln(pᵢ) + (1−yᵢ)·ln(1−pᵢ)) for true binary labels y and predicted probabilities p. A loop applies math.log per sample. Library: sklearn.metrics.log_loss(y_true, p_pred, labels=[0,1]). RESULT: log loss (rounded).

By hand

With scikit-learn

log_loss(y_true, p_pred, labels=[0,1]) uses the same natural-log formula. labels=[0,1] is passed explicitly to fix class ordering for the binary case.

naive.py 
import math
y_true = [1, 0, 1, 0, 1]
p_pred = [0.9, 0.2, 0.7, 0.3, 0.8]
n = len(y_true)
total = 0.0
for i in range(n):
    yi = y_true[i]
    pi = p_pred[i]
    total = total - (yi * math.log(pi) + (1 - yi) * math.log(1 - pi))
loss = total / n
print('RESULT:', round(loss, 4))
library.py 
from sklearn.metrics import log_loss
from dalib.display import set_display
set_display()

y_true = [1, 0, 1, 0, 1]
p_pred = [0.9, 0.2, 0.7, 0.3, 0.8]
loss = float(log_loss(y_true, p_pred, labels=[0, 1]))
print('y_true:', y_true)
print('p_pred:', p_pred)
print('RESULT:', round(loss, 4))

y_true: [1, 0, 1, 0, 1]
p_pred: [0.9, 0.2, 0.7, 0.3, 0.8]
RESULT: 0.253

Implementation notes

  • Log loss penalises confident wrong predictions heavily: predicting p=0.01 for a true positive adds −ln(0.01)≈4.6 to the sum vs −ln(0.9)≈0.1.
  • Natural log (base e), not log₂ or log₁₀. sklearn uses the same convention.
  • sklearn clips probabilities by default (eps≈1e-15) to avoid log(0). Choosing p in (0,1) open — here [0.2,0.9] — means no clipping occurs and naive math.log matches sklearn exactly.
  • For y=1 the term reduces to −ln(p); for y=0 it reduces to −ln(1−p). The formula handles both in one expression via the y/1−y multipliers.
  • Lower log loss is better. A model predicting 0.5 everywhere gives log loss = ln(2) ≈ 0.693 regardless of labels — the random baseline.
  • Cross-reference: sigmoid-function (this chapter) produces the probabilities that log loss evaluates.