Evaluation Metrics

Precision, Recall, and F1

Compute precision, recall, and F1 from given TP, FP, FN counts. Precision = TP/(TP+FP); recall = TP/(TP+FN); F1 = 2PR/(P+R). Library: sklearn precision_score, recall_score, f1_score on the same y_true/y_pred. RESULT: (precision, recall, f1) rounded.

Learning path

Prerequisite: Confusion Matrix Counts.

By hand

TP=2, FP=1, FN=2 (from Confusion Matrix Counts, same data). precision=2/(2+1)=0.6667; recall=2/(2+2)=0.5; F1=20.66670.5/(0.6667+0.5)=4/7=0.5714.

naive.py
Replay: real traced execution (multi-file project)
tp = 2
fp = 1
fn = 2
precision = tp / (tp + fp)
recall = tp / (tp + fn)
f1 = 2 * precision * recall / (precision + recall)
print('RESULT:', (round(precision, 4), round(recall, 4), round(f1, 4)))

  1. tp ← 2

    1tp = 22fp = 1
    values this step2tp

  2. fp ← 1

    1tp = 22fp = 13fn = 2
    values this step1fp

  3. fn ← 2

    2fp = 13fn = 24precision = tp / (tp + fp)
    values this step2fn

  4. precision ← 0.6666666666666666

    3fn = 24precision = tp / (tp + fp)5recall = tp / (tp + fn)
    values this step0.6666666666666666precision

  5. recall ← 0.5

    4precision = tp / (tp + fp)5recall = tp / (tp + fn)6f1 = 2 * precision * recall / (precision + recall)
    values this step0.5recall

  6. f1 ← 0.5714285714285715

    5recall = tp / (tp + fn)6f1 = 2 * precision * recall / (precision + recall)7print('RESULT:', (round(precision, 4), round(recall, 4), round(f1, 4)))
    values this step0.5714285714285715f1

  7. stdout ← RESULT: (0.6667, 0.5, 0.5714)

    6f1 = 2 * precision * recall / (precision + recall)7print('RESULT:', (round(precision, 4), round(recall, 4), round(f1, 4)))
    values this stepRESULT: (0.6667, 0.5, 0.5714)stdout

With scikit-learn

precision_score, recall_score, f1_score each take y_true and y_pred; default average='binary' treats label 1 as positive.

library.py 
from sklearn.metrics import precision_score, recall_score, f1_score
from dalib.display import set_display
set_display()

y_true = [1, 0, 1, 1, 0, 1]
y_pred = [1, 1, 0, 1, 0, 0]
p = round(float(precision_score(y_true, y_pred)), 4)
r = round(float(recall_score(y_true, y_pred)), 4)
f = round(float(f1_score(y_true, y_pred)), 4)
print('RESULT:', (p, r, f))

RESULT: (0.6667, 0.5, 0.5714)

Implementation notes

  • F1 is computed from the raw (unrounded) precision and recall so floating- point rounding doesn't compound. The rounded values appear only in the final print.
  • Precision answers "of all predicted positives, how many were correct?"; recall answers "of all actual positives, how many did we find?". F1 is their harmonic mean — harmonic mean punishes a metric that is very high on one axis and very low on the other, more than the arithmetic mean would.
  • Accuracy = (TP+TN)/n = 3/6 = 0.5 on this data (lower than F1 = 0.57), showing that accuracy can understate classifier performance when classes are imbalanced. Compare with Accuracy Score and Confusion Matrix Counts.