Evaluation Metrics

Confusion Matrix Counts

Count TP, FP, FN, TN from binary true/predicted label lists. Loop over pairs: TP=true=1,pred=1; FP=true=0,pred=1; FN=true=1,pred=0; TN=true=0,pred=0. Library: sklearn confusion_matrix(labels=[0,1]) returns [[TN,FP],[FN,TP]]; extract TP=cm[1][1], FP=cm[0][1], FN=cm[1][0], TN=cm[0][0]. RESULT: (TP, FP, FN, TN) tuple.

Learning path

Prerequisite: Threshold Probabilities.

By hand

y_true=[1,0,1,1,0,1], y_pred=[1,1,0,1,0,0]. TP=2 (i=0,3); FP=1 (i=1); FN=2 (i=2,5); TN=1 (i=4).

naive.py 
y_true = [1, 0, 1, 1, 0, 1]
y_pred = [1, 1, 0, 1, 0, 0]
tp = 0
fp = 0
fn = 0
tn = 0
for i in range(len(y_true)):
    t = y_true[i]
    p = y_pred[i]
    if t == 1 and p == 1:
        tp = tp + 1
    elif t == 0 and p == 1:
        fp = fp + 1
    elif t == 1 and p == 0:
        fn = fn + 1
    else:
        tn = tn + 1
print('RESULT:', (tp, fp, fn, tn))

With scikit-learn

confusion_matrix(y_true, y_pred, labels=[0,1]) returns a 2×2 matrix with layout [[TN,FP],[FN,TP]]: rows index true class (0 then 1), columns index predicted class (0 then 1).

library.py 
from sklearn.metrics import confusion_matrix
from dalib.display import set_display
set_display()

y_true = [1, 0, 1, 1, 0, 1]
y_pred = [1, 1, 0, 1, 0, 0]
cm = confusion_matrix(y_true, y_pred, labels=[0, 1])
print('cm:', cm.tolist())
tn, fp, fn, tp = int(cm[0][0]), int(cm[0][1]), int(cm[1][0]), int(cm[1][1])
print('RESULT:', (tp, fp, fn, tn))

cm: [[1, 1], [2, 2]]
RESULT: (2, 1, 2, 1)

Implementation notes

  • sklearn's layout [[TN,FP],[FN,TP]] indexes by (true_class, pred_class). With labels=[0,1]: cm[0][0]=TN, cm[0][1]=FP, cm[1][0]=FN, cm[1][1]=TP. Passing labels explicitly pins the row/column ordering regardless of which classes appear in the data.
  • TP and TN are correct predictions; FP is a false alarm (predicted positive, actually negative); FN is a miss (predicted negative, actually positive).
  • Accuracy = (TP+TN)/n; see Accuracy Score. Precision, Recall, and F1 covers metrics that handle class imbalance better than accuracy.
  • Threshold Probabilities shows how raising the decision threshold lowers FP at the cost of raising FN.