Evaluation Metrics

Accuracy Score

Compute accuracy: fraction of predictions that match the true labels. Loop over index, count matches where y_true[i]==y_pred[i], divide by n. Library: sklearn.metrics.accuracy_score(y_true, y_pred). RESULT: accuracy (rounded).

By hand

With scikit-learn

accuracy_score(y_true, y_pred) returns the fraction of matching pairs as a float.

naive.py 
y_true = [1, 0, 1, 1, 0, 0, 1, 0]
y_pred = [1, 1, 0, 1, 0, 0, 0, 0]
n = len(y_true)
correct = 0
for i in range(n):
    if y_true[i] == y_pred[i]:
        correct = correct + 1
accuracy = correct / n
print('RESULT:', round(accuracy, 4))
library.py 
from sklearn.metrics import accuracy_score
from dalib.display import set_display
set_display()

y_true = [1, 0, 1, 1, 0, 0, 1, 0]
y_pred = [1, 1, 0, 1, 0, 0, 0, 0]
print('correct:', sum(t == p for t, p in zip(y_true, y_pred)))
acc = accuracy_score(y_true, y_pred)
print('RESULT:', round(float(acc), 4))

correct: 5
RESULT: 0.625

Implementation notes

  • Accuracy = (TP+TN)/n = (2+3)/8 = 0.625; cross-reference confusion-matrix-counts (this chapter) for the component counts.
  • On balanced data accuracy is intuitive, but on imbalanced data a classifier predicting only the majority class can still score high — motivating precision, recall, and F1 as complementary metrics.