Evaluation Metrics

Accuracy Score

Compute accuracy: fraction of predictions that match the true labels. Loop over index, count matches where y_true[i]==y_pred[i], divide by n. Library: sklearn.metrics.accuracy_score(y_true, y_pred). RESULT: accuracy (rounded).

Learning path

Prerequisite: Confusion Matrix Counts.

By hand

y_true=[1,0,1,1,0,0,1,0], y_pred=[1,1,0,1,0,0,0,0]. Matches at i=0,3,4,5,7 → correct=5, n=8, accuracy=5/8=0.625.

naive.py
Replay: real traced execution (multi-file project)
y_true = [1, 0, 1, 1, 0, 0, 1, 0]
y_pred = [1, 1, 0, 1, 0, 0, 0, 0]
n = len(y_true)
correct = 0
for i in range(n):
    if y_true[i] == y_pred[i]:
        correct = correct + 1
accuracy = correct / n
print('RESULT:', round(accuracy, 4))

  1. y_true ← [1, 0, 1, 1, 0, 0, 1, 0]

    1y_true = [1, 0, 1, 1, 0, 0, 1, 0]2y_pred = [1, 1, 0, 1, 0, 0, 0, 0]
    values this step[1, 0, 1, 1, 0, 0, 1, 0]y_true

  2. y_pred ← [1, 1, 0, 1, 0, 0, 0, 0]

    1y_true = [1, 0, 1, 1, 0, 0, 1, 0]2y_pred = [1, 1, 0, 1, 0, 0, 0, 0]3n = len(y_true)
    values this step[1, 1, 0, 1, 0, 0, 0, 0]y_pred

  3. n ← 8

    2y_pred = [1, 1, 0, 1, 0, 0, 0, 0]3n = len(y_true)4correct = 0
    values this step8n

  4. correct ← 0

    3n = len(y_true)4correct = 05for i in range(n):
    values this step0correct

  5. i ← 0

    4correct = 05for i in range(n):6    if y_true[i] == y_pred[i]:
    values this step0i

  6. if y_true[i] == y_pred[i]:

    5for i in range(n):6    if y_true[i] == y_pred[i]:7        correct = correct + 1

  7. correct ← 1

    6    if y_true[i] == y_pred[i]:7        correct = correct + 18accuracy = correct / n
    values this step0 → 1correct

  8. i ← 1

    4correct = 05for i in range(n):6    if y_true[i] == y_pred[i]:
    values this step0 → 1i

  9. if y_true[i] == y_pred[i]:

    5for i in range(n):6    if y_true[i] == y_pred[i]:7        correct = correct + 1

  10. i ← 2

    4correct = 05for i in range(n):6    if y_true[i] == y_pred[i]:
    values this step1 → 2i

  11. if y_true[i] == y_pred[i]:

    5for i in range(n):6    if y_true[i] == y_pred[i]:7        correct = correct + 1

  12. i ← 3

    4correct = 05for i in range(n):6    if y_true[i] == y_pred[i]:
    values this step2 → 3i

  13. if y_true[i] == y_pred[i]:

    5for i in range(n):6    if y_true[i] == y_pred[i]:7        correct = correct + 1

  14. correct ← 2

    6    if y_true[i] == y_pred[i]:7        correct = correct + 18accuracy = correct / n
    values this step1 → 2correct

  15. i ← 4

    4correct = 05for i in range(n):6    if y_true[i] == y_pred[i]:
    values this step3 → 4i

  16. if y_true[i] == y_pred[i]:

    5for i in range(n):6    if y_true[i] == y_pred[i]:7        correct = correct + 1

  17. correct ← 3

    6    if y_true[i] == y_pred[i]:7        correct = correct + 18accuracy = correct / n
    values this step2 → 3correct

  18. i ← 5

    4correct = 05for i in range(n):6    if y_true[i] == y_pred[i]:
    values this step4 → 5i

  19. if y_true[i] == y_pred[i]:

    5for i in range(n):6    if y_true[i] == y_pred[i]:7        correct = correct + 1

  20. correct ← 4

    6    if y_true[i] == y_pred[i]:7        correct = correct + 18accuracy = correct / n
    values this step3 → 4correct

  21. i ← 6

    4correct = 05for i in range(n):6    if y_true[i] == y_pred[i]:
    values this step5 → 6i

  22. if y_true[i] == y_pred[i]:

    5for i in range(n):6    if y_true[i] == y_pred[i]:7        correct = correct + 1

  23. i ← 7

    4correct = 05for i in range(n):6    if y_true[i] == y_pred[i]:
    values this step6 → 7i

  24. if y_true[i] == y_pred[i]:

    5for i in range(n):6    if y_true[i] == y_pred[i]:7        correct = correct + 1

  25. correct ← 5

    6    if y_true[i] == y_pred[i]:7        correct = correct + 18accuracy = correct / n
    values this step4 → 5correct

  26. for i in range(n):

    4correct = 05for i in range(n):6    if y_true[i] == y_pred[i]:

  27. accuracy ← 0.625

    7        correct = correct + 18accuracy = correct / n9print('RESULT:', round(accuracy, 4))
    values this step0.625accuracy

  28. stdout ← RESULT: 0.625

    8accuracy = correct / n9print('RESULT:', round(accuracy, 4))
    values this stepRESULT: 0.625stdout

With scikit-learn

accuracy_score(y_true, y_pred) returns the fraction of matching pairs as a float.

library.py 
from sklearn.metrics import accuracy_score
from dalib.display import set_display
set_display()

y_true = [1, 0, 1, 1, 0, 0, 1, 0]
y_pred = [1, 1, 0, 1, 0, 0, 0, 0]
print('correct:', sum(t == p for t, p in zip(y_true, y_pred)))
acc = accuracy_score(y_true, y_pred)
print('RESULT:', round(float(acc), 4))

correct: 5
RESULT: 0.625

Implementation notes

  • Accuracy = (TP+TN)/n = (2+3)/8 = 0.625; see Confusion Matrix Counts for the component counts.
  • On balanced data accuracy is intuitive, but on imbalanced data a classifier predicting only the majority class can still score high — motivating precision, recall, and F1 as complementary metrics.