Accuracy Can Hide

Confusion Counts Show the Miss

Count TP, FN, FP, and TN with P as the positive class.

highlighted = computed this step

Count true positives

Treat P as the positive class. There are 0 true positives because no row is marked P.

TP=0TP=0
Class Imbalance ExactlyExact row counts for one always-N baseline.label countslabelcountN4P1always-N baseline on one imbalanced tablerowtruepredcorrect?cellANNyesTNBNNyesTNCNNyesTNDNNyesTNEPNmissFNconfusion counts with P as positivecountvalueTP0FN1FP0TN4metric fractionsmetricfractionaccuracy4/5positive recall0/1 = 0accuracy can be high while the lone P is missedone pinned imbalanced table; one always-N baselinethe lone P is missed; this is exact row countingdoes not claim: NOT fairness solved; NOT future performanceNOT calibrated; NOT safe; NOT best metric; NOT accuracy guaranteeNOT generalization; NOT probability truth

Count false negatives

The P row is marked N, so FN is 1.

FN=1FN=1
Class Imbalance ExactlyExact row counts for one always-N baseline.label countslabelcountN4P1always-N baseline on one imbalanced tablerowtruepredcorrect?cellANNyesTNBNNyesTNCNNyesTNDNNyesTNEPNmissFNconfusion counts with P as positivecountvalueTP0FN1FP0TN4metric fractionsmetricfractionaccuracy4/5positive recall0/1 = 0accuracy can be high while the lone P is missedone pinned imbalanced table; one always-N baselinethe lone P is missed; this is exact row countingdoes not claim: NOT fairness solved; NOT future performanceNOT calibrated; NOT safe; NOT best metric; NOT accuracy guaranteeNOT generalization; NOT probability truth

Count the N side

No N row is marked P, so FP is 0 and TN is 4.

FP=0,TN=4FP=0,\quad TN=4
Class Imbalance ExactlyExact row counts for one always-N baseline.label countslabelcountN4P1always-N baseline on one imbalanced tablerowtruepredcorrect?cellANNyesTNBNNyesTNCNNyesTNDNNyesTNEPNmissFNconfusion counts with P as positivecountvalueTP0FN1FP0TN4metric fractionsmetricfractionaccuracy4/5positive recall0/1 = 0accuracy can be high while the lone P is missedone pinned imbalanced table; one always-N baselinethe lone P is missed; this is exact row countingdoes not claim: NOT fairness solved; NOT future performanceNOT calibrated; NOT safe; NOT best metric; NOT accuracy guaranteeNOT generalization; NOT probability truth