Evaluation Metrics

Mistake Costs Change the Choice

Assigned mistake costs can rank the same threshold rows differently from a plain summary column. This lesson pins the costs first, then recomputes exact totals from visible FP and FN counts.

highlighted = computed this step

Assign costs first

Before choosing from the same threshold rows, pin the toy costs. A false positive costs 4, and a false negative costs 1. The table recomputes every total from the visible FP and FN counts.

cost⁡=4⋅FP+1⋅FN\operatorname{cost}=4\cdot FP + 1\cdot FN
Mistake costs change the choiceExact assigned-cost comparison for the listed thresholds.assigned mistake cost tablethresholdFPFNF1costformulatotalcostchoice5021/24*0+1*22choose3112/34*1+1*15skip1203/44*2+1*08skipFP cost=4, FN cost=1; cost = FP cost*FP + FN cost*FNlowest assigned cost chooses threshold 5assigned costs for this toy setting; not a universalmetric; not a claim about future rows; NOT training; NOTgeneralizationthreshold fixed first; metric alone is not a claim; NOT training

Threshold 5 has low assigned cost

At threshold 5, FP=0 and FN=2. The assigned total is 2.

t=5:4⋅0+1⋅2=2t=5:\quad 4\cdot0 + 1\cdot2 = 2
Mistake costs change the choiceExact assigned-cost comparison for the listed thresholds.assigned mistake cost tablethresholdFPFNF1costformulatotalcostchoice5021/24*0+1*22choose3112/34*1+1*15skip1203/44*2+1*08skipFP cost=4, FN cost=1; cost = FP cost*FP + FN cost*FNlowest assigned cost chooses threshold 5assigned costs for this toy setting; not a universalmetric; not a claim about future rows; NOT training; NOTgeneralizationthreshold fixed first; metric alone is not a claim; NOT training

The other totals are larger

Threshold 3 has assigned total 5. Threshold 1 has assigned total 8 even though its F1 summary is 3/4.

t=3⇒5,t=1⇒8,F1(t=1)=3/4t=3\Rightarrow 5,\quad t=1\Rightarrow 8,\quad F_{1}(t=1)=3/4
Mistake costs change the choiceExact assigned-cost comparison for the listed thresholds.assigned mistake cost tablethresholdFPFNF1costformulatotalcostchoice5021/24*0+1*22choose3112/34*1+1*15skip1203/44*2+1*08skipFP cost=4, FN cost=1; cost = FP cost*FP + FN cost*FNlowest assigned cost chooses threshold 5assigned costs for this toy setting; not a universalmetric; not a claim about future rows; NOT training; NOTgeneralizationthreshold fixed first; metric alone is not a claim; NOT training

Cost is a stated policy

With these assigned toy costs, the lowest total chooses threshold 5. This is an assigned-cost table for this toy setting, not a universal metric and not a claim about future rows.

min⁡{2,5,8}=2\min\{2,5,8\}=2
Mistake costs change the choiceExact assigned-cost comparison for the listed thresholds.assigned mistake cost tablethresholdFPFNF1costformulatotalcostchoice5021/24*0+1*22choose3112/34*1+1*15skip1203/44*2+1*08skipFP cost=4, FN cost=1; cost = FP cost*FP + FN cost*FNlowest assigned cost chooses threshold 5assigned costs for this toy setting; not a universalmetric; not a claim about future rows; NOT training; NOTgeneralizationthreshold fixed first; metric alone is not a claim; NOT training