Count the low bucket observed fraction exactly.

highlighted = computed this step

Count the low bucket positives

Only row E has label 1. Rows F, G, and H have label 0.

1+0+0+0=11 + 0 + 0 + 0 = 1
Calibration Bins ExactlyExact observed fractions inside pinned score buckets.score buckets with exact observed fractionshigh bucketscore=3/4Alabel 1Blabel 1Clabel 1Dlabel 0positives=3rows=4observed=3/4 = 3/4score matches observedlow bucketscore=1/4Elabel 1Flabel 0Glabel 0Hlabel 0positives=1rows=4observed=1/4 = 1/4score matches observedcontrast high bucketscore=3/4Ilabel 1Jlabel 1Klabel 0Llabel 0positives=2rows=4observed=2/4 = 1/2mismatch: 3/4 -> 1/2reliability table: score vs observed fractionbucketscoreobservedstatushigh bucket3/43/4same fractionlow bucket1/41/4same fractioncontrast high bucket3/41/2visible mismatchObserved fraction is positives over rows inside one displayed bucketcontrast is one displayed mismatch onlydoes not claim: NOT probability truth; NOT future calibrationNOT accuracy; NOT fairness; NOT safety; NOT generalizationNOT best model; NOT future performance

Use all four low-bucket rows

The low bucket also has 4 rows.

rows in low bucket=4\text{rows in low bucket}=4
Calibration Bins ExactlyExact observed fractions inside pinned score buckets.score buckets with exact observed fractionshigh bucketscore=3/4Alabel 1Blabel 1Clabel 1Dlabel 0positives=3rows=4observed=3/4 = 3/4score matches observedlow bucketscore=1/4Elabel 1Flabel 0Glabel 0Hlabel 0positives=1rows=4observed=1/4 = 1/4score matches observedcontrast high bucketscore=3/4Ilabel 1Jlabel 1Klabel 0Llabel 0positives=2rows=4observed=2/4 = 1/2mismatch: 3/4 -> 1/2reliability table: score vs observed fractionbucketscoreobservedstatushigh bucket3/43/4same fractionlow bucket1/41/4same fractioncontrast high bucket3/41/2visible mismatchObserved fraction is positives over rows inside one displayed bucketcontrast is one displayed mismatch onlydoes not claim: NOT probability truth; NOT future calibrationNOT accuracy; NOT fairness; NOT safety; NOT generalizationNOT best model; NOT future performance

Observed fraction matches the low score

The observed positive fraction is 1 over 4.

observed low=14\text{observed low}={1\over4}
Calibration Bins ExactlyExact observed fractions inside pinned score buckets.score buckets with exact observed fractionshigh bucketscore=3/4Alabel 1Blabel 1Clabel 1Dlabel 0positives=3rows=4observed=3/4 = 3/4score matches observedlow bucketscore=1/4Elabel 1Flabel 0Glabel 0Hlabel 0positives=1rows=4observed=1/4 = 1/4score matches observedcontrast high bucketscore=3/4Ilabel 1Jlabel 1Klabel 0Llabel 0positives=2rows=4observed=2/4 = 1/2mismatch: 3/4 -> 1/2reliability table: score vs observed fractionbucketscoreobservedstatushigh bucket3/43/4same fractionlow bucket1/41/4same fractioncontrast high bucket3/41/2visible mismatchObserved fraction is positives over rows inside one displayed bucketcontrast is one displayed mismatch onlydoes not claim: NOT probability truth; NOT future calibrationNOT accuracy; NOT fairness; NOT safety; NOT generalizationNOT best model; NOT future performance