Count the high bucket observed fraction exactly.

highlighted = computed this step

Count the high bucket positives

Rows A, B, and C have label 1. Row D has label 0.

1+1+1+0=31 + 1 + 1 + 0 = 3
Calibration Bins ExactlyExact observed fractions inside pinned score buckets.score buckets with exact observed fractionshigh bucketscore=3/4Alabel 1Blabel 1Clabel 1Dlabel 0positives=3rows=4observed=3/4 = 3/4score matches observedlow bucketscore=1/4Elabel 1Flabel 0Glabel 0Hlabel 0positives=1rows=4observed=1/4 = 1/4score matches observedcontrast high bucketscore=3/4Ilabel 1Jlabel 1Klabel 0Llabel 0positives=2rows=4observed=2/4 = 1/2mismatch: 3/4 -> 1/2reliability table: score vs observed fractionbucketscoreobservedstatushigh bucket3/43/4same fractionlow bucket1/41/4same fractioncontrast high bucket3/41/2visible mismatchObserved fraction is positives over rows inside one displayed bucketcontrast is one displayed mismatch onlydoes not claim: NOT probability truth; NOT future calibrationNOT accuracy; NOT fairness; NOT safety; NOT generalizationNOT best model; NOT future performance

Use all four rows as the denominator

The high bucket has 4 rows.

rows in high bucket=4\text{rows in high bucket}=4
Calibration Bins ExactlyExact observed fractions inside pinned score buckets.score buckets with exact observed fractionshigh bucketscore=3/4Alabel 1Blabel 1Clabel 1Dlabel 0positives=3rows=4observed=3/4 = 3/4score matches observedlow bucketscore=1/4Elabel 1Flabel 0Glabel 0Hlabel 0positives=1rows=4observed=1/4 = 1/4score matches observedcontrast high bucketscore=3/4Ilabel 1Jlabel 1Klabel 0Llabel 0positives=2rows=4observed=2/4 = 1/2mismatch: 3/4 -> 1/2reliability table: score vs observed fractionbucketscoreobservedstatushigh bucket3/43/4same fractionlow bucket1/41/4same fractioncontrast high bucket3/41/2visible mismatchObserved fraction is positives over rows inside one displayed bucketcontrast is one displayed mismatch onlydoes not claim: NOT probability truth; NOT future calibrationNOT accuracy; NOT fairness; NOT safety; NOT generalizationNOT best model; NOT future performance

Observed fraction matches the score

The observed positive fraction is 3 over 4, the same fraction as the bucket score.

observed high=34\text{observed high}={3\over4}
Calibration Bins ExactlyExact observed fractions inside pinned score buckets.score buckets with exact observed fractionshigh bucketscore=3/4Alabel 1Blabel 1Clabel 1Dlabel 0positives=3rows=4observed=3/4 = 3/4score matches observedlow bucketscore=1/4Elabel 1Flabel 0Glabel 0Hlabel 0positives=1rows=4observed=1/4 = 1/4score matches observedcontrast high bucketscore=3/4Ilabel 1Jlabel 1Klabel 0Llabel 0positives=2rows=4observed=2/4 = 1/2mismatch: 3/4 -> 1/2reliability table: score vs observed fractionbucketscoreobservedstatushigh bucket3/43/4same fractionlow bucket1/41/4same fractioncontrast high bucket3/41/2visible mismatchObserved fraction is positives over rows inside one displayed bucketcontrast is one displayed mismatch onlydoes not claim: NOT probability truth; NOT future calibrationNOT accuracy; NOT fairness; NOT safety; NOT generalizationNOT best model; NOT future performance