Held-Out Rows

Score Held-Out Rows

Apply the frozen baseline to the test rows and compute exact errors.

highlighted = computed this step

Score row D

Row D has y=3 and prediction 4, so the absolute error is 1.

∣3−4∣=1|3 - 4| = 1
Train Test Split ExactlyExact train-only statistic and held-out scoring rows.train-only mean and held-out scoringrowroleytrain stat?predictionabs errorAtrain2yes--Btrain4yes--Ctrain6yes--Dtest3held out41Etest7held out43train mean: (2+4+6)/3 = 4test MAE: (1+3)/2 = 2train statistic uses train rows onlytest rows are held out and scored onceNOT training; NOT learning; NOT generalizationNOT future performance; NOT unbiased; NOT safeNOT best split; NOT an accuracy guarantee

Score row E

Row E has y=7 and prediction 4, so the absolute error is 3.

∣7−4∣=3|7 - 4| = 3
Train Test Split ExactlyExact train-only statistic and held-out scoring rows.train-only mean and held-out scoringrowroleytrain stat?predictionabs errorAtrain2yes--Btrain4yes--Ctrain6yes--Dtest3held out41Etest7held out43train mean: (2+4+6)/3 = 4test MAE: (1+3)/2 = 2train statistic uses train rows onlytest rows are held out and scored onceNOT training; NOT learning; NOT generalizationNOT future performance; NOT unbiased; NOT safeNOT best split; NOT an accuracy guarantee

These are held-out scoring rows

The error cells shown for D and E are held-out scoring cells, not train-statistic cells.

held-out scoring rows only\text{held-out scoring rows only}
Train Test Split ExactlyExact train-only statistic and held-out scoring rows.train-only mean and held-out scoringrowroleytrain stat?predictionabs errorAtrain2yes--Btrain4yes--Ctrain6yes--Dtest3held out41Etest7held out43train mean: (2+4+6)/3 = 4test MAE: (1+3)/2 = 2train statistic uses train rows onlytest rows are held out and scored onceNOT training; NOT learning; NOT generalizationNOT future performance; NOT unbiased; NOT safeNOT best split; NOT an accuracy guarantee