Held-Out Rows

Train Mean Uses Train Rows

Compute the baseline statistic from train rows only.

highlighted = computed this step

Add only train rows

The train sum is 2 + 4 + 6 = 12.

2+4+6=122 + 4 + 6 = 12
Train Test Split ExactlyExact train-only statistic and held-out scoring rows.train-only mean and held-out scoringrowroleytrain stat?predictionabs errorAtrain2yes--Btrain4yes--Ctrain6yes--Dtest3held out41Etest7held out43train mean: (2+4+6)/3 = 4test MAE: (1+3)/2 = 2train statistic uses train rows onlytest rows are held out and scored onceNOT training; NOT learning; NOT generalizationNOT future performance; NOT unbiased; NOT safeNOT best split; NOT an accuracy guarantee

Divide by train count

There are 3 train rows, so the train mean is 12 divided by 3 = 4.

123=4{12\over3} = 4
Train Test Split ExactlyExact train-only statistic and held-out scoring rows.train-only mean and held-out scoringrowroleytrain stat?predictionabs errorAtrain2yes--Btrain4yes--Ctrain6yes--Dtest3held out41Etest7held out43train mean: (2+4+6)/3 = 4test MAE: (1+3)/2 = 2train statistic uses train rows onlytest rows are held out and scored onceNOT training; NOT learning; NOT generalizationNOT future performance; NOT unbiased; NOT safeNOT best split; NOT an accuracy guarantee

Test rows are not in this statistic

Rows D and E do not contribute to the train sum, train count, or train mean.

train statistic uses train rows only\text{train statistic uses train rows only}
Train Test Split ExactlyExact train-only statistic and held-out scoring rows.train-only mean and held-out scoringrowroleytrain stat?predictionabs errorAtrain2yes--Btrain4yes--Ctrain6yes--Dtest3held out41Etest7held out43train mean: (2+4+6)/3 = 4test MAE: (1+3)/2 = 2train statistic uses train rows onlytest rows are held out and scored onceNOT training; NOT learning; NOT generalizationNOT future performance; NOT unbiased; NOT safeNOT best split; NOT an accuracy guarantee