Held-Out Rows

The Role Column

Start by making train and test row roles visible.

highlighted = computed this step

Show the role column

Rows A, B, and C are marked train. Rows D and E are marked test. The role column is part of the displayed data.

A,B,C: trainD,E: testA,B,C:\text{ train}\quad D,E:\text{ test}
Train Test Split ExactlyExact train-only statistic and held-out scoring rows.train-only mean and held-out scoringrowroleytrain stat?predictionabs errorAtrain2yes--Btrain4yes--Ctrain6yes--Dtest3held out41Etest7held out43train mean: (2+4+6)/3 = 4test MAE: (1+3)/2 = 2train statistic uses train rows onlytest rows are held out and scored onceNOT training; NOT learning; NOT generalizationNOT future performance; NOT unbiased; NOT safeNOT best split; NOT an accuracy guarantee

Read the train y values

The train y values are 2, 4, and 6.

ytrain=(2,4,6)y_{\text{train}}=(2,4,6)
Train Test Split ExactlyExact train-only statistic and held-out scoring rows.train-only mean and held-out scoringrowroleytrain stat?predictionabs errorAtrain2yes--Btrain4yes--Ctrain6yes--Dtest3held out41Etest7held out43train mean: (2+4+6)/3 = 4test MAE: (1+3)/2 = 2train statistic uses train rows onlytest rows are held out and scored onceNOT training; NOT learning; NOT generalizationNOT future performance; NOT unbiased; NOT safeNOT best split; NOT an accuracy guarantee

Keep test rows held out

The test y values are 3 and 7. They are visible, but they are not used in the train statistic.

ytest=(3,7)y_{\text{test}}=(3,7)
Train Test Split ExactlyExact train-only statistic and held-out scoring rows.train-only mean and held-out scoringrowroleytrain stat?predictionabs errorAtrain2yes--Btrain4yes--Ctrain6yes--Dtest3held out41Etest7held out43train mean: (2+4+6)/3 = 4test MAE: (1+3)/2 = 2train statistic uses train rows onlytest rows are held out and scored onceNOT training; NOT learning; NOT generalizationNOT future performance; NOT unbiased; NOT safeNOT best split; NOT an accuracy guarantee