Data Preparation
Train-Test Split (Fixed)
Partition a dataset into a training set and a test set using a fixed
pre-determined index order. The trace shows X_train and y_train
growing one point at a time for the first six positions, then X_test
and y_test receiving the final two.
By hand
Given a fixed index order (representing a pre-determined shuffle), walk
each position with enumerate. Append to the training lists for the first
six positions and to the test lists for the remainder.
naive.py
Replay: real traced execution (multi-file project)
X = [1, 2, 3, 4, 5, 6, 7, 8]
y = [10, 20, 30, 40, 50, 60, 70, 80]
order = [2, 5, 0, 7, 4, 1, 3, 6]
X_train = []
y_train = []
X_test = []
y_test = []
for i, idx in enumerate(order):
if i < 6:
X_train.append(X[idx])
y_train.append(y[idx])
else:
X_test.append(X[idx])
y_test.append(y[idx])
print('RESULT:', (len(X_train), len(X_test)))
X ← [1, 2, 3, 4, 5, 6, 7, 8]
1X = [1, 2, 3, 4, 5, 6, 7, 8]2y = [10, 20, 30, 40, 50, 60, 70, 80]values this step[1, 2, 3, 4, 5, 6, 7, 8]Xy ← [10, 20, 30, 40, 50, 60, 70, 80]
1X = [1, 2, 3, 4, 5, 6, 7, 8]2y = [10, 20, 30, 40, 50, 60, 70, 80]3order = [2, 5, 0, 7, 4, 1, 3, 6]values this step[10, 20, 30, 40, 50, 60, 70, 80]yorder ← [2, 5, 0, 7, 4, 1, 3, 6]
2y = [10, 20, 30, 40, 50, 60, 70, 80]3order = [2, 5, 0, 7, 4, 1, 3, 6]4X_train = []values this step[2, 5, 0, 7, 4, 1, 3, 6]orderX_train ← []
3order = [2, 5, 0, 7, 4, 1, 3, 6]4X_train = []5y_train = []values this step[]X_trainy_train ← []
4X_train = []5y_train = []6X_test = []values this step[]y_trainX_test ← []
5y_train = []6X_test = []7y_test = []values this step[]X_testy_test ← []
6X_test = []7y_test = []8for i, idx in enumerate(order):values this step[]y_testi ← 0, idx ← 2, X_train ← [3], y_train ← [30]
pass 1 of 67y_test = []8for i, idx in enumerate(order):9 if i < 6:10 X_train.append(X[idx])11 y_train.append(y[idx])12 else:values this step0i2idx[] → [3]X_train[] → [30]y_trainAll 6 passes — pass 1 is the card above pass iidxX_trainy_trainX_testy_test1 0 2 [] → [3] [] → [30] — — 2 0 → 1 2 → 5 [3] → [3, 6] [30] → [30, 60] — — 3 1 → 2 5 → 0 [3, 6] → [3, 6, 1] [30, 60] → [30, 60, 10] — — 4 2 → 3 0 → 7 [3, 6, 1] → [3, 6, 1, 8] [30, 60, 10] → [30, 60, 10, 80] — — 5 3 → 4 7 → 4 [3, 6, 1, 8] → [3, 6, 1, 8, 5] [30, 60, 10, 80] → [30, 60, 10, 80, 50] — — 6 4 → 5 4 → 1 [3, 6, 1, 8, 5] → [3, 6, 1, 8, 5, 2] [30, 60, 10, 80, 50] → [30, 60, 10, 80, 50, 20] [] → [4] [] → [40] i ← 6, idx ← 3, X_test ← [4], y_test ← [40]
pass 1 of 27y_test = []8for i, idx in enumerate(order):9 if i < 6:10 X_train.append(X[idx])11 y_train.append(y[idx])12 else:13 X_test.append(X[idx])14 y_test.append(y[idx])15print('RESULT:', (len(X_train), len(X_test)))values this step5 → 6i1 → 3idx[] → [4]X_test[] → [40]y_testi ← 7, idx ← 6, X_test ← [4, 7], y_test ← [40, 70]
pass 2 of 27y_test = []8for i, idx in enumerate(order):9 if i < 6:10 X_train.append(X[idx])11 y_train.append(y[idx])12 else:13 X_test.append(X[idx])14 y_test.append(y[idx])15print('RESULT:', (len(X_train), len(X_test)))values this step6 → 7i3 → 6idx[4] → [4, 7]X_test[40] → [40, 70]y_testfor i, idx in enumerate(order):
7y_test = []8for i, idx in enumerate(order):9 if i < 6:stdout ← RESULT: (6, 2)
14 y_test.append(y[idx])15print('RESULT:', (len(X_train), len(X_test)))values this stepRESULT: (6, 2)stdout
With scikit-learn
train_test_split handles the shuffle and partition in one call.
test_size=0.25 requests 25% of 8 points = 2 test items; random_state=42
makes the shuffle reproducible. The actual rows selected differ from the
hand-coded fixed order above.
library.py
from sklearn.model_selection import train_test_split
from dalib.display import set_display
set_display()
X = [1, 2, 3, 4, 5, 6, 7, 8]
y = [10, 20, 30, 40, 50, 60, 70, 80]
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.25, random_state=42, shuffle=True
)
print('train X:', X_train)
print('test X:', X_test)
print('RESULT:', (len(X_train), len(X_test)))
train X: [1, 8, 3, 5, 4, 7]
test X: [2, 6]
RESULT: (6, 2)
Implementation notes
- RESULT reports only
(train_size, test_size). Both halves produce 6/2, but the specific rows in each split differ: the naive version uses a hand-written order while sklearn applies its own seeded permutation. Aligning the two selections exactly would require hard-coding sklearn's internal shuffle, which would be contrived. random_state=42makes the sklearn split deterministic across runs within a pinned environment (same sklearn and NumPy versions); omitting it produces a different split every run.- A 75/25 train/test ratio is a common starting point; real workflows often
also carve out a validation set (
train_test_splitcalled twice) or use cross-validation instead of a single hold-out. shuffle=Trueis the default but is stated explicitly here for clarity.