Decision Trees

Decision Stump Predict

Apply a given depth-1 decision tree (stump): if x < threshold → left_class, else → right_class. A single loop classifies each new point. Library: DecisionTreeClassifier(max_depth=1).fit(X_train, y_train).predict(X_new). RESULT: predicted class list.

By hand

threshold=4.0 (from best-split-find). left_class=0, right_class=1. X_new=[0,3,5,9]. Predictions: 0<4→0, 3<4→0, 5≥4→1, 9≥4→1.

naive.py
Replay: real traced execution (multi-file project)
threshold = 4.0
left_class = 0
right_class = 1
X_new = [0, 3, 5, 9]
preds = []
for x in X_new:
    c = left_class if x < threshold else right_class
    preds.append(c)
print('RESULT:', preds)

  1. threshold ← 4.0

    1threshold = 4.02left_class = 0
    values this step4.0threshold

  2. left_class ← 0

    1threshold = 4.02left_class = 03right_class = 1
    values this step0left_class

  3. right_class ← 1

    2left_class = 03right_class = 14X_new = [0, 3, 5, 9]
    values this step1right_class

  4. X_new ← [0, 3, 5, 9]

    3right_class = 14X_new = [0, 3, 5, 9]5preds = []
    values this step[0, 3, 5, 9]X_new

  5. preds ← []

    4X_new = [0, 3, 5, 9]5preds = []6for x in X_new:
    values this step[]preds

  6. x ← 0

    5preds = []6for x in X_new:7    c = left_class if x < threshold else right_class
    values this step0x

  7. c ← 0

    6for x in X_new:7    c = left_class if x < threshold else right_class8    preds.append(c)
    values this step0c

  8. preds ← [0]

    7    c = left_class if x < threshold else right_class8    preds.append(c)9print('RESULT:', preds)
    values this step[] → [0]preds

  9. x ← 3

    5preds = []6for x in X_new:7    c = left_class if x < threshold else right_class
    values this step0 → 3x

  10. c = left_class if x < threshold else right_class

    6for x in X_new:7    c = left_class if x < threshold else right_class8    preds.append(c)

  11. preds ← [0, 0]

    7    c = left_class if x < threshold else right_class8    preds.append(c)9print('RESULT:', preds)
    values this step[0] → [0, 0]preds

  12. x ← 5

    5preds = []6for x in X_new:7    c = left_class if x < threshold else right_class
    values this step3 → 5x

  13. c ← 1

    6for x in X_new:7    c = left_class if x < threshold else right_class8    preds.append(c)
    values this step0 → 1c

  14. preds ← [0, 0, 1]

    7    c = left_class if x < threshold else right_class8    preds.append(c)9print('RESULT:', preds)
    values this step[0, 0] → [0, 0, 1]preds

  15. x ← 9

    5preds = []6for x in X_new:7    c = left_class if x < threshold else right_class
    values this step5 → 9x

  16. c = left_class if x < threshold else right_class

    6for x in X_new:7    c = left_class if x < threshold else right_class8    preds.append(c)

  17. preds ← [0, 0, 1, 1]

    7    c = left_class if x < threshold else right_class8    preds.append(c)9print('RESULT:', preds)
    values this step[0, 0, 1] → [0, 0, 1, 1]preds

  18. for x in X_new:

    5preds = []6for x in X_new:7    c = left_class if x < threshold else right_class

  19. stdout ← RESULT: [0, 0, 1, 1]

    8    preds.append(c)9print('RESULT:', preds)
    values this stepRESULT: [0, 0, 1, 1]stdout

With scikit-learn

DecisionTreeClassifier(max_depth=1) trains on the same data; .predict applies the identical threshold rule to new points.

library.py 
from sklearn.tree import DecisionTreeClassifier
from dalib.display import set_display
set_display()

X_train = [[1], [2], [3], [5], [7], [8]]
y_train = [0, 0, 0, 1, 1, 1]
clf = DecisionTreeClassifier(max_depth=1, random_state=0)
clf.fit(X_train, y_train)
X_new = [[0], [3], [5], [9]]
preds = clf.predict(X_new).tolist()
print('threshold:', round(float(clf.tree_.threshold[0]), 4))
print('RESULT:', preds)

threshold: 4.0
RESULT: [0, 0, 1, 1]

Implementation notes

  • The stump rule is hardcoded from best-split-find (this chapter) — this lesson isolates the predict step from the fit step.
  • sklearn's left condition is feature ≤ threshold; the naive uses strict <. Both agree here because no test point equals 4.0.
  • Majority class per leaf: left (training X=1,2,3) → all class 0; right (training X=5,7,8) → all class 1. Impure leaves would use majority vote.
  • Cross-reference: knn-classify-majority (ch02) for another simple classifier; a stump is O(1) at predict time vs O(n) for kNN.