Gini Splits

A Split and Its Weighted Impurity

A candidate question creates child nodes. The split score weights each child impurity by how many rows reached that child.

highlighted = computed this step

Feature A creates two leaves

Feature A splits the parent into counts [2,1] and [1,2].

A: [2,1], [1,2]A:\ [2,1],\ [1,2]
Feature A splitThe table and tree show the same binary split.Feature A split tablenodecountsGnoteparent[3, 3]1/2left[2, 1]4/9right[1, 2]4/9weighted4/9weighted childrengain1/18parent - weightedFeature A split treeroot splitcounts=[3, 3]G=1/2left leafcounts=[2, 1]G=4/9right leafcounts=[1, 2]G=4/9

Why the children are weighted

Each child impurity is 4/9. The weighted impurity uses each child sample share, giving 4/9.

Gchild=4/9,Gweighted=4/9G_{\text{child}}=4/9,\quad G_{\text{weighted}}=4/9
Feature A splitThe table and tree show the same binary split.Feature A split tablenodecountsGnoteparent[3, 3]1/2left[2, 1]4/9right[1, 2]4/9weighted4/9weighted childrengain1/18parent - weightedFeature A split treeroot splitcounts=[3, 3]G=1/2left leafcounts=[2, 1]G=4/9right leafcounts=[1, 2]G=4/9

Summary

The split table and the tree are locked to the same displayed child counts. A hidden third child is rejected.

binary split only\text{binary split only}
Feature A node compositionParent and child node class bars are derived from integer counts.Feature A node compositionparent node [3, 3]1/21/2positive3 of 6negative3 of 6class count / node total1/2 + 1/2 = 1left child [2, 1]2/31/3positive2 of 3negative1 of 3class count / node total2/3 + 1/3 = 1right child [1, 2]1/32/3positive1 of 3negative2 of 3class count / node total1/3 + 2/3 = 1child counts sum exactly to parent counts