Summary - Decision trees

In this group you learned decision trees, splitting criteria, and overfitting.

Summary: decision trees

In this group:

A tree splits data with yes/no questions; leaves decide and the rules are read with export_text.
Splits are chosen with Gini (default) or entropy.
max_depth, min_samples_leaf, ccp_alpha limit the tree.
An unlimited tree overfits; the training-test gap is the sign.
No scaling is needed; feature_importances_ gives feature contributions.

The instability of single trees is fixed by random forest and boosting, which you'll see later.

Exercise Tree regression

Train a DecisionTreeRegressor(max_depth=1). Because there is a single split there are only two distinct predictions; print the predictions for [[2], [8]].

Show example solution

from sklearn.tree import DecisionTreeRegressor

X = [[1], [2], [3], [7], [8], [9]]
y = [10, 12, 11, 30, 32, 31]

m = DecisionTreeRegressor(max_depth=1).fit(X, y)
print(m.predict([[2], [8]]))

Test yourself

Which model doesn't need scaling?
- Every distance-based model
- Decision tree
- KNN
- Ridge without scaling
Show answer

Decision tree — A tree compares each feature with its own threshold; units don't change the decision.

Require at least 5 examples in every leaf.

m = DecisionTreeClassifier(____=5)

Show answer

m = DecisionTreeClassifier(min_samples_leaf=5)

Match the concepts.

Gini

Impurity measure
export_text

Prints the rules
ccp_alpha

Pruning strength
DecisionTreeRegressor

Mean at the leaf

Next lesson: What is classification?

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies