Depth

The tree depth (max_depth) prevents overfitting.

max_depth

An unlimited tree keeps splitting until only one class remains in each leaf and memorises the training data. max_depth sets at most how many steps the question chain can have:

for d in [1, 2, 3, None]:
    m = DecisionTreeClassifier(max_depth=d, random_state=0).fit(X_train, y_train)
    print(d, m.score(X_train, y_train), m.score(X_test, y_test))

Shallow trees learn more general rules, deep trees more detailed ones.

Other limits

Besides depth there are other stopping rules:

DecisionTreeClassifier(
    max_depth=4,
    min_samples_split=10,   # at least 10 examples to split
    min_samples_leaf=5,     # at least 5 examples in every leaf
    max_leaf_nodes=12,      # at most 12 leaves
)

min_samples_leaf is especially effective because it stops a separate leaf being opened for a single outlier.

Exercise Try the depth

Train a DecisionTreeClassifier(random_state=0) on Iris for max_depth 1, 2 and 3. On each line print the depth and the test accuracy with round(..., 3).

Show example solution

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)

for d in [1, 2, 3]:
    m = DecisionTreeClassifier(max_depth=d, random_state=0).fit(X_train, y_train)
    print(d, round(m.score(X_test, y_test), 3))

Test yourself

What does <code>max_depth=None</code> mean?
- There is no depth limit
- The depth is 0
- No tree is built
- The depth is 1
Show answer

There is no depth limit — The default is None; the tree grows until the leaves are pure.

Limit the tree to at most 3 levels.

m = DecisionTreeClassifier(____=3)

Show answer

m = DecisionTreeClassifier(max_depth=3)

Match the parameters.

max_depth

Maximum depth
min_samples_split

Minimum examples to split
min_samples_leaf

Minimum examples per leaf
max_leaf_nodes

Maximum leaves

Next lesson: Classification

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies