Depth
- Machine Learning course
- Beginner
- Lesson 55/112
- 5 min
The tree depth (max_depth) prevents overfitting.
max_depth
An unlimited tree keeps splitting until only one class remains in each leaf and memorises the training data. max_depth sets at most how many steps the question chain can have:
for d in [1, 2, 3, None]:
m = DecisionTreeClassifier(max_depth=d, random_state=0).fit(X_train, y_train)
print(d, m.score(X_train, y_train), m.score(X_test, y_test))Shallow trees learn more general rules, deep trees more detailed ones.
Other limits
Besides depth there are other stopping rules:
DecisionTreeClassifier(
max_depth=4,
min_samples_split=10, # at least 10 examples to split
min_samples_leaf=5, # at least 5 examples in every leaf
max_leaf_nodes=12, # at most 12 leaves
)min_samples_leaf is especially effective because it stops a separate leaf being opened for a single outlier.
Practice Try the depth
Train a DecisionTreeClassifier(random_state=0) on Iris for max_depth 1, 2 and 3. On each line print the depth and the test accuracy with round(..., 3).
Starter code
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)
# buraya yazSample solution
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)
for d in [1, 2, 3]:
m = DecisionTreeClassifier(max_depth=d, random_state=0).fit(X_train, y_train)
print(d, round(m.score(X_test, y_test), 3))Expected output
1 0.667 2 0.956 3 0.956
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 What does <code>max_depth=None</code> mean?
- There is no depth limit Correct answer
- The depth is 0
- No tree is built
- The depth is 1
Explanation: The default is None; the tree grows until the leaves are pure.
2 Limit the tree to at most 3 levels.
m = DecisionTreeClassifier(____=3)m = DecisionTreeClassifier(max_depth=3)Explanation: max_depth is the length of the longest path from root to leaf.
3 Match the parameters.
| Term | Meaning |
|---|---|
max_depth |
Maximum depth |
min_samples_split |
Minimum examples to split |
min_samples_leaf |
Minimum examples per leaf |
max_leaf_nodes |
Maximum leaves |