Classification
- Machine Learning course
- Beginner
- Lesson 56/112
- 5 min
A decision tree assigns the majority class at leaf nodes.
Classifying with a tree
A new example starts at the root and reaches a leaf by going left or right according to the questions. The prediction is the majority class of the training examples in that leaf; predict_proba gives the class shares in the leaf.
m = DecisionTreeClassifier(max_depth=2, random_state=0).fit(X, y)
print(m.predict(X[:3]))
print(m.predict_proba(X[100:101]))In depth-limited trees leaves can be mixed; the probabilities reflect this.
Feature importances
feature_importances_ shows how much each feature reduces impurity in the splits; the total is 1:
from sklearn.datasets import load_iris
d = load_iris()
m = DecisionTreeClassifier(random_state=0).fit(d.data, d.target)
for ad, o in zip(d.feature_names, m.feature_importances_):
print(f"{ad}: {o:.2f}")In Iris the petal measurements carry almost all of the importance; the sepal measurements contribute little.
Practice The most important feature
Train a DecisionTreeClassifier(random_state=0) on Iris. Find the index of the most important feature with argmax and print it, then print the sum of the importances with round(..., 3).
Starter code
from sklearn.datasets import load_iris
from sklearn.tree import DecisionTreeClassifier
X, y = load_iris(return_X_y=True)
# buraya yazSample solution
from sklearn.datasets import load_iris
from sklearn.tree import DecisionTreeClassifier
X, y = load_iris(return_X_y=True)
m = DecisionTreeClassifier(random_state=0).fit(X, y)
print(m.feature_importances_.argmax())
print(round(m.feature_importances_.sum(), 3))Expected output
3 1.0
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 If a leaf has 8 A and 2 B examples, what is <code>predict_proba</code> for an example landing there?
- [1.0, 0.0]
- [0.5, 0.5]
- [8, 2]
- [0.8, 0.2] Correct answer
Explanation: The probability is the class shares in the leaf: 8/10 and 2/10.
2 Read the feature importances.
onem = m.____onem = m.feature_importances_Explanation: In tree-based models importances live in the feature_importances_ attribute.
3 Match the outputs.
| Term | Meaning |
|---|---|
predict |
The leaf's majority class |
predict_proba |
Shares in the leaf |
feature_importances_ |
Feature contributions |
get_n_leaves() |
Number of leaves |