What is a decision tree?

A decision tree is a tree-like model that classifies data by splitting it with questions.

What is a decision tree?

A decision tree splits the data step by step with yes/no questions: "height > 170?", "age < 30?". Each inner node is a question and each leaf a decision.

from sklearn.tree import DecisionTreeClassifier, export_text
X = [[20, 0], [25, 1], [40, 0], [50, 1]]   # age, home owner?
y = [0, 0, 1, 1]
m = DecisionTreeClassifier(random_state=0).fit(X, y)
print(export_text(m, feature_names=["yas", "ev"]))

Trees are interpretable: why the model made a decision can be explained by the rules read up to the root.

No scaling needed

Trees compare each feature with a threshold on its own; they don't compute distances. So scaling doesn't change the result, and they work directly with mixed-unit data.

print(m.get_depth())      # depth of the tree
print(m.get_n_leaves())   # number of leaves

The same family works for regression: DecisionTreeRegressor predicts the mean of the examples in that region at the leaves.

Exercise Build a tree

Train a DecisionTreeClassifier(random_state=0); print the predictions for [[30, 1]] and [[45, 0]], then the depth of the tree.

Show example solution

from sklearn.tree import DecisionTreeClassifier

X = [[20, 0], [25, 1], [40, 0], [50, 1]]
y = [0, 0, 1, 1]

m = DecisionTreeClassifier(random_state=0).fit(X, y)
print(m.predict([[30, 1], [45, 0]]))
print(m.get_depth())

Test yourself

What does a leaf node represent in a decision tree?
- A feature
- The training error
- The final decision (prediction)
- A question
Show answer

The final decision (prediction) — Inner nodes ask questions; leaves give a class or value.

Print the tree's rules as text.

print(____(m, feature_names=["yas", "ev"]))

Show answer

print(export_text(m, feature_names=["yas", "ev"]))

Match the parts.

root

The first question
internal node

An intermediate question
leaf

Decision
get_depth()

Length of the question chain

Next lesson: Splitting criterion

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies