Summary - Random forest

In this group you learned random forest, ensemble, and bagging.

Summary: random forest

In this group:

A random forest trains many trees on bootstraps and random feature subsets and combines them; it reduces variance.
Voting is strong when errors are independent; the classifier averages probabilities.
n_estimators doesn't cause overfitting; max_features and min_samples_leaf set the complexity.
A bootstrap contains ~63% of examples; the OOB score is free validation.
Impurity importance is fast but biased; permutation importance is more reliable.

Exercise Regression forest

Split the diabetes data with test_size=0.2, random_state=0. Train LinearRegression() and RandomForestRegressor(n_estimators=300, min_samples_leaf=5, random_state=0); print both test R² values with round(..., 3) on separate lines.

Show example solution

from sklearn.datasets import load_diabetes
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.ensemble import RandomForestRegressor

X, y = load_diabetes(return_X_y=True)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
for m in [LinearRegression(), RandomForestRegressor(n_estimators=300, min_samples_leaf=5, random_state=0)]:
    print(round(m.fit(X_train, y_train).score(X_test, y_test), 3))

Test yourself

Which is unnecessary for a random forest?
- Setting aside a test set
- Choosing n_estimators
- Scaling the features
- Giving random_state
Show answer

Scaling the features — Trees compare against thresholds; scale doesn't change the decision.

Create a regression forest.

rf = RandomForest____(n_estimators=200, random_state=0)

Show answer

rf = RandomForestRegressor(n_estimators=200, random_state=0)

Match the tools.

RandomForestClassifier

Classification forest
BaggingClassifier

General bagging
oob_score_

Out-of-bag accuracy
permutation_importance

Model-agnostic importance

Next lesson: What is boosting?

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies