Summary - Random forest
- Machine Learning course
- Intermediate
- Lesson 40/112
- 6 min
In this group you learned random forest, ensemble, and bagging.
Summary: random forest
In this group:
- A random forest trains many trees on bootstraps and random feature subsets and combines them; it reduces variance.
- Voting is strong when errors are independent; the classifier averages probabilities.
n_estimatorsdoesn't cause overfitting;max_featuresandmin_samples_leafset the complexity.- A bootstrap contains ~63% of examples; the OOB score is free validation.
- Impurity importance is fast but biased; permutation importance is more reliable.
Practice Regression forest
Split the diabetes data with test_size=0.2, random_state=0. Train LinearRegression() and RandomForestRegressor(n_estimators=300, min_samples_leaf=5, random_state=0); print both test R² values with round(..., 3) on separate lines.
Starter code
from sklearn.datasets import load_diabetes
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.ensemble import RandomForestRegressor
X, y = load_diabetes(return_X_y=True)
# buraya yazSample solution
from sklearn.datasets import load_diabetes
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.ensemble import RandomForestRegressor
X, y = load_diabetes(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
for m in [LinearRegression(), RandomForestRegressor(n_estimators=300, min_samples_leaf=5, random_state=0)]:
print(round(m.fit(X_train, y_train).score(X_test, y_test), 3))Expected output
0.332 0.287
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 Which is unnecessary for a random forest?
- Setting aside a test set
- Choosing n_estimators
- Scaling the features Correct answer
- Giving random_state
Explanation: Trees compare against thresholds; scale doesn't change the decision.
2 Create a regression forest.
Question
rf = RandomForest____(n_estimators=200, random_state=0)
Answer
rf = RandomForestRegressor(n_estimators=200, random_state=0)Explanation: RandomForestRegressor gives the mean of the tree predictions.
3 Match the tools.
| Term | Meaning |
|---|---|
RandomForestClassifier |
Classification forest |
BaggingClassifier |
General bagging |
oob_score_ |
Out-of-bag accuracy |
permutation_importance |
Model-agnostic importance |