Machine Learning
n_estimators
n_estimators sets the number of trees in the forest.
n_estimators
As the number of trees grows, the forest stabilises, but after a point the gain stops; it doesn't cause overfitting, only longer runtimes. It is the most forgiving setting of a random forest:
for n in [1, 10, 50, 200]:
rf = RandomForestClassifier(n_estimators=n, random_state=0).fit(X_train, y_train)
print(n, round(rf.score(X_test, y_test), 3))
Rule of thumb: increase until the score settles, then decide by runtime; 100-500 is enough for most problems.
Other important settings
What really sets the forest's complexity are the per-tree settings:
RandomForestClassifier(
n_estimators=300,
max_features="sqrt", # candidate features per split; small = more diverse trees
max_depth=None, # unlimited trees by default
min_samples_leaf=1, # increasing it smooths
n_jobs=-1, # trees are trained in parallel
)
Because the trees are independent, n_jobs=-1 speeds training up almost linearly with the number of cores.
Exercise Effect of the tree count
Train forests for n_estimators = 1, 10, 100 (random_state=0). On each line print the tree count and the test accuracy with round(..., 3).
Show example solution
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
for n in [1, 10, 100]:
rf = RandomForestClassifier(n_estimators=n, random_state=0).fit(X_train, y_train)
print(n, round(rf.score(X_test, y_test), 3))
Test yourself
-
Does making n_estimators very large cause overfitting?
- Only with little data
- No; only the time grows
- Yes, always
- Only in regression
Show answer
No; only the time grows — More trees only make the average more stable.
-
Consider the square root of the features at each split.
rf = RandomForestClassifier(max_features=____)
Show answer
rf = RandomForestClassifier(max_features="sqrt")
-
Match the settings.
- n_estimators
- Number of trees
- max_features
- Tree diversity
- min_samples_leaf
- Leaf smoothness
- n_jobs
- Parallel training
Next lesson: Bagging
