n_estimators
- Machine Learning course
- Intermediate
- Lesson 37/112
- 5 min
n_estimators sets the number of trees in the forest.
n_estimators
As the number of trees grows, the forest stabilises, but after a point the gain stops; it doesn't cause overfitting, only longer runtimes. It is the most forgiving setting of a random forest:
for n in [1, 10, 50, 200]:
rf = RandomForestClassifier(n_estimators=n, random_state=0).fit(X_train, y_train)
print(n, round(rf.score(X_test, y_test), 3))Rule of thumb: increase until the score settles, then decide by runtime; 100-500 is enough for most problems.
Other important settings
What really sets the forest's complexity are the per-tree settings:
RandomForestClassifier(
n_estimators=300,
max_features="sqrt", # candidate features per split; small = more diverse trees
max_depth=None, # unlimited trees by default
min_samples_leaf=1, # increasing it smooths
n_jobs=-1, # trees are trained in parallel
)Because the trees are independent, n_jobs=-1 speeds training up almost linearly with the number of cores.
Practice Effect of the tree count
Train forests for n_estimators = 1, 10, 100 (random_state=0). On each line print the tree count and the test accuracy with round(..., 3).
Starter code
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
# buraya yazSample solution
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
for n in [1, 10, 100]:
rf = RandomForestClassifier(n_estimators=n, random_state=0).fit(X_train, y_train)
print(n, round(rf.score(X_test, y_test), 3))Expected output
1 0.881 10 0.951 100 0.944
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 Does making n_estimators very large cause overfitting?
- Only with little data
- No; only the time grows Correct answer
- Yes, always
- Only in regression
Explanation: More trees only make the average more stable.
2 Consider the square root of the features at each split.
rf = RandomForestClassifier(max_features=____)rf = RandomForestClassifier(max_features="sqrt")Explanation: "sqrt" is the default in classification.
3 Match the settings.
| Term | Meaning |
|---|---|
n_estimators |
Number of trees |
max_features |
Tree diversity |
min_samples_leaf |
Leaf smoothness |
n_jobs |
Parallel training |