best_params_
- Machine Learning course
- Intermediate
- Lesson 22/112
- 5 min
The best_params_ property gives the best hyperparameter combination.
best_params_ and best_estimator_
When the search ends, the best combination sits in best_params_ and the retrained model in best_estimator_. best_estimator_ is an ordinary pipeline; its steps can be accessed and it can be saved:
print(gs.best_params_)
en_iyi = gs.best_estimator_
print(en_iyi.named_steps["svc"].C)
print(en_iyi.named_steps["svc"].n_support_) # support vectors per classgs.predict and gs.best_estimator_.predict give the same result.
cv_results_
cv_results_ is a dict holding, for each combination, the mean and standard deviation scores, times and ranking:
r = gs.cv_results_
for p, o, s in zip(r["params"], r["mean_test_score"], r["std_test_score"]):
print(p, round(o, 3), round(s, 3))
print(r["rank_test_score"])If a simpler combination (like a smaller C) gives almost the same score as the best, it may be preferred.
Practice Rank the results
From the cv_results_ dict of the given gs search, find the combination ranked 1 and its mean score. Print its params and the score with round(..., 3) on separate lines; then print that it equals best_params_.
Starter code
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
pipe = Pipeline([("olcek", StandardScaler()), ("svc", SVC())])
gs = GridSearchCV(pipe, {"svc__C": [0.1, 1, 10], "svc__gamma": [0.01, 0.1]}, cv=5).fit(X_train, y_train)
# buraya yazSample solution
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
pipe = Pipeline([("olcek", StandardScaler()), ("svc", SVC())])
gs = GridSearchCV(pipe, {"svc__C": [0.1, 1, 10], "svc__gamma": [0.01, 0.1]}, cv=5).fit(X_train, y_train)
r = gs.cv_results_
i = list(r["rank_test_score"]).index(1)
print(r["params"][i])
print(round(r["mean_test_score"][i], 3))
print(r["params"][i] == gs.best_params_)Expected output
{'svc__C': 10, 'svc__gamma': 0.01}
0.981
True
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 What should be done if the best value is at the edge of the grid (e.g. C=100, grid max 100)?
- Look at the test set
- Extend the grid in that direction and search again Correct answer
- Accept the result
- Narrow the grid
Explanation: An optimum at the edge shows the true best may lie outside the grid.
2 Get the retrained best model.
model = gs.____model = gs.best_estimator_Explanation: best_estimator_ exists only when refit=True.
3 Match the attributes.
| Term | Meaning |
|---|---|
best_params_ |
Winning settings |
best_estimator_ |
Trained model |
cv_results_ |
All results |
rank_test_score |
Ranking |