Machine Learning
best_params_
The best_params_ property gives the best hyperparameter combination.
best_params_ and best_estimator_
When the search ends, the best combination sits in best_params_ and the retrained model in best_estimator_. best_estimator_ is an ordinary pipeline; its steps can be accessed and it can be saved:
print(gs.best_params_)
en_iyi = gs.best_estimator_
print(en_iyi.named_steps["svc"].C)
print(en_iyi.named_steps["svc"].n_support_) # support vectors per class
gs.predict and gs.best_estimator_.predict give the same result.
cv_results_
cv_results_ is a dict holding, for each combination, the mean and standard deviation scores, times and ranking:
r = gs.cv_results_
for p, o, s in zip(r["params"], r["mean_test_score"], r["std_test_score"]):
print(p, round(o, 3), round(s, 3))
print(r["rank_test_score"])
If a simpler combination (like a smaller C) gives almost the same score as the best, it may be preferred.
Exercise Rank the results
From the cv_results_ dict of the given gs search, find the combination ranked 1 and its mean score. Print its params and the score with round(..., 3) on separate lines; then print that it equals best_params_.
Show example solution
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
pipe = Pipeline([("olcek", StandardScaler()), ("svc", SVC())])
gs = GridSearchCV(pipe, {"svc__C": [0.1, 1, 10], "svc__gamma": [0.01, 0.1]}, cv=5).fit(X_train, y_train)
r = gs.cv_results_
i = list(r["rank_test_score"]).index(1)
print(r["params"][i])
print(round(r["mean_test_score"][i], 3))
print(r["params"][i] == gs.best_params_)
Test yourself
-
What should be done if the best value is at the edge of the grid (e.g. C=100, grid max 100)?
- Look at the test set
- Extend the grid in that direction and search again
- Accept the result
- Narrow the grid
Show answer
Extend the grid in that direction and search again — An optimum at the edge shows the true best may lie outside the grid.
-
Get the retrained best model.
model = gs.____
Show answer
model = gs.best_estimator_
-
Match the attributes.
- best_params_
- Winning settings
- best_estimator_
- Trained model
- cv_results_
- All results
- rank_test_score
- Ranking
Next lesson: best_score_
