Grid search
- Machine Learning course
- Beginner
- Lesson 89/112
- 5 min
GridSearchCV finds the best parameters by trying all combinations.
Grid search
GridSearchCV tries every combination of the given parameter values with cross-validation and picks the best:
from sklearn.model_selection import GridSearchCV
izgara = {"n_neighbors": [1, 3, 5, 7, 9], "weights": ["uniform", "distance"]}
gs = GridSearchCV(KNeighborsClassifier(), izgara, cv=5)
gs.fit(X_train, y_train)
print(gs.best_params_, gs.best_score_)5 × 2 = 10 combinations, each with 5 folds: 50 trainings in total. As the grid grows, the time grows multiplicatively.
refit and best_estimator_
When the search ends, GridSearchCV retrains the model with the best parameters on all training data (refit=True). That model is best_estimator_, and gs.predict uses it directly:
print(gs.best_estimator_)
print(gs.score(X_test, y_test)) # test only at the very endThe right flow: split → grid search on the training set → measure the best model once on the test set.
Practice Find K with a grid
Run GridSearchCV(KNeighborsClassifier(), ..., cv=5) with the grid {"n_neighbors": [1, 3, 5, 7, 9, 11]} on the Iris training set. Print best_params_ and the test accuracy with round(..., 3) on separate lines.
Starter code
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.neighbors import KNeighborsClassifier
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)
# buraya yazSample solution
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.neighbors import KNeighborsClassifier
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)
gs = GridSearchCV(KNeighborsClassifier(), {"n_neighbors": [1, 3, 5, 7, 9, 11]}, cv=5)
gs.fit(X_train, y_train)
print(gs.best_params_)
print(round(gs.score(X_test, y_test), 3))Expected output
{'n_neighbors': 7}
1.0
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 With one 3-valued and one 4-valued parameter and <code>cv=5</code>, how many trainings does a grid search do (excluding the final refit)?
- 60 Correct answer
- 12
- 35
- 20
Explanation: 3 × 4 = 12 combinations × 5 folds = 60.
2 Read the best parameters.
print(gs.____)print(gs.best_params_)Explanation: best_params_ gives the best combination as a dict.
3 Match the attributes.
| Term | Meaning |
|---|---|
best_params_ |
Best settings |
best_score_ |
Best CV mean |
best_estimator_ |
Retrained model |
cv_results_ |
All trials |