Machine Learning
Grid search
GridSearchCV finds the best parameters by trying all combinations.
Grid search
GridSearchCV tries every combination of the given parameter values with cross-validation and picks the best:
from sklearn.model_selection import GridSearchCV
izgara = {"n_neighbors": [1, 3, 5, 7, 9], "weights": ["uniform", "distance"]}
gs = GridSearchCV(KNeighborsClassifier(), izgara, cv=5)
gs.fit(X_train, y_train)
print(gs.best_params_, gs.best_score_)
5 × 2 = 10 combinations, each with 5 folds: 50 trainings in total. As the grid grows, the time grows multiplicatively.
refit and best_estimator_
When the search ends, GridSearchCV retrains the model with the best parameters on all training data (refit=True). That model is best_estimator_, and gs.predict uses it directly:
print(gs.best_estimator_)
print(gs.score(X_test, y_test)) # test only at the very end
The right flow: split → grid search on the training set → measure the best model once on the test set.
Exercise Find K with a grid
Run GridSearchCV(KNeighborsClassifier(), ..., cv=5) with the grid {"n_neighbors": [1, 3, 5, 7, 9, 11]} on the Iris training set. Print best_params_ and the test accuracy with round(..., 3) on separate lines.
Show example solution
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.neighbors import KNeighborsClassifier
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)
gs = GridSearchCV(KNeighborsClassifier(), {"n_neighbors": [1, 3, 5, 7, 9, 11]}, cv=5)
gs.fit(X_train, y_train)
print(gs.best_params_)
print(round(gs.score(X_test, y_test), 3))
Test yourself
-
With one 3-valued and one 4-valued parameter and <code>cv=5</code>, how many trainings does a grid search do (excluding the final refit)?
Show answer
60 — 3 × 4 = 12 combinations × 5 folds = 60.
-
Read the best parameters.
print(gs.____)
Show answer
print(gs.best_params_)
-
Match the attributes.
- best_params_
- Best settings
- best_score_
- Best CV mean
- best_estimator_
- Retrained model
- cv_results_
- All trials
Next lesson: Score comparison
