Machine Learning
param_grid
param_grid is a dictionary of hyperparameter values to try.
Structure of param_grid
param_grid is a dict: the key is the parameter name and the value the list of values to try. The Cartesian product of all lists is tried:
izgara = {"svc__C": [0.1, 1, 10], "svc__kernel": ["linear", "rbf"]}
# 3 × 2 = 6 combinations
from sklearn.model_selection import ParameterGrid
print(len(ParameterGrid(izgara))) # 6
ParameterGrid lets you see how many combinations will be tried before the search.
A list of dicts: a conditional grid
Some parameters only matter in certain settings: gamma isn't used with the linear kernel. A list of dicts prevents needless combinations:
izgara = [
{"svc__kernel": ["linear"], "svc__C": [0.1, 1, 10]},
{"svc__kernel": ["rbf"], "svc__C": [0.1, 1, 10], "svc__gamma": [0.01, 0.1]},
]
print(len(ParameterGrid(izgara))) # 3 + 6 = 9
The same method even lets different models be compared in one search: {"svc": [SVC(), LogisticRegression()]}.
Exercise Conditional grid
Build a list of dicts with C ∈ [0.1, 1, 10] for the linear kernel, and C ∈ [1, 10] and gamma ∈ [0.01, 0.1] for the rbf kernel. Print the number of combinations with ParameterGrid, then run GridSearchCV(..., cv=5) and print best_params_.
Show example solution
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
pipe = Pipeline([("olcek", StandardScaler()), ("svc", SVC())])
from sklearn.model_selection import ParameterGrid
izgara = [
{"svc__kernel": ["linear"], "svc__C": [0.1, 1, 10]},
{"svc__kernel": ["rbf"], "svc__C": [1, 10], "svc__gamma": [0.01, 0.1]},
]
print(len(ParameterGrid(izgara)))
gs = GridSearchCV(pipe, izgara, cv=5).fit(X_train, y_train)
print(gs.best_params_)
Test yourself
-
How many combinations does a grid of three parameters with 3, 4 and 5 values contain?
Show answer
60 — 3 × 4 × 5 = 60; each is additionally trained in every fold.
-
See the number of combinations before the search.
print(len(____(izgara)))
Show answer
print(len(ParameterGrid(izgara)))
-
Match the grids with the combination counts.
- {"a": [1, 2], "b": [1, 2]}
- 4
- {"a": [1, 2, 3]}
- 3
- [{"a": [1]}, {"b": [1, 2]}]
- 3 (1 + 2)
- {"a": [1, 2], "b": [1, 2, 3]}
- 6
Next lesson: best_params_
