param_grid
- Machine Learning course
- Intermediate
- Lesson 21/112
- 5 min
param_grid is a dictionary of hyperparameter values to try.
Structure of param_grid
param_grid is a dict: the key is the parameter name and the value the list of values to try. The Cartesian product of all lists is tried:
izgara = {"svc__C": [0.1, 1, 10], "svc__kernel": ["linear", "rbf"]}
# 3 × 2 = 6 combinations
from sklearn.model_selection import ParameterGrid
print(len(ParameterGrid(izgara))) # 6ParameterGrid lets you see how many combinations will be tried before the search.
A list of dicts: a conditional grid
Some parameters only matter in certain settings: gamma isn't used with the linear kernel. A list of dicts prevents needless combinations:
izgara = [
{"svc__kernel": ["linear"], "svc__C": [0.1, 1, 10]},
{"svc__kernel": ["rbf"], "svc__C": [0.1, 1, 10], "svc__gamma": [0.01, 0.1]},
]
print(len(ParameterGrid(izgara))) # 3 + 6 = 9The same method even lets different models be compared in one search: {"svc": [SVC(), LogisticRegression()]}.
Practice Conditional grid
Build a list of dicts with C ∈ [0.1, 1, 10] for the linear kernel, and C ∈ [1, 10] and gamma ∈ [0.01, 0.1] for the rbf kernel. Print the number of combinations with ParameterGrid, then run GridSearchCV(..., cv=5) and print best_params_.
Starter code
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
pipe = Pipeline([("olcek", StandardScaler()), ("svc", SVC())])
from sklearn.model_selection import ParameterGrid
# buraya yazSample solution
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
pipe = Pipeline([("olcek", StandardScaler()), ("svc", SVC())])
from sklearn.model_selection import ParameterGrid
izgara = [
{"svc__kernel": ["linear"], "svc__C": [0.1, 1, 10]},
{"svc__kernel": ["rbf"], "svc__C": [1, 10], "svc__gamma": [0.01, 0.1]},
]
print(len(ParameterGrid(izgara)))
gs = GridSearchCV(pipe, izgara, cv=5).fit(X_train, y_train)
print(gs.best_params_)Expected output
7
{'svc__C': 0.1, 'svc__kernel': 'linear'}
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 How many combinations does a grid of three parameters with 3, 4 and 5 values contain?
- 20
- 15
- 60 Correct answer
- 12
Explanation: 3 × 4 × 5 = 60; each is additionally trained in every fold.
2 See the number of combinations before the search.
print(len(____(izgara)))print(len(ParameterGrid(izgara)))Explanation: ParameterGrid produces each combination as a dict.
3 Match the grids with the combination counts.
| Term | Meaning |
|---|---|
{"a": [1, 2], "b": [1, 2]} |
4 |
{"a": [1, 2, 3]} |
3 |
[{"a": [1]}, {"b": [1, 2]}] |
3 (1 + 2) |
{"a": [1, 2], "b": [1, 2, 3]} |
6 |