Hyperparameter
- Machine Learning course
- Beginner
- Lesson 91/112
- 5 min
A hyperparameter is a parameter set before model training.
What is a hyperparameter?
Parameters are learned from the data during training (coef_, tree splits). Hyperparameters are the settings you choose before training (n_neighbors, max_depth, C):
m = DecisionTreeClassifier(max_depth=3) # hyperparameter: you choose
m.fit(X, y)
print(m.tree_.node_count) # learned structureHyperparameters set the model's complexity; the right value changes from dataset to dataset and is found by validation.
get_params and the validation curve
get_params() shows all the hyperparameters of a model. validation_curve gives training and validation scores together while varying a single hyperparameter:
from sklearn.model_selection import validation_curve
d = [1, 2, 3, 5, 8]
tr, va = validation_curve(DecisionTreeClassifier(random_state=0), X, y,
param_name="max_depth", param_range=d, cv=5)
print(tr.mean(axis=1).round(2), va.mean(axis=1).round(2))The point where the validation score starts to drop while the training score keeps rising is where overfitting begins.
Practice Validation curve
Run validation_curve on Iris for max_depth with the values [1, 2, 3, 5] (cv=5). Print the training and validation means (mean(axis=1).round(3)) on separate lines.
Starter code
from sklearn.datasets import load_iris
from sklearn.model_selection import validation_curve
from sklearn.tree import DecisionTreeClassifier
X, y = load_iris(return_X_y=True)
# buraya yazSample solution
from sklearn.datasets import load_iris
from sklearn.model_selection import validation_curve
from sklearn.tree import DecisionTreeClassifier
X, y = load_iris(return_X_y=True)
tr, va = validation_curve(DecisionTreeClassifier(random_state=0), X, y, param_name="max_depth", param_range=[1, 2, 3, 5], cv=5)
print(tr.mean(axis=1).round(3))
print(va.mean(axis=1).round(3))Expected output
[0.667 0.962 0.973 0.998] [0.667 0.933 0.96 0.96 ]
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 Which is a hyperparameter?
- mean_ in StandardScaler
- The thresholds a tree learns
- n_neighbors in KNN Correct answer
- coef_ in LinearRegression
Explanation: n_neighbors is given before training; the others are learned during fit.
2 List all the model's hyperparameters.
print(m.____())print(m.get_params())Explanation: get_params returns a dict; set_params changes it.
3 Classify the values.
| Term | Meaning |
|---|---|
max_depth |
Hyperparameter |
coef_ |
Learned parameter |
validation_curve |
Sweeps one setting |
set_params |
Changes a setting |