Summary - Model selection
- Machine Learning course
- Beginner
- Lesson 92/112
- 6 min
In this group you learned cross-validation, grid search, and hyperparameter tuning.
Summary: model selection
In this group:
- Model selection needs a fair comparison; the test set stays out of the selection.
- The balance between bias (too simple) and variance (too flexible).
- K-fold validation with
cross_val_scoreandcross_validate; mean ± std. GridSearchCVtries combinations;best_params_,best_estimator_.- Hyperparameters are what you choose, parameters what is learned;
validation_curveshows the effect.
At the intermediate level you'll see cross-validation and grid search with pipelines in detail.
Practice Choose the tree depth
Run GridSearchCV(DecisionTreeClassifier(random_state=0), ..., cv=5) with the grid {"max_depth": [2, 3, 4, 6, None]} on the breast cancer training set. Print best_params_, best_score_ (round(..., 3)) and the test score (round(..., 3)) on separate lines.
Starter code
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.tree import DecisionTreeClassifier
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
# buraya yazSample solution
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.tree import DecisionTreeClassifier
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
gs = GridSearchCV(DecisionTreeClassifier(random_state=0), {"max_depth": [2, 3, 4, 6, None]}, cv=5)
gs.fit(X_train, y_train)
print(gs.best_params_)
print(round(gs.best_score_, 3))
print(round(gs.score(X_test, y_test), 3))Expected output
{'max_depth': 2}
0.927
0.888
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 Which is the right model selection flow?
- Run CV on the test set
- Split → choose with CV on training → measure once on test Correct answer
- Try on test → pick the best
- Train and measure on all data
Explanation: The test set is only for the final, independent measurement.
2 Set up the grid search with 5 folds.
Question
gs = GridSearchCV(model, izgara, ____=5)
Answer
gs = GridSearchCV(model, izgara, cv=5)Explanation: cv takes the number of folds or a splitter object.
3 Match the tools.
| Term | Meaning |
|---|---|
cross_val_score |
K scores |
GridSearchCV |
Combination search |
validation_curve |
Single-setting sweep |
best_estimator_ |
Chosen model |