Summary - Model selection

In this group you learned cross-validation, grid search, and hyperparameter tuning.

Summary: model selection

In this group:

Model selection needs a fair comparison; the test set stays out of the selection.
The balance between bias (too simple) and variance (too flexible).
K-fold validation with cross_val_score and cross_validate; mean ± std.
GridSearchCV tries combinations; best_params_, best_estimator_.
Hyperparameters are what you choose, parameters what is learned; validation_curve shows the effect.

At the intermediate level you'll see cross-validation and grid search with pipelines in detail.

Exercise Choose the tree depth

Run GridSearchCV(DecisionTreeClassifier(random_state=0), ..., cv=5) with the grid {"max_depth": [2, 3, 4, 6, None]} on the breast cancer training set. Print best_params_, best_score_ (round(..., 3)) and the test score (round(..., 3)) on separate lines.

Show example solution

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.tree import DecisionTreeClassifier

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)

gs = GridSearchCV(DecisionTreeClassifier(random_state=0), {"max_depth": [2, 3, 4, 6, None]}, cv=5)
gs.fit(X_train, y_train)
print(gs.best_params_)
print(round(gs.best_score_, 3))
print(round(gs.score(X_test, y_test), 3))

Test yourself

Which is the right model selection flow?
- Run CV on the test set
- Split → choose with CV on training → measure once on test
- Try on test → pick the best
- Train and measure on all data
Show answer

Split → choose with CV on training → measure once on test — The test set is only for the final, independent measurement.

Set up the grid search with 5 folds.

gs = GridSearchCV(model, izgara, ____=5)

Show answer

gs = GridSearchCV(model, izgara, cv=5)

Match the tools.

cross_val_score

K scores
GridSearchCV

Combination search
validation_curve

Single-setting sweep
best_estimator_

Chosen model

Next lesson: Preparing data

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies