Cross-validation
- Machine Learning course
- Beginner
- Lesson 88/112
- 5 min
Cross-validation evaluates the model by splitting data into multiple folds.
Cross-validation
A single split depends on luck. K-fold cross-validation splits the data into K parts; each time one part is for validation and the rest for training. The mean of the K scores is a more reliable estimate:
from sklearn.model_selection import cross_val_score
skorlar = cross_val_score(KNeighborsClassifier(), X, y, cv=5)
print(skorlar.round(3)) # 5 separate scores
print(skorlar.mean().round(3), skorlar.std().round(3))The standard deviation shows how much the score moves from split to split.
Stratified folds for classification
When a classifier is given, cross_val_score uses StratifiedKFold by default; class proportions are kept in each fold. The metric is chosen with scoring:
cross_val_score(model, X, y, cv=5, scoring="f1_macro")
cross_val_score(reg, X, y, cv=5, scoring="neg_mean_absolute_error")Error metrics are given negative with "neg_", because scikit-learn always applies the "bigger is better" rule.
Practice 5-fold validation
Run 5-fold cross-validation on Iris for a DecisionTreeClassifier(random_state=0). Print the scores with round(3), then the mean with round(..., 3).
Starter code
from sklearn.datasets import load_iris
from sklearn.model_selection import cross_val_score
X, y = load_iris(return_X_y=True)
from sklearn.tree import DecisionTreeClassifier
# buraya yazSample solution
from sklearn.datasets import load_iris
from sklearn.model_selection import cross_val_score
X, y = load_iris(return_X_y=True)
from sklearn.tree import DecisionTreeClassifier
skorlar = cross_val_score(DecisionTreeClassifier(random_state=0), X, y, cv=5)
print(skorlar.round(3))
print(round(skorlar.mean(), 3))Expected output
[0.967 0.967 0.9 0.967 1. ] 0.96
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 With <code>cv=10</code>, how many times is each example in the validation set?
- 0
- 1 Correct answer
- 10
- 9
Explanation: Each example belongs to exactly one fold; it is measured once when that fold is the validation set.
2 Use macro F1 as the metric.
cross_val_score(model, X, y, cv=5, ____="f1_macro")cross_val_score(model, X, y, cv=5, scoring="f1_macro")Explanation: scoring takes the metric name as text.
3 Match the concepts.
| Term | Meaning |
|---|---|
cv=5 |
5 folds |
skorlar.mean() |
Mean success |
scores.std() |
Split instability |
neg_mean_absolute_error |
Negative MAE |