Summary - Cross-validation

In this group you learned k-fold cross-validation and cross_val_score.

Summary: cross-validation

In this group:

CV removes the luck of a single split; the result is mean ± standard deviation.
KFold, StratifiedKFold, GroupKFold, TimeSeriesSplit: the splitter must mimic the real use of the data.
cross_val_score, cross_validate (multiple metrics, training score), cross_val_predict (out-of-fold predictions).
Repeated CV gives stability, nested CV an honest score including selection.

Even after CV, the test set is used only once, at the very end.

Exercise Time series CV

Walk through the folds of TimeSeriesSplit(n_splits=4) for a 20-step X. On each line print the training set size and the first and last value of the validation indices.

Show example solution

import numpy as np
from sklearn.model_selection import TimeSeriesSplit

X = np.arange(20).reshape(-1, 1)

for egitim, dogrulama in TimeSeriesSplit(n_splits=4).split(X):
    print(len(egitim), dogrulama[0], dogrulama[-1])

Test yourself

Which is the right splitter for data with several transactions per customer?
- KFold
- LeaveOneOut
- No split needed
- GroupKFold (by customer id)
Show answer

GroupKFold (by customer id) — If a customer's transactions land in different folds, the model memorises the customer.

Use the splitter that keeps the time order.

cv = ____(n_splits=5)

Show answer

cv = TimeSeriesSplit(n_splits=5)

Match the outputs.

cross_val_score

Array of scores
cross_validate

Dict
cross_val_predict

Array of predictions
get_n_splits

Number of folds

Next lesson: What is grid search?

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies