Machine Learning
Train-test split
Data is split into train and test sets with train_test_split.
Why a separate test?
A model's real value is what it does on data it has never seen. train_test_split shuffles the data randomly and splits it; shuffle=True is the default.
from sklearn.model_selection import train_test_split
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2,
random_state=42, stratify=y)
With time series shuffling is wrong: it means learning from the future to predict the past. There the first part goes to training and the last part to testing with shuffle=False.
random_state and reproducibility
Different splits give different scores; with small datasets the difference can be large. Fixing random_state makes the result reproducible:
for rs in [0, 1, 2]:
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=rs)
print(rs, model.fit(X_tr, y_tr).score(X_te, y_te))
Seeing how much the score moves with the split teaches you not to over-trust a single number; cross-validation solves this problem.
Exercise Ordered split
Split the ordered 10-element data with test_size=0.3, shuffle=False and print the test inputs with X_test.ravel().
Show example solution
import numpy as np
from sklearn.model_selection import train_test_split
X = np.arange(10).reshape(-1, 1)
y = np.arange(10) * 2
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, shuffle=False)
print(X_test.ravel())
Test yourself
-
If you'll predict next month from daily sales data, how do you split?
- Without a test set
- Last days training, old days testing
- Without shuffling: old days for training, last days for testing
- With random shuffling
Show answer
Without shuffling: old days for training, last days for testing — Time order must be kept; the model may learn only from the past.
-
Split the time series without shuffling.
train_test_split(X, y, test_size=0.2, ____=False)
Show answer
train_test_split(X, y, test_size=0.2, shuffle=False)
-
Match the parameters.
- test_size
- Test share
- random_state
- Reproducible split
- stratify
- Keeps class proportion
- shuffle=False
- Keeps the order
Next lesson: Accuracy
