CodeBlanks Logo Icon CodeBlanks
Login Register

Training and testing

The model is trained on training data and its performance is measured on test data.

Why separate test data?

If you measure the model with the data you trained it on, you can't tell whether it has just memorised it. So the data is split in two: the model learns with the training set and its success on unseen examples is measured with the test set.

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42)

test_size=0.2 sets aside 20% of the data for testing; random_state makes the split the same on every run.

The four returned parts

train_test_split returns four arrays and the order matters: first the two X parts, then the two y parts.

model.fit(X_train, y_train)          # training only
print(model.score(X_test, y_test))   # test only

Test data must never be used in training; otherwise the measured success doesn't reflect reality. This is called data leakage.

Practice Split the data

Split the 20-example data with test_size=0.25 and random_state=0. Print the lengths of the training and test sets separated by a space.

Starter code

model.py
from sklearn.model_selection import train_test_split

X = [[i] for i in range(20)]
y = [i % 2 for i in range(20)]

# buraya yaz

Sample solution

model.py
from sklearn.model_selection import train_test_split

X = [[i] for i in range(20)]
y = [i % 2 for i in range(20)]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)
print(len(X_train), len(X_test))

Expected output

15 5

Questions and answers

Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.

1 With 100 examples and <code>test_size=0.25</code>, how many examples remain for training?

  • 25
  • 100
  • 50
  • 75 Correct answer

Explanation: 25% (25 examples) goes to the test set and the remaining 75 are for training.

2 Set aside 30% of the data for testing.

Question 
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=____, random_state=0)
Answer 
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)

Explanation: test_size takes a ratio between 0 and 1 or an integer count; 0.3 means 30 percent.

3 Match the parts with their meanings.

Term Meaning
X_train Training inputs
y_train Training labels
X_test Test inputs
y_test Test labels

All lessons in Beginner

1 What is ML?

  1. What is machine learning?
  2. Supervised and unsupervised
  3. Data and model
  4. Training and testing
  5. Feature and label notation
  6. Summary - ML fundamentals

2 NumPy basics

  1. What is NumPy?
  2. Creating arrays
  3. Array properties
  4. Array indexing
  5. Reshaping arrays
  6. Summary - NumPy basics

3 Pandas basics

  1. What is Pandas?
  2. Creating a DataFrame
  3. Column operations
  4. Selecting and filtering data
  5. Viewing first rows
  6. Summary - Pandas basics

4 Data preprocessing

  1. Data cleaning
  2. Missing data
  3. Data types
  4. Label encoding
  5. Data splitting
  6. Summary - Data preprocessing

5 Statistics basics

  1. Mean
  2. Median
  3. Standard deviation
  4. Variance
  5. Correlation
  6. Summary - Statistics

6 Linear algebra

  1. Vectors
  2. Matrices
  3. Matrix multiplication
  4. Transpose
  5. Identity matrix
  6. Summary - Linear algebra

7 Project: Data analysis

  1. Loading data
  2. Cleaning data
  3. Computing statistics
  4. Printing results

8 Regression

  1. What is regression?
  2. Linear regression
  3. Model training
  4. Prediction
  5. Error measurement
  6. Summary - Regression

9 KNN

  1. What is KNN?
  2. The K parameter
  3. Distance measurement
  4. Classification
  5. Model training
  6. Summary - KNN

10 Decision trees

  1. What is a decision tree?
  2. Splitting criterion
  3. Depth
  4. Classification
  5. Overfitting
  6. Summary - Decision trees

11 Classification

  1. What is classification?
  2. Binary classification
  3. Multi-class
  4. Accuracy
  5. Confusion matrix
  6. Summary - Classification

12 Model evaluation

  1. Train-test split
  2. Accuracy
  3. Precision
  4. Recall
  5. F1 score
  6. Summary - Model evaluation

13 Project: Classification

  1. Preparing data
  2. Training model
  3. Prediction
  4. Evaluation

14 Scaling

  1. What is scaling?
  2. StandardScaler
  3. MinMaxScaler
  4. When to use?
  5. Pipeline
  6. Summary - Scaling

15 Feature engineering

  1. What is a feature?
  2. Feature selection
  3. Feature extraction
  4. Categorical features
  5. Numerical features
  6. Summary - Feature engineering

16 Model selection

  1. What is model selection?
  2. Cross-validation
  3. Grid search
  4. Score comparison
  5. Hyperparameter
  6. Summary - Model selection

17 Project: Regression

  1. Preparing data
  2. Training model
  3. Prediction
  4. Error measurement

18 Metrics

  1. What is a metric?
  2. Accuracy
  3. Precision
  4. Recall
  5. F1 score
  6. Summary - Metrics

19 Data visualization

  1. What is visualization?
  2. Scatter plot
  3. Histogram
  4. Line plot
  5. Box plot
  6. Summary - Visualization

20 Beginner final

  1. Challenge 1
  2. Challenge 2
  3. Challenge 3
  4. Challenge 4