Machine Learning
Training model
Train a KNN model.
A model with a pipeline
Instead of managing scaling and the model separately, we make a single object with make_pipeline. The fit call learns the scaler only on training data; predict automatically applies the same transformation to the test data:
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
model = make_pipeline(StandardScaler(), KNeighborsClassifier(n_neighbors=5))
model.fit(X_train, y_train)
That way the test data's mean can't leak into training; data leakage is prevented structurally.
Unscaled versus scaled
To see the effect of scaling, also train the same KNN without scaling:
ham = KNeighborsClassifier(n_neighbors=5).fit(X_train, y_train)
print(ham.score(X_test, y_test)) # proline dominates: low
print(model.score(X_test, y_test)) # scaled: clearly higher
Unscaled KNN in practice looks only at the feature with the largest units (proline); scaling lets all features vote.
Exercise Train the model
Train an unscaled KNeighborsClassifier(n_neighbors=5) and a make_pipeline(StandardScaler(), KNeighborsClassifier(n_neighbors=5)). Print the test accuracy of both with round(..., 3) on separate lines.
Show example solution
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
X, y = load_wine(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42, stratify=y)
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
ham = KNeighborsClassifier(n_neighbors=5).fit(X_train, y_train)
model = make_pipeline(StandardScaler(), KNeighborsClassifier(n_neighbors=5)).fit(X_train, y_train)
print(round(ham.score(X_test, y_test), 3))
print(round(model.score(X_test, y_test), 3))
Test yourself
-
What does the scaler do when <code>predict(X_test)</code> is called on a pipeline?
- It errors
- It transforms with the mean and std learned in training
- It refits on the test data
- It does nothing
Show answer
It transforms with the mean and std learned in training — In the test phase the pipeline only calls transform.
-
Combine the scaler and the model in one object.
model = ____(StandardScaler(), KNeighborsClassifier())
Show answer
model = make_pipeline(StandardScaler(), KNeighborsClassifier())
-
Match the steps.
- make_pipeline
- Chains the steps
- StandardScaler
- Mean 0, std 1
- model.fit
- Learns scale and model
- ham KNN
- Large unit dominates
Next lesson: Prediction
