Machine Learning
Model training
KNN is a lazy algorithm; it stores training data as-is.
Scale, then train
Distance-based KNN decides by the large-unit feature on unscaled data. First every feature is given mean 0 and standard deviation 1 with StandardScaler:
from sklearn.preprocessing import StandardScaler
sc = StandardScaler().fit(X_train)
m = KNeighborsClassifier(5).fit(sc.transform(X_train), y_train)
print(m.score(sc.transform(X_test), y_test))
The scaler is fit only on the training data and the same transformation is applied to the test data.
The Iris dataset
scikit-learn ships with small example datasets. Iris describes 150 examples of 3 flower species with 4 measurements and is a classic testing ground for KNN:
from sklearn.datasets import load_iris
X, y = load_iris(return_X_y=True)
print(X.shape, set(y)) # (150, 4) {0, 1, 2}
return_X_y=True returns the X and y arrays directly.
Exercise KNN on Iris
Load Iris and split it with test_size=0.3, random_state=1, stratify=y. fit a StandardScaler on the training data, train a KNeighborsClassifier(n_neighbors=5) and print the test accuracy with round(..., 3).
Show example solution
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=1, stratify=y)
sc = StandardScaler().fit(X_train)
m = KNeighborsClassifier(n_neighbors=5).fit(sc.transform(X_train), y_train)
print(round(m.score(sc.transform(X_test), y_test), 3))
Test yourself
-
With age (20-60) and salary (10,000-90,000) features, what does unscaled KNN look at?
- Neither
- Almost only at salary
- Only at age
- Both equally
Show answer
Almost only at salary — Salary differences are thousands of units; age differences contribute almost nothing to the distance.
-
Apply the scaling learned in training to the test data.
X_test_s = sc.____(X_test)
Show answer
X_test_s = sc.transform(X_test)
-
Match the steps.
- load_iris
- Example data
- StandardScaler
- Mean 0, std 1
- sc.fit(X_train)
- Learn the scale
- m.score(X_test_s, y_test)
- Test accuracy
Next lesson: Summary - KNN
