Model training
- Machine Learning course
- Beginner
- Lesson 51/112
- 5 min
KNN is a lazy algorithm; it stores training data as-is.
Scale, then train
Distance-based KNN decides by the large-unit feature on unscaled data. First every feature is given mean 0 and standard deviation 1 with StandardScaler:
from sklearn.preprocessing import StandardScaler
sc = StandardScaler().fit(X_train)
m = KNeighborsClassifier(5).fit(sc.transform(X_train), y_train)
print(m.score(sc.transform(X_test), y_test))The scaler is fit only on the training data and the same transformation is applied to the test data.
The Iris dataset
scikit-learn ships with small example datasets. Iris describes 150 examples of 3 flower species with 4 measurements and is a classic testing ground for KNN:
from sklearn.datasets import load_iris
X, y = load_iris(return_X_y=True)
print(X.shape, set(y)) # (150, 4) {0, 1, 2}return_X_y=True returns the X and y arrays directly.
Practice KNN on Iris
Load Iris and split it with test_size=0.3, random_state=1, stratify=y. fit a StandardScaler on the training data, train a KNeighborsClassifier(n_neighbors=5) and print the test accuracy with round(..., 3).
Starter code
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
X, y = load_iris(return_X_y=True)
# buraya yazSample solution
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=1, stratify=y)
sc = StandardScaler().fit(X_train)
m = KNeighborsClassifier(n_neighbors=5).fit(sc.transform(X_train), y_train)
print(round(m.score(sc.transform(X_test), y_test), 3))Expected output
0.933
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 With age (20-60) and salary (10,000-90,000) features, what does unscaled KNN look at?
- Neither
- Almost only at salary Correct answer
- Only at age
- Both equally
Explanation: Salary differences are thousands of units; age differences contribute almost nothing to the distance.
2 Apply the scaling learned in training to the test data.
X_test_s = sc.____(X_test)X_test_s = sc.transform(X_test)Explanation: There is no refitting on the test data.
3 Match the steps.
| Term | Meaning |
|---|---|
load_iris |
Example data |
StandardScaler |
Mean 0, std 1 |
sc.fit(X_train) |
Learn the scale |
m.score(X_test_s, y_test) |
Test accuracy |