Summary - KNN
- Machine Learning course
- Beginner
- Lesson 52/112
- 6 min
In this group you learned the KNN algorithm, the K parameter, and classification.
Summary: KNN
In this group:
- KNN classifies by the majority vote of the K nearest examples;
fitonly stores the data. - Small K overfits, large K underfits; K is chosen by validation, odd numbers preferred for two classes.
- Euclidean (default) and Manhattan distances; the
metricparameter. kneighbors,predict_proba,weights="distance".- Because it is distance-based, scaling is a must.
For regression the same idea lives in KNeighborsRegressor: the neighbours' mean is taken.
Practice The best K
For K = 1, 3, 5, 7, 9 compute the mean 5-fold cross-validation score on Iris (cross_val_score(..., cv=5).mean()). Print the K that gives the best score.
Starter code
from sklearn.datasets import load_iris
from sklearn.model_selection import cross_val_score
from sklearn.neighbors import KNeighborsClassifier
X, y = load_iris(return_X_y=True)
# buraya yazSample solution
from sklearn.datasets import load_iris
from sklearn.model_selection import cross_val_score
from sklearn.neighbors import KNeighborsClassifier
X, y = load_iris(return_X_y=True)
skorlar = {}
for k in [1, 3, 5, 7, 9]:
skorlar[k] = cross_val_score(KNeighborsClassifier(n_neighbors=k), X, y, cv=5).mean()
print(max(skorlar, key=skorlar.get))Expected output
7
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 Which is the most important preprocessing step when working with KNN?
- Scaling the features Correct answer
- Shuffling the labels
- Sorting the columns
- None is needed
Explanation: Distances are computed from the differences of all features; units must be equalised.
2 Weight the neighbours by distance.
Question
m = KNeighborsClassifier(n_neighbors=5, weights=____)
Answer
m = KNeighborsClassifier(n_neighbors=5, weights="distance")Explanation: The default "uniform" gives every neighbour an equal vote.
3 Match the parameters.
| Term | Meaning |
|---|---|
n_neighbors |
The K value |
metric |
Distance type |
weights |
Vote weighting |
KNeighborsRegressor |
Neighbour mean |