Classification
- Machine Learning course
- Beginner
- Lesson 50/112
- 5 min
KNN classifies a new example by majority vote of its k nearest neighbors.
Seeing the neighbours
The kneighbors method returns a query's nearest neighbours and their distances. It is very useful for understanding where a decision comes from:
d, idx = m.kneighbors([[2, 2]])
print(idx) # [[2 1 0]] indices of the three nearest training examples
print(d) # their distancesThe neighbours' labels can be read with np.array(y)[idx]; the majority gives the prediction.
Probability and weighted votes
predict_proba gives the share of votes for each class. With weights="distance", nearer neighbours' votes weigh more:
m = KNeighborsClassifier(n_neighbors=5).fit(X, y)
print(m.predict_proba([[3, 3]])) # [[0.6 0.4]] class order: m.classes_
m2 = KNeighborsClassifier(n_neighbors=5, weights="distance")The probabilities show how "close a call" the decision is.
Practice Vote shares
Train a KNeighborsClassifier(n_neighbors=5). Print the predict_proba result and the predict result for [[4, 4]] on separate lines.
Starter code
from sklearn.neighbors import KNeighborsClassifier
X = [[1, 1], [1, 2], [2, 1], [6, 6], [6, 7], [7, 6]]
y = ["A", "A", "A", "B", "B", "B"]
# buraya yazSample solution
from sklearn.neighbors import KNeighborsClassifier
X = [[1, 1], [1, 2], [2, 1], [6, 6], [6, 7], [7, 6]]
y = ["A", "A", "A", "B", "B", "B"]
m = KNeighborsClassifier(n_neighbors=5).fit(X, y)
print(m.predict_proba([[4, 4]]))
print(m.predict([[4, 4]]))Expected output
[[0.4 0.6]] ['B']
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 With K=5, if the neighbours' labels are A, B, A, B, A, what is the prediction?
- Undecided
- An error
- A Correct answer
- B
Explanation: 3 votes for A, 2 for B; the majority is A.
2 Get the class probabilities.
p = m.____([[3, 3]])p = m.predict_proba([[3, 3]])Explanation: predict_proba returns one column per class.
3 Match the methods.
| Term | Meaning |
|---|---|
predict |
Majority class |
predict_proba |
Vote shares |
kneighbors |
Neighbour indices |
classes_ |
Class order |