Challenge 2
- Machine Learning course
- Beginner
- Lesson 110/112
- 8 min
Process data with NumPy and Pandas.
Challenge 2: write KNN from scratch
The whole logic of KNN is a few lines: compute the distances, pick the k nearest examples, take the majority of their labels.
def knn_tahmin(X_egitim, y_egitim, q, k):
d = np.sqrt(((X_egitim - q) ** 2).sum(axis=1))
yakin = np.argsort(d)[:k]
return np.bincount(y_egitim[yakin]).argmax()Comparing your implementation with scikit-learn's shows that you really understand the algorithm.
Practice Your own KNN
Write the knn_tahmin(X_egitim, y_egitim, q, k) function. Produce a prediction with k=5 for every example of the Iris test set and print the accuracy with round(..., 3).
Starter code
import numpy as np
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)
# buraya yazSample solution
import numpy as np
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)
def knn_tahmin(X_egitim, y_egitim, q, k):
d = np.sqrt(((X_egitim - q) ** 2).sum(axis=1))
yakin = np.argsort(d)[:k]
return np.bincount(y_egitim[yakin]).argmax()
p = np.array([knn_tahmin(X_train, y_train, q, 5) for q in X_test])
print(round((p == y_test).mean(), 3))Expected output
1.0
Practice Compare with scikit-learn
Train a KNeighborsClassifier(n_neighbors=5) on the same split. Print on one line in how many examples your predictions (p) match its predictions, using (p == m.predict(X_test)).sum(), and the test set size.
Starter code
import numpy as np
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)
d = np.sqrt(((X_train[None, :, :] - X_test[:, None, :]) ** 2).sum(axis=2))
p = np.array([np.bincount(y_train[np.argsort(r)[:5]]).argmax() for r in d])
# buraya yazSample solution
import numpy as np
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)
d = np.sqrt(((X_train[None, :, :] - X_test[:, None, :]) ** 2).sum(axis=2))
p = np.array([np.bincount(y_train[np.argsort(r)[:5]]).argmax() for r in d])
m = KNeighborsClassifier(n_neighbors=5).fit(X_train, y_train)
print((p == m.predict(X_test)).sum(), len(X_test))Expected output
45 45
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 What does <code>np.argsort(d)[:k]</code> give?
- The indices of the k smallest distances Correct answer
- The k largest distances
- The k-th distance
- The sorted distances
Explanation: argsort gives indices from smallest to largest; the first k are the nearest neighbours.
2 Take the majority of the neighbour labels.
sinif = np.bincount(y_egitim[yakin]).____()sinif = np.bincount(y_egitim[yakin]).argmax()Explanation: bincount counts each class's votes; argmax gives the one with the most.
3 Match the steps.
| Term | Meaning |
|---|---|
((X - q) ** 2).sum(axis=1) |
Squared distances |
np.argsort(d)[:k] |
Nearest k |
np.bincount |
Vote count |
argmax |
Winning class |