Machine Learning
KMeans
KMeans is a popular clustering algorithm that divides data into k clusters.
KMeans
KMeans assigns each example to the nearest cluster centre and moves the centres to the mean of the assigned examples; these two steps repeat until the centres stop changing. The goal is to minimise the sum of squared distances of examples to their centres.
km = KMeans(n_clusters=3, n_init=10, random_state=0).fit(X)
print(km.cluster_centers_) # 3 centres
print(km.predict([[0, 0]])) # the cluster of a new point
predict assigns new examples to the nearest centre; that is why KMeans can be applied to new data.
KMeans's assumptions
KMeans assumes clusters are round, of similar size and similar density. It fails on moon-shaped or nested ring clusters:
from sklearn.datasets import make_moons
Xm, ym = make_moons(200, noise=0.05, random_state=0)
print(adjusted_rand_score(ym, KMeans(2, n_init=10).fit_predict(Xm))) # low
The number of clusters n_clusters must be given beforehand; you'll see how to choose it in the elbow method lesson.
Exercise Find the centres
Train a KMeans(n_clusters=3, n_init=10, random_state=0) on small data with three clear groups. Sort the centres by their first coordinate (c[np.argsort(c[:, 0])]) and print them with np.round(..., 2). Then print that the cluster of [[5, 5]] equals that of [[5.2, 4.9]].
Show example solution
import numpy as np
from sklearn.cluster import KMeans
X = np.array([[0, 0], [0, 1], [1, 0], [5, 5], [5, 6], [6, 5], [10, 0], [10, 1], [11, 0]])
km = KMeans(n_clusters=3, n_init=10, random_state=0).fit(X)
c = km.cluster_centers_
print(np.round(c[np.argsort(c[:, 0])], 2))
print(km.predict([[5, 5]])[0] == km.predict([[5.2, 4.9]])[0])
Test yourself
-
Where is a centre moved in the KMeans update step?
- To the mean of the assigned examples
- To the farthest example
- To a random point
- To the origin
Show answer
To the mean of the assigned examples — The point minimising the sum of squared distances is the mean.
-
Train and get the labels in one step.
etiket = KMeans(n_clusters=3, n_init=10).____(X)
Show answer
etiket = KMeans(n_clusters=3, n_init=10).fit_predict(X)
-
Match the steps.
- assignment
- To the nearest centre
- update
- Move to the mean
- cluster_centers_
- Centres
- predict
- Assign a new example
Next lesson: Centroid
