Machine Learning
Elbow method
The elbow method is used to find the optimal k value.
The elbow method
KMeans can't find the number of clusters by itself. The elbow method plots the total squared distance (inertia_) for different values of k. As k grows inertia always falls; the "elbow" where the drop clearly slows is a sensible k:
for k in range(1, 8):
km = KMeans(n_clusters=k, n_init=10, random_state=0).fit(X)
print(k, round(km.inertia_))
The elbow isn't always clear; then extra criteria like the silhouette score are checked.
The silhouette score
The silhouette score measures for each example, between -1 and 1, "how close to its own cluster and how far from the nearest neighbouring cluster". The k with the highest mean is preferred:
from sklearn.metrics import silhouette_score
for k in range(2, 7):
etiket = KMeans(n_clusters=k, n_init=10, random_state=0).fit_predict(X)
print(k, round(silhouette_score(X, etiket), 3))
The silhouette needs at least 2 clusters; values near 1 show well-separated clusters, values near 0 overlapping ones.
Exercise The best k
Train a KMeans(n_clusters=k, n_init=10, random_state=0) for k = 2-6. On each line print k, the inertia_ value (as an integer with round) and the silhouette score with round(..., 3).
Show example solution
from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
X, y_gercek = make_blobs(n_samples=300, centers=4, cluster_std=0.8, random_state=0)
from sklearn.metrics import silhouette_score
for k in range(2, 7):
km = KMeans(n_clusters=k, n_init=10, random_state=0).fit(X)
print(k, round(km.inertia_), round(silhouette_score(X, km.labels_), 3))
Test yourself
-
Why isn't the k with the smallest inertia chosen?
- Inertia increases
- It is chosen
- At k = number of examples inertia is 0; that is meaningless
- It can't be computed
Show answer
At k = number of examples inertia is 0; that is meaningless — If every example is its own cluster the distances are zero, but nothing has been learned.
-
Compute the mean silhouette score.
s = ____(X, etiket)
Show answer
s = silhouette_score(X, etiket)
-
Match the criteria.
- inertia_
- Sum of squared distances
- dirsek
- Where the drop slows
- siluet ≈ 1
- Well-separated clusters
- siluet ≈ 0
- Overlapping clusters
Next lesson: Labels
