Elbow method
- Machine Learning course
- Intermediate
- Lesson 50/112
- 5 min
The elbow method is used to find the optimal k value.
The elbow method
KMeans can't find the number of clusters by itself. The elbow method plots the total squared distance (inertia_) for different values of k. As k grows inertia always falls; the "elbow" where the drop clearly slows is a sensible k:
for k in range(1, 8):
km = KMeans(n_clusters=k, n_init=10, random_state=0).fit(X)
print(k, round(km.inertia_))The elbow isn't always clear; then extra criteria like the silhouette score are checked.
The silhouette score
The silhouette score measures for each example, between -1 and 1, "how close to its own cluster and how far from the nearest neighbouring cluster". The k with the highest mean is preferred:
from sklearn.metrics import silhouette_score
for k in range(2, 7):
etiket = KMeans(n_clusters=k, n_init=10, random_state=0).fit_predict(X)
print(k, round(silhouette_score(X, etiket), 3))The silhouette needs at least 2 clusters; values near 1 show well-separated clusters, values near 0 overlapping ones.
Practice The best k
Train a KMeans(n_clusters=k, n_init=10, random_state=0) for k = 2-6. On each line print k, the inertia_ value (as an integer with round) and the silhouette score with round(..., 3).
Starter code
from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
X, y_gercek = make_blobs(n_samples=300, centers=4, cluster_std=0.8, random_state=0)
from sklearn.metrics import silhouette_score
# buraya yazSample solution
from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
X, y_gercek = make_blobs(n_samples=300, centers=4, cluster_std=0.8, random_state=0)
from sklearn.metrics import silhouette_score
for k in range(2, 7):
km = KMeans(n_clusters=k, n_init=10, random_state=0).fit(X)
print(k, round(km.inertia_), round(silhouette_score(X, km.labels_), 3))Expected output
2 1349 0.496 3 709 0.515 4 370 0.58 5 326 0.503 6 292 0.43
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 Why isn't the k with the smallest inertia chosen?
- Inertia increases
- It is chosen
- At k = number of examples inertia is 0; that is meaningless Correct answer
- It can't be computed
Explanation: If every example is its own cluster the distances are zero, but nothing has been learned.
2 Compute the mean silhouette score.
s = ____(X, etiket)s = silhouette_score(X, etiket)Explanation: silhouette_score takes the data and the labels.
3 Match the criteria.
| Term | Meaning |
|---|---|
inertia_ |
Sum of squared distances |
dirsek |
Where the drop slows |
siluet ≈ 1 |
Well-separated clusters |
siluet ≈ 0 |
Overlapping clusters |