Machine Learning
inertia_
inertia_ gives the total within-cluster sum of squares.
inertia_
inertia_ is the sum of squared distances of examples to their assigned centre. A small value shows tight clusters but doesn't measure quality alone: it always falls as k grows and depends on the data's scale.
km = KMeans(n_clusters=5, n_init=10, random_state=0).fit(X)
d = km.transform(X).min(axis=1)
print(np.isclose((d ** 2).sum(), km.inertia_)) # True
score(X) returns the negative of the same value; it is for consistency with scikit-learn's "bigger is better" rule.
Other internal criteria
There are also internal criteria that reduce scale dependence:
from sklearn.metrics import calinski_harabasz_score, davies_bouldin_score
print(calinski_harabasz_score(X, km.labels_)) # bigger is better
print(davies_bouldin_score(X, km.labels_)) # smaller is better
Calinski-Harabasz is the ratio of between-cluster to within-cluster spread; Davies-Bouldin is each cluster's average similarity to its most similar cluster. They are used together with the silhouette when choosing k.
Exercise Verifying inertia
Train KMeans (5 clusters). Print with np.isclose that the sum of squares of the km.transform(X).min(axis=1) distances equals inertia_, then with np.isclose that km.score(X) is -inertia_, on separate lines.
Show example solution
import numpy as np
from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
X, _ = make_blobs(n_samples=500, centers=5, cluster_std=1.0, random_state=3)
km = KMeans(n_clusters=5, n_init=10, random_state=0).fit(X)
d = km.transform(X).min(axis=1)
print(np.isclose((d ** 2).sum(), km.inertia_))
print(np.isclose(km.score(X), -km.inertia_))
Test yourself
-
If the data is multiplied by 10 (every feature), how does inertia change?
- It becomes 10 times larger
- It doesn't change
- It decreases
- It becomes 100 times larger
Show answer
It becomes 100 times larger — Distances grow 10 times, squared distances 100 times.
-
Compute the internal criterion where smaller is better.
db = ____(X, km.labels_)
Show answer
db = davies_bouldin_score(X, km.labels_)
-
Match the criteria with their directions.
- inertia_
- Smaller is tighter
- silhouette_score
- Bigger is better (≤1)
- calinski_harabasz_score
- Bigger is better
- davies_bouldin_score
- Smaller is better
Next lesson: random_state
