Machine Learning
Agglomerative
Agglomerative starts each example as a cluster and merges them.
AgglomerativeClustering
Its scikit-learn counterpart is AgglomerativeClustering. Either the number of clusters or a distance threshold is given:
from sklearn.cluster import AgglomerativeClustering
ag = AgglomerativeClustering(n_clusters=3, linkage="ward")
etiket = ag.fit_predict(X)
ag2 = AgglomerativeClustering(n_clusters=None, distance_threshold=10).fit(X)
print(ag2.n_clusters_) # cluster count from the threshold
Unlike KMeans there is no predict method: a new example can't be added to the tree; only the training data is labelled.
Scale and cost
Agglomerative clustering needs all pairwise distances; memory and time grow with the square of the number of examples. It slows down on tens of thousands of examples. One fix is allowing merges only between neighbours with connectivity:
from sklearn.neighbors import kneighbors_graph
bag = kneighbors_graph(X, n_neighbors=10, include_self=False)
ag = AgglomerativeClustering(n_clusters=3, connectivity=bag).fit(X)
A neighbourhood graph also helps find long, winding clusters.
Exercise Cluster with a threshold
Train an AgglomerativeClustering(n_clusters=None, distance_threshold=30) on the synthetic data; print the resulting cluster count (n_clusters_). Then train with n_clusters=3 and print the ARI against the true labels with round(..., 3).
Show example solution
from sklearn.datasets import make_blobs
from sklearn.cluster import AgglomerativeClustering
from sklearn.metrics import adjusted_rand_score
X, y = make_blobs(n_samples=300, centers=3, cluster_std=[0.5, 1.5, 0.8], random_state=4)
ag = AgglomerativeClustering(n_clusters=None, distance_threshold=30).fit(X)
print(ag.n_clusters_)
etiket = AgglomerativeClustering(n_clusters=3).fit_predict(X)
print(round(adjusted_rand_score(y, etiket), 3))
Test yourself
-
What is the way to cluster a new example with AgglomerativeClustering?
- Using transform
- Calling fcluster
- There is none directly; retrain, or train a classifier on the labels
- Using predict
Show answer
There is none directly; retrain, or train a classifier on the labels — The practical fix: train a KNN on the cluster labels and assign new examples with it.
-
Give a distance threshold instead of a cluster count.
ag = AgglomerativeClustering(n_clusters=None, ____=10)
Show answer
ag = AgglomerativeClustering(n_clusters=None, distance_threshold=10)
-
Match the properties.
- n_clusters
- Fixed cluster count
- distance_threshold
- Cut by threshold
- n_clusters_
- Resulting cluster count
- connectivity
- Limiting by neighbourhood
Next lesson: Linkage
