What is hierarchical clustering?
- Machine Learning course
- Intermediate
- Lesson 59/112
- 5 min
Hierarchical clustering groups data in a tree-like structure.
What is hierarchical clustering?
Hierarchical clustering produces not a single partition but a tree of nested clusters. In the agglomerative approach every example starts as its own cluster; at each step the two closest clusters merge until a single cluster remains.
from scipy.cluster.hierarchy import linkage
Z = linkage(X, method="ward")
print(Z.shape) # (n-1, 4): each row is one merge"Cutting" the tree at the desired level gives the desired number of clusters; unlike KMeans, k doesn't have to be given up front.
The linkage matrix Z
Each row of the linkage output is one merge: [cluster_a, cluster_b, distance, examples_in_new_cluster]. Numbers below n are original examples; n and above are clusters formed in earlier steps.
print(Z[:3])
# [[0. 1. 1. 2.] examples 0 and 1, distance 1, 2 examples
# ... ]The distances increase row by row; a large jump shows that two far-apart groups were merged and is a good cut point.
Practice Inspect the linkage
Compute the linkage matrix with linkage(X, method="single"). Print its shape and the distance of the last merge with round(..., 2) on separate lines. Then print after which row the largest jump between consecutive distances occurs, using np.diff(Z[:, 2]).argmax().
Starter code
import numpy as np
from scipy.cluster.hierarchy import linkage, fcluster
X = np.array([[0, 0], [0, 1], [1, 0], [5, 5], [5, 6], [6, 5], [12, 0], [12, 1]])
# buraya yazSample solution
import numpy as np
from scipy.cluster.hierarchy import linkage, fcluster
X = np.array([[0, 0], [0, 1], [1, 0], [5, 5], [5, 6], [6, 5], [12, 0], [12, 1]])
Z = linkage(X, method="single")
print(Z.shape)
print(round(Z[-1, 2], 2))
print(np.diff(Z[:, 2]).argmax())Expected output
(7, 4) 7.21 4
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 How many merges does agglomerative clustering make on 8 examples?
- 7 Correct answer
- 8
- 4
- 28
Explanation: Each merge reduces the cluster count by one: 7 steps from 8 to 1.
2 Compute the linkage matrix with the Ward method.
Z = ____(X, method="ward")Z = linkage(X, method="ward")Explanation: linkage lives in scipy.cluster.hierarchy.
3 Match the Z columns.
| Term | Meaning |
|---|---|
Z[:, 0], Z[:, 1] |
Merged clusters |
Z[:, 2] |
Merge distance |
Z[:, 3] |
Example count |
n-1 rows |
Total merges |