What is hierarchical clustering?

Hierarchical clustering groups data in a tree-like structure.

What is hierarchical clustering?

Hierarchical clustering produces not a single partition but a tree of nested clusters. In the agglomerative approach every example starts as its own cluster; at each step the two closest clusters merge until a single cluster remains.

from scipy.cluster.hierarchy import linkage
Z = linkage(X, method="ward")
print(Z.shape)   # (n-1, 4): each row is one merge

"Cutting" the tree at the desired level gives the desired number of clusters; unlike KMeans, k doesn't have to be given up front.

The linkage matrix Z

Each row of the linkage output is one merge: [cluster_a, cluster_b, distance, examples_in_new_cluster]. Numbers below n are original examples; n and above are clusters formed in earlier steps.

print(Z[:3])
# [[0. 1. 1. 2.]   examples 0 and 1, distance 1, 2 examples
#  ...           ]

The distances increase row by row; a large jump shows that two far-apart groups were merged and is a good cut point.

Exercise Inspect the linkage

Compute the linkage matrix with linkage(X, method="single"). Print its shape and the distance of the last merge with round(..., 2) on separate lines. Then print after which row the largest jump between consecutive distances occurs, using np.diff(Z[:, 2]).argmax().

Show example solution

import numpy as np
from scipy.cluster.hierarchy import linkage, fcluster

X = np.array([[0, 0], [0, 1], [1, 0], [5, 5], [5, 6], [6, 5], [12, 0], [12, 1]])

Z = linkage(X, method="single")
print(Z.shape)
print(round(Z[-1, 2], 2))
print(np.diff(Z[:, 2]).argmax())

Test yourself

How many merges does agglomerative clustering make on 8 examples?
- 7
- 8
- 4
- 28
Show answer

7 — Each merge reduces the cluster count by one: 7 steps from 8 to 1.
Compute the linkage matrix with the Ward method.
```
Z = ____(X, method="ward")
```
Show answer
```
Z = linkage(X, method="ward")
```
Match the Z columns.

Z[:, 0], Z[:, 1]

Merged clusters
Z[:, 2]

Merge distance
Z[:, 3]

Example count
n-1 rows

Total merges

Next lesson: Dendrogram

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies