Machine Learning
Challenge 3
Perform clustering with KMeans.
Challenge 3: segments + classifier
A scenario combining unsupervised and supervised learning: first split the digit data into 10 clusters with PCA + KMeans, then measure how well each cluster overlaps the true labels. Even though the clusters were found without labels, how much does the structure resemble the real classes?
Z = PCA(n_components=30, random_state=0).fit_transform(StandardScaler().fit_transform(X))
kume = KMeans(n_clusters=10, n_init=10, random_state=0).fit_predict(Z)
print(adjusted_rand_score(y, kume))
Then we can assign each cluster the most frequent true label inside it and evaluate this like a "classifier".
Exercise Evaluate the clusters
Reduce the scaled digit data with PCA(n_components=30, random_state=0) and cluster it with KMeans(n_clusters=10, n_init=10, random_state=0). Print the ARI with round(..., 3). Then assign each cluster its most frequent true label and print the accuracy obtained with this mapping with round(..., 3).
Show example solution
import numpy as np
from sklearn.datasets import load_digits
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.cluster import KMeans
from sklearn.metrics import adjusted_rand_score
X, y = load_digits(return_X_y=True)
Z = PCA(n_components=30, random_state=0).fit_transform(StandardScaler().fit_transform(X))
kume = KMeans(n_clusters=10, n_init=10, random_state=0).fit_predict(Z)
print(round(adjusted_rand_score(y, kume), 3))
eslem = np.array([np.bincount(y[kume == k]).argmax() for k in range(10)])
print(round((eslem[kume] == y).mean(), 3))
Exercise Compare with supervised
Print the 5-fold CV accuracy of a KNeighborsClassifier(n_neighbors=5) in the same 30-component PCA space with round(..., 3). See how much difference using labels makes.
Show example solution
from sklearn.datasets import load_digits
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import cross_val_score
X, y = load_digits(return_X_y=True)
pipe = make_pipeline(StandardScaler(), PCA(n_components=30, random_state=0), KNeighborsClassifier(n_neighbors=5))
print(round(cross_val_score(pipe, X, y, cv=5).mean(), 3))
Test yourself
-
Why does assigning a cluster its most frequent label mean looking at labelled data?
- It uses the labels; it isn't a fully unsupervised evaluation
- It uses no labels
- It is random
- It is wrong
Show answer
It uses the labels; it isn't a fully unsupervised evaluation — The clustering is unlabelled but the cluster-label mapping needs label information.
-
Find the most frequent label in each cluster.
etiket = np.bincount(y[kume == k]).____()
Show answer
etiket = np.bincount(y[kume == k]).argmax()
-
Match the steps.
- PCA(30)
- Noise reduction
- KMeans(10)
- Unlabelled grouping
- ARI
- Structural similarity
- most frequent label
- Cluster-class mapping
Next lesson: Challenge 4
