Machine Learning
Visualization
Dimensionality reduction helps visualize high-dimensional data in 2D or 3D.
Reducing for visualisation
The fastest way to understand high-dimensional data is to reduce it to 2 dimensions and plot it with class colours. If the classes separate on the plot, even a simple model will work well; if they are intertwined, the problem is hard:
Z = PCA(n_components=2).fit_transform(Xs)
fig, ax = plt.subplots()
sc = ax.scatter(Z[:, 0], Z[:, 1], c=y, cmap="tab10", s=8)
ax.set_xlabel("1. bileşen")
ax.set_ylabel("2. bileşen")
fig.colorbar(sc)
Writing the explained variance ratio in the axis labels tells how much information the plot shows.
Summarising with centres
In addition to the plot, each class's mean in the 2-D space can be computed; centres close to each other show beforehand which class pairs the model will confuse:
merkez = np.array([Z[y == k].mean(axis=0) for k in range(10)])
from scipy.spatial.distance import pdist, squareform
D = squareform(pdist(merkez))
np.fill_diagonal(D, np.inf)
print(np.unravel_index(D.argmin(), D.shape)) # the closest class pair
These pairs usually match the classes most confused in the confusion matrix.
Exercise A 2-D map
Reduce the digit data to 2 dimensions with PCA and plot it with class colours, then save. Print the total variance ratio explained by the first two components with round(..., 3). Then find and print the class pair whose 2-D centres are closest, using pdist/squareform.
Show example solution
import numpy as np
from sklearn.datasets import load_digits
X, y = load_digits(return_X_y=True)
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
from scipy.spatial.distance import pdist, squareform
pca = PCA(n_components=2)
Z = pca.fit_transform(X)
fig, ax = plt.subplots()
ax.scatter(Z[:, 0], Z[:, 1], c=y, s=8)
fig.savefig("rakam.png")
print(round(pca.explained_variance_ratio_.sum(), 3))
merkez = np.array([Z[y == k].mean(axis=0) for k in range(10)])
D = squareform(pdist(merkez))
np.fill_diagonal(D, np.inf)
print(tuple(int(v) for v in np.unravel_index(D.argmin(), D.shape)))
Test yourself
-
If the first two components explain only 20% of the variance, what does it mean that the classes look mixed on the plot?
- The data is wrong
- The classes may separate in other components
- The classes definitely can't be separated
- PCA is faulty
Show answer
The classes may separate in other components — The 2-D view is a small part of the information; the mixing isn't conclusive.
-
Plot colouring the classes.
ax.scatter(Z[:, 0], Z[:, 1], ____=y)
Show answer
ax.scatter(Z[:, 0], Z[:, 1], c=y)
-
Match the steps.
- PCA(2)
- Reduce to 2 dimensions
- c=y
- Class colours
- class centers
- Mean position
- close centers
- Classes that will mix
Next lesson: Advantages
