What is dimensionality reduction?

Dimensionality reduction reduces feature count while preserving information.

What is dimensionality reduction?

Dimensionality reduction means representing many-feature data with fewer features while keeping most of the information. There are two families:

Feature selection: keeps a subset of the original columns (interpretable).
Feature extraction: produces new columns (PCA, t-SNE, UMAP).

from sklearn.datasets import load_digits
X, y = load_digits(return_X_y=True)
print(X.shape)   # (1797, 64): 8x8-pixel digit images

64 pixels are more than needed to describe a digit; neighbouring pixels are strongly correlated.

Constant and correlated columns

In image data, edge pixels are almost always 0; their variance is near zero and they carry no information. Correlated columns repeat the same information:

print((X.var(axis=0) == 0).sum())      # pixels that never change
C = np.corrcoef(X[:, X.var(axis=0) > 0], rowvar=False)
print((np.abs(C) > 0.8).sum())         # strongly correlated pairs

Both cases show why dimensionality reduction is possible.

Exercise Get to know the digit data

Print the shape of the digit data, the number of pixels that never change and the number of classes (len(np.unique(y))) on separate lines.

Show example solution

import numpy as np
from sklearn.datasets import load_digits

X, y = load_digits(return_X_y=True)

print(X.shape)
print((X.var(axis=0) == 0).sum())
print(len(np.unique(y)))

Test yourself

Which family does PCA belong to?
- Feature extraction
- Feature selection
- Clustering
- Classification
Show answer

Feature extraction — PCA produces linear combinations of the original columns, not the columns themselves.

Count the columns that never change.

print((X.____(axis=0) == 0).sum())

Show answer

print((X.var(axis=0) == 0).sum())

Match the concepts.

feature selection

Subset of original columns
feature extraction

New columns
constant column

Carries no information
correlated columns

Repeated information

Next lesson: Why reduce?

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies