Summary - Dimensionality reduction

In this group you learned PCA, t-SNE, and dimensionality reduction advantages.

Summary: dimensionality reduction

In this group:

Two families: feature selection (original columns) and feature extraction (new columns).
Reasons: speed, noise reduction, visualisation, the curse of dimensionality.
PCA is linear and applicable to new data; t-SNE is a visualisation tool preserving local neighbourhoods.
2-D maps hint at class separability and which classes will mix.
The gain is always measured against the model without reduction; KernelPCA for non-linear structures.

Exercise The gain of reduction

Build make_pipeline(StandardScaler(), PCA(n_components=20), SVC()) on the digit data. Compute the 5-fold CV accuracy first as is, then with set_params(pca="passthrough"). Print the two scores with round(..., 3) on one line (with PCA first).

Show example solution

import numpy as np
from sklearn.datasets import load_digits

X, y = load_digits(return_X_y=True)
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.svm import SVC
from sklearn.model_selection import cross_val_score

pipe = make_pipeline(StandardScaler(), PCA(n_components=20), SVC())
a = cross_val_score(pipe, X, y, cv=5).mean()
pipe.set_params(pca="passthrough")
b = cross_val_score(pipe, X, y, cv=5).mean()
print(round(a, 3), round(b, 3))

Test yourself

Which can't be used for dimensionality reduction inside a model pipeline?
- PCA
- KernelPCA
- SelectKBest
- TSNE
Show answer

TSNE — TSNE has no transform method; the test data can't be moved to the same space.

Turn the reduction step off for comparison.

pipe.set_params(pca=____)

Show answer

pipe.set_params(pca="passthrough")

Match the tools.

load_digits

8x8 digit images
TSNE

2-D visualisation
KernelPCA

Curved structures
explained_variance_ratio_

Share of information on the map

Next lesson: What is feature selection?

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies