Summary - Dimensionality reduction
- Machine Learning course
- Intermediate
- Lesson 80/112
- 6 min
In this group you learned PCA, t-SNE, and dimensionality reduction advantages.
Summary: dimensionality reduction
In this group:
- Two families: feature selection (original columns) and feature extraction (new columns).
- Reasons: speed, noise reduction, visualisation, the curse of dimensionality.
- PCA is linear and applicable to new data; t-SNE is a visualisation tool preserving local neighbourhoods.
- 2-D maps hint at class separability and which classes will mix.
- The gain is always measured against the model without reduction;
KernelPCAfor non-linear structures.
Practice The gain of reduction
Build make_pipeline(StandardScaler(), PCA(n_components=20), SVC()) on the digit data. Compute the 5-fold CV accuracy first as is, then with set_params(pca="passthrough"). Print the two scores with round(..., 3) on one line (with PCA first).
Starter code
import numpy as np
from sklearn.datasets import load_digits
X, y = load_digits(return_X_y=True)
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.svm import SVC
from sklearn.model_selection import cross_val_score
# buraya yazSample solution
import numpy as np
from sklearn.datasets import load_digits
X, y = load_digits(return_X_y=True)
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.svm import SVC
from sklearn.model_selection import cross_val_score
pipe = make_pipeline(StandardScaler(), PCA(n_components=20), SVC())
a = cross_val_score(pipe, X, y, cv=5).mean()
pipe.set_params(pca="passthrough")
b = cross_val_score(pipe, X, y, cv=5).mean()
print(round(a, 3), round(b, 3))Expected output
0.941 0.946
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 Which can't be used for dimensionality reduction inside a model pipeline?
- PCA
- KernelPCA
- SelectKBest
- TSNE Correct answer
Explanation: TSNE has no transform method; the test data can't be moved to the same space.
2 Turn the reduction step off for comparison.
Question
pipe.set_params(pca=____)
Answer
pipe.set_params(pca="passthrough")Explanation: "passthrough" turns the step off so the data passes through unchanged.
3 Match the tools.
| Term | Meaning |
|---|---|
load_digits |
8x8 digit images |
TSNE |
2-D visualisation |
KernelPCA |
Curved structures |
explained_variance_ratio_ |
Share of information on the map |