Summary - PCA

In this group you learned PCA, variance, and n_components.

Summary: PCA

In this group:

PCA rotates the data towards the directions of largest variance; the components are perpendicular.
Scaling first is a must; otherwise the large-unit feature dominates.
Component variances are the eigenvalues of the covariance matrix; PCA with all components is a lossless rotation.
n_components: a count or a variance ratio; best chosen with CV inside a pipeline.
inverse_transform is an approximate inverse; explained_variance_ratio_ and the loadings give interpretation.

Exercise Classification with PCA

Search the pca__n_components values among [1, 2, 3, 5, 8] for make_pipeline(StandardScaler(), PCA(), LogisticRegression(max_iter=1000)) with GridSearchCV(..., cv=5). Print best_params_ and best_score_ (round(..., 3)) on separate lines.

Show example solution

from sklearn.datasets import load_wine
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import GridSearchCV

X, y = load_wine(return_X_y=True)

pipe = make_pipeline(StandardScaler(), PCA(), LogisticRegression(max_iter=1000))
gs = GridSearchCV(pipe, {"pca__n_components": [1, 2, 3, 5, 8]}, cv=5).fit(X, y)
print(gs.best_params_)
print(round(gs.best_score_, 3))

Test yourself

Which kind of relationships can't PCA capture?
- It captures them all
- Non-linear structures (e.g. a spiral)
- Linear correlations
- Variance differences
Show answer

Non-linear structures (e.g. a spiral) — PCA is a linear transformation; curved structures need methods like t-SNE or UMAP.

Reduce while keeping 95% of the variance.

pca = PCA(n_components=____)

Show answer

pca = PCA(n_components=0.95)

Match the tools.

PCA

Linear dimensionality reduction
StandardScaler

Before PCA
np.linalg.eigvalsh

Eigenvalues
GridSearchCV

Choosing the component count

Next lesson: What is dimensionality reduction?

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies