Machine Learning
n_components
n_components sets the number of dimensions to reduce to.
Choosing n_components
n_components can be an integer (number of components) or a ratio between 0 and 1. If a ratio is given, PCA itself picks the fewest components that preserve that much variance:
pca = PCA(n_components=0.9).fit(Xs) # 90% of the variance
print(pca.n_components_) # chosen component count
2 or 3 for visualisation, 90-95% of the variance for compression before a model are common choices.
Choosing inside a pipeline
The number of components is a hyperparameter; the most accurate choice is made with cross-validation by looking at the next model's success:
pipe = make_pipeline(StandardScaler(), PCA(), LogisticRegression(max_iter=1000))
gs = GridSearchCV(pipe, {"pca__n_components": [2, 4, 6, 8]}, cv=5).fit(X, y)
print(gs.best_params_)
A variance ratio isn't a rule but a starting point; a low-variance component might carry the information that separates the classes.
Exercise Choice by ratio
Train PCA on the scaled wine data for n_components 0.8, 0.9 and 0.95. On each line print the ratio and the chosen number of components (n_components_).
Show example solution
import numpy as np
from sklearn.datasets import load_wine
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
X, y = load_wine(return_X_y=True)
Xs = StandardScaler().fit_transform(X)
for oran in [0.8, 0.9, 0.95]:
print(oran, PCA(n_components=oran).fit(Xs).n_components_)
Test yourself
-
What does <code>PCA(n_components=0.95)</code> do?
- Picks the fewest components keeping 95% of the variance
- Keeps 0.95 components
- Drops 95% of the features
- It errors
Show answer
Picks the fewest components keeping 95% of the variance — The chosen number is in the n_components_ attribute after fit.
-
Search the number of components in a grid.
gs = GridSearchCV(pipe, {"____": [2, 4, 6]}, cv=5)
Show answer
gs = GridSearchCV(pipe, {"pca__n_components": [2, 4, 6]}, cv=5)
-
Match the values.
- n_components=2
- Visualisation
- n_components=0.95
- Choice by variance ratio
- n_components_
- Chosen count
- pca__n_components
- Grid parameter
Next lesson: fit_transform
