What is PCA?
- Machine Learning course
- Intermediate
- Lesson 69/112
- 5 min
PCA reduces dimensions while preserving the most variance in data.
What is PCA?
Principal component analysis (PCA) is a linear transformation that rotates the data towards the directions in which it spreads most. The first component is the direction of largest variance, the second is perpendicular to it with the largest remaining variance, and so on.
from sklearn.decomposition import PCA
pca = PCA(n_components=2)
Z = pca.fit_transform(Xs) # 13 features -> 2 components
print(Z.shape)If a few components carry most of the data's information, the dimension can be reduced a lot.
Scale first
Since PCA looks at variance, the feature with large units takes over the first component. On the wine data, if PCA is done without scaling, the first component is almost entirely proline (~1000):
ham = PCA(n_components=1).fit(X)
print(abs(ham.components_[0]).argmax()) # 12: proline
olcekli = PCA(n_components=1).fit(Xs)
print(abs(olcekli.components_[0]).round(2)) # the load spreads over many featuresWith features in different units, StandardScaler before PCA is almost always necessary.
Practice The effect of scale
Train a one-component PCA on the raw X and on the scaled Xs. For each, print the index of the feature with the largest absolute loading in the first component on one line (raw first).
Starter code
import numpy as np
from sklearn.datasets import load_wine
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
X, y = load_wine(return_X_y=True)
Xs = StandardScaler().fit_transform(X)
# buraya yazSample solution
import numpy as np
from sklearn.datasets import load_wine
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
X, y = load_wine(return_X_y=True)
Xs = StandardScaler().fit_transform(X)
ham = PCA(n_components=1).fit(X)
olcekli = PCA(n_components=1).fit(Xs)
print(abs(ham.components_[0]).argmax(), abs(olcekli.components_[0]).argmax())Expected output
12 6
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 How is PCA's second component related to the first?
- It is the same direction
- No relation, it is random
- It is perpendicular (uncorrelated) Correct answer
- It is parallel to it
Explanation: The components are perpendicular; there is no correlation between the transformed columns.
2 Reduce the data to two principal components.
Z = PCA(____=2).fit_transform(Xs)Z = PCA(n_components=2).fit_transform(Xs)Explanation: n_components is the number of components to keep.
3 Match the concepts.
| Term | Meaning |
|---|---|
1st component |
Direction of largest variance |
2nd component |
Perpendicular to the first |
components_ |
Component directions |
scaling |
Prevents unit dominance |