Machine Learning
explained_variance_
explained_variance_ gives the variance ratio of each component.
explained_variance_ratio_
explained_variance_ratio_ gives how much of the total variance each component explains. Its cumulative sum answers "what percentage of the information do the first k components keep?":
pca = PCA().fit(Xs)
print(pca.explained_variance_ratio_[:3].round(3))
kum = np.cumsum(pca.explained_variance_ratio_)
print(np.argmax(kum >= 0.9) + 1) # components needed for 90%
When these values are drawn as a line plot it is called a "scree plot"; the elbow is a hint for the number of components.
Reading component loadings
The rows of components_ are the components; the values in each row (loadings) show which features that component is made of. Loadings with large absolute values give the component's meaning:
d = load_wine()
yuk = pca.components_[0]
for i in np.argsort(-abs(yuk))[:3]:
print(d.feature_names[i], round(yuk[i], 2))
The sign of a loading alone isn't meaningful (a component can be flipped); what matters is the features' signs relative to each other within the same component.
Exercise Explained variance
Train a full PCA on the scaled wine data. Print the variance ratios of the first three components with round(3), then the number of components needed to reach 80% of the variance with np.argmax(kum >= 0.8) + 1.
Show example solution
import numpy as np
from sklearn.datasets import load_wine
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
X, y = load_wine(return_X_y=True)
Xs = StandardScaler().fit_transform(X)
pca = PCA().fit(Xs)
print(pca.explained_variance_ratio_[:3].round(3))
kum = np.cumsum(pca.explained_variance_ratio_)
print(np.argmax(kum >= 0.8) + 1)
Test yourself
-
What is the sum of <code>explained_variance_ratio_</code> over all components?
- 0
- The total variance
- 1
- The number of features
Show answer
1 — The ratios are values divided by the total variance; they sum to 1.
-
Take the cumulative sum of the explained variance.
kum = np.____(pca.explained_variance_ratio_)
Show answer
kum = np.cumsum(pca.explained_variance_ratio_)
-
Match the attributes.
- explained_variance_
- Component variance
- explained_variance_ratio_
- Variance ratio
- components_
- Loadings
- mean_
- Centring mean
Next lesson: Summary - PCA
