Machine Learning
fit_transform
fit_transform trains PCA and transforms the data.
fit_transform and inverse_transform
fit_transform learns the components and moves the data into component space. inverse_transform brings it back; since the dropped components' information is lost, the result is an approximation of the original data:
pca = PCA(n_components=4)
Z = pca.fit_transform(Xs) # (178, 4)
X_geri = pca.inverse_transform(Z) # (178, 13), approximate
hata = ((Xs - X_geri) ** 2).mean()
print(round(hata, 3))
The reconstruction error is proportional to the dropped variance; it is also used in anomaly detection: examples that are hard to reconstruct are unusual.
A model in component space
PCA is often used before a model: fewer dimensions, faster training and combining correlated features. The right use is again inside a pipeline so that PCA learns only on training data:
pipe = make_pipeline(StandardScaler(), PCA(n_components=5), LogisticRegression(max_iter=1000))
print(cross_val_score(pipe, X, y, cv=5).mean())
The components are no longer original features; they are abstract columns like "component 1", which makes interpretation harder.
Exercise Reconstruction error
For component counts 2, 6 and 13, reduce the scaled data with PCA and transform it back. On each line print the component count and the mean squared reconstruction error with round(..., 3).
Show example solution
import numpy as np
from sklearn.datasets import load_wine
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
X, y = load_wine(return_X_y=True)
Xs = StandardScaler().fit_transform(X)
for k in [2, 6, 13]:
pca = PCA(n_components=k)
X_geri = pca.inverse_transform(pca.fit_transform(Xs))
print(k, round(((Xs - X_geri) ** 2).mean(), 3))
Test yourself
-
If all components are kept, what does <code>inverse_transform(transform(X))</code> give?
- A zero matrix
- An approximate X
- An error
- X itself (to floating-point precision)
Show answer
X itself (to floating-point precision) — A full rotation is lossless; so is the inverse.
-
Go back from component space to the original space.
X_geri = pca.____(Z)
Show answer
X_geri = pca.inverse_transform(Z)
-
Match the methods.
- fit_transform
- Learn and reduce
- transform
- Reduce new data
- inverse_transform
- Approximate inverse
- reconstruction error
- Dropped information
Next lesson: explained_variance_
