Summary - PCA
- Machine Learning course
- Intermediate
- Lesson 74/112
- 6 min
In this group you learned PCA, variance, and n_components.
Summary: PCA
In this group:
- PCA rotates the data towards the directions of largest variance; the components are perpendicular.
- Scaling first is a must; otherwise the large-unit feature dominates.
- Component variances are the eigenvalues of the covariance matrix; PCA with all components is a lossless rotation.
n_components: a count or a variance ratio; best chosen with CV inside a pipeline.inverse_transformis an approximate inverse;explained_variance_ratio_and the loadings give interpretation.
Practice Classification with PCA
Search the pca__n_components values among [1, 2, 3, 5, 8] for make_pipeline(StandardScaler(), PCA(), LogisticRegression(max_iter=1000)) with GridSearchCV(..., cv=5). Print best_params_ and best_score_ (round(..., 3)) on separate lines.
Starter code
from sklearn.datasets import load_wine
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import GridSearchCV
X, y = load_wine(return_X_y=True)
# buraya yazSample solution
from sklearn.datasets import load_wine
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import GridSearchCV
X, y = load_wine(return_X_y=True)
pipe = make_pipeline(StandardScaler(), PCA(), LogisticRegression(max_iter=1000))
gs = GridSearchCV(pipe, {"pca__n_components": [1, 2, 3, 5, 8]}, cv=5).fit(X, y)
print(gs.best_params_)
print(round(gs.best_score_, 3))Expected output
{'pca__n_components': 8}
0.978
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 Which kind of relationships can't PCA capture?
- It captures them all
- Non-linear structures (e.g. a spiral) Correct answer
- Linear correlations
- Variance differences
Explanation: PCA is a linear transformation; curved structures need methods like t-SNE or UMAP.
2 Reduce while keeping 95% of the variance.
Question
pca = PCA(n_components=____)
Answer
pca = PCA(n_components=0.95)Explanation: A value between 0 and 1 is interpreted as a variance ratio.
3 Match the tools.
| Term | Meaning |
|---|---|
PCA |
Linear dimensionality reduction |
StandardScaler |
Before PCA |
np.linalg.eigvalsh |
Eigenvalues |
GridSearchCV |
Choosing the component count |