CodeBlanks Logo Icon CodeBlanks
Login Register

What is dimensionality reduction?

Dimensionality reduction reduces feature count while preserving information.

What is dimensionality reduction?

Dimensionality reduction means representing many-feature data with fewer features while keeping most of the information. There are two families:

  • Feature selection: keeps a subset of the original columns (interpretable).
  • Feature extraction: produces new columns (PCA, t-SNE, UMAP).
from sklearn.datasets import load_digits
X, y = load_digits(return_X_y=True)
print(X.shape)   # (1797, 64): 8x8-pixel digit images

64 pixels are more than needed to describe a digit; neighbouring pixels are strongly correlated.

Constant and correlated columns

In image data, edge pixels are almost always 0; their variance is near zero and they carry no information. Correlated columns repeat the same information:

print((X.var(axis=0) == 0).sum())      # pixels that never change
C = np.corrcoef(X[:, X.var(axis=0) > 0], rowvar=False)
print((np.abs(C) > 0.8).sum())         # strongly correlated pairs

Both cases show why dimensionality reduction is possible.

Practice Get to know the digit data

Print the shape of the digit data, the number of pixels that never change and the number of classes (len(np.unique(y))) on separate lines.

Starter code

model.py
import numpy as np
from sklearn.datasets import load_digits

X, y = load_digits(return_X_y=True)

# buraya yaz

Sample solution

model.py
import numpy as np
from sklearn.datasets import load_digits

X, y = load_digits(return_X_y=True)

print(X.shape)
print((X.var(axis=0) == 0).sum())
print(len(np.unique(y)))

Expected output

(1797, 64)
3
10

Questions and answers

Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.

1 Which family does PCA belong to?

  • Feature extraction Correct answer
  • Feature selection
  • Clustering
  • Classification

Explanation: PCA produces linear combinations of the original columns, not the columns themselves.

2 Count the columns that never change.

Question 
print((X.____(axis=0) == 0).sum())
Answer 
print((X.var(axis=0) == 0).sum())

Explanation: A column with zero variance is constant.

3 Match the concepts.

Term Meaning
feature selection Subset of original columns
feature extraction New columns
constant column Carries no information
correlated columns Repeated information

All lessons in Intermediate

1 scikit-learn basics

  1. What is scikit-learn?
  2. Estimator API
  3. The fit method
  4. The predict method
  5. The score method
  6. Summary - scikit-learn

2 Pipeline

  1. What is a Pipeline?
  2. Steps
  3. fit and predict
  4. Advantages
  5. ColumnTransformer
  6. Summary - Pipeline

3 Cross-validation

  1. What is cross-validation?
  2. K-fold
  3. cross_val_score
  4. Stratified
  5. Advantages
  6. Summary - Cross-validation

4 Grid search

  1. What is grid search?
  2. GridSearchCV
  3. param_grid
  4. best_params_
  5. best_score_
  6. Summary - Grid search

5 SVM

  1. What is SVM?
  2. Margin
  3. Kernel
  4. C parameter
  5. Classification
  6. Summary - SVM

6 Project: SVM

  1. Preparing data
  2. Training model
  3. Prediction
  4. Evaluation

7 Random forest

  1. What is random forest?
  2. Ensemble
  3. n_estimators
  4. Bagging
  5. Feature importance
  6. Summary - Random forest

8 Boosting

  1. What is boosting?
  2. AdaBoost
  3. Gradient boosting
  4. Learning rate
  5. Advantages
  6. Summary - Boosting

9 Clustering

  1. What is clustering?
  2. KMeans
  3. Centroid
  4. Elbow method
  5. Labels
  6. Summary - Clustering

10 KMeans in depth

  1. KMeans algorithm
  2. Initialization
  3. n_init
  4. inertia_
  5. random_state
  6. Summary - KMeans

11 Hierarchical clustering

  1. What is hierarchical clustering?
  2. Dendrogram
  3. Agglomerative
  4. Linkage
  5. Advantages
  6. Summary - Hierarchical clustering

12 Project: Clustering

  1. Preparing data
  2. Training KMeans
  3. Label analysis
  4. Printing results

13 PCA

  1. What is PCA?
  2. Variance
  3. n_components
  4. fit_transform
  5. explained_variance_
  6. Summary - PCA

14 Dimensionality reduction

  1. What is dimensionality reduction?
  2. Why reduce?
  3. PCA vs t-SNE
  4. Visualization
  5. Advantages
  6. Summary - Dimensionality reduction

15 Feature selection

  1. What is feature selection?
  2. Variance threshold
  3. SelectKBest
  4. Recursive elimination
  5. Feature importance
  6. Summary - Feature selection

16 Imbalanced data

  1. What is imbalanced data?
  2. SMOTE
  3. Class weight
  4. Oversampling
  5. Metric selection
  6. Summary - Imbalanced data

17 Project: Features

  1. Feature extraction
  2. Feature selection
  3. Training model
  4. Evaluation

18 Ensemble methods

  1. What is ensemble?
  2. Voting
  3. Bagging
  4. Boosting
  5. Stacking
  6. Summary - Ensemble

19 Regularization

  1. What is regularization?
  2. L1 (Lasso)
  3. L2 (Ridge)
  4. alpha
  5. ElasticNet
  6. Summary - Regularization

20 Intermediate final

  1. Challenge 1
  2. Challenge 2
  3. Challenge 3
  4. Challenge 4