What is scaling?

Scaling brings feature values to a specific range.

What is scaling?

Features can come in different units: age 20-60, salary 10,000-90,000. Scaling moves them all into a comparable range so none dominates just because of its unit.

a = np.array([30, 40000])
b = np.array([60, 41000])
print(np.linalg.norm(a - b))   # ~1000.4  the age gap (30) is almost invisible

Scaling doesn't change the shape of the data; it only stretches or squeezes the axes.

Which models are affected?

Affected: models using distances or gradients (KNN, SVM, K-means, PCA, regularised linear models, neural networks). Not affected: tree-based models (decision tree, random forest, gradient boosting).

# affected: KNN, SVC, LogisticRegression(C=...), Ridge, Lasso, KMeans, PCA, MLP
# not affected: DecisionTree, RandomForest, GradientBoosting

In regularisation (Ridge/Lasso) the penalty is applied equally to all weights; if scales differ, the penalty is distributed unfairly.

Exercise See the dominance

Compute the distance between a and b, round it to 1 digit and print it. Then divide salary by 1000 (thousand TL), recompute the same distance and print it.

Show example solution

import numpy as np

a = np.array([30.0, 40000.0])
b = np.array([60.0, 41000.0])

print(round(np.linalg.norm(a - b), 1))
olcek = np.array([1, 1000])
print(round(np.linalg.norm(a / olcek - b / olcek), 1))

Test yourself

In unscaled data, what mostly determines the distance between two examples?
- The most important feature
- The first column
- The label
- The feature with the widest range
Show answer

The feature with the widest range — Since squared differences are summed, the large-valued feature crushes the total.

Compute the vector length.

d = np.linalg.____(a - b)

Show answer

d = np.linalg.norm(a - b)

Match the models with their scale sensitivity.

KNN

Uses distance, affected
RandomForest

Uses thresholds, unaffected
Ridge

Penalty becomes unfair
PCA

Looks at variance, affected

Next lesson: StandardScaler

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies