What is scaling?
- Machine Learning course
- Beginner
- Lesson 75/112
- 5 min
Scaling brings feature values to a specific range.
What is scaling?
Features can come in different units: age 20-60, salary 10,000-90,000. Scaling moves them all into a comparable range so none dominates just because of its unit.
a = np.array([30, 40000])
b = np.array([60, 41000])
print(np.linalg.norm(a - b)) # ~1000.4 the age gap (30) is almost invisibleScaling doesn't change the shape of the data; it only stretches or squeezes the axes.
Which models are affected?
Affected: models using distances or gradients (KNN, SVM, K-means, PCA, regularised linear models, neural networks). Not affected: tree-based models (decision tree, random forest, gradient boosting).
# affected: KNN, SVC, LogisticRegression(C=...), Ridge, Lasso, KMeans, PCA, MLP
# not affected: DecisionTree, RandomForest, GradientBoostingIn regularisation (Ridge/Lasso) the penalty is applied equally to all weights; if scales differ, the penalty is distributed unfairly.
Practice See the dominance
Compute the distance between a and b, round it to 1 digit and print it. Then divide salary by 1000 (thousand TL), recompute the same distance and print it.
Starter code
import numpy as np
a = np.array([30.0, 40000.0])
b = np.array([60.0, 41000.0])
# buraya yazSample solution
import numpy as np
a = np.array([30.0, 40000.0])
b = np.array([60.0, 41000.0])
print(round(np.linalg.norm(a - b), 1))
olcek = np.array([1, 1000])
print(round(np.linalg.norm(a / olcek - b / olcek), 1))Expected output
1000.4 30.0
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 In unscaled data, what mostly determines the distance between two examples?
- The most important feature
- The first column
- The label
- The feature with the widest range Correct answer
Explanation: Since squared differences are summed, the large-valued feature crushes the total.
2 Compute the vector length.
d = np.linalg.____(a - b)d = np.linalg.norm(a - b)Explanation: np.linalg.norm gives the Euclidean length by default.
3 Match the models with their scale sensitivity.
| Term | Meaning |
|---|---|
KNN |
Uses distance, affected |
RandomForest |
Uses thresholds, unaffected |
Ridge |
Penalty becomes unfair |
PCA |
Looks at variance, affected |