Machine Learning
Numerical features
Numerical features are scaled to improve model performance.
Numeric transformations
Numeric features can also be transformed to make them more useful. For right-skewed distributions like income, the logarithm compresses the extreme values:
gelir = np.array([20, 25, 30, 40, 500])
print(np.log1p(gelir).round(2)) # [3.04 3.26 3.43 3.71 6.22]
np.log1p(x) means log(1 + x) and works safely at 0 too.
Binning and polynomials
A continuous value can be split into ranges (KBinsDiscretizer, np.digitize). To teach a linear model a curved relationship, PolynomialFeatures adds squares and products:
yas = np.array([15, 25, 45, 70])
print(np.digitize(yas, [18, 40, 65])) # [0 1 2 3] young, adult, middle, old
from sklearn.preprocessing import PolynomialFeatures
print(PolynomialFeatures(2, include_bias=False).fit_transform([[2, 3]]))
# [[2. 3. 4. 6. 9.]] x1, x2, x1², x1·x2, x2²
The number of polynomial features grows fast; degree 2 or 3 is usually enough.
Exercise Teach a curve to a linear model
For the relationship y = x², train a plain LinearRegression and a LinearRegression extended with PolynomialFeatures(2). Print the R² score (score) of both with round(..., 3) on separate lines.
Show example solution
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import PolynomialFeatures
X = np.arange(-5, 6).reshape(-1, 1)
y = X.ravel() ** 2
duz = LinearRegression().fit(X, y)
Xp = PolynomialFeatures(2).fit_transform(X)
egri = LinearRegression().fit(Xp, y)
print(round(duz.score(X, y), 3))
print(round(egri.score(Xp, y), 3))
Test yourself
-
For which kind of distribution is the log transform most useful?
- Right-skewed, long-tailed
- Symmetric
- Categorical
- Negative-valued
Show answer
Right-skewed, long-tailed — It compresses large values and makes the distribution more symmetric.
-
Split the values into groups by the boundaries.
grup = np.____(yas, [18, 40, 65])
Show answer
grup = np.digitize(yas, [18, 40, 65])
-
Match the transformations with their purposes.
- np.log1p
- Reduces skew
- np.digitize
- Splits into ranges
- PolynomialFeatures
- Adds curved relations
- x1·x2
- Interaction feature
Next lesson: Summary - Feature engineering
