Machine Learning
Feature extraction
Derive new features.
Project: house prices
In this project we'll predict the prices of 500 houses. Features: square metres, room count, building age, floor, distance to the centre and five meaningless noise columns. The project's focus is feature engineering and selection.
X = np.column_stack([m2, oda, yas, kat, merkez_km, gurultu])
print(X.shape) # (500, 10)
In real projects nobody knows which column is noise; here we added them on purpose so we can see whether the selection methods find them.
Feature extraction
We derive two new features from domain knowledge: square metres per room and the logarithm of the distance to the centre (the first kilometres make more difference than the last ones):
m2_oda = m2 / oda
log_merkez = np.log1p(merkez_km)
X2 = np.column_stack([X, m2_oda, log_merkez])
print(X2.shape) # (500, 12)
Whether derived features help is again measured by validation; intuition is a hypothesis, not a result.
Exercise Derive the features
Build X2 by adding the columns m2 / oda and np.log1p(merkez_km) to X. Print X2.shape, then the means of the two new columns with np.round(..., 2).
Show example solution
import numpy as np
rng = np.random.default_rng(11)
n = 500
m2 = rng.uniform(40, 200, n)
oda = np.clip(np.round(m2 / 35 + rng.normal(0, 0.7, n)), 1, 6)
yas = rng.uniform(0, 40, n)
kat = rng.integers(0, 15, n)
merkez_km = rng.uniform(0.5, 25, n)
gurultu = rng.normal(size=(n, 5))
fiyat = 20 * m2 + 150 * m2 / oda - 2500 * np.log1p(merkez_km) - 40 * yas + 60 * kat + rng.normal(0, 300, n)
X = np.column_stack([m2, oda, yas, kat, merkez_km, gurultu])
X2 = np.column_stack([X, m2 / oda, np.log1p(merkez_km)])
print(X2.shape)
print(np.round(X2[:, -2:].mean(axis=0), 2))
Test yourself
-
Why might square metres per room be a useful derived feature?
- It increases the row count
- It is required
- It describes how spacious the house is in one number
- It reduces noise
Show answer
It describes how spacious the house is in one number — Fewer rooms at the same area mean larger rooms; the two raw columns don't say this directly.
-
Add the new columns to the feature matrix.
X2 = np.____([X, m2_oda, log_merkez])
Show answer
X2 = np.column_stack([X, m2_oda, log_merkez])
-
Match the features.
- m2
- Raw area
- m2 / oda
- Spaciousness
- log1p(merkez_km)
- Distance with diminishing effect
- gurultu
- Meaningless columns
Next lesson: Feature selection
