Preparing data
- Machine Learning course
- Intermediate
- Lesson 65/112
- 7 min
Prepare data for clustering.
Project: customer segmentation
In this project we'll split a shop's 400 customers into groups by monthly spending, number of visits and average basket size. There are no labels; the goal is to give the marketing team meaningful segments.
harcama = ... # monthly TL
ziyaret = ... # monthly visits
sepet = harcama / ziyaret
X = np.column_stack([harcama, ziyaret, sepet])Basket size is a derived feature: it separates the "big basket" shopper who spends the same amount in few visits from the frequent "small basket" shopper.
Preparation
First the feature ranges are checked, then each is scaled to mean 0 and standard deviation 1. Since clustering is unlabelled there is no train/test split; all customers are clustered.
print(X.min(axis=0).round(0), X.max(axis=0).round(0))
Xs = StandardScaler().fit_transform(X)
print(Xs.std(axis=0).round(2)) # [1. 1. 1.]When new customers arrive the same scaler and km.predict will be used, so the scaler object must be kept.
Practice Inspect the customer data
Print the shape of X, then its column means with np.round(..., 0). Scale with StandardScaler and print the column standard deviations of the scaled data with np.round(..., 2).
Starter code
import numpy as np
rng = np.random.default_rng(5)
n = 400
grup = rng.integers(0, 3, n)
harcama = np.array([300, 1500, 800])[grup] + rng.normal(0, 120, n)
ziyaret = np.array([2, 4, 12])[grup] + rng.normal(0, 1.2, n)
sepet = harcama / np.maximum(ziyaret, 1)
X = np.column_stack([harcama, ziyaret, sepet])
from sklearn.preprocessing import StandardScaler
# buraya yazSample solution
import numpy as np
rng = np.random.default_rng(5)
n = 400
grup = rng.integers(0, 3, n)
harcama = np.array([300, 1500, 800])[grup] + rng.normal(0, 120, n)
ziyaret = np.array([2, 4, 12])[grup] + rng.normal(0, 1.2, n)
sepet = harcama / np.maximum(ziyaret, 1)
X = np.column_stack([harcama, ziyaret, sepet])
from sklearn.preprocessing import StandardScaler
print(X.shape)
print(np.round(X.mean(axis=0), 0))
Xs = StandardScaler().fit_transform(X)
print(np.round(Xs.std(axis=0), 2))Expected output
(400, 3) [858. 6. 224.] [1. 1. 1.]
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 Why is scaling a must on this data?
- Because there are no labels
- It isn't needed
- Spending is hundreds of TL, visits single digits; spending dominates the distance Correct answer
- Clustering speeds up with scaling
Explanation: Unscaled KMeans in practice clusters only by spending.
2 Scale the features.
Xs = StandardScaler().____(X)Xs = StandardScaler().fit_transform(X)Explanation: fit_transform learns and applies the mean and std.
3 Match the features.
| Term | Meaning |
|---|---|
harcama |
Monthly TL |
ziyaret |
Monthly visit count |
sepet |
Derived: TL / visit |
Xs |
Scaled matrix |