Machine Learning
Prediction
Make predictions with SVM.
Prediction and threshold
With the chosen model, decision scores are taken for the test customers. In SVC, decision_function > 0 means class 1. If the business wants to catch more customers, the threshold can be pulled below 0:
skor = gs.decision_function(X_test)
varsayilan = (skor > 0).astype(int) # the same as predict
hassas = (skor > -0.5).astype(int) # more alarms
Changing the threshold shifts the precision-recall balance without retraining the model.
A priority list
The business team often needs a ranked list like "the 20 riskiest customers". Scores are sorted in descending order and the first k customers are taken; the share who really leave in this list is called precision@k:
sira = np.argsort(-skor)[:20]
print(y_test[sira].mean()) # share of real leavers in the top 20
If the campaign budget is limited, this metric is a more direct business measure than F1.
Exercise Riskiest customers
Get the test decision scores with the given gs. Print on one line how many customers are flagged with the default and the -0.5 threshold. Then print the share of real leavers among the 20 riskiest customers with round(..., 2).
Show example solution
import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
X, y = make_classification(n_samples=600, n_features=8, n_informative=4, weights=[0.8, 0.2], random_state=7)
rng = np.random.default_rng(7)
X[rng.random(X.shape) < 0.05] = np.nan
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
from sklearn.pipeline import make_pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV
pipe = make_pipeline(SimpleImputer(strategy="median"), StandardScaler(), SVC(class_weight="balanced"))
gs = GridSearchCV(pipe, {"svc__C": [0.1, 1, 10], "svc__gamma": ["scale", 0.1]}, cv=5, scoring="f1").fit(X_train, y_train)
skor = gs.decision_function(X_test)
print((skor > 0).sum(), (skor > -0.5).sum())
sira = np.argsort(-skor)[:20]
print(round(y_test[sira].mean(), 2))
Test yourself
-
How does pulling the threshold from 0 to -0.5 affect recall?
- Zeroes it
- Never affects it
- Raises it (or leaves it the same)
- Lowers it
Show answer
Raises it (or leaves it the same) — More examples are counted positive, so fewer positives are missed.
-
Get the indices sorting the scores from largest to smallest.
sira = np.argsort(____)[:20]
Show answer
sira = np.argsort(-skor)[:20]
-
Match the expressions.
- skor > 0
- Default decision
- skor > -0.5
- Sensitive decision
- np.argsort(-skor)
- Riskiest first
- y_test[sira].mean()
- precision@k
Next lesson: Evaluation
