Class weight
- Machine Learning course
- Intermediate
- Lesson 89/112
- 5 min
The class_weight parameter gives more weight to the minority class.
class_weight
Instead of changing the data, the minority's errors can be given more weight in the model's loss function. class_weight="balanced" weights each class by n / (k · n_class):
m = LogisticRegression(class_weight="balanced").fit(X_train, y_train)
from sklearn.utils.class_weight import compute_class_weight
print(compute_class_weight("balanced", classes=np.array([0, 1]), y=y_train))
# [0.526 10.] a minority error is ~19 times costlierBecause it generates no new examples it is fast, and it is available directly in most scikit-learn models (logistic, SVM, trees, forests).
Manual weights
The weights can also be given manually with a dict, so the real business cost of errors is reflected in the model. For example, if missing a fraud is 50 times costlier than a false alarm:
m = LogisticRegression(class_weight={0: 1, 1: 50})
# or as example weights in fit:
w = np.where(y_train == 1, 50, 1)
m.fit(X_train, y_train, sample_weight=w)As the weight grows recall rises and precision falls; the best value is chosen by validation.
Practice Weighted model
Train an unweighted and a class_weight="balanced" logistic regression. For each print the test precision and recall with round(..., 3) on one line (unweighted first).
Starter code
import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
X, y = make_classification(n_samples=2000, n_features=10, n_informative=4, weights=[0.95, 0.05], flip_y=0, random_state=1)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import precision_score, recall_score
# buraya yazSample solution
import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
X, y = make_classification(n_samples=2000, n_features=10, n_informative=4, weights=[0.95, 0.05], flip_y=0, random_state=1)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import precision_score, recall_score
for cw in [None, "balanced"]:
p = LogisticRegression(class_weight=cw).fit(X_train, y_train).predict(X_test)
print(round(precision_score(y_test, p), 3), round(recall_score(y_test, p), 3))Expected output
0.5 0.033 0.19 0.9
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 With 1000 examples of which 900 majority and 100 minority, what are the balanced weights?
- 1 and 9
- 0.9 and 0.1
- 1 and 1
- 0.556 and 5.0 Correct answer
Explanation: 1000 / (2 · 900) ≈ 0.556; 1000 / (2 · 100) = 5.
2 Give the minority class 20 times the weight.
m = LogisticRegression(class_weight=____)m = LogisticRegression(class_weight={0: 1, 1: 20})Explanation: The dict gives each class's weight separately.
3 Match the weighting forms.
| Term | Meaning |
|---|---|
"balanced" |
n / (k · n_class) |
{0: 1, 1: 50} |
Manual cost |
sample_weight |
Per-example weight |
compute_class_weight |
Computes the weights |