What is imbalanced data?
- Machine Learning course
- Intermediate
- Lesson 87/112
- 5 min
Imbalanced data has classes with unequal numbers of examples.
What is imbalanced data?
If one class has far fewer examples than the others, the data is imbalanced: fraudulent transactions (0.1%), rare diseases (1%), machine failures. A model gets high accuracy by predicting the majority class but misses the minority that is actually sought.
print(np.bincount(y)) # [1900 100]
print(np.bincount(y) / len(y)) # [0.95 0.05]Imbalance isn't an error but the nature of the problem; the fix is the right metric and the right training strategy.
The effect of imbalance
Because most models try to minimise the total error, they disregard the minority's errors. The result: the minority class's recall drops and the decision boundary shifts into the minority.
m = LogisticRegression().fit(X_train, y_train)
p = m.predict(X_test)
print(accuracy_score(y_test, p)) # high
print(recall_score(y_test, p)) # minority: lowIn this group you'll see the remedies: resampling, class weights, threshold tuning and the right metric.
Practice See the problem
Print the class distribution. Train a LogisticRegression(); print the test accuracy and the minority recall with round(..., 3) on one line.
Starter code
import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
X, y = make_classification(n_samples=2000, n_features=10, n_informative=4, weights=[0.95, 0.05], flip_y=0, random_state=1)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, recall_score
# buraya yazSample solution
import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
X, y = make_classification(n_samples=2000, n_features=10, n_informative=4, weights=[0.95, 0.05], flip_y=0, random_state=1)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, recall_score
print(np.bincount(y))
m = LogisticRegression().fit(X_train, y_train)
p = m.predict(X_test)
print(round(accuracy_score(y_test, p), 3), round(recall_score(y_test, p), 3))Expected output
[1900 100] 0.95 0.033
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 With a 5% minority, what is the accuracy of a model that says "majority" to everything?
- 0
- 0.95 Correct answer
- 0.05
- 0.5
Explanation: Accuracy looks high but the minority recall is 0.
2 Compute the class proportions.
print(np.bincount(y) / ____)print(np.bincount(y) / len(y))Explanation: Dividing the counts by the number of examples gives the proportions.
3 Match the examples.
| Term | Meaning |
|---|---|
fraud |
Very rare minority |
majority prediction |
High accuracy, zero recall |
stratify |
Keeps the proportion |
recall |
Catching the minority |