Machine Learning
Confusion matrix
A confusion matrix shows classification results in detail.
Confusion matrix
The confusion matrix shows which class was confused with which. In a binary problem there are four cells:
from sklearn.metrics import confusion_matrix
y_true = [1, 0, 1, 1, 0, 0]
y_pred = [1, 0, 0, 1, 1, 0]
print(confusion_matrix(y_true, y_pred))
# [[2 1] row: true 0 -> TN=2, FP=1
# [1 2]] row: true 1 -> FN=1, TP=2
Rows are the true classes and columns the predictions. The diagonal counts correct predictions, the off-diagonal counts errors.
Reading the cells
With ravel() the four values can be unpacked in order. Which error is costlier depends on the problem:
tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()
# spam filter: FP (important e-mail went to spam) is costly
# cancer screening: FN (patient missed) is costly
In a multi-class problem the matrix becomes k×k; large off-diagonal values show which classes the model can't separate.
Exercise Get the matrix
Get test predictions on the breast cancer data with a scaled logistic regression pipeline. Print the confusion matrix, then unpack it with ravel() and print the FN and FP counts separated by a space.
Show example solution
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
from sklearn.metrics import confusion_matrix
m = make_pipeline(StandardScaler(), LogisticRegression()).fit(X_train, y_train)
cm = confusion_matrix(y_test, m.predict(X_test))
print(cm)
tn, fp, fn, tp = cm.ravel()
print(fn, fp)
Test yourself
-
Calling a sick person "healthy" falls into which cell?
- True positive (TP)
- False negative (FN)
- False positive (FP)
- True negative (TN)
Show answer
False negative (FN) — Truly positive (sick), predicted negative (healthy): a false negative.
-
Unpack the matrix into four values.
tn, fp, fn, tp = confusion_matrix(y_true, y_pred).____()
Show answer
tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()
-
Match the cells.
- TP
- Positive, predicted right
- TN
- Negative, predicted right
- FP
- False alarm
- FN
- Missed positive
Next lesson: Summary - Classification
