Accuracy

Accuracy shows the ratio of correct predictions.

Multi-class accuracy

In a multi-class problem the overall accuracy can look good while some classes are predicted badly. Per-class accuracy is found by dividing the confusion matrix diagonal by the row sums:

cm = confusion_matrix(y_true, y_pred)
print(cm.diagonal() / cm.sum(axis=1))   # each class's own accuracy

If the big class is predicted well and the small one badly, overall accuracy hides it.

Top-k accuracy

In multi-class problems (for example image recognition with 1000 classes), the correct class being among the k most likely predictions can count as success too:

from sklearn.metrics import top_k_accuracy_score
olasilik = [[0.5, 0.3, 0.2], [0.1, 0.3, 0.6], [0.4, 0.4, 0.2]]
print(top_k_accuracy_score([1, 2, 1], olasilik, k=1))   # 0.333
print(top_k_accuracy_score([1, 2, 1], olasilik, k=2))   # 1.0

In search and recommendation systems, "is the right one in the top 5 results?" is exactly this question.

Exercise Per-class accuracy

Print the overall accuracy for the given labels. Then print the per-class accuracy from the confusion matrix (cm.diagonal() / cm.sum(axis=1)) with round(3).

Show example solution

from sklearn.metrics import accuracy_score, confusion_matrix

y_true = [0, 0, 0, 0, 0, 0, 1, 1, 2, 2]
y_pred = [0, 0, 0, 0, 0, 1, 1, 0, 2, 2]

print(accuracy_score(y_true, y_pred))
cm = confusion_matrix(y_true, y_pred)
print((cm.diagonal() / cm.sum(axis=1)).round(3))

Test yourself

What is the accuracy of a model that gets all 90 examples of class A and none of the 10 of class B?
- 1.0
- 0.1
- 0.9
- 0.5
Show answer

0.9 — 90 / 100 = 0.9; but class B's accuracy is 0, and balanced accuracy would be 0.5.

Measure whether the correct class is in the top 3 predictions.

top_k_accuracy_score(y_true, olasilik, ____=3)

Show answer

top_k_accuracy_score(y_true, olasilik, k=3)

Match the measures.

overall accuracy

Share of all correct
per-class accuracy

Diagonal / row sum
top-k accuracy

Correct within the top k
balanced accuracy

Mean of class accuracies

Next lesson: Precision

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies