Summary - Metrics

In this group you learned accuracy, precision, recall, and F1 metrics.

Summary: metrics

In this group you deepened the core metrics:

Loss is minimised in training, a metric is interpreted; log-loss measures probability quality.
Per-class accuracy and top-k accuracy with many classes.
macro, weighted, micro averages; micro equals accuracy in single-label problems.
The ROC curve and threshold-independent AUC.
Adjusting the precision-recall balance with F-beta; choosing the threshold on validation data.

Exercise The best F1 threshold

Run precision_recall_curve for probabilities p and labels y, compute each threshold's F1 and print the threshold giving the highest F1.

Show example solution

import numpy as np
from sklearn.metrics import precision_recall_curve

y = np.array([0, 0, 1, 0, 1, 1, 0, 1])
p = np.array([0.1, 0.3, 0.35, 0.4, 0.6, 0.7, 0.75, 0.9])

pr, re, esik = precision_recall_curve(y, p)
f1 = 2 * pr * re / (pr + re + 1e-12)
print(esik[f1[:-1].argmax()])

Test yourself

Which fits comparing the ranking power of two classifiers without choosing a threshold?
- F1
- Precision
- ROC AUC
- Accuracy
Show answer

ROC AUC — AUC evaluates all thresholds together.

Get multi-class F1 with equal weight for classes.

f1_score(y_true, y_pred, average=____)

Show answer

f1_score(y_true, y_pred, average="macro")

Match the metrics.

log_loss

Probability quality
top_k_accuracy_score

Correct within top k
roc_auc_score

Ranking power
fbeta_score

Adjustable balance

Next lesson: What is visualization?

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies