F1 score
- Machine Learning course
- Beginner
- Lesson 101/112
- 5 min
F1 is the harmonic mean of precision and recall.
F-beta
F1 gives precision and recall equal weight. F-beta adjusts the balance: β > 1 favours recall, β < 1 favours precision.
from sklearn.metrics import fbeta_score
print(fbeta_score(y_true, y_pred, beta=2)) # recall first
print(fbeta_score(y_true, y_pred, beta=0.5)) # precision firstF2 is the natural choice for disease screening, F0.5 for a spam filter.
The threshold that maximises F1
precision_recall_curve gives precision and recall for all thresholds; from there each threshold's F1 can be computed and the best chosen:
from sklearn.metrics import precision_recall_curve
pr, re, esik = precision_recall_curve(y_val, p_val)
f1 = 2 * pr * re / (pr + re + 1e-12)
en_iyi = esik[f1[:-1].argmax()]The threshold must be chosen on validation data; choosing it on test data again gives an optimistic result. The pr and re arrays are one longer than the thresholds, which is why f1[:-1] is used.
Practice Three F scores
Compute fbeta_score with beta 0.5, 1 and 2 for the given labels. On each line print beta and the score with round(..., 3).
Starter code
from sklearn.metrics import fbeta_score
y_true = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
y_pred = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]
# buraya yazSample solution
from sklearn.metrics import fbeta_score
y_true = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
y_pred = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]
for b in [0.5, 1, 2]:
print(b, round(fbeta_score(y_true, y_pred, beta=b), 3))Expected output
0.5 0.833 1 0.667 2 0.556
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 Which fits a problem where missed positives are very costly?
- F0.5
- Precision only
- Accuracy
- F2 Correct answer
Explanation: β = 2 gives recall more weight than precision.
2 Compute the F-score that favours recall.
f = fbeta_score(y_true, y_pred, beta=____)f = fbeta_score(y_true, y_pred, beta=2)Explanation: beta = 2 counts recall as twice as important as precision.
3 Match the values.
| Term | Meaning |
|---|---|
beta = 1 |
F1 |
beta = 2 |
Recall first |
beta = 0.5 |
Precision first |
precision_recall_curve |
All thresholds |