What is a metric?

A metric is a numerical value that measures model performance.

What is a metric?

A metric summarises a model's success in a single number. Two separate concepts mustn't be confused:

Loss: the function optimised during training (log-loss, MSE).
Metric: the measure people use to interpret success (accuracy, F1, MAE).

from sklearn.metrics import get_scorer_names
print(len(get_scorer_names()))          # dozens of ready-made scorers
print("roc_auc" in get_scorer_names())  # True

The metric must reflect the real cost of the problem; the wrong metric leads to a model that looks right but is useless.

Log-loss: quality of probabilities

Log-loss is the mean negative logarithm of the probability given to the correct class. It punishes confident but wrong predictions very hard:

from sklearn.metrics import log_loss
print(round(log_loss([1, 0], [[0.1, 0.9], [0.8, 0.2]]), 3))   # 0.164  good
print(round(log_loss([1, 0], [[0.9, 0.1], [0.2, 0.8]]), 3))   # 1.956  confident and wrong

Accuracy measures only the decision; log-loss measures how well the probabilities are calibrated.

Exercise Compare log-loss

For two probability sets that give the same decisions but with different confidence, print the log_loss values with round(..., 3) on separate lines.

Show example solution

from sklearn.metrics import log_loss

y = [1, 0, 1]
emin = [[0.05, 0.95], [0.9, 0.1], [0.2, 0.8]]
kararsiz = [[0.45, 0.55], [0.55, 0.45], [0.4, 0.6]]

print(round(log_loss(y, emin), 3))
print(round(log_loss(y, kararsiz), 3))

Test yourself

What does logistic regression minimise in training?
- Accuracy
- F1
- R²
- Log-loss (cross-entropy)
Show answer

Log-loss (cross-entropy) — Accuracy isn't differentiable; differentiable log-loss is used in training.

Compute the log-loss of the probabilities.

kayip = ____(y_true, olasiliklar)

Show answer

kayip = log_loss(y_true, olasiliklar)

Match the concepts.

loss

Minimised in training
metric

Interprets success
log_loss

Probability quality
get_scorer_names

Ready-made scorers

Next lesson: Accuracy

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies