Summary - Model evaluation

In this group you learned train-test split, accuracy, precision, and F1.

Summary: model evaluation

In this group:

Test data is kept completely apart from training; shuffle=False for time series.
Accuracy is good for balanced data; for imbalanced data balanced accuracy is more honest.
Precision measures false alarms, recall misses; the threshold balances them.
F1 is their harmonic mean; classification_report gives everything per class.

Choosing a metric isn't a technical decision but a business one: which error is costlier?

Exercise Produce the report

Get test predictions on the breast cancer data with scaled logistic regression. Print the F1 score with round(..., 3), then find the accuracy line among the lines of the classification_report(..., digits=3) output and print its second value using split().

Show example solution

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import f1_score, classification_report

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
m = make_pipeline(StandardScaler(), LogisticRegression()).fit(X_train, y_train)

p = m.predict(X_test)
print(round(f1_score(y_test, p), 3))
rapor = classification_report(y_test, p, digits=3)
for s in rapor.splitlines():
    if s.strip().startswith("accuracy"):
        print(s.split()[1])

Test yourself

In fraud detection, if both misses and false alarms matter, which metric fits?
- R²
- F1
- Accuracy
- Precision only
Show answer

F1 — F1 accounts for both and doesn't mislead on imbalanced data like accuracy does.

Print the per-class report with three digits.

print(classification_report(y_test, y_pred, ____=3))

Show answer

print(classification_report(y_test, y_pred, digits=3))

Match the problems with the metrics.

Cancer screening

Recall
Spam filter

Precision
Balanced 3 classes

Accuracy
Fraud detection

F1

Next lesson: Preparing data

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies