Summary - Model evaluation
- Machine Learning course
- Beginner
- Lesson 70/112
- 6 min
In this group you learned train-test split, accuracy, precision, and F1.
Summary: model evaluation
In this group:
- Test data is kept completely apart from training;
shuffle=Falsefor time series. - Accuracy is good for balanced data; for imbalanced data balanced accuracy is more honest.
- Precision measures false alarms, recall misses; the threshold balances them.
- F1 is their harmonic mean;
classification_reportgives everything per class.
Choosing a metric isn't a technical decision but a business one: which error is costlier?
Practice Produce the report
Get test predictions on the breast cancer data with scaled logistic regression. Print the F1 score with round(..., 3), then find the accuracy line among the lines of the classification_report(..., digits=3) output and print its second value using split().
Starter code
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import f1_score, classification_report
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
m = make_pipeline(StandardScaler(), LogisticRegression()).fit(X_train, y_train)
# buraya yazSample solution
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import f1_score, classification_report
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
m = make_pipeline(StandardScaler(), LogisticRegression()).fit(X_train, y_train)
p = m.predict(X_test)
print(round(f1_score(y_test, p), 3))
rapor = classification_report(y_test, p, digits=3)
for s in rapor.splitlines():
if s.strip().startswith("accuracy"):
print(s.split()[1])Expected output
0.967 0.958
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 In fraud detection, if both misses and false alarms matter, which metric fits?
- R²
- F1 Correct answer
- Accuracy
- Precision only
Explanation: F1 accounts for both and doesn't mislead on imbalanced data like accuracy does.
2 Print the per-class report with three digits.
Question
print(classification_report(y_test, y_pred, ____=3))
Answer
print(classification_report(y_test, y_pred, digits=3))Explanation: digits is the number of decimal places in the report.
3 Match the problems with the metrics.
| Term | Meaning |
|---|---|
Cancer screening |
Recall |
Spam filter |
Precision |
Balanced 3 classes |
Accuracy |
Fraud detection |
F1 |