Evaluation
- Machine Learning course
- Beginner
- Lesson 74/112
- 7 min
Calculate model accuracy.
Evaluation
A single accuracy number isn't enough to evaluate the project. The confusion matrix shows which classes get mixed up, and classification_report shows precision and recall per class:
from sklearn.metrics import confusion_matrix, classification_report
tahmin = model.predict(X_test)
print(confusion_matrix(y_test, tahmin))
print(classification_report(y_test, tahmin, digits=3))The class with the lowest recall in the report is the class the model struggles with most.
Per-class recall
recall_score(..., average=None) returns each class's recall as a separate array; it equals the diagonal of the confusion matrix divided by the row sums:
from sklearn.metrics import recall_score
print(recall_score(y_test, tahmin, average=None).round(3))
cm = confusion_matrix(y_test, tahmin)
print((cm.diagonal() / cm.sum(axis=1)).round(3)) # the sameNaming the weakest class when writing up results shows where the next improvement should focus.
Practice Report the result
Print the confusion matrix with the test predictions. Then print the per-class recall with recall_score(..., average=None).round(3) and the index of the class with the lowest recall with argmin.
Starter code
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
X, y = load_wine(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42, stratify=y)
model = make_pipeline(StandardScaler(), KNeighborsClassifier(n_neighbors=5)).fit(X_train, y_train)
from sklearn.metrics import confusion_matrix, recall_score
# buraya yazSample solution
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
X, y = load_wine(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42, stratify=y)
model = make_pipeline(StandardScaler(), KNeighborsClassifier(n_neighbors=5)).fit(X_train, y_train)
from sklearn.metrics import confusion_matrix, recall_score
tahmin = model.predict(X_test)
print(confusion_matrix(y_test, tahmin))
r = recall_score(y_test, tahmin, average=None).round(3)
print(r)
print(r.argmin())Expected output
[[15 0 0] [ 0 16 2] [ 0 1 11]] [1. 0.889 0.917] 1
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 In the confusion matrix, what does a 3 in row 2, column 1 mean?
- Correct predictions of class 1
- There are 3 classes
- Class 0 precision
- 3 examples truly of class 1 were predicted as class 0 Correct answer
Explanation: The row shows the true class (index 1), the column the prediction (index 0).
2 Get each class's recall separately.
r = recall_score(y_test, tahmin, average=____)r = recall_score(y_test, tahmin, average=None)Explanation: average=None returns per-class values without averaging.
3 Match the outputs.
| Term | Meaning |
|---|---|
confusion_matrix |
Mixed-up classes |
classification_report |
Per-class table |
average=None |
Per-class array |
cm.diagonal() |
Correct predictions |