The score method
- Machine Learning course
- Intermediate
- Lesson 5/112
- 5 min
The score() method returns the model's R² or accuracy value.
The score method
score(X, y) returns the default metric for each estimator type: accuracy for classifiers, R² for regressors. If a different metric is needed, the sklearn.metrics functions or get_scorer are used:
from sklearn.metrics import get_scorer
f1 = get_scorer("f1_macro")
print(f1(m, X_test, y_test)) # (model, X, y) signatureScorer objects take (estimator, X, y); this is the form cross-validation and grid search use internally.
make_scorer
You can turn your own metric into a scorer with make_scorer. If the error metric is better when smaller, greater_is_better=False is given; the scorer returns a negative result:
from sklearn.metrics import make_scorer
def ort_yuzde_hata(y, p):
return np.mean(np.abs(y - p) / y)
olcut = make_scorer(ort_yuzde_hata, greater_is_better=False)
cross_val_score(model, X, y, cv=5, scoring=olcut)Business-specific costs (like a false negative being 10 times costlier) enter model selection directly this way.
Practice Custom scorer
Write a maliyet(y, p) function counting a false negative as 5 points and a false positive as 1 point. Make a scorer with make_scorer(maliyet, greater_is_better=False) and print the model's value on the test set.
Starter code
import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import make_scorer
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
m = make_pipeline(StandardScaler(), LogisticRegression()).fit(X_train, y_train)
# buraya yazSample solution
import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import make_scorer
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
m = make_pipeline(StandardScaler(), LogisticRegression()).fit(X_train, y_train)
def maliyet(y, p):
fn = np.sum((y == 1) & (p == 0))
fp = np.sum((y == 0) & (p == 1))
return 5 * fn + fp
olcut = make_scorer(maliyet, greater_is_better=False)
print(olcut(m, X_test, y_test))Expected output
-18
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 What does <code>RandomForestRegressor().score(X, y)</code> return?
- Accuracy
- MAE
- MSE
- R² Correct answer
Explanation: The default score metric of every regressor is R².
2 Make a scorer from a smaller-is-better metric.
olcut = make_scorer(hata, ____=False)olcut = make_scorer(hata, greater_is_better=False)Explanation: greater_is_better=False makes the result negative so the "bigger is better" rule holds.
3 Match the default metrics.
| Term | Meaning |
|---|---|
Classifier score |
Accuracy |
Regressor score |
R² |
get_scorer |
Scorer by name |
make_scorer |
Custom scorer |