Machine Learning
The predict method
The predict() method makes predictions for new data.
The predict family
Classifiers have several prediction methods:
predict: the class label.predict_proba: class probabilities (column order classes_).decision_function: the raw score before the threshold; in SVMs and linear models.
m = LogisticRegression(max_iter=1000).fit(X_train, y_train)
print(m.predict(X_test[:2]))
print(m.predict_proba(X_test[:2]).round(3))
print(m.decision_function(X_test[:2]).round(2))
SVC doesn't offer predict_proba by default; it is enabled with probability=True, but training slows down.
Shape of the prediction input
predict always needs two-dimensional input with the training column count. For a single example, reshape(1, -1) or the slice X[i:i+1] is used:
ornek = X_test[0] # shape (4,)
m.predict(ornek.reshape(1, -1)) # shape (1, 4)
m.predict(X_test[0:1]) # the same
The slice X[0:1] keeps two dimensions; the index X[0] drops one.
Exercise Score and probability
Train scaled logistic regression on the breast cancer data. For the first test example print the decision_function score and the predict_proba[:, 1] probability with round(..., 3) on one line. Then print with np.isclose that the probability equals 1 / (1 + np.exp(-skor)).
Show example solution
import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
m = make_pipeline(StandardScaler(), LogisticRegression()).fit(X_train, y_train)
skor = m.decision_function(X_test[:1])[0]
olasilik = m.predict_proba(X_test[:1])[0, 1]
print(round(skor, 3), round(olasilik, 3))
print(np.isclose(olasilik, 1 / (1 + np.exp(-skor))))
Test yourself
-
In binary logistic regression, if <code>decision_function</code> is 0, what is the probability?
Show answer
0.5 — The probability is sigmoid(score); sigmoid(0) = 0.5, the decision boundary.
-
Make the single example two-dimensional.
m.predict(ornek.reshape(____))
Show answer
m.predict(ornek.reshape(1, -1))
-
Match the methods.
- predict
- Label
- predict_proba
- Probabilities
- decision_function
- Raw score
- classes_
- Column order
Next lesson: The score method
