Binary classification

Binary classification has two classes: positive and negative.

Binary classification

In two-class problems the classes are usually encoded as 0 (negative) and 1 (positive). "Positive" doesn't mean good or bad; it is the event we are looking for: the disease is present, the transaction is fraud, the customer will leave.

m = LogisticRegression().fit(X, y)
print(m.predict_proba([[5]]))   # [[P(0), P(1)]]
print(m.predict([[5]]))         # 1 if P(1) > 0.5

predict_proba returns two columns; the positive-class probability is taken with [:, 1].

Decision threshold

0.5 is only the default threshold. If missing a patient in cancer screening is costly, the threshold is lowered; more people are flagged positive:

p = m.predict_proba(X_test)[:, 1]
karar = (p >= 0.3).astype(int)   # more sensitive threshold

Changing the threshold shifts the balance of decisions without retraining the model; in the evaluation lesson you'll see how this connects to the metrics.

Exercise Decide with a threshold

Train a LogisticRegression. Print the positive-class probabilities for X_yeni with np.round(..., 2). Then print the decisions with a 0.3 threshold ((p >= 0.3).astype(int)).

Show example solution

import numpy as np
from sklearn.linear_model import LogisticRegression

X = np.array([[1], [2], [3], [4], [5], [6], [7], [8]])
y = np.array([0, 0, 0, 0, 1, 1, 1, 1])
X_yeni = np.array([[3.5], [4.2], [6.0]])

m = LogisticRegression().fit(X, y)
p = m.predict_proba(X_yeni)[:, 1]
print(np.round(p, 2))
print((p >= 0.3).astype(int))

Test yourself

If <code>predict_proba</code> gives <code>[[0.3, 0.7]]</code>, what does <code>predict</code> return?
- [1]
- [0]
- [0.7]
- [0.3]
Show answer

[1] — The positive-class probability 0.7 > 0.5, so class 1 is chosen.

Take the positive class's probability column.

p = m.predict_proba(X_test)[____]

Show answer

p = m.predict_proba(X_test)[:, 1]

Match the terms.

positive class

The event sought (1)
negative class

Absence of the event (0)
predict_proba[:, 1]

Positive probability
threshold

From probability to decision

Next lesson: Multi-class

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies