Multi-class
- Machine Learning course
- Beginner
- Lesson 61/112
- 5 min
Multi-class classification has more than two classes.
Multi-class classification
If there are more than two classes (the 3 species of Iris, 10 digits), most scikit-learn classifiers support it directly. predict_proba gives one column per class and each row sums to 1:
from sklearn.datasets import load_iris
X, y = load_iris(return_X_y=True)
m = LogisticRegression(max_iter=1000).fit(X, y)
print(m.classes_) # [0 1 2]
print(m.predict_proba(X[:1]).round(3))max_iter gives the optimisation more steps to converge; increase it if you get a warning.
One-vs-rest
Models that are binary by nature are extended to many classes with the one-vs-rest (OvR) strategy: a separate model is trained for each class asking "is it this class or not?", and the one with the highest score wins.
from sklearn.multiclass import OneVsRestClassifier
from sklearn.svm import LinearSVC
m = OneVsRestClassifier(LinearSVC()).fit(X, y)
print(len(m.estimators_)) # 3: one model per classThe predicted class is found with argmax on the probability array.
Practice Three classes on Iris
Train a LogisticRegression(max_iter=1000) on Iris. Get the probabilities for the first example, the 51st (index 50) and the 101st (index 100); print the classes with argmax(axis=1) and print that this matches the predict result with (... == ...).all().
Starter code
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
X, y = load_iris(return_X_y=True)
secim = X[[0, 50, 100]]
# buraya yazSample solution
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
X, y = load_iris(return_X_y=True)
secim = X[[0, 50, 100]]
m = LogisticRegression(max_iter=1000).fit(X, y)
p = m.predict_proba(secim)
sinif = p.argmax(axis=1)
print(sinif)
print((sinif == m.predict(secim)).all())Expected output
[0 1 2] True
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 In a 3-class problem, what is the shape of <code>predict_proba(X)</code> (10 examples in X)?
- (10,)
- (3, 10)
- (10, 1)
- (10, 3) Correct answer
Explanation: For every example, the probability of every class: 10 rows, 3 columns.
2 Find the predicted classes from the probabilities.
sinif = p.____(axis=1)sinif = p.argmax(axis=1)Explanation: argmax(axis=1) gives the index of the most probable class in each row.
3 Match the concepts.
| Term | Meaning |
|---|---|
OvR |
One binary model per class |
predict_proba row |
Sums to 1 |
argmax(axis=1) |
Most likely class |
max_iter |
Optimisation step limit |