Machine Learning
Multi-class
Multi-class classification has more than two classes.
Multi-class classification
If there are more than two classes (the 3 species of Iris, 10 digits), most scikit-learn classifiers support it directly. predict_proba gives one column per class and each row sums to 1:
from sklearn.datasets import load_iris
X, y = load_iris(return_X_y=True)
m = LogisticRegression(max_iter=1000).fit(X, y)
print(m.classes_) # [0 1 2]
print(m.predict_proba(X[:1]).round(3))
max_iter gives the optimisation more steps to converge; increase it if you get a warning.
One-vs-rest
Models that are binary by nature are extended to many classes with the one-vs-rest (OvR) strategy: a separate model is trained for each class asking "is it this class or not?", and the one with the highest score wins.
from sklearn.multiclass import OneVsRestClassifier
from sklearn.svm import LinearSVC
m = OneVsRestClassifier(LinearSVC()).fit(X, y)
print(len(m.estimators_)) # 3: one model per class
The predicted class is found with argmax on the probability array.
Exercise Three classes on Iris
Train a LogisticRegression(max_iter=1000) on Iris. Get the probabilities for the first example, the 51st (index 50) and the 101st (index 100); print the classes with argmax(axis=1) and print that this matches the predict result with (... == ...).all().
Show example solution
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
X, y = load_iris(return_X_y=True)
secim = X[[0, 50, 100]]
m = LogisticRegression(max_iter=1000).fit(X, y)
p = m.predict_proba(secim)
sinif = p.argmax(axis=1)
print(sinif)
print((sinif == m.predict(secim)).all())
Test yourself
-
In a 3-class problem, what is the shape of <code>predict_proba(X)</code> (10 examples in X)?
- (10,)
- (3, 10)
- (10, 1)
- (10, 3)
Show answer
(10, 3) — For every example, the probability of every class: 10 rows, 3 columns.
-
Find the predicted classes from the probabilities.
sinif = p.____(axis=1)
Show answer
sinif = p.argmax(axis=1)
-
Match the concepts.
- OvR
- One binary model per class
- predict_proba row
- Sums to 1
- argmax(axis=1)
- Most likely class
- max_iter
- Optimisation step limit
Next lesson: Accuracy
