Classification
- Machine Learning course
- Intermediate
- Lesson 29/112
- 5 min
SVM is used for binary and multi-class classification.
Classification with SVMs
In practice an SVM is always used in a pipeline with a scaler. For multi-class problems SVC internally applies a one-vs-one (OvO) strategy: one model per pair of classes.
m = make_pipeline(StandardScaler(), SVC()).fit(X_train, y_train)
print(m.score(X_test, y_test))
print(m.decision_function(X_test[:1]).shape) # (1, n_classes)If probabilities are needed, SVC(probability=True) is used; it slows training because it calibrates with an extra internal CV.
Regression: SVR
The same idea in regression is SVR: there is a tube of width epsilon around the prediction; errors inside the tube aren't penalised, and only the points sticking out become support vectors:
from sklearn.svm import SVR
r = make_pipeline(StandardScaler(), SVR(C=10, epsilon=0.1)).fit(X, y)As epsilon grows, fewer support vectors and a smoother function are obtained.
Practice SVM on wine
Split the wine data with test_size=0.3, random_state=0, stratify=y. Train make_pipeline(StandardScaler(), SVC()); print the test accuracy with round(..., 3) and the shape of the decision_function result for the first test example.
Starter code
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
X, y = load_wine(return_X_y=True)
# buraya yazSample solution
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
X, y = load_wine(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)
m = make_pipeline(StandardScaler(), SVC()).fit(X_train, y_train)
print(round(m.score(X_test, y_test), 3))
print(m.decision_function(X_test[:1]).shape)Expected output
1.0 (1, 3)
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 How many binary models does one-vs-one train for a 4-class problem?
- 16
- 6 Correct answer
- 4
- 3
Explanation: The number of class pairs is 4·3/2 = 6.
2 Make the SVC produce probabilities.
m = SVC(____=True)m = SVC(probability=True)Explanation: probability=True performs Platt calibration with an extra internal CV.
3 Match the concepts.
| Term | Meaning |
|---|---|
SVC |
SVM classifier |
SVR |
SVM regressor |
epsilon |
Penalty-free tube width |
OvO |
A model per class pair |