What is SVM?

SVM finds the hyperplane that separates classes with the widest margin.

What is an SVM?

A support vector machine (SVM) chooses, among the boundaries separating two classes, the one farthest from the classes. The training points closest to that boundary are called support vectors; the boundary's position depends only on them.

from sklearn.svm import SVC
m = SVC(kernel="linear").fit(X, y)
print(m.support_vectors_.shape)   # support vector count x features
print(m.n_support_)               # per class

If the other points were deleted the model wouldn't change; that is why an SVM is memory-efficient and relatively robust to outliers.

Strengths of SVMs

SVMs are strong on medium-sized data (up to a few tens of thousands of examples) with many features: text classification, bioinformatics. With the kernel trick they learn non-linear boundaries.

Very scale-sensitive: StandardScaler first.
Training time grows fast with the number of examples; for very large data LinearSVC or SGDClassifier is preferred.
By default it produces a decision score, not a probability.

Exercise Linear SVM

Train an SVC(kernel="linear") on separable two-cluster data. Print the number of support vectors (n_support_) and the predictions for [[0, 0], [4, 4]] on separate lines.

Show example solution

import numpy as np
from sklearn.svm import SVC

X = np.array([[0, 0], [1, 0], [0, 1], [1, 1], [3, 3], [4, 3], [3, 4], [4, 4]])
y = np.array([0, 0, 0, 0, 1, 1, 1, 1])

m = SVC(kernel="linear").fit(X, y)
print(m.n_support_)
print(m.predict([[0, 0], [4, 4]]))

Test yourself

Which points determine an SVM's boundary?
- The class means
- The support vectors
- All points equally
- The points farthest from the boundary
Show answer

The support vectors — The points closest to the boundary define the margin; those far away have no effect.
Read the number of support vectors per class.
```
print(m.____)
```
Show answer
```
print(m.n_support_)
```
Match the terms.

support vector

Point closest to the boundary
support_vectors_

Their coordinates
LinearSVC

Fast for large data
StandardScaler

Required before SVM

Next lesson: Margin

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies