Margin
- Machine Learning course
- Intermediate
- Lesson 26/112
- 5 min
The margin is the distance from the hyperplane to the nearest points of each class.
The margin
The margin is the gap between the boundary and the nearest points. A linear SVM chooses the boundary w·x + b = 0 so that the margin width 2 / ‖w‖ is as large as possible. A wide margin means safer decisions on new points.
m = SVC(kernel="linear", C=1000).fit(X, y) # close to a hard margin
w = m.coef_[0]
print(2 / np.linalg.norm(w)) # margin widthFor the support vectors |w·x + b| = 1; these points sit on the edge of the margin.
Hard and soft margin
A hard margin allows no point inside the margin; if the data can't be separated perfectly there is no solution, and a single outlier shifts the boundary a lot. A soft margin allows some violations at a cost; the balance is set with C.
# large C: violations costly -> narrow margin, tight fit to training
# small C: violations cheap -> wide margin, smoother boundaryAlmost all real data is noisy, so in practice a soft margin is always used.
Practice Measure the margin
Train an SVC(kernel="linear", C=1000). Compute the margin width with 2 / np.linalg.norm(m.coef_[0]) and print it with round(..., 3). Then print the absolute decision_function values at the support vectors with np.round(..., 2).
Starter code
import numpy as np
from sklearn.svm import SVC
X = np.array([[0, 0], [1, 0], [0, 1], [3, 3], [4, 3], [3, 4]])
y = np.array([0, 0, 0, 1, 1, 1])
# buraya yazSample solution
import numpy as np
from sklearn.svm import SVC
X = np.array([[0, 0], [1, 0], [0, 1], [3, 3], [4, 3], [3, 4]])
y = np.array([0, 0, 0, 1, 1, 1])
m = SVC(kernel="linear", C=1000).fit(X, y)
print(round(2 / np.linalg.norm(m.coef_[0]), 3))
print(np.round(np.abs(m.decision_function(m.support_vectors_)), 2))Expected output
3.536 [1. 1. 1.]
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 What does widening the margin mean for ‖w‖?
- ‖w‖ shrinks Correct answer
- ‖w‖ grows
- ‖w‖ doesn't change
- ‖w‖ becomes zero
Explanation: The margin is 2/‖w‖, so as the weight norm shrinks the margin widens.
2 Compute the norm of the weight vector.
marj = 2 / np.linalg.____(m.coef_[0])marj = 2 / np.linalg.norm(m.coef_[0])Explanation: norm gives the Euclidean length ‖w‖.
3 Match the concepts.
| Term | Meaning |
|---|---|
margin |
2 / ‖w‖ |
hard margin |
No violations allowed |
soft margin |
Penalised violations |
|w·x + b| = 1 |
Support vectors |