Summary - SVM

In this group you learned SVM, margin, kernel, and C parameter.

Summary: SVMs

In this group:

An SVM picks the boundary farthest from the classes; the support vectors determine it.
The margin is 2/‖w‖; in a soft margin the penalty for violations is C.
Kernels: linear, poly, rbf; in RBF, gamma sets the area of influence.
C and gamma are searched together on a logarithmic grid.
Always a pipeline with a scaler; OvO for many classes; SVR for regression.

Exercise C and gamma search

Run GridSearchCV(..., cv=5) for make_pipeline(StandardScaler(), SVC()) on the moon data with the grid {"svc__C": [0.1, 1, 10, 100], "svc__gamma": [0.1, 1, 10]}. Print best_params_ and the test accuracy with round(..., 3) on separate lines.

Show example solution

from sklearn.datasets import make_moons
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC

X, y = make_moons(n_samples=300, noise=0.25, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)
from sklearn.model_selection import GridSearchCV
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

gs = GridSearchCV(make_pipeline(StandardScaler(), SVC()), {"svc__C": [0.1, 1, 10, 100], "svc__gamma": [0.1, 1, 10]}, cv=5)
gs.fit(X_train, y_train)
print(gs.best_params_)
print(round(gs.score(X_test, y_test), 3))

Test yourself

If an RBF SVM has training accuracy 1.0 and low test accuracy, which setting change should be tried first?
- Lowering gamma and/or C
- Raising gamma
- Removing scaling
- Raising C
Show answer

Lowering gamma and/or C — Both reduce the model's flexibility and reduce overfitting.

Create the SVM regressor.

r = ____(C=10, epsilon=0.1)

Show answer

r = SVR(C=10, epsilon=0.1)

Match the parameters.

C

Violation penalty
gamma

Narrowness of influence
kernel

Similarity function
epsilon

Tube width

Next lesson: Preparing data

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies