Kernel

Kernel increases linear separability by mapping data to higher dimensions.

The kernel trick

Data that isn't linearly separable can become separable when moved to a higher-dimensional space. A kernel function computes the similarity between points directly, without doing that move explicitly:

SVC(kernel="linear")             # straight boundary
SVC(kernel="poly", degree=3)     # polynomial boundary
SVC(kernel="rbf", gamma=0.5)     # default; flexible, local boundaries

The RBF kernel exp(-γ‖x - x'‖²) measures how close two points are.

gamma

In RBF, gamma sets how narrow a point's area of influence is. Small gamma gives wide, smooth boundaries; large gamma draws small islands around each point (overfitting):

for g in [0.1, 1, 100]:
    m = SVC(gamma=g).fit(X_train, y_train)
    print(g, m.score(X_train, y_train), m.score(X_test, y_test))

The default gamma="scale" uses 1 / (n_features · X.var()) and is a good start on scaled data.

Exercise Compare kernels

Train SVCs with the linear, poly and rbf kernels on the moon data. On each line print the kernel name and the test accuracy with round(..., 3).

Show example solution

from sklearn.datasets import make_moons
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC

X, y = make_moons(n_samples=300, noise=0.25, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)

for k in ["linear", "poly", "rbf"]:
    m = SVC(kernel=k).fit(X_train, y_train)
    print(k, round(m.score(X_test, y_test), 3))

Test yourself

Which kernel suits two moon-shaped classes (make_moons) best?
- linear
- The kernel doesn't matter
- None can learn it
- rbf
Show answer

rbf — A curved boundary can't be drawn with the linear kernel; rbf gives local flexibility.

Use the polynomial kernel.

m = SVC(kernel=____, degree=3)

Show answer

m = SVC(kernel="poly", degree=3)

Match the kernels.

linear

Straight boundary
poly

Polynomial curve
rbf

Local similarity
large gamma

Islands, overfitting

Next lesson: C parameter

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies