C parameter
- Machine Learning course
- Intermediate
- Lesson 28/112
- 5 min
C controls the trade-off between margin width and classification error.
The C parameter
C is the penalty for margin violations. A large C counts training errors as very costly: the model tries to classify every point correctly and the boundary bends. A small C accepts some errors and the boundary is smoother:
for C in [0.01, 1, 100]:
m = SVC(C=C).fit(X_train, y_train)
print(C, len(m.support_), m.score(X_test, y_test))With a small C many more points become support vectors, because every point inside the wide margin affects the boundary.
C and gamma together
In an RBF SVM, C and gamma set the complexity together; neither can be tuned alone. The typical approach is to search both together on a logarithmic grid:
izgara = {"svc__C": np.logspace(-2, 3, 6), "svc__gamma": np.logspace(-3, 1, 5)}
gs = GridSearchCV(make_pipeline(StandardScaler(), SVC()), izgara, cv=5)The combination of high C with high gamma is the most flexible, most overfitting model.
Practice The effect of C
Train an RBF SVC on the moon data for C = 0.01, 1, 100. On each line print C, the number of support vectors (len(m.support_)) and the test accuracy with round(..., 3).
Starter code
from sklearn.datasets import make_moons
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
X, y = make_moons(n_samples=300, noise=0.25, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)
# buraya yazSample solution
from sklearn.datasets import make_moons
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
X, y = make_moons(n_samples=300, noise=0.25, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)
for C in [0.01, 1, 100]:
m = SVC(C=C).fit(X_train, y_train)
print(C, len(m.support_), round(m.score(X_test, y_test), 3))Expected output
0.01 195 0.411 1 62 0.967 100 34 0.967
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 How does the number of support vectors usually change as C grows?
- It doesn't change
- It becomes zero
- It decreases Correct answer
- It increases
Explanation: As the margin narrows, fewer points stay inside it.
2 Get the indices of the support vectors.
idx = m.____idx = m.support_Explanation: support_ holds the support vectors' indices in the training set.
3 Match the settings with the behaviour.
| Term | Meaning |
|---|---|
small C |
Wide margin, many support vectors |
large C |
Narrow margin, tight fit |
small gamma |
Smooth boundary |
large C and gamma |
Most overfitting |