Summary - Regularization

In this group you learned L1, L2, and ElasticNet regularization.

Summary: regularisation

In this group:

Regularisation adds a penalty on coefficient size to reduce overfitting; features must be scaled first.
Lasso (L1): pushes coefficients to zero, selects features; may pick one of correlated features arbitrarily.
Ridge (L2): shrinks coefficients, shares the weight, is stable.
ElasticNet: a mix of both, l1_ratio.
alpha is chosen with CV: RidgeCV, LassoCV, ElasticNetCV; in logistic regression and SVMs C = 1/α.

Exercise Compare the three models

Compare LinearRegression, RidgeCV(alphas=np.logspace(-2, 3, 11)) and LassoCV(cv=5, random_state=0) on the scaled data by their mean 5-fold CV R². Print the three scores with round(..., 3) on one line.

Show example solution

import numpy as np
from sklearn.preprocessing import StandardScaler

rng = np.random.default_rng(0)
X = rng.normal(size=(60, 20))
y = 3 * X[:, 0] - 2 * X[:, 1] + 1.5 * X[:, 2] + rng.normal(0, 1, 60)
Xs = StandardScaler().fit_transform(X)
from sklearn.linear_model import LinearRegression, RidgeCV, LassoCV
from sklearn.model_selection import cross_val_score

modeller = [LinearRegression(), RidgeCV(alphas=np.logspace(-2, 3, 11)), LassoCV(cv=5, random_state=0)]
print(*[round(cross_val_score(m, Xs, y, cv=5).mean(), 3) for m in modeller])

Test yourself

If you want an interpretable model with few features, which do you choose?
- No regularisation
- Lasso
- Ridge
- Ordinary regression
Show answer

Lasso — Lasso's zero coefficients amount to feature selection.

Apply a strong penalty in logistic regression.

m = LogisticRegression(____=0.01)

Show answer

m = LogisticRegression(C=0.01)

Match the models.

Lasso

L1, sparse
Ridge

L2, shared
ElasticNet

L1 + L2
RidgeCV

Chooses alpha

Next lesson: Challenge 1

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies