L2 (Ridge)
- Machine Learning course
- Intermediate
- Lesson 105/112
- 5 min
L2 regularization shrinks coefficients but doesn't zero them.
Ridge (L2)
Ridge uses a squared penalty. It shrinks the coefficients towards zero but usually doesn't make them exactly zero. With correlated features it shares the weight between them; that's why it is more stable than Lasso:
from sklearn.linear_model import Ridge
for a in [0.01, 1, 100]:
m = Ridge(alpha=a).fit(Xs, y)
print(a, round(np.abs(m.coef_).sum(), 2), (m.coef_ == 0).sum()) # no zeros at allRidge has a closed-form solution: w = (XᵀX + αI)⁻¹Xᵀy; the αI term makes the solution possible even when XᵀX isn't invertible.
Ridge, Lasso and identical columns
The clearest way to see the difference is to copy a column:
X2 = np.column_stack([Xs[:, 0], Xs[:, 0]])
print(Ridge(alpha=1).fit(X2, y).coef_.round(2)) # [1.4 1.4] shares
print(Lasso(alpha=0.1).fit(X2, y).coef_.round(2)) # [2.7 0. ] picks oneWhich behaviour is wanted depends on the question: Lasso for an interpretable sparse model, Ridge for stable predictions.
Practice Identical columns
Train Ridge(alpha=1) and Lasso(alpha=0.1) with X2, made of two copies of the first column. Print the two coefficient arrays with np.round(..., 2) on separate lines.
Starter code
import numpy as np
from sklearn.preprocessing import StandardScaler
rng = np.random.default_rng(0)
X = rng.normal(size=(60, 20))
y = 3 * X[:, 0] - 2 * X[:, 1] + 1.5 * X[:, 2] + rng.normal(0, 1, 60)
Xs = StandardScaler().fit_transform(X)
from sklearn.linear_model import Ridge, Lasso
X2 = np.column_stack([Xs[:, 0], Xs[:, 0]])
# buraya yazSample solution
import numpy as np
from sklearn.preprocessing import StandardScaler
rng = np.random.default_rng(0)
X = rng.normal(size=(60, 20))
y = 3 * X[:, 0] - 2 * X[:, 1] + 1.5 * X[:, 2] + rng.normal(0, 1, 60)
Xs = StandardScaler().fit_transform(X)
from sklearn.linear_model import Ridge, Lasso
X2 = np.column_stack([Xs[:, 0], Xs[:, 0]])
print(np.round(Ridge(alpha=1).fit(X2, y).coef_, 2))
print(np.round(Lasso(alpha=0.1).fit(X2, y).coef_, 2))Expected output
[1.43 1.43] [2.78 0. ]
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 With two identical columns, how does Ridge distribute the coefficients?
- It shares them equally Correct answer
- It zeroes one
- It zeroes both
- It errors
Explanation: The squared penalty prefers equal coefficients when the total effect is the same.
2 Create the squared-penalty model.
m = ____(alpha=1.0)m = Ridge(alpha=1.0)Explanation: Ridge applies the L2 penalty.
3 Match the behaviours.
| Term | Meaning |
|---|---|
Ridge |
Shares the weight |
Lasso |
Picks one |
αI |
Ensures invertibility |
kare ceza |
Near zero, not zero |