ElasticNet
- Machine Learning course
- Intermediate
- Lesson 107/112
- 5 min
ElasticNet combines L1 and L2 regularization.
ElasticNet
ElasticNet combines the L1 and L2 penalties. l1_ratio is the mixing ratio: 1 is pure Lasso, 0 pure Ridge. It offers Lasso's sparsity together with Ridge's stability on correlated features:
from sklearn.linear_model import ElasticNet, ElasticNetCV
en = ElasticNet(alpha=0.1, l1_ratio=0.5).fit(Xs, y)
encv = ElasticNetCV(l1_ratio=[0.1, 0.5, 0.9, 1.0], cv=5, random_state=0).fit(Xs, y)
print(encv.alpha_, encv.l1_ratio_)With groups of correlated features, ElasticNet tends to keep the group together; Lasso picks a single one from the group.
Regularisation in classification
Logistic regression is regularised too; but in scikit-learn the penalty strength is given as C = 1/α: small C means a strong penalty. A suitable solver must be chosen for the L1 penalty:
LogisticRegression(C=0.1) # strong L2
LogisticRegression(penalty="l1", C=0.1, solver="liblinear") # sparse
LogisticRegression(penalty="elasticnet", l1_ratio=0.5, solver="saga", C=1)The C in SVMs follows the same logic: large C means little penalty, a tight fit.
Practice Grouped features
Train Lasso(alpha=0.1) and ElasticNet(alpha=0.1, l1_ratio=0.3) with G, made of three noisy copies of the first column. For each print the number of non-zero coefficients on one line (Lasso first).
Starter code
import numpy as np
from sklearn.preprocessing import StandardScaler
rng = np.random.default_rng(0)
X = rng.normal(size=(60, 20))
y = 3 * X[:, 0] - 2 * X[:, 1] + 1.5 * X[:, 2] + rng.normal(0, 1, 60)
Xs = StandardScaler().fit_transform(X)
from sklearn.linear_model import Lasso, ElasticNet
G = np.column_stack([Xs[:, 0] + rng.normal(0, 0.05, 60) for _ in range(3)])
# buraya yazSample solution
import numpy as np
from sklearn.preprocessing import StandardScaler
rng = np.random.default_rng(0)
X = rng.normal(size=(60, 20))
y = 3 * X[:, 0] - 2 * X[:, 1] + 1.5 * X[:, 2] + rng.normal(0, 1, 60)
Xs = StandardScaler().fit_transform(X)
from sklearn.linear_model import Lasso, ElasticNet
G = np.column_stack([Xs[:, 0] + rng.normal(0, 0.05, 60) for _ in range(3)])
a = (Lasso(alpha=0.1).fit(G, y).coef_ != 0).sum()
b = (ElasticNet(alpha=0.1, l1_ratio=0.3).fit(G, y).coef_ != 0).sum()
print(a, b)Expected output
1 3
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 What is <code>ElasticNet(l1_ratio=1.0)</code> equal to?
- LinearRegression
- None of them
- Lasso Correct answer
- Ridge
Explanation: l1_ratio = 1 means the penalty is entirely L1.
2 Mix L1 and L2 equally.
en = ElasticNet(alpha=0.1, ____=0.5)en = ElasticNet(alpha=0.1, l1_ratio=0.5)Explanation: l1_ratio is the L1 share of the penalty.
3 Match the parameters.
| Term | Meaning |
|---|---|
l1_ratio=0 |
Pure Ridge |
l1_ratio=1 |
Pure Lasso |
small C |
Strong penalty |
solver="liblinear" |
L1 logistic |