L1 (Lasso)
- Machine Learning course
- Intermediate
- Lesson 104/112
- 5 min
L1 regularization performs feature selection by shrinking coefficients to zero.
Lasso (L1)
Lasso uses an absolute-value penalty. This penalty has an important property: it pushes the coefficients of weak features to exactly zero. That way the model both learns and selects features:
from sklearn.linear_model import Lasso
m = Lasso(alpha=0.1).fit(Xs, y)
print((m.coef_ != 0).sum()) # number of non-zero coefficients
print(np.where(m.coef_ != 0)[0]) # selected featuresA sparse (mostly zero) solution produces interpretable, fast models.
Scale and instability
The penalty is applied equally to all coefficients; that's why features must be scaled first, otherwise a large-unit feature gets off "cheaply" with a small coefficient. Among correlated features, Lasso may also pick one arbitrarily:
pipe = make_pipeline(StandardScaler(), Lasso(alpha=0.1))
# x1 and x1 + small noise: Lasso usually keeps only oneWith groups of correlated features, Lasso's choice may change from sample to sample; ElasticNet softens this problem.
Practice Selection with Lasso
Train a Lasso(alpha=0.2) on the scaled data. Print the indices of the non-zero coefficients, then those coefficients with np.round(..., 2), on separate lines.
Starter code
import numpy as np
from sklearn.preprocessing import StandardScaler
rng = np.random.default_rng(0)
X = rng.normal(size=(60, 20))
y = 3 * X[:, 0] - 2 * X[:, 1] + 1.5 * X[:, 2] + rng.normal(0, 1, 60)
Xs = StandardScaler().fit_transform(X)
from sklearn.linear_model import Lasso
# buraya yazSample solution
import numpy as np
from sklearn.preprocessing import StandardScaler
rng = np.random.default_rng(0)
X = rng.normal(size=(60, 20))
y = 3 * X[:, 0] - 2 * X[:, 1] + 1.5 * X[:, 2] + rng.normal(0, 1, 60)
Xs = StandardScaler().fit_transform(X)
from sklearn.linear_model import Lasso
m = Lasso(alpha=0.2).fit(Xs, y)
idx = np.where(m.coef_ != 0)[0]
print(idx)
print(np.round(m.coef_[idx], 2))Expected output
[ 0 1 2 3 5 15] [ 2.78 -1.64 1.32 0.05 -0.11 -0.06]
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 What property sets Lasso apart from Ridge?
- It uses no penalty
- It can set coefficients to exactly zero Correct answer
- It needs no scaling
- It is faster
Explanation: The cornered geometry of the L1 penalty pushes the solution onto the axes, i.e. zero coefficients.
2 Count the non-zero coefficients.
print((m.coef_ ____ 0).sum())print((m.coef_ != 0).sum())Explanation: In Lasso the non-zero coefficients are the selected features.
3 Match the properties.
| Term | Meaning |
|---|---|
L1 |
Absolute-value penalty |
sparse solution |
Most coefficients zero |
scaling |
Required for a fair penalty |
correlated features |
Risk of arbitrary choice |