Training model
- Machine Learning course
- Intermediate
- Lesson 95/112
- 7 min
Train a model with selected features.
Model training
We combine selection and the model in one pipeline and search k by MAE. For comparison we also train the same chain without the derived features:
gs = GridSearchCV(pipe, {"selectkbest__k": [3, 5, 7, 9, 12]}, cv=5,
scoring="neg_mean_absolute_error").fit(X_train, y_train)The test set stays untouched throughout the project; all decisions are made with CV on the training data.
Reading the selected columns
Seeing which columns the best model uses checks whether the selection makes sense:
sec = gs.best_estimator_["selectkbest"]
print(sec.get_support(indices=True))On this data CV selects all 12 columns: since Ridge's penalty already keeps the noise columns' coefficients small, dropping them doesn't improve the score, while dropping real but weak-looking columns makes it worse. The value of selection shows up with many noise columns or with noise-sensitive models like KNN. In this project the main gain will come from the derived features.
Practice Search and inspect
Search selectkbest__k among [3, 5, 7, 9, 12] for make_pipeline(StandardScaler(), SelectKBest(f_regression), Ridge(alpha=1.0)) with GridSearchCV(..., cv=5, scoring="neg_mean_absolute_error"). Print best_params_, the CV MAE (round(-gs.best_score_, 1)) and the selected column indices on separate lines.
Starter code
import numpy as np
rng = np.random.default_rng(11)
n = 500
m2 = rng.uniform(40, 200, n)
oda = np.clip(np.round(m2 / 35 + rng.normal(0, 0.7, n)), 1, 6)
yas = rng.uniform(0, 40, n)
kat = rng.integers(0, 15, n)
merkez_km = rng.uniform(0.5, 25, n)
gurultu = rng.normal(size=(n, 5))
fiyat = 20 * m2 + 150 * m2 / oda - 2500 * np.log1p(merkez_km) - 40 * yas + 60 * kat + rng.normal(0, 300, n)
X = np.column_stack([m2, oda, yas, kat, merkez_km, gurultu])
X2 = np.column_stack([X, m2 / oda, np.log1p(merkez_km)])
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X2, fiyat, test_size=0.25, random_state=0)
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.feature_selection import SelectKBest, f_regression
from sklearn.linear_model import Ridge
from sklearn.model_selection import GridSearchCV
# buraya yazSample solution
import numpy as np
rng = np.random.default_rng(11)
n = 500
m2 = rng.uniform(40, 200, n)
oda = np.clip(np.round(m2 / 35 + rng.normal(0, 0.7, n)), 1, 6)
yas = rng.uniform(0, 40, n)
kat = rng.integers(0, 15, n)
merkez_km = rng.uniform(0.5, 25, n)
gurultu = rng.normal(size=(n, 5))
fiyat = 20 * m2 + 150 * m2 / oda - 2500 * np.log1p(merkez_km) - 40 * yas + 60 * kat + rng.normal(0, 300, n)
X = np.column_stack([m2, oda, yas, kat, merkez_km, gurultu])
X2 = np.column_stack([X, m2 / oda, np.log1p(merkez_km)])
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X2, fiyat, test_size=0.25, random_state=0)
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.feature_selection import SelectKBest, f_regression
from sklearn.linear_model import Ridge
from sklearn.model_selection import GridSearchCV
pipe = make_pipeline(StandardScaler(), SelectKBest(f_regression), Ridge(alpha=1.0))
gs = GridSearchCV(pipe, {"selectkbest__k": [3, 5, 7, 9, 12]}, cv=5, scoring="neg_mean_absolute_error").fit(X_train, y_train)
print(gs.best_params_)
print(round(-gs.best_score_, 1))
print(gs.best_estimator_["selectkbest"].get_support(indices=True))Expected output
{'selectkbest__k': 12}
255.0
[ 0 1 2 3 4 5 6 7 8 9 10 11]
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 Why is <code>scoring="neg_mean_absolute_error"</code> written rather than "mean_absolute_error"?
- scikit-learn treats a larger score as better; the error is negated Correct answer
- MAE can't be computed
- It is faster
- There is no difference
Explanation: best_score_ is the negative MAE; the sign is flipped when reporting.
2 Get the selector step in the best model.
sec = gs.best_estimator_[____]sec = gs.best_estimator_["selectkbest"]Explanation: The step name in make_pipeline is the lowercase class name.
3 Match the decisions.
| Term | Meaning |
|---|---|
k |
Chosen with CV |
MAE |
Selection criterion |
test set |
Untouched until the end |
selected columns |
Sanity check |