Training model
- Machine Learning course
- Beginner
- Lesson 94/112
- 7 min
Train a linear regression model.
Model training
We'll train a baseline and two candidates: DummyRegressor predicting the mean, LinearRegression and lightly regularised Ridge:
from sklearn.dummy import DummyRegressor
from sklearn.linear_model import LinearRegression, Ridge
adaylar = {
"ortalama": DummyRegressor(),
"dogrusal": LinearRegression(),
"ridge": Ridge(alpha=1.0),
}Ridge penalises large weights and makes the model more stable against noise; you'll see the details at the intermediate level.
Choosing with cross-validation
We compare the candidates with 5-fold validation on the training set, without looking at the test set. We'll use MAE as the metric:
for ad, m in adaylar.items():
s = -cross_val_score(m, X_train, y_train, cv=5,
scoring="neg_mean_absolute_error")
print(ad, s.mean().round(1))The leading minus sign turns the error that scikit-learn gives as negative back into a positive. Because the features in this data are on a very small scale, alpha=1.0 penalises too much and Ridge falls behind; in the next steps we'll continue with LinearRegression, which gives the lowest MAE.
Practice Validate the candidates
Compare the three candidates on MAE with 5-fold validation on the training set. On each line print the name and the mean MAE with round(..., 1).
Starter code
from sklearn.datasets import load_diabetes
from sklearn.model_selection import train_test_split
X, y = load_diabetes(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
from sklearn.model_selection import cross_val_score
from sklearn.dummy import DummyRegressor
from sklearn.linear_model import LinearRegression, Ridge
adaylar = {"ortalama": DummyRegressor(), "dogrusal": LinearRegression(), "ridge": Ridge(alpha=1.0)}
# buraya yazSample solution
from sklearn.datasets import load_diabetes
from sklearn.model_selection import train_test_split
X, y = load_diabetes(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
from sklearn.model_selection import cross_val_score
from sklearn.dummy import DummyRegressor
from sklearn.linear_model import LinearRegression, Ridge
adaylar = {"ortalama": DummyRegressor(), "dogrusal": LinearRegression(), "ridge": Ridge(alpha=1.0)}
for ad, m in adaylar.items():
s = -cross_val_score(m, X_train, y_train, cv=5, scoring="neg_mean_absolute_error")
print(ad, round(s.mean(), 1))Expected output
ortalama 67.4 dogrusal 44.0 ridge 49.9
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 What does <code>DummyRegressor()</code> predict by default?
- A random value
- The median
- The mean of the training target Correct answer
- Zero
Explanation: strategy="mean" is the default; its R² comes out around 0.
2 Use negative MAE as the metric.
cross_val_score(m, X_train, y_train, cv=5, scoring=____)cross_val_score(m, X_train, y_train, cv=5, scoring="neg_mean_absolute_error")Explanation: Error metrics are given with the "neg_" prefix.
3 Match the models.
| Term | Meaning |
|---|---|
DummyRegressor |
Baseline |
LinearRegression |
Plain linear model |
Ridge |
Penalised linear model |
neg_mean_absolute_error |
MAE for CV |