Machine Learning
Training model
Train a linear regression model.
Model training
We'll train a baseline and two candidates: DummyRegressor predicting the mean, LinearRegression and lightly regularised Ridge:
from sklearn.dummy import DummyRegressor
from sklearn.linear_model import LinearRegression, Ridge
adaylar = {
"ortalama": DummyRegressor(),
"dogrusal": LinearRegression(),
"ridge": Ridge(alpha=1.0),
}
Ridge penalises large weights and makes the model more stable against noise; you'll see the details at the intermediate level.
Choosing with cross-validation
We compare the candidates with 5-fold validation on the training set, without looking at the test set. We'll use MAE as the metric:
for ad, m in adaylar.items():
s = -cross_val_score(m, X_train, y_train, cv=5,
scoring="neg_mean_absolute_error")
print(ad, s.mean().round(1))
The leading minus sign turns the error that scikit-learn gives as negative back into a positive. Because the features in this data are on a very small scale, alpha=1.0 penalises too much and Ridge falls behind; in the next steps we'll continue with LinearRegression, which gives the lowest MAE.
Exercise Validate the candidates
Compare the three candidates on MAE with 5-fold validation on the training set. On each line print the name and the mean MAE with round(..., 1).
Show example solution
from sklearn.datasets import load_diabetes
from sklearn.model_selection import train_test_split
X, y = load_diabetes(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
from sklearn.model_selection import cross_val_score
from sklearn.dummy import DummyRegressor
from sklearn.linear_model import LinearRegression, Ridge
adaylar = {"ortalama": DummyRegressor(), "dogrusal": LinearRegression(), "ridge": Ridge(alpha=1.0)}
for ad, m in adaylar.items():
s = -cross_val_score(m, X_train, y_train, cv=5, scoring="neg_mean_absolute_error")
print(ad, round(s.mean(), 1))
Test yourself
-
What does <code>DummyRegressor()</code> predict by default?
- A random value
- The median
- The mean of the training target
- Zero
Show answer
The mean of the training target — strategy="mean" is the default; its R² comes out around 0.
-
Use negative MAE as the metric.
cross_val_score(m, X_train, y_train, cv=5, scoring=____)
Show answer
cross_val_score(m, X_train, y_train, cv=5, scoring="neg_mean_absolute_error")
-
Match the models.
- DummyRegressor
- Baseline
- LinearRegression
- Plain linear model
- Ridge
- Penalised linear model
- neg_mean_absolute_error
- MAE for CV
Next lesson: Prediction
