Linear regression

Linear regression finds the linear relationship in data and fits a line.

Fitting a line

Linear regression fits the line y = w·x + b to the data. w is the slope, b the intercept. With several features each feature has its own weight: y = w₁x₁ + w₂x₂ + ... + b.

m = LinearRegression().fit([[1], [2], [3]], [2, 4, 6])
print(m.coef_)       # [2.]
print(m.intercept_)  # 0.0 (approximately)

The weights are interpretable: w = 2 says y rises by two units when x rises by one.

Least squares

The "best" line is the one that minimises the sum of squared prediction errors. Squaring both equalises negative and positive errors and penalises large errors more.

x = np.array([1, 2, 3, 4])
y = np.array([2.1, 3.9, 6.2, 7.8])
w, b = np.polyfit(x, y, 1)   # degree 1: a line
print(round(w, 2), round(b, 2))   # 1.94 0.15

np.polyfit and LinearRegression find the same least-squares solution.

Exercise Slope and intercept

Train a LinearRegression with the two-feature X. Round coef_ and intercept_ with np.round(..., 2) and print them on separate lines.

Show example solution

import numpy as np
from sklearn.linear_model import LinearRegression

X = np.array([[1, 0], [0, 1], [1, 1], [2, 1]])
y = 3 * X[:, 0] + 2 * X[:, 1] + 5

m = LinearRegression().fit(X, y)
print(np.round(m.coef_, 2))
print(np.round(m.intercept_, 2))

Test yourself

In a house model, what does a square-metre weight of 15 mean?
- The price is 15 units
- 15 m² houses are the most expensive
- The model is 15 percent wrong
- Each extra m² raises the price by 15 units
Show answer

Each extra m² raises the price by 15 units — The weight is the one-unit effect of that feature with the others held fixed.
Read the learned slope.
```
egim = m.____[0]
```
Show answer
```
egim = m.coef_[0]
```
Match the terms.

w

Slope (weight)
b

Intercept
least squares

Minimising squared error sum
np.polyfit

Polynomial fitting

Next lesson: Model training

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies