Challenge 1
- Machine Learning course
- Beginner
- Lesson 109/112
- 8 min
Combine ML fundamentals to analyze data.
Challenge 1: linear regression by hand
In this challenge you'll solve single-variable linear regression with NumPy, without scikit-learn. The least-squares solution has a closed form:
w = ((x - x.mean()) * (y - y.mean())).sum() / ((x - x.mean()) ** 2).sum()
b = y.mean() - w * x.mean()The slope is the ratio of how x and y vary together to how x varies on its own; the intercept makes the line pass through the point of means.
Practice Your own regression
Compute w and b with the formula and print them on one line with round(..., 3). Then verify the result with LinearRegression: print np.isclose(w, m.coef_[0]).
Starter code
import numpy as np
from sklearn.linear_model import LinearRegression
x = np.array([1.0, 2.0, 3.0, 4.0, 5.0, 6.0])
y = np.array([2.9, 5.1, 7.0, 8.8, 11.2, 13.1])
# buraya yazSample solution
import numpy as np
from sklearn.linear_model import LinearRegression
x = np.array([1.0, 2.0, 3.0, 4.0, 5.0, 6.0])
y = np.array([2.9, 5.1, 7.0, 8.8, 11.2, 13.1])
w = ((x - x.mean()) * (y - y.mean())).sum() / ((x - x.mean()) ** 2).sum()
b = y.mean() - w * x.mean()
print(round(w, 3), round(b, 3))
m = LinearRegression().fit(x.reshape(-1, 1), y)
print(np.isclose(w, m.coef_[0]))Expected output
2.031 0.907 True
Practice Compute R² by hand
Compute the predictions with the line you found. Print R² with round(..., 4) using R² = 1 - SS_res / SS_tot (SS_res = Σ(y - ŷ)², SS_tot = Σ(y - ȳ)²).
Starter code
import numpy as np
x = np.array([1.0, 2.0, 3.0, 4.0, 5.0, 6.0])
y = np.array([2.9, 5.1, 7.0, 8.8, 11.2, 13.1])
w = ((x - x.mean()) * (y - y.mean())).sum() / ((x - x.mean()) ** 2).sum()
b = y.mean() - w * x.mean()
# buraya yazSample solution
import numpy as np
x = np.array([1.0, 2.0, 3.0, 4.0, 5.0, 6.0])
y = np.array([2.9, 5.1, 7.0, 8.8, 11.2, 13.1])
w = ((x - x.mean()) * (y - y.mean())).sum() / ((x - x.mean()) ** 2).sum()
b = y.mean() - w * x.mean()
p = w * x + b
ss_res = ((y - p) ** 2).sum()
ss_tot = ((y - y.mean()) ** 2).sum()
print(round(1 - ss_res / ss_tot, 4))Expected output
0.9987
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 Which point does the line found with this formula always pass through?
- (max of x, max of y)
- (mean of x, mean of y) Correct answer
- (0, 0)
- The first data point
Explanation: Since b = ȳ - w·x̄, the prediction for x = x̄ is ȳ.
2 Compute the intercept.
b = y.mean() - w * ____b = y.mean() - w * x.mean()Explanation: The intercept comes from the condition of passing through the point of means.
3 Match the pieces.
| Term | Meaning |
|---|---|
x - x.mean() |
Centred x |
pay |
Joint variation |
payda |
Variation of x |
b |
Intercept |