Loss computation
- Machine Learning course
- Advanced
- Lesson 27/78
- 7 min
Compute MSE loss.
Computing the loss
In regression the loss is the mean squared error (MSE). Since the loss is a scalar, backward() can be called directly:
def mse(tahmin, hedef):
return ((tahmin - hedef) ** 2).mean()
kayip = mse(model(x), y)
kayip.backward()
print(w.grad, b.grad)While the parameters are zero the gradients are negative: they say w and b must increase to lower the loss.
Verifying the gradient by hand
MSE's derivatives are known in closed form: ∂L/∂w = mean(2(ŷ - y)·x), ∂L/∂b = mean(2(ŷ - y)). Comparing the autograd result with these formulas shows the code is correct:
fark = model(x) - y
print(torch.allclose(w.grad, (2 * fark * x).mean())) # True
print(torch.allclose(b.grad, (2 * fark).mean())) # TrueIn large models derivatives aren't written by hand, but such small tests help grasp the logic of autograd.
Practice First loss and gradient
With the parameters at zero, compute the MSE loss and call backward(). Print the loss, w.grad and b.grad with item() on one line. (The y values are 2, 5, ..., 29; mean square = 314.5.)
Starter code
import torch
x = torch.arange(10, dtype=torch.float32).unsqueeze(1)
y = 3 * x + 2
w = torch.zeros(1, requires_grad=True)
b = torch.zeros(1, requires_grad=True)
def model(x):
return x * w + b
# buraya yazSample solution
import torch
x = torch.arange(10, dtype=torch.float32).unsqueeze(1)
y = 3 * x + 2
w = torch.zeros(1, requires_grad=True)
b = torch.zeros(1, requires_grad=True)
def model(x):
return x * w + b
kayip = ((model(x) - y) ** 2).mean()
kayip.backward()
print(kayip.item(), w.grad.item(), b.grad.item())Expected output
314.5 -189.0 -31.0
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 If the gradient is negative, how does gradient descent change the parameter?
- Decreases it
- Zeroes it
- Doesn't change it
- Increases it Correct answer
Explanation: w ← w - lr·grad; subtracting a negative grad increases w.
2 Compute the mean squared error.
kayip = ((tahmin - hedef) ** 2).____()kayip = ((tahmin - hedef) ** 2).mean()Explanation: MSE is the mean of the squared differences.
3 Match the derivatives.
| Term | Meaning |
|---|---|
∂L/∂w |
mean(2(ŷ - y)·x) |
∂L/∂b |
mean(2(ŷ - y)) |
negative gradient |
The parameter must grow |
MSE |
Regression loss |