Machine Learning
Preparing tensors
Convert training data to tensors.
Project: linear regression from scratch
In this project we'll build linear regression by hand using PyTorch's building blocks: we'll prepare the tensors, define the model, compute the loss and learn the parameters with gradient descent. The data comes from the relationship y = 3x + 2; we expect the model to find w ≈ 3, b ≈ 2.
x = torch.arange(10, dtype=torch.float32).unsqueeze(1) # (10, 1)
y = 3 * x + 2 # (10, 1)
In the next groups we'll shorten the same job with nn.Module, nn.MSELoss and optim.SGD; seeing the mechanism first is important.
Column format
Models expect input of shape (example, feature). With single-feature data this means turning a (10,) vector into a (10, 1) column. If the target is kept in the same shape too, there is no unwanted broadcasting in the loss computation:
print(x.shape, y.shape) # torch.Size([10, 1]) torch.Size([10, 1])
# careful: (10, 1) - (10,) broadcasts to (10, 10), the loss comes out wrong
This is one of the most common silent mistakes in PyTorch: the code runs but the result is wrong.
Exercise Prepare the tensors
Print the shapes of x and y on one line. Then print the shape produced by the wrong broadcasting with (y - x.squeeze(1)).shape and the correct shape with (y - x).shape on separate lines.
Show example solution
import torch
x = torch.arange(10, dtype=torch.float32).unsqueeze(1)
y = 3 * x + 2
print(x.shape, y.shape)
print((y - x.squeeze(1)).shape)
print((y - x).shape)
Test yourself
-
Why is <code>dtype=torch.float32</code> given?
- For the GPU
- arange produces integers; gradients need a floating type
- Because it is faster
- It isn't required
Show answer
arange produces integers; gradients need a floating type — Parameters multiplied by integer tensors have mismatched types and can't be differentiated.
-
Turn the vector into a column.
x = torch.arange(10, dtype=torch.float32).____(1)
Show answer
x = torch.arange(10, dtype=torch.float32).unsqueeze(1)
-
Match the shapes.
- (10,)
- Vector
- (10, 1)
- Column
- (10, 1) - (10,)
- (10, 10) error
- (10, 1) - (10, 1)
- Correct difference
Next lesson: Model definition
