Preparing tensors
- Machine Learning course
- Advanced
- Lesson 25/78
- 7 min
Convert training data to tensors.
Project: linear regression from scratch
In this project we'll build linear regression by hand using PyTorch's building blocks: we'll prepare the tensors, define the model, compute the loss and learn the parameters with gradient descent. The data comes from the relationship y = 3x + 2; we expect the model to find w ≈ 3, b ≈ 2.
x = torch.arange(10, dtype=torch.float32).unsqueeze(1) # (10, 1)
y = 3 * x + 2 # (10, 1)In the next groups we'll shorten the same job with nn.Module, nn.MSELoss and optim.SGD; seeing the mechanism first is important.
Column format
Models expect input of shape (example, feature). With single-feature data this means turning a (10,) vector into a (10, 1) column. If the target is kept in the same shape too, there is no unwanted broadcasting in the loss computation:
print(x.shape, y.shape) # torch.Size([10, 1]) torch.Size([10, 1])
# careful: (10, 1) - (10,) broadcasts to (10, 10), the loss comes out wrongThis is one of the most common silent mistakes in PyTorch: the code runs but the result is wrong.
Practice Prepare the tensors
Print the shapes of x and y on one line. Then print the shape produced by the wrong broadcasting with (y - x.squeeze(1)).shape and the correct shape with (y - x).shape on separate lines.
Starter code
import torch
x = torch.arange(10, dtype=torch.float32).unsqueeze(1)
y = 3 * x + 2
# buraya yazSample solution
import torch
x = torch.arange(10, dtype=torch.float32).unsqueeze(1)
y = 3 * x + 2
print(x.shape, y.shape)
print((y - x.squeeze(1)).shape)
print((y - x).shape)Expected output
torch.Size([10, 1]) torch.Size([10, 1]) torch.Size([10, 10]) torch.Size([10, 1])
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 Why is <code>dtype=torch.float32</code> given?
- For the GPU
- arange produces integers; gradients need a floating type Correct answer
- Because it is faster
- It isn't required
Explanation: Parameters multiplied by integer tensors have mismatched types and can't be differentiated.
2 Turn the vector into a column.
x = torch.arange(10, dtype=torch.float32).____(1)x = torch.arange(10, dtype=torch.float32).unsqueeze(1)Explanation: unsqueeze(1) turns the shape (10,) into (10, 1).
3 Match the shapes.
| Term | Meaning |
|---|---|
(10,) |
Vector |
(10, 1) |
Column |
(10, 1) - (10,) |
(10, 10) error |
(10, 1) - (10, 1) |
Correct difference |