Tensor operations
- Machine Learning course
- Advanced
- Lesson 13/78
- 5 min
Addition, multiplication, and reshape operations are done on tensors.
Tensor operations
Arithmetic operations are element-wise and broadcasting follows NumPy rules. Matrix multiplication is done with @ or torch.matmul:
a = torch.tensor([[1., 2.], [3., 4.]])
b = torch.tensor([10., 20.])
print((a + b).tolist()) # [[11.0, 22.0], [13.0, 24.0]] broadcasting
print((a @ a).tolist()) # [[7.0, 10.0], [15.0, 22.0]]
print(a.sum(dim=0).tolist()) # [4.0, 6.0] column sumsIn PyTorch you write dim instead of NumPy's axis (most functions accept both).
In-place operations
Methods ending in an underscore change the tensor in place: x.add_(1), x.mul_(2), x.zero_(). They save memory but must be used carefully with autograd; overwriting a value needed for backpropagation causes an error.
x = torch.tensor([1., 2., 3.])
x.mul_(2)
print(x.tolist()) # [2.0, 4.0, 6.0]
y = x.add(1) # a new tensor; x unchangedIn the training loop, the weight update and gradient zeroing are done in place by the optimiser.
Practice Basic operations
For matrix a, print the row means (mean(dim=1)), the result of a @ a.T and the largest element of a with tolist()/item() on separate lines.
Starter code
import torch
a = torch.tensor([[1., 2., 3.], [4., 5., 6.]])
# buraya yazSample solution
import torch
a = torch.tensor([[1., 2., 3.], [4., 5., 6.]])
print(a.mean(dim=1).tolist())
print((a @ a.T).tolist())
print(a.max().item())Expected output
[2.0, 5.0] [[14.0, 32.0], [32.0, 77.0]] 6.0
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 What does <code>torch.tensor([[1, 2], [3, 4]]).sum(dim=1).tolist()</code> give?
- [4, 6]
- 10
- [1, 2, 3, 4]
- [3, 7] Correct answer
Explanation: dim=1 sums within each row: 1+2 = 3, 3+4 = 7.
2 Multiply the tensor by two in place.
x.____(2)x.mul_(2)Explanation: The trailing underscore means an in-place operation.
3 Match the operations.
| Term | Meaning |
|---|---|
a * b |
Element-wise multiplication |
a @ b |
Matrix multiplication |
a.sum(dim=0) |
Column sum |
x.add_(1) |
In-place addition |