Tensor operations

Addition, multiplication, and reshape operations are done on tensors.

Tensor operations

Arithmetic operations are element-wise and broadcasting follows NumPy rules. Matrix multiplication is done with @ or torch.matmul:

a = torch.tensor([[1., 2.], [3., 4.]])
b = torch.tensor([10., 20.])
print((a + b).tolist())    # [[11.0, 22.0], [13.0, 24.0]]  broadcasting
print((a @ a).tolist())    # [[7.0, 10.0], [15.0, 22.0]]
print(a.sum(dim=0).tolist())   # [4.0, 6.0]  column sums

In PyTorch you write dim instead of NumPy's axis (most functions accept both).

In-place operations

Methods ending in an underscore change the tensor in place: x.add_(1), x.mul_(2), x.zero_(). They save memory but must be used carefully with autograd; overwriting a value needed for backpropagation causes an error.

x = torch.tensor([1., 2., 3.])
x.mul_(2)
print(x.tolist())     # [2.0, 4.0, 6.0]
y = x.add(1)          # a new tensor; x unchanged

In the training loop, the weight update and gradient zeroing are done in place by the optimiser.

Exercise Basic operations

For matrix a, print the row means (mean(dim=1)), the result of a @ a.T and the largest element of a with tolist()/item() on separate lines.

Show example solution

import torch

a = torch.tensor([[1., 2., 3.], [4., 5., 6.]])

print(a.mean(dim=1).tolist())
print((a @ a.T).tolist())
print(a.max().item())

Test yourself

What does <code>torch.tensor([[1, 2], [3, 4]]).sum(dim=1).tolist()</code> give?
- [4, 6]
- 10
- [1, 2, 3, 4]
- [3, 7]
Show answer

[3, 7] — dim=1 sums within each row: 1+2 = 3, 3+4 = 7.
Multiply the tensor by two in place.
```
x.____(2)
```
Show answer
```
x.mul_(2)
```
Match the operations.

a * b

Element-wise multiplication
a @ b

Matrix multiplication
a.sum(dim=0)

Column sum
x.add_(1)

In-place addition

Next lesson: Shape

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies