CodeBlanks Logo Icon CodeBlanks
Login Register

Tensor

A tensor is PyTorch's core data structure, a multi-dimensional array.

The tensor

The tensor is PyTorch's core data structure: an n-dimensional array of numbers of the same type. It is named by its number of dimensions: scalar (0), vector (1), matrix (2), beyond that a tensor. A batch of images is typically 4-dimensional: (example, channel, height, width).

s = torch.tensor(3.0)               # 0 dimensions
v = torch.tensor([1.0, 2.0])        # 1 dimension
m = torch.tensor([[1, 2], [3, 4]])  # 2 dimensions
print(s.dim(), v.dim(), m.dim())    # 0 1 2

Element-wise operations, broadcasting and indexing work as in NumPy.

The bridge to NumPy

torch.from_numpy and .numpy() move data between the two worlds without copying; because they share the same memory, a change in one shows up in the other:

a = np.array([1, 2, 3])
t = torch.from_numpy(a)
a[0] = 100
print(t.tolist())       # [100, 2, 3]  shared memory
k = torch.tensor(a)     # a copy: independent

If an independent copy is wanted, torch.tensor(a) or t.clone() is used.

Practice Shared memory

From the array a, create t1 with torch.from_numpy and t2 with torch.tensor. Then set a[1] = 50; print t1.tolist() and t2.tolist() on separate lines.

Starter code

model.py
import numpy as np
import torch

a = np.array([1, 2, 3])

# buraya yaz

Sample solution

model.py
import numpy as np
import torch

a = np.array([1, 2, 3])

t1 = torch.from_numpy(a)
t2 = torch.tensor(a)
a[1] = 50
print(t1.tolist())
print(t2.tolist())

Expected output

[1, 50, 3]
[1, 2, 3]

Questions and answers

Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.

1 How many dimensions does a batch of 32 colour (3-channel) 64×64 images have as a tensor?

  • 2
  • 5
  • 4 Correct answer
  • 3

Explanation: The shape is (32, 3, 64, 64): example, channel, height, width.

2 Turn the NumPy array into a tensor without copying.

Question 
t = torch.____(a)
Answer 
t = torch.from_numpy(a)

Explanation: from_numpy shares memory; torch.tensor copies.

3 Match the dimensions.

Term Meaning
scalar 0 dimensions
vector 1 dimension
matrix 2 dimensions
image batch 4 dimensions

All lessons in Advanced

1 Neural network basics

  1. What is a neural network?
  2. Neuron
  3. Layers
  4. Activation
  5. Forward propagation
  6. Summary - Neural network

2 PyTorch basics

  1. What is PyTorch?
  2. Tensor
  3. Creating tensors
  4. Tensor properties
  5. GPU support
  6. Summary - PyTorch

3 Tensor operations

  1. Tensor operations
  2. Shape
  3. Dtype
  4. Reshape
  5. Indexing
  6. Summary - Tensor operations

4 Autograd

  1. What is autograd?
  2. requires_grad
  3. backward()
  4. grad
  5. Computational graph
  6. Summary - Autograd

5 Project: Neural network

  1. Preparing tensors
  2. Model definition
  3. Loss computation
  4. Optimizer

6 nn.Module

  1. What is nn.Module?
  2. __init__
  3. forward
  4. nn.Linear
  5. parameters()
  6. Summary - nn.Module

7 Training loop

  1. Training loop
  2. Epoch
  3. Loss
  4. backward()
  5. optimizer.step()
  6. Summary - Training loop

8 CNN

  1. What is CNN?
  2. Conv2d
  3. MaxPool2d
  4. Feature map
  5. Flatten
  6. Summary - CNN

9 RNN

  1. What is RNN?
  2. Hidden state
  3. Sequence
  4. Vanishing gradient
  5. Embedding
  6. Summary - RNN

10 LSTM

  1. What is LSTM?
  2. Gates
  3. Cell state
  4. input_size
  5. hidden_size
  6. Summary - LSTM

11 Project: CNN

  1. Preparing data
  2. CNN model
  3. Loss and optimizer
  4. Training

12 Transfer learning

  1. What is transfer learning?
  2. Pre-trained model
  3. Fine-tuning
  4. Feature extraction
  5. Advantages
  6. Summary - Transfer learning

13 Optimization

  1. What is an optimizer?
  2. SGD
  3. Adam
  4. Learning rate
  5. zero_grad()
  6. Summary - Optimization

14 Advanced final

  1. Challenge 1
  2. Challenge 2
  3. Challenge 3
  4. Challenge 4