CodeBlanks Logo Icon CodeBlanks
Login Register

What is PyTorch?

PyTorch is a deep learning library developed by Facebook.

What is PyTorch?

PyTorch is one of the most widely used libraries for deep learning. It offers three things:

  • Tensor: a multi-dimensional array like a NumPy array that can also run on the GPU.
  • Autograd: a system that automatically computes the derivatives of the operations performed.
  • nn: a toolbox of layers, loss functions and optimisers.
import torch
x = torch.tensor([1.0, 2.0, 3.0])
print(x * 2)          # tensor([2., 4., 6.])
print(x.sum().item()) # 6.0

Code runs line by line (eager); debugging is like ordinary Python.

item() and tolist()

To go from tensors to ordinary Python values, item() (single-element tensor) and tolist() (any shape) are used. They are handy when printing and saving outputs:

t = torch.tensor([[1, 2], [3, 4]])
print(t.sum().item())   # 10   (Python int)
print(t.tolist())       # [[1, 2], [3, 4]]
print(t.max().item())   # 4

On a multi-element tensor, item() raises an error.

Practice First tensor

Create torch.tensor([[1, 2, 3], [4, 5, 6]]). Print it multiplied by 10 with tolist(), then the sum of all elements with item().

Starter code

model.py
import torch

# buraya yaz

Sample solution

model.py
import torch

t = torch.tensor([[1, 2, 3], [4, 5, 6]])
print((t * 10).tolist())
print(t.sum().item())

Expected output

[[10, 20, 30], [40, 50, 60]]
21

Questions and answers

Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.

1 What are the two key features that set PyTorch apart from NumPy?

  • Shorter code and coloured output
  • Reading tables and plotting
  • There is no difference
  • GPU support and automatic differentiation Correct answer

Explanation: Tensors can be moved to the GPU and autograd computes gradients by itself.

2 Turn a single-element tensor into a Python number.

Question 
toplam = t.sum().____()
Answer 
toplam = t.sum().item()

Explanation: item() returns the value of a scalar tensor.

3 Match the concepts.

Term Meaning
torch.tensor Multi-dimensional array
autograd Automatic differentiation
torch.nn Layers and losses
tolist() To a Python list

All lessons in Advanced

1 Neural network basics

  1. What is a neural network?
  2. Neuron
  3. Layers
  4. Activation
  5. Forward propagation
  6. Summary - Neural network

2 PyTorch basics

  1. What is PyTorch?
  2. Tensor
  3. Creating tensors
  4. Tensor properties
  5. GPU support
  6. Summary - PyTorch

3 Tensor operations

  1. Tensor operations
  2. Shape
  3. Dtype
  4. Reshape
  5. Indexing
  6. Summary - Tensor operations

4 Autograd

  1. What is autograd?
  2. requires_grad
  3. backward()
  4. grad
  5. Computational graph
  6. Summary - Autograd

5 Project: Neural network

  1. Preparing tensors
  2. Model definition
  3. Loss computation
  4. Optimizer

6 nn.Module

  1. What is nn.Module?
  2. __init__
  3. forward
  4. nn.Linear
  5. parameters()
  6. Summary - nn.Module

7 Training loop

  1. Training loop
  2. Epoch
  3. Loss
  4. backward()
  5. optimizer.step()
  6. Summary - Training loop

8 CNN

  1. What is CNN?
  2. Conv2d
  3. MaxPool2d
  4. Feature map
  5. Flatten
  6. Summary - CNN

9 RNN

  1. What is RNN?
  2. Hidden state
  3. Sequence
  4. Vanishing gradient
  5. Embedding
  6. Summary - RNN

10 LSTM

  1. What is LSTM?
  2. Gates
  3. Cell state
  4. input_size
  5. hidden_size
  6. Summary - LSTM

11 Project: CNN

  1. Preparing data
  2. CNN model
  3. Loss and optimizer
  4. Training

12 Transfer learning

  1. What is transfer learning?
  2. Pre-trained model
  3. Fine-tuning
  4. Feature extraction
  5. Advantages
  6. Summary - Transfer learning

13 Optimization

  1. What is an optimizer?
  2. SGD
  3. Adam
  4. Learning rate
  5. zero_grad()
  6. Summary - Optimization

14 Advanced final

  1. Challenge 1
  2. Challenge 2
  3. Challenge 3
  4. Challenge 4