CodeBlanks Logo Icon CodeBlanks
Login Register

GPU support

PyTorch moves tensors to GPU for fast computation.

GPU support

Deep learning's speed comes from doing matrix multiplications in parallel on the GPU. In PyTorch a tensor or model is moved to the GPU with .to(device). Code is written so it falls back to the CPU if there is no GPU:

cihaz = "cuda" if torch.cuda.is_available() else "cpu"
x = torch.randn(1000, 1000).to(cihaz)
model = model.to(cihaz)
y = model(x)          # computed on the chosen device

On Apple chips the "mps" device is used: torch.backends.mps.is_available().

Device patterns

In the training loop each batch is moved to the same device before being given to the model. To turn results into NumPy, you must first go back to the CPU:

for xb, yb in yukleyici:
    xb, yb = xb.to(cihaz), yb.to(cihaz)
    cikti = model(xb)
tahmin = cikti.argmax(dim=1).cpu().numpy()   # GPU -> CPU -> NumPy

Copying between GPU and CPU is slow; needless .cpu() calls inside the loop are avoided.

Practice Device-independent code

Choose the device with "cuda" if torch.cuda.is_available() else "cpu". Move x = torch.ones(2, 3) to the device, multiply by 3, bring the result back with .cpu() and print it with tolist(). Then print that the result's device.type is "cpu".

Starter code

model.py
import torch

# buraya yaz

Sample solution

model.py
import torch

cihaz = "cuda" if torch.cuda.is_available() else "cpu"
x = torch.ones(2, 3).to(cihaz)
sonuc = (x * 3).cpu()
print(sonuc.tolist())
print(sonuc.device.type == "cpu")

Expected output

[[3.0, 3.0, 3.0], [3.0, 3.0, 3.0]]
True

Questions and answers

Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.

1 What happens if you add two tensors, one on the CPU and one on the GPU?

  • It is moved automatically
  • The result is on the CPU
  • It works fine
  • It errors; both must be on the same device Correct answer

Explanation: All tensors in an operation and the model must be on the same device.

2 Move the tensor to the chosen device.

Question 
x = x.____(cihaz)
Answer 
x = x.to(cihaz)

Explanation: .to() can change both device and type.

3 Match the expressions.

Term Meaning
torch.cuda.is_available() Is there a GPU?
.to(cihaz) Move to device
.cpu() Back to the CPU
"mps" Apple GPU

All lessons in Advanced

1 Neural network basics

  1. What is a neural network?
  2. Neuron
  3. Layers
  4. Activation
  5. Forward propagation
  6. Summary - Neural network

2 PyTorch basics

  1. What is PyTorch?
  2. Tensor
  3. Creating tensors
  4. Tensor properties
  5. GPU support
  6. Summary - PyTorch

3 Tensor operations

  1. Tensor operations
  2. Shape
  3. Dtype
  4. Reshape
  5. Indexing
  6. Summary - Tensor operations

4 Autograd

  1. What is autograd?
  2. requires_grad
  3. backward()
  4. grad
  5. Computational graph
  6. Summary - Autograd

5 Project: Neural network

  1. Preparing tensors
  2. Model definition
  3. Loss computation
  4. Optimizer

6 nn.Module

  1. What is nn.Module?
  2. __init__
  3. forward
  4. nn.Linear
  5. parameters()
  6. Summary - nn.Module

7 Training loop

  1. Training loop
  2. Epoch
  3. Loss
  4. backward()
  5. optimizer.step()
  6. Summary - Training loop

8 CNN

  1. What is CNN?
  2. Conv2d
  3. MaxPool2d
  4. Feature map
  5. Flatten
  6. Summary - CNN

9 RNN

  1. What is RNN?
  2. Hidden state
  3. Sequence
  4. Vanishing gradient
  5. Embedding
  6. Summary - RNN

10 LSTM

  1. What is LSTM?
  2. Gates
  3. Cell state
  4. input_size
  5. hidden_size
  6. Summary - LSTM

11 Project: CNN

  1. Preparing data
  2. CNN model
  3. Loss and optimizer
  4. Training

12 Transfer learning

  1. What is transfer learning?
  2. Pre-trained model
  3. Fine-tuning
  4. Feature extraction
  5. Advantages
  6. Summary - Transfer learning

13 Optimization

  1. What is an optimizer?
  2. SGD
  3. Adam
  4. Learning rate
  5. zero_grad()
  6. Summary - Optimization

14 Advanced final

  1. Challenge 1
  2. Challenge 2
  3. Challenge 3
  4. Challenge 4