GPU support

PyTorch moves tensors to GPU for fast computation.

GPU support

Deep learning's speed comes from doing matrix multiplications in parallel on the GPU. In PyTorch a tensor or model is moved to the GPU with .to(device). Code is written so it falls back to the CPU if there is no GPU:

cihaz = "cuda" if torch.cuda.is_available() else "cpu"
x = torch.randn(1000, 1000).to(cihaz)
model = model.to(cihaz)
y = model(x)          # computed on the chosen device

On Apple chips the "mps" device is used: torch.backends.mps.is_available().

Device patterns

In the training loop each batch is moved to the same device before being given to the model. To turn results into NumPy, you must first go back to the CPU:

for xb, yb in yukleyici:
    xb, yb = xb.to(cihaz), yb.to(cihaz)
    cikti = model(xb)
tahmin = cikti.argmax(dim=1).cpu().numpy()   # GPU -> CPU -> NumPy

Copying between GPU and CPU is slow; needless .cpu() calls inside the loop are avoided.

Exercise Device-independent code

Choose the device with "cuda" if torch.cuda.is_available() else "cpu". Move x = torch.ones(2, 3) to the device, multiply by 3, bring the result back with .cpu() and print it with tolist(). Then print that the result's device.type is "cpu".

Show example solution

import torch

cihaz = "cuda" if torch.cuda.is_available() else "cpu"
x = torch.ones(2, 3).to(cihaz)
sonuc = (x * 3).cpu()
print(sonuc.tolist())
print(sonuc.device.type == "cpu")

Test yourself

What happens if you add two tensors, one on the CPU and one on the GPU?
- It is moved automatically
- The result is on the CPU
- It works fine
- It errors; both must be on the same device
Show answer

It errors; both must be on the same device — All tensors in an operation and the model must be on the same device.
Move the tensor to the chosen device.
```
x = x.____(cihaz)
```
Show answer
```
x = x.to(cihaz)
```
Match the expressions.

torch.cuda.is_available()

Is there a GPU?
.to(cihaz)

Move to device
.cpu()

Back to the CPU
"mps"

Apple GPU

Next lesson: Summary - PyTorch

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies