CodeBlanks Logo Icon CodeBlanks
Login Register

Tensor properties

A tensor has shape, dtype, and device properties.

shape, dtype, device

Every tensor has three basic properties:

x = torch.zeros(2, 3)
print(x.shape)     # torch.Size([2, 3])
print(x.dtype)     # torch.float32
print(x.device)    # cpu
print(x.numel())   # 6 elements

In deep learning the default type is float32; a tensor created from a list of integers is int64. Class labels should be int64 (torch.long), inputs float32.

Type conversion

To change the type, .to(dtype) or its shortcuts (.float(), .long()) are used. Operating on different types sometimes gives an error, sometimes a silent type promotion:

a = torch.tensor([1, 2, 3])
b = a.float()                 # float32
c = a.to(torch.float64)
print((a / 2).dtype)          # torch.float32: division produces decimals
print((a // 2).tolist())      # [0, 1, 1]

Decimal arrays coming from NumPy are float64; before giving them to a model they must be converted to float32 with .float().

Practice Types

For a = torch.tensor([1, 2, 3]), print a.dtype, (a / 2).dtype and a.float().dtype on separate lines. Then print the shape and numel() of torch.zeros(3, 4, 5) on one line.

Starter code

model.py
import torch

# buraya yaz

Sample solution

model.py
import torch

a = torch.tensor([1, 2, 3])
print(a.dtype)
print((a / 2).dtype)
print(a.float().dtype)
z = torch.zeros(3, 4, 5)
print(z.shape, z.numel())

Expected output

torch.int64
torch.float32
torch.float32
torch.Size([3, 4, 5]) 60

Questions and answers

Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.

1 What is <code>torch.tensor([1, 2, 3]).dtype</code>?

  • torch.int64 Correct answer
  • torch.float32
  • torch.int32
  • torch.float64

Explanation: Integer lists produce an int64 tensor.

2 Convert the tensor to float32.

Question 
x = a.____()
Answer 
x = a.float()

Explanation: The .float() shortcut produces float32.

3 Match the properties.

Term Meaning
shape Dimensions
dtype Element type
device cpu or cuda
numel() Element count

All lessons in Advanced

1 Neural network basics

  1. What is a neural network?
  2. Neuron
  3. Layers
  4. Activation
  5. Forward propagation
  6. Summary - Neural network

2 PyTorch basics

  1. What is PyTorch?
  2. Tensor
  3. Creating tensors
  4. Tensor properties
  5. GPU support
  6. Summary - PyTorch

3 Tensor operations

  1. Tensor operations
  2. Shape
  3. Dtype
  4. Reshape
  5. Indexing
  6. Summary - Tensor operations

4 Autograd

  1. What is autograd?
  2. requires_grad
  3. backward()
  4. grad
  5. Computational graph
  6. Summary - Autograd

5 Project: Neural network

  1. Preparing tensors
  2. Model definition
  3. Loss computation
  4. Optimizer

6 nn.Module

  1. What is nn.Module?
  2. __init__
  3. forward
  4. nn.Linear
  5. parameters()
  6. Summary - nn.Module

7 Training loop

  1. Training loop
  2. Epoch
  3. Loss
  4. backward()
  5. optimizer.step()
  6. Summary - Training loop

8 CNN

  1. What is CNN?
  2. Conv2d
  3. MaxPool2d
  4. Feature map
  5. Flatten
  6. Summary - CNN

9 RNN

  1. What is RNN?
  2. Hidden state
  3. Sequence
  4. Vanishing gradient
  5. Embedding
  6. Summary - RNN

10 LSTM

  1. What is LSTM?
  2. Gates
  3. Cell state
  4. input_size
  5. hidden_size
  6. Summary - LSTM

11 Project: CNN

  1. Preparing data
  2. CNN model
  3. Loss and optimizer
  4. Training

12 Transfer learning

  1. What is transfer learning?
  2. Pre-trained model
  3. Fine-tuning
  4. Feature extraction
  5. Advantages
  6. Summary - Transfer learning

13 Optimization

  1. What is an optimizer?
  2. SGD
  3. Adam
  4. Learning rate
  5. zero_grad()
  6. Summary - Optimization

14 Advanced final

  1. Challenge 1
  2. Challenge 2
  3. Challenge 3
  4. Challenge 4