Dtype

Dtype specifies the data type of tensor elements.

Dtype

The element type determines memory, speed and accuracy:

float32: the default, standard for training.
float16 / bfloat16: half precision; twice the speed and half the memory on the GPU (mixed-precision training).
int64 (long): indices and class labels.
bool: masks.

x = torch.tensor([1.0, 2.0])
print(x.element_size())             # 4 bytes
print(x.half().element_size())      # 2 bytes
print((x > 1).dtype)                # torch.bool

Precision and overflow

Low-precision types lose small differences and overflow on large values. The largest value of float16 is 65504:

print(torch.tensor(70000.0).half().item())        # inf: overflow
print(torch.tensor(1.0001).half().item())         # 1.0: the detail is lost
print(torch.tensor(70000.0).bfloat16().item())    # 70144.0: bfloat16 has a wide range

bfloat16 has the same exponent range as float32, so it doesn't overflow but keeps fewer decimal digits; it is often used when training large models.

Exercise Memory calculation

Print the size in bytes of x = torch.zeros(1000, 1000) (x.element_size() * x.numel()). Do the same calculation for x.half(). Then print torch.tensor(70000.0).half().item().

Show example solution

import torch

x = torch.zeros(1000, 1000)

print(x.element_size() * x.numel())
h = x.half()
print(h.element_size() * h.numel())
print(torch.tensor(70000.0).half().item())

Test yourself

Roughly how much memory does a 1-billion-parameter model take in float32?
- 8 GB
- 4 GB
- 1 GB
- 32 GB
Show answer

4 GB — 4 bytes per parameter: 10⁹ × 4 = 4·10⁹ bytes ≈ 4 GB (2 GB in float16).
Convert the tensor to half precision.
```
y = x.____()
```
Show answer
```
y = x.half()
```
Match the types.

float32

4 bytes, default
float16

2 bytes, narrow range
bfloat16

2 bytes, wide range
bool

Mask

Next lesson: Reshape

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies