parameters()
- Machine Learning course
- Advanced
- Lesson 33/78
- 5 min
The parameters() method returns the model's learnable parameters.
parameters()
parameters() walks all the learned tensors in the module; this is what is given to the optimiser. named_parameters() returns them with their names:
model = nn.Sequential(nn.Linear(4, 8), nn.ReLU(), nn.Linear(8, 3))
for ad, p in model.named_parameters():
print(ad, tuple(p.shape))
# 0.weight (8, 4) 0.bias (8,) 2.weight (3, 8) 2.bias (3,)
print(sum(p.numel() for p in model.parameters())) # 67Because ReLU has no parameters, the name with number 1 doesn't appear in the list.
Freezing and state_dict
If a parameter's requires_grad is turned off, that parameter is frozen: it gets no gradient and isn't updated. In transfer learning pretrained layers are protected this way. state_dict() collects all parameters and buffers by name in a dict; it is the standard way to save a model:
for p in model[0].parameters():
p.requires_grad = False # freeze the first layer
egitilebilir = sum(p.numel() for p in model.parameters() if p.requires_grad)
torch.save(model.state_dict(), "model.pt")
model.load_state_dict(torch.load("model.pt"))Practice Count and freeze parameters
For nn.Sequential(nn.Linear(4, 8), nn.ReLU(), nn.Linear(8, 3)), print the parameter names as a list and the total parameter count on separate lines. Then freeze the first layer and print the number of trainable parameters.
Starter code
import torch
from torch import nn
model = nn.Sequential(nn.Linear(4, 8), nn.ReLU(), nn.Linear(8, 3))
# buraya yazSample solution
import torch
from torch import nn
model = nn.Sequential(nn.Linear(4, 8), nn.ReLU(), nn.Linear(8, 3))
print([ad for ad, _ in model.named_parameters()])
print(sum(p.numel() for p in model.parameters()))
for p in model[0].parameters():
p.requires_grad = False
print(sum(p.numel() for p in model.parameters() if p.requires_grad))Expected output
['0.weight', '0.bias', '2.weight', '2.bias'] 67 27
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 How many parameters does <code>nn.Sequential(nn.Linear(10, 5), nn.ReLU(), nn.Linear(5, 1))</code> contain?
- 61 Correct answer
- 56
- 50
- 66
Explanation: 10·5 + 5 = 55; 5·1 + 1 = 6; 61 in total.
2 Walk the parameters with their names.
for ad, p in model.____():for ad, p in model.named_parameters():Explanation: named_parameters yields (name, tensor) pairs.
3 Match the methods.
| Term | Meaning |
|---|---|
parameters() |
Learned tensors |
named_parameters() |
With their names |
state_dict() |
Dict to save |
requires_grad = False |
Freezing |