__init__
- Machine Learning course
- Advanced
- Lesson 30/78
- 5 min
Layers are defined in the __init__ method.
__init__
Inside __init__, super().__init__() is called first, then the layers are assigned as attributes. Every nn.Module or nn.Parameter assigned to the module is registered automatically:
class Siniflandirici(nn.Module):
def __init__(self, girdi, gizli, sinif):
super().__init__()
self.gizli = nn.Linear(girdi, gizli)
self.cikti = nn.Linear(gizli, sinif)
self.olcek = nn.Parameter(torch.ones(1)) # a custom parameterIf super().__init__() is forgotten, the first assignment raises "cannot assign module before Module.__init__() call".
ModuleList and ModuleDict
If a variable number of layers is needed, nn.ModuleList is used, and nn.ModuleDict for named layers; their contents become registered submodules:
class Derin(nn.Module):
def __init__(self, boyutlar):
super().__init__()
self.katmanlar = nn.ModuleList(
nn.Linear(a, b) for a, b in zip(boyutlar[:-1], boyutlar[1:]))
def forward(self, x):
for k in self.katmanlar[:-1]:
x = torch.relu(k(x))
return self.katmanlar[-1](x)With an ordinary list, parameters() returns empty and the model never learns; it doesn't error either.
Practice The list trap
Two model classes are given: one keeps the layers in a Python list, the other in an nn.ModuleList. Print both parameter counts (sum(p.numel() for p in m.parameters())) on one line.
Starter code
import torch
from torch import nn
class ListeModel(nn.Module):
def __init__(self):
super().__init__()
self.katmanlar = [nn.Linear(4, 4), nn.Linear(4, 2)]
class ModulModel(nn.Module):
def __init__(self):
super().__init__()
self.katmanlar = nn.ModuleList([nn.Linear(4, 4), nn.Linear(4, 2)])
# buraya yazSample solution
import torch
from torch import nn
class ListeModel(nn.Module):
def __init__(self):
super().__init__()
self.katmanlar = [nn.Linear(4, 4), nn.Linear(4, 2)]
class ModulModel(nn.Module):
def __init__(self):
super().__init__()
self.katmanlar = nn.ModuleList([nn.Linear(4, 4), nn.Linear(4, 2)])
say = lambda m: sum(p.numel() for p in m.parameters())
print(say(ListeModel()), say(ModulModel()))Expected output
0 30
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 What happens if the layers are kept in an ordinary Python list?
- It works fine
- It errors
- It is faster
- Their parameters aren't registered; the optimiser doesn't see them Correct answer
Explanation: nn.ModuleList must be used for a list.
2 Call the parent class's constructor.
def __init__(self):
____().__init__()def __init__(self):
super().__init__()Explanation: super().__init__() sets up the module machinery.
3 Match the containers.
| Term | Meaning |
|---|---|
nn.ModuleList |
Registered layer list |
nn.ModuleDict |
Named layers |
nn.Parameter |
Custom learned tensor |
Python listesi |
Not registered |