Machine Learning
__init__
Layers are defined in the __init__ method.
__init__
Inside __init__, super().__init__() is called first, then the layers are assigned as attributes. Every nn.Module or nn.Parameter assigned to the module is registered automatically:
class Siniflandirici(nn.Module):
def __init__(self, girdi, gizli, sinif):
super().__init__()
self.gizli = nn.Linear(girdi, gizli)
self.cikti = nn.Linear(gizli, sinif)
self.olcek = nn.Parameter(torch.ones(1)) # a custom parameter
If super().__init__() is forgotten, the first assignment raises "cannot assign module before Module.__init__() call".
ModuleList and ModuleDict
If a variable number of layers is needed, nn.ModuleList is used, and nn.ModuleDict for named layers; their contents become registered submodules:
class Derin(nn.Module):
def __init__(self, boyutlar):
super().__init__()
self.katmanlar = nn.ModuleList(
nn.Linear(a, b) for a, b in zip(boyutlar[:-1], boyutlar[1:]))
def forward(self, x):
for k in self.katmanlar[:-1]:
x = torch.relu(k(x))
return self.katmanlar[-1](x)
With an ordinary list, parameters() returns empty and the model never learns; it doesn't error either.
Exercise The list trap
Two model classes are given: one keeps the layers in a Python list, the other in an nn.ModuleList. Print both parameter counts (sum(p.numel() for p in m.parameters())) on one line.
Show example solution
import torch
from torch import nn
class ListeModel(nn.Module):
def __init__(self):
super().__init__()
self.katmanlar = [nn.Linear(4, 4), nn.Linear(4, 2)]
class ModulModel(nn.Module):
def __init__(self):
super().__init__()
self.katmanlar = nn.ModuleList([nn.Linear(4, 4), nn.Linear(4, 2)])
say = lambda m: sum(p.numel() for p in m.parameters())
print(say(ListeModel()), say(ModulModel()))
Test yourself
-
What happens if the layers are kept in an ordinary Python list?
- It works fine
- It errors
- It is faster
- Their parameters aren't registered; the optimiser doesn't see them
Show answer
Their parameters aren't registered; the optimiser doesn't see them — nn.ModuleList must be used for a list.
-
Call the parent class's constructor.
def __init__(self):
____().__init__()
Show answer
def __init__(self):
super().__init__()
-
Match the containers.
- nn.ModuleList
- Registered layer list
- nn.ModuleDict
- Named layers
- nn.Parameter
- Custom learned tensor
- Python listesi
- Not registered
Next lesson: forward
