Epoch
- Machine Learning course
- Advanced
- Lesson 36/78
- 5 min
An epoch is one complete pass through all training data.
Epoch
An epoch is one pass of the whole training data through the model. With N examples and batch size B, an epoch takes ⌈N/B⌉ steps. Models are usually trained for tens or hundreds of epochs:
for epoch in range(20):
toplam = 0.0
for xb, yb in yukleyici:
opt.zero_grad()
kayip = kayip_fn(model(xb), yb)
kayip.backward()
opt.step()
toplam += kayip.item() * len(xb)
print(epoch, toplam / len(ds)) # epoch mean lossWeighting the epoch loss by batch size stops the last small batch from distorting the mean.
Training and evaluation mode
Layers like dropout and BatchNorm behave differently in training and evaluation. model.train() and model.eval() switch this mode; during evaluation torch.no_grad() is used as well:
model.eval()
with torch.no_grad():
tahmin = model(X_val).argmax(dim=1)
dogruluk = (tahmin == y_val).float().mean().item()
model.train() # before going back to trainingeval() doesn't turn off gradient computation; it only changes layer behaviour. They are two separate jobs.
Practice Epoch loop
With the given model, loss and optimiser, build a batch_size=32, shuffle=True loader and train for 30 epochs. Collect each epoch's example-weighted mean loss in a list. Print son < ilk for the first and last epoch losses, then that the training accuracy is above 0.9.
Starter code
import torch
from torch import nn
from torch.utils.data import TensorDataset, DataLoader
torch.manual_seed(0)
X = torch.randn(200, 2)
y = (X[:, 0] + X[:, 1] > 0).long()
model = nn.Sequential(nn.Linear(2, 16), nn.ReLU(), nn.Linear(16, 2))
kayip_fn = nn.CrossEntropyLoss()
opt = torch.optim.SGD(model.parameters(), lr=0.1)
# buraya yazSample solution
import torch
from torch import nn
from torch.utils.data import TensorDataset, DataLoader
torch.manual_seed(0)
X = torch.randn(200, 2)
y = (X[:, 0] + X[:, 1] > 0).long()
model = nn.Sequential(nn.Linear(2, 16), nn.ReLU(), nn.Linear(16, 2))
kayip_fn = nn.CrossEntropyLoss()
opt = torch.optim.SGD(model.parameters(), lr=0.1)
yukleyici = DataLoader(TensorDataset(X, y), batch_size=32, shuffle=True)
kayiplar = []
for epoch in range(30):
toplam = 0.0
for xb, yb in yukleyici:
opt.zero_grad()
kayip = kayip_fn(model(xb), yb)
kayip.backward()
opt.step()
toplam += kayip.item() * len(xb)
kayiplar.append(toplam / len(X))
print(kayiplar[-1] < kayiplar[0])
with torch.no_grad():
print((model(X).argmax(dim=1) == y).float().mean().item() > 0.9)Expected output
True True
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 With 1000 examples and batch_size=32, how many steps is one epoch?
- 32 Correct answer
- 31
- 1000
- 33
Explanation: ⌈1000/32⌉ = 32: 31 full batches and a final batch of 8 examples.
2 Put the model in evaluation mode.
model.____()model.eval()Explanation: eval() turns dropout off and uses BatchNorm's stored statistics.
3 Match the concepts.
| Term | Meaning |
|---|---|
epoch |
A full pass over the data |
step |
One batch update |
model.eval() |
Evaluation behaviour |
torch.no_grad() |
Gradient-free computation |