Summary - Training loop
- Machine Learning course
- Advanced
- Lesson 40/78
- 6 min
In this group you learned epoch, loss, backward, and optimizer.
Summary: the training loop
In this group:
- Five steps:
zero_grad→ forward → loss →backward→step. TensorDataset+DataLoader(batch_size, shuffle); an epoch = a full pass over the data.model.train()/model.eval()is for layer behaviour,torch.no_grad()for gradient tracking.- Losses: MSE, CrossEntropy (raw logits), BCEWithLogits;
reduction. - Gradient clipping, accumulation; the SGD rule and learning rate schedulers.
Practice Train and validate
Let the first 150 examples be training and the last 50 validation. Train 40 epochs with batch_size=32, shuffle=True; at the end of every epoch compute the validation accuracy inside model.eval() and torch.no_grad(), collect it in a list, then return to model.train(). Print on one line that the final accuracy is above 0.9 and that the list holds 40 values.
Starter code
import torch
from torch import nn
from torch.utils.data import TensorDataset, DataLoader
torch.manual_seed(0)
X = torch.randn(200, 2)
y = (X[:, 0] + X[:, 1] > 0).long()
model = nn.Sequential(nn.Linear(2, 16), nn.ReLU(), nn.Linear(16, 2))
kayip_fn = nn.CrossEntropyLoss()
opt = torch.optim.SGD(model.parameters(), lr=0.1)
# buraya yazSample solution
import torch
from torch import nn
from torch.utils.data import TensorDataset, DataLoader
torch.manual_seed(0)
X = torch.randn(200, 2)
y = (X[:, 0] + X[:, 1] > 0).long()
model = nn.Sequential(nn.Linear(2, 16), nn.ReLU(), nn.Linear(16, 2))
kayip_fn = nn.CrossEntropyLoss()
opt = torch.optim.SGD(model.parameters(), lr=0.1)
yukleyici = DataLoader(TensorDataset(X[:150], y[:150]), batch_size=32, shuffle=True)
dogruluk = []
for epoch in range(40):
for xb, yb in yukleyici:
opt.zero_grad()
kayip_fn(model(xb), yb).backward()
opt.step()
model.eval()
with torch.no_grad():
dogruluk.append((model(X[150:]).argmax(dim=1) == y[150:]).float().mean().item())
model.train()
print(dogruluk[-1] > 0.9, len(dogruluk) == 40)Expected output
True True
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 What is needed while computing validation accuracy?
- model.eval() and torch.no_grad() Correct answer
- opt.step()
- kayip.backward()
- shuffle=True
Explanation: No parameters are updated in evaluation; layers must be in test behaviour.
2 Put the model back in training mode.
Question
model.____()
Answer
model.train()Explanation: train() returns dropout and BatchNorm to training behaviour.
3 Match the tools.
| Term | Meaning |
|---|---|
DataLoader |
Produces batches |
CrossEntropyLoss |
Classification loss |
clip_grad_norm_ |
Gradient limiting |
lr_scheduler |
Rate scheduling |