Summary - Training loop

In this group you learned epoch, loss, backward, and optimizer.

Summary: the training loop

In this group:

Five steps: zero_grad → forward → loss → backward → step.
TensorDataset + DataLoader (batch_size, shuffle); an epoch = a full pass over the data.
model.train()/model.eval() is for layer behaviour, torch.no_grad() for gradient tracking.
Losses: MSE, CrossEntropy (raw logits), BCEWithLogits; reduction.
Gradient clipping, accumulation; the SGD rule and learning rate schedulers.

Exercise Train and validate

Let the first 150 examples be training and the last 50 validation. Train 40 epochs with batch_size=32, shuffle=True; at the end of every epoch compute the validation accuracy inside model.eval() and torch.no_grad(), collect it in a list, then return to model.train(). Print on one line that the final accuracy is above 0.9 and that the list holds 40 values.

Show example solution

import torch
from torch import nn
from torch.utils.data import TensorDataset, DataLoader

torch.manual_seed(0)
X = torch.randn(200, 2)
y = (X[:, 0] + X[:, 1] > 0).long()
model = nn.Sequential(nn.Linear(2, 16), nn.ReLU(), nn.Linear(16, 2))
kayip_fn = nn.CrossEntropyLoss()
opt = torch.optim.SGD(model.parameters(), lr=0.1)

yukleyici = DataLoader(TensorDataset(X[:150], y[:150]), batch_size=32, shuffle=True)
dogruluk = []
for epoch in range(40):
    for xb, yb in yukleyici:
        opt.zero_grad()
        kayip_fn(model(xb), yb).backward()
        opt.step()
    model.eval()
    with torch.no_grad():
        dogruluk.append((model(X[150:]).argmax(dim=1) == y[150:]).float().mean().item())
    model.train()
print(dogruluk[-1] > 0.9, len(dogruluk) == 40)

Test yourself

What is needed while computing validation accuracy?
- model.eval() and torch.no_grad()
- opt.step()
- kayip.backward()
- shuffle=True
Show answer

model.eval() and torch.no_grad() — No parameters are updated in evaluation; layers must be in test behaviour.
Put the model back in training mode.
```
model.____()
```
Show answer
```
model.train()
```
Match the tools.

DataLoader

Produces batches
CrossEntropyLoss

Classification loss
clip_grad_norm_

Gradient limiting
lr_scheduler

Rate scheduling

Next lesson: What is CNN?

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies