Training
- Machine Learning course
- Advanced
- Lesson 62/78
- 7 min
Write a training loop.
Training and evaluation
Full training takes a few epochs with batches; the test accuracy is measured at the end of every epoch:
from torch.utils.data import TensorDataset, DataLoader
yukleyici = DataLoader(TensorDataset(X_train, y_train), batch_size=32, shuffle=True)
for epoch in range(5):
model.train()
for xb, yb in yukleyici:
opt.zero_grad()
kayip_fn(model(xb), yb).backward()
opt.step()
model.eval()
with torch.no_grad():
dogruluk = (model(X_test).argmax(1) == y_test).float().mean().item()
print(epoch, dogruluk)For such a clear pattern, almost perfect accuracy is expected within a few epochs.
On a new example
To try the trained model on a single image, a batch dimension is added and the probabilities are taken with softmax:
yeni = torch.zeros(1, 16, 16)
yeni[0, :, 7:9] = 1.0 # a vertical line
model.eval()
with torch.no_grad():
p = torch.softmax(model(yeni.unsqueeze(0)), dim=1)
print(p.argmax(1).item()) # 1: verticalForms never seen in training (like a diagonal line) show the model's limits: you look at which class it assigns and with what confidence.
Practice Train and measure
Train for 5 epochs with batch_size=32, shuffle=True. Print that the test accuracy is above 0.95. Then print the predictions for a hand-drawn vertical and a horizontal line image as a list (vertical first).
Starter code
import torch
from torch import nn
torch.manual_seed(0)
def veri(n):
X = torch.rand(n, 1, 16, 16) * 0.3
y = torch.randint(0, 2, (n,))
for k in range(n):
if y[k] == 0:
X[k, 0, 7:9, :] += 1.0
else:
X[k, 0, :, 7:9] += 1.0
return X, y
X_train, y_train = veri(256)
X_test, y_test = veri(64)
model = nn.Sequential(
nn.Conv2d(1, 8, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2),
nn.Conv2d(8, 16, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2),
nn.Flatten(), nn.Linear(16 * 4 * 4, 2))
kayip_fn = nn.CrossEntropyLoss()
opt = torch.optim.Adam(model.parameters(), lr=0.01)
from torch.utils.data import TensorDataset, DataLoader
# buraya yazSample solution
import torch
from torch import nn
torch.manual_seed(0)
def veri(n):
X = torch.rand(n, 1, 16, 16) * 0.3
y = torch.randint(0, 2, (n,))
for k in range(n):
if y[k] == 0:
X[k, 0, 7:9, :] += 1.0
else:
X[k, 0, :, 7:9] += 1.0
return X, y
X_train, y_train = veri(256)
X_test, y_test = veri(64)
model = nn.Sequential(
nn.Conv2d(1, 8, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2),
nn.Conv2d(8, 16, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2),
nn.Flatten(), nn.Linear(16 * 4 * 4, 2))
kayip_fn = nn.CrossEntropyLoss()
opt = torch.optim.Adam(model.parameters(), lr=0.01)
from torch.utils.data import TensorDataset, DataLoader
yukleyici = DataLoader(TensorDataset(X_train, y_train), batch_size=32, shuffle=True)
for epoch in range(5):
model.train()
for xb, yb in yukleyici:
opt.zero_grad()
kayip_fn(model(xb), yb).backward()
opt.step()
model.eval()
with torch.no_grad():
print((model(X_test).argmax(1) == y_test).float().mean().item() > 0.95)
dikey = torch.zeros(1, 1, 16, 16)
dikey[0, 0, :, 7:9] = 1.0
yatay = torch.zeros(1, 1, 16, 16)
yatay[0, 0, 7:9, :] = 1.0
print(model(torch.cat([dikey, yatay])).argmax(1).tolist())Expected output
True [1, 0]
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 If test accuracy is still around 0.5 after training, what should be checked first?
- That labels match the data and the loss is falling Correct answer
- The GPU driver
- The image colours
- Nothing
Explanation: 0.5 means random guessing for two classes; the model has learned nothing.
2 Add a batch dimension to a single image.
p = torch.softmax(model(yeni.____(0)), dim=1)p = torch.softmax(model(yeni.unsqueeze(0)), dim=1)Explanation: The model expects (N, C, H, W); a single image is (C, H, W).
3 Match the steps.
| Term | Meaning |
|---|---|
model.train() |
Training mode |
model.eval() |
Evaluation mode |
argmax(1) |
Class prediction |
softmax |
Probabilities |