Challenge 1
- Machine Learning course
- Advanced
- Lesson 75/78
- 8 min
Perform tensor operations with PyTorch.
Challenge 1: a non-linear boundary
You'll write a neural network to separate two interleaving half-moon classes. A linear model reaches only ~85% accuracy on this data; a network with hidden layers can bend the boundary:
dogrusal = nn.Linear(2, 2)
mlp = nn.Sequential(nn.Linear(2, 32), nn.ReLU(), nn.Linear(32, 32), nn.ReLU(), nn.Linear(32, 2))A few hundred Adam steps with the full data (no batches) are enough; since the data is small, every step can use all the examples.
Practice Linear and MLP
Write a train function: train the model for 500 steps on all the data with Adam(lr=0.01) and CrossEntropyLoss and return the training accuracy. Print on one line that the linear model's accuracy is below 0.9 and the MLP's is above 0.97.
Starter code
import math
import torch
from torch import nn
torch.manual_seed(0)
def ay(n):
t = torch.rand(n) * math.pi
ust = torch.stack([torch.cos(t), torch.sin(t)], 1)
alt = torch.stack([1 - torch.cos(t), 0.5 - torch.sin(t)], 1)
X = torch.cat([ust, alt]) + 0.1 * torch.randn(2 * n, 2)
y = torch.cat([torch.zeros(n), torch.ones(n)]).long()
return X, y
X, y = ay(200)
# buraya yazSample solution
import math
import torch
from torch import nn
torch.manual_seed(0)
def ay(n):
t = torch.rand(n) * math.pi
ust = torch.stack([torch.cos(t), torch.sin(t)], 1)
alt = torch.stack([1 - torch.cos(t), 0.5 - torch.sin(t)], 1)
X = torch.cat([ust, alt]) + 0.1 * torch.randn(2 * n, 2)
y = torch.cat([torch.zeros(n), torch.ones(n)]).long()
return X, y
X, y = ay(200)
def train(model):
opt = torch.optim.Adam(model.parameters(), lr=0.01)
kayip_fn = nn.CrossEntropyLoss()
for _ in range(500):
opt.zero_grad()
kayip_fn(model(X), y).backward()
opt.step()
with torch.no_grad():
return (model(X).argmax(1) == y).float().mean().item()
dogrusal = train(nn.Linear(2, 2))
mlp = train(nn.Sequential(nn.Linear(2, 32), nn.ReLU(), nn.Linear(32, 32), nn.ReLU(), nn.Linear(32, 2)))
print(dogrusal < 0.9, mlp > 0.97)Expected output
True True
Practice Early stopping with validation
Shuffle the data and split it into 300 training and 100 validation examples. Train the MLP for 1000 steps; every 50 steps measure the validation loss and keep the state_dict at the best loss with copy.deepcopy. Load the best weights back and print that the validation accuracy is above 0.95.
Starter code
import math
import torch
from torch import nn
torch.manual_seed(0)
def ay(n):
t = torch.rand(n) * math.pi
ust = torch.stack([torch.cos(t), torch.sin(t)], 1)
alt = torch.stack([1 - torch.cos(t), 0.5 - torch.sin(t)], 1)
X = torch.cat([ust, alt]) + 0.1 * torch.randn(2 * n, 2)
y = torch.cat([torch.zeros(n), torch.ones(n)]).long()
return X, y
X, y = ay(200)
import copy
# buraya yazSample solution
import math
import torch
from torch import nn
torch.manual_seed(0)
def ay(n):
t = torch.rand(n) * math.pi
ust = torch.stack([torch.cos(t), torch.sin(t)], 1)
alt = torch.stack([1 - torch.cos(t), 0.5 - torch.sin(t)], 1)
X = torch.cat([ust, alt]) + 0.1 * torch.randn(2 * n, 2)
y = torch.cat([torch.zeros(n), torch.ones(n)]).long()
return X, y
X, y = ay(200)
import copy
sira = torch.randperm(len(X))
egit, dog = sira[:300], sira[300:]
model = nn.Sequential(nn.Linear(2, 32), nn.ReLU(), nn.Linear(32, 32), nn.ReLU(), nn.Linear(32, 2))
opt = torch.optim.Adam(model.parameters(), lr=0.01)
kayip_fn = nn.CrossEntropyLoss()
en_iyi, en_iyi_durum = float("inf"), None
for adim in range(1000):
opt.zero_grad()
kayip_fn(model(X[egit]), y[egit]).backward()
opt.step()
if adim % 50 == 0:
with torch.no_grad():
k = kayip_fn(model(X[dog]), y[dog]).item()
if k < en_iyi:
en_iyi, en_iyi_durum = k, copy.deepcopy(model.state_dict())
model.load_state_dict(en_iyi_durum)
with torch.no_grad():
print((model(X[dog]).argmax(1) == y[dog]).float().mean().item() > 0.95)Expected output
True
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 Why does a linear model fall short on this data?
- There is too little data
- There are too many features
- It doesn't
- The classes can't be separated by a straight line Correct answer
Explanation: The moon shapes reach into each other; the boundary must be curved.
2 Add the non-linearity in between.
mlp = nn.Sequential(nn.Linear(2, 32), nn.____(), nn.Linear(32, 2))mlp = nn.Sequential(nn.Linear(2, 32), nn.ReLU(), nn.Linear(32, 2))Explanation: Layers without activation collapse into one linear transformation.
3 Match the models.
| Term | Meaning |
|---|---|
nn.Linear(2, 2) |
Straight boundary |
MLP |
Curved boundary |
ReLU |
Adds the bend |
Adam |
Optimiser |