CodeBlanks Logo Icon CodeBlanks
Login Register

Summary - LSTM

In this group you learned LSTM, gates, and cell state.

Summary: LSTM

In this group:

  • An LSTM carries a hidden state h and a cell state c; the gates (f, i, o) and the candidate g control information flow.
  • The additive cell update c = f·c + i·g reduces the vanishing gradient.
  • A state (h0, c0) can be given and carried between pieces.
  • input_size: features per step; hidden_size: capacity; parameters 4·(HI + HH + 2H).
  • bidirectional and num_layers; GRU is a lighter alternative (nn.GRU, 3 components).

Practice Sequence classifier

Write an LSTM classifier: nn.LSTM(3, 32, batch_first=True) and an nn.Linear(32, 2) on the last hidden state (h_n[-1]). Print the output shape for an (8, 20, 3) input, then build the same structure with nn.GRU and print the parameter counts of the LSTM and GRU layers on one line.

Starter code

model.py
import torch
from torch import nn

x = torch.zeros(8, 20, 3)

# buraya yaz

Sample solution

model.py
import torch
from torch import nn

x = torch.zeros(8, 20, 3)

class DiziSiniflandirici(nn.Module):
    def __init__(self):
        super().__init__()
        self.lstm = nn.LSTM(3, 32, batch_first=True)
        self.bas = nn.Linear(32, 2)

    def forward(self, x):
        _, (h_n, _) = self.lstm(x)
        return self.bas(h_n[-1])

print(DiziSiniflandirici()(x).shape)
say = lambda m: sum(p.numel() for p in m.parameters())
print(say(nn.LSTM(3, 32)), say(nn.GRU(3, 32)))

Expected output

torch.Size([8, 2])
4736 3552

Questions and answers

Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.

1 Why isn't a bidirectional LSTM used in a live model forecasting the next day's sales?

  • The loss can't be computed
  • It is used
  • It sees future steps; at prediction time there is no future Correct answer
  • It is too slow

Explanation: The backward pass requires knowing the steps at the end.

2 Build a lighter gated RNN.

Question 
gru = nn.____(16, 32, batch_first=True)
Answer 
gru = nn.GRU(16, 32, batch_first=True)

Explanation: GRU has 3 components and no cell state.

3 Match the architectures.

Term Meaning
RNN 1 component
GRU 3 components
LSTM 4 components
bidirectional Two directions

All lessons in Advanced

1 Neural network basics

  1. What is a neural network?
  2. Neuron
  3. Layers
  4. Activation
  5. Forward propagation
  6. Summary - Neural network

2 PyTorch basics

  1. What is PyTorch?
  2. Tensor
  3. Creating tensors
  4. Tensor properties
  5. GPU support
  6. Summary - PyTorch

3 Tensor operations

  1. Tensor operations
  2. Shape
  3. Dtype
  4. Reshape
  5. Indexing
  6. Summary - Tensor operations

4 Autograd

  1. What is autograd?
  2. requires_grad
  3. backward()
  4. grad
  5. Computational graph
  6. Summary - Autograd

5 Project: Neural network

  1. Preparing tensors
  2. Model definition
  3. Loss computation
  4. Optimizer

6 nn.Module

  1. What is nn.Module?
  2. __init__
  3. forward
  4. nn.Linear
  5. parameters()
  6. Summary - nn.Module

7 Training loop

  1. Training loop
  2. Epoch
  3. Loss
  4. backward()
  5. optimizer.step()
  6. Summary - Training loop

8 CNN

  1. What is CNN?
  2. Conv2d
  3. MaxPool2d
  4. Feature map
  5. Flatten
  6. Summary - CNN

9 RNN

  1. What is RNN?
  2. Hidden state
  3. Sequence
  4. Vanishing gradient
  5. Embedding
  6. Summary - RNN

10 LSTM

  1. What is LSTM?
  2. Gates
  3. Cell state
  4. input_size
  5. hidden_size
  6. Summary - LSTM

11 Project: CNN

  1. Preparing data
  2. CNN model
  3. Loss and optimizer
  4. Training

12 Transfer learning

  1. What is transfer learning?
  2. Pre-trained model
  3. Fine-tuning
  4. Feature extraction
  5. Advantages
  6. Summary - Transfer learning

13 Optimization

  1. What is an optimizer?
  2. SGD
  3. Adam
  4. Learning rate
  5. zero_grad()
  6. Summary - Optimization

14 Advanced final

  1. Challenge 1
  2. Challenge 2
  3. Challenge 3
  4. Challenge 4