Summary - LSTM

In this group you learned LSTM, gates, and cell state.

Summary: LSTM

In this group:

An LSTM carries a hidden state h and a cell state c; the gates (f, i, o) and the candidate g control information flow.
The additive cell update c = f·c + i·g reduces the vanishing gradient.
A state (h0, c0) can be given and carried between pieces.
input_size: features per step; hidden_size: capacity; parameters 4·(HI + HH + 2H).
bidirectional and num_layers; GRU is a lighter alternative (nn.GRU, 3 components).

Exercise Sequence classifier

Write an LSTM classifier: nn.LSTM(3, 32, batch_first=True) and an nn.Linear(32, 2) on the last hidden state (h_n[-1]). Print the output shape for an (8, 20, 3) input, then build the same structure with nn.GRU and print the parameter counts of the LSTM and GRU layers on one line.

Show example solution

import torch
from torch import nn

x = torch.zeros(8, 20, 3)

class DiziSiniflandirici(nn.Module):
    def __init__(self):
        super().__init__()
        self.lstm = nn.LSTM(3, 32, batch_first=True)
        self.bas = nn.Linear(32, 2)

    def forward(self, x):
        _, (h_n, _) = self.lstm(x)
        return self.bas(h_n[-1])

print(DiziSiniflandirici()(x).shape)
say = lambda m: sum(p.numel() for p in m.parameters())
print(say(nn.LSTM(3, 32)), say(nn.GRU(3, 32)))

Test yourself

Why isn't a bidirectional LSTM used in a live model forecasting the next day's sales?
- The loss can't be computed
- It is used
- It sees future steps; at prediction time there is no future
- It is too slow
Show answer

It sees future steps; at prediction time there is no future — The backward pass requires knowing the steps at the end.

Build a lighter gated RNN.

gru = nn.____(16, 32, batch_first=True)

Show answer

gru = nn.GRU(16, 32, batch_first=True)

Match the architectures.

RNN

1 component
GRU

3 components
LSTM

4 components
bidirectional

Two directions

Next lesson: Preparing data

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies