Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

So, let’s go straight to creating a model using a long short-term memory.
Square Model III — The Sorcerer
This model is pretty much the same as the original "Square Model," except for two
differences: Its recurrent neural network is not a plain RNN anymore, but an LSTM,
and it produces two states as output instead of one. Everything else stays exactly
the same.
Model Configuration
1 class SquareModelLSTM(nn.Module):
2 def __init__(self, n_features, hidden_dim, n_outputs):
3 super(SquareModelLSTM, self).__init__()
4 self.hidden_dim = hidden_dim
5 self.n_features = n_features
6 self.n_outputs = n_outputs
7 self.hidden = None
8 self.cell = None 2
9 # Simple LSTM
10 self.basic_rnn = nn.LSTM(self.n_features,
11 self.hidden_dim,
12 batch_first=True) 1
13 # Classifier to produce as many logits as outputs
14 self.classifier = nn.Linear(self.hidden_dim,
15 self.n_outputs)
16
17 def forward(self, X):
18 # X is batch first (N, L, F)
19 # output is (N, L, H)
20 # final hidden state is (1, N, H)
21 # final cell state is (1, N, H)
22 batch_first_output, (self.hidden, self.cell) = \
23 self.basic_rnn(X) 2
24
25 # only last item in sequence (N, 1, H)
26 last_output = batch_first_output[:, -1]
27 # classifier will output (N, 1, n_outputs)
28 out = self.classifier(last_output)
29 # final output is (N, n_outputs)
30 return out.view(-1, self.n_outputs)
650 | Chapter 8: Sequences