Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Equation 8.9 - LSTM—candidate hidden state
Then, let’s turn our attention to the cell state (c). What about computing the new
cell state (c') using a weighted sum of the old cell state and the candidate hidden
state (g)?
Equation 8.10 - LSTM—new cell state
"What about i and f? What are they?"
They are gates, of course: the input (i) gate and the forget (f) gate. Now, we’re only
missing the new hidden state (h'). If the cell state is unbounded, what about making
it bounded again?
Equation 8.11 - LSTM—new hidden state
Can you guess what that o is? It is yet another gate, the output (o) gate.
The cell state corresponds to the long-term memory, while the
hidden state corresponds to the short-term memory.
That’s it; we’ve (re)invented the long short-term memory cell on our own!
LSTM Cell
If we place the three expressions next to each another, we can more easily see the
differences between them:
Long Short-Term Memory (LSTM) | 641