Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Output
PackedSequence(data=tensor([[ 1.1264, 1.1571],
[ 0.9225, -0.8989],
[ 0.8738, -1.0075],
[ 1.0942, 0.9230],
[-0.9150, -1.0915],
[-1.0867, 1.0773]]), batch_sizes=tensor([2, 2, 1, 1]),
sorted_indices=tensor([0, 1]), unsorted_indices=tensor([0, 1]))
Two sequences of different sizes go in, one packed sequence comes out. Now we
can create a data loader that uses our collate function:
Data Preparation
1 train_var_loader = DataLoader(train_var_data,
2 batch_size=16,
3 shuffle=True,
4 collate_fn=pack_collate)
5 x_batch, y_batch = next(iter(train_var_loader))
And now every batch coming out of our data loader has a packed sequence.
"Do I have to change the model too?"
Square Model IV — Packed
There are some changes we need to make to the model. Let’s illustrate them by
creating a model that uses a bidirectional LSTM and expects packed sequences as
inputs.
First, since X is a packed sequence now, it means that the output is packed, and
therefore we need to unpack it to a padded output.
Once it is unpacked, we can get the last output by using the fancier indexing (from
a couple of pages ago) to get the last (actual) element of the padded sequences.
Moreover, using a bidirectional LSTM means that the output for each sequence has
an (N, 1, 2*H) shape.
Variable-Length Sequences | 667