Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Model Configuration & Training
Model Configuration
1 torch.manual_seed(42)
2 # Layers
3 enclayer = EncoderLayer(n_heads=3, d_model=6,
4 ff_units=10, dropout=0.1)
5 declayer = DecoderLayer(n_heads=3, d_model=6,
6 ff_units=10, dropout=0.1)
7 # Encoder and Decoder
8 enctransf = EncoderTransf(enclayer, n_layers=2)
9 dectransf = DecoderTransf(declayer, n_layers=2)
10 # Transformer
11 model_transf = EncoderDecoderTransf(enctransf,
12 dectransf,
13 input_len=2,
14 target_len=2,
15 n_features=2)
16 loss = nn.MSELoss()
17 optimizer = torch.optim.Adam(model_transf.parameters(), lr=0.01)
Weight Initialization
1 for p in model_transf.parameters():
2 if p.dim() > 1:
3 nn.init.xavier_uniform_(p)
Model Training
1 sbs_seq_transf = StepByStep(model_transf, loss, optimizer)
2 sbs_seq_transf.set_loaders(train_loader, test_loader)
3 sbs_seq_transf.train(50)
sbs_seq_transf.losses[-1], sbs_seq_transf.val_losses[-1]
Output
(0.019648547226097435, 0.011462601833045483)
Putting It All Together | 875