Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Data Generation & Preparation
1 X_reg, y_reg = make_regression(
2 n_samples=1000, n_features=10, noise=0.1, random_state=42
3 )
4 X_reg = torch.as_tensor(X_reg).float()
5 y_reg = torch.as_tensor(y_reg).float().view(-1, 1)
6
7 dataset = TensorDataset(X_reg, y_reg)
8 train_loader = DataLoader(
9 dataset=dataset, batch_size=32, shuffle=True
10 )
Even though we cannot plot a ten-dimensional regression, we can still visualize the
distribution of both features and target values.
Figure E.4 - Distributions of feature and target values
It’s all good and fine with our feature values since they are inside a typical
standardized range (-3, 3). The target values, though, are on a very different scale,
from -400 to 400. If the target variable represents a monetary value, for example,
these ranges are fairly common. Sure, we could standardize the target value as well,
but that would ruin the example of exploding gradients!
Model Configuration & Training
We can build a fairly simple model to tackle this regression problem: a network
with one hidden layer with 15 units, a ReLU as activation function, and an output
layer.
572 | Extra Chapter: Vanishing and Exploding Gradients