Deep-Learning-with-PyTorch

218 CHAPTER 8 Using convolutions to generalize
almost any architecture thanks to the expressivity of PyTorch. The purpose of this
section is precisely to provide conceptual tools that will allow us to read the latest
research paper and start implementing it in PyTorch—or, since authors often release
PyTorch implementations of their papers, to read the implementations without choking
on our coffee.
8.5.1 Adding memory capacity: Width
Given our feed-forward architecture, there are a couple of dimensions we’d likely
want to explore before getting into further complications. The first dimension is the
width of the network: the number of neurons per layer, or channels per convolution.
We can make a model wider very easily in PyTorch. We just specify a larger number of
output channels in the first convolution and increase the subsequent layers accordingly,
taking care to change the forward function to reflect the fact that we’ll now
have a longer vector once we switch to fully connected layers:
# In[40]:
class NetWidth(nn.Module):
def __init__(self):
super().__init__()
self.conv1 = nn.Conv2d(3, 32, kernel_size=3, padding=1)
self.conv2 = nn.Conv2d(32, 16, kernel_size=3, padding=1)
self.fc1 = nn.Linear(16 * 8 * 8, 32)
self.fc2 = nn.Linear(32, 2)
def forward(self, x):
out = F.max_pool2d(torch.tanh(self.conv1(x)), 2)
out = F.max_pool2d(torch.tanh(self.conv2(out)), 2)
out = out.view(-1, 16 * 8 * 8)
out = torch.tanh(self.fc1(out))
out = self.fc2(out)
return out
If we want to avoid hardcoding numbers in the definition of the model, we can easily
pass a parameter to init and parameterize the width, taking care to also parameterize
the call to view in the forward function:
# In[42]:
class NetWidth(nn.Module):
def __init__(self, n_chans1=32):
super().__init__()
self.n_chans1 = n_chans1
self.conv1 = nn.Conv2d(3, n_chans1, kernel_size=3, padding=1)
self.conv2 = nn.Conv2d(n_chans1, n_chans1 // 2, kernel_size=3,
padding=1)
self.fc1 = nn.Linear(8 * 8 * n_chans1 // 2, 32)
self.fc2 = nn.Linear(32, 2)
def forward(self, x):
out = F.max_pool2d(torch.tanh(self.conv1(x)), 2)
out = F.max_pool2d(torch.tanh(self.conv2(out)), 2)