Deep-Learning-with-PyTorch

Named tensors
49
For operations with two inputs, in addition to the usual dimension checks—whether
sizes are the same, or if one is 1 and can be broadcast to the other—PyTorch will now
check the names for us. So far, it does not automatically align dimensions, so we need
to do this explicitly. The method align_as returns a tensor with missing dimensions
added and existing ones permuted to the right order:
# In[9]:
weights_aligned = weights_named.align_as(img_named)
weights_aligned.shape, weights_aligned.names
# Out[9]:
(torch.Size([3, 1, 1]), ('channels', 'rows', 'columns'))
Functions accepting dimension arguments, like sum, also take named dimensions:
# In[10]:
gray_named = (img_named * weights_aligned).sum('channels')
gray_named.shape, gray_named.names
# Out[10]:
(torch.Size([5, 5]), ('rows', 'columns'))
If we try to combine dimensions with different names, we get an error:
gray_named = (img_named[..., :3] * weights_named).sum('channels')
RuntimeError: Error when
attempting to broadcast dims ['channels', 'rows',
'columns'] and dims ['channels']: dim 'columns' and dim 'channels'
are at the same position from the right but do not match.
If we want to use tensors outside functions that operate on named tensors, we need to
drop the names by renaming them to None. The following gets us back into the world
of unnamed dimensions:
# In[12]:
gray_plain = gray_named.rename(None)
gray_plain.shape, gray_plain.names
# Out[12]:
(torch.Size([5, 5]), (None, None))
Given the experimental nature of this feature at the time of writing, and to avoid
mucking around with indexing and alignment, we will stick to unnamed in the
remainder of the book. Named tensors have the potential to eliminate many sources
of alignment errors, which—if the PyTorch forum is any indication—can be a source
of headaches. It will be interesting to see how widely they will be adopted.