Deep-Learning-with-PyTorch

LibTorch: PyTorch in C++
467
Our tensor is only 1D, so we need to reshape it. Conveniently, CImg uses the same
ordering as PyTorch (channel, rows, columns). If not, we would need to adapt the
reshaping and permute the axes as we did in chapter 4. As CImg uses a range of
0…255 and we made our model to use 0…1, we divide here and multiply later.
This could, of course, be absorbed into the model, but we wanted to reuse our
traced model.
A common pitfall to avoid: pre- and postprocessing
When switching from one library to another, it is easy to forget to check that the conversion
steps are compatible. They are non-obvious unless we look up the memory
layout and scaling convention of PyTorch and the image processing library we use. If
we forget, we will be disappointed by not getting the results we anticipate.
Here, the model would go wild because it gets extremely large inputs. However, in
the end, the output convention of our model is to give RGB values in the 0..1 range.
If we used this directly with CImg, the result would look all black.
Other frameworks have other conventions: for example OpenCV likes to store images
as BGR instead of RGB, requiring us to flip the channel dimension. We always want
to make sure the input we feed to the model in the deployment is the same as what
we fed into it in Python.
Loading the traced model is very straightforward using torch::jit::load. Next, we
have to deal with an abstraction PyTorch introduces to bridge between Python and
C++: we need to wrap our input in an IValue (or several IValues), the generic data type
for any value. A function in the JIT is passed a vector of IValues, so we declare that
and then push_back our input tensor. This will automatically wrap our tensor into an
IValue. We feed this vector of IValues to the forward and get a single one back. We
can then unpack the tensor in the resulting IValue with .toTensor.
Here we see a bit about IValues: they have a type (here, Tensor), but they could also
be holding int64_ts or doubles or a list of tensors. For example, if we had multiple
outputs, we would get an IValue holding a list of tensors, which ultimately stems from
the Python calling conventions. When we unpack a tensor from an IValue using
.toTensor, the IValue transfers ownership (becomes invalid). But let’s not worry about
it; we got a tensor back. Because sometimes the model may return non-contiguous data
(with gaps in the storage from chapter 3), but CImg reasonably requires us to provide
it with a contiguous block, we call contiguous. It is important that we assign this
contiguous tensor to a variable that is in scope until we are done working with the
underlying memory. Just like in Python, PyTorch will free memory if it sees that no
tensors are using it anymore.
So let’s compile this! On Debian or Ubuntu, you need to install cimg-dev,
libjpeg-dev, and libx11-dev to use CImg.