Deep-Learning-with-PyTorch

472 CHAPTER 15 Deploying to production
as long as we do not want gradients. Just like in Python, we set the model into evaluation
mode calling model->eval(). This time around, we call model->forward with our input
tensor and get a tensor as a result—no JIT is involved, so we do not need IValue packing
and unpacking.
Phew. Writing this in C++ was a lot of work for the Python fans that we are. We are
glad that we only promised to do inference here, but of course LibTorch also offers
optimizers, data loaders, and much more. The main reason to use the API is, of
course, when you want to create models and neither the JIT nor Python is a good fit.
For your convenience, CMakeLists.txt contains also the instructions for building
cyclegan-cpp-api, so building is just like in the previous section.
We can run the program as
./cyclegan_cpp_api ../traced_zebra_model.pt ../../data/p1ch2/horse.jpg
But we knew what the model would be doing, didn’t we?
15.5 Going mobile
As the last variant of deploying a model, we will consider deployment to mobile
devices. When we want to bring our models to mobile, we are typically looking at
Android and/or iOS. Here, we’ll focus on Android.
The C++ parts of PyTorch—LibTorch—can be compiled for Android, and we
could access that from an app written in Java using the Android Java Native Interface
(JNI). But we really only need a handful of functions from PyTorch—loading a JITed
model, making inputs into tensors and IValues, running them through the model,
and getting results back. To save us the trouble of using the JNI, the PyTorch developers
wrapped these functions into a small library called PyTorch Mobile.
The stock way of developing apps in Android is to use the Android Studio IDE,
and we will be using it, too. But this means there are a few dozen files of administrativa—which
also happen to change from one Android version to the next. As such, we
focus on the bits that turn one of the Android Studio templates (Java App with Empty
Activity) into an app that takes a picture, runs it through our zebra-CycleGAN, and
displays the result. Sticking with the theme of the book, we will be efficient with the
Android bits (and they can be painful compared with writing PyTorch code) in the
example app.
To infuse life into the template, we need to do three things. First, we need to define
a UI. To keep things as simple as we can, we have two elements: a TextView named headline
that we can click to take and transform a picture; and an ImageView to show our
picture, which we call image_view. We will leave the picture-taking to the camera app
(which you would likely avoid doing in an app for a smoother user experience), because
dealing with the camera directly would blur our focus on deploying PyTorch models. 15
15 We are very proud of the topical metaphor.