Learning Data Mining with Python

Classifying Objects in Images Using Deep Learning
GPU optimization
Neural networks can grow quite large in size. This has some implications for
memory use; however, efficient structures such as sparse matrices mean that we
don't generally run into problems fitting a neural network in memory.
The main issue when neural networks grow large is that they take a very long time
to compute. In addition, some datasets and neural networks will need to run many
epochs of training to get a good fit for the dataset. The neural network we will train
in this chapter takes more than 8 minutes per epoch on my reasonably powerful
computer, and we expect to run dozens, potentially hundreds, of epochs. Some
larger networks can take hours to train a single epoch. To get the best performance,
you may be considering thousands of training cycles.
The math obviously doesn't give a nice result here.
One positive is that neural networks are, at their core, full of floating point
operations. There are also a large number of operations that can be performed
in parallel, as neural network training is composed of mainly matrix operations.
These factors mean that computing on GPUs is an attractive option to speed up
this training.
When to use GPUs for computation
GPUs were originally designed to render graphics for display. These graphics
are represented using matrices and mathematical equations on those matrices,
which are then converted into the pixels that we see on our screen. This process
involves lots of computation in parallel. While modern CPUs may have a number
of cores (your computer may have 2, 4, or even 16—or more!), GPUs have
thousands of small cores designed specifically for graphics.
A CPU is therefore better for sequential tasks, as the cores tend to be individually
faster and tasks such as accessing the computer's memory are more efficient. It is
also, honestly, easier to just let the CPU do the heavy lifting. Almost every machine
learning library defaults to using the CPU, and there is extra work involved before
you can use the GPU for computing. The benefits though, can be quite significant.
GPUs are therefore better suited for tasks in which there are lots of small
operations on numbers that can be performed at the same time. Many machine
learning tasks are like this, lending themselves to efficiency improvements through
the use of a GPU.
[ 258 ]