Learning Data Mining with Python

Classifying Objects in Images Using Deep Learning
Next, we have our output layer that takes its input from the hidden layer, which
has three nodes (which is the same as the number of classes), and uses the softmax
nonlinearity. Softmax is more typically used in the final layer of neural networks:
output_layer = lasagne.layers.DenseLayer(hidden_layer, num_units=3,
nonlinearity=lasagne.
nonlinearities.softmax)
In Lasagne's usage, this output layer is our network. When we enter a sample
into it, it looks at this output layer and obtains the layer that is inputted into it
(the first argument). This continues recursively until we reach an input layer, which
applies the samples to itself, as it doesn't have an input layer to it. The activations
of the neurons in the input layer are then fed into its calling layer (in our case, the
hidden_layer), and that is then propagated up all the way to the output layer.
In order to train our network, we now need to define some training functions,
which are Theano-based functions. In order to do this, we need to define a Theano
expression and a function for the training. We start by creating variables for the
input samples, the output given by the network, and the actual output:
import theano.tensor as T
net_input = T.matrix('net_input')
net_output = output_layer.get_output(net_input)
true_output = T.ivector('true_output')
We can now define our loss function, which tells the training function how to improve
the network—it attempts to train the network to minimize the loss according to this
function. The loss we will use is the categorical cross entropy, a metric on categorical
data such as ours. This is a function of the output given by the network and the actual
output we expected:
loss = T.mean(T.nnet.categorical_crossentropy(net_output,
true_output))
Next, we define the function that will change the weights in our network.
In order to do this, we obtain all of the parameters from the network and create
a function (using a helper function given by Lasagne) that adjusts the weights to
minimize our loss;
all_params = lasagne.layers.get_all_params(output_layer)
updates = lasagne.updates.sgd(loss, all_params, learning_rate=0.1)
[ 252 ]