Dissertation - HQ

A general modelling framework for larval behaviour 125
Numerical solution
Given the description of the evolution of the state through transition
matrices (Figure 6.2), solving the backward computation of gain and
decisions (equation 6.4) is a matter of manipulating matrices. The
final gain is a vector indexed by state. In the simple case presented in
Figure 6.2, it is a column vector of length 12. Now, computing the value
function (i.e. the optimal gain) at time T − 1 is a three step process:
Multiply transition
matrices and gain,
backward in time
1. Fill the matrices describing the transition probabilities from all
states at t = T −1 to all other states at t = T , for the two decisions.
These probabilities come from the description of the dynamical
system (advection by currents, energy consumption, etc.) and this
process has been detailed for three examples on page 121.
2. Multiply each matrix by the final gain vector. For each initial state,
this means multiplying the gain associated with every reachable
final state by the probability to reach it and then summing all
those products. These sums are therefore the mean gain at T − 1,
for each state and each decision.
3. For each state, compare the gain values in the two mean gain
vectors (one for each decision), choose the maximum, and record
to which decision it corresponds. This gives optimal mean gain
and optimal decisions.
To compute the value function at t = T − 2, repeat these steps with the
optimal mean gain at t = T − 1 instead of the final gain. And similarly
until t = 0.
6.2.3 Example trajectories
Given the description of the environment and the characteristics of
the larva, optimal strategies (giving sequences of optimal decisions)
and optimal trajectories (state trajectories for which the sequence of
decisions is optimal) can be computed. There is no finite number of
optimal trajectories. Indeed, in this version of the model, stochasticity
is introduced by random survival.
Two characteristic examples of optimal trajectories are presented in
Figure 6.3. As remarked for the last two decisions, the behaviour of
larvae is very intuitive. When it survives (left plot), the larva forages
and lets itself be taken away by currents until it reaches its maximum
energy resources. Then, it alternates swimming and foraging in order
to keep energy close to its maximum at final time. In the right plot, the
behaviour begins the same way but the larva dies at time step 25. We
can conclude from the results of this simple model that the algorithm
used to simulate larval behaviour and to solve the optimisation problem
is correct.
Predictable and sensible
decisions: the problem
is solved correctly