Dissertation - HQ

168 Oceanography vs. behaviour
6.A Choice of the last two optimal decisions
In the simple model presented in section 6.2 (page 119 and following)
it is possible to describe analytically the choice of the last two decisions.
It helps in understanding how the optimisation algorithm works.
Last optimal decision d # (θ, x, T − 1)
The value function is
0
V (θ + ∆θ 0 , x + ∆x 0 1
, T ),
| {z }
Foraging, d = 0
V (θ, x, T − 1) = p · max
d
B
@ V (θ − ∆θ 1 , x − ∆x 1 C
, T ) A
| {z }
Swimming, d = 1
= p · max (θ + !
∆θ0 ) · 1 {x+∆x 0 =0},
(θ − ∆θ 1 ) · 1 {x−∆x 1 =0}
Now, x + ∆x 0 ≠ 0 because x ≥ 0 and ∆x 0 > 0. Thus
(
V (θ, x, T − 1) = p · (θ − ∆θ 1 ) · 1 {x−∆x 1 =0}
d # (θ, x, T − 1) = 1
Therefore, the optimal decision at time T −1 is swimming if x−∆x 1 = 0.
It means that the larva will swim if it can reach the reef by choosing
swimming. But, if x is not equal to ∆x 1 (it cannot reach the island), V
equals zero for any decision. In this case it does not have any favourite
decision for its last choice.
Before the last optimal decision u # (θ, x, T − 2) Assuming the last
decision was swimming (d # (θ, x, T − 1) = 1), the value function at
T − 2 is
0
V (θ + ∆θ 0 , x + ∆x 0 1
, T − 1) ,
| {z }
Foraging, d = 0
V (θ, x, T − 2) = p · max
d
B
@ V (θ − ∆θ 1 , x − ∆x 1 C
, T − 1) A
| {z }
Swimming, d = 1
= p · max (θ + !
∆θ0 − ∆θ 1 ) · 1 {x+∆x 0 =∆x 1 },
(θ − 2∆θ 1 ) · 1 {x−∆x 1 =∆x 1 }
As we cannot have at the same time x+∆x 0 = ∆x 1 and x−∆x 1 = ∆x 1 ,
it comes that:
“
V (θ, x, T − 2) = p · (θ + ∆θ 0 − ∆θ 1 ) · 1 {x+∆x 0 =∆x 1 }
+ (θ − 2∆θ 1 ) · 1 {x−∆x 1 =∆x 1 }
”