Ivancevic_Applied-Diff-Geom

1022 Applied Differential Geometry: A Modern Introductionproblem is to find the control path u i (·) that minimizes〈∫ tfC(x i , t, u i (·)) = φ(x i (t f )) + dτ( 1 〉t 2 u i(τ)Ru i (τ) + V (x i (τ), τ)) ,x i(6.24)with R a matrix, V (x i , t) a time–dependent potential, and φ(x i ) the endcost. The brackets 〈〉 x i denote expectation value with respect to the stochastictrajectories (6.23) that start at x i .One defines the optimal cost–to–go function from any time t and statex i asJ(x i , t) = minu i (·) C(xi , t, u i (·)).J satisfies the following stochastic HJB equation [Kappen (2006)]( 1− ∂ t J(x i , t) = minu i 2 u iRu i +V +(b i +u i )∂ x iJ(x i , t)+ 1 )2 ν ij∂ xi x j J(xi , t)= − 1 2 R−1 ∂ x iJ(x i , t)∂ x iJ+V +b i ∂ x iJ(x i , t)+ 1 2 ν ij∂ x i x j J(xi , t),where b i = (b i ) T , and u i = (u i ) T , and(6.25)u i = −R −1 ∂ x iJ(x i , t) (6.26)is the optimal control at the point (x i , t). The HJB equation is nonlinearin J and must be solved with end boundary condition J(x i , t f ) = φ(x i ).Let us define ψ(x i , t) through the Log Transformand assume that there exists a scalar λ such thatJ(x i , t) = −λ log ψ(x i , t), (6.27)λδ ij = (Rν) ij , (6.28)with δ ij the Kronecker delta. In the one dimensional case, such a λ canalways be found. In the higher dimensional case, this restricts the matricesR ∝ (ν ij ) −1 . Equation (6.28) reduces the dependence of optimalcontrol on the nD noise matrix to a scalar value λ that will play the role oftemperature, while (6.25) reduces to the linear equation (6.22) withH = − V λ + b i∂ x i + 1 2 ν ij∂ xi x j J(xi , t).