Controlled Markov Diffusions and - University of Leeds

Controlled Markov Diffusions and”Mean-Variance” OptimisationGeorgios AivaliotisSubmitted in accordance with the requirements for the degree ofDoctor of PhilosophyThe University of LeedsDepartment of StatisticsJune 2009The candidate confirms that the work submitted is his own, exceptwhere work which has formed part of jointly-authored publicationshas been included.The contribution of the candidate and the other

iiauthors to this work has been explicitly indicated overleaf. Thecandidate confirms that appropriate credit has been given within thethesis where reference has been made to the work of others. Thiscopy has been supplied on the understanding that it is copyrightmaterial and that no quotation from the thesis may be publishedwithout proper acknowledgement.

iiiDetails of jointly-authored publications that have been used to form part of the thesisChapters 2, 3 and 4, include parts from a jointly-authored preprint accepted forpublication in the journal ”Stochastics: An International Journal of Probabilityand Stochastic Processes”.Title:diffusion.On Bellman’s equations for mean and variance control for a MarkovAuthors: Mr. Georgios Aivaliotis and Prof. Alexander. Yu. Veretennikov.In particular, the candidate confirms that sections 2.3 and 2.3.1 from chapter2, section 3.2 from chapter 3 and 4.2 from chapter 4 have been included in theabove mentioned preprint. The preprint has been accepted for publication andis going to appear soon in the journal ”Stochastics: An International Journal ofProbability and Stochastic Processes”.The candidate confirms that Theorem 2 and Corollary 1 of the preprint areattributed to the co-author, while the rest is the contribution of the candidate.

To my parents Leonidas and Athena.iv

vAcknowledgementsThe author would like to thank all these people, who made possible the completionof this thesis. First, I would like to express my deep gratitude to my primarysupervisor, Prof. Alexander Yu. Veretennikov for all his patience, help andguidance during my studies in the University of Leeds. It has been an honourand a pleasure to study under his supervision.I am also indebted to my co-supervisor, Prof. John T. Kent for his criticalcomments and constructive suggestions that have always resulted in theimprovement of the material in this thesis. Furthermore, I would like to thankDr. Leonid V. Bogachev for some very useful questions that gave us some newperspectives of this work and my colleagues Niloufar Abourashchi and RiemmanAbu-Shanab, for the discussions we had during my studies and their usefulcomments on my work.I would also like to thank my internal and external examiners for pointingout errors in the thesis and making very useful suggestions which led to theimprovement of the material.Finally, I would like to thank EPSRC for funding and last but not least a bigthanks to my family and to Ariadni for their support, financial and emotional, andfor being always there for me. Apologies to any person whose name has not beenacknowledged, either by mistake or by ignorance.

viAbstractMean-Variance control problems have long been a topic of interest in financialmathematics. However, even for control of the second moment of a Markovianfunctional, classical Hamilton-Jacobi-Bellman theory was not available (at leastfor the general case). For that reason, alternative methods have been developedmainly based on the optimisation of a utility function. In this work, we try toreestablish the simplicity of HJB theory on this nontrivial problem.For diffusions without control, we examine the second moment of a cost functionfor the finite horizon problem and the exit time problem. A system of two wellposedlinear PDEs is derived, similar to Kac’s and Dynkin’s moment equations,along with an equivalent single degenerate equation which turns out to be wellposed,too. An extension to higher moments is also proposed.Finally a controlled diffusion process is considered with cost functions of “meanand variance” type. A regularization is proposed for computing the value of thecost function via Bellman’s equations. In particular, the latter are useful becausethey imply sufficiency of markovian strategies which prove to be almost optimalfor the degenerate versions of the value functions as well.

viiList of NotationsE d is the d-dimensional Euclidian space, x = (x 1 , . . . , x d ) is an arbitrary pointin it.E d+1 is the (d + 1)-dimensional Euclidian space, (t, x) is an arbitrary point in it,where t ∈ [0, ∞).E d+ is the half space in E d , ie E d := {x ∈ E d |x i > 0, ∀i = 1, . . . , d}.D is an open set in E d .∂D is the boundary of D.¯D is the closure of D.Q is an open set in E d+1 .∂Q is the boundary of Q.¯Q is the closure of Q.C is the set of all continuous functions.C n (E d ) is the set of continuous functions, n-times continuously differentiable inE d .C n 1,n 2(E d+1 ) is the set of continuous functions, n 1 -times continuouslydifferentiable with respect to t ∈ [0, ∞) and n 2 -times continuouslydifferentiable with respect to x ∈ E d .C n,l (E d ) is the set of continuous functions, n-times continuously differentiablein E d with Lipschitz derivatives.

viiiC b is the set of all continuous bounded functions.C n b (E d) is the set of continuous bounded functions, n-times continuouslydifferentiable in E d with bounded derivatives.C n 1,n 2b(E d+1 ) is the set of continuous bounded functions, n 1 -times continuouslydifferentiable with respect to t ∈ [0, ∞) and n 2 -times continuouslydifferentiable with respect to x ∈ E, where all the derivatives are alsobounded.Cβ 1(E d), C 1,2β(E d+1) same as above, but the derivatives are Hölder continuouswith β ∈ (0, 1).φ x1 ,x 2 ,...,x d=∂ n φ∂x 1 ∂x 2 ...∂x nfor φ ∈ C n (E d ).L p , (p ≥ 1) (Lebesgue spaces) are the Banach spaces of Borel measurablefunctions summable with order p.‖f‖ p,D = (∫ D |f(x)|p dx ) 1 p.For a local L p space L D p,loc = {f(t, x) : 0 ≤ t ≤ T, x ∈∫ T ∫E d , sup |f(t, z∈Ed 0 |x−z|≤1∩D x)|p dt dx < ∞}.W n (E d ) are the Sobolev Spaces of functions with generalised derivatives withrespect to x ∈ E d of order n.W n 1,n 2(E d+1 ) are the Sobolev Spaces of functions with generalised derivativeswith respect to t ∈ [0, ∞) of order n 1 and generalised derivatives withrespect to x ∈ E d of order n 2 .‖V ‖ W 2D= ∑ di,j=1 ‖V x i x j‖ p,D + ∑ di=1 ‖V x i‖ p,D + ‖V ‖ p,D .

ix‖V ‖ W1,2Q= ‖V t ‖ p,Q + ∑ di,j=1 ‖V x i x j‖ p,Q + ∑ di=1 ‖V x i‖ p,Q + ‖V ‖ p,Q .W 1,2 = ⋂ p>1 W 1,2p,loc .W 2 = ⋂ p>1 W 2 p .‖x‖ = ( ∑ di=1 x2 i) 1/2for x ∈ Ed .‖M‖ = ( tr(MM ∗ ) ) 1/2, for a d × d1 matrix M.‖f‖ B = sup (t,x)∈Ed+1 f(t, x).Summation agreement: pairs of equal indices imply summation from 1 to d. Inparticular∂a∂U xkU xk ,x i= ∑ d ∂ak=1 ∂U xkU xk ,x i.

xTable of ContentsDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . .Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ivvviList of Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiTable of Contentsx1 Introduction 11.1 Fundamental Notation and Definitions . . . . . . . . . . . . . . . 41.1.1 General Notation . . . . . . . . . . . . . . . . . . . . . . 41.1.2 Notation on Spaces, Definitions and Basic Properties . . . 111.2 The General Problem Setting . . . . . . . . . . . . . . . . . . . . 141.3 History and Literature Review . . . . . . . . . . . . . . . . . . . 182 Higher Moments of a non-controlled Markovian functional 252.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252.2 A preliminary analysis for the finite horizon problem (ParabolicCase) with the use of the artificial variable y . . . . . . . . . . . . 28

CONTENTSxi2.3 A system of equations for the finite horizon non-controlled secondmoment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412.3.1 A single linear PDE for diffusion without control . . . . . 472.4 A system of equations for the non-controlled second moment inthe exit time problem . . . . . . . . . . . . . . . . . . . . . . . . 482.4.1 A single linear elliptic PDE for the exit time problemwithout control . . . . . . . . . . . . . . . . . . . . . . . 542.5 The d-dimensional exit time problem with ”final payment” . . . . 562.6 The exit time problem in dimension one . . . . . . . . . . . . . . 582.7 Higher moments . . . . . . . . . . . . . . . . . . . . . . . . . . . 643 Controlling the second moment 663.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 663.1.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . 693.2 Control of the first and second moment of a Markov functionalwithout final payment . . . . . . . . . . . . . . . . . . . . . . . . 713.2.1 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . 783.3 Control of the first and second moment of a Markov functionalwith final payment . . . . . . . . . . . . . . . . . . . . . . . . . 794 Mean-Variance Optimization 904.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 904.2 Without final payment . . . . . . . . . . . . . . . . . . . . . . . 924.3 With final payment . . . . . . . . . . . . . . . . . . . . . . . . . 1044.4 The ”first-second moment” variation of the problem . . . . . . . . 109

CONTENTSxii5 Conclusions and Further Research 1135.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1135.2 Further Research . . . . . . . . . . . . . . . . . . . . . . . . . . 115

1Chapter 1IntroductionThe main topic of this thesis is stochastic control for mean and variance ofmarkovian cost (or payoff) functionals. The results are going to develop graduallywith several variations on the theme, combining existing literature from differentareas. The first part is mainly concerned with the second moment (and extensionto higher moments) of a non-controlled markovian cost functional. We will provea ”Dynamic Principle” type argument to end up with one degenerate parabolicPDE and make use of a chain of PDE’s to describe the second moment. Then therest of the work will focus on controlled problems and specifically mean-variancecontrol.The motivation for this work emerges mainly from some problems from finance

Chapter 1. Introduction 2and especially from the famous ”mean-variance” optimisation theory. Initially,what in this work is called second moment of a cost functional, might describe thequadratic payoff of a financial option. Financial institutions nowadays are keen inintroducing new products for their clients, given that there is a reliable method tocalculate a fair price for this product. Furthermore, the moments of any functionof a random variable can give information about its distribution and especially thefirst two moments can be used to calculate the mean and the variance. Therefore,regarding another option (for example with some integral payoff), it would be ofinterest to be able to calculate its variance, thus quantifying the risk this derivativecarries. Consequently, it is obvious that the calculation of the variance of a costfunctional could be very useful.Moreover, application of control techniques could much more extend the scopeof the above issues.Controlling the first two moments of a payoff function(or a cost function) is essential in many financial applications.We refer toMarkowitz’s mean-variance optimisation criterion as a benchmark in optimalportfolio selection.However, introducing control to the analysis, introducesautomatically some deviations from the smoothness of the functions to whichcontrol is applied. The controls used in this thesis will affect the functions appliedto (drift and diffusion coefficients, cost function f) in a uniformly continuous

Chapter 1. Introduction 3way, but these functions might not be differentiable any more. In the sections thatfollow, the results are formulated under quite relaxed assumptions regarding theintegrability of the functions involved, although further relaxation is possible.In the next section, the basic notation and definitions are given. The relevant tothis analysis spaces will be defined and a few useful properties will be described.In section 1.3 we set up the framework of the problem examined and discusssome previous results that will be referred in the sequel. A preliminary analysis(intuitive but not rigorous) is worked out in section 2.2. It is the finite horizonproblem, presented in a way that gives intuition on the derivation of the PDE’sthat describe the problem. The main theorems for the problem of the secondmoment without control are given in sections 2.3 and 2.4 for fixed time and exittime interval respectively, while the one-dimensional case and its explicit solutionis discussed in section 2.6. Finally, an extension to higher moments is proposedin section 2.7.Chapter 3 mainly refers to the calculation of the second moment in the controlledcase, for fixed time interval, where regularisation is used in order to avoiddegeneracy problems.Finally, the ”mean-variance” control problem will beexamined in chapter 4 for functionals without and with a ”final payment”.

Chapter 1. Introduction 41.1 Fundamental Notation and DefinitionsIn this section the basic notation used throughout this work is introduced. Thiswill remain the same for all chapters unless otherwise stated. Notation includesspaces, domains, space and time variables etc. Importance will be given to thedefinition and properties of the spaces, which will allow to establish the desirableresults in an appropriate way.1.1.1 General NotationLet R be the set of real numbers. Also, let E d denote the d-dimensional Euclidianspace and D be an open set in E d with boundary ∂D and closure ¯D. We denoteby E d+ the half space in E d , ie E d := {x ∈ E d |x i > 0, ∀i = 1, . . . , d}. Inwhat follows, T indicates a real nonnegative number such that the interval [0, T ]is a time interval from 0 until T , while t and t 0 are used as time variables. E d+1 isdefined to be the space [0, ∞)×E d , which in general includes d spatial dimensions(here will be denoted by x i with i = 1, ..., d) and one dimension reserved for thetime variables. In several cases, an artificial, single-dimensional, space variabley will be introduced apart from x. The scope and the role of this variable willbe explained properly in due course.Q is a domain in E d+1 with boundary

Chapter 1. Introduction 5∂Q and closure ¯Q. In other words, Q is the cylinder (0, T ) × D. Often in thesequel Q will be called Parabolic domain. For both boundaries mentioned (∂Dand ∂Q) certain regularity conditions are usually required. For the purposes ofthis analysis, we assume that ∂D is a C 1,l elliptic boundary to get existence anduniqueness results for Partial Differential Equations. For parabolic domains likeQ it is usually enough to assume a Lipschitz condition for the boundary. It ispossible that other conditions could be used like i.e piecewise smooth (smoothmeans infinitely continuously differentiable), or even smooth, which is too strictthough. One possible definition of a Lipschitz boundary is the one that follows.Definition 1.1.1. An open set D has a Lipschitz boundary if for every z ∈ ∂Dthere exist r Z> 0, an orthonormal basis e 1 , ..., e d and a Lipschitz continuousfunction F of x ′ = (x 1 , ..., x d−1 ), such that{x ∈ D ∣ ∣ |x − z| < rz } = {x ∈ E d∣ ∣|x − z| < rz , x d > F (x ′ )}.We will consider a C 1,l boundary to be as follows (like in [17]).Definition 1.1.2. A bounded domain D ∈ E d and its boundary are of class C 1,l ifat each point x 0 ∈ ∂D there is a ball B = B(x 0 ) and a one-to-one mapping Ψ ofB onto D ⊂ E d,+ such that:• Ψ(B ⋂ D) ⊂ E d,+ ,• Ψ(B ⋂ ∂D) ⊂ E d,+ ,

Chapter 1. Introduction 6• Ψ is a once differentiable function with Lipschitz derivative in B (Ψ ∈C 1,l (B)),• Ψ −1 ∈ C 1,l (D).Note also, that in this work, the domain Q is not particularly useful and we wouldrather replace it by [0, ∞) × E d . That is because we would like to use this spacein the finite horizon problem, where the time horizon T ∈ [0, ∞) is fixed and thediffusion is evolving freely with no restriction on the space domain until time T .Let (Ω, F, F t , P ) be a complete, filtered probability space. F t , t ≥ 0 is a leftcontinuous,increasing family of σ−algebras, completed with respect to P , andF t ⊂ F,∀t ≥ 0 . P stands for the probability measure on that space.Next, the F t -adapted Wiener process is defined. The standard Wiener process isthe basic building block of diffusion processes.Definition 1.1.3. Let t ∈ [0, T ]∀T ∈ [0, ∞) , then the process W t is a onedimensionalF t −Wiener process if the following conditions are fulfilled:(i) W t1 − W t0 , W t2 − W t1 , . . . , W tn − W tn−1are independent random variablesfor every 0 ≤ t 0 ≤ t 1 ≤ . . . ≤ t n−1 ≤ t n . In particular, for all t ′ > t,(W t ′ − W t ) is independent of F t and W t ∈ F t .(ii) W t − W s are Gaussian random variables with mean zero and variance t − sfor all 0 ≤ s < t.

Chapter 1. Introduction 7(iii) W 0 = 0 and W t is continuous for every t.The extension to the d-dimensions is natural.Definition 1.1.4. Let W it be one-dimensional independent F t −Wiener processesas defined in Def. 1.1.3, for all 0 < i ≤ d . Then the vector ⃗ W t =[W 1t , W 2t , · · · , W dt ] is a d-dimensional F t −Wiener process.For existence and properties of the Wiener process see for example [30].Definition 1.1.5. A process X : [0, ∞) → R defined on a probability space(Ω, F, F t , P ) by the mapping (t, ω) ↦→ X t (ω) is called progressively measurableif for every s ∈ [0, ∞) the stopped process X t∧s is B[0, s] ⊗ F s measurable (Brefers to Borel σ−algebra).Let W t be a d 1 −dimensional F t −Wiener process and σ t a random d × d 1 matrix,progressively measurable with respect to F t .Definition 1.1.6. We call stochastic integral a continuous, progressivelymeasurable process ∫ tσ 0 s dW s , with the property E ∣ ∫ ∣t ∣∣ σ 2 ∫ t0 s dW s = E ‖σ 0 s‖ 2 dsand under the assumption that E ∫ t0 ‖σ s‖ 2 ds < ∞.The most general process we are going to consider in the chapters that followwill be the following non-homogeneous diffusion process, starting from the time

Chapter 1. Introduction 8t 0 ≥ 0:∫ t∫ tX t = x + b(s, X s ) ds +t 0σ(s, X s ) dW s ,t 0t ≥ t 0 , X t0 = x. (1.1.1)Assumptions on b, σ are provided on the next page. However, in parts of Chapter2, when we examine some exit time problems, we will consider a simpler, timehomogeneousdiffusion, starting from t = 0:X t = x +∫ t0b(X s ) ds +∫ t0σ(X s ) dW s , t ≥ 0, X 0 = x. (1.1.2)In both equations (1.1.1) and (1.1.2) also we assume that W t is a d 1 -dimensionalWiener Process (d 1 ≥ d).Equivalently, we write (1.1.1) and (1.1.2) in differential form as follows:dX t = b(t, X t ) dt + σ(t, X t ) dW t , t ≥ t 0 , X t0 = x (1.1.3)anddX t = b(X t ) dt + σ(X t ) dW t , t ≥ 0, X 0 = x (1.1.4)respectively, in (Ω, F, F t , P ).Furthermore, in (1.1.1) x ∈ E d is a d-dimensional vector, b(t, x) ∈ E d isa d-dimensional Borel-vector function and σ(t, x) represents a d × d 1 Borelmatrixfunction. In all cases we assume that d ≤ d 1 to avoid degeneracy.

Chapter 1. Introduction 9Accordingly we define the d × d Borel-matrix a(t, x) = σσ ∗ (t, x), where ∗ standsfor transpose of a matrix. a(t, x) is taken to be positive definite throughout thiswork. Corresponding definitions for (1.1.2) follow naturally.Definition 1.1.7. By a solution of equation (1.1.3) we mean a quadruplet(Ω, F, F t , P ) and stochastic processes X t , t ≥ t 0 and W t , t ≥ t 0 such that1. X t is continuous in t almost surely,2. X t is an F t -adapted process and W t is an F t -Wiener process,3. P(X t − x − ∫ tt 0b(s, X s ) ds − ∫ tt 0σ(s, X s ) dW s , ∀t ≥ t 0)= 1.Remark 1.1.1. The above solution is called Strong Solution if X t ∈ F t = F x ∧FtW , ∀t ≥ t 0 , completed with respect to P . F x ∧ FtWis the filtration generatedby the initial data and/or the considered fixed Wiener process. See also [30] or[24].Suppose, finally that τ is the first exit time of the process X x tfrom the domainD (in this case we assume that x ∈ D). In other words, τ is the first time thatXt x hits the boundary ∂D (τ = inf{t ≥ 0 : X t ∈ ∂D}). Therefore, τ is astopping (Markov) time. Sometimes it is useful for practical purposes to add afinal ”payment” to the cost at the time when the process hits the boundary or atthe terminal time T . We denote this payment by a function Φ(Xτ x ) or Φ(XT x)accordingly.

Chapter 1. Introduction 10We now introduce a new variable and a new dependence of the coefficients band σ on some ”tuning” or control parameter α. That means b and σ are nowfunctions of three variables (α, t, x). Consider a d-dimensional SDE with a d 1 ≥d-dimensional Wiener process (W t , F t , t ≥ 0) as before, now with controlledcoefficients b and σ:dX t = b(α t , t, X t ) dt + σ(α t , t, X t ) dW t , t ≥ t 0 , X t0 = x. (1.1.5)α t ∈ A ⊂ E l is a control process for each t 0 ≤ t and we assume that A ≠ ∅,bounded and closed. There several types of control strategies. Here, we give therelevant definitions similar to Krylov as in [29].Definition 1.1.8. A process α t (ω) : ω ⊂ Ω → A ⊂ E d progressivelymeasurable with respect to F = (F t , t ≥ 0) is called an admissible strategyif there exists a strong solution of equation (1.1.5).To each such strategy , we associate the solution of equation (1.1.5), which willexist and will be a unique strong solution because of the assumptions that we makein all cases.Let A be the set of admissible strategies.Definition 1.1.9. The admissible strategy α t (ω) is said to be a Markov strategyif α t (ω) = ˆα t (X t ) for some Borel function ˆα t (x) and X t ,t ≥ t 0 is a strongsolution of (1.1.5).

Chapter 1. Introduction 11We denote the set of all Markov (non-homogeneous in general) strategies as A M .1.1.2 Notation on Spaces, Definitions and Basic PropertiesLet C be the set of all continuous functions in E d . The space C 2 (D) includes allfunctions that are twice continuously differentiable with respect to space variablesin D, while the space C 1,2 (Q) contains all the functions once continuouslydifferentiable with respect to time and twice with respect to the space variablesin Q. The subindex b on the above mentioned spaces will indicate that the setis restricted to bounded functions only and that their derivatives (if they exist)are also bounded.All the previous spaces have the advantageous propertiesof differentiability and continuity. However, since the above class of functionsis quite narrow, the use of spaces of functions with more relaxed properties isappropriate.We are going to refer to the Banach spaces of Borel measurable functionssummable with order p as Lebesgue spaces L p , (p ≥ 1). For the functions fbelonging to L p in a domain D, ‖f‖ p,D denotes the following norm:(∫ ) 1‖f‖ p,D = |f(x)| p pdx . (1.1.6)D

Chapter 1. Introduction 12Furthermore, a local L p space is considered in the following sense∫ T ∫L D p,loc = {f(t, x) : 0 ≤ t ≤ T, x ∈ E d , sup|f(t, x)| p dt dx < ∞}.z∈E d 0 |x−z|≤1∩DIn other words, we take the supremum on all norms over all balls of radius ≤ 1(D i ⊆ D) in a neighborhood of x. This set of functions is wider in general (maycoincide) than the L p and is enough for the applications of this analysis so it mayreplace it.Starting from the set of functions L p , a more complex set can be constructedif we additionally require the functions included in the set to have generalisedderivatives. A function g is the generalised derivative of a function V (x) of orderl, in the domain D, if for all smooth test functions φ∫∫φ(x)g(x) dx = (−1) l φ x1 ,x 2 ,...,x lV (x) dx, (1.1.7)DDwhere φ x1 ,x 2 ,...,x l=∂ l φ∂x 1 ∂x 2 ...∂x l.In the previous definition (which is not theone we will apply here), we used the formula of integration by parts to makeclear the notion of the generalised derivative. In fact, the function V (x) maynot be differentiable at all. The derivatives exist in the sense that there exists asequence of functions V n (x) that converge to the function V and their derivativesto the generalized derivatives of V as defined (1.1.7) in L p,D . In case that the

Chapter 1. Introduction 13function V is differentiable, then its generalized derivatives would coincide withthe regular derivatives. Thus, we define formally the space of functions W 2 (D)in the following way. If the function V is defined in the closure of the open set D,then V ∈ W 2 (D) if there exists a sequence of functions V n ∈ C 2 (D), such thatthe following conditions are satisfied when n → ∞:andsup |V − V n | → 0 (1.1.8)x∈ ¯D‖V − V n ‖ W 2 (D) → 0. (1.1.9)In equation (1.1.9) the norm ‖V − V n ‖ W 2 (D) is defined to be the following for Vbounded Borel functions:d∑‖V ‖ W 2D= ‖V xi x j‖ p,D +i,j=1d∑‖V xi ‖ p,D + ‖V ‖ p,D . (1.1.10)i=1Condition (1.1.8) implies continuity for functions in W 2 p (D), while (1.1.9) ensuresconvergence of the V nx iand V nx i ,x jto the generalized derivatives of first and secondorder respectively. In full correspondence with the previous, regarding a domainof E d+1= [0, ∞) × E d , namely Q, the definition is analogous and equation(1.1.10) takes the form:‖V ‖ W1,2Qd∑= ‖V t ‖ p,Q + ‖V xi x j‖ p,Q +i,j=1d∑‖V xi ‖ p,Q + ‖V ‖ p,Q . (1.1.11)i=1

Chapter 1. Introduction 14Denote W 1,2 = ⋂ p>1 W 1,2p,loc and respectively W 2 (D) = ⋂ p>1 W 2 p . These are thespaces of equivalence classes of functions that we are going to search for solutionsof the PDE’s in this thesis. For functions of one time and two spacial variables(t, x, y) we will use the notation W 1,2,2 , which simply indicates two generalisedderivatives with respect to y.We would like in most cases to extend our results to unbounded parabolicdomains. This kind of extension is mentioned in the standard reference [31]. Alsoa result of Veretennikov [45, Theorem 3.1] extends standard PDE results frombounded domains to [0, T ] × E d . There are also various appropriate embeddingtheorems often concentrated under the title Sobolev’s theorem of embedding [43],that embed the spaces W 1,2 , W 2into the spaces of continuous differentiablefunctions for suitably large p and some assumptions on the smoothness of theboundaries of the domains. The spaces W 2 (D) and W 1,2 (Q) may be mentionedas Sobolev spaces with generalized derivatives summable up to order p.1.2 The General Problem SettingEventually, to achieve the objectives of this work, we will have to apply stochasticcontrol on the first and second moment of a markovian functional. However, we

Chapter 1. Introduction 15start the analysis by examining these same moments in the absence of control inorder to build some intuition for the problem and we postpone dealing with thecomplications introduced by stochastic control until the second chapter.There are various approaches and variations to the main problem for noncontrolleddiffusions. In the first approach, we would like to calculate the firstand second moment of a cost functional taken on a finite horizon interval or onan exit time problem. It is well known (details in the next section) that the firstmoment of such functionals is a solution to some partial differential equation, withappropriate initial conditions or boundary data respectively. However, the secondmoment requires some extra effort.When the time horizon over which the functional is taken is fixed (also calledfinite horizon problems), we consider the following function:(∫ TV (t 0 , x) = Et 0) 2f(t, Xt x ) dt , (1.2.1)where t 0 is the time variable, T is considered to be a fixed parameter, X x tis asolution of the time-inhomogeneous SDE (1.1.1) and function f : [0, T ] × E d →R. In the exit time problem, the function takes the form:(∫ τ2V (x) = E f(Xt x ) dt), (1.2.2)0

Chapter 1. Introduction 16where X x tis a solution of the time-homogeneous SDE (1.1.2), τ is the exit timefrom the domain D and a function f : E d → R.It is natural to try to calculate, at least numerically, both versions of functionV . The ideal result, would be the existence of an explicit formula allowing tocalculate directly the value of the function given some parameters. However, thisis not always possible (in the next chapter we will produce an explicit formulabut only for the one-dimensional process Xt x in the ”exit time case” (1.2.2)).Nevertheless, sometimes it is enough to be able to approximate the value of thefunction by using the knowledge of the system that describes its dynamics. From atheoretical point of view at least, it is often enough to prove that the value functionis the unique solution of a specified PDE in a certain space of functions. Then,normally, numerical methods can approximate the real solution. This will be thecase when the process X x tis multivariate.To the other end, another perspective of the problem is that of solvingprobabilistically partial differential equations which are degenerate. This happensto be the case for the second moment. This kind of PDE is difficult to be solvedprobabilistically and can be done only under restrictive assumptions (ex. nondegeneracy).However, there are established techniques (as will be discussed inthe sequel), that could simplify the solution of such PDE’s by transforming them

Chapter 1. Introduction 17into a system of much simpler PDE’s, at least for the non-controlled case.No matter under which perspective the problem is examined, there is always thedemand for general applicability of the solution.For this reason, we are going to look for solutions in equivalence classes offunctions with generalized derivatives. In modern PDE theory, these spaces offunctions (Sobolev spaces) have become a standard. In the chapters that follow,an analysis using optimal control will follow. Then it is really a severe restrictionto require that the solutions belong to C 1,2 or C 2 and therefore this kind of theoryis necessary.With control introduced, we seek the cost functions among the solutions ofHamilton-Jacobi-Bellman equations. We consider only the parabolic case in thiscontext and the value function now is of the form:(∫ T2v 2 (t 0 , x) = sup E t0 ,x f(α t , t, X x,αt ) dt). (1.2.3)α∈At 0Again the case for the first moment is well known (see [29]) but we will presentan idea of how to extend it to the second moment. Regularisation gives solutionsfor the control problem by overcoming the degeneracy of HJB equation, whilethe discrepancy between the regularised and the ”real” value function remainsbounded. After that, it is natural to try and find a Bellman’s equation for problems

Chapter 1. Introduction 18like mean-variance optimisation and mean-variance optimal stopping. Details ofthe existing theory used and our contribution in the field follow in the next section.1.3 History and Literature ReviewIn the pages that follow, we make an attempt to condense and briefly describe thenumerous contributions of other researchers in the topic we are going to examine.The history of this area is relatively recent as most of the relevant literature startedto appear in 1950s.This is the era that stochastic control is developed (seefor example Girsanov [18], Howard [21] and Bellman [2]) first as discrete timecontrol and then in continuous time. At the same time, research reports appearthat examine the distribution of Wiener functionals (see Kac and Darling [7] [23])and also a mathematical formulation is introduced in finance by Markowitz whocalculated optimal portfolios of financial assets for a single period model.As it is clear from the previous sections of the introduction, we would like todivide the history and the material in this work along three main axes: thefirst refers to moments of functionals without control, the second to controlledmoments and the third to mean-variance optimisation.When the underlying process is not controlled, there is an extensive literature on

Chapter 1. Introduction 19the calculation of moments of certain markovian functionals. Partial differentialequations that have as a solution (or that describe the dynamics of) non-controlledfunctionals like (1.2.1) and (1.2.2) are called Feynman-Kac equations. Kac in [23]published this result based on an unpublished thesis of Feynman. The Feynman-Kac moment formula was derived in order to calculate the distribution of certainWiener functionals (also continued in [7] by Darling and Kac), however it isproved to be suitable for many types of diffusion functionals. Fitzsimmons andPitman have extended this topic by proving results on the moments of stochasticfunctionals similar to the ones we use but they do not explicitly refer to partialdifferential equations and they only examine the exit time problem.In the same direction, Dynkin in ([8], chapter XIII, §4), presents theorem 13.17 inwhich, the m-th moment of a stochastic functional taken on a random time interval(function V m (x) (1.2.2)) represents the solution of a chain of m consecutiveelliptic PDE’s (”Dynkin’s chain of equations”) under certain assumptions. Forthe proof of this theorem, Dynkin assumes that the function V m has all theexponential moments (it is a very smooth function). This restriction excludes arange of problems from its applications and in particular problems with (at leastlocally) unbounded cost function f. Moreover, it refers only to elliptic PDE’s,with no control, excluding this way the fixed time interval controlled functionals

Chapter 1. Introduction 20like (1.2.3).Regarding the assumptions made and the space of solutions, Ladyzhenskaya,Solonnikov and Uralceva in [31], apart from the classical cases, present solutionsof Parabolic PDE’s in Sobolev and Hölder spaces. The results shown in this bookare analytic and ensure existence and uniqueness of solutions to linear and quasilinearPDE’s. Veretennikov has also expanded on these results in [45]. We aregoing to refer often to their results, mainly for the existence and uniqueness of asolution of a parabolic PDE with zero initial data, but corresponding results arealso available for elliptic PDE’s in [32] and [17].In the first part of this work, we extend the existing theory by showing that thesecond moment of a functional like (1.2.2) and (1.2.1) is the solution of a systemof two PDE’s (see Chapter 2, sections 2.4 and 2.3), similarly to Dynkin’s cascadeof equations. All the solutions are found in the space of functions with generalisedderivatives (Sobolev spaces), thus relaxing the assumptions made regarding thedifferentiability of the functions. This makes use of the results of [31] mentionedabove. These combined with a ”dynamic principle” argument (see Chapter 2,section 2.2), give existence of solution to a single degenerate PDE for the secondmoment of such functionals (see theorem 2.3.3 and 2.4.2).

Chapter 1. Introduction 21The spaces mentioned above are particularly useful when control is introducedto the problem. In this second part, the analogue of the Feymnan-Kac equationis the so called Bellman’s equation or HJB, which stands for Hamilton-Jacobi-Bellman equation. Krylov, in [29] applies stochastic control to functionals ofdiffusion processes. Bellman’s PDE (see [29], ch1) implies application of Itô’sformula, while on the same time the functions to which it must be applied includeexpressions like inf or sup, which automatically make them non-smooth incommon sense. Thus, Krylov is establishing a version of Itô’s formula applicableto functions that belong to Sobolev spaces (or spaces of functions with generalisedderivatives). The result, also known as Itô-Krylov formula, is an equality whenapplied to non-degenerate processes and an estimation when this assumption isrelaxed. Krylov’s proof, uses a sequence of functions that converge in some L psense, as explained in the previous section, to the function differentiated and toit’s derivatives respectively. In order to prove convergence to the limit, Krylovuses estimates for the distribution of a markovian functional.It happens thatthe assumptions used by Krylov are very close to those used by Ladyzhenskaya,Solonnikov and Uralceva. Indeed, Krylov’s theory may be used to solve a PDEprobabilistically in the same space in which they solve a PDE of the same typeanalytically.

Chapter 1. Introduction 22Except from Krylov, control problems for markovian functionals have beenextensively studied by several others (see for example [39], [41] and [13]). Wealso refer to Borkar (see [3] and [4]) who seeks solutions using a compactnesscontinuity argument for the space of controls. Nisio semigroup (see in this context[35], [36], [38] and [37]) is a powerful tool in this direction for HJB equations asit can play the same role as the transition semigroup in Feynman-Kac equation. Inthe context of dynamic programming, which yields the HJB equation, Rishel andFleming (see [40], [13]) introduced the notion of a weak solution for a degenerateHJB equation. They also prove that this weak solution has indeed a density undersome general assumptions (although control is only applied to the drift and notto the diffusion coefficient) and coincides with the value function of the initialcontrol problem.In fact, Fleming’s approach to degeneracy, is similar to ourapproach of regularisation as Fleming approximates the degenerate differentialoperator by series of non-degenerate ones. The result is a regularising term on thePDE depending on a constant ε.All the previous results, have established the solutions of HJB equations for thefirst moment of a controlled markovian functional. However, none of these resultsrefers to the second or higher moments. In chapter 3 we give a degenerate HJBequation for the second moment of a controlled functional of the type (1.2.1),

Chapter 1. Introduction 23thus reducing the problem so that existing theory can be applied.A solutionwill be found after regularisation in order to overcome the degeneracy problem.Furthermore, we will prove that the cost of regularisation remains bounded.The third part of this work, is mainly inspired by the financial applications andparticularly that of ”portfolio selection”. Philosophically this has always been anissue, but it is due to Markowitz [34] in 50’s that a mathematical formulationof the problem has been established.Markowitz’s selection was based on a”mean-variance” optimisation criterion, which is natural under the perspectivethat the variance represents the risk and the mean the expected return of aninvestment. The initial model is a single-period model but for about half a centuryit served as the basis for various mathematical-based studies in Economics andFinance and is still highly influential. In the last two decades this concept wasextensively investigated in modern stochastic financial theories, where a diffusionis involved and the optimisation is in continuous time. Föllmer and Sondermann[16] initiated the study of this approach for semimartingale models of stochasticmarkets, continued further in [15], [42] and in the following years in [42], [33],[19], et al. Unlike in the more classical theory of stochastic control, here Bellmanequations (often called HJB for Hamilton–Jacobi–Bellman) did not seem to playany significant role, except in linear–quadratic (LQ) theory [48] (not covered in

Chapter 1. Introduction 24our context).Here, we propose a parametric Bellman’s equation, depending on a realparameter from a bounded interval, which is promising in terms of numericalapproximations.What is more, this is important as it implies sufficiency ofmarkovian strategies for the regularised value function.The optimal strategyfor the regularised problem, proves to be also ɛ-optimal for the degenerateproblem and therefore we conclude that markovian strategies are sufficient forthe degenerate problem as well. If the ”mean-variance” criterion is changed to”first-second moment”, then the results are significantly simpler.

25Chapter 2Higher Moments of a non-controlledMarkovian functional2.1 IntroductionIn this chapter, the first and the second moment of a markovian cost functionalare examined in connection to the corresponding Partial Differential Equations.The types of cost functionals considered were given in the previous chapter (seeequation 1.2.1 and 1.2.2). The cost function depends on a non-controlled diffusionprocess like (1.1.1). The main concern of this chapter will be the representationof the second moment of such a functional as the unique solution of a PDE.Several variations will be studied according to the choice of the time interval, thedimensions of the stochastic process and the addition of a final payment. Thus,

Chapter 2. Higher Moments of a non-controlled Markovian functional 26for instance, regarding (1.2.1), we look for the solution of a Parabolic PDE, whilefor the exit time problem (1.2.2) Elliptic PDE’s are examined.In general, a d-dimensional diffusion will be used. However, in the exceptionalcase, where the diffusion process is of dimension one, there is the advantage ofan explicit solution.A final payment function will be added later to the costfunctionals considered. Although this type of final payment is not very general, inthat instance, it introduces the idea on a rather specific, but interesting problem.In the following chapters, a more general final payment will be used. This willresult to some more complicated calculations, although the main idea of the resultsremains the same. The diffusion involved is not a controlled one, so we are notactually using HJB equations, but Feynman-Kac type PDE’s instead. However,ideas from that theory are still used in order to provide intuition and links to thetheory for controlled diffusions.In comparison with the existing literature, we could say that we introduce a newmethod for the calculation of moments of markovian functionals. This methodhas the potential to cover cases that are not possible to cover using the existingtheory.Earlier methods require exponential moments of the random variable i.e.

Chapter 2. Higher Moments of a non-controlled Markovian functional 27E t,x exp ( α ∫ Ttf(s, X s ) ) < ∞ with some α > 0, while for the exit problem,correspondingly, E x exp ( α ∫ τ0 f(X s) ) < ∞. Although in the current text f isassumed bounded, we believe that the same results can be applied for a functionf belonging in some L p space (with suitable p). In particular because the methoditself does not require E t,x exp ( α ∫ Ttf(s, X s ) ) < ∞ any more. Similarly, for theexit problem, we do not require E x exp ( α ∫ τ0 f(X s) ) < ∞, and here it is evenmore subtle because even f bounded is not enough, due to τ, which is potentiallyunbounded (something that we will examine in Section 2.4).However, it ispossible and certainly much easier to require E x( ∫ τ0 f(X s) ) 2< ∞ compared toE x exp ( α ∫ τ0 f(X s) ) < ∞. Note that although the first case (bounded f) can berecovered from the existing literature (Dynkin’s chain of equations or Feynman-Kac equations), the latter is not covered.The procedure can be extended to higher than the second moments although thecalculus gradually becomes complicated. The first and the second moment arevery important though, as they produce the variance of the functional. We startour analysis with a restricted case and then we present two general cases for thefinite horizon problem and the exit time problem.

Chapter 2. Higher Moments of a non-controlled Markovian functional 282.2 A preliminary analysis for the finite horizonproblem (Parabolic Case) with the use of theartificial variable yIn this section, a degenerate parabolic PDE is proposed for the second momentof a cost functional over a fixed time interval. The result is severely restrictedby the assumptions made, however, it enhances the intuition about the solutionsrelated to the second moment and gives the motivation for the rest of the results.A dynamic Principle is used for Feynman-Kac equations, similarly to HJBequations, although there is no control.This section has strong links to thecorresponding theory for controlled functionals. In that case, the second momentof the value function is a solution to a very similar HJB PDE.Let X x t be a d-dimensional diffusion process as defined in equation (1.1.1),with X t0 = x ∈ E d and the first moment of the cost functional:U(t 0 , x) = E t0 ,x∫ Tt 0f(s, X s ) ds.Consider again the function (1.2.1), which is the second moment of our cost

Chapter 2. Higher Moments of a non-controlled Markovian functional 29functional:V (t 0 , x) = E t0 ,x( ∫ Tt 0) 2.f(s, X s ) ds(2.2.1)It is useful to work out a transformation of V above as follows. We have, due tothe Fubini theorem:V (t 0 , x) = E t0 ,x( ∫ Tt 0) ( ∫ T )f(s, X s ) ds f(t, X t ) dtt 0( ∫ T ( ∫ T= 2E t0 ,x f(t, X t )t 0 t))f(s, X s ) ds dt∫ (T∫ )T= 2 E t0 ,x f(t, X t ) f(s, X s ) ds dtt 0 t∫ (T ( ∫ T) )= 2 E t0 ,x E f(t, X t ) f(s, X s ) ds|F t dtt 0 t∫ (T( ∫ T) )= 2 E t0 ,x f(t, X t )E f(s, X s ) ds|F t dtt 0 t( ∫ T= 2E t0 ,x f(t, X t )E( ∫ T) )f(s, X s ) ds|X t dtt 0 t( ∫ )T= 2E t0 ,x f(t, X t ) U(t, X t ) dt ,t 0(2.2.2)

Chapter 2. Higher Moments of a non-controlled Markovian functional 30due to the Markov property, where∫ TU(t, x) := E t,x f(s, X s ) ds.tThe result of this transformation is to express the second moment of a simple costfunctional as the first moment of a more complicated functional. In the sequel,similar transformations of the second moment are to be used with some variationswhen we refer to the exit time problem or when a ”final payment” is added to theproblem.The internal integral in (2.2.2) can be interpreted in the sense of the loss startingfrom time t, as in the derivation of the Bellman’s equation (see [29]) for controlproblems. An alternative way to see the function (2.2.2) is the following. Wetransform V in a similar way as beforeV (t 0 , x) = E t0 ,x( ∫ Tt 0) 2f(t, X t ) dt( ∫ T ) ( ∫ T )= E t0 ,x f(s, X s ) ds f(t, X t ) dtt 0t( 0∫ T ( ∫ )t )= 2E t0 ,x f(t, X t ) f(s, X s ) ds dtt 0 t 0(2.2.3)and then we add a new variable y in a way that it becomes now:V (t 0 , x, y) = 2E t0 ,x∫ T(t 0f(t, X t ) y +∫ t)f(s, X s ) ds dt, (2.2.4)t 0

Chapter 2. Higher Moments of a non-controlled Markovian functional 31where we take T as fixed. After adding this variable y ∈ R, we can look at thefunction V as the first moment of a functional related to a pair of processes:X t = x +∫ t∫ tYt x = y + f(s, Xs x ) ds,t 0(2.2.5)t 0b(s, X s ) ds +∫ tt 0σ(s, X s ) dW s . (2.2.6)In particular,V (t 0 , x, y) = 2E t0 ,x,y∫ Tt 0f(t, X t )Y t dt,at y = 0. Apparently, the first process does not contain any diffusion part andhas only a drift. On the other hand the process X x tincludes both a drift and adiffusion part. Therefore, some kind of degeneracy is inherited in the system. Inthe analysis that follows, we will see that this is indeed true.In the rest of this section we are going to use the general idea of DynamicProgramming on which Bellman’s Principle is based (see [29, Chapter 1, §1] fordetails) along with some restrictive assumptions on the function f in order toproduce the Feynman-Kac PDE that is satisfied by the function V . We would liketo warn the reader that the following analysis is not rigorous and it is presentedhere only to enhance the intuition on the problem. More rigorous results follow inthe next sections. For the rest of this section, the following assumptions will be inplace:

Chapter 2. Higher Moments of a non-controlled Markovian functional 32(A 0 ) • Function (b(t, x), 0 ≤ t ≤ T, x ∈ E d ) is Borel measurable withvalues in E d , function (σ(t, x), 0 ≤ t ≤ T, x ∈ E d ) is d × d–matrix valued Borel measurable, both σ, b ∈ C 1,2β(E d+1) (continuouslydifferentiable with respect to t, x, with Hölder continuous derivativeswith coefficient β ∈ (0, 1) of first and second order respectively).• Matrix function σσ ∗ is uniformly non-degenerate, a(t, ·) := σσ ∗ (t, ·)is uniformly continuous, and this continuity is uniform with respect to0 ≤ t ≤ T .• ∃ β ∈ (0, 1) and constant K, such that ‖b(t, x) − b(t, x ′ )‖ + ‖σ(t, x) −σ(t, x ′ )‖ ≤ K‖x − x ′ ‖ β (Hölder continuous).• Function (f(t, x), 0 ≤ t ≤ T, x ∈ E d ) is Borel measurable andf(·, ·) ∈ C 1,2β(E d+1).First, some requirements on the continuity and differentiability of V must besatisfied. We check them in the lemmas that follow. Notice that in this section,the summation agreement (see the list of notations) is used quite often.As a preparation for these lemmas, we notice that the derivative of X x t∈ E d withrespect to x is a d × d matrix valued process Z t . Componentwise, for k, j =

Chapter 2. Higher Moments of a non-controlled Markovian functional 331, ..., d:∂(Xt x ) k= (2.2.7)∂x j= ∂x k∂x j+= ∂x k∂x j+∫ t∂b k (s, Xs x )t 0∂x j ′∫ t∂(X x s ) j ′∂x jds +∫ tt 0< ∇b k (s, X x s ), ∂(Xx s )∂x j> ds +∂σ k,i (s, Xs x )t 0∂x j ′∫ t∂(X x s ) j ′∂x jdW i st 0∂σ k,i (s, X x s )∂x k∂(X x s ) j ′∂x jdW i s.Furthermore, the following lemma comes from [9].Theorem 2.2.1. Assume A 0 . Z t itself is a solution of a linear SDE system and iscontinuously differentiable.Proof. It follows directly from [9, Ch. II, Theorem 3.3].Therefore we can formulate the following lemmas about the continuity anddifferentiability of the value function V under the assumptions A 0 .Lemma 2.2.1. Under the set of assumptions A 0 the first and second derivatives ofV with respect to x exist and are continuous and bounded.Proof. The gradient of V with respect to x is a d−dimensional vector with thefollowing components for k, j = 1, ..., d:∫∂V (t 0 , x, y)T(∫ T∂f(s, Xs x ) ∂Xsx,j= 2E t0 ,x∂x k t 0 t ∂x j ∂x k∫ T( ∫ T) ∂f(t, Xx+2E t0 ,x y + f(s, X s ) dst )t 0 t∂x j)ds f(t, X t ) dt∂X x,jt∂x kdt,

Chapter 2. Higher Moments of a non-controlled Markovian functional 34which is continuous. Thus,the second derivative of V with respect to x becomes:∂ 2 V (t 0 , x, y)= 2E t0 ,x∂x k ∂x i+ 2E t0 ,x+ 4E t0 ,x+ 2E t0 ,x+ 2E t0 ,x∫ Tt 0∫ Tt 0∫ Tt 0∫ T(t 0(∫ Tt(∫ T∂ 2 f(s, X x s )∂x j ∂x i ′∂f(s, X x s )∂x j∂X x,js∂x k∂ 2 Xsx,j∂x k ∂x i ′∂X x,i′s∂x i∂X x,i′s)ds f(t, X t ) dtt∂x i(∫ T)∂f(s, Xs x ) ∂Xsx,i′ ∂f(t, Xxdst )t ∂x i ′ ∂x i ∂x i ′∫ T) ∂ 2 f(t, Xt x )y + f(s, X s ) dst∂x j ∂x i ′∫ T) ∂f(t, Xxf(s, X s ) dst )∂x j∫ Tt 0(y +t)ds f(t, X t ) dt∂X x,jt∂x k∂X x,i′t∂x i∂ 2 X x,jt∂x k ∂x i ′dt∂X x,i′t∂x i∂X x,i′t∂x iBecause of assumptions A 0 and the previous Theorem from [9], all the aboveexpressions are continuous and bounded. This completes the proof.dtdt.Here, we use Z x t as the derivative of the process X x t with respect to x forillustrative purposes.The representation is not compact and the expressionsbecome cumbersome. Instead of using classical derivatives, we could make useof the generalised or also-called L p -derivatives. For a definition and description,we refer to the introduction and to ([30] Chapter 2, §1).Lemma 2.2.2. Under assumptions A 0 , the first derivative of V (t 0 , x, y) withrespect to t 0 exists and is continuous.Proof. From the Feynman-Kac equation we know that functions of the formu(t, x) := E t,x∫ Ttg(t, X t ) dt are probabilistic solutions of PDE’s of the following

Chapter 2. Higher Moments of a non-controlled Markovian functional 35form:( ∂∂t + L) u(t, x) + g(t, x) = 0, 0 ≤ t ≤ T, ∀x ∈ E du(T, x) = 0, (2.2.8)whereL =d∑i=1b i (t, x) ∂∂x i+ 1 2d∑ ∂ 2a i,j (t, x) . (2.2.9)∂x i x ji,j=1In equation (2.2.2) the function V is given the following representationV (t 0 , x) = 2E t0 ,x∫ Tt 0f(t, X t )U(t, X t ) dt,where U as well as f are bounded because of the assumptions A 0 .Hence,according to Feynman-Kac equation, the above representation of V suggests thatwe should consider the following PDE( ∂∂t + L) V (t, x) + 2f(t, x)U(t, x) = 0, 0 ≤ t ≤ T, ∀x ∈ E dV (T, x) = 0. (2.2.10)According to [31, Theorem 5.1, Chapter IV], under the assumptions A 0 the aboveequation has a unique classical solution V ∈ C 2,1β(E d+1). In particular, all thederivatives V t , ∂V∂x i,∂ 2 V∂x i ∂x j, i, j = 1, . . . , d exist and are Hölder continuous.Now, we consider the above solution and we plug in X t . We then apply Itô’sformula to V (t, X t ) between t 0 and T . We get that:V (T, X T ) − V (t 0 , x) =∫ Tt 0∫( ∂T∂t + L) V (t, X t ) dt + ∇ x V (t, X t )σ(t, X t ) dW t .t 0(2.2.11)

Chapter 2. Higher Moments of a non-controlled Markovian functional 36By our assumptions the quantity ∇ x V (t, X t )σ(t, X t ) is bounded in [t 0 , T ] × E dand therefore the stochastic integral in the above equation is a martingale with zeromean. Hence, after taking expectations in (2.2.11) and using the PDE (2.2.10)along with its terminal condition, we get:V (t 0 , x) = 2E t0 ,x∫ Tt 0f(t, X t )U(t, X t ) dt. (2.2.12)Therefore, we see that our function V as expressed in (2.2.2) coincides with thesolution of (2.2.10). It is then continuously differentiable with respect to t 0 .Remark 2.2.1. We sketch another way of proving the last Lemma.For our non-homogeneous SDE, dX s = b(s, X s ) ds + σ(s, X s ) dW s , s ≥ t, weobserve that the process X s is equivalent in law to a new process Yst,x , which issolution to dY s = b(t + s, X s ) ds + σ(t + s, X s ) dW s , s ≥ 0, Y t = x. Here tplays the role of a parameter and Y is L p −differentiable for any p ≥ 1 (see [29,Chapter 2] for details). Denote this derivative by Z s := ∂Yst,x /∂t and note thatthis is also a solution of an SDE. Then we can writeE t,x∫ Ttf(s, X t,xs ) ds =∫ T −tFinally, we can now differentiate the last expression∂∂t∫ T −t0∫ T −t00Ef(t + s, Yst,x ) ds.Ef(t + s, Yst,x ) ds = −Ef(t + T − t, Y t,xEf t (t + s, Y t,xs ) ds +∫ T −t0T −t )Ef x (t + s, Y t,xs )Z s ds, (2.2.13)where all three terms are well-defined and continuous (see for example [9]).

Chapter 2. Higher Moments of a non-controlled Markovian functional 37Lemma 2.2.3. The first derivative of V with respect to y exists and is continuous.Proof. Differentiation of V with respect to y gives the following continuousfunction:∫∂VT∂y = 2E t 0 ,x f(t, X t ) dt. (2.2.14)t 0Given those preliminary ”results”, the following theorem can be formulated andproved. It suggests that the function V (2.2.2) represents the solution of a secondorder degenerate parabolic PDE. We will see in the sequel though, that this kind ofdegeneracy can be overcome using another technique. For this example, we canjust ignore it since our ”strict” assumptions ensure the existence of the solution.Theorem 2.2.2. Assume A 0 . Then V ∈ C 1,2bParabolic Partial Differential Equation:is the solution of the followingV t0 +d∑b i (t 0 , x)V xi + f(t 0 , x)V y + 1 2i=1d∑a ij (t 0 , x)V xi x ji,j=1= −2yf(t 0 , x),(2.2.15)with initial condition:V | t0 =T = 0.Proof. For the proof of this theorem we are going to apply the idea of Bellman’sPrinciple, which will be explained while used.Starting with V as defined

Chapter 2. Higher Moments of a non-controlled Markovian functional 38previously by (2.2.4), we choose conveniently the cost function to be thefollowing:( ∫ t)g(t, Xt x , Y x,yt ) = 2Y x,yt f(t, Xt x ) = y + f(t, Xs x ) ds 2f(t, Xt x ). (2.2.16)t 0Now the function V could be represented as the integral for a time period fromt 0 to T of the cost function. Thus we have reexpressed the initial problem as theproblem of calculating the expected total cost over a period of time given a costfunction g:V (t 0 , x, y) = E t0 ,x,y∫ Tt 0g(t, X t , Y t )dt. (2.2.17)We can write down the function V as follows:V (t 0 , x, y) = E t0 ,x,y(∫ S0t 0(∫ T))g(t, X t , Y t ) dt + E g(t, X t , Y t ) dt|F S0 ,S 0where F S0 is the ”history” of the process up to time S 0 . Then let us assume thatwe have all the information about the process X x tup to a certain time S 0 (thatassumption does not restrict us at all as we will see). But the inner expectationis the function V itself just starting at a later time S 0 . Thus we can rewrite theprevious equation in the following form:V (t 0 , x, y) = E t0 ,x,y(∫ S0)t 0g(t, X t , Y t ) dt + V (S 0 , X S0 , Y S0 ) .Now we bring inside the expectation the left hand side and this gives:E t0 ,x,y(∫ S0)t 0g(t, X t , Y t ) dt + V (S 0 , X S0 , Y S0 ) − V (t 0 , x, y) = 0.

Chapter 2. Higher Moments of a non-controlled Markovian functional 39Now, by dividing all terms by S 0 − t 0 and taking limits where S 0 goes to t 0 itresults to the following:limS 0 →t 0E t0 ,x,y( 1S 0 − t 0∫ S0g(t, X t , Y t ) dt + V (S )0, X S0 , Y S0 ) − V (t 0 , x, y)= 0.t 0S 0 − t 0It is clear from the definition of the function g that the first limit equals(2.2.18)yf(t 0 , X t0 ) = yf(t 0 , x) (2.2.19)as S 0 tends to t 0 . In order to calculate the second limit, we need to do the followinganalysis.Since Lemmas 2.2.1, 2.2.2 and 2.2.3 hold true we can apply Itô’s formula to thefunction V (S 0 , X S0 , Y S0 ). This gives the following expression:V (S 0 , X S0 , Y S0 ) = V (t 0 , X t0 , Y t0 ) +++d∑i=1∫ S0t 0∫ S0t 0b i (s, X x s ) ∂V∂x i(s, X x s ) + 1 2d∑i=1Now define the operator ˜L such that:( ∂V∂t (s, Xx s ) + f(s, Xs x ) ∂V∂y (s, Xx s )d∑i=1,j=1a ij (t, X x t ) ∂2 V∂x i ∂x j(s, X x s )∂V∂x i(s, X x s )σ i (t, X x t ) dW t . (2.2.20))dtThen:˜LV = ∂V∂t+ f(t, x)∂V∂y +d∑i=1b i (t, x) ∂V∂x i+ 1 2d∑i=1,j=1a ij (t, x) ∂2 V∂x i ∂x j.V (S 0 , X S0 , Y S0 ) = V (t 0 , x, y)+∫ S0t 0∫ S0˜LV (t, Xxt , Y yt ) dt+ σ ∗ ∇ x V (t, Xt x , Y yt ) dW t .t 0

Chapter 2. Higher Moments of a non-controlled Markovian functional 40Now, by taking expectations in the above equation, we can estimate the secondlimit in equation (2.2.18):( V (S0 , X S0 , Y S0 ) − V (t 0 , X t0 , Y t0 ))lim E=S 0 →t 0 S 0 − t ˜LV (t 0 , x, y). (2.2.21)0The stochastic integral disappears under the expectation as we have showed that∂V (t 0 ,x,y)∂x iis finite (Lemma 2.2.1). Interchanging the expectation with the limit inall previous cases is allowed because of the assumptions A 0 and the MonotoneConvergence Theorem.Then by replacing (2.2.21) and (2.2.19) to equation (2.2.18) we have:2yf(t 0 , x) + ˜LV (t 0 , x, y) = 0 ⇔ ˜LV (t 0 , x, y) = −2yf(t 0 , x),orV t0 +d∑b i (t 0 , x)V xi + f(t 0 , x)V y + 1 2i=1d∑a ij (t 0 , x)V xi x ji,j=1= −2yf(t 0 , x).Initial condition taken on (2.2.15) is natural, since when the cost functional istaken on a zero time interval it equals to zero.We will prefer in the sequel to write down the above PDE in slightly different andmore elegant notation in order to be in line with standard literature asV t +d∑b i (t, x)V xi + f(t, x)V y + 1 2i=1Vd∑a ij (t, x)V xi x ji,j=1∣t=T= 0,= −2yf(t, x),

Chapter 2. Higher Moments of a non-controlled Markovian functional 41without changing the meaning of any of the variables.It has been shown that the second moment of the functional we are interested incan be the solution of a second order parabolic PDE. However, the assumptionsmade about the function f and the coefficients b and σ were too restrictive and thusthe above analysis has rather narrow applications. Nevertheless, it is very usefulto show the final form of the PDE one has to solve for V as well as to observethe degeneracy inherited in the problem. We will see in the sequel, that we canavoid degeneracy and even the introduction of the variable y in the non-controlledcase. The reason for this is that, by following the idea of ”Dynkin’s chain ofequations” [8], we are able to solve a system of two second-order non-degeneratePDEs instead. In fact, V y will be considered as the solution of the first PDE. Then,the result will be replaced in the initial PDE, which will become non-degenerate.This analysis is the content of the next section.2.3 A system of equations for the finite horizon noncontrolledsecond momentIn this section, the result from the previous section is reformulated in an alternativeway. Equation (2.2.15) will be solved as a system of two ”simpler” equations.

Chapter 2. Higher Moments of a non-controlled Markovian functional 42By observing equation (2.2.2), it can be seen that the second moment of the costfunctional can be transformed into a composition of two simple functionals. FromLemma 2.2.3, someone can notice that the derivative V y that appears in (2.2.15) isin fact the first moment of the cost functional. Thus, the first moment implicitlyexists in the PDE, which gives the second moment. That is an indication of a chainof equations in the problem, which actually transfers the first moment to the righthand side of the PDE as a solution of a separate PDE of the same type. Indeed,”Dynkin’s chain of Equations” suggests that the second moment of a functionalcan be expressed as the solution of two consecutive PDE’s (in our case Parabolic,although Dynkin’s result involves only elliptic PDE’s). The first equation admitsas a solution the first moment and the second, which includes the first moment inthe right hand side, admits as a solution the second moment.The problem setting remains the same as in the previous section including all thedefinitions and notation. However, the artificial variable y is not going to be usedhere. Furthermore, a dramatic difference from the previous section is going tobe the space in which we solve the PDE’s. In section 1.4, we restrict functionV to be an element of the space C 1,2 , which limits somehow the applicabilityof this theory to quite smooth functions. In this section, we are going to extendour results to functions with generalized derivatives (see [31]). The notion of

Chapter 2. Higher Moments of a non-controlled Markovian functional 43the generalized derivative as well as the properties of the space W 1,2 have beendiscussed in the introduction. Thus, the solution of the problem can be formulatedwith the following two theorems. The first of the two theorems shows a trivialwell known result (see for example [30]), however, we present it here to give thecomplete image of the method for this case only. Throughout this section weassume the following:(A 2.3 ) • Function (f(t, x), 0 ≤ t ≤ T, x ∈ E d ) is Borel measurable andbounded.• Function (b(t, x), 0 ≤ t ≤ T, x ∈ E d ) is Borel measurable withvalues in E d , function (σ(t, x), 0 ≤ t ≤ T, x ∈ E d ) is d × d–matrixvalued Borel measurable, both σ and b are bounded.• Matrix function σσ ∗ is uniformly non-degenerate, a(t, ·) := σσ ∗ (t, ·)is uniformly continuous, and this continuity is uniform with respect to0 ≤ t ≤ T .Theorem 2.3.1. Let assumptions A 2.3 hold true. Then, the function U(t 0 , x) =∫ TE t0 ,xt 0f(t, X t ) dt is the unique solution in W 1,2 of the following PDE:d∑U t + b i (t, x)U xi + 1 d∑a ij (t, x)U xi x2j= −f(t, x), 0 ≤ t ≤ T, (2.3.1)i=1i,j=1with initial condition:U| t=T = 0.

Chapter 2. Higher Moments of a non-controlled Markovian functional 44Proof. Existence and uniqueness of solutions to equations like (2.3.1) in Sobolevspaces are well established (see for example [31, ch. IV, §9]). We onlyshow here that the solution indeed coincides with the expectation of the costfunctional.Under the assumptions A 2.3 , we can apply Itô’s formula withgeneralised derivatives (Itô-Krylov’s formula, see [1]). Since U ∈ W 1,2 there existsequences of regular functions U n that approximate U in the following sense:sup |U − U n | → 0,x‖U − U n ‖ W 1,2 → 0,recall that ‖U‖ W 1,2 = ‖U t ‖ p + ∑ di,j=1 ‖U x i x j‖ p + ∑ di=1 ‖U x i‖ d+1and ‖|∇ x (U − U n )| 2 ‖ p → 0.Since U n is a smooth function, Itô’s formula can be applied to it. Then:U n (t, X x t )−U n (t 0 , x) =whereL s U =∫ tt 0( ∂ ∂s +L s)U n (s, X x s ) ds+d∑b i (t, x)U xi + 1 2i=1∫ tt 0σ ∗ (s, X s )∇ x U n (s, X x s ) dW s ,(2.3.2)d∑a ij (t, x)U xi x j. (2.3.3)i,j=1”Krylov’s estimate” (see [29]) allows to pass equation (2.3.2) to the limit whenn → ∞. Thus, we write down directly the limit version of Itô’s formula in theappropriate time interval for this problem:U(T, X x T )−U(t 0 , x) =∫ Tt 0( ∂ ∫ T∂s +L s)U(s, Xs x ) ds+ σ ∗ (s, Xs x )∇ x U(s, Xs x ) dW s .t 0(2.3.4)

Chapter 2. Higher Moments of a non-controlled Markovian functional 45Furthermore, after taking expectations and using (2.3.1), (2.3.4) can be written asE t0 ,xU(T, X T ) − U(t 0 , x) = −E t0 ,x∫ Tt 0f(t, X t ) dt, (2.3.5)after noting that the stochastic integral dissappears under the expectation as∇ x U ∈ L p , ∀p andE t0 ,x∫ Tt 0|σ ∗ (s, X s )∇ x V (s, X s )| 2 ds ≤ C 1 ‖∇ x V ‖ 2d < ∞,(see for example [29, Chapter 2]). Finally, the boundary condition of equation(2.3.1) suggests that E t0 ,xU(T, X T ) = 0 and thusU(t 0 , x) = E t0 ,x∫ Tt 0f(t, X t ) dt. (2.3.6)It has been shown that the function U ∈ W 1,2 , which solves equation (2.3.1), is infact the first moment of the cost function.In theorem 2.3.1, we have verified that the function U(t 0 , x) =∫ TE t0 ,x t 0f(t, X t ) dt coincides with the solution of equation (2.3.1). Therefore themain result that will be formulated with the following theorem uses the functionU to simplify equation (2.2.15).Theorem 2.3.2. Let assumptions A 2.3 be satisfied. Then the function V (t 0 , x) isthe unique solution in W 1,2 of the following PDE:d∑V t0 + b i (t 0 , x)V xi + 1 d∑a ij (t 0 , x)V xi x2j= −2f(t 0 , x)U(t 0 , x), 0 ≤ t 0 ≤ T,i=1i,j=1(2.3.7)

Chapter 2. Higher Moments of a non-controlled Markovian functional 46with initial condition:V | t0 =T = 0.Proof. In a very similar way as in Theorem 2.3.1, the function V can beapproximated from by a sequence V n such that: sup x |V − V n | → 0, ‖V −V n ‖ W 1,2 → 0 and ‖|∇ x (V − V n )| 2 ‖ p → 0 for p large enough. Under theassumptions made in the theorem, we can apply Ito’s formula to V n (s, Xs x ) andpass to the limit when n tends to infinity. Thus we have the following version ofItô-Krylov’s formula:V (T, X x T )−V (t 0 , x) =∫ Tt 0(L s + ∂ ∫ T∂s )V (s, Xx s ) ds+ σ ∗ (s, Xs x )∇ x V (s, Xs x ) dW s ,t 0(2.3.8)where the operator L was defined in (2.3.3). Furthermore, by taking expectationsand using (2.3.7) and Krylov’s estimate (as in the previous theorem) to show thatthe stochastic integral is a martingale, (2.3.8) can be written as:E t0 ,xV (T, X T ) − V (t 0 , x) = −2E t0 ,x∫ Tt 0f(t, X t )U(t, X t ) dt. (2.3.9)Finally, the boundary condition of equation (2.3.7) suggests thatE t0 ,xV (T, X T ) = 0 and thus:V (t 0 , x) = 2E t0 ,x∫ Tt 0f(t, X t )U(t, X t ) ds. (2.3.10)Thus, if there exists a unique solution of the equation (2.3.7), then it coincides withthe second moment of the cost function. However existence and uniqueness of

Chapter 2. Higher Moments of a non-controlled Markovian functional 47such a PDE is ensured by the assumptions and the proof can be found in litterature(see for example [31, ch. IV §9] ).Comparing equation (2.3.10) with equation (2.2.2) given the information until( ∫ ) 2Ttime t, we see that indeed we get the solution V (t 0 , x) = E t0 ,x t 0f(t, X t ) dtas expected. Thus it has been shown that, when the process X x tis d-dimensional,under fairly relaxed assumptions, the second moment of the cost functional takenover the time interval (t 0 , T ) is the solution of the Parabolic equation (2.3.7),which in fact represents an implied system of two linear second order PDE’s(equations (2.3.7) and (2.3.1)).2.3.1 A single linear PDE for diffusion without controlRecall equation (2.2.15). We have seen that it admits the same solution as (2.3.7),namely the second moment of the cost functional.V t +d∑b i (t, x)V xi + 1 2i=1d∑ d∑a ij (t, x)V xi x j+ f(t, x)V y + f(t, x)y = 0,i=1 j=1(2.3.11)V (T, x, y) ≡ 0.

Chapter 2. Higher Moments of a non-controlled Markovian functional 48Simultaneously, consider the equation (2.3.1). We already know that there is asolution to the latter equation in the appropriate Sobolev class. Hence, we obtainthe following result.Theorem 2.3.3. There is a unique solution of the equation (2.3.11) in the class offunctions V : V (·, ·, y) ∈ W 1,2 for each y, such that(1) the function V is affine in y. That is say V (t 0 , x, y) = 2Ṽ (t 0, x) + y ˆV (t 0 , x)and ˆV is a solution to the equation (2.3.1) in W 1,2 ;(2) the function Ṽ is a solution to the the equation (2.3.7) with U = V y in W 1,2 .All the assertions follow easily from the previous section.2.4 A system of equations for the non-controlledsecond moment in the exit time problemIn this section, the problem setting slightly changes. In particular, there are somedifferences regarding the time interval over which the cost functional is taken andthe diffusion process considered. We remind that in this section we will considerX x tto be solution of the following SDE:X t = x +∫ tb(X s ) ds +∫ t00σ(X s ) dW s , t ≥ 0, X 0 = x. (2.4.1)

Chapter 2. Higher Moments of a non-controlled Markovian functional 49We also assume that the function f does not depend on time. In this new frame,the cost functional will be taken from time t = 0 up to an exit time τ. We definedτ in Section (1.1.1) to be the first exit time of the process X x tfrom the boundeddomain D (remember that D is an open, bounded domain of E d , with Lipschitzboundary ∂D and closure ¯D). That means that the process X x tevolves until itleaves the domain D (hits for first time the boundary). It is trivial to prove thatthis is indeed a stopping time.The assumptions made in this section are the following:(A 2.4 ) • Function (f(x), x ∈ E d ) is Borel measurable and bounded.• Function (b(x), x ∈ E d ) is Borel measurable with values in E d ,function (σ(x), x ∈ E d ) is d × d–matrix valued Borel measurable,both σ and b are bounded.• Matrix function σσ ∗ is uniformly non-degenerate, a(·) := σσ ∗ (·) isuniformly continuous with respect to x ∈ E d .• Γ (the boundary of the domain D) is C 1,l .Thus, now the function V takes the form( ∫ τ2.V (x) = E x f(X t ) dt)(2.4.2)0

Chapter 2. Higher Moments of a non-controlled Markovian functional 50Let us ensure ourselves that the above expression is well defined. We have alreadyassumed that the function f is bounded and we will show that the exit time hasa finite expectation as well, although this is considered to be a well-known fact.We follow the approach of [20]. We have already assumed that the domain Dis bounded. From the assumptions made above, it is clear that |b i (·)| < K and|a ii (·)| < K for i = 1, ..., d and we remind that the function a(·) is uniformly nondegenerateby assumptions A 2.4 . Then, invoking [20, Theorem 7.1 and Corollaries1 and 2], E s,x e γτ exists for sufficiently small positive constant γ. That means theexit time has exponential moments and in particular the first two moments arefinite.Following the same steps as in section 2.2 for equation (2.2.4), it is possible toshow that the function (2.4.2) can be transformed into the following function:( ∫ τ ) 2 ( ∫ ∞V (x) = E x f(X t ) dt = Ex00∫ ∞) 21(t < τ)f(X t ) dt][∫ ∞= E x 1(t < τ)f(X t ) dt 1(s < τ)f(X s ) ds (2.4.3)00[∫ ∞∫ ∞]= 2E x 1(t < τ)f(X t ) 1(s < τ)f(X s ) ds dt0t∫ ∞(∫ ∞)]= 2 E x[E 1(t < τ)f(X t ) 1(s < τ)f(X s ) ds|F t dt0t

Chapter 2. Higher Moments of a non-controlled Markovian functional 51∫ ∞(∫ ∞)]= 2 E x[1(t < τ)f(X t )E 1(s < τ)f(X s ) ds|F t dt0t∫ ∞(∫ ∞)= 2E x 1(t < τ)f(X t )E 1(s < τ)f(X s ) ds|F t dt0t∫ τ(∫ τ) ∫ τ= 2E x f(X t )E Xt f(X s ) ds dt = 2E x f(X t )U(X t ) dt,0t0where∫ τU(x) = E x f(X t ) dt (2.4.4)0We know that V (x) is well defined (< ∞) and therefore all equations aboveare valid. In particular, we are allowed to change the order of integration andtherefore insert the expectation inside the integrals. In a similar way as in theprevious section, equation (2.4.4) implies the existence of a system of PDE’s. Wewill present the result in a more ”compact” form than previously. We are goingto formulate only one theorem, since the result regarding the first moment U isknown.In particular, it is known that (see [32], or [29]) the function U (see (2.4.4))coincides with the solution of the equation:d∑b i (x)U xi + 1 d∑a ij (x)U xi x2j= −f(x) (2.4.5)i=1i,j=1U| ∂D = 0.Now, the relevant Theorem about the function V is the following:

Chapter 2. Higher Moments of a non-controlled Markovian functional 52Theorem 2.4.1. Let assumptions A 2.4 be satisfied.Then V (x) is the uniquesolution in W 2 (D) of the following PDE:d∑b i (x)V xi + 1 d∑a ij (x)V xi x2j= −2f(x)U(x) (2.4.6)i=1i,j=1V | ∂D = 0.Proof. We remind that due to the assumptions and the results of [20] the functionU is bounded. It is known that there exists a unique solution V of (2.4.6) in thespace W 2 (see for example [17, Theorem 9.15] or [32]). That means there is asequence of regular functions V n that approximate V and its spatial derivatives(with respect to the vector x) up to second order in L p norm (the sequence V napproximates the function V and its derivatives in the sense ‖V − V n ‖ W2 → 0.In particular ‖V − V n ‖ D → 0 and ‖|∇ x (V − V n )| 2 ‖ p,D → 0. Furthermore wecan apply Itô’s formula to V n :V n (X x t∧τ) − V n (x) =∫ t∧τ0∫ t∧τ(L s Vs n ) ds + σ ∗ (Xs x )∇ x Vs n dW s , (2.4.7)0where L s is the generator of the process as it appears in (2.3.3). Recall thatL =d∑i=1b i (x) ∂∂x i+ 1 2d∑ ∂ 2a ij (x) .∂x i x ji,j=1In that sense, the function V does not necessarily have ordinary derivatives, buthas ”generalised”, or also called Sobolev derivatives. The assumptions made inthe theorem and Krylov’s estimates (similar to (2.4.10) below, see [29]) providebounds for all the quantities on the right hand side of equation (2.4.7) and allow us

Chapter 2. Higher Moments of a non-controlled Markovian functional 53to pass equation (2.4.7) to the limit and have the generalised form of Itô’s formula:V (X x t∧τ) − V (x) =∫ t∧τ0LV (X x s ) ds +∫ t∧τTaking expectations on (2.4.8) gives the following:as ∇ x V ∈ L p , ∀p and0σ ∗ (X x s )∇ x V (X x s ) dW s . (2.4.8)∫ t∧τE x V (X t∧τ ) − V (x) = E x LV (X s ) ds + 0, (2.4.9)0∫ t∧τE x |σ ∗ (X s )∇ x V (X s )| 2 ds ≤ C 1 ‖∇ x V ‖ 2d < ∞, (2.4.10)0using Krylov’s estimate (see for example [29, Chapter 2]). Then, from (2.4.6) wehave∫ t∧τE x V (X t∧τ ) − V (x) = −2E x f(X s )U(X s ) ds. (2.4.11)Finally when t → ∞, due to Lebesque dominated convergence theorem and sinceE(τ) < ∞, we can pass to the limit the integral∫ t∧τ∫ τlim E x f(X s )U(X s ) ds = E x f(X s )U(X s ) ds.t→∞000Then:∫ τE x V (X τ ) − V (x) = −2E x f(X s )U(X s ) ds (2.4.12)and by making use of the boundary condition of (2.4.6) we take the following:∫ τV (x) = 2E x f(X s )U(X s ) ds,00

Chapter 2. Higher Moments of a non-controlled Markovian functional 54which is equal to (2.4.4) as we showed in the beginning of this section. On theother hand, U(x) is the solution of the following PDE:d∑b i (x)U xi + 1 d∑a ij (x)U xi x2j= −f(x), (2.4.13)i=1i,j=1U| ∂D = 0.The expression for U is proved by repeating the steps of the first part of thisproof.Summarizing the analysis of this section, the second moment of a payofffunctional in the exit time problem was proved to be the unique solution of asystem of two elliptic PDE’s. This was in full accordance with Dynkin’s result [8],but under slightly more relaxed assumptions. Actually, although our assumptionscan be relaxed further, we assume that f is bounded, which implies existence ofall moments, but we do not require exponential moments. In both the previousand this section, the result came out from the solution of a chain of two PDE’s.2.4.1 A single linear elliptic PDE for the exit time problemwithout controlSimilarly to what was shown for the parabolic system of equations the systemof the two elliptic PDE’s can also be replaced by the following single linear

Chapter 2. Higher Moments of a non-controlled Markovian functional 55degenerate PDE:d∑b i (x)V xi + 1 2i=1d∑ d∑a ij (x)V xi x j+ f(x)V y + 2f(x)y = 0,i=1 j=1V | ∂D = 0. (2.4.14)Here, in a similar to the previous sections way, we consider a differentrepresentation of the function V . In particular, starting from (2.4.3), we can write:V (x) = 2E x∫ ∞∫ τ= E x 2f(X t )00[1(t < τ) f(X t )∫ t0∫ t0f(X s ) ds dt.Then we can write V with the help of the auxiliary process:Y xt =∫ t0f(X x s ) ds]f(X s ) ds dtas∫ τV (x) = E x 2f(X t )Y t dt.0Then, as usually, we allow some initial data for Y t , namely,Y xt = y +∫ t0f(X x s ) dsand we get that∫ τV (x, y) = E x,y 2f(X t )Y t dt0

Chapter 2. Higher Moments of a non-controlled Markovian functional 56at y = 0.Now, simultaneously with equation (2.4.14), consider the equation (2.4.6). Wealready know that there is a solution to the latter equation in the appropriateSobolev class. Hence, we obtain the following result.Theorem 2.4.2. There is a unique solution of the equation (2.4.14) in the class offunctions V : V (·, y) ∈ W 2 for each y, such that:(1) the function V is affine in y. That is say V (x, y) = Ṽ (x) + 2y ˆV (x) and ˆV isa solution to the equation (2.4.5) in W 2 ;(2) the function Ṽ is a solution to the the equation (2.4.6) with U = V y in W 2 .All the assertions follow easily from the previous section.2.5 The d-dimensional exit time problem with ”finalpayment”In some problems, it is useful and reasonable to add a final payment to thecost functional. We stay in the previous case of the exit time problem. Thusall definitions are taken from the previous section.What changes here is theintroduction of a rather specific final payment. That means, when the processX x texits for the first time from the domain D, the function V is not any more

Chapter 2. Higher Moments of a non-controlled Markovian functional 57equal to zero but there is an additional cost (or payment) given by the functionΦ(X x τ ). So the function V is now of the following form:V (x) = E x( ( ∫ τ0)2f(X t ) dt)+ Φ(Xτ ) . (2.5.1)Note that this is not a general form of final payment since it is considered as a finalpayment for a quadratic cost functional and is not included in the second momentfor simplicity. More general final payment will be considered in later chapters.The method of solution is going to be exactly the same, through the solution of asystem of two consecutive equations. Naturally, what is going to be different is theterminal condition of the PDE each time. Therefore, we present the correspondingtheorem.Theorem 2.5.1. Let assumptions A 2.4 hold. Then V (x) =( ( )∫ 2 τE x f(X 0 t) dt)+ Φ(Xτ ) is the unique solution in W 2 ( ¯D) of thefollowing PDE:d∑b i (x)V xi + 1 2i=1d∑a ij (x)V xi x j= −f(x)U(x) (2.5.2)i,j=1V | ∂D = Φ(x).Proof. Again, we invoke well known results (see for example [17] or [32])for existence and uniqueness of solutions to the above equation.As for therepresentation of the solution, following step by step the method followed theorem

Chapter 2. Higher Moments of a non-controlled Markovian functional 582.4.1 we end up with:∫ t∧τE x V (X t ) − V (x) = −2E x f(X s )U(X s ) ds (2.5.3)and by letting t go to ∞ and making use of the boundary condition of (2.5.2) wetake the following:∫ τV (x) = 2E x f(X s )U(X s ) ds + Φ(x).Finally, it is known that U(x) is the solution of the following PDE00d∑b i (x)U xi + 1 d∑a ij (x)U xi x2j= −f(x), (2.5.4)i=1i,j=1U(x)| ∂D = 0.The solution for U is proved by repeating the steps of the first part of this proof.2.6 The exit time problem in dimension oneUntil this point, the variable x, appearing in the problem, was taken to bea d-dimensional vector.In this context we were able to prove existence anduniqueness of solution under relaxed assumptions. However, nothing could bementioned about explicit solutions. Indeed, in the multidimensional case it is toodifficult to obtain explicit solutions. On the other hand, if the problem is reduced

Chapter 2. Higher Moments of a non-controlled Markovian functional 59to one dimension, there is the possibility to work out an explicit solution for thecorresponding ODE’s, which would coincide with the second moment of a payofffunction taken over an exit time interval (elliptic case). Note that this calculusis not new and has been performed in the past by many authors (again see forexample [29]). However, it has not been performed for the second moment. Thissolution is presented in the rest of this section.LetX t = x +∫ tb(X s ) ds +∫ t00σ(X s ) dW t , (2.6.1)where x ∈ R, b(x) takes values in R and σ(x) represents a 1 × d 1 matrix.Accordingly we define the previously used a(x) = σσ ∗ (x) ∈ R. In equation(2.6.1), we also assume as usually that W t is a d 1 -dimensional Wiener Process.Suppose also that τ is now the first exit time of the process X x tfrom an interval(α, β), again assumed to be finite. Therefore, τ is again a stopping (Markov) time.In the next two theorems an explicit solution will be calculated, corresponding tothe second moment of a payoff functional. We first define the function V to be aspreviously of the following form:( ∫ τ2.V (x) = E x f(X t ) dt)(2.6.2)The idea of solving a system of two equations was adequately described and0

Chapter 2. Higher Moments of a non-controlled Markovian functional 60explained in the last two sections. Therefore, we are going to apply it here inexactly the same way.However, since we are trying to work out an explicitsolution, both the equations of the system will need to be solved explicitly.Theorem 2.6.1. Assuming that the ratios b(x)a(x)andf(x)a(x)are continuous and alsothat inf x a(x) > 0 , there exists a unique solution u in C 2 (α, β) of the followingODE:with boundary condition:b(x)u x + 1 2 a(x)u xx = −f(x), (2.6.3)u| x=α,x=β = 0.Furthermore u(x) = U(x) := E x∫ τ0 f(X t) dt.Proof. We can solve explicitly (2.6.3) as follows. Start by multiplying all theterms with e∫ x02b(s)a(s) ds . Then the ODE becomesb(x)e∫ x02b(s)a(s) ds u x + 1 2 a(x)e ∫ x02b(s)a(s) ds u xx = −e∫ x02b(s)a(s) ds f(x). (2.6.4)Divide all terms by a(x). This can be done since a(x) is bounded away from zero.Then (2.6.4) can be written as:∂( 1∫∂x 2 e x0)2b(s)a(s) ds u x = − 1 ∫a(x) e x02b(s)a(s) ds f(x). (2.6.5)Now integration of both parts of (2.6.5) with respect to x and a rearrangementgives the following:u x = −2e − ∫ x0∫2b(s) xa(s) ds01a(y) e ∫ y02b(s)a(s) ds f(y) dy + c 1 , (2.6.6)

Chapter 2. Higher Moments of a non-controlled Markovian functional 61where c 1 is a constant. Finally, integration of (2.6.6) with respect to x gives thefollowing:u = −2∫ x0e − ∫ z0∫2b(s) za(s) ds01a(y) e ∫ y02b(s)a(s) ds f(y) dy dz + c 1 (x − α) + c 2 . (2.6.7)In the previous equation the constant c 1 was integrated starting from α. This littletrick doesn’t change the result and simplifies the determination of the constantsfrom the boundary conditions. Then, for x = α we take the following:c 2 = 2∫ α0e − ∫ z0∫2b(s) za(s) dsFurthermore, for x = β, we find that:01a(y) e ∫ y02b(s)a(s) ds f(y) dy dz =: g(b). (2.6.8)c 1 =g(β) − g(α), (2.6.9)β − αwhere in generalg(x) = 2∫ x0e − ∫ z0∫2b(s) za(s) ds01a(y) e ∫ y02b(s)a(s) ds f(y) dy dz.The general verification theorems we had in Section 2.4 are still valid in this caseand we have that u(x) = U(x) = E x∫ τ0 f(X t) dt.The previous theorem shows the calculation of an explicit solution u of the ODE,corresponding to the first moment of the cost functional. What is left is to calculatethe expression for the second moment. As we explained in the previous sections,the first moment is integrated into the ODE for the second moment. We solve thisODE in the following theorem.

Chapter 2. Higher Moments of a non-controlled Markovian functional 62Theorem 2.6.2. Assuming that the ratio b(x)a(x)andf(x)a(x)are continuous and alsothat inf x a(x) > 0, there exists a unique solution v in C 2 (α, β) of the followingODE:with boundary condition:andb(x)v x + 1 2 a(x)v xx = −2f(x)u(x), (2.6.10)v| x=α,x=β = 0v(x) = V (x).Proof. Equation (2.6.10) can be solved explicitly in the same way as equation(2.6.3). We start by multiplying all terms with e∫ x02b(s)a(s) ds . Then, equation (2.6.10)becomes:b(x)e∫ x02b(s)a(s) ds v x + 1 2 a(x)e ∫ x02b(s)a(s) ds v xx = −e∫ x02b(s)a(s) ds u(x)f(x). (2.6.11)After dividing all terms by a(s), (2.6.11) can be written as follows:∂( 1∫∂x 2 e x0)2b(s)a(s) ds v x = − 1 ∫a(x) e x02b(s)a(s) ds u(x)f(x). (2.6.12)Integration of both parts in (2.6.12) and rearrangement gives the following:v x = −2e − ∫ x0∫2b(s) xa(s) ds01a(z) e ∫ z02b(s)a(s) ds u(z)f(z) dz + c 3 . (2.6.13)With one more integration of both parts, we take the following expression for v:v(x) = −2∫ x0e − ∫ g0∫2b(s) ga(s) ds01a(z) f(z)u(z)e ∫ z02b(s)a(s) ds dz dg + c 3 (x − a) + c 4 .(2.6.14)

Chapter 2. Higher Moments of a non-controlled Markovian functional 63Just like in the previous theorem, we have slightly modified the first constant withno loss of generality, in order to make easier the calculation of both constants.Thus, V (x) should satisfy the boundary conditions of (2.6.10) and thus for x = αwherec 4 = 2∫ α0e − ∫ g0h(x) = 2∫2b(s) ga(s) ds∫ x00e − ∫ g01a(z) f(z)u(z)e ∫ z0∫2b(s) ga(s) ds02b(s)a(s) ds dz dg =: h(α), (2.6.15)1a(z) f(z)u(z)e ∫ z02b(s)a(s) ds dz dg.Finally, by replacing x = b, the constant c 3 is calculated to be equal toc 3 =h(β) − h(α). (2.6.16)β − αAgain the proof is completed by using known verification theorems to show thatv(x) = E x∫ τ0 2f(X t)U(X t ) dt. Finally, we have seen in previous sections thatV (x) = E x∫ τ0 2f(X t)U(X t ) dt.Concluding this section, an explicit result was formulated for the second momentof a functional, when the stochastic process involved is one dimensional undernot very restricting assumptions.In general, the results discussed in sections1.5-1.7, provide the means to calculate that second moment, either numerically(since the existence and uniqueness is assured theoretically) or explicitly in theone dimensional case. The second moment is a very useful tool in describing adistribution as it is involved in the calculation of the Variance. In the next section,

Chapter 2. Higher Moments of a non-controlled Markovian functional 64three corollaries will be formulated in order to demonstrate a direct application ofthe previous theory.2.7 Higher momentsIn this section we illustrate how the previous results can be extended to higher thanthe second moments. Similarly to the transformation of the second moment, onecan take for the third moment (here with different notation, where the superscriptdenotes the moment v 3 ) the function:v 3 (t 0 , x) = E t0 ,xWe have, due to the Fubini theorem:( ∫ Tt 0) 3.f(s, X s ) ds(2.7.1)( ∫ T ) ( ∫ T ) ( ∫ Tv 3 (t 0 , x) = E t0 ,x f(t, X t ) ds f(s, X s ) dtt 0t 0t( 0∫ T ( ∫ T ∫ T)= 3E t0 ,x f(t, X t ) f(s, X s )( f(z, X z ) dz) ds dtt 0 ts( ∫ T( ∫ T∫ T= 3E t0 ,x f(t, X t )E t,Xt f(s, X s )E s,Xs (t 0( ∫ T∫ )T= 3E t0 ,x f(t, X t )E t,Xt f(s, X s )v 1 (s, X s ) dst 0( ∫ )T= 3E t0 ,x f(t, X t ) v 2 (t, X t ) dt ,t 0tt)f(z, X z ) dt)s))f(z, X z ) dz) ds dt

Chapter 2. Higher Moments of a non-controlled Markovian functional 65due to the Markov property, where∫ Tv 2 (t, x) := 2E t,x f(s, X s )v 1 (s, X s ) dstand∫ Tv 1 (t, x) := E t,x f(s, X s ) ds.tBy induction, it is clear that someone can express the m th moment as follows:v m (t 0 , x) = mE t0 ,x∫ Tt 0f(t, X t )v m−1 (t, X t ) dt. (2.7.2)Then, in complete accordance with the Dynkin’s chain of equation we obtain thefollowing theorem.Theorem 2.7.1. Let Assumptions A 2.3 hold. Then exists function V m , which is theunique solution in W 1,2 (Q) of the following PDE:d∑vt m + b i (t, x)vx m i+ 1 d∑a ij (t, x)vx m 2i x j= −mf(t, x)v m−1 (t, x), (2.7.3)i=1i,j=1with initial condition:v m | t=T = 0.The corresponding extension to higher moments for the exit time problem followssimilarly. In fact, the requirement that f is bounded is too strict and is in placeonly for simplification. The above theorem can be possibly be proved with f ∈ L pfor some large p, but this is in the future plans of the author.

66Chapter 3Controlling the second moment3.1 IntroductionIn this chapter, the problem of the optimal control of a Markov functional isdiscussed. By inserting control in the first and the second moment, the previousanalysis based on a system of equations is no longer applicable, at least for generalcontrol problems. However, control is very important for the purposes of this workas it will allow applications like mean-variance control and potentially meanvarianceoptimal stopping control. These will be developed in the next chapters.From this chapter and in the sequel, we abandon the exit time problem and thefocus will be on the finite horizon problem.Control techniques for diffusion processes has been a well developed and

Chapter 3. Controlling the second moment 67popular topic during the past decades. There are various approaches, includingmathematical dynamic programming and Bellman’s principle. In this work, weconsider a controlled diffusion and a controlled function f, which always leadto a well defined Bellman’s PDE and strong solutions to the relevant stochasticdifferential equation, similarly to the context of Krylov in [29]. In particular, theprocess that we consider has bounded and quite smooth coefficients (Lipschitz forexample) and some conditions (here bounded) apply to f as well. All the relevantassumptions will be summarised in the beginning of each section.Regarding the control of the first moment of Markov functionals, results have beenwell established through various approaches, like the Bellman’s PDE approach in[29] or the ”compactness-continuity” argument used for example in [3]. For thesecond moment however, a Bellman’s PDE approach seemed to be unfeasibledue to the nonlinear nature of the problem. However, the method followed heretranslates nonlinearity by means a transformation of the payoff function intodegeneracy of the Bellman’s PDE’s.As we have seen in the previous chapter, equations for the second moment canbe obtained in two ways.The first idea was to follow the Dynkin’s chain ofequations scheme (see [29]) for parabolic PDE’s, which leads to a system of twonon-degenerate partial differential equations. It is not immediate at least how

Chapter 3. Controlling the second moment 68to extend this idea to the controlled case. The problem arises from the fact thatsolving a HJB equation and obtaining a verification theorem would also provide anoptimal control as a byproduct. Then, solving two HJB’s would require to take thesupremum over all supremma for the available strategies. The second idea, whichis followed here mainly focuses in the reduction of the non-linear second momentproblem to a simpler, linear problem with a transformation of the functionaland the introduction of a new process with artificial initial data. Nevertheless,this procedure inserts a degeneracy into the problem, which fortunately can beovercome by a regularization method. The ”cost someone has to pay” for theregularisation of the value function is well bounded as it is proved in the sequel.Notice that in this chapter we follow the notation of section 2.7 in order to reservesome notation for the control variables.Before we start with the main results, we give a very simple example to acquaintthe reader with the notion of control. We must say that this well known problemhas been solved explicitly in many cases. Here, we do not attempt to solve it, butwe just formulate it in order to give a real example of control.

Chapter 3. Controlling the second moment 693.1.1 ExampleConsider a financial market with only two assets available: a risk free asset(usually a bond) and a risky one (a stock). We accept that the price of the bondevolves as:dp t = p t r dt, (3.1.1)where r is the risk free interest rate, considered as a constant for simplicity.The stock follows a geometric diffusion:dS = S t [µ dt + σ dW t ], (3.1.2)where W t is a one dimensional F t −Wiener process. Again µ and σ are taken asconstants.At time t, an investor will invest a proportion π t of his wealth in the stock and1 − π t in the bond, while he will be consuming at a rate C t . We can consider π tand C t as F t −adapted processes.Let X t be the investor’s wealth at time t. Then:[dX t = X t (1 − π t ) dp ]t dS t+ π t − C t dt, (3.1.3)p t S twith initial wealth X 0 = x. Equation (3.1.3) can be rewritten using (3.1.1) and(3.1.2) as

Chapter 3. Controlling the second moment 70dX t = X t [(1 − π t )r dt + π t (µ dt + σ dW t )] − C t dt. (3.1.4)Then the investor would like to maximise the following functional:v 1,π,C (t 0 , x) = E t0 ,x∫ Tt 0f πt,Ct (X πt,Ctt ) dt (3.1.5)over a period (t 0 , T ), where f could be a utility function that satisfies someconditions however.That means, choose the optimal consumption rates andinvestment weights so thatv 1 (t 0 , x) = sup v 1,π,C (t 0 , x). (3.1.6)π,CIf the couple π, C is denoted by the strategy α ∈ A with values u ∈ A ⊂ E l ,then it is well known (see [29, Chapters 3 and 4]) that this type of function is theunique solution of an equation of the type:or[ ]∂v1sup + L u tu∈A ∂t 0v 1 − f u = 0, (3.1.7)0[ ∂v1sup + x[(1 − π)r + πµ − c]vx 1 + 1 ]u∈A ∂t 0 2 xπ2 σ 2 vxx 1 − f u (t 0 , x) = 0, (3.1.8)with zero initial data∣v 1 ∣∣t0= 0.=T

Chapter 3. Controlling the second moment 713.2 Control of the first and second moment of aMarkov functional without final paymentConsider the following control problem for the first moment of a Markovfunctional:∫ Tv 1 (t 0 , x) = sup E t0 ,xα∈A t 0f(α t , t, Xt α ) dt, (3.2.1)where the diffusion process X t depends on the control strategy α ∈ A in thefollowing sense:or equivalentlyX α t = x +∫ tt 0b(α s , s, X α s ) ds +∫ tt 0σ(α s , s, X α s ) dW s , (3.2.2)dX t = b(α t , t, X α t ) dt + σ(α t , t, X α t ) dW t , t ≥ t 0 , X α t 0= x. (3.2.3)X x,αtis a solution of the above SDE under certain assumptions for some classesof control strategies.Throughout this section, we assume the following conditions (cf. [30, Chapter3]), which may be actually relaxed:(A 3.2 ) • The functions σ, b, f are Borel with respect to (u, t, x), continuouswith respect to (u, x) and continuous with respect to x uniformly overu for each t; moreover,

Chapter 3. Controlling the second moment 72• ‖σ(u, t, x) − σ(u, t, x ′ )‖ ≤ K‖x − x ′ ‖,• ‖b(u, t, x) − b(u, t, x ′ )‖ ≤ K‖x − x ′ ‖,• ‖σ(u, t, x)‖ + ‖b(u, t, x)‖ + |f(u, t, x)| ≤ K,• |f(u, t, x) − f(u, t, x ′ )| ≤ K ‖x − x ′ ‖.Remark 3.2.1. Note that because of the above standing assumptions and thedefinition of Admissible Strategies, if the control strategy α ∈ A then there existsa unique strong solution of (3.2.3). Furthermore, if α ∈ A M , under the aboveassumptions, equation (3.2.3) has a unique strong solution which is a strongMarkov process (see Veretennikov [44]).The above remark follows from the definition of the notion of strategies. In mostoccasions, the usual convention where the control variable will be denoted inthe expectation is followed. That means sup α∈A E α t 0 ,x∫ Tcompact equivalent notation for sup α∈A E t0 ,x t 0f(α t , t, Xt α ) dt.∫ Tt 0f(t, X t ) dt is a moreIt is well known (see Krylov) that problem (3.2.1) corresponds to the solution ofa Bellman’s Parabolic PDE of the form:(( ) ) ∂sup + L u v 1 − f u (t 0 , x) = 0, (3.2.4)u∈A ∂t 0with zero initial datav 1 (t 0 , x) ∣∣t0 =T= 0,

Chapter 3. Controlling the second moment 73whereL (u) (t 0 , x) = 1 2∑∂ 2a ij (u, t 0 , x) + ∑ ∂xi,ji ∂x jjb j (u, t 0 , x) ∂∂x j.Note that the solution exists and is unique (see for example [29]) for functionsin W 1,2 (with one generalised derivative with respect to time and two generalisedderivatives with respect to x) .Correspondingly, the second moment of the functional (3.2.1) can be written inthe following form:∫ Tv 2 (t 0 , x, 0) = sup Et α 0 ,x,0α∈At 02f(s, X s )Y s ds, (3.2.5)with the help of the auxiliary processdY t 0,x,αs= f(α s , s, X t 0,x,αs ) ds, s ≥ t 0 , Y t0 = 0. (3.2.6)In (3.2.5), a new index zero in the expectation E α t 0 ,x,0 stands for the initial valueof the process Y , that is, for Y t0 = 0. As usually with Bellman’s equations, it willbe useful to allow a variable initial data y for this new component,dY t 0,x,y,αs= f(α s , s, X t 0,x,αs ) ds, s ≥ t 0 , Y t0 = y. (3.2.7)Now the function in (3.2.5) can be regarded as a value function for the controlledextended Markov diffusion (X, Y ),∫ Tv 2 (t 0 , x, y) = sup Et α 0 ,x,y 2f(s, X s )Y s ds,α∈At 0

Chapter 3. Controlling the second moment 74at y = 0. In this case the corresponding PDE is degenerate with zero initial data.[sup vt 2 +u∈Ad∑b i (u, t, x)vx 2 i+ 1 2i=1d∑i=1d∑j=1]a ij (u, t, x)vx 2 i x j+f(u, t, x)vy+2f(u, 2 t, x)y = 0v 2 (t, x, y)= 0.∣t=T(3.2.8)In the absence of control (see chapter 2), this equation has a unique solution inW 1,2 (it can be expressed as a system of 2 equations), however in the controlledcase, the degeneracy of v 2 does not allow for solutions even with generalisedderivatives (see [29]).In order to simplify the resulting equation and restorenon-degeneracy, a small constant diffusion coefficient is added to the degenerateprocess (3.2.6). Then the process Y x,ytlatter becomesis replaced by the process Y x,y,εtso that the∫ tY x,y,ε,αt = y + f(α s , s, X s ) ds + ε( ˜W t − ˜W t0 ),t 0(3.2.9)where ˜W t is a new Wiener process independent of W t . It will be shown thatwith a proper choice of the constant ε the regularised value function v 2,ε (whichis a solution to a PDE) is close to the original degenerate value function (noticethat this value function does not normally correspond to a solution of a PDE asexplained before) and their difference does not exceed a constant depending onε. Note that equation (3.2.10) below, for the ε-case, is a non-degenerate Parabolic

Chapter 3. Controlling the second moment 75PDE and it is now easier to prove existence and uniqueness and potentially tosolve the problem numerically.[sup v 2,εt +u∈Ad∑i=1b i (u, t, x)v 2,εx i+ 1 2+f(u, t, x)v 2,εyd∑i=1d∑j=1a ij (u, t, x)v 2,εx i x j+ ε 2 v2,ε yy + 2f(u, t, x)y]= 0, (3.2.10)v 2,ε ∣ ∣∣∣∣t=T= 0.Lemma 3.2.1. The new processˆX t =⎡⎢⎣ X tY εt⎤⎥⎦is non-degenerate.Proof. It is enough to examine the matrix a of that process and check that it isindeed positive definite. The new diffusion matrix is⎡⎤σ 11 (u, t, x) σ 12 (u, t, x) ... σ 1d1 (u, t, x) 0σ 21 (u, t, x) σ 22 (u, t, x) ... σ 2d1 (u, t, x) 0˜σ(u, t, x) =⎢ σ d1 (u, t, x) σ d2 (u, t, x) ... σ dd1 (u, t, x) 0 ⎥⎣⎦0 0 ... 0 ε

Chapter 3. Controlling the second moment 76and the matrix a = ˜σ˜σ ∗ is⎛a(u, t, x) =⎜⎝∑ d1j=1 σ2 1jTherefore, the scalar product⎞∑ d1j=1 σ ∑ d11jσ 2j j=1 σ ∑1jσ 3j ...d1j=1 σ 1jσ dj 0∑ d1j=1 σ 2jσ dj 0... ... ... ... ... ...∑ d1j=1 σ2 dj 0⎟⎠0 0 0 0 0 ε 2∑ d1i=1 σ i1σ i2∑ d1j=1 σ2 2j ... ...∑ d1i=1 σ i1σ id ... ... ...[] Λ = λ 1 λ 2 ... λ d λ d+1× a ×⎢⎣=d∑κ=1λ k⎡⎤λ 1λ 2...λ d⎥⎦λ d+1d∑a κj λ j + λ 2 d+1ε 2 > 0.j=1Note that the matrix a included here is the product a(u, t, x) =σ(u, t, x)σ ∗ (u, t, x).Theorem 3.2.1. Let conditions A 3.2 hold. Then, the function v 2,ε is the uniquesolution of the equation (3.2.10) in the class of functions W 1,2,2 . Moreover, forC ≥ 0, independent of (x, y),sup ∣ ( v 2,ε − v 2) (t 0 , x, y) ∣ ≤ Cε.t 0 ,x,y

Chapter 3. Controlling the second moment 77Proof. The fact that v 2,ε is a unique solution of (3.2.10) follows, e.g., from ([29])similarly to the first moment. To show the second assertion we estimate:= sup≤v 2,ε (t 0 , x, y) − v 2 (t 0 , x, y)Et α 0 ,x,yα∈Asupα∈A(E α t 0 ,x,y= sup Et α 0 ,x,yα∈A= sup Et α 0 ,x,yα∈A∫ Tt 0∫ Tt 0∫ Tt 0∫ Tt 02f(t, X t )Yt ε dt − sup∫ TEt α 0 ,x,yα∈At 0∫ T2f(t, X t )Y εt dt − E α t 0 ,x,y2f(t, X t ) ( Y εt− Y t)dt2f(t, X t )ε( ˜W t − ˜W t0 ) dtt 02f(t, X t )Y t dt)2f(t, X t )Y t dt≤ 2‖f‖ B sup E| ˜W s − ˜W t0 |(T − t 0 ) ε ≤ 2‖f‖ B (T − t 0 ) 3 2 E| ˜W1 | εt 0 ≤s≤T= 2‖f‖ B (T − t 0 ) 3 2√2πε, (3.2.11)becauseE| ˜W 1 | == 2∫ ∞−∞∫ ∞01|x| √ e − x22 dx 2πx 1 √2πe − x22 dx=√2π . (3.2.12)Similarly we obtain a lower boundv 2,ε − v 2 ≥ −2‖f‖ B (T − t 0 ) 3 2√2π ε.Note that although in the previous chapter we required that the cost function(running cost) f(t, X t ) is bounded, this is not true here.In fact, our ”new”

Chapter 3. Controlling the second moment 78modified cost function f(t, X t )Y t grows linearly in y. However, this cannot affectthe result, since the theory we invoke (see [29]) for existence and uniquenessallows even a polynomial growth to the cost function.3.2.1 ExtensionsAlthough we restrict ourselves to the case of bounded function f, more generalcases are available. Here, we give an idea of how the above lemmas could bemodified. We do not mention anything about the HJB PDE’s however.First, we can allow to the function f to grow polynomially with x (i.e. |f(t, x)| ≤C(1 + |x| m )) , so that the norm‖f‖ polm := supt,xf(t, x)∣(1 + |x| m ) ∣ < ∞ (3.2.13)is bounded. In this case, the assertion of Theorem 3.2.1 will take the form:Lemma 3.2.2. If |f(x)| ≤ C(1 + |x| m ), C > 0 then sup t0 ,x,y |v 2,ε (t 0 , x, y) −v 2 (t 0 , x, y)| ≤ cε, c ≥ 0, where c = 2‖f‖ polm (T − t 0 ) 3 2 C T (x).Proof. We estimate:|v 2,ε (t 0 , x, y) − v 2 (t 0 , x, y)| == | sup Et α 0 ,x,yα≤sup |Et α 0 ,x,yα∫ Tt 0∫ Tt 02f(t, X t )Y t dt − sup Et α 0 ,x,yα2f(t, X t )Y εt dt − E α t 0 ,x,y∫ Tt 0∫ Tt 02f(t, X t )Y εt dt|2f(t, X t )Y t dt|

Chapter 3. Controlling the second moment 79∫ T= sup |Et α 0 ,x,yα≤≤sup Et α 0 ,x,yα2‖f‖ polm εt 0∫ Tt 0∫ Tt 0∫ T2f(t, X t ) ( )Y t − Ytε dt|2|f(t, X t )|ε| ˜W t − ˜W t0 | dtE(1 + |X x t | m )| ˜W t − ˜W t0 | dt(≤ 2‖f‖ polm ε C T (x) 1 2 E | ˜W t − ˜W) 1t0 | 2 2dtt 0= 2‖f‖ polm ε(T − t 0 ) 3 2 CT (x), (3.2.14)because sup t≤T E|X x t | 2m ≤ C T (x) see for example [29, Chapter 2].An alternative would be to assume f ∈ (L p ) ⋂ (L 2p )p ≥ d + 1. In that caselemma 3.2.2 can be reproduced by just replacing the norm ‖f‖ B by the relevantL p norms.3.3 Control of the first and second moment of aMarkov functional with final paymentIn this case the cost functional consists of a final payment additionally to therunning cost functional and the value function for the first moment has the form:[ ∫ T]ṽ 1 (t 0 , x) = sup Et α 0 ,xα∈A t 0f(t, X t ) dt + Φ(X T ) . (3.3.1)

Chapter 3. Controlling the second moment 80The Bellman’s PDE for ṽ 1 is the following:with initial data(sup Luṽ 1 − f u) = 0, (3.3.2)u∈Aṽ 1 ∣ ∣∣t0=T= Φ(x).Then, the value function for the second moment is[∫ T] 2ṽ 2 (t 0 , x) = sup Et α 0 ,x f(t, X t ) dt + Φ(X T ) . (3.3.3)α∈At 0Throughout this section, we assume the following:A 3.3 • The functions σ, β, c, f are continuous with respect (u, x) andcontinuous with respect to x uniformly over u for each t.• ‖σ(u, t, x) − σ(u, t, x ′ )‖ ≤ K‖x − x ′ ‖.• ‖b(u, t, x) − b(u, t, x ′ )‖ ≤ K‖x − x ′ ‖.• ‖σ(u, t, x)‖ + ‖b(u, t, x)‖ + |f(u, t, x)| ≤ K.• |f(u, t, x) − f(u, t, x ′ )| ≤ K‖x − x ′ ‖.• Φ(x) ∈ C 2 b .We need to assume that Φ is twice continuously bounded differentiable so thatItô’s formula is applicable to it and to preserve the boundness of the value

Chapter 3. Controlling the second moment 81function. Itô’s formula will be used in the transformation of the second momentin a similar way to what we did in the case that there was no final payment. Wedo the following calculations without any control just for simplicity. Introductionof control will not alter the result because of our assumptions on the diffusioncoefficients. Then:Φ(XT x ) − Φ(Xt x ) =+∫ T[t i=1∫ Tt i=1d∑b i (s, Xs x )Φ xi (Xs x ) + 1 d∑ d∑a(s, Xs x )Φ xi x2j(Xs x )] dsi=1 j=1(d∑ ∑ d1)σ ij (s, Xs x ) Φ xi (Xs x ) dW s (3.3.4)j=1⇔ Φ(XT x ) = Φ(Xt x ) +where∫ TΦ 1 (s, X x s ) ds +∫ TttΦ 2 (s, X x s ) dW s ,Φ 1 (t, x) =d∑b i (t, x)Φ xi (x) + 1 2i=1d∑ d∑a(t, x)Φ xi x j(x) (3.3.5)i=1 j=1andΦ 2 (t, x) =d∑(d1i=1 j=1)∑σ ij (t, x) Φ xi (x). (3.3.6)For simplicity, we write down the second moment (3.3.3) without control.[∫ T] 2V (t 0 , x) = E t0 ,x f(t, X t ) dt + Φ(X T )t 0(∫ T) 2= E t0 ,x (Φ(X T )) 2 + E t0 ,x f(t, X t ) dtt(0∫ T)+ 2E t0 ,x Φ(X T ) f(t, X t ) dt . (3.3.7)t 0

Chapter 3. Controlling the second moment 82The second term from the right hand side of equation (3.3.7) is the second momentof the cost function without final payment. The third term, however, can be writtenas==2E t0 ,x∫ TE t0 ,xt 0∫ Tt 0E t0 ,x( ∫ T) ∫ TΦ(X T ) f(t, X t ) dt = E t0 ,x Φ(X T )2f(t, X t ) dtt 0t 0(Φ(XT )2f(t, X t ) ) dt(2f(t, X t ) ( ∫ T∫ T) )Φ(X t ) + Φ 1 (s, X s ) ds + Φ 2 (s, X s ) dW s dtttbecause of Fubini theorem and of equation (3.3.4).So,=2E t0 ,x∫ Tt 0E t0 ,x( ∫ T )Φ(X T ) f(t, X t ) dt =t 0(2f(t, Xt )Φ(X t ) ) ∫ Tdt +t 0E t0 ,x(2f(t, X t )∫ Tt)Φ 1 (s, X s ) ds dt+∫ T= E t0 ,x+E t0 ,xt 0∫ T∫ TE t0 ,xt 0∫ T( ∫ T)2f(t, X t ) Φ 2 (s, X s ) dW s dtt∫ T∫ T2f(t, X t )Φ(X t ) dt + E t0 ,x 2f(t, X t ) Φ 1 (s, X s ) ds dtt 0 t 0 t( ∫ T))(E t,Xt 2f(t, X t ) Φ 2 (s, X s ) dW s dtt= E t0 ,x 2f(t, X t )Φ(X t ) dt + E t0 ,x 2f(t, X t ) Φ 1 (s, X s ) ds dtt 0 t 0 t∫ T(∫ T))+ E t0 ,x2f(t, X t )(E t,Xt Φ 2 (s, X s ) dW s dt. (3.3.8)t 0t∫ T∫ T

Chapter 3. Controlling the second moment 83In equation (3.3.8), the expectation of the stochastic integral equals to zero since,because of the assumptions A 3.3 , all the quantities involved are bounded and so∫ Tt(Φ 2 (s, X s ) ) 2ds < ∞. Therefore,2E t0 ,x= E t0 ,x( ∫ TΦ(X T )∫ Tt 0)f(t, X t ) dt = (3.3.9)t 02f(t, X t )Φ(X t ) dt + E t0 ,x∫ Tt 02f(t, X t )∫ TtΦ 1 (s, X s ) ds dt.Equation (3.3.7) can then be written as:[∫ T] 2V (t 0 , x) = E t0 ,x f(t, X t ) dt + Φ(X T )t 0= E t0 ,x(Φ(XT ) ) (∫ T) 22+ Et0 ,x f(t, X t ) dt+ E t0 ,x∫ T= E t0 ,x(Φ(XT ) ) 2+ Et0 ,x∫ Tt 02f(t, X t )Φ(X t ) dt + E t0 ,x∫ Tt 0t 02f(t, X t )∫ Tt∫ T∫ Tt 02f(t, X t )f(s, X s ) ds dt∫ TtΦ 1 (s, X s ) ds dt+ E t0 ,x 2f(t, X t )Φ(X t ) dt + E t0 ,x 2f(t, X t ) Φ 1 (s, X s ) ds dtt 0 t 0 t(= E t0 ,x Φ(XT ) ) 2(3.3.10)∫ T( ∫ T∫ T)+ E t0 ,x 2f(t, X t ) Φ(X t ) + f(s, X s ) ds + Φ 1 (s, X s ) ds dt.t 0 ttUsing Fubibi theorem, tower property of expectation and the Markov property ofX t as we did before, the above expression can be written as:V (t 0 , x) = E t0 ,x(Φ(XT ) ) 2+ Et0 ,x∫ T∫ Tt 02f(t, X t )V ′ (t, X t ) dt, (3.3.11)

Chapter 3. Controlling the second moment 84where∫ T(V ′ (t, X t ) = Φ(X t ) + E t,Xt f(s, Xs ) + Φ 1 (s, X s ) ) ds. (3.3.12)tThe above form is linked directly to the system of equations examined in theprevious chapter. However, as we explained in Section 3.1, in the controlled case,which we examine here, there are additional difficulties. Therefore, we would liketo express the second moment as the first moment of a functional of an extendedMarkov process similarly to the case without final payment.For the latter, we can proceed from (3.3.10) as follows:(V (t 0 , x) = E t0 ,x Φ(XT ) ) 2∫ T+ E t0 ,x 2f(t, X t )t( 0= E t0 ,x+∫ T= E t0 ,xΦ(X T ) 22f(t, X t )Φ(X t ) dt +t 0((Φ(X T ) 2 +( ∫ T∫ T)Φ(X t ) + f(s, X s ) ds + Φ 1 (s, X s ) ds dttt∫ T∫ Tt 02f(t, X t )t 0g(t, X t ) dt +∫ T∫ T∫ Ttt 02f(t, X t )∫ T(f(s, Xs ) + Φ 1 (s, X s ) ) ds dt∫ Tt∫ t˜Φ 1 (s, X s ) ds dt= E t0 ,x Φ(X T ) 2 + g(t, X t ) dt + ˜Φ1 (t, X t ) 2f(s, X s ) ds dtt 0 t 0 t 0( ∫ )T (= E t0 ,x Φ(X T ) 2 + g(t, X t ) + ˜Φ)1 (t, X t )Ỹt dt , (3.3.13)t 0)))

Chapter 3. Controlling the second moment 85whereg(t, X x t ) = 2f(t, X x t )Φ(X x t )˜Φ 1 (t, X x t ) = f(t, X x t ) + Φ 1 (t, X x t )∫ tỸt x = 2f(t, Xt x ) dt.t 0We have the following two equivalent representations for the controlled secondmoment (3.3.3):( (Φ(XTṽ 2 (t 0 , x) = sup Et α 0 ,x ) ) ∫ T)2+ 2f(t, X t )V ′ (t, X t ) dtα∈At 0(3.3.14)and( (Φ(XTṽ 2 (t 0 , x) = sup Et α 0 ,x ) ) ∫ T (2+ g(t, X t ) + ˜Φ(t,) )X t )Ỹt dt .α∈At 0(3.3.15)As usually, we will allow to Ỹ xtwe will finally set y = 0. So, we can writeandto depend on its own initial data y ∈ R, for which∫ tỸ x,αt = y + 2f(α s , s, Xs x,α ) dst 0(3.3.16)( (Φ(XTṽ 2 (t 0 , x, y) = sup Et α 0 ,x,y ) ) ∫ T (2+ g(t, X t ) + ˜Φ(t,) )X t )Ỹt dt .α∈At 0(3.3.17)

Chapter 3. Controlling the second moment 86Now, the goal would be to show that ṽ 2 is the solution of a Bellman’s equation ofthe following type:[ d∑sup ṽt 2 + b i (u, t, x)ṽx 2 i+ 1 d∑ d∑a ij (u, t, x)ṽx 2u∈A2i x ji=1i=1 j=1+ 2f(u, t, x)ṽy 2 + g(u, t, x) + y ˜Φ]1 (u, t, x) = 0, (3.3.18)but equation (3.3.18) is degenerate and thus we cannot claim even existenceof a solution.Therefore, we again try to regularise the process Ỹ x,yt . Thecorresponding modified for non-degeneracy process is:and∫ tỸ x,y,ε,αt = y + 2f(α s , s, Xs x,α ) ds + ε ˜W t−t0t 0(3.3.19)( (Φ(XTṽ 2,ε (t 0 , x, y) = sup Et α 0 ,x,y ) ) ∫ T (2+ g(t, X t ) + ˜Φ(t,)X t )Ỹ tεα∈At 0)dt .(3.3.20)The resulting Bellman’s PDE which describe the dynamics of ṽ 2,ε is the following:[sup ṽ 2,εt +u∈Ad∑i=1+ 2f(u, t, x)ṽ 2,εyb i (u, t, x)ṽ 2,εx i+ 1 2d∑i=1d∑j=1+ ε 2ṽ2,εyy + g(u, t, x) + y˜Φ 1 (u, t, x)a ij (u, t, x)ṽ 2,εx i x j(3.3.21)]= 0,with initial condition∣ṽ 2,ε ∣∣t0= Φ(x) 2 .=T

Chapter 3. Controlling the second moment 87Lemma 3.3.1. The new processis non-degenerate.⎡ˆX x,y ⎢t = ⎣Xx tỸ x,y,εt⎤⎥⎦Proof. It is enough to examine the new diffusion coefficient of that process andcheck that it is indeed positive definite. The new matrix is⎡σ 11 (u, t, x) σ 12 (u, t, x) ... σ 1d1 (u, t, x) 0σ 21 (u, t, x) σ 22 (u, t, x) ... σ 2d1 (u, t, x) 0˜σ(u, t, x) =⎢ σ d1 (u, t, x) σ d2 (u, t, x) ... σ dd1 (u, t, x) 0⎣0 0 ... 0 ε⎤⎥⎦and the matrix ã = ˜σ˜σ ∗ is⎛ã(α, t, x) =⎜⎝⎞∑ d1j=1 σ ∑1jσ 2j ...d1j=1 σ 1jσ dj 0∑ d1j=1 σ 2jσ dj 2... ... ... ... ...∑ d1j=1 σ2 dj 0⎟⎠0 0 ... 0 ε 2∑ d1j=1 σ2 1j∑ d1i=1 σ i1σ i2∑ d1j=1 σ2 2j ...∑ d1i=1 σ i1σ id ... ...

Chapter 3. Controlling the second moment 88Therefore, for vector λ : λ|λ| = 1 the scalar product Therefore, the scalar product⎡ ⎤λ 1[] λ 2Λ = λ 1 λ 2 ... λ d λ d+1× ã ×...⎢ λ d⎥⎣ ⎦λ d+1=d∑λ kκ=1 j=1d∑ã κj λ j + λ 2 d+1ε 2 > 0.Note that the matrix ã included here is the product ã(α, t, x) =σ(u, t, x)σ ∗ (u, t, x) and is by assumption positive definite.Then, finally, following the same steps as in the previous section and since thefunction f is bounded, it can be proved that=|ṽ 2,ε − ṽ 2 |∣ supα∈AE α t 0 ,x,y( ∫ T (Φ 2 (X T ) + g(t, X t ) + ˜Φ)1 (t, X t )Ỹ tεt 0)dt

Chapter 3. Controlling the second moment 89−≤−( ∫ Tsup Et α 0 ,x,y Φ 2 (X T ) +α∈A∣∣sup∣Et α 0 ,x,yα∈AE α t 0 ,x,y∣= sup(Φ 2 (X T ) +( ∫ TΦ 2 (X T ) +∫ T∣E t0 ,x,yα∈At 0∫ T∣∣E t0 ,x,yα∈At 0= sup≤‖˜Φ 1 (t, X t )‖ B εE∣t 0(t 0(∫ Tt 0(g(t, X t ) + ˜Φ) ∣∣∣1 (t, X t )Ỹt dt)g(t, X t ) + ˜Φ) )1 (t, X t )Ỹ ε dtg(t, X t ) + ˜Φ) ∣∣∣1 (t, X t )Ỹt dt)˜Φ1 (t, X t ) ( Ỹ εt − Ỹt)dt∣ ∣∣˜Φ1 (t, X t )ε( ˜W t − ˜W t0 ) dt∣∫ Tt 0˜Ws ds∣ ≤ ‖˜Φ 1 (t, X t )‖ B ε(T − t 0 ) 3 2 E| ˜W1 |= ‖˜Φ 1 (t, X t )‖ B ε(T − t 0 ) 3 2 c1 , (3.3.22)twhere c 1 has been calculated in (3.2.12).Theorem 3.3.1. Under the assumptions A 3.3 , equation (3.3.21) has a uniquesolution ṽ 2,ε in the class of functions W 1,2,2 . Moreover, there exists C ≥ 0,independent of (x, y), such that:supt,x,y∣ ( ṽ 2,ε − ṽ 2) (t, x, y) ∣ ∣ ≤ Cε.Proof. Due to the above bound and similar to the one in the previous section.

90Chapter 4Mean-Variance Optimization4.1 IntroductionIn this chapter, we present the results on the optimal control of a linearcombination of mean and variance of a Markov functional with and without finalpayment. The notion of control remains quite general. For instance, it could reflectthe weights allocated to each component (dimension) of the process. That, in afinancial application, would represent the weights of the wealth an investor placeson each of the d assets that compose its portfolio. This reminds automatically theMarkowitz Optimisation concept. Indeed, the expression is the following with Zrepresenting the integrated cost function:supα∈A{E α (Z) − θV ar α (Z)} = sup{E α (Z) − θE α (Z 2 ) + θ ( E α (Z) ) 2}, (4.1.1)α∈A

Chapter 4. Mean-Variance Optimization 91where θ is normally a positive constant (since in most cases we are interested inminimising risk). However, this is not a restriction as it can take negative valuesas well.Apparently, the above linear combination is not very meaningful in terms of units.It is however the existence (and the selection) of an optimal control that someonewould be looking for, rather than a numerical result of that combination. Forexample, when a ”Markowitz’s type” optimal portfolio is constructed, the investoris not interested in the numerical result of this combination, but in the optimalweights that maximise the expected return and minimise risk simultaneously.In the results that follow, no reference is made on the construction of optimalcontrol strategy and the class of strategies required. Initially, the analysis bellowcan be done for any admissible control strategy α ∈ A. However, the fact that aBellman’s equation is solved for the optimal control problem is enough to ensuresufficiency of Markovian strategies (see [29]). Construction of such strategies isalso standard (see [29, Chapter 5]).

Chapter 4. Mean-Variance Optimization 924.2 Without final paymentThroughout this section, we make the following assumptions which may beactually relaxed:(A 4.2 ) • The functions σ, b, f are Borel with respect to (u, t, x), continuouswith respect to (u, x) and continuous with respect to x uniformly overu for each t; moreover,• ‖σ(u, t, x) − σ(u, t, x ′ )‖ ≤ K‖x − x ′ ‖.• ‖b(u, t, x) − b(u, t, x ′ )‖ ≤ K‖x − x ′ ‖.• ‖σ(u, t, x)‖ + ‖b(u, t, x)‖ + |f(u, t, x)| ≤ K.• |f(u, t, x) − f(u, t, x ′ )| ≤ K ‖x − x ′ ‖.Along with the function v 1 – see (3.2.1) in the previous chapter – consider thefollowing function to represent the linear combination (4.1.1):{v ε (t 0 , x, y) := sup v 1,α (t 0 , x) − θv 2,ε,α (t 0 , x, y) + θ (v 1,α (t 0 , x)) 2} . (4.2.1)α∈ANotice the change in the notation. Now the supremum is reserved for the mainvalue function v ε (t, x, y) and naturally the moments depend on the controls so

Chapter 4. Mean-Variance Optimization 93thatv 1,α (t 0 , x) := E α t 0 ,x∫ Tt 0f(t, X t ) dt, (4.2.2)v 2,ε,α (t 0 , x, y) := E α t 0 ,x,yLater, we will also need the notationand the correspondingv 2,α (t 0 , x, y) := E α t 0 ,x,y∫ Tt 0∫ T2f(t, X t )Y εt dt. (4.2.3)t 02f(t, X t )Y t dt (4.2.4){v(t 0 , x, y) := sup v 1,α (t 0 , x) − θv 2,α (t 0 , x, y) + θ (v 1,α (t 0 , x)) 2} . (4.2.5)α∈AThe key point in the representation is that the term (v 1,α ) 2 (t, x) in (4.2.1) may berepresented in the form:(v 1,α ) 2 = sup{−ψ − 2ψv 1,α }, ψ ∈ R. (4.2.6)ψNotice that the expression −ψ 2 − 2ψv 1,α attains its supremum with respect to ψfor ψ = −v 1,α and the supremum equals (v 1,α ) 2 indeed. Thus, the optimizationproblem (4.1.1) can be rewritten as:[v ε (t 0 , x, y) = sup sup v 1,α (t 0 , x) + θ[−ψ 2 − 2ψv 1,α (t 0 , x)] − θv 2,ε,α (t 0 , x, y) ]α∈A ψ= sup supα∈A ψ[v 1,α (t 0 , x)[1 − 2θψ] − θψ 2 − θv 2,ε,α (t 0 , x, y) ] . (4.2.7)

Chapter 4. Mean-Variance Optimization 94It is possible to change the order of the optimization in the following way:[v ε (t 0 , x, y) = supψsupα∈Awhere we will be finally interested in y = 0.(v 1,α (t 0 , x)[1 − 2θψ] − θv 2,ε,α (t 0 , x, y) ) − θψ 2 ],(4.2.8)Given ψ, denote:and let(V 1,ε (t 0 , x, y, ψ) := sup v 1,α (t 0 , x)[1 − 2θψ] − θv 2,ε,α (t 0 , x, y) ) , (4.2.9)α∈A(V 1 (t 0 , x, y, ψ) := sup v 1,α (t 0 , x)[1 − 2θψ] − θv 2,α (t 0 , x, y) ) (4.2.10)α∈AL (u,ε) := 1 ∑∂ 2a ij (u, t, x) + ∑ 2∂x i ∂x jji,jb j (u, t, x) ∂∂x j+ ε22 ∆ yy + f(u, t, x) ∂ ∂y .The function in the left hand side of (4.2.9) is the unique solution of the BellmanPDE:[( )]∂supu∈A ∂t + L(u,ε) V 1,ε (t, x, y, ψ) + [1 − 2θψ + 2θy]f(u, t, x) = 0,V 1,ε ∣ ∣∣∣∣t=T= 0.(4.2.11)Under our standing assumptions above, we have existence and uniqueness ofsolution of the equation (4.2.11), due to [29, Chapters 3 and 4]. Now, the external

Chapter 4. Mean-Variance Optimization 95optimization problem (with respect to ψ) takes the form:[v ε (t 0 , x, y) = sup V 1,ε (t 0 , x, y, ψ) − θψ 2] . (4.2.12)ψObviously, this is not just a simple quadratic equation with respect to ψ, since thefunction V 1,ε depends on ψ. However, the function V 1,ε , being locally Lipschitzin ψ, grows in this variable at most linearly, as shown in the next Lemma below.Hence, the supremum is attained at some ψ from a certain bounded interval.Lemma 4.2.1. The functions V 1 (t 0 , x, y, ψ) and V 1,ε (t 0 , x, y, ψ) satisfy thebounds:|V 1,ε (t 0 , x, y, ψ)| + |V 1 (t 0 , x, y, ψ)| ≤ C (1 + |ψ|),andmax ( |V 1 (t 0 , x, y, ψ) − V 1 (t 0 , x, y, ψ ′ )|, |V 1,ε (t 0 , x, y, ψ) − V 1,ε (t 0 , x, y, ψ ′ )| )≤ C |ψ − ψ ′ |,for some C > 0.Proof. The first inequality of the lemma (linear growth with respect to ψ) followsfrom the definitions (4.2.9) and (4.2.10), because all three functions v 2,ε,α , v 2,αand v 1,α are bounded. Let us show that the function V 1,ε is locally Lipschitz with

Chapter 4. Mean-Variance Optimization 96respect to ψ. Then, the function V 1 may be considered similarly. We estimate:V 1,ε (t 0 , x, y, ψ) − V 1,ε (t 0 , x, y, ψ ′ )(= sup v 1,α (t 0 , x)[1 − 2θψ] − θv 2,ε,α (t 0 , x, y) )α∈A−(sup v 1,α (t 0 , x)[1 − 2θψ ′ ] − θv 2,ε,α (t 0 , x, y) )α∈A≤2θ sup ∣ v 1,α (t 0 , x) ∣ |ψ ′ − ψ| ≤ C θ |ψ ′ − ψ|.α∈AA lower bound follows similarly, which proves the Lemma 4.2.1.Corollary 4.2.1. Any supremum in (4.2.12) is attained at λ − ≤ |ψ| ≤ λ + , whereλ ± = C ± √ C 2 + θC(2 − C/4θ).2θC > 0 is the constant from the second inequality of the Lemma 4.2.1.Proof. Consider the function G(ψ) := V 1,ε (t 0 , x, y, ψ) − θψ 2 . We have |V 1,ε | ≤C(1 + |ψ|). So, G(ψ) → −∞ when |ψ| → ∞ and G is also continuous. Hence,exists ¯ψ ∈ R such thatThen[sup V 1,ε (t 0 , x, y, ψ) − θψ 2] = V 1,ε (t 0 , x, y, ¯ψ) − θ ¯ψ 2 .ψ−C ( 1 + | ¯ψ| ) − θ ¯ψ 2 ≤ V 1,ε (t 0 , x, y, ¯ψ) − θ ¯ψ 2 ≤ C ( 1 + | ¯ψ| ) − θ ¯ψ 2 .

Chapter 4. Mean-Variance Optimization 97In order to find the interval in which the supremum is attained, we first find themaximum value of the lower parabola and this is C24θ − C. Hence, max ψV 1,ε ≥C 24θ− C and therefore it cannot be attained outside the interval where the upperparabola exceeds this value. So, we solve the inequality − ( θ| ¯ψ| 2 − C| ¯ψ| − C ) ≥−C + C24θ . Hence λ − ≤ | ¯ψ| ≤ λ + .Theorem 4.2.1. The approximate cost function v ε satisfies (4.2.12), and exists aconstant C ≥ 0, independent of (x, y), such thatsup |v ε (t 0 , x, y) − v(t 0 , x, y)| ≤ C θ ε. (4.2.13)t 0 ,x,yProof. Only (4.2.13) needs to be established. We have, due to the calculus in(3.2.11),≤v ε (t 0 , x, y) − v(t 0 , x, y)(sup V 1,ε (t 0 , x, y, ψ) − θψ 2 − V 1 (t 0 , x, y, ψ) + θψ 2)ψ(= supψsupα∈A(v 1,α (t 0 , x)[1 − 2θψ] − θv 2,ε,α (t 0 , x, y) )(− sup v 1,α (t 0 , x)[1 − 2θψ] − θv 2,α (t 0 , x, y) ))α∈A≤supψ(sup v 1,α (t 0 , x)[1 − 2θψ] − v 1,α (t 0 , x)[1 − 2θψ]α∈A+θ ( v 2,α (t 0 , x, y) − v 2,ε,α (t 0 , x, y) ))(= θ sup v 2,α (t 0 , x, y) − v 2,ε,α (t 0 , x, y) ) ≤ θ C ε.α∈A

Chapter 4. Mean-Variance Optimization 98The lower bound follows similarly. The Theorem 4.2.1 is proved.Remarks:Remark 4.2.1. Notice that the control variable ψ is one-dimensional, whichfacilitates numerical approximation results.Remark 4.2.2. Let ψ take values in I = [λ ∗ , λ ∗ ], λ ∗= |λ − | ∨ |λ + |. We candiscretise the interval I by dividing it into k ∈ Z subintervals of length 2λ∗k , sothat ψ takes discrete values [kψ]kin the interval I k = [0, ± 2λ∗k , ± 4λ∗k , . . . , ±λ ∗].Then, we define:v ε,k (t 0 , x, y, ψ) = supψ∈I k [V 1 (t 0 , x, y, ψ) − θψ 2] ≤ supψ∈I k [1 + |ψ| − θψ 2 ]. (4.2.14)Since v ε,k (t 0 , x, y) is Lipschitz continuous in ψ, then v ε,k (t 0 , x, y) → v ε (t 0 , x, y)as k → ∞.Proof. We can write:∣∣v ε,k (t 0 , x, y) − v ε (t 0 , x, y) ∣ [ ] [= ∣ sup V 1,ε (t 0 , x, y, ψ n ) − θψn2 − sup V 1,ε (t 0 , x, y, ψ) − θψ 2] ∣ ∣ψ n=2nc/k−∞

Chapter 4. Mean-Variance Optimization 99say at ¯ψ. Then:∣ v ε,k (t 0 , x, y) − v ε (t 0 , x, y) ∣ ∣= V 1,ε (t 0 , x, y, ¯ψ) − θ ¯ψ[ ] 2 − sup V 1,ε (t 0 , x, y, ψ n ) − θψn2 .ψ n=2nc/k(4.2.16)Now, let ¯ψ n be the closest from the left discrete value of ψ n to ¯ψ. We have:0 ≤ ∣ ∣ v ε,k (t 0 , x, y) − v ε (t 0 , x, y) ∣ ∣= V 1,ε (t 0 , x, y, ¯ψ) − θ ¯ψ 2 − V 1,ε (t 0 , x, y, ¯ψ n ) − θ ¯ψ 2 n≤ Cθ ( ( ¯ψ − ¯ψ n ) − ( ¯ψ 2 − ¯ψ 2 n) )= Cθ ( ( ¯ψ − ¯ψ n ) − ( ¯ψ − ¯ψ n )( ¯ψ + ¯ψ n ) )( 2λ∗≤ Cθk− 2λ )∗k (2λ ∗)= Cθ 2λ ∗k (1 − (2λ ∗)) ,for constant C > 0. Then the above converges to 0 when k → ∞.The last remark is unambiguously of numerical character.It shows that adiscretisation scheme could efficiently approximate numerically the control ψ.Remark 4.2.3. Similarly to the previous chapter, here the assumption on function⋂f may be relaxed to sup u |f(u, ·)| ∈ L p L2p with any p ≥ d+1. The bounds like(3.2.11) then should use L p and L 2p norms instead of ‖f‖ B . Again it is probable⋂that sup u |f(u, ·)| ∈ L p,loc L2p,loc with any p ≥ d + 1 also suffices and thatfurther extensions are possible related to a certain growth of L p,loc and L 2p,locnorms at infinity. However, some estimates may become more complicated.

Chapter 4. Mean-Variance Optimization 100We have proved that for the regularised equation, there exists a unique solutionof the Bellman’s PDE for the mean variance problem, although this dependson a one-dimensional parameter ψ, which complicates the problem. A solutionto a Bellman’s equation for a control problem implies sufficiency of Markovianstrategies α ∈ A M . What would be interesting to know though is that this sameoptimal strategy that has been constructed using the regularised process Y y,εtgivesat least an ”almost-optimal” result for the degenerate problem as well. We willuse notation ˜α ε , where ε relates to the regularised problem, where it appears asthe regularisation constant. We will also use another notation ɛ, which is differentfrom ε and is related to ”almost-optimal” strategies as they are defined below.Definition 4.2.1. Let ɛ ≥ 0. A strategy α ∈ A is said to be ɛ−optimal for (t, x) ifv(t, x) ≤ v α (t, x) + ɛ, where v(t, x) = sup α∈A v α (t, x).Lemma 4.2.2. For any strategy α ∈ A, we have the following bounds:|v ε,α (t 0 , x, y) − v α (t 0 , x, y)| ≤ Cε, (4.2.17)for a constant C ≥ 0, independent of (x, y).Proof. By the definition of v ε,α (t 0 , x, y) and v α (t 0 , x, y), we can write the

Chapter 4. Mean-Variance Optimization 101inequality (4.2.17) as:∣∣ Eα t 0 ,x∫ Tt 0− θ[ (E α t 0 ,xf(t, X t ) dt∫ Tt 0) 2 ∫ ]Tf(t, X t ) dt − Et α 0 ,x,y 2f(t, X t )Yt ε dtt 0− E α t 0 ,x∫ Tt 0+ θ[ (E α t 0 ,x==∣∣E α t 0 ,x,y∣∣E α t 0 ,x,y∫ Tt 0∫ Tf(t, X t ) dtt 0) 2 ∫ ∣T∣∣∣∣f(t, X t ) − Et α 0 ,x,y 2f(t, X t )Y t dt]t 0 t 0∫ T2f(t, X t )Y εt dt − E α t 0 ,x,y2f(t, X t ) (Y εt− Y t ) dt∣∫ Tt 02f(t, X t )Y t dt∣≤ 2‖f‖ B sup E| ˜W s − ˜W t0 |(T − t 0 ) ε = 2‖f‖ B (T − t 0 ) 3 2 E| ˜W1 | εt 0 ≤s≤T= 2‖f‖ B (T − t 0 ) 3 2√2π ε,similarly to (3.2.11). So the inequality (4.2.17) is proved and thereforev ε,α (t 0 , x, y) − Cε ≤ v α (t 0 , x, y) ≤ v ε,α (t 0 , x, y) + Cε. (4.2.18)For ε > 0, let ᾱ ε denote an (any) optimal strategy for the problem (4.2.1), if itexists, and let ˜α ε,δ denote any δ-optimal strategy for the same problem (v ε ).Theorem 4.2.2. There exists C ≥ 0, independent of (x, y), such that:

Chapter 4. Mean-Variance Optimization 1021. If an optimal strategy exists for the degenerate problem (4.2.5), then anyoptimal strategy for v ε (4.2.1) is at most 2Cε-optimal for (4.2.5).2. For any ν > 0, a 2Cε + ν-optimal Markov strategy for the original problemexists which can be constructed using the δ-optimal strategy for v ε .Proof. Suppose that the value function of the degenerate problem attains itssupremum for a strategy ᾱ. Then, we would have from Lemma 4.2.2:vᾱ(t 0 , x, y) − Cε ≤ v ε,ᾱ (t 0 , x, y) ≤ vᾱ(t 0 , x, y) + Cε. (4.2.19)Furthermore, because the strategy ᾱ ε is optimal for the regularised value function(assuming that the supremum is attained), we know thatv ε,ᾱε (t 0 , x, y) ≥ v ε,ᾱ (t 0 , x, y). (4.2.20)So,vᾱ(t 0 , x, y)−Cε ≤ v ε,ᾱ (t 0 , x, y) ≤ v ε,ᾱε (t 0 , x, y) ≤ vᾱε (t 0 , x, y)+Cε. (4.2.21)Hence,vᾱ(t 0 , x, y) − 2Cε ≤ (t vᾱε 0 , x, y), (4.2.22)i.e. ᾱ ε is 2Cε−optimal for v α .In the case that the degenerate value function does not attain a supremum, aγ−optimal strategy exists, namely ˜α. Thenv ˜α (t 0 , x, y) ≥ sup v α (t 0 , x, y) − γ (4.2.23)α∈A

Chapter 4. Mean-Variance Optimization 103and due to the bounds (4.2.17) and inequality (4.2.23) we havesup v α (t 0 , x, y)−γ −Cε ≤ v ˜α (t 0 , x, y)−Cε ≤ v ε,˜α (t 0 , x, y) ≤ v ˜α (t 0 , x, y)+Cε.α∈A(4.2.24)Now, suppose again that v ε,ᾱε (t 0 , x, y) = sup α∈A v ε,α (t 0 , x, y). That meansv ε,ᾱε (t 0 , x, y) ≥ v ε,˜α (t 0 , x, y).So,sup v α (t 0 , x, y) − γ − Cε ≤ v ˜α (t 0 , x, y) − Cε ≤ v ε,˜α (t 0 , x, y)α∈A≤ v ε,ᾱε (t 0 , x, y) ≤ vᾱε (t 0 , x, y) + Cε. (4.2.25)Thus,sup v α (t 0 , x, y) ≤ vᾱε (t 0 , x, y) + γ + 2Cε, (4.2.26)α∈Ai.e. ᾱ ε is an (γ + 2Cε)−optimal strategy for v α (t 0 , x, y).Finally, in the case that sup α∈A v ε,α (t 0 , x, y) is not attained, a δ−optimal strategy˜α ε is attained instead, such that v ε,˜αε (t 0 , x, y) ≤ sup α∈A v ε,α (t 0 , x, y) − δ.Following the same reasoning as previously (note that ˜α is γ−optimal for v ε ),we getsup v α (t 0 , x, y) − γ − δ − Cε ≤ v ˜α (t 0 , x, y) − Cε − δα∈A≤ v ε,˜α (t 0 , x, y) − δ ≤ v ε,˜αε (t 0 , x, y) ≤ v ˜αε (t 0 , x, y) + Cε. (4.2.27)Therefore,sup v α (t 0 , x, y) ≤ v ˜αε (t 0 , x, y) + γ + δ + 2Cε, (4.2.28)α∈A

Chapter 4. Mean-Variance Optimization 104i.e. ᾱ ε is an (γ + δ + 2Cε)−optimal strategy for v α (t 0 , x, y). Now, without lossof generality, we can take γ = δ = ν/2, which completes the proof.Remark 4.2.4. Recall that for v ε , nearly optimal strategies can always beconstructed among Markov strategies.Corollary 4.2.2. For any ε > 0, there exist C, ν > 0, such that Markov strategiesare at most 2Cε + ν−optimal for the degenerate value function.Proof. Since we deal everywhere with ”first moment theory” and additionally anoptimal ¯ψ can always be found in a real interval, we know that Markov strategiesare sufficient for the problem (see [29]).Therefore, because of the previoustheorem, Markov strategies are sufficient in obtaining an 2Cε + ν−optimalstrategy for the degenerate value function.4.3 With final paymentIn this case we consider the controlled functionals ṽ 1,α (t 0 , x) and ṽ 2,ε,α (t 0 , x).To maintain consistency with the notation of the last chapter, the ˜. over the costfunctionals and the value functions indicates the presence of a final payment.Then, the optimization problem (4.1.1) can be solved in exactly the same way as inthe previous section by just replacing the notation for the first and second moment(instead of v 1 there is ṽ 1 and in the place of v 2,ε there is now ṽ 2,ε ), along with the

Chapter 4. Mean-Variance Optimization 105relevant HJB equations. For example we are now examining the following twoproblems (regularised and degenerate respectively):andṽ ε (t 0 , x, y) := sup{ṽ1,α (t 0 , x) − θṽ 2,ε,α (t 0 , x, y) + θ (ṽ 1,α (t 0 , x)) 2} (4.3.1)α∈Aṽ(t 0 , x, y) := sup{ṽ1,α (t 0 , x) − θṽ 2,α (t 0 , x, y) + θ (ṽ 1,α (t 0 , x)) 2} . (4.3.2)α∈AThroughout this section, we assume the following:A 4.3 • The functions σ, β, c, f are continuous with respect (u, x) andcontinuous with respect to x uniformly over u for each t.• ‖˜σ(u, t, x) − ˜σ(u, t, x ′ )‖ ≤ K‖x − x ′ ‖.• ‖b(u, t, x) − b(u, t, x ′ )‖ ≤ K‖x − x ′ ‖.• ‖˜σ(u, t, x)‖ + ‖ ˜β(u, t, x)‖ + |f(u, t, x)| ≤ K.• |f(u, t, x) − f(u, t, x ′ )| ≤ K‖x − x ′ ‖.• Φ(x) ∈ C 2 b .Using the same representation for (ṽ 1 ) 2 as in (4.2.6), the optimisation problemtakes the following form:[ṽ ε (t 0 , x, y) = supψsupα∈A(ṽ1,α (t 0 , x)[1 − 2θψ] − θṽ 2,ε,α (t 0 , x, y) ) − θψ 2 ].(4.3.3)

Chapter 4. Mean-Variance Optimization 106Therefore, it is enough for this case only to present the slightly different resultingBellman’s equation. The Bellman’s PDE for the optimal solution of the internaloptimization problem which corresponds to (4.2.11) given ψ is for this case[( ) ∂(supu∈A ∂t + Lu V 2,ε + [1 − 2θψ]f(u, t, x) + θ g(u, t, x) + y ˜Φ 1 (u, t, x)) ] = 0,with initial dataV 2,ε ∣ ∣∣∣∣t=T= [1 − 2θψ]Φ(x) − θΦ 2 (x),(4.3.4)where we will be interested finally in y = 0. The notation follows naturally:andg(α t , t, X t ) = 2f(α t , t, X αt,xt )Φ(X αt,xt )˜Φ 1 (α t , t, X αt,xt∫ tYt x =) = f(α t , t, X αt,x ) + Φ 1 (X αt,x2f(α t , t, X αt,xtt 0) dttt )[ ∫ T]ṽ 1,α (t 0 , x) = Et α 0 ,x f(t, X t ) dt + Φ(X T )t 0( (Φ(XTṽ 2 (t 0 , x, y) = sup Et α 0 ,x,y ) ) ∫ T2+αṽ 2,ε (t 0 , x, y) = sup Et α 0 ,x,yα( (Φ(XT) ) 2+∫ Tt 0(t 0(g(t, X t ) + ˜Φ(t, X t )Ỹt)g(t, X t ) + ˜Φ(t,)X t )Ỹ εt)dt)dtV 2,ε (t 0 , x, y, ψ) := sup(ṽ1,α (t 0 , x)[1 − 2θψ] − θṽ 2,ε,α (t 0 , x, y) ) . (4.3.5)α∈A

Chapter 4. Mean-Variance Optimization 107Correspondingly,andV 2 (t 0 , x, y, ψ) := sup(ṽ1,α (t 0 , x)[1 − 2θψ] − θṽ 2,α (t 0 , x, y) ) (4.3.6)α∈AL (u,ε) := ∑ i,j∂ 2a ij (u, t, x)∂x i ∂x + ∑ j jb j (u, t, x) ∂∂x j+ (ε2 )2 ∆ yy + 2f(u, t, x) ∂ ∂y . (4.3.7)Theorem 4.3.1. Equation (4.3.4) has a unique solution in W 1,2,2 , underassumptions A 4.3 .The result follows from [29, chapter 3 and 4].Then, as in the previous section, the function V 2,ε is locally Lipschitz in ψ andgrows at most linearly in this variable. Hence, the supremum is again attained atsome ψ from a closed interval. Then the external optimisation problem becomes[ṽ ε (t 0 , x, y) = sup V 2,ε (t 0 , x, y, ψ) − θψ 2] . (4.3.8)ψLemma 4.3.1. The functions V 2 (t 0 , x, y, ψ) and V 2,ε (t 0 , x, y, ψ) satisfy thebounds:|V 2,ε (t 0 , x, y, ψ)| + |V 2 (t 0 , x, y, ψ)| ≤ C (1 + |ψ|)

Chapter 4. Mean-Variance Optimization 108andmax ( |V 2 (t 0 , x, y, ψ) − V 2 (t 0 , x, y, ψ ′ )|, |V 2,ε (t 0 , x, y, ψ) − V 2,ε (t 0 , x, y, ψ ′ )| )≤ C |ψ − ψ ′ |,for some constant C > 0.Proof. The first inequality of the lemma (linear growth with respect to ψ) followsfrom the definitions (4.3.5) and (4.3.6), since all three functions ṽ 2,ε (α, . . .),ṽ 2 (α, . . .) and ṽ 1 (α, . . .) are bounded.To show that the function V 1,ε is locally Lipschitz with respect to ψ, it is enoughto follow exactly the proof of Lemma 4.2.1.Corollary 4.3.1. Any supremum in (4.3.8) is attained between λ − ≤ ψ ≤ λ + ,where λ + , λ − are the same as in Corollary 4.2.1.The proof is exactly the same as for Corollary 4.2.1.Theorem 4.3.2. The approximate cost function ṽ ε satisfies (4.3.8) and there existsa constant C ≥ 0, independent of (x, y), such thatsupt,x,y|ṽ ε (t 0 , x, y) − ṽ(t 0 , x, y)| ≤ C θ ε. (4.3.9)The proof is also the same as the one of Theorem 4.2.1.Let the strategy ᾱ ε ∈ A be the optimal strategy for the problem (4.3.3), or if thesupremum is not attained, let the strategy ˜α ε ∈ A be a δ−optimal strategy for the

Chapter 4. Mean-Variance Optimization 109same problem. Notice that the Remarks 4.2.1 and 4.2.2 are also relevant here andcorresponding to theorem 4.2.2 we have:Theorem 4.3.3. There exists C ≥ 0, independent of (x, y), such that:1. If an optimal strategy exists for the degenerate problem (4.3.2), then anyoptimal strategy for ṽ ε (4.3.1) is at most 2Cε-optimal for (4.3.2).2. For any ν > 0, a 2Cε + ν-optimal Markov strategy for the original problemexists which can be constructed using the δ-optimal strategy for ṽ ε .Proof is the same as in Theorem 4.2.2.Corollary 4.3.2. For any ε > 0, there exist C ≥ 0, ν > 0, such that Markovstrategies are at most 2Cε + ν−optimal for the degenerate value function.The proof is the same as in Corollary 4.2.2.4.4 The ”first-second moment” variation of theproblemIn the previous two sections, the main goal was to achieve complete meanvariancecontrol through the solution of Bellman’s equations. The choice of theoptimisation criteria was standard, well-established in mathematical finance and

Chapter 4. Mean-Variance Optimization 110certainly attributed to Markowitz. The main reason for using variance is that,traditionally, it has been chosen to represent the risk involved in financial assets.One main weakness of the previous analysis is the extra control parameter ψ,which arises from the squared first moment term, included in the variance.Therefore, if we change convention and agree that the risk can efficiently berepresented solely by the second moment of the cost function, the analysis is muchsimplified. There is no need to use any additional parameters and the problem issolved in terms of one Bellman’s PDE. We just present below the relevant results,without any proofs, as these are the same or simplified versions of the ones inthe previous sections. We confine ourselves to the case without final payment asextension to final payment is then obvious.The ”first-second moment” problem would be formulated as follows forregularised and degenerate case respectively:{ˆv ε (t 0 , x, y) := sup v 1,α (t 0 , x) − θv 2,ε,α (t 0 , x, y) } , (4.4.1)α∈A{ˆv(t 0 , x, y) := sup v 1,α (t 0 , x) − θv 2,α (t 0 , x, y) } , (4.4.2)α∈Ausing same notation as in section 4.2.

Chapter 4. Mean-Variance Optimization 111The function in the left hand side of (4.4.1) is a solution of the Bellman PDE:[( )]∂supu ∂t + L(u,ε) ˆv ε (t, x, ψ) + [1 − 2θy]f(u, t, x) = 0,ˆv ε ∣ ∣∣∣∣t=T= 0.(4.4.3)Under the usual assumptions, we have existence and uniqueness of solution of theequation (4.4.3), due to [29, Chapters 3 and 4].Moreover, the regularised value function is close to the real value function.Theorem 4.4.1. There exists a constant C ≥ 0, independent of (x, y), such thatsup |ˆv ε (t 0 , x, y) − ˆv(t 0 , x, y)| ≤ C θ ε. (4.4.4)t 0 ,x,yFinally, the optimal strategy for the regularised cost function is also nearly-optimalfor the degenerate one. Let the strategy ᾱ ε ∈ A be the optimal strategy for theproblem (4.4.1), or if the supremum is not attained, let the strategy ˜α ε ∈ A be aδ−optimal strategy for the same problem .Theorem 4.4.2. There exists C ≥ 0, independent of (x, y), such that:1. If an optimal strategy exists for the degenerate problem (4.4.2), then anyoptimal strategy for (4.4.1) is at most 2Cε-optimal for (4.4.2).

Chapter 4. Mean-Variance Optimization 1122. For any ν > 0, a (2Cε + ν)-optimal strategy for the original problem existsand can be constructed using the δ-optimal strategy for ˆv ε .Proof is similar to the one in Theorem 4.2.2.Corollary 4.4.1. For any ε > 0, there exists C ≥ 0, ν > 0, such that Markovstrategies are at most 2Cε + ν−optimal for the degenerate value function.The proof is similar to the one in Corollary 4.2.2.

113Chapter 5Conclusions and Further Research5.1 ConclusionsThe work presented in this thesis mainly addresses the long existed problem ofcontinuous time mean-variance control of markovian value functions. Althoughthe problem is not solved here, the contribution of this work is the introductionof some new approaches that reestablish the use of Bellman’s equations inthe construction of markovian optimal strategies. However, work towards thisdirection gave rise to a number of smaller problems that had been addressed here.Naturally, when thinking in terms of variance, the second moment must betaken into account.Then, non-trivial problems start to appear as there is nostandard HJB theory for the second moment. Initially, in order to provide some

Chapter 5. Conclusions and Further Research 114understanding and intuition for the second moment of a markovian functional, weconfined ourselves to the problem without control. In this case, we managed toextend somewhat Dynkin’s and Kac’s chain of equations in that we do not assumeexponential moments for the value function and still we can express the secondmoment through a system of two equations. Then, a single equivalent PDE alsoleads to the solution of the same problem and although it is degenerate, it is wellposed.The method can also give higher moments.Then, returning to the problem of the controlled second moment, the degeneracyof the HJB equation was the main obstacle. We used regularisation to solve theequation by means of standard HJB theory and then proved that the ”cost” of theregularisation was a well bounded quantity.With the second moment in hand, we tried then to attack the ”mean-variance”problem. At this point, another obstacle emerged from the squared first momentterm included in the variance. This had been overcome with the introductionof a second, but trivial single-dimensional control parameter ψ. Thus, we couldfind an optimal strategy for the problem through the solution of a parameterisedsystem of HJB equations. What is more, Bellman’s equation imply sufficiency ofMarkovian strategies and we also show that these strategies are also ɛ-optimal forthe degenerate value function.

Chapter 5. Conclusions and Further Research 115From practical point of view, we have to admit that no significant advance has beenmade. We do not provide any numerical results, but these are a natural extensionof this, mainly theoretical work, and will be attempted by the author in the nearfuture.5.2 Further ResearchAs we mentioned above, a necessary and natural extension of this work istowards numerical approximations and proper algorithms for the construction ofthe optimal strategies. These, will hopefully make clear the importance of theabove results and provide useful tools for the solution of real problems.Moreover, some work has already been done to the direction of Viscosity solutionsfor the PDE’s we examine here.These are particularly useful in numericalapproximation schemes. Furthermore, optimal stopping using a ”mean-variance”stopping criterion and the method of randomised stopping is also a naturalextension of this work. Unfortunately, there was not enough time to include thesein this thesis.Finally, there are several, more ”delicate” extensions that can be made and thesemainly concern relaxations of the assumptions used for the function f. All along

Chapter 5. Conclusions and Further Research 116this thesis, we assumed f to be bounded. In fact, we could allow some polynomialgrowth for f or even consider f from some L p spaces. Another rather restrictiveassumption, especially for real problems is the requirement that the final paymentΦ(X t ) ∈ Cb 2 . There have been some efforts to remove this assumption but doesnot seem to be plausible for the moment.

117Bibliography[1] Adams R.A., Fournier J.J.F., Sobolev Spaces, Pure and Applies mathematicsSeries, Elsevier Science, 2nd Edition, (2003).[2] Bellman R.S., Dynamic Programming, Princeton University Press,Princeton, N.J., (1957).[3] Borkar V.S., Optimal control of diffusion processes, Longman Scientific andTechnical, (1989).[4] ———- , The probabilistic structure of controlled diffusion processes, ActaAppl. Math. 11, 19-48, (1988).[5] Crandall M.G., Ishii H. and Lions P.L., A User’s guide to viscosity solutions,Bulletin A.M.S., N.S. 27, 1-67, (1992).[6] Crandall M.G. and Lions P.L., Viscosity solutions of Hamilton-Jacobiequations, Trans. A.M.S., 277, 1-42, (1984).[7] Darling D.A., Kac M., On Occupation Times for Markoff Processes,Transactions of the American Mathematical Society, 84(2), 444-458, (1957).[8] Dynkin E.B., Introduction to Markov Processes, Vol 2, Springer-Verlag,(1965).[9] Dudley R.M., Kunita H., Ledrappier F., Ecole D’Ete De Probabilites DeSaint-Flour Xii, 1982, Lecture Notes in Mathematics, Springer, (1985).[10] Elton, E. J., Gruber, M. J., Brown, S. J., Goetzmann, W. N., Modern PortfolioTheory and Investment Analysis, (6th edition), John Wiley & Sons, Inc,(2003).

BIBLIOGRAPHY 118[11] Feller, W., An Introduction to Probability Theory and Its Applications,Volume I, 3rd Edition, John Wiley and Sons, Inc, New York-London-Sydney,(1970).[12] Fitzsimmons P.J., Pitman J., Kac’s moment formula and the Feynman-Kacformula for additive functionals of a Markov process, Stochastic processesand their applications, 79, 117-134, (1999).[13] Fleming W.H., Duality and a priori estimates in Markovian optimizationproblems, J. Math. Anal. Appl., 16, 254-279, (1966).[14] Fleming W.H., Soner H.M., Controlled Markov Processes and ViscositySolutions, Springer, Stochastic Modelling and Applied Probability, SecondEdition, (2006).[15] Föllmer H., Schweizer M., Hedging of contingent claims under incompleteinformation. In: Davis,M. H. A., Elliott, R. J. (Eds.) Applied StochasticAnalysis, Gordon and Breach, London et al., 389-414, (1991).[16] Föllmer H., Sondermann D., Hedging of non-redundant contingent claims.In: Contributions to Mathematical Economics, In honor of Gérard Debreu.Ed. by W. Hildenbrand, A. Mas-Colell, North–Holland, Amsterdam et al.,205-223, (1986).[17] Gilbarg D., Trudinger N.S., Elliptic Partial Differential Equations of SecondOrder, Springer-Verlag, second edition (1983).[18] Girsanov I.V., Minimax problems in the theory of diffusion processes, SovietMathematics, Doklady Akademii Nauk SSSR 136(4), 118-121 (EnglishTranslation), (1961).[19] Gourieroux C., Laurent J.P., Pham H., Mean-variance hedging andnuméraire, Math. Finance, 8(3), 179-200, (1998).[20] Has’minskiǐ R., Z., Stochastic Stability of Differential Equations, Sijthoffand Noordhoff, Alphen aan den Rijn, The Netherlands-Rockville, Maryland,USA, (1980).[21] Howard R. A., Dynamic Programming and Markov Processes, Wiley, NewYork, (1960).

BIBLIOGRAPHY 119[22] Ikeda N., Watanabe S., Stochastic Differential Equations and DiffusionProcesses, North Holland Publishing Company, (1981).[23] Kac M., On Distributions of Certain Wiener Functionals, Transactions ofthe American Mathematical Society, 65(1), 1-13, (1949).[24] Karatzas I., Shreve S.E., Brownian Motion and Stochastic Calculus,Springer-Verlag, Second Edition, (1991).[25] Krylov N.V., Bounded inhomogeneous nonlinear elliptic and parabolicequations in the plane, Math. USSR Sbornik, 11(1), 89-99 (Englishtranslation), (1970).[26] ———- , Control of Markov processes and W-spaces, Math. USSR Izvestija,5(1), 233-265 (English translation), (1971).[27] ———- , On control of the solution of a stochastic integral equation, TheoryProb. Appl., 17(1), 114-131 (English translation), (1972).[28] ———- , On the selection of a Markov process from a system of processesand the construction of quasi-diffusion processes, Math. USSR Izestija, 7(3),691-708 (English translation), (1973).[29] ———- , Controlled Diffusion Processes, Springer-Verlag, (1980).[30] ———- , Introduction to the Theory of Diffusion Processes, Translationsof Mathematical Monographs vol 142, American Mathematical Society,Providence, Rhode Island, (1994).[31] Ladyzhenskaya O.A., Solonnikov V.A., Uralceva N.N., Linear andQuasi-Linear Equations of Parabolic Type, Translations of MathematicalMongraphs, Vol 23, American Mathematical Society, (1998).[32] Ladyzhenskaya O.A., Uralceva N.N., Linear and Quasilinear EllipticEquations, Translated from the Russian, Leon Ehrenpreis Academic Press,New-York-London, (1968).[33] Laurent J.P., Pham H., Dynamic programming and mean-variance hedging,Finance Stochast., 3, 83-110, (1999).[34] Markowitz H.M., Portfolio Selection, Journal of Finance 7 (1), 77-91,(1952).

BIBLIOGRAPHY 120[35] Nisio M., On a nonlinear semigroup attached to stochastic optimal control,Publ. RIMS, Kyoto University, (1976).[36] ———- , On stochastic optimal controls and envelopes of MarkovianSemigroups, in K. Itô (ed.).Proc. Int. Symp. on stochastic differentialequations, Wiley, 297-325, (1978).[37] ———- , Note on the uniqueness of semigroup associated with Bellmanoperator, in W. Fleming, L.G. Gorostiza (eds), Advances in filteringand optimal stochastic control, Lecture Notes in Control and InformationSciences No. 42, Springer, 267-75, (1982).[38] ———- , Lectures on Stochastic control theory, ISI Lecture Notes No. 9,Macmillan, (1983).[39] Rishel R., Necesary and Sufficient Dynamic Programming Conditions forContinuous Time Stochastic Optimal Control, SIAM J. Control, 8(4), 559-571, (1970).[40] ———- , Weak Solutions of a Partial Differential Equation of DynamicProgramming, SIAM J. Control, 9(4), 519-528, (1971).[41] ———- , Optimal Control of Stochastic Integrals and Hamilton Jacobi-Bellman Equations, 1340-1344, (1981).[42] Schweizer M., Mean-Variance hedging for general claims, Ann. Appl. Prob.,2, 171-179, (1992).[43] Sobolev S.L., Applications of functional analysis in mathematical physics,Izdat. Leningrad. Gos. Univ., Leningrad, (1950); English translation, Trasl.of Math. Monographs, vol. 7, American Mathematical Society, Providence,R.I. MR14 565; MR29 2624, (1963).[44] Veretennikov A.Yu., On strong solutions and explicit formulas for solutionsof stochastic differential equations, Math. USSR Sbornik 39, 387-401,(1981).[45] ———- , Parabolic equations and Ito stochastic equations with coefficientsdiscontinuous in the time variable, Math. Notes. 31(3-4), 278-283, (1982).

BIBLIOGRAPHY 121[46] ———- , On polynomial mixing estimates for stochastic differentialequations with a gradient drift, Theory Probab. Appl. 45(1), 160–163,(2001).[47] Wiener N., Differential Space, J. Math. Phys., 2, 131-174, (1923).[48] Zhou X.Y., Li, D., Continuous-Time Mean-Variance Portfolio Selection:A Stochastic LQ Framework, Appl. Math. Optim. 42(19-33), DOI:10.1007/s002450010003, (2000).