An Introduction to Viscosity Solutions: theory, numerics and ... - BCAM

An Introduction to Viscosity Solutions:theory, numerics and applicationsM. FalconeDipartimento di MatematicaOPTPDE-BCAM Summer School”Challenges in Applied Control and Optimal Design”July 4-8, 2011, Bilbao – Lecture 1/4

OUTLINE OF THE COURSE:• Lecture 1: Introduction to viscosity solutions• Lecture 2: Approximation schemes for viscosity solutions• Lecture 3: Approximation of optimal control problems via DP• Lecture 4: Efficient methods and perspectives

OUTLINE OF THIS LECTURE:Introduction to viscosity solutions• Motivations (related to control problems)• Viscosity solutions for 1st order PDEs• Uniqueness for viscosity solutions• Some properties of viscosity solutions• Hopf–Lax representation formula• Some hints on viscosity solutions for 2nd order PDEs• Existence and uniqueness

MotivationsOur approach to the solution of optimal control problems andgames is based on Dynamic Programming, another approach isbased on Pontryagin Maximum Principle.By the Dynamic Programming Principle, we will derive the characterizationof the value function in terms of a first order partialdifferential equation (PDE), the Bellman equation (or the Isaacsequation for games ).

MotivationsThis approach is interesting for the continuous control problemsas well for numerical purposes and can be applied to all classicalcontrol problems.In one of the following lectures we will deal with the constructionof approximation schemes via Discrete Dynamic Programming.We will also deal with the algorithms which allow to computethe value function, optimal controls in feedback form and thecorresponding optimal trajectories.

The finite horizon problemDynamics⎧⎨⎩ẏ(t) = b(y(t), α(t))y(t 0 ) = x 0where α(·) ∈ A ≡ {α : [0,+∞[→ A, measurable} and A ⊂ R M iscompactCostJ (t0 ,x 0 ) (α) ≡ ∫ tft 0f(y(s), α(s))e −λs ds + ψ(y(t f )), λ > 0Value function v(t 0 , x 0 ) ≡ infα∈A J (t 0 ,x 0 ) (α) .

The infinite horizon problemDynamics⎧⎨⎩ẏ(t) = b(y(t), α(t))y(0) = xwhere α(·) ∈ A ≡ {α : [0,+∞[→ A, measurable} and A ⊂ R M iscompactCostJ x (α) ≡∫ ∞0 f(y(s), α(s))e−λs ds , λ > 0Value function v(x) ≡ infα∈A J x(α) .

The infinite horizon problem with state constraintsDynamics⎧⎨⎩ẏ(t) = b(y(t), α(t))y(0) = xwhere y ∈ R N . Now we require that y x (t) ∈ Ω ⊂ R N for any t.Admissible controlsα(·) ∈ A x ≡ {α : [0,+∞[→ A, measurable such that y x (t) ∈ Ω}where A ⊂ R M is compact.

CostJ x (α) ≡∫ ∞0 f(y(s), α(s))e−λs ds , λ > 0Value function v(x) ≡infα∈A xJ x (α) .

The nonlinear minimum time problemDynamics⎧⎨⎩ẏ(t) = b(y(t), α(t))y(0) = xy ∈ R Nand α(·) ∈ AwhereA ≡ {α(·) : [0,+∞[→ A measurable} and A ⊂ R M is compactTarget T is a compact set with nonempty interior

The minimum time function is:T(x) ≡inf {t : y x(t, α(t)) ∈ T }α(·)∈AA priori T is not defined everywhere, its domain of definition isthe reachable set ,R ≡ { x ∈ R N : T(x) < +∞ }Note that R is NOT given and can have a rather complicatedshape even for simple dynamics. This is a free-boundary problemwhere we have to detect the couple (T, R).

Value function and HJB equationThe link is the followingThe value function is the unique viscosity solution of the Bellmanequation associated to the problem via Dynamic Programming.For the infinite horizon problem this isλv(x) + max{−b(x, a) · ∇u(x) − f(x, a)} = 0, xa∈A∈ RN

Value function for a minimum time problem32.521.510.52010−1−2−2 −1.5 −1 −0.5 0 0.5 1 1.5 2

Value function for a state constraint problem

Viscosity solutions for 1st order PDEsLet us start with the stationary model problem⎧⎨⎩H(x, u, Du) = 0u(x) = g(x)in Ωon ∂Ω(HJ)where H : R n × R × R n → R is the Hamiltonian and g is a givenboundary condition.We want to define a good notion of weak solution since theproblem is nonlinear and we cannot expect a classical solutionto (HJ).

Stationary 1st order PDEsThe previous examples have shown that we can have:• jumps in the gradient of the solution• jumps in the solution for the constrained problem andfor differential gamesThe typical assumptions on the Hamiltonian H(x, u, p) to set upthe theory areA1: H(·, ·, ·) is uniformly continuousA2: H(x, u, ·) is convexA3: H(x, ·, p) is monotone

Example: |u x | = 1Consider the following Dirichlet problem for the eikonal equationin 1 dimension⎧⎨⎩|u x | = 1 in Ω ≡ (−1,1)u(x) = 0on ∂Ω(E)Obviously, u 1 (x) = x and u 2 (x) = −x satisfy the equation pointwisein (-1,1) but they do not satisfy the boundary conditions.Moreover, a classical solution can not exist due to Rolle Theorem.Both functions u 3 (x) = |x| − 1 and u 4 (x) = 1 − |x| satisfy theequation a.e. and satisfy the boundary conditions.

Example: |u x | = 1One can easily seen that there are infinitely many a.e. solutionsof the equation.In fact, collecting piecewise affine functions parallel to x or −xone can built a new a.e. solution.

Vanishing viscosity limitConsider the second order problem−εu xx + |u x | = 1, in (−1,1) (1)with the homogeneous boundary condition u(−1) = u(1). Thishas a regular solution u ε ∈ C 2 (−1,1) for every positive εWe can pass to the limit at every x ∈ Ω gettinglimε→0 uε (x) = u(x) (2)and define the limiting function u to be the weak solution of ourproblem (Kruzkov, 60s).Note that this is the reason for the name.

Definition of viscosity solutionsLet us consider the ”classical definition” for u ∈ BUC(Ω) (thespace of Bounded Uniformly Continuous function over the openset Ω).Note that the definition is direct (without any reference to aregularization and/or a limit) and LOCAL .

Definition of viscosity solutionsDEFINITIONu ∈ BUC(Ω) is a viscosity solution ofH(x, u, Du) = 0 in Ωif and only if, for any ϕ ∈ C 1 (Ω) the following conditions hold:i) at every local maximum point x o ∈ Ω for u − ϕH(x 0 , u(x 0 ), Dϕ(x 0 )) ≤ 0i.e. u is a viscosity sub-solution.ii) at every local minimum point x o ∈ Ω for u − ϕH(x 0 , u(x 0 ), Dϕ(x 0 )) ≥ 0i.e. u is a viscosity super-solution.

Example: |u x | = 1Let us go back to the example.Take any a.e solution u which has a local minimum at x 0 andchoose (for example) ϕ = costant. Clearly, x 0 is a local minimumpoint for u − ϕ so that we should havewhich is FALSE since ϕ x ≡ 0.|ϕ x (x 0 ) ≥ 1Conclusion: every a.e. solution having a local minimum pointcannot be a viscosity super-solution.

Example: |u x | = 1The same argument will not work for sub-solutions.Take any a.e solution u which has a local maximum at x 0 andchoose (for example) ϕ = costant. Clearly, x 0 is a local maximumpoint for u − ϕ so that we should have|ϕ x (x 0 )| ≤ 1which is TRUE since ϕ x ≡ 0.The only viscosity solution of our problem is v(x) = 1 − |x|.

Some properties of viscosity solutions1. If u is a classical C 1 (Ω) solution then it is also a viscositysolution.2. If u is a regular viscosity solution then it is also a classicalsolution (i.e. satisfies the equation pointwise).3. the viscosity solution u is the maximal sub-solution , i.e.w ≤ u, for any w ∈ S ≡ {space of sub-solutions}

4. (stability) the viscosity solution u is the uniform limit of u εwherewhich means−εu ε xx + H(x, u ε , Du ε ) = 0u(x) = limε→0 + uε (x)WARNING: as we have seen in the example, if u is a viscositysolution this DOES NOT IMPLY that −u is a viscosity solution(unless u is regular).

Comparison pricipleThe crucial point for viscosity solution is to prove uniqueness.This is done via a comparison principle (also called maximumprinciple).THEOREMLet u, v ∈ BUC(Ω) be respectively a sub and a super-solutionforand letH(x, u, Du) = 0in ΩThen,u(x) ≤ v(x)for any x ∈ ∂Ω.u(x) ≤ v(x) for any x ∈ Ω.

Uniquess via the comparison principleThe comparison principle is enough to get uniqueness.In fact, let u and v be two viscosity solutions of the equationsatisfying the same boundary condition, clearly they are (both)sub and super-solutions. Then, we haveu(x) ≤ v(x) for any x ∈ ΩAND, reverting the role of u and v, alsou(x) ≥ v(x) for any x ∈ Ωwhich implies u(x) = v(x) in Ω.

Sufficient conditions for uniquenessLet ϕ : R → R + be continuous and ω(·) be a modulus of continuity.Let us assume thatA4 : |H(x, u, p) − H(y, u, p)| ≤ ω(|x − y|(1 + |p|))Q R (x, y, u, p)for any x, y ∈ Ω, −R ≤ u ≤ R and p ∈ R n , whereQ R (x, y, u, p) ≡ max(ϕ(H(x, u, p)) , ϕ(H(y, u, p))THEOREMLet the assumptions (A1–A4) be satisfied, then the comparisonprinciple holds for (HJ), i.e. the viscosity solution is unique.

Variable doublingThe goal is to prove that M = max(u − v) is negative.x∈ΩWe introduce a test function depending on two variables|x − y|2ψ ε (x, y) ≡ u(x) − v(y) −ε 2 .Due to the penalization term, we can expect that the maximumpoints (x ε , y ε ) for ψ ε should have x ε and y ε close enough for εsmall. Moreover, for ε → 0 + we have:M ε → M,|x ε − y ε | 2ε 2→ 0 and u(x ε ) − v(y ε ) → MThose properties allow to pass to the limit and get the comparisonresult.

Boundary conditionsAnother peculiar point is the way boundary conditions are satisfied.One can consider Dirichlet boundary conditions, Neumann boundaryconditions and ”state constraints” boundary conditions.The typical compact form for boundary conditions ismin(H(x, u(x), Du(x)), B(x, u(x), Du(x))) ≤ 0max(H(x, u(x), Du(x)), B(x, u(x), Du(x))) ≥ 0on ∂Ωon ∂Ωwhere B represents the ”boundary” operator.The technical point is that the equation plays a role up to theboundary.

Examples of Boundary conditionsFor example, the two classical Dirichlet and Neumann boundaryconditions correspond to:The typical compact form for boundary conditions isB(x, u(x), Du(x))) ≡ u − gB(x, u(x), Du(x))) ≡ ∂u∂η − gon ∂Ωon ∂ΩWARNING: not all the boundary conditions are compatible withthe equations.

Viscosity solutions for evolutive 1st order PDEsA typical example is the evolutive equation related to the levelsetformulation of a front propagating in the normal directionwith a (known) velocity:⎧⎨⎩u t + c(x)|Du| = 0u(x,0) = u 0 (x)in R × (0, T),in R(3)where u 0 : R → R must be a proper representation of the frontΓ 0 .

This means that u 0 has to change sign on Γ ≡ ∂Ω 0⎧⎪⎨⎪⎩u 0 (x) < 0 in Ω − 0u 0 (x) = 0 in Γu 0 (x) > 0 in Ω + 0so that the front at time 0 is identified as the 0-level set of u 0 .

Definition of viscosity solutions (evolutive case)DEFINITIONu ∈ BUC(Ω × (0, T) is a viscosity solution ofu t + H(x, u, Du) = 0in Ωif and only if, for any ϕ ∈ C 1 (Ω ×(0, T)) the following conditionshold:i) at every local maximum point (x 0 , t 0 ) ∈ Ω × (0, T) for u − ϕϕ t (x 0 , t 0 ) + (H(x 0 , u(x 0 , t 0 ), Dϕ(x 0 , t 0 )) ≤ 0i.e. u is a viscosity sub-solution.ii) at every local minimum point (x 0 , t 0 ) ∈ Ω × (0, T) for u − ϕϕ t (x 0 , t 0 ) + (H(x 0 , u(x 0 , t 0 ), Dϕ(x 0 , t 0 )) ≥ 0i.e. u is a viscosity super-solution.

Hopf-Lax representation formulaIt is interesting to note that in some cases we have a representationformula for the viscosity solution.For example, let us consider the problem⎧⎨⎩u t + H(Du) = 0 in R n × (0, T),u(x,0) = u 0 (x) in R n (EHJ)where H(·) is continuous and convex andlim|p|→+∞H(p)|p|= +∞ coercivity

Hopf-Lax formulaDEFINITION (Legendre-Fenchel conjugate)H ∗ (p) = sup · q − H(q)} for p ∈ Rnq∈Rn{p PROPERTIES• H ∗ : R n → R• H = H ∗∗

Hopf-Lax representation formulaA typical example is the followingH 2 (p) = |p|22 for which H∗ |p|22 (p) =2 .A similar construction works also for the caseH 1 (p) = |p|( WARNING: H 1 is not coercive!)In this case the Legendre-Fenchel conjugate is not defined everywhere,in factH ∗ 1 (p) = ⎧⎨⎩0 for |p| ≤ 1+∞ elsewhere

Hopf-Lax representation formulaThe unique viscosity solution of (EHJ) is given by the Hopf-Laxrepresentation formula{(v(x, t) = inf v 0 (y) + tH ∗ x − yy∈R n twhich after the change of variable a = x−ytas)}v(x, t) = infa∈R n {v 0(x − ta) + tH ∗ (a)}can also be written

HJ equation and Conservation Laws in RAnother interesting remark is the link between the entropy solutionsand viscosity solutions.This link is only valid in R. Consider the two problems, theevolutive Hamilton-Jacobi equation⎧⎨⎩v t + H(v x ) = 0v(x,0) = v 0 (x)and the associated conservation law⎧⎨⎩u t + H(u) x = 0u(x,0) = u 0 (x)in R × (0, T),in Rin R × (0, T),in R(HJ)(CL)

HJ equation and Conservation Laws in RAssume thatv 0 (x) ≡∫ x−∞ u 0(ξ)dξIf u is the entropy solution of (CL), thenv(x, t) =∫ x−∞u(ξ, t)dξis the unique viscosity solution of (HJ).Viceversa, let v be the viscosity solution of (HJ), then u =v x is the unique entropy solution for (CL) (note that v is a.e.differentiable).This link will be also useful for numerical purposes.

Discontinuous viscosity solutionIn several applications, e.g. to image processing and to games,it is natural to look for discontinous solutions. This can be donein the framework of viscosity solutions.ENVELOPESLet z : R n → R be a bounded function.Lower semi-continuous envelopez ∗ (x) ≡ liminfy→x z(y)Upper semi-continuous envelopez ∗ (x) ≡ limsupy→xz(y)

Discontinuous viscosity solutionDEFINITIONu ∈ B(Ω) is a viscosity solution ofH(x, u, Du) = 0 in Ωif and only if, for any ϕ ∈ C 1 (Ω) the following conditions hold:i) at every local maximum point x 0 ∈ Ω for u ∗ − ϕH(x 0 , u ∗ (x 0 ), Dϕ(x 0 )) ≤ 0i.e. u ∗ is a viscosity sub-solution.ii) at every local minimum point x 0 ∈ Ω for u ∗ − ϕH(x 0 , u ∗ (x 0 ), Dϕ(x 0 )) ≥ 0i.e. u ∗ is a viscosity super-solution.

Viscosity solutions for evolutive 2nd order PDEsThe two fully non linear Hamilton-Jacobi equations are in thiscaseH(x, u, Du, D 2 u) = 0 in Ωu t + H(x, u, Du, D 2 u) = 0 in Ω × (O, T)Naturally, one can deal with easier problems likeu t − ∆u + H(Du) = 0 in Ω × (O, T)u t + div(Du|Du|)|Du| = 0 in Ω × (O, T)

Maximum principle for 2nd order PDEsLet us consider the Dirichlet problem⎧⎨⎩u + H(Du, D 2 u) = f in Ω,u(x) = g(x) on ∂Ω(HJ 2 )where f, g ∈ C 0 (Ω).THEOREMLet H(p, X) be continuous and (degenerate) elliptic.Assume that :1. u is an u.s.c. subsolution of (HJ 2 )2. v is a l.s.c. super-solution of (HJ 2 )3. u ≤ v on ∂Ω.Then, u ≤ v in Ω.

Maximum principle for 2nd order PDEsNote that the usual proof by variable doubling does NOT work.In fact, if we define|x − y|2ψ(x, y) ≡ u(x) − v(y) −2εand (x ε , y ε ) ∈ Ω × Ω for small ε. By the definitions of sub andsuper solutions (in the viscosity sense) we getu(x ε ) + H( xε − y ε, 1 )ε ε I ≤ f(x ε )v(y ε ) + H( xε − y ε, − 1 )ε ε I ≥ f(y ε )

which impliesu(x ε ) − v(y ε ) ≤ f(x ε ) − f(y ε )+H( xε − y ε, 1 )ε ε I − H( xε − y ε, − 1 )ε ε Iand we cannot end the proof since I ≥ −I impliesH( xε − y ε, 1 )ε ε I − H( xε − y ε, − 1 )ε ε I > 0.so that in the limit we will not getu(x) ≤ v(x)

Existence via the Perron MethodLet the comparison result (maximum principle) hold for (HJ 2 ).The following result gives the existence of viscosity solutions alwaysin terms of ”maximal subsolution”.THEOREMAssume that there exist a sub-solution u and a super-solution uof (HJ 2 ) satisfyingu ∗ (x) = u ∗ (x) = g(x), for any x ∈ ∂ΩLet S be the set of sub-solutions of (HJ 2 ).Then,W(x) = sup{w(x) : u ≤ w(x) ≤ u}w∈Sis a viscosity solution of (HJ 2 ).

ConclusionsThe theory of viscosity solutions gives a good framework for theanalysis of nonlinear PDEs of first and second order.Existence and uniqueness results have been proved under verygeneral hypotheses.Applications of this theory range from control problems andgames, to front propagation and image processing.

Basic referencesGENERAL THEORYG. Barles, Solutions de viscositè des equations d’Hamilton–Jacobi,Springer–Verlag, 1998.M.G. Crandall, H. Ishii, P.L. Lions, User’s guide to viscosity solutionsof second order partial differential equations, Bull. Amer.Math. Soc. 27 (1992), 1-67.A very readable introduction to HJ equations is also containedin the bookC. Evans, Partial Differential Equations, American MathematicalSociety, 1999.

Basic referencesVISCOSITY SOLUTIONS AND CONTROL THEORYM. Bardi, I. Capuzzo Dolcetta, Optimal control and viscosity solutionsof Hamilton-Jacobi-Bellman equations, Birkhäuser, 1997.W.H. Fleming, H.M. Soner, Controlled Markov processes andviscosity solutions, Springer–Verlag, New York, 1993.SOME RECENT DEVELOPMENTSRecent papers by L. Caffarelli and X. Cabrè deal with fully nonlinear2nd order equations.