1 General Algebraic and Differential Riccati ... - Bradley Bradley

1
General Algebraic and Differential
Riccati Equations from Stochastic LQR Problem
Libin Mou
Department of Mathematics
Bradley University
Peoria, IL 61625
Abstract. In this paper we consider a class of matrix Riccati equations arising from stochastic
LQR problems. We prove a monotonicity of solutions to the differential Riccati equations, which
leads to a necessary and sufficient condition for the existence of solutions to the algebraic Riccati
equations. In addition, we obtain results on comparison, uniqueness, stabilizability and approximation
for solutions of the algebraic Riccati equations.
§ 1. Introduction
The following differential Riccati equation has been studied in [11] by using the method of
upper and lower solutions.
Ú w X
X
Ý T €E T€TE€G TG€ K € CaTb
X X X
X " X X
Û aF T€HTG€WbaV€HTHb aF T€HTG€Wb
œ!ß
Ý
Ü T a>
b œR,
"
(1)
where E is the transpose of E, T œ
.>
, R is a symmetric matrix, C is a linear map of symmetric
matrices, and EßFßGßHß K ßV and W are bounded and measurable matrix functions with appropriate
dimensions. The main results in [11] include an interpretation of upper and lower solutions,
comparison theorems, an upper-lower solution theorem, necessary and sufficient conditions for
existence of solutions, an estimation of maximal existence intervals of solutions and an approximation
of solutions.
X w .T
(1):
This paper is a continuation of
X
X
E T€TE€G TG€ K € CaTb
[11]. The focus here is the algebraic equation associated with
X
X X X X X
aF T€HTG€WbaV€HTHb aF T€HTG€Wb
œ! ,
"
(2)
where EßFßGßHßK ßV and W are constant matrices with appropriate dimensions; see (5) below.
The inclusion of the term C is important to stochastic control problems with Markovian
jumping noises and differential game problems with state-dependent noises; see [17] and [10], for
example.
Equation (2) with C œ! becomes
X
X
E T€TE€G TG€ K
X
X X X X X
aF T€HTG€WbaV€HTHb aF T€HTG€Wb
œ! .
"
(3)
Equation (3) arises from the following stochastic
linear quadratic regulator ( LQR) problem of infinite-

horizon with control-dependent noises:
Ú minimize/maximize Na b for ­hÒ!ß_Ñ, where
_
Û a b e' X X X
N œI
!
aB KB€#B W€V b.>
f subject to (4.1)
Ü .B œ aEB€F b.>€ aGB€H b.[ß> 0 àBa!
b œD,
(4.2)
2
(4)
where [ ab > is a standard Brownian motion on a complete probability space with [ a!
b œ! almost
surely, Ief is the expectation of the enclosed variable, and hÒ!ß_Ñ is the set of all admissible control
processes ; see Section 2 for a further description. For a detailed account of this problem, see [1]
and [22]. Through approaches of convex optimization and semidefinite programming, interesting
relationships between equation (3) and the LQR problem (4) have established in [1] and [22]. These
approaches are especially attractive for purpose of approximating solutions.
A monotonicity property for solutions to equation (1) will be proved, which leads to necessary
and sufficient conditions for the existence of solutions to (2). For solutions to equation (2), we will
prove results on comparison, uniqueness, stabilizability and approximation. We use a method of upper
and lower solutions, which satisfy associated Riccati inequalities. This method is closely related to the
methods of linear matrix inequalities in [1] [18] [22] and semidefinite programming in [22]. As
pointed out in [11], our method has several desirable merits. For example, it directly links equation
(3) with the LQR problem (4); see Theorem 2. It derives the main results under general assumptions.
It gives verifiable necessary and sufficient conditions for the existence of solutions [Theorems 8 and
9]. It also gives algorithms for approximating solutions. It can be used to estimate the maximal
existence intervals of solutions to differential Riccati equations without solving the equations [11]. It
applies to differential as well as algebraic Riccati equations.
The paper is organized as follows. In Section 2, we set up the notations used in this paper and
define upper and lower solutions. In addition, we describe the LQR problem (4) and interpret upper
and lower solutions to (2) in Theorem 2.
Section 2 is ended with some results from [11] that are needed in this paper. Specifically,
Theorem 5 is an upper-lower solution theorem and Propositions 3 and 4 are some structural
properties of equations (1), (2) and (3).
In Section 3, we prove a monotonicity [Theorem 7] for solutions to (1), which leads to a
general necessary and sufficient condition [Theorem 8] for the existence of solutions to (2). A refined
existence result [Theorem 9] for (2) is proved under ms-stabilizability. Theorem 9 generalizes a main
result in [1, Theorem 10] for (2) with C œWœ! . The definitions of ms-stability, ms-stabilizability
and ms-detectability for equation (2) are given in this section.
In Section 4, we study the comparison, uniqueness, stabilizability and approximation of
solutions to (2). Theorem 11 is a comparison theorem for ms-stabilizing solutions. Theorem 12 gives
the uniqueness and extreme properties of stabilizing solutions. Generalizing a classical relationship
between detectability and stability, Theorem 14 shows that ms-detectability implies the msstabilizability
of solutions to (2). Theorem 16 gives an algorithm for approximating solutions to (2).
The results here generalize known results for equation (2) with GœHœWœ! obtained in [5], [6]
and [21]. For results for equation (2) with GœHœWœ C œ! , see [2], [3], [4], [8], [9], [14],
[15], [16], [18], [19], [20] and [24]. The proofs are reduced to proving the same results for linear
equations in Theorems 10 and 13, which may be of interest in their own right.
§ 2. Preliminary Results

3
Denote ’ 8 ‘
8‚8 X
X
œ eT­ ÀT œTf, where T is the transpose of T. We write Q " Q#
8 8 8
( Q" žQ# ) if Q" Q#
­ ’ is a positive semidefinite (definite). For a map C À ’ Ä ’ we write
C ! if CaQ b ! for each Q .
" " !
Assumption. We assume that EßFßGßHßKßVßW and R in equations (1), (2) and (3) are constant
matrices and C À ’ 8 Ä ’
8 is a linear map, which satisfy
8‚8 8‚5 8
X
EßG­ ‘ àFßHßW ­ ‘ àV­ ’ 5 ; K, R ­ ’ ; C 0 Þ
(5)
For a Hilbert space — and an interval M, P
_ aMß—
b is the space of all bounded and measurable
"ß_ _ w _
functions from M to —. Furthermore, we define P aMß— b œÖT ­P aMß— bßT ­P aMß — b×Þ
The
solution T to (1) is assumed to be in P "ß_ aMß’ 8 b. Since all matrices in (1) are constant, a solution T
on M is actually smooth. The solution T to (2) and (3) is assumed to be in ’ 8 , which may be
considered as a constant solution to the associated differential equation.
As in [11], we abbreviate (1), (2) and (3) as
Ú T w € LQaT b€ CaTb œ!ß T a> " b œ Rß (1)
Û LQaT b€ CaTb
œ!ß
(2)
Ü LQaTb
œ!ß
(3)
where LQaTb œK€ L aTb
QaT
b and
X
X
L aTb
œE T€TE€G TG,
œ
Q a b
X X
aF T€HTG€Wba X
X " X X
T œ
V€HTHb aF T€HTG€Wb
(6)
X
We remark that Q aTb and LQ aTb
make sense even if V€HTH is singular. To see this, we
introduce the following notations
X X X
eaTb œV€HTH, faTb
œ F T€HTG€W. (7)
"
e f e
^a b œ a T b a T b if a T b is nonsingular,
T œ
€
eaTb faTb if eaTb
is singular,
(8)
€
where eaTb is the pseudoinverse of eaTb. Recall that any matrix Q has a unique Moore-Penrose
pseudoinverse Q € with the following properties (see [13] and [1]).
€ € € €
QQ QœQßQ QQ œQ . (9)
8 € 8 € €
If Q­ ’ , then Q ­ ’ , and QQ œQ Q.
Q
€
! if and only if Q !.
€
Although ^aTb œ eaTb faTb
always exists, but it may not be bounded nor satisfy
faTb œ eaTb^aT b
(10)
on the interval M. We recall some definitions in [11]. A function T­P "ß_ aMß’ 8 b is said to be feasible
if ^ aTb ­P _ aMß‘
5‚8 b and (10) holds . Afunction O ­ P _ aMß‘
5‚8 b is called a feedback matrix
associated with T­P
"ß_ aMß’ 8 w
b if it satisfies faTb œ eaTbO.
The set of all such O= is denoted by

4
_ 5‚8
ŠaTb. Suppose ^aTb
­ P aMß‘ b, then T is feasible if and only if ŠaTb Ág, and in this case,
X
X
Q aTb
œ ^aTb eaTb^aTb œO eaTbO,
(11)
for each O­ŠaTb; see [11] for proof. In other words, Q aTb
is well-defined by (11) whenever T is
feasible.
If T­ ’ 8 , then T is feasible as long as (10) holds. The term "feasible" is consist with that
defined in [22] for the associated semidefinite programming problem.
Definition 1. T ­P
"ß_ a
Mß’ 8 bis a solution (upper solution, lower solution) to (1) if
LQaTb€ L Q aT b€ CaTb
œ! ( Ÿ! , !ßT ) a> b œR ( R,
ŸRÑÞ
An upper or lower solution is strict if one of the inequalities is strict. Similarly, T­’ 8 is a solution
(upper, lower solution) to (2) if L QaT b€ CaTb
œ! ( Ÿ! , !, respectively).
Upper and lower solutions can be interpreted in terms of the well-definedness of the LQR
Problem (4). Let P
# 5 (same as P
# 8
a‘ b
Ya‘
b in [1]) be space of ‘ 5 -valued processes on Ò!ß_Ñ that
are adapted to the 5 -field generated by [ ab > such that I' _ #
!
ll .> _(square-integrable). Each
­ P
# a‘ 5 b is an open-loop control. We say that ­ P
# a‘
5 b is ms-stabilizing if the solution B to
equation (4.2) satisfies IelBab
> l # f Ä! as >Ä_ . The state equation (4.2) is stabilizable if there
exists a feedback matrix O­ ‘ 5‚8 such that œ OB is stabilizing, where B is the solution to
.B œ aE FObB.>€ aG HO bB.[ßBa! b œ D, (12)
which is induced from (4.2) by œ OB. In this case, O is called a stabilizing feedback matrix. A
8
solution T­ ’ to (3) is said to be stabilizing if œ ^aTbB
is stabilizing.
These concepts will be generalized for equation (2) without referring to the state equation
(4.2).
Let h Ò!ß_Ñ be the set of all ­P # a‘
5 b such that the solution B to equation (4.2) is P
# -
_
integrable; that is, Ie'
#
!
lB ab >l.> f _. Clearly N a b is well-defined for each ­hÒ!ß_Ñ.
Furthermore, the following relationship holds.
"
Proposition 1. Each ­hÒ!ß_Ñ is stabilizing. Conversely, if O­‘ 5‚8 such that œ OB
stabilizing, then ­ hÒ!ß_Ñ.
is
_
Proof. If ­hÒ!ß_Ñ, then the solution B to (4.2) satisfies Ie'
#
!
lB ab >l.> f _, which implies
#
that IelBab
> l f Ä! as > Ä_ . In other words, is stabilizing. Conversely, if œ OB is
stabilizing, then by [1, Thm 1] , there exists T­ ’ 8 , Tž! , such that _aOàTb
!, where
_aOàTb
œ aE FOb T€TaE FO b€ aG X
HOb TaG HOb, (13)
which is L aTb
with E and G replaced by E FO and G HO, respectively. By Ito's lemma and
the fundamental theorem of calculus, we have
"
X X
.
X
IeB a> " bTB> a " b f œD TD€I(
B ab >TB>.> ab
.>
>
!
>
X
X
œD TD€I( B _ aOàTbB.>
.
!
"
(14)

5
#
From (14) and the fact that IelBa> " bl f Ä! as > " Ä_ , one has that
I' _ X
X
_
!
B _ aOàTbB.> œ D TD. Since _ aOàTb
!, it follows that Ie'
#
!
lB ab >l.> f _. So
­ h Ò!ß_Ñ. ¨
Theorem 2. Suppose T­’ 8 is feasible and stabilizing.
(i) If LQ aT b ! and eaTb
!, then Na b D X TD for ­ hÒ!ß_Ñ.
(ii) If LQ aTb
Ÿ! and eaT b Ÿ !, then Na b Ÿ D X TD for ­ hÒ!ß_Ñ.
X
(iii) If LQ aTb
œ! and eaTb !(or
eaT b Ÿ !), then D TD is the minimum (maximum,
respectively) value of Na b over hÒ!ß_Ñ , which occurs when œ OB, where O­ ŠaTband
B
satisfies (12).
Proof. Suppose ­ h Ò!ß_Ñ and B is the solution to equation (4.2).
By the Fundamental Theorem of
calculus and Ito's formula, we have
"
X X
.
X
IeB a> " bTB> a " b f œD TD€I(
B ab >TB>.> ab
.>
>
!
>
X X X X X X X
œD TD€I( eB L aTbB€# aF T€HTGB€HTH b
f.>ß
!
"
X X
where L aTb
œE T€TE€G TG as defined in (6). Taking limits in (15), we obtain
_
X X X X X X X
!œD TD€I( eB L aTbB€# aF T€HTGB€HTH b
f.>
. (16)
!
Adding (16) to Na band using the notations eaTb and faTb
in (7), we obtain
_
X X X X
Na
b œD TD€I( eB aL aT b€KB€# b faTbB€ eaT b f.>
. (17)
!
Since faTb œ eaTbO for each O­ ŠaTb, we can write
# X faTbB€ X eaT bœ a€OBb X
eaT ba€OBb O X eaTb O.
Recall that LQaTb œK€ L aTb O X
eaTbO. It follows that
_
X X X
Na
b œD TD€I( ˜B LQaTbB€ a€OBb e aT ba€OB b.>Þ
(18)
!
X
In case (i), we have LQaTb 0 and e aT b !, so (18) implies that Na
b D TD for every
X
­ hÒ!ß_Ñ. Similarly, in case (ii), (18) implies that N a b Ÿ D TD for every ­ hÒ!ß_Ñ. In case
(iii), (18) implies that for every ­ hÒ!ß_Ñ,
_
X
X
N a b œD TD€I ( ˜a€OBb eaT ba€OB b.>Þ
!
X
It follows that N a b has a minimum (maximum) D TD at œ OB if eaT b !Ðif eaTb
Ÿ!Ñ.
Equation (12) is precisely the state equation (4.2) with œ OB.
¨
Note that when œ OBß the cost N a b becomes N œ ' B X
ZaOB.>
b , where
O
_
!
(15)
X X X
ZaOb
œ O VO O W W O€K. (19)

6
The following two propositions and Theorem 5 are proved in [11].
Proposition 3. Suppose T­P "ß_ 8 _ 5‚8
aMß’ b is feasible and O ­ P aMß‘
b. Let LQ aTb,
_ aOàTb
and ZaOb
be defined in (6), (13) and (19), respectively. Then
(i) LQ aT b€ a^aTb Ob X e aTba^aTb
Ob
œ Z aO b€ _ aOàTb,
(20)
(ii)
Ú LQaTb Ÿ Z aO b€ _ aOàTb, if eaT b ! (21.1)
Û LQaT b Z aO b€ _ aOàTb, if eaTb
Ÿ! (21.2)
Ü LQaTb œ Z a^aTbb€ _ a^aTbàTb. (21.3)
(21)
Proposition 4. Suppose ] ß^ ­P "ß_ aMß’ 8 b are feasible, O ­ P _ aMß‘
5‚8 b. Denote Tœ] ^,
EœE s F^a^bßGœG s H^a^b and Vœ s ea^b. Then
(i) LQ a] b LQa^b
(22)
X X X
œEs T€TE€G s s TGs
X X X €
ˆ ‰ˆ ‰ ˆ
X X
F T€HTGs V€HTH s F T€HTGs‰
(ii) If ^ is given, then ] satisfies equation (1) if and only if T œ] ^ satisfies
Ú
X
X
Ý w
T € K€ s Es T€TE€G s s TG€ s CaTb
X
€
Û ˆ X X X X X
Ý
F T€HTGs‰ˆ V€HTH s ‰ ˆ F T€HTGs‰
œ!ß
Ü T a> b œ R ^> a b,
" "
(23)
where Ks œ^ w € LQa^b€ Ca^bÞ
Note that ^ is a lower solution to (1) if and only if K s ! and R ^> a " b !, which are
equivalent to that ! is a lower solution to (23). Similarly, ^ is an upper solution to (1) if and only if !
an upper solution to (23). In other words, equation (1) has an upper or lower solutions is equivalent
to that (1) can be translated to a standard problem that has ! as an upper or lower solution.
Also note that Propositions 3 and 4 hold, in particular, for Tß]ß^­ ’ 8 and O­‘ 5‚8 Þ
Theorem 5 (upper-lower solution theorem). Suppose that a] , ^b
is a pair of upper-lower solutions
to (1).
(i) If either ea^ b ! or ea] b Ÿ! , then ] ^ . In addition, if one of ] and ^ is strict, then
] ž ^.
(ii) If either ea^ b ž! or ea] b !, then equation (1) has a unique solution T with ] T ^.
Remark 6. As noted in [11], if a lower solution ^ (if it exists) to (1) with ea^b ž! has certain
property, then an upper solution ] (if it exists) to (1) with ea]
b ! also has the corresponding
property. Same is true for equations (2) and (3). Because of this, we will state properties of upper
solutions without proof.
§3. Monotonicity of Solutions to (1) and Existence of Solutions to (2)
Now we consider equations (1) and (2), which are

w
XaT b ´ T € LQaT b€ CaTb
œ!ß Ta> " b œ R, (1)
XaT b ´ LQaT b€ C aTb
œ! . (2)
By the local theory of differential equations, equation (1) has a unique solution in a maximum interval
Ð> ! ß> " Ó. We show that this solution is monotone if and only if R is a lower or upper solution to (2);
that is, XaR b ! or XaRb
Ÿ! .
Theorem 7 (Monotonicity of Solutions) . Suppose T is the feasible solution of (1) in Ð> ! ß>
" Ó. Then we
have
(i) XaR b ! if and only if T is increasing in Ð> ! ß>
" Ó as > decreases.
(ii) XaR b Ÿ ! if and only if T is decreasing in Ð> ! ß>
" Ó as > decreases.
Proof. (i) If XaRb !, then R is a lower solution to (1). By Theorem 5, T ab > R for all
8
>­Ð> ! ß> " ÓÞ For any number 7 ­ a!ß> " > ! b, define T‡ ÀÐ> ! € 7ß> " € 7ÓÄ ’ by T ‡ ab > œT a>
7b.
Since (1) is time-invariant, T ‡a> b is a solution to ( 1 ) with T* a> " b œT a> " 7b R œTa>
" b. By
Theorem 5 again, T ‡a> b T ab > for >­Ð> ! € 7ß>
" Ó, or equivalently, T a> 7b T ab > for every
7 ­ a!ß> " > ! b. In other words, T ab > is increasing in Ð> ! ß>
" Ó as > decreases. Part (ii) is proved
similarly by using the fact that R is an upper solution.
¨
Theorem 7 implies that if T is a bounded solution to (1) on Ð _ß>Ó " with R being an upper
or lower solution to (2), then T ´ lim T ab > exists and T is a solution to (2). As a result, we have
_
>Ä _
the following necessary and sufficient existence condition for solutions to (2).
Theorem 8. Equation (2) has a solution T­ ’ 8 with e a T b ž! a eaT b ! b if and only if it has an
upper solution ] and a lower solution ^ such that ] ^ with ea^b
ž!
ae a] b !, respectively b.
Proof. The necessity is obvious by choosing ] œ ^ œT. For the sufficiency, consider equation (1)
with Rœ] and ^ respectively. Since ] is an upper solution and ^ is a lower solution (1) in
Ð _ß>" Ó, by Theorem 5, there exist solutions T] and T^ on Ð _ß>"
Ó such that T ] a> " b œ] ,
T ^ a> " b œ^ and ] T ] T ^ ^. By Theorem 7, both T] and T^
are monotone. So
]_ œ lim>Ä _ T ] ab > and ^_ œ lim>Ä _ T ^ ab > exist. Clearly ]_ and ^_
are constant solutions to
(2) satisfying ] ]_ ^ _ ^. If ea^b ž! then ea] _ b ea^_ b ž! . If ea] b ! then
!žea]_
b ea^_ b. ¨
In fact, ]_
and ^_
are the maximal and minimal solutions of (2) in the "interval"
c^ ß] d œ eT­ ’ 8 ß^ŸTŸ] f, respectively. Indeed, if Q­ c^ ß]
d is a solution to Ð2, Ñ then
] T ab > Q T ab > ^, which imply that ] Q ^ .
] ^ _ _
We now define stabilizability and detectability for equation (2). Since equation (2) may arise
from problems with different state equations, we will avoid using the state equations in our definitions
of stabilizability and detectability.
Definition 2. Suppose EßFßGßHßKß C are as in (5) and T­ ­ ’ 8 . We define the following
terms.
_
7

aEßGßCb
is ms-stable if there exists Y ­ ’ 8 ß Y ž! such that
X
X
L aYb€ CaYb œE Y € YE€G YG€ CaYb
!Þ
(24)
In other words, aEßGß Cb is ms-stable if L aTb€ CaTb
œ! has a strict upper solution Yž! .
aEßFßGßHßCb
is ms-stabilizable if there exists O­‘ 5‚8 such that
aE FOßG HOßC b is ms-stable, or equivalently, _ aOàT b€ CaTb
has a strict upper solution
Yž! . Such a matrix O is also called an ms-stabilizing feedback matrix.
Š È KßEßGß C ‹ is ms-detectable if KœJ
X J for some J­‘
;‚8
and there exist
;
Q" ßQ# ­V 8‚ such that aE Q" JßG Q#
JßCb
is ms-stable.
8
T­ ’ is ms-stabilizing if aE F^aTbßG H^aTbß Cb
is ms-stableÞ
As proved in [1, Theorem 1], the ms-stabilizability of aEßFßGßHß!
b is equivalent to the
existence of a matrix O­ ‘ 5‚8 such that the solution B to (12) satisfies IelBab
> l # f Ä! as >Ä_ ;
see the paragraph before Proposition 1. When GœHœ! , the ms-stabilility, ms-stabilizability and
ms-detectability defined here are equivalent to those defined in [5, Definitions 3.1 and 3.2].
Assuming that aEßFßGßHß C b ms-stabilizble, we show that (2) has an upper or lower
solution and we obtain a refined necessary and sufficient condition for solutions to (2).
Theorem 9. Suppose aEßFßGßHß Cb
is stabilizable.
(i) Equation (2) has a solution T with eaTb ž! if and only if it has a lower solution ^ with
eaZ b ž! .
(ii) Equation (2) has a solution T with eaT b ! if and only if it has an upper solution ] with
ea] b !.
Proof. (i) The necessity is trivial. For sufficiency, suppose that aEßFßGßHß Cb
is ms-stabilizable and
that ^ is a lower solution with ea^b ž! . Then _ aOà Yb€ CaY b ! for some O­‘ 5‚8 and
Y ž! . Let ]œ ! Y. Choose an ! ž! such that ] ^and
_ aOà] b€ Ca] b€ ZaO b !ß
where ZaO b is defined in (19). It follows that ea]
b ea^b
ž! . By Proposition 3 (ii) with Tœ] ,
we have that
LQa] b€ C a] b Ÿ _ aOà] b€ Ca] b€ Z aOb
!Þ
That is, ] is a strict upper solution to (2). By Theorem 8, (2) has a solution in c^ß]
d. The proof of
(ii) is similar or it follows from Remark 6. ¨
8
§ 4. Comparison, Uniqueness, Stabilizability and Approximation of Solutions to (2)
Now we consider the uniqueness, ms-stabilizability and approximation of solutions to (2). We
first prove some equivalent descriptions of the ms-stability.
Theorem 10.
(i) The following are equivalent.
(a) aEßGßCb
is ms-stable.

8 8
(b) Re -aE b ! and < 5a7b ", where 7 : ’ Ä ’ is defined as 7aY
b œ
_
' E> E>
!
/ X
ÐG X
YG€ C aYbÑ/ .>.
(c) LaYb€ C aYb€ K œ! has a unique solution for every K ­ ’ 8 . If K ! then Y !
and if K ž! then Yž! .
(ii) Suppose aEßGßCb is ms-stable and aYßZb
is a pair of upper and lower solutions to
L aT b€ CaT
b€Kœ! , then Y ZÞ
Proof. (a) Ê (b). Suppose Y ­ ’ 8 ß Y ž! satisfies L aY b€ CaY b !. Let
L œ cL
aYb€ CaYbd
ž! , then we have
X
X
E Y € YE€G YG€ CaYb € L œ! . (25)
In particular, E X
Y € Y E !, which implies that E must be stable in the sense that all eigenvalues of
E have negative real parts; that is, Re-aE b !. Now (25) can be rewritten as
_
X
Y œ ( / Ð G X YG€
CaYb€ LÑ/ .> œ 7 aYb€ Qß
!
E> E>
_ X
where Q œ ' E> E> 5€"
/ L/ .> ž!Þ It follows that for all integers 5 !, Y œ 7 aYb€0 aQb,
!
5
3
where 0 aQ b ´ !7 aQb. Since Q ž! , 0 aQb is an increasing sequence and 0 aQb
ŸY.
5 5 5
3œ!
_
0 _ a Q b ´ ! 7 3a Q b 0_
a Q b œ YÞ T­’
8
3œ!
€ „ _ _
_
_ a b _ a 3
€ b _ a b - 7 !-
3œ!
Therefore, the series converges and Since each can be
written as TœT T where T ! and 0 is linear, 0 is well-defined for all T­’ 8 by
0 T œ0 T 0 T . Now if is an eigenvalue of , then must converge to an
eigenvalue of 0. Therefore, l- l " and so < 5a7 b ".
_
(b) Ê (c) Suppose V/ -aE b ! and < a7b
". Then 0 a T b œ ! 7 3a
T b is defined for every
_
5 _
X
8 E> E>
T ­ ’ . Take T œ '
!
/ K/ .>. It is easily checked that Y œ0aTb satisfies that Y œ T € 7aYb,
which is equivalent to L aYb€ CaYb€ K œ! . Since any solution to L aYb€ CaYb€ K œ! is
_ E> E>
represented as 0_
aTb , where T œ ' / X
!
K/ .>, the solution has to be unique. If K ! or ž! ,
then T ! or ž! , which implies that Y ! or ž! , respectively.
(c) Ê (a).Let K œI , then L aYb€ CaYb€Iœ! has a solution Y ž!Þ So aEßGßCb
is msstable.
This finishes the proof of (i).
To prove (ii), consider TœY Z . Then T satisfies L aT b€ CaT
b€ L œ! for some
L !. Part (i.c) implies that Y Zި
Now we can prove the following comparison result for (2).
Theorem 11. Suppose that ] is an upper solution and ^ a lower solution to equation (2). Then
] ^ if either (i) ] is ms-stabilizing and ea^b ž! , or (ii) ^ is ms-stabilizing and ea]
b !.
Proof. Suppose ] is ms-stabilizing and ea^ b ž! . Let Tœ ] ^ and L œ Xa^ b Xa] b !.
Then L œ CaT b€
LQa]
b LQ a^Þ
b By Proposition 3 (i),
3œ!
5
9

10
Lœ CaT b€
LQa]
b LQa^b
œ
X
X
CaT b€
_ aOàTb
a^a] b O b ea] ba^a] b O b€ a^a^b Ob ea^ba^a^b
Ob,
(26)
where _aOàTb
œ aE FOb X
X
T€TaE FO b€ aG HOb TaG HOb
Setting Oœ^a]
b in (26), we get
as defined in (13).
X
_ a^a] bàT b€ CaT b€a^a^b ^a] bb ea^ba^a^ b ^a] bb€ L œ! .
Since ea^ b ! and L !, T is an upper solution to _ a^a] bàT b€ CaTb
œ!. By assumption,
aE F^a] bßG H^a] bß
Cb
is ms-stable. Therefore, we can apply Theorem 10 (ii) to the
equation _ a^a] bàT b€ CaTb
œ! to conclude that T !; that is, ] ^Þ
The proof (ii) is similar
or it follows from Remark 6.¨
8
Denote by m€
( m ) be set of all solutions T­ ’ to (2) with eaTb
ž! ( !, respectively).
By Theorem 11, if T­ m€ is ms-stabilizing, then T X for every X­m€
. It follows that the msstabilizing
solutions in m€
must be maximal and so unique. Similarly, ms-stabilizing solutions in m
are minimal and unique. Therefore, we have the following uniqueness of ms-stabilizing solutions.
Proposition 12.
(i) If T­ m€ is ms-stabilizing, then T is maximal in m€
and T is the unique ms-stabilizing solution
in m € .
(ii) If T­ m is ms-stabilizing, then T is minimal in m and T is the unique ms-stabilizing solution
in m .
Now we consider the question on the
linear equation associated with (2):
and its generalization with O ­
ms-stabilizability of solutions to (2). First consider the
X
X
L aT b€ CaT b ´E T€ TE€G T G € CaT
b€Kœ!Þ
(27)
‘ 5‚8
where _aOàTb
is defined as in (13). We have
Theorem 13. Suppose that
À
X
_ aOàT b€ CaT b€K€ O VOœ!
, (28)
Š È KßEßGßC ‹
is ms-detectable.
(i) If (27) has an upper solution T !, then aEßGßCb
is ms-stable.
5‚8 8
(ii) More generally, if for some O ­V and V­ ’ with Vž! , (28) has an upper solution
T !, then aE FOßG HOßC b is ms-stable.
Proof. (i) The ms-detectability of Š È KßEßGßC ‹ implies that KœJ X J for some J­ ‘;‚8
and
8‚ ;
there exist Q ßQ ­V such that aE Q JßG Q JßCb
is ms-stable; that is,
for some
" # " #
X
X
aE Q J b Z € Z aE Q J b€ aG Q Jb Z aG Q J b€ C aZ
b ! (29)
" " # #
! Z ­ ’ 8 . Expanding (29) we obtain
X X X X X X X
L aZ b€ CaZ b aJ Q Z€Z Q J€J Q ZG€G Z Q J b€J Q Z Q J !Þ (30au)
" " # # # #

X X X X
where L aZ b œE Z € Z E€G Z G. With J Q# Z Q#
J 0 dropped, (30) still holds. It follows
that for some & ž! ,
L aZ b€ C aZ b aJ X Q X Z€Z Q J€J X Q X ZG€G X Z Q J b€ % Z€ % G X ZG !.(31)
" " #
#
Let + be the largest eigenvalue of aQ Z Q € Q Z Q bÎ
X X
" " # # & #
. Then one has
# #
X X X X
J aQ Z Q" € Q Z Q# bJÎ& # Ÿ+J JŸ + cL aT b€ CaTbd, (32)
" #
where the last inequality is just (27). Define [ œ+T €Z . Then [ž!. Combining (31) and (32),
we obtain that
La[ b€ Ca[ b œ+ aLaT b€ CaT bb€
LaZ b€ CaZ
b
# X X # X X X X X
& Z J Q Z Z J€ G Z J J ZG€+J J‘
" Q" & G ZG Q#
Q#
X
X
Ÿ a& I Q JÎ& b Z a& I Q JÎ & b€
a& G Q JÎ& b Za& G Q JÎ&
b‘
Ÿ !Þ
" " # #
This shows that aEßGßCb
is ms-stable.
J
(ii) Let Y œ ” . Then Y Y œ
By assumption,
V
O
•
J X J€ O VO. aE Q" JßG Q# JßCb
is ms-stable for some Q" and Q# Þ Let `" œ Q" ß FV
"Î#‘ and `# œ Q#
ß HV
"Î#‘. Then
we have that
E FO ` Y œE Q JßG HO ` Y œG Q J.
" " # #
So aE FO `" YßG HO `# YßCb œ aE Q" JßG Q#
JßCb, which is ms-stable. In
other words, ˆ ÈYXYßE FO ßG HOßC‰
is ms-detectable. By (i) applied to (28),
aE FOßG HOßC b is ms-stable. The proof (ii) is similar or it follows from Remark 6.¨
Remark 3. In Theorem 13, a sufficient condition for Š È KßEßGßC ‹ to be ms-detectable is that
#
Kž! . Indeed, let Jž! such that KœJ and ! ­ a!ß_ b such that ! I CaIÎ#
b . Take
" "
Q" œ aE ! IJ b and Q#
œ GJ . Then the left-hand side of (29) with Z œI becomes
#! I€ CaI b !. So Š ÈKßEßGßC‹
is ms-detectable.
Now we prove the ms-stabilizability of solutions to (2).
Theorem 14. Suppose a] ß ^ b is a pair of upper-lower solutions to (2) and ] ^.
(i) If ea^b ž! and ˆ ÈXa^ bßE F^a^bßG H^a^bßC‰
is ms-detectable, then ] is ms-
stabilizing.
(ii) If ea] b ! and ˆ È Xa] bßE F ^a] bßG H ^a]
bßC‰
is ms-detectable, then ^ is ms-
stabilizing.
In both cases, (2) has a unique solution T in c^ ß]
d and T is ms-stabilizing.
Proof. (i) We first assume Wœ! and ^ œ! . The assumptions imply that K !ß Vž! and
Š È KßEßGß C ‹ is ms-detectable. By (20) with Oœ ^ a T b , equation (2) is equivalent to
_^ a aTbàT b€ CaT b€ Z^ a aTbb
œ!. (33)
11

X
Since Wœ! , Z^ a aTbb œK€ ^aTb V^aTb. So (33) is precisely (28) with Oœ ^aTb.
Because ]
is an upper solution to (2) and so it also an upper solution to (33), Theorem 13 (ii) implies that
aE F ^a] bßG H ^a]
bßC b is ms-stable; that is, ] is ms-stabilizing. For the general case,
consider T œ] ^. Then by (23), T satisfies equation (2) is equivalent to
Ú
X
X
Ý K€ s Es T€TE€G s s TG€ s CaTb
X
€
Û ˆ X X X X X
Ý
F T€HTGs‰ˆ V€HTH s ‰ ˆ F T€HTGs‰
Ÿ!ß
Ü T a> b œ R ^> a b !,
" "
where EœE s F^a^bß GœG s H^a^b, Kœ s LQ a^ b€ Ca^ b ! and Vœ s ea^b
ž! . The
assumptions imply that (34) has a lower solution ! with ms-detectable Š È Kß s EG s ß s ß C ‹. This is
precisely the case we just proved with EßGßKßV replaced by EßGßKßV s s s s , respectively. Therefore, T
is ms-stabilizing in the sense that Š Es F^saT bßGs H^saT bßC‹
is ms-stable, where
"
^s aTb œ ˆ V€HTH s X ‰ ˆ X X
F X€HTG s‰ . Since ea] b ea^b
ž! , it is directly checked that
^s aTb œ^a] b ^a^ b; see [11, proof of Proposition 5]. It follows that
aE F ^a] bßG H ^a] bßCb œ Š Es F^saT bßGs H^saT bßC‹
is ms-stable. So ] is ms-stabilizing.
In particular, all solutions of (2) in c^ ß ] d are ms-stabilizing and
so they must be unique by Proposition 12. The proof (ii) is similar or it follows from Remark 6.
¨
By Remark 3, the ms-detectability condition in Theorem 14 hold if ^ is a strict lower solution (i.e.,
Xa^ b ž! ) or ] is a strict upper solution ( Xa]
b !). Thus we have the following corollary.
Corollary 15. Suppose a] ß ^ b is a pair of upper-lower solutions and ] ^.
(i) If ea^b ž! and Xa^ b ž! , then ] is ms-stabilizing.
(ii) If ea] b ! and Xa] b !, then ^ is ms-stabilizing.
In both cases, (2) has a unique solution in c^ ß ] d and it is ms-stabilizing.
Finally we show that the stabilizing solution to (2) can be approximated by solutions to linear
equations.
Theorem 16. Suppose a] ß ^ b is a pair of upper and lower solutions such that ] ^.
(i) Suppose ea^b ž! and ˆ ÈXa^bßE F^a^bßG H^a^bßC‰
is ms-detectable. Define
L œ ] and L be the unique solution to
" 3€"
12
(34)
_^ a aL3bàT b€ Z^ a aL 3 bb € CaTb
œ!
(35)
for 3 "Þ Then L L â ^ and Lœ lim L is the unique ms-stabilizing solution to (2)in
" # 3
3Ä_
c^ ß ] d.
(ii) Suppose ea] b ! and ˆ È Xa]
bßE F ^a] bßG H ^a]
bßC‰
is ms-detectable. Define
L œ ^ and H be the solution to (35) for 3 "Þ Then L ŸL ŸâŸ ] and Lœ lim L is
" 3€" " # 3
3Ä_
the unique ms stabilizing solution to (2) in c^ ß ] d.

Proof. (i) We first show that L 3€" ^. Since ^ is a lower solution with ea^ b !. By (21.1) with
Tœ ^ and Oœ ^aL
b, we have
3
!Ÿ LQa^b€ C a^b
Ÿ _^ a aL bà ^b€ Z^ a aL bb€
Ca^b
3 3 .
It follows that aL3€" ß^b
is a pair of upper-lower solution to (35). By Theorem 10 (ii), L 3€" ^.
Consequently, L3€" is a solution to (35) with eaL 3€" b ea^b
ž! . By (21.1) with TœL3€"
and
Oœ ^aL
3 b, we have
LQaL b€ C aL b Ÿ _^ a aL bàL b€ Z^ a aL bb € CaL b œ!Þ
3€" 3€"
3 3€" 3 3€"
So L3€" is an upper solution to (2). By Theorem 14, L3€"
must be ms-stabilizing.
Next we show that L L . By (20) with TœL, we have
3 3€" 3
! LQaL b€ C aL b œ _^ a aL bàL b€ Z^ a aL bb € CaL bÞ
3 3
3 3 3 3
In other words, aLßL
3 b is a pair of upper-lower solutions to (35). By Theorem 10 (ii) again,
3€"
L 3 L3€".
Now it is clear that the limit Lœ lim L exists and satisfies (2). By Theorem 14, L is
3Ä_ 3
stabilizing. The proof of (ii) is similar or it follows from Remark 6.
¨
References
[1] M. Ait Rami and X.Y. Zhou, Linear Matrix Inequalities, Riccati Equations, Indefinite
Stochastic Linear Quadratic Controls, IEEE, Transactions on Automatic Control, Vol. 45,
No.6 (2000) 1131-1143.
_
[2] T. Basar and P. Bernhard, H -Optimal control and related minimax design problems,
Birkhauser, 1995.
[3] S. Bittanti, A.J. Laub, and J.C. Willems, The Riccati equations, Springer, 1991.
[4] R. W. Brockett, Finite dimensional linear systems, J. Wiley, New York, 1970.
[5] M.D. Fragoso, O.L.V. Costa and C.E. de Souza, A new approach to linearly perturbed
Riccati equations arising in stochastic control, Applied Mathematics & Optimization, 37
(1998) 99-126.
[6] G. Freiling and G. Jank, Existence and comparison theorems for algebraic Riccati equations
and Riccati differential and difference equations, J. Dynam. Contr. System, 2 (1996) 529-547.
[7] I. Gohberg, P. Lancaster, and L. Rodman, On Hermitian solutions of the symmetric algebraic
Riccati equations, SIAM J. Contr. Optimiz. 24 (1986), 1323-1334.
[8] V. Ionescu, C. Oara and M. Weiss, Generalized Riccati theory and robust control, A Popov
Approach, John Wiley & Sons, 1999.
[9] P. Lancaster and L. Rodman, The Algebraic Riccati equations, Oxford University Press,
Oxford, 1995.
[10] M. McAsey and L. Mou, Riccati Algebraic and differential equations from stochastic
differential games. In preparation.
[11] L. Mou, Upper-Lower Solution Method for Differential Riccati Equations from Stochastic
LQR Problems. Preprint, 2001.
[12] L. Mou and S. R. Liberty, Estimation of Maximal Existence Intervals for Solutions to a
Riccati Equation via an Upper-Lower Solution Method. To appear on the Proceedings of
Thirty-Nineth Allerton Conference on Communication, Control, and Computing, 2001.
13

[13] R. Penrose, A generalized inverse of matrices, ., 52 (1955), 17-
Proc. Cambridge Philos. Soc
19.
[14] A.C.M. Ran and R. Vreugdenhil, Existence and comparison theorems for algebraic Riccati
equations for continuous- and discrete-time systems, Linear Algebra Appl. 99 (1988), 63-83.
[15] W.T. Reid, Riccati differential equations. Academic Press, New York and London, 1972.
[16] C. Scherer, The solution set of the algebraic Riccati equation and the algebraic Riccati
inequality. Linear Algebra and its applications, 153 (1991) 99-122.
[17] D.D. Sworder, Feedback control of a class of linear systems with jump parameters, IEEE
Trans. Automat. Contr., Vol. AC-14 (1969) 9-14.
[18] J. C. Willems, Least squares stationary optimal control and the algebraic Riccati equation.
IEEE Trans. Automatic Control AC-16 (1971), 621--634.
[19] C. W. Wimmer, Monotonicity of maximal solutions of algebraic Riccati equations, Syst.
Contr. Lett. 5 (1985), 317-319.
[20] H.K. Wimmer, A Galois correspondence between sets of semidefinite solutions of continuoustime
algebraic Riccati equations, Linear Algebra and its Applications, 205-206 (1994) 1253-
1270.
[21] W.M. Wonham, On a matrix Riccati equation of stochastic control, SIAM J. Control, Vol. 6,
No.4 (1968) 681-697.
[22] D.D. Yao, S. Zhang and X.Y. Zhou, Stochastic linear-quadratic control via semidefinite
programming, SIAM J. Control and Optimizations 40: 813-823 (2001).
[23] J. Yong and X.Y. Zhou, Stochastic Controls, Hamiltonian Systems and HJB Equations,
Springer 1999.
[24] K. Zhou with J. Doyle and K. Glover, Robust optimal control, Prentice Hall 1995.
14