Sum-of-Squares Applications in Nonlinear Controller Synthesis

NomenclatureGreek Small Lettersδγuncertainty boundL 2 gain boundIndicesdfeasILQRudisturbancebased on the feasibility problembased on the identitiy matrixbased on the linear quadratic regulatorinputMiscellaneous CharactersA ≻ (≽)0A ≺ (≼)0minA is positive (semi)definiteA is negative (semi)definiteb a minimize the objective a over the parameter b∇A gradient of A∅∀∃the empty setfor allthere existsNotationxxXẋx TX −1‖x‖‖x‖ 2x ∗scalarvectormatrixtime derivativetransposeinverseEuclidian norm of xL 2 norm of xoptimizeriii

Introduction1 IntroductionControl Lyapunov functions introduced by Artstein and Sontag [1, 20] in the 80’s offera valuable source for systematic design of controllers for nonlinear systems. They extendthe classical Lyapunov theory to systems with inputs, making it directly applicable in thedesign process. Moreover, by use of differential geometric concepts, it is possible to derivean explicit universal feedback law, known as the Sontag formula, once a control Lyapunovfunction for a system is found.Interestingly, a slightly modified version of this feedback law was shown to be asolution to the infinite time optimal control problem by Freeman and Kokotovic [6]. Theirnotion of inverse optimality can consequently be seen as a natural extension of the linearquadratic regulator theory to nonlinear systems. The extension of optimal control designtechniques to nonlinear systems is tempting, recalling that since Kalman [11] solved theLQR problem in the 60’s, optimal pole-placement has proved useful for actual application.Up to today, it is a useful starting point for the design of regulatory controllers. Reasons forthis are the systematic way of tuning, the guaranteed stability and good classical robustnessmargins that are associated with the method.One of the problems with the Sontag formula so far was to find suitable controlLyapunov functions, as no general analytical method is developed yet. The sum-of-squaresmethodolgy yields an approach to this issue for polynomial systems and provides a practicalway of finding control Lyapunov functions. Polynomial systems are an important class ofnonlinear systems and can be seen as a logical next step to extend theoretical results to. Theyarise wherever empirically obtained data is fit by polynomials, e.g., to obtain aerodynamiccoefficients for aircraft. Furthermore, it is often possible to turn a general nonlinear systeminto polynomial form with a change of variables. Last but not least, Taylor’s theorem orleast square regression allows to approximate every function by polynomials.Restricting polynomials to be a sum-of-squares implies their positive semidefinitenessand is a condition that can be computationally verified with semidefinite programming,a still growing discipline of convex optimization that was started by the introduction ofefficient interior point methods in the 90’s. Several software tools [13, 14, 16] are availableand can be employed in the process, providing a promising framework for the computeraided search for control Lyapunov functions.A reason for our interest in the topic is that nonlinear control is still widely disregardedfor actual application. Despite the existence of theoretically well established techniques, theoverwhelming majority of controllers used in industry is still designed using linear methodology.Regarding that most real world systems are nonlinear in nature, validity of linearapproximations in the design is often questionable and leads to a loss of potential perfor-1

Introductionmance. Arguments against the use of available nonlinear design techniques include thatthey usually require full state access. However, with the ongoing development in microelectromechanicalsystems, more sensors become cheaply available and consequently enablemeasurement of all states. The importance of state feedback for application is thereforelikely to further increase in the near future.In this research project, the application of sum-of-squares programming to the taskof regulator design for polynomial systems is investigated. A static state feedback law basedon the Sontag formula is introduced and shown to resemble LQR control for linear systems.The method provides easy means of tuning and still produces controllers that comeinherently with a Lyapunov certificate, anticipating a separate model based validation. Furtherextensions to ensure robustness for bounded parameter uncertainty and to guaranteedisturbance attenuation properties in terms of the L 2 norm are outlined.2

Control Lyapunov Functions and Sum-of-Squares2 Control Lyapunov Functions and Sum-of-SquaresThroughout this text, polynomial control affine dynamic systemsẋ(t) = f(x(t)) + g(x(t)) u(t)(1)with x(t) ∈ R n , f(t) ∈ R n , g(t) ∈ R n × R nu , u(t) ∈ R nuwith possibly multiple inputs are considered. From this point on, the time dependence isimplicitly assumed and therefore omitted. For historical reasons, the fundamental resultspresented in this chapter are given in a formulation for single input systems, i.e., n u = 1.They extend naturally to the multi input case that is considered from chapter 3 on. LiederivativesL f V def = ∇ V · f (2)are used for notational convenience whenever derivatives along trajectories of a system aredenoted.2.1 Artstein-Sontag TheoremThe conceptual important foundation for our approach to nonlinear controller synthesis isTheorem 1: Artstein-Sontag Theorem.A system ẋ = f(x) + g(x) u is globally asymptotically stabilizable by a statefeedback law u(x) if and only if there exists a positive definite, radially unboundedscalar function V (x) with()minu L f V + L g V u < 0 ∀x ≠ 0 .V is then called a control Lyapunov function (CLF).The theorem is an extension of classical Lyapunov theory to systems with inputs. It wasstated by Artstein [1] and was proved in a constructive way by Sontag [20] using the feedbacklaw⎧⎪⎨ 0 if L g V = 0( √ )u =( ) 2 ( ) 2⎪⎩ − 1 L f V + L f V + L g V if L g V ≠ 0 ,L g V(3)which is now known as the Sontag formula. This control law is continuous everywhere,except possibly at the origin. It requires full state access and results in global stability as3

2.1 Artstein-Sontag Theorem Control Lyapunov Functions and Sum-of-SquaresV becomes a Lyapunov function for the closed loop system:˙V = L f V + L g V u⎧⎪⎨ L f V < 0 if L g V = 0 , x ≠ 0= ( ) 2 ( ) 2 ⎪⎩ −√L f V + L g V < 0 if L g V ≠ 0 , x ≠ 0 .(4)It is easy to verify that the condition of theorem 1 reduces to the question of whether thereexists a suitable V withL f V < 0 ∀x ≠ 0 such that L g V = 0 , (5)as for any L g V ≠ 0 the inequality can be satisfied by some choice of u. This can looselybe interpreted as “the Lyapunov function being accessible from the input, wherever itsdynamics are unstable”. Once a control Lyapunov function is found, the Sontag feedbackformula (3) can be used to guarantee global stability via state feedback. The design taskthus reduces mainly to finding a CLF. 1As an introductory example of CLFs, the special case of single input linear systemsẋ = A x + B u (6)is presented. To show global stability, we consider a quadratic Lyapunov functionV = x T P x , P ≻ 0 . (7)The time derivative along the closed loop system’s trajectories is˙V = (A x + B u) T P x + x T P (A x + B u)( )= x T A T P + P A x +}2 x T {{P B}u . (8)} {{ } L g VL f VThe first term (L f V ) describes the system dynamics without input and is in fact the classicalLyapunov inequality, as the autonomous system is stable if and only if A T P + P A ≺ 0.The control term relaxes this requirement, as the autonomous system has to be stable onlyin those regions of the state space where the input does not affect ˙V .1 It is noteworthy that for any control Lyapunov function V , the scaled function α V with α ∈ R + is alsoa CLF and that the Sontag formula yields the same feedback law for all these scaled CLFs.4

2.1 Artstein-Sontag Theorem Control Lyapunov Functions and Sum-of-SquaresThis is exactly what condition (5) states and consequently V is a control Lyapunov functionif and only if( )x T A T P + P A x < 0 ∀x ≠ 0 such that 2 x T P B = 0 (9)To verify this condition let N be a basis for the left Nullspace of B, i.e., N T B = 0. Thenand thus(ξ T N T ) B = 0∀ξx T P = ξ T N T ⇒ x = P −1 N ξ .With this, condition (9) becomes( )ξ T N T P −1 A T P + P A P −1 N ξ < 0 ∀ξ (10)and we can stateLemma 1: Control Lyapunov Functions for Linear Systems.V = x T P x is a control Lyapunov function for the linear system ẋ = A x+B uif and only if there exists a symmetric positive definite matrix P such thatN T P −1 A T N + N T A P −1 N ≺ 0 ,where N is a basis for the left Nullspace of B.Thus a criterion to check whether a given candidate function is a control Lyapunov functionfor linear systems is found.One manner to parametrize a whole family of CLFs is to introduce the free parameterQ ≻ 0 and obtain the matrix P as the symmetric positive definite solution of the algebraicRiccati equationA T P + P A = P B B T P − Q . (11)To shows that V is indeed a CLF for all positive definite matrices Q, we substitute equation(11) into equation (8). The time derivative of V then becomes˙V ( )= x T P B B T P − Q x + 2 x T P B u()= −x T Qx + x T P B B T P x + 2 u ,(12)5

2.2 Sum-of-Squares Programming Control Lyapunov Functions and Sum-of-Squareswhich satisfies the condition of theorem 1 asminu ˙V =⎧⎨−x T Qx if x T P B = 0⎩−∞ if x T P B ≠ 0 .(13)We introduce this concept not only for its own interest but also to use a similar approach inchapter 3 to parametrize a controller for nonlinear systems. Furthermore, the parametrizationpresented here is directly applicable to nonlinear systems that are input-to-state feedbacklinearizable. As for arbitrary polynomial systems no systematic method of finding CLFsis yet discovered, we use the relatively recent approach of sum-of-squares programming [22].2.2 Sum-of-Squares ProgrammingWe start with a short review on polynomials and semidefinite programming to familarizethe reader with the basic ideas. The application of sum-of-squares programming to findcontrol Lyapunov functions is covered in section 2.3.First note that a monomial of n variables is a function of the form{}z(x 1 , x 2 , . . . , x n ) ∈ M (x 1 , x 2 , . . . , x n ) = x α 11 xα 22 · · · x αnn ∣ α i ∈ N 0 . (14)Equivalently, the short notation z(x) will be used to indicate a monomial in the componentsof a vector x. Furthermore, a vector representation will be convenient, ordering the entrieslexically:z(x) =[1 x 1 x 2 · · · x n x 2 1 x 1 x 2 · · · x 1 x n · · · x 2 1 x 2 · · ·] T. (15)A polynomial is then a finite linear combination of monomials with real coefficientsp(x 1 , x 2 , . . . , x n ) = ∑ ic i z i , c i ∈ R , z i ∈ M(x 1 , x 2 , . . . , x n ) , (16)which can also be written in vector notation asp(x 1 , x 2 , . . . , x n ) = c z , c ∈ R 1×m , z ∈ M m .The space of polynomials will be denoted by P and the space of sum-of-squares polynomialswill be defined as{S defn }= p ∈ P∣ p = ∑qi 2 , q i ∈ P, i = 1, . . . , n . (17)i=16

2.2 Sum-of-Squares Programming Control Lyapunov Functions and Sum-of-SquaresIt is obvious that if p ∈ S, then p is positive semidefinite everywhere. As the converseis not true, replacing the requirement of a polynomial to be positive semidefinite with asum-of-squares constraint is a relaxation to sufficient conditions.In general, sum-of-squares problems, i.e., the question whether a given polynomialis a sum-of-squares polynomial, can be solved computational tractably via semidefiniteprogramming. This is due toTheorem 2: Sum-of-Squares Decomposition.Given a polynomial p of degree 2d in the variables x, p is a sum-of-squares ifand only if there exists a symmetric positive semidefinite matrix Q such thatp = z T Q zwith z being the vector of all monomials of x up to degree d. Q is then calledthe Gram matrix of the sum-of-squares polynomial p.This result was first obtained by Choi et. al. [5] and shown by Parillo [15] to be a semidefiniteprogramming problem. Note that Q is not unique since the monomials are not algebraicallyindependent.Semidefinite programs can be seen as a generalization of linear programs. The goalis to minimize a linear function, subject to the constraint that an affine combination ofsymmetric matrices is positive semidefinite.min c T yr∑subject to: Q 0 + Q k y k ≽ 0k=1(18a)(18b)These constraints are equivalently known as linear matrix inequalities (LMI) in the controlcommunity, although they are usually nonlinear (see Boyd et. al. [4] for an in-depth survey).They are however always convex and efficient solvers such as SeDuMi are freely available.The computational method to check whether a given polynomial is a sum-of-squares isto use a parametric sum-of-squares polynomial in the monomials of the function in question.With theorem 2, this means to introduce a parametric Gram matrix. The entries are thenconstrained to match the coefficients of the given polynomial. A sum-of-squares constraintthus is equivalent to a set of equality constraints and a linear matrix inequality constraint.If the problem is feasible, the solution is a sum of squares decomposition, if it is infeasible,the polynomial in question is not a sum-of-squares.7

2.3 Sum-of-Squares Relaxation for CLFs Control Lyapunov Functions and Sum-of-Squares2.3 Sum-of-Squares Relaxation for Control Lyapunov FunctionsRecall that the conditions for a control Lyapunov function are the positive definiteness ofthe function itself and the negative definiteness of L f V wherever L g V = 0. For polynomialsystems and polynomial control Lyapunov functions, the question whether a given functionis a CLF thus evolves around the positiveness of polynomials at values of x where anotherpolynomial’s value is zero.The important result used to assess positivity of a polynomial wherever another polynomialhas certain properties is taken from real algebraic geometry [3] and isTheorem 3: Positivstellensatz.The empty set condition⎧ ∣ ∣∣∣∣∣∣∣⎪⎨ q 1 ≥ 0, . . . , q nq ≥ 0x ∈ R n r 1 ≠ 0, . . . , r nr ≠ 0⎪⎩t 1 = 0, . . . , t nt = 0⎫⎪⎬= ∅⎪⎭is equivalent to the existence of polynomialsq ∈ C ( q 1 , . . . , q nq)r ∈ M (r 1 , . . . , r nr )t ∈ I (t 1 , . . . , t nt )such that q + r 2 + t = 0.The sets used in the theorem are the Multiplicative Monoid, Cone and IdealM (r 1 , . . . , r nr ) ={r k 11 rk 22 · · · r knrn r∣ }∣∣∣k 1 , k 2 , . . . , k nr ∈ NC ( ∣)l∑ ∣∣∣q 1 , . . . , q nq ={s 0 + s i b i l ∈ N, s i ∈ S, b i ∈ M ( ) } q 1 , . . . , q nq ,i=1{∑ nt ∣ }∣∣∣I (t 1 , . . . , t nt ) = t k p k p k ∈ P .k=1Observe, that for single polynomial constraints, i.e., n q = n r = n t = 1, these sets become,M (r) =C (q) =I (t) ={ ∣ } ∣∣∣r k k ∈ N{s 0 + s 1 q,{t p,}∣ s 0, s 1 ∈ S}∣ p ∈ P .,8

2.3 Sum-of-Squares Relaxation for CLFs Control Lyapunov Functions and Sum-of-SquaresThus theorem 3 says that the empty set condition⎧ ∣ ∣∣∣∣∣∣∣⎪⎨ q ≥ 0x ∈ R n r ≠ 0⎪⎩t = 0⎫⎪⎬= ∅⎪⎭is equivalent to the existence of polynomials s 0 , s 1 ∈ S and p ∈ P such thats 0 + q s 1 + r 2k + t p = 0 ,k ∈ NTo use this result, condition (5) has to be formulated as the empty set condition{ ∣ }∣∣∣x ∈ R n L g V = 0 , L f V ≥ 0 , x ≠ 0 = ∅ . (19)Furthermore, V being a radially unbounded positive definite function requiresV (x) > 0 ∀x ≠ 0 , V (0) = 0 and ‖V ‖ → ∞ as ‖x‖ → ∞ . (20)The first condition can be equivalently stated as the empty set condition{ ∣ }∣∣∣x ∈ R n − V ≥ 0, x ≠ 0 = ∅ . (21)The second can be guaranteed by not allowing constant terms in V and the third holdstrivially for any polynomial V that satisfies the other conditions.With the essential idea of replacing x ≠ 0 with the polynomial constraint l(x) ≠ 0,l ≻ 0, the empty set conditions (19) and (21) are by theorem 3 equivalent to the existenceof polynomials s 0 , s 1 , s 2 , s 3 ∈ S and p, l 1 , l 2 ∈ P, l 1 , l 2 ≻ 0 with k 1 , k 2 ∈ N such that0 = s 0 − s 1 V + l 2k 11 ,0 = s 2 + s 3 L f V + l 2k 22 + p L g V .We follow the lines of Tan and Packard [21, 22] to further simplify the problem in order toapply computational methods. By fixing k 1 = 1, s 1 = l 1 and s 0 = ŝ 0 l 1 the first equation isreduced tol 1 (ŝ 1 − V + l 1 ) = 0 ⇒ V − l 1 = ŝ 1 ,9

2.3 Sum-of-Squares Relaxation for CLFs Control Lyapunov Functions and Sum-of-Squaresand by fixing k 2 = 1 and s 2 = ŝ 2 l 2 , s 3 = ŝ 3 l 2 , p = ˆp l 2 the second equation is reduced to())( ))l 2 ŝ 2 + ŝ 3(L f V + l 2 + ˆp L g V = 0 ⇒ −(ŝ 3 L f V + l 2 + ˆp L g V = ŝ 2 .A sufficient condition for V to be a control Lyapunov function then is∃ s ∈ S, p, l 1 , l 2 ∈ P, l 1 , l 2 ≻0 such thatV − l 1 ∈ S( ( ))− s L f V + l 2 + p L g V ∈ S .(22)Using semidefinite programming, it is possible to evaluate condition (22) and to verifywhether a given candidate function V is a control Lyapunov function. 2 This is stated asLemma 2: Sum-of-Squares Certificate for Control Lyapunov Functions.V is a control Lyapunov function for the polynomial system ẋ = f(x) + g(x) ifV − l( ( )) 1− s L f V + l 2 + p L g Vs∈ S∈ S∈ Sp, l 1 , l 2 ∈ Pl 1 , l 2 ≻ 0is a feasible semidefinite programming problem.Note, that this can only be used to assess a candidate function V that has to be provided,e.g., by guessing.A way to actually synthesize a control Lyapunov function is to allow the coeffcients ofa parametric polynomial V , i.e., V = c z, c ∈ R 1×m , z ∈ M m (x), to be decision variables.The problem stated in lemma 2 can then be solved asProgram 1.Find V such that:V − l( ( )) 1− s L f V + l 2 + p L g Vs∈ S∈ S∈ Sp, l 1 , l 2 ∈ Pl 1 , l 2 ≻ 0As both the polynomial multiplier’s and the CLF’s coefficients are now decision variables,2 For actual implementation, the positive definite polynomials l 1 and l 2 need to be specified. A commonapproach is to use l i = ɛ i x T x with a decision variable ɛ.10

2.3 Sum-of-Squares Relaxation for CLFs Control Lyapunov Functions and Sum-of-Squaresthe associated semidefinite program will include bilinear constraints of the formn∑n∑ n∑Q 0 + Q k y k + R kl y k y l ≽ 0. (23)k=1k=1 l=1The numerical solution of these bilinear matrix inequalities (BMI) is still subject to ongoingresearch, since these problems are no longer convex. Current approaches include iterativeschemes which fix one bilinear variable and solve the resulting linear matrix inequality inthe other variables. In a next step, a different bilinear variable is fixed and the procedureis repeated. This is computanially expensive and convergence to a global extremum cannotbe guaranteed. The only available direct solver for bilinear problems currently is PENBMI,a commercial solver that uses a local penalty method. YALMIP [13] and its sum-of-squaresmodule [14] can be used as a parser.We consider this software to be still in an early stage and in our opinion it lacksboth performance and reliability compared to the dedicated polynomial manipulation toolboxMULTIPOLY [2]. For linear sum-of-squares programs, i.e., to verify that a candidatefunction is a CLF, we therefore recommend the use of the toolbox SOSTOOLS [16], thatprovides a number of advantages including the integration of MULTIPOLY. Furthermore,we recommend to verify results obtained from bilinear sum-of-squares problems throughYALMIP/PENBMI with SOSTOOLS. We nevertheless conclude that the available toolsallow for our method to attack the problem and if they are successful produce a meaningfulcertificate. If unsuccessful, however, no conclusions about possible intractability of theproblem can be drawn. We expect further development in this field.11

Sontag Formula Feedback3 Sontag Formula FeedbackIt might seem that having established a method to find a control Lyapunov function, thetask of global regulatory control with the use of the Sontag formula is already solved. Butas this feedback law is in fact a function of the control Lyapunov function, it yields differentresults for different CLFs. This makes the intentional design of the controller particularlydifficult, especially considering the fact that the original Sontag feedback formula (3) doesnot include any design parameters.3.1 Modifications for PerformanceFreeman and Primbs [7] therefore use a modified Sontag formula feedback law that, besidesincorporating multi-input systems, introduces a tuning parameter to exchange state decayspeed for control action.⎧⎪⎨ 0 if L g V = 0( √ )u =(⎪⎩ − (L g V ) 2 ( ) ( ) T )TL(L g V ) (L g V ) T f V + L f V + q(x) L g V L g V if L g V ≠ 0(24)Primbs and Doyle [17] showed that the tuning parameter q(x) can be interpreted in thecontext of the Hamilton-Jacobi-Bellman equation. This interesting result will be brieflyoutlined here, as it motivates our own approach.Consider the infinite horizon nonlinear optimal control problemminu∫ ∞0()q(x) + u T u dtsubject to: ẋ = f + g uwith q(x) continuously differentiable, positive definite. Applying the Bellman principle ofoptimality yields the steady-state Hamilton-Jacobi equation(25)− ∂V ∗∂t= 0 = q + u ∗T u ∗ + L f V ∗ + L g V ∗ u ∗ . (26)The unknown V ∗ is called the value function and V ∗ (x 0 ) represents the optimal cost to gofrom an initial condition x 0 to the origin. The optimal input u ∗ can be calculated from thefirst order optimality condition0 = ∂∂u ∗ (q + u ∗T u ∗ + L f V ∗ + L g V ∗ u ∗) = 2 u ∗T + L g V ∗ (27)12

3.1 Modifications for Performance Sontag Formula Feedbackwhich yields the state feedback lawu ∗ = − 1 2(L g V ∗) T. (28)This is substituted into equation (27) to obtain the Hamilton-Jacobi-Bellman equation0 = L f V ∗ − 1 4(L g V ∗) ( L g V ∗) T+ q . (29)Assume now that the respective level sets of the value function V ∗ and the control Lyapunovfunction V posess the same shape. This implies that their gradients are co-linear at everypoint, i.e., ∇V ∗ = λ ∇V . Using this together with equation (29) allows to solve for theparameter λ which turns out to beλ =(L g V2) (L g VConsequently, equation (28) can be written as√ )( ) 2 ( ) ( ) T) T(L f V + L f V + q L g V L g V. (30)u ∗ = − λ 2= −( ) TL g V( ) TL g V) (L g V(L g V√ )( ) 2 ( ) ( ) T) T(L f V + L f V + q L g V L g V.(31)The optimizer for problem (25) thus coincides with the modified Sontag formula (24).This of course is only the case if V and V ∗ have level sets with the same shape, anassumption that is neither assessable nor likely to be fulfilled by chance. The control law(24) therefore cannot be expected to be optimal when using an arbitrary CLF. AlthoughFreeman and Kokotovic [6] emphasize inverse optimality of the control law i.e., the existenceof a functional that is minimized by it, actual performance decreases dramatically whenvalue function and control Lyapunov function differ significantly in the shape of their levelsets. The fact that the feedback law (24) uses the directional information provided bythe gradient of the CLF explains this dependence on the shape of the underlying controlLyapunov function.This is, however, widely ignored by researchers following the sum-of-squares approachto Lyapunov based control. The naïve use of program 1 to find a CLF with semidefiniteprograming will usually yield the first feasible point that the solver can find. The resultthus depends on the algorithm and starting point that the solver uses.We therefore see the need to introduce design objectives to the search for a CLF. Ourapproach is to impose a certain shape on the control Lyapunov function, that consequently13

3.1 Modifications for Performance Sontag Formula Feedbackdetermines the performance of the control law. A desirable shape might be obtained fromthe solution of the linear quadratic regulator (LQR) problem.We motivate this idea by introducing the modified Sontag feedback law⎧0 if L g V = 0(⎪⎨√u = −R−1 (L g V ) T( ) 2 ( ) ( )) TL(L g V ) R −1 (L g V ) T f V + L f V + x T Q x L g V R −1 L g V⎪⎩if L g V ≠ 0 .which for any Q ≻ 0 , R ≻ 0 still yields global stability of the closed loop system because⎧⎪⎨ L f V < 0 if L g V = 0 , x ≠ 0˙V = ( ) 2 ( )⎪⎩ −√ ( ) TL f V + x T Q x L g V R −1 L g V < 0 if L g V ≠ 0 , x ≠ 0 .It still is continuous everywhere except possibly at x = 0. Furthermore for linear systemsẋ = A x + B u (34)the control law (32) is equivalent to the linear quadratic regulator(32)(33)u = −R −1 B T P x (35)with 0 = A T P + P A + Q − P B R −1 B T P , P ≻ 0 , P = P T , (36)that is known to minimize the cost functional∫ ∞0x T Q x + u T R u dt . (37)To see this, note that for a quadratic control Lyapunov function V = x T P x, the feedbacklaw (32) can be written in explicit form asu = −R−1 g T P xx T P g R −1 g T P xFor linear systems this becomes( √)x T P f + (x T P f) 2 + x T Q x x T P g R −1 g T P x . (38)√u = −R −1 B T P x xT P A x + (x T P A x) 2 + x T Q x x T P B R −1 B T P xx T P B R −1 B T . (39)P x14

3.1 Modifications for Performance Sontag Formula FeedbackObviously (39) is equivalent to (35) if and only if√x T P A x + (x T P A x) 2 + x Q x x T P B R −1 B T P x = x T P B R −1 B T P x .With P B R −1 B T P = A T P + P A + Q, it follows that( )x T A T P + Q x =√(x T P A x) 2 ()+ x Q x x T A T P + P A + Q x( ( ) ) 2 (2⇔ x T A T P + Q x − x T P A x) ( )= x Q x xTA T P + P A + Q xwhich can be verified by using a 2 − b 2 = (a + b) (a − b) and x T P A x = x T A T P x.For a certain region around the origin, we therefore expect near optimal performancein terms of the cost functional (37) when using the modified Sontag feedback formula (32)together with the control Lyapunov function V = x T P x obtained from the LQR problemfor the linearized system. To check whether this is a control Lyapunov function for thenonlinear system, lemma 2 can be used. As this will usually not be case, the idea is to usean augmented control Lyapunov functionV = x T P x + m(x) (40)with 0 = ĀT P + P Ā + Q − P ¯B R −1 ¯BT P , (41)∣P ≻ 0 , P = P T ∣∣∣x=0, Ā = ∇f∣ , ¯B = g .x=0where m(x) is a higher order polynomial, i.e., it consists only of monomials with degreegreater than 2. To keep the influence of the higher order terms generally as small as possible,we rewrite m(x) = m z(x) in vector notation and introduce the optimization objective ofminimizing the norm of the coefficient vector m. This leads to the bilinear sum-of-squaresProgram 2.min √ m T msubject to: V − l 1 ∈ S( ( ))− s L f V + l 2 + p L g V ∈ Ss∈ Sp, l 1 , l 2 ∈ Pl 1 , l 2 ≻ 0We thus add as little higher order terms as necessary to render the solution of the LQR problema control Lyapunov function while preserving the shape within a certain region aroundthe origin. Using the modified Sontag feedback law (32), global stability is guaranteed and15

3.2 Modifications for Systems with Bounded Uncertainties Sontag Formula Feedbacknear optimal performance for small deviations from the origin is expected. Focusing on theperformance around the origin is reasonable, as any trajectory will eventually enter thisregion due to global stability. Even more important from a practical point of view, we haveintroduced convenient tuning parameters for both state decay rates and control action.3.2 Modifications for Systems with Bounded UncertaintiesAn important question for actual application is how to deal with uncertainties in a system.Often uncertainty can be reasonably bounded, as parameters are usually known to varywithin a certain range or because estimates for external disturbances are available. This typeof uncertainty for nonlinear systems is very successfully addressed by sliding mode control(see Slotine and Li [18]). These (usually discontinuous) controllers seek to overpower an apriori bounded uncertainty term with control action. A crucial assumption for this is thematching condition, i.e., that the uncertainty enters the system through the same channelas the input. Recently developed dynamic surface control [19] generalizes these ideas alsoto mismatched systems. Although sliding mode controllers are usually used for trackingcontrol, we regard them as a valuable inspiration for the regulator task.We consider a polynomial system, affine in both control and disturbance, given bythe equationẋ = f(x) + g u (x) u + g d (x) dwith x(t) ∈ R n , f(t) ∈ R n , u(t) ∈ R nu , d(t) ∈ R n d,(42)g d (t) ∈ R n × R n d, g u (t) ∈ R n × R nu ,and restrict the unknown disturbance d to satisfy an a priori bound‖d‖ < δ ∀d with δ ∈ R n d(43)This can be used, e.g., to model parameter uncertainty for a family of systems. Recall thatwe seek to decrease the control Lyapunov function V to eventually drive the system to theorigin. For the disturbed system, the time derivative of the CLF is˙V = L f V + L guV u + L gdV d . (44)The disturbance d is unknown and thus sign indefinite, but with the uncertainty bound(43) the worst case is governed by the inequality˙V ≤ L f V + L guV u + ‖ L gdV δ‖ . (45)16

3.2 Modifications for Systems with Bounded Uncertainties Sontag Formula FeedbackThus, using the reasoning from the Artstein-Sontag theorem, the conditionL f V + ‖ L gdV δ‖ < 0 ∀x ≠ 0 such that L guV = 0 . (46)guarantees the existence of a globally stabilizing controller for the uncertain system. Sucha controller is easily constructed by replacing L f V in the Sontag feedback formula withα = L f V + ‖ L gdV δ‖. For the closed loop system, this yields˙V = L f V + L gdV d < L f V + ‖ L gdV δ‖ < 0˙V = −√α 2 + x T Q xif L guV = 0 , x ≠ 0( )L guV R −1( ) TL guV + Lgd V d − ‖ L gdV δ‖ < 0(47)if L guV ≠ 0 , x ≠ 0which proves stability even in the presence of bounded uncertainty. Note, that the controllaw still is continuous and that the formulation requires no matching condition. In fact, wecan conclude that for matched systems 3 L gdV and L guV have the same roots and thus anycontrol Lyapunov function will satisfy condition (46) for arbitrary δ. This is expected andreflects the conceptual similarity to sliding mode controllers: it is possible to overpower anymatched uncertainty with sufficient control action.Condition (46) is equivalent to the two conditionsL f V + L gdV δ < 0 ∀x ≠ 0 such that L guV = 0L f V − L gdV δ < 0 ∀x ≠ 0 such that L guV = 0(48)for whom a sum-of-squares relaxation can be obtained using theorem 3 and the proceduredescribed in section 2.3. A control Lyapunov function can then be found withProgram 3.Find V such that:(−(s 1−V − l)) 1L f V + L gdV δ + l 2 + p 1 L guV))+ l 3 + p 2 L guV(s 2(L f V − L gdV δ∈ S∈ S∈ Ss 1 , s 2 ∈ Sp 1 , p 2 , l 1 , l 2 , l 3 ∈ Pl 1 , l 2 , l 3 ≻ 03 Recall that the precise definition according to Khalil [12] is that the disturbance term belongs to therange space of the input matrix.17

3.3 Modifications for Disturbance Attenuation Sontag Formula Feedback3.3 Modifications for Disturbance AttenuationFor linear systems, H ∞ design has become a widely accepted design paradigm that systematicallyaddresses performance and allows for conclusions in terms of uncertainties anddisturbance attenuation. For nonlinear systems, the corresponding property is the L 2 gain.Considering a disturbed systemẋ = f(x) + g d (x) dwith x(t) ∈ R n , f(t) ∈ R n , g d (t) ∈ R n × R n d, d(t) ∈ R n d,(49)we seek to guarantee a bound from the disturbance d to a certain performance index h interms of the L 2 norm√ ∫ ∞‖h‖ 2 = h(t) T h(t) dt . (50)0Note, that any closed loop systems using state feedback can be written in the form (49).The results in this section therefore implicitly apply to systems that use the Sontag formulafeedback developed in the previous sections.As an important conceptual difference to the approach of section 3.2, where we wereinterested in the stability of an equilibrium point, we now use the framework of boundedinput/bounded output stability. We are therefore interested in results of the form‖h‖ 2 < γ ‖d‖ 2 , ∀d with γ ∈ R + , (51)which for linear systems is equivalent to the H ∞ norm (see, e.g., Khalil [12]). From thetheory of dissipative systems [23], it is known that this inequality can be guaranteed to holdby finding a storage function with the supply rate γ 2 d T d − h T h. We formalize this asLemma 3:The system (49) has an L 2 gain from d to h that is less then γ if ∀d and ∀x ≠ 0there exists a positive definite, radially unbounded function V such that˙V < γ 2 d T d − h T h.Lemma 3 can be proved by integrating V along the system’s trajectories, which yieldsV (x ∞ ) − V (x 0 ) < γ 2 ‖d‖ 2 2 − ‖h‖ 2 2 ∀d and ∀x 0 (52)18

3.3 Modifications for Disturbance Attenuation Sontag Formula Feedbackand due to the positive definiteness of V also‖h‖ 2 2 ≤ ‖h‖ 2 2 + V (x ∞ ) < γ 2 ‖d‖ 2 2 + V (x 0 ) ∀d and ∀x 0 . (53)Again using that V is positive definite, this implies inequality (51). In the absence of disturbances,it furthermore provides an initial condition to output bound‖h‖ 2 2 < V (x 0 ) ∀x 0 . (54)Lemma 3 suggests to use sum-of-squares programming to search for a suitable storagefunction similar to the search for a control Lyapunov function in section 2.3. This approachto L 2 gain analysis is developed in [10]. It is however not possible to use this method togetherwith the Sontag formula, as the feedback law renders the closed loop system non-polynomial.We therefore proceed with a slightly different approach and again consider the system(42) with both disturbance and control input. From lemma 3, we conclude that if∀d and ∀x ≠ 0,∃V such that˙V = L f V + L guV u + L gdV d < γ 2 d T d − h T h ,(55)then the L 2 gain from d to h is less than γ. Condition (55) can be restated as∀x ≠ 0, ∃V such that()max L f V + L guV u + L gdV d − γ 2 d T d + h T h < 0 .d(56)We follow the proposition of Isidori [8] and calculate the maximum disturbance from thefirst order optimality conditionL gdV − 2 γ 2 d ∗T = 0 ⇔ d ∗ = 12 γ 2 (L gdV) T. (57)Substituting this expression into equation (56), the condition becomes∀x ≠ 0, ∃V such thatL f V + L guV u + 1 ( ) ( ) T4 γ 2 L gdV L gdV + h T h < 0 .(58)As long as L guV ≠ 0, this can always be achieved by a suitable choice of u. This is the samereasoning that was used in section 2.1 with the Artstein-Sontag theorem. This furthermoreimplies the existence of a control law which guarantees that the closed loop system has anL 2 gain less than γ, once a storage function V that satisfies condition (58) is found.19

3.3 Modifications for Disturbance Attenuation Sontag Formula Feedback) ( ) TReplacing L f V in the Sontag formula (32) with β = L f V +(L 14 γ 2 gdV L gdV + h T hyields the desired result, as for the closed loop system condition (58) becomes−√β 2 + x T Q x( )L guV R −1( ) TL guV < 0 ∀x ≠ 0which trivially holds. We summarize these results inLemma 4:There exists a state feedback law which guarantees the L 2 gain from d to h isless than γ if there exists a positive definite, radially unbounded function V withL f V + 1 ( ) ( ) T4 γ 2 L gdV L gdV + h T h < 0 ∀x ≠ 0 such that L guV = 0 .It is now possible to again use sum-of-squares programming to search for a suitable functionV. As the condition in lemma 4 is quadratic in V , we use the Schur complement[ ]M LT≺ 0 ⇔ M − L T N −1 L ≺ 0 and N ≺ 0 (59)L Nto transform it into the equivalent form⎡(L gdV⎣ L f V + hT 1h2 γ( ) TL gdV −I12 γ) ⎤To formulate this as a sum-of-squares problem, we then useLemma 5.⎦ ≺ 0 ∀x ≠ 0 such that L guV = 0 . (60)A polynomial matrix M(x) is positive semidefinite for all x if the scalar polynomialy T M(x) y is a sum-of-squares in x and y.Using theorem 3 and the procedure described in section 2.3 then allows a sum-of-squaresrelaxation of condition (60) that leads toProgram 4.Find V such that:(L gdV⎛ ⎛ ⎡− ⎝s ⎝y T ⎣ L f V + hT 1h2 γ( ) TL gdV −I12 γ) ⎤⎞V − l⎞ 1⎦ y⎠ + l 2 + p L g V ⎠s∈ S∈ S∈ Sp, l 1 , l 2 ∈ Pl 1 , l 2 ≻ 0 .20

3.3 Modifications for Disturbance Attenuation Sontag Formula FeedbackAn important question is, which performance index h should be used with this approach.In the spirit of our earlier LQR based design, a logical choice would beh =[√ ]Q x√R u(61)which then would result in the quadratic cost performance metric∫ ∞0x T Q x + u T R u dt < γ 2 ‖d‖ 2 2 ∀d . (62)However, one major problem is that the Sontag formula does not result in a polynomialfeedback law, which consequently prevents the use of sum-of-squares methods. Note, thata similar problem arises even if a polynomial control law as suggested by Jarvis-Wloszek[9] is used. The quadratic term in the cost functional (62) will always result in quadraticdecision variables that the available bilinear solvers cannot handle. The output thus cannotbe used directly in the performance index. Using only state weights, however, can result inexcessively high control action and is therefore of limited use.One idea to include an input weight is to use an augmented system[ẋ ] [ ] [ ] [ ]f(x) gu (x) gd (x)= + u +(63a)˙v −ω v ω0or alternatively[ẋ ] [ ] [ [ ]f(x) + gu (x) v 0 gd (x)=+ u +˙v −ω v ω]0. (63b)The former introduces new states that are used to keep track of the input. The latter takesthis idea one step further and drives the original system solely through these new states,effectively introducing a low pass filter with bandwidth ω at the input. As the theorypresented so far does hold also for the augmented system, with the performance indexh =[√ ]Q x√R z(64)it should be possible to include a penalty on control effort. We note here that it is yet unclear,to what extent possible results are valid for the original system. Although we expect thesystem to behave asymptotically for large ω, no quantitative results could be obtained sofar to support this.21

Illustrative Examples4 Illustrative ExamplesIn this section, we illustrate first the relationship between controller performance and theshape of the control Lyapunov function used to derive the control law, and second, thepossible advantages of our method. Two controller designs based on the ideas described inchapter 3 are presented for different polynomial systems.4.1 Quadratic Cost PerformanceWe start with a two dimensional system, to simplify visualization and point out importantfeatures. The system is a truncation of an example used in [21]. It is given by the equation[ ] [−2 x1 , 1 + x2ẋ =3 x 2 + x 3 + 2 1 − x 2 ] [ ]1 u1. (65)1 1 3 u 2The open loop system has a one dimensionalstable manifold. The phase portrait for u = 0is depicted in figure 1. The linearization is[ ] [ ]−2 0 1 1ẋ = x + u (66)0 3 1 3x21050which shows the saddle characteristic of theequilibrium point. As a choice for the tuningparameters in our feedback law (32), we use[ ][ ]1 01 0Q = , R = . (67)0 10 1Solving the associated LQR algebraic Riccatiequation for the linearized system yields−5−10−10 −5 0 5 10x 1Figure 1: Open loop system’s phase portrait.V LQR =[ ] T [ ] [ ]x1 0.2436 −0.1105 x1x 2 −0.1105 0.8090 x 2. (68)This is a control Lyapunov function for the nonlinear system, as can be verified by theuse of lemma 2. Figure 2 shows the control Lyapunov function and its Lie derivative withrespect to the autonomous dynamics. From figure 2b it is apparent that there are regionswhere L f V is negative, i.e., where the system tends towards the origin without externalinput. Although depending on the Q/R weighting, we expect a good control law to takeadvantage of this fact.22

4.1 Quadratic Cost Performance Illustrative ExamplesThe control action is depicted in figure 3. It is notable that the Sontag formula indeed takesadvantage of the higher order dynamics. Wherever the autonomous dynamics are stable,little control action is applied, whereas in those regions where the autonomous dynamicsare unstable, the control action increases.·10 4 x 1V10050010L f V10−110x 20−10 −5 0 5 10−10x 1(a) Control Lyapunov Functionx 20−10 −5 0 5 10−10(b) Lie Deriviative of the Control Lyapunov Functionwith respect to the autonomous dynamicsFigure 2: LQR based control Lyapunov function for the system.5050u10u20−50−501010x 20−10 −5 0 5 10−10x 1x 20−10 −5 0 5 10−10x 1(a) First input.(b) Second input.Figure 3: LQR based Sontag formula state feedback law.23

4.1 Quadratic Cost Performance Illustrative Examples5050u10u20−50−501010x 20−10 −5 0 5 10−10x 1x 20−10 −5 0 5 10−10x 1(a) First input.(b) Second input.Figure 4: Linear quadratic regulator state feedback law.This is an important difference to the use of feedback linearization techniques that tend tocancel benevolent higher order dynamics. Furthermore, it can be verified that the controllaw is tangential to the LQR control law (fig. 4) at the origin.To show the advantages of our systematic method of synthesizing control Lyapunovfunctions, two other CLFs are used. A classical approach to the problem would be to guessa candidate function and verify that it is a CLF, e.g., by using lemma 2. A common firstchoice isV I =[ ] T [ ] [ ]x1 1 0 x1x 2 0 1 x 2, (69)which for this problem indeed is a CLF. The second possibility is to use sum-of-squaresprogramming to obtain a feasible point of program 1 with V = x T P x and P now being adecision variable. A feasible point returned by PENBMI isV feas =[ ] T [ ] [ ]x1 12.7294 6.5225 x1x 2 6.5225 6.9634 x 2. (70)Note, that because the input matrix g has full rank everywhere, any choice of a positivedefinite P leads to a stabilizing feedback law with the Sontag formula. The P matrixobtained from the feasibility problem thus indeed is arbitrary.Figure 5 shows the phase portraits of the closed loop system using the globally stabilizingcontrol laws. Obviously the closed loop behavior varies considerably, although the24

4.1 Quadratic Cost Performance Illustrative Examplesexact same feedback formula (32) is used. These differences are caused by using the differentunderlying control Lyapunov functions (68), (69) and (70), which proves our point toexplicitly shape the CLF in the design process of the controller. 4101055x20x20−5−5−10−10 −5 0 5 1010x 1(a) Sontag Formula (using V LQR)−10−10 −5 0 5 10(b) Linear Quadratic Regulator (not globally stable)10x 155x20x20−5−5−10−10 −5 0 5 10−10−10 −5 0 5 10x 1x 1(c) Sontag Formula (using V I)(d) Sontag Formula (using V feas )Figure 5: Phase portraits of the closed loop system with different control laws.4 It should be noted here, that the CLFs V I and V LQR can also be obtained via our method, e.g., withQ = [ 6 4 44 ] and R = [ 1 0 01 01] the solution to the algebraic Riccati equation is exactly P = [0 1]. This is, ofcourse, a consequence of the inverse optimality of the LQR problem. Our point here is that intentionallyaddressing the corresponding cost functional yields superior performance in the desired metric compared toblindly relying on the inverse optimality.25

4.1 Quadratic Cost Performance Illustrative ExamplesClosed loop simulations are used to illustrate the benefits that can be gained by usingthe proposed method. Direct application of the linear quadratic regulator, which does notprovide global stability, is used as a performance benchmark. As a metric, the quadraticcost determined by the functional ∫ τ0 xT Q x + u T R u dt is used, with τ being a sufficientlylarge time to allow the system to settle at the equilibrium. A disk of radius 2 around theorigin is uniformly sampled in state-space, which can be seen as a circular approximationof the LQR’s region of stability. Using polar coordinates with steps of 0.05 in radial and 5 ◦in angular direction, the sample consists of 2993 points. Each of these is used as an initialcondition and the cost along all trajectories is summed up. Table 1 provides the results fordifferent weighting matrices Q and R.Table 1: Quadratic cost of the different feedback laws. Uniform sample within a disk of radius 2around the origin with 2993 sample points. Cost of all trajectories added up to τ = 5.Weighting factors LQR Sontag (V LQR ) Sontag (V I ) Sontag (V feas )Q = [ 1.0 0 1.0 0 R = [ 1.0 0 1.0 0 ] 2117 2068 2331 3240Q = [ 1.0 0 1.0 0 R = [ 1.0 0 4.0 0 ] 2026 2003 2132 3115Q = [ 1.0 0 1.0 0 R = [ 30 0 20 0 ] 1253 1100 1494 2420Q = [ 5.0 0 0.5 0 R = [ 1.0 0 1.0 0 ] 3688 3579 3723 4977Q = [ 5.0 0 0.5 0 R = [ 1.0 0 4.0 0 ] 3689 3205 3420 4487Q = [ 5.0 0 0.5 0 R = [ 30 0 20 0 ] 3776 3348 3274 4404Q = [ 0.1 0 0.3 0 R = [ 1.0 0 1.0 0 ] 1446 1565 1901 2684Q = [ 0.1 0 0.3 0 R = [ 1.0 0 4.0 0 ] 1334 1399 1332 2159Q = [ 0.1 0 0.3 0 R = [ 30 0 20 0 ] 290 273 410 717Overall 19619 18267 20017 28203Summarized, the LQR based Sontag formula provides good results throughout all tuningsconsidered. In some cases the guessed control Lyapunov function and the linear quadraticregulator provide slightly better results. The control law based upon the feasibility problemyields consistently the worst performance. Our results thus suggest that improved performancecan be achieved with the systematic way of constructing the Lyapunov functioncompared to using some arbitrary feasible point.26

4.2 Controller Tuning Illustrative Examples4.2 Controller TuningAs a second system, we consider the inverted pendulum which is a well studied example inboth nonlinear dynamics and control. It has some interesting properties that make it prototypicalfor various real world systems. Most obviously it is unstable at its upper equilibriumpoint and therefore requires stabilizing control. It is also an underactuated non-minimumphase system and therefore inherently difficult to control. Examples for this class of systemsinclude rockets and aircraft.m 2m 1(a) Modelgπ2lx 2π4x40− π 4ux 1− π 2− π 2− π 40π4x 2(b) Phase portrait of the angular dynamicsπ2Figure 6: Second example: the inverted pendulum.The model is illustrated in figure 6, together with the phase portrait of the angular dynamics.The model equations are derived from physical interpretation of the system. The simplestmodel that captures the nonlinear dynamics of the inverted pendulum is a point mass m 2 ,attached to a cart of mass m 1 through a rigid rod of length l without mass. Effects of frictionand damping are neglected. A force u is applied to the cart and used as the control input.⎡ ⎤ ⎡ẋ 1ẋ 2=⎢ẋ ⎣ 3 ⎥⎦ ⎢⎣ẋ 4x 3x 4m 2 g cos(x 2 ) sin(x 2 )−m 2 l x 2 4 sin(x 2)m 1 +m 2 −m 2 cos 2 (x 2 )(m 1 +m 2 ) g sin(x 2 )−m 2 l x 2 4 sin(x 2) cos(x 2 )m 1 l+m 2 l−m 2 l cos 2 (x 2 )⎤⎡+⎥ ⎢⎦ ⎣001m 1 +m 2 −m 2 cos 2 (x 2 )cos(x 2 )m 1 l+m 2 l−m 2 l cos 2 (x 2 )⎤u . (71)⎥⎦As this model is rational with trigonometric terms and it is not possible to turn it into apolynomial form with a change of variables, an approximation for a certain region around27

4.2 Controller Tuning Illustrative Examplesthe origin is used. This of course implies, that the global stability for the approximatesystem does not hold in general for the full model. Still our method provides a very straightforward way to come up with a control law for a system that is particularly difficult tocontrol.Table 2: Parameters for the simulation.m 1 m 2 l g2.5 1.0 0.5 9.81We use the parameters from table 2 and the polynomial approximationwith⎡ ⎤ ⎡ ⎤ ⎡ ⎤ẋ 1 x 30ẋ 2x 40=+u . (72)⎢⎣ẋ 3⎥ ⎢⎦ ⎣f 3 (x 2 , x 4 ) ⎥ ⎢⎦ ⎣g 3 (x 2 ) ⎥⎦ẋ 4 f 4 (x 2 , x 4 ) g 4 (x 2 )f 3 (x 2 , x 4 ) = 0.11707 x 5 2 + 0.035908 x 3 2 x 2 4 − 1.6032 x 3 2 − 0.17201 x 2 x 2 4 + 3.0313 x 2 ,f 4 (x 2 , x 4 ) = 0.24902 x 5 2 + 0.13049 x 3 2 x 2 4 − 5.6188 x 3 2 − 0.29147 x 2 x 2 4 + 23.9892 x 2 ,g 3 (x 2 ) = 0.02905 x 4 2 − 0.11289 x 2 2 + 0.3955 ,g 4 (x 2 ) = 0.096371 x 4 2 − 0.54277 x 2 2 + 0.7831 ,which is obtained as a least-square approximation of the full equations for x 2 × x 4(− π 2 , π 2 ) × (− 3 π ). The linearized system then is2 , 3 π2⎡ ⎤ ⎡⎤ ⎡ ⎤ ⎡ ⎤ẋ 1 0 0 1 0 x 1 0ẋ 20 0 0 1x 20=+u (73)⎢⎣ẋ 3⎥ ⎢⎦ ⎣0 3.0313 0 0⎥⎢⎦ ⎣x 3⎥ ⎢⎦ ⎣0.3955⎥⎦ẋ 4 0 23.9892 0 0 x 4 0.7831and solving the associated algebraic Riccati equations for Q =[ 1 0 0 0]0 1 0 00 0 1 00 0 0 1and R = 1 yields⎡⎤3.1990 −15.2287 4.6168 −3.6086−15.2287 541.5121 −45.1076 115.4561P =. (74)⎢⎣ 4.6168 −45.1076 13.0926 −10.6973⎥⎦−3.6086 115.4561 −10.6973 24.8493∈28

4.2 Controller Tuning Illustrative ExamplesThe method of lemma 2 is not able to prove that this results in a control Lyapunov functionfor the nonlinear system. We thus employ the globalization strategy and solve program 2,where for simplicity only quartic terms with a diagonal Gram matrix are used. Minimizingthe higher order coefficients then means to minimize the Frobenius norm of the Gram matrix.We further simplify the procedure by directly imposing a positive definiteness constraint onM, so that V is also guaranteed to be positive definite and by fixing l 2 = ɛ x T x.√min trace M T Msubject to M ≻ 0 , M diagonal ,( ( ))− s L f V + ɛ x T x + p L g V ∈ S , ɛ ∈ R(75)ɛ > 0 ,p ∈ P .s ∈ SThe result is⎡⎤9.957 0 0 0 0 0 0 0 0 00 9.957 0 0 0 0 0 0 0 00 0 9.957 0 0 0 0 0 0 00 0 0 9.957 0 0 0 0 0 0M = 10 −6 0 0 0 0 9.957 0 0 0 0 0·0 0 0 0 0 9.955 0 0 0 00 0 0 0 0 0 9.951 0 0 00 0 0 0 0 0 0 9.957 0 0⎢⎥⎣ 0 0 0 0 0 0 0 0 9.959 0 ⎦0 0 0 0 0 0 0 0 0 9.974(76)and thus we found a control Lyapunov function⎡ ⎤x 1x 2V =⎢⎣x 3⎥⎦x 4TP⎡x 2 ⎤T1x 1 x 2⎡ ⎤x 2 2x 1x 1 x 3x 2x 2 x 3+⎢⎣x 3⎥⎦x 2 3x 4 x 1 x 4x 2 x 4⎢⎣x 3 x ⎥ 4 ⎦M⎡x 2 ⎤1x 1 x 2x 2 2x 1 x 3x 2 x 3x 2 3x 1 x 4x 2 x 4⎢⎣x 3 x ⎥ 4 ⎦(77)x 2 4x 2 429

4.2 Controller Tuning Illustrative Examplesthat is in shape similar to the function suggested by the solution of the LQR problem. Usingthe feedback law (32), we can guarantee global stability for the polynomial system. We thusconclude stability for the full model, where the polynomial approximation is valid.Furthermore, we can tune the controller by choosing different weighting matrices andrepeat the procedure. For every different combination of weighting matrices, the solutionto the Riccati equation is of course different, and thus also a different higher order Grammatrix is returned by the solver. Nevertheless, the coefficients stay in the same order ofmagnitude, i.e., are small compared to the coefficients of the quadratic part.Closed loop simulations using the full trigonometric model are carried out to demonstratetuning capabilities. The initial condition x 0 = [ 10 55 π180 0 0 ]T is outside of the regimewhere linear controllers can usually be applied. From the results depicted in figure 7, it isapparent that the design procedure allows for systematic tuning.angle x2/ ◦input u/ Nposition x1/ m1050−50 2 4 6 8 10 12 14 16 18 20time / s100500Q = diag [ 1 1 1 1 ] R = 1Q = diag [ 0.5 10 0.5 0.5 ] R = 5Q = diag [ 10 1 1 1 ] R = 1Q = diag [ 1 1 1 1 ] R = 1Q = diag [ 0.5 10 0.5 0.5 ] R = 5Q = diag [ 10 1 1 1 ] R = 1−500 2 4 6 8 10 12 14 16 18 20time / s500−50Q = diag [ 1 1 1 1 ] R = 1Q = diag [ 0.5 10 0.5 0.5 ] R = 5Q = diag [ 10 1 1 1 ] R = 1−1000 2 4 6 8 10 12 14 16 18 20time / sFigure 7: Influence of different tuning parameters.30

Conclusions and Remarks5 Conclusions and RemarksIt was shown that the solution of sum-of-squares problems with semidefinte programmingcan be used to synthesize nonlinear regulatory controllers based on control Lyapunov functions.A systematic approach was developed that includes performance in terms of quadraticcost, allows for tuning and guarantees global stability by use of an explicit state feedbacklaw. Furthermore, possible modifications to include robustness specifications such as disturbanceattenuation and parameter uncertainty were outlined. The design procedure wasapplied to example systems and simulations indicate that the controller performs as expected.The global results presented in this report can be turned into local ones by consideringonly sublevelsets of the Lyapunov function (see [22]). This might be desirable, wheneverglobal results are of limited use, e.g., because the model itself is only valid in some region.Also, issues of robustness could be addressed differently, e.g., by replacing the controlLyapunov functions used here with robust control Lyapunov functions, as introduced byFreeman and Kokotovic [6]. As their notion is based upon set valued maps, we consider itout of scope for the current project.A subtle implication of our results is that even without the use of sum-of-squaresprogramming, obtaining a local control Lyapunov function from the linear quadratic regulatorproblem and using our modified Sontag formula will produce effective control laws fornonlinear systems. Furthermore, including robustness for matched uncertainties is straightforward and can be achieved by a simple modification of the feedback law. Together, thismight be of practical importance and should be compared in detail to the use of linearcontrol techniques for nonlinear systems.Open topics include the question of existence of solutions to our problems. This isespecially true for the L 2 gain design. Obtaining the quadratic part of the storage functionfrom the solution to linear H ∞ or mixed H ∞ / H 2 problems might be a valid strategy thatdeserves further investigation. 5 It should also be noted that the control law in general canproduce high control action so that actuator saturation might become a problem. Furtherresearch is needed in this direction.As a last remark, we emphasize that in our opinion the universal feedback law providedby Sontag is largely underappreciated. For practical application, we consider it amongthe most important results from geometric nonlinear control theory. We therefore like toencourage future research in this field.5 It is known that the H ∞ gain of the linearized system is a lower bound for the nonlinear L 2 gain.31

REFERENCESREFERENCESReferences[1] Artstein, Z.: Stabilization with Relaxed Controls. In: Nonlinear Analysis, Theory,Methods & Applications 7 (1983), No. 11, pp. 1163–1173[2] Balas, G. ; Packard, A. ; Seiler, P. ; Topcu, U.: Robustness Analysis of NonlinearSystems. 2009. – Available at http://www.aem.umn.edu/~AerospaceControl/[3] Bochnak, J. ; Coste, M. ; Roy, M.-F.: Real Algebraic Geometry. Berlin : Springer,1991[4] Boyd, S. ; El Ghaoui, L. ; Feron, E. ; Balakrishnan, V.: Linear Matrix Inequalitiesin System and Control Theory. SIAM, 1994[5] Choi, M. ; Lam, T. ; Reznick, B.: Sums of Squares of Real Polynomials. In: Procededingsof Symposia in Pure Matematics Vol. 58, 1995, pp. 103–126[6] Freeman, R. A. ; Kokotovic, P. V.: Robust Nonlinear Control Design: State-Spaceand Lyapunov Techniques. 1st ed. Birkhäuser Boston, 1996[7] Freeman, R. A. ; Primbs, J. A.: Control Lyapunov Functions: New Ideas from an oldSource. In: Decision and Control, 1996., Proceedings of the 35th IEEE Vol. 4, 1996,pp. 3926–3931[8] Isidori, A.: Nonlinear Control Systems. 3rd ed. Springer, 1995[9] Jarvis-Wloszek, Z.: Lyapunov Based Analysis and Controller Synthesis for PolynomialSystems using Sum-of-Squares Optimization, University of California, Berkeley,Ph.D. thesis, 2003[10] Jarvis-Wloszek, Z. ; Feeley, R. ; Tan, W. ; Sun, K. ; Packard, A.: ControlApplications of Sum of Squares Programming. In: Henrion, D. (Edt.) ; Garulli, A.(Edt.): Positive Polynomials in Control. Berlin : Springer, 2005[11] Kalman, R.: Contributions to the Theory of Optimal Control. In: Boletin de laSociedad Matematica Mexicana 5 (1960), pp. 102–119[12] Khalil, H. K.: Nonlinear Systems. 3rd ed. Prentice Hall, 2001[13] Löfberg, J.: YALMIP : A Toolbox for Modeling and Optimization in MATLAB. In:Proceedings of the CACSD Conference. Taipei, Taiwan, 2004. – Available at http://users.isy.liu.se/johanl/yalmip32

REFERENCESREFERENCES[14] Löfberg, J.: Pre- and Post-Processing Sum-of-Squares Programs in Practice. In:IEEE Transactions on Automatic Control 54 (2009), No. 5, pp. 1007–1011[15] Parrilo, P.: Semidefinite Programming Relaxations for Semialgebraic Problems. In:Mathematical Programming 96 (2003), No. 2, pp. 293–320[16] Prajna, S. ; Papachristodoulou, A. ; Seiler, P. ; Parrilo, P. A.: SOSTOOLS:Sum-of-Squares Optimization Toolbox for MATLAB. 2004. – Available at http://www.cds.caltech.edu/sostools[17] Primbs, J. A. ; Nevistic, V. ; Doyle, J. C.: Nonlinear Optimal Control: A ControlLyapunov Function and Receding Horizon Perspective. In: Asian Journal of Control1 (1999), pp. 14—24[18] Slotine, J.-J. E. ; Li, W.: Applied Nonlinear Control. Upper Saddle River, NJ :Prentice Hall, 1991[19] Song, B. ; Hedrick, J. K.: Dynamic Surface Control of Uncertain Nonlinear Systems.An LMI Approach. London : Springer, 2011[20] Sontag, E. D.: A “universal” Construction of Artstein’s Theorem on Nonlinear Stabilization.In: Systems & Control Letters 13 (1989), pp. 117–123[21] Tan, W.: Nonlinear Control Analysis and Synthesis Using Sum-of-Squares Programming,University of California, Berkeley, Ph.D. thesis, 2006[22] Tan, W. ; Packard, A.: Searching for Control Lyapunov Functions using Sum-of-Squares Programming. In: 42nd Annual Allerton Conference on Communications,Control and Computing, 2004, pp. 210–219[23] Willems, J. C.: Dissipative dynamical systems part I: General theory. In: Archive forRational Mechanics and Analysis 45 (1972), No. 5, pp. 321–35133