Optimal Tracking Control for a Class of Nonlinear Discrete-Time ...

ZHANG et al.: OPTIMAL TRACKING CONTROL FOR A CLASS OF NONLINEAR DISCRETE-TIME SYSTEMS WITH TIME DELAYS BASED ON HDP 1855In the following part, we present the convergence analysisof the iteration about (21)–(28).B. Convergence Analysis of the Iteration HDP AlgorithmLemma 1: Let v [i] (k) { and J [i+1] (e [i] (k)) be expressed as(25) and (27), and let μ [i] (k) } be arbitrary sequence ofcontrol policy, [i+1] be defined by( ) ( T ( T [i+1] e [i] (k) = e (k)) [i] Qe [i] (k) + μ (k)) [i] Rμ [i]( (k))+ [i] e [i−1] (k + 1) , i ≥ 0. (29)Thus, if J [0] = [0] = 0, then J [i+1] (e [i] (k)) ≤ [i+1] (e [i] (k)) ∀e [i] (k).Proof: For given ε 1 (k)(−σ m ≤ k ≤ 0) of (1), it can bestraight forward from the fact that J [i+1] (e [i] (k)) is obtainedby the control input v [i] (k), while [i+1] (e [i] (k)) is a result ofarbitrary control input.Lemma 2: Let the sequence J [i+1] (e [i] (k)) be defined by(27), v [i] (k) is the control policy expressed as (25). There isan upper bound Y , such that 0 ≤ J [i+1] (e [i] (k)) ≤ Y , ∀e [i] (k).Proof: Let γ [i] (k) be any control policy that satisfiesAssumption 1. Let J [0] = P [0] = 0, and P [i+1] (e [i] (k)) isdefined ( as follows:) ( T ( TP [i+1] e [i] (k) = e (k)) [i] Qe [i] (k) + γ (k)) [i] Rγ [i]( (k))+ P [i] e [i−1] (k + 1) . (30)From (30), we can obtain( ) (TP [i] e [i−1] (k + 1) = e [i−1] (k + 1))Qe [i−1] (k + 1)(T+ γ [i−1] (k + 1))Rγ [i−1]( (k + 1))+ P [i−1] e [i−2] (k + 2) . (31)Thus we can get( ) ( TP [i+1] e [i] (k) = e (k)) [i] Qe [i] (k)( T+ γ (k)) [i] Rγ [i] (k)(T+ e [i−1] (k + 1))Qe [i−1] (k + 1)(T+ γ [i−1] (k + 1))Rγ [i−1] (k + 1)(T+ ···+ e [0] (k + i))Qe [0] (k + i)(T+ γ [0] (k + i))Rγ [0] (k + i). (32)For j = 0,...,i, welet(TL(k + j) = e [i− j] (k + j))Qe[i− j] (k + j)(T+ γ [i− j] (k + j))Rγ[i− j] (k + j). (33)Then (32) can further be written as follows:( )P [i+1] e [i] ∑(k) = i L(k + j). (34)j=0Furthermore, we can see that( )i∑∀i : P [i+1] e [i] (k) ≤ lim L(k + j). (35)i→∞Noting that { γ [i] (k) } satisfies Assumption 1, according to thethird condition of Assumption 1, there exists an upper boundY such thatlimi→∞j=0i∑L(k + j) ≤ Y. (36)j=0From Lemma 1 we ( can obtain) ( )∀i : J [i+1] e [i] (k) ≤ P [i+1] e [i] (k) ≤ Y. (37)Theorem 1: For (1), the iteration algorithm is as in(21)–(28), then we have J [i+1] (e [i] (k)) ≥ J [i] (e [i] (k)),∀e [i] (k).{ Proof: For convenience of analysis, define a new sequence [i] (e(k)) } as follows:( T [i] (e(k)) = e T (k)Qe(k) + v (k)) [i] Rv [i] (k)+ [i−1] (e(k + 1)) (38)with v [i] (k) defined by (25), [0] (e(k)) = 0.In the following part, we will prove [i] (e(k)) ≤J [i+1] (e(k)) by mathematical induction.First, we prove it holds for i = 0. Notice thatJ [1] (e(k)) − [0] (e(k)) = e T (k)Qe(k)( T+ v (k)) [0] Rv [0] (k) ≥ 0 (39)thus for i = 0, we haveJ [1] (e(k)) ≥ [0] (e(k)). (40)Second, we assume it holds for i, i.e., J [i] (e(k)) ≥ [i−1] (e(k)), for∀e(k).Then, from (26) and (38), we can getJ [i+1] (e(k)) − [i] (e(k))= J [i] (e(k + 1)) − [i−1] (e(k + 1)) ≥ 0 (41)the following inequality holds: [i] (e(k)) ≤ J [i+1] (e(k)). (42)Therefore, (42) is proved by mathematical induction, for∀e(k).On the other hand, from Lemma 1, we have ∀e [i] (k),J [i+1] (e [i] (k)) ≤ [i+1] (e [i] (k)). So we have J [i] (e(k)) ≤ [i] (e(k)).Therefore ∀e(k), wehaveJ [i] (e(k)) ≤ [i] (e(k)) ≤ J [i+1] (e(k)). (43)So, for ∀e [i] (k), wehave( )J [i+1] e [i] (k)( )≥ J [i] e [i] (k) . (44)