Mathematical Theory of Networks and Systems, Melbourne 9-13 July **2012**

Fritz Colonius

Universität Augsburg

An Approach to Minimal Bit Rates and Entropy

for Deterministic Control

or

The Limits of Control

Supported by DFG Priority Program 1305 ‘Control Theory of Digitally Networked Dynamical Systems’

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 1 / 35

Introduction

Control problems under data rate constraints: control of one or more

dynamical systems using multiple sensors and actuators transmitting

information over a digital communication network.

Early work includes: Delchamps (1990), Wong/Brockett (1999), ...

Basic mathematical question:

What are universal bounds for the minimal amount of “information” that

is needed for performing control tasks?

Here: Explain an approach for deterministic …nite dimensional systems to

the analysis of

- exponential stabilization

- invariance of compact sets

- controlled invariant subspaces

- H∞-control

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 1 / 35

Minimal bit rates

Consider

˙x(t) = f (x(t), u(t)), u 2 U := fu : [0, ∞) ! U R m g.

Any feedback u = F (x) requires continual measurements of the state (or

of observed values).

For a (noiseless) digital communication channel, minimal bit rates are of

interest.

How much information will a controller need?

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 2 / 35

Data rates

Quantization:

Decompose the relevant part of the state space into cells and transfer only

the information, in which cell the present state is.

Event based control:

For example, change the control only if the state crosses from one cell into

another; also other events may trigger control actions.

Model-predictive control

The controller computes (via optimal control) at time ti a control function

u(t), which is used on [ti , ti+1] until at time ti+1 new information is

available (adaptive sampling allowed).

Symbolic controllers

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 3 / 35

Entropy for control

The approach presented here is motivated by ergodic theory.

Nair/Evans/Mareels/Moran: topological feedback entropy

IEEE TAC ’04

Kawan: invariance entropy

SIAM J. Control ’11, Nonlinearity ’11, and a monograph, in preparation

Determine the limits of control by computing minimal bit rates!

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 4 / 35

Outline

Topological entropy for autonomous linear ODE

Exponential stabilization under communication constraints

- Entropy for stabilization

- A second way to count bits

controlled invariance for compact sets

controlled invariant subspaces

topological entropy of H∞-control

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 5 / 35

A (very) short history of entropy

Claude Shannon 1948

The expected information of an experiment (i.e., given a partition)

Kolmogorov and Sinai 1960s

The information generated by a dynamical system: measure-theoretic

entropy

Adler/Konheim/McAndrew 1965

Topological entropy

Bowen and Dinaburg 1971

Topological entropy using distances

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 6 / 35

Topological Entropy for Linear ODE I

Consider for λ > 0

˙x(t) = λx(t), t 0, x(0) = x0 2 K := [ 1, 1].

How many “di¤erent” trajectories are there?

Fix T , ε > 0.

Two trajectories are “the

same” if

e λt x0 e λt y0 < ε

for all t 2 [0, T ].

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 7 / 35

Topological Entropy for Linear ODE II

Fix T , ε > 0. A …nite set R [ 1, 1] is called (T,ε)-spanning if for all

x0 2 [ 1, 1] there is y0 2 R with

e λt jx0 y0j = e λt x0 e λt y0 < ε for all t 2 [0, T ].

The minimal number #R =: r(T , ε) of elements in such a set grows like

eλT and hence

1

lim log r(T , ε) = λ.

T !∞ T

Interpretation: log 2 r(T , ε) is the number of bits generated by the system

on [0, T ] modulo ε. If r(T , ε) = 2 k , then the elements of R can be

labelled by sequences (s1 ... sk ) with si = 0, 1.

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 8 / 35

Topological Entropy for Linear ODE III

˙x(t) = Ax(t), t 0, x(0) = x0 2 K R n , K compact.

Fix T , ε > 0. A …nite set R K is called (T , ε)-spanning if for all x0 2 K

there is y0 2 R with

e At x0 e At y0 < ε for all t 2 [0, T ].

Let #R =: r(T , ε) be minimal. Then the topological entropy is de…ned

as

1

htop(A) := limε!0 limT !∞ log r(T , ε).

T

Theorem (Bowen 1971). For every K with nonvoid interior

htop(A) = ∑Re λ>0 Re λ.

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 9 / 35

Stabilization for digitally connected systems

- quantization

- event based control

- symbolic controllers

- model predictive control

In any case: The controller must be able to generate a set of control

functions stabilizing for every initial value. Since only …nitely many bits are

available on a …nite time interval, only …nitely many controls should be

generated on a …nite time interval. The controller must be able to

distinguish between them.

We may count the associated number of bits.

This gives at least a lower bound for the bit rate.

Tatikonda/Mitter 2004

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 10 / 35

Feedbacks

A feedback u = F (x) generates controls (depending on x0)

where x(t, x0) solves

u(t) := F (x(t, x0)), x0 2 R n ,

˙x(t, x0) = f (x(t, x0), F (x(t, x0))), x(0, x0) = x0.

However, with a digital connection, only …nitely many controls can be

generated on …nite time intervals. We have to mimic the e¤ects of

feedback.

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 11 / 35

Exponential stabilization

Let K R n be a bounded set of initial states with 0 2 intK.

Assumption: There are M, α > 0 such that for all 0 6= x0 2 K there is

u 2 U with

kx(t, x0, u)k < Me αt kx0k for all t 0.

For stable systems it may also be of interest to increase the exponential

decay rate α.

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 12 / 35

Finitely many controls do not su¢ ce!

Consider the scalar system

˙x(t) = u(t), u(t) 2 R, x(0) = K := [ 1, 1].

Let α, M > 0 and …x T > 0. Then there is no …nite set of controls

u : [0, T ] ! R such that for every x0 6= 0 in K

Proof: By contradiction.

jx(t, x0, u)j < Me αt jx0j for all t 2 [0, T ].

True for general linear control systems if Re λ 0 for some eigenvalue λ.

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 13 / 35

Proof

Let R = fu1, ..., ung s.t. for every 0 6= x0 2 K := [ 1, 1] there is u 2 R

with

and de…ne

Z t

jx(t, x0, u)j = x0 + u(s)ds < Me

0

αt jx0j for all t 2 [0, T ]

Kj := fx0 2 K j jx(t, x0, uj )j < Me αt jx0j for all t 2 [0, T ]g.

W.l.o.g. u0(t) 0 62 R. Hence for every j one …nds tj 2 [0, T ] with

cj := max

t2[0,T ]

Z t

uj (s)ds =

0

Z tj

0

uj (s)ds > 0.

cj

Let x0 2 K with jx0j < minj 2M . Then jx0j < M jx0j < 1

2 minj cj, since

M > 1, and we arrive at the contradiction

Z tj

x0 + uj (s)ds

0

Z tj

0

uj (s)ds jx0j > cj

cj

2

> e αtj cj

2 > Me αtj jx0j .

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 14 / 35

De…nition of stabilization entropy

For T , ε > 0 a …nite set R of control functions is (T , ε)-spanning if for

every x0 2 K there is u 2 R with

kx(t, x0, u)k < e αt (ε + M kx0k ) for all t 2 [0, T ].

Let #R =: r(T , ε) be minimal. The stabilization entropy is de…ned as

h(α) := lim

ε!0 lim

T !∞

1

log r(T , ε).

T

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 15 / 35

Existence of …nite spanning sets

If there are M, α > 0 such that for all x0 2 K there is u 2 U with

kx(t, x0, u)k < Me αt kx0k for all t 0,

then for all ε > 0 and all T > 0 there are …nite (T , ε)-spanning

sets.

Proof. For x0 2 K pick u0 2 U with

kx(t, x0, u0)k < Me αt kx0k , t 2 [0, T ].

There is δ < ε/M such that for kx0 y0k < δ and t 2 [0, T ]

kx(t, y0, u0)k < e αt M kx0k

e αt M (kx0 y0k + ky0k)

< e αt (Mδ + M ky0k) < e αt (ε + M ky0k) .

By compactness of K, …nitely many xi and ui su¢ ce.

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 16 / 35

Existence of …nite spanning sets

If there are M, α > 0 such that for all x0 2 K there is u 2 U with

kx(t, x0, u)k < Me αt kx0k for all t 0,

then for all ε > 0 and all T > 0 there are …nite (T , ε)-spanning sets.

Proof. For x0 2 K pick u0 2 U with

kx(t, x0, u0)k < Me αt kx0k , t 2 [0, T ].

There is δ < ε/M such that for kx0 y0k < δ and t 2 [0, T ]

kx(t, y0, u0)k < e αt M kx0k

e αt M (kx0 y0k + ky0k)

< e αt (Mδ + M ky0k) < e αt (ε + M ky0k) .

By compactness of K, …nitely many xi and ui su¢ ce.

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 16 / 35

Properties of stabilization entropy

Let K R n be a compact set of initial states with 0 2 intK.

Assumption: There are M, α > 0 such that for all 0 6= x0 2 K there is

u 2 U with

kx(t, x0, u)k < Me αt kx0k for all t 0.

Then the stabilization entropy satis…es

1

T !∞

T

h(α) := lim

ε!0 lim

log r(T , ε) (L + α) n.

In particular, it is …nite.

Estimate from below using the trace of the Jacobian.

Proofs based on methods from topological theory of dynamical systems.

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 17 / 35

A formula for the stabilization entropy in the linear case

Consider

˙x(t) = Ax(t) + Bu(t), u(t) 2 U.

Assumption. There are M, α > 0 such that for all 0 6= x0 2 R n there is

u 2 U with

kx(t, x0, u)k < Me αt kx0k for all t 0.

Theorem

For every compact neighborhood K of the origin the stabilization entropy is

h(α) = htop(A + αI ) = ∑ (α + Re λ),

Re λ> α

where summation is over all eigenvalues λ of A with Re λ > α.

In particular, it is always …nite. Also of interest for stable A.

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 18 / 35

Interpretation as quantization

Consider a (T , ε)-spanning set R of control functions, i.e. for every

x0 2 K there is u 2 R with

De…ne for u 2 R

kx(t, x0, u)k < e αt (ε + M kx0k ) for t 2 [0, T ].

K (u, T ) := fx0 2 K kx(t, x0, u)k < e αt (ε + M kx0k ) for t 2 [0, T ] g.

Then

K = S

u2R K (u, T )

is a quantization of the set K of initial values and log 2 #R are the

associated bits.

However: For T 0 > T , we do not obtain a re…nement of this

quantization. Furthermore, we do not know what happens on [0, ∞).

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 19 / 35

A second way to count bits for stabilization

Fix ε > 0 and consider a decreasing function γ ε on [0, ∞) with

γ ε (0) = ε and lim

t!∞ γ ε (t) = 0

(e.g. γ ε (t) = εe αt ). A set S U is called γ ε -stabilizing if for

all x0 2 K there is u 2 S with

kx(t, x0, u)k < γ ε (t) + Me αt kx0k for all t 0.

Suppose that for T > 0

ST := fu j[0,T ] j u 2 Sg

is …nite. Then the number of bits for discerning the elements of S on

[0, T ] is log #ST .

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 20 / 35

A second way to count bits for stabilization

Fix ε > 0 and consider a decreasing function γ ε on [0, ∞) with

γ ε (0) = ε and lim

t!∞ γ ε (t) = 0

(e.g. γ ε (t) = εe αt ). A set S U is called γ ε -stabilizing if for all

x0 2 K there is u 2 S with

kx(t, x0, u)k < γ ε (t) + Me αt kx0k for all t 0.

Suppose that for T > 0

ST := fu j[0,T ] j u 2 Sg

is …nite. Then the number of bits for discerning the elements of

S on [0, T ] is log #ST .

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 20 / 35

Minimal bit rates

The bit rate for a γ ε -stabilizing set S U is

1

b(S) := limT !∞

T log #ST .

The minimal bit rate for exponential stabilization is

bstab(α, M) := lim inf

ε!0 γε

inf

S b(S),

where the inner in…mum is taken over all γ ε -stabilizing sets S U;

the outer in…mum is taken over all functions γ ε .

Quantization?

Relation to stabilization entropy ?

Note: The controls in S are de…ned on [0, ∞)!

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 21 / 35

Minimal bit rates

The bit rate for a γ ε -stabilizing set S U is

1

b(S) := limT !∞

T log #ST .

The minimal bit rate for exponential stabilization is

bstab(α, M) := lim inf

ε!0 γε

inf

S b(S),

where the inner in…mum is taken over all γ ε -stabilizing sets

S U; the outer in…mum is taken over all functions γ ε .

Quantization?

Relation to stabilization entropy ?

Note: The controls in S are de…ned on [0, ∞)!

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 21 / 35

Minimal bit rates

The bit rate for a γ ε -stabilizing set S U is

1

b(S) := limT !∞

T log #ST .

The minimal bit rate for exponential stabilization is

bstab(α, M) := lim inf

ε!0 γε

inf

S b(S),

where the inner in…mum is taken over all γ ε -stabilizing sets S U;

the outer in…mum is taken over all functions γ ε .

Quantization?

Relation to stabilization entropy ?

Note: The controls in S are de…ned on [0, ∞)!

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 21 / 35

Interpretation as quantization

Consider for u 2 S

K (u, T ) = fx0 2 K kx(t, x0, u)k < γ ε (t) + Me αt kx0k , t 2 [0, T ] g.

Then

K = S

u2S K (u, T )

is a quantization of the set K of initial values and log 2 #ST are the

associated bits.

For T 0 > T , we obtain a re…nement of the quantization for T .

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 22 / 35

Comparison to entropy

Theorem

Assume that there are M, α > 0 such that for all 0 6= x0 2 K there is

u 2 U with

kx(t, x0, u)k < Me α t kx0k for all t 0.

Then the minimal bit rate and the entropy for stabilization satisfy for

α 2 (0, α )

b stab(α, M) h stab(α , M).

Idea for the proof: Concatenate (Tn, ε n )-spanning controls.

Conjecture: In the linear case h stab(α, M) = b stab(α, M).

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 23 / 35

Compensators

Stabilize

˙x = Ax + Bu, y = Cx

using u = Fz with z generated by a compensator on the form

˙z = Jz + Nu Ky,

where J, K and N are matrices of appropriate dimensions. Suppose that

˙x

˙z =

A BF

KC J + NF

is exponentially stable.

Then, what is the entropy for the transfer from the system to the

compensator? When the implementation of y = Cx is not possible,

replace y = Cx by an input v(t) for the compensator.

˙x

˙z

= A BF

0 J + NF

x

z +

x

z

0

K v(t).

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 24 / 35

() March 2, 2011 2 / 5

Compensators

Stabilize

˙x = Ax + Bu, y = Cx

using u = Fz with z generated by a compensator on the form

˙z = Jz + Nu Ky,

where J, K and N are matrices of appropriate dimensions. Suppose that

˙x

˙z =

A BF

KC J + NF

is exponentially stable.

Then, what is the entropy for the transfer from the system to the

compensator? When the implementation of y = Cx is not possible,

replace y = Cx by an input v(t) for the compensator.

˙x

˙z

= A BF

0 J + NF

x

z +

x

z

0

K v(t).

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 25 / 35

Compensators

Then the entropy ˆh stab for α-stabilization of

is given by

˙x

˙z

= A BF

0 J + NF

x

z +

0

K v(t).

ˆh stab(α) = ∑ (α + Re λi ) + ∑ (α + Re µ i ),

Re λi > α

Re µ i > α

where summation is over the eigenvalues λi of A with Re λi > α and

over the eigenvalues µ i of J + NF with Re µ i > α, respectively.

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 26 / 35

Example

This entropy may be strictly larger than the stabilization entropy of the

system given by ˙x = Ax + Bu. Let

A =

0 1

1 0

, B = 1

0

, C = [1, 0].

The system is controllable and observable with eigenvalues 1 and hence

h stab(α) = α + 1 for α 2 (0, 1). Stabilization gives rise to

˙x

˙z

= A BF

0 A + BF + GC

x

z +

0

G v(t).

A direct computation shows that A + BF , A + GC, and A + BF + GC

cannot all be stable for any F and G and hence the entropy satis…es

ˆh stab(α) > h stab(α).

This is a classical problem in linear control: In general, one cannot …nd a

stable controller (of arbitrary order) for a stabilizable system.

SISO systems: parity interlacing condition on the zeros and poles.

Youla/Bongiorno/Lu (1974).

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 27 / 35

The approach so far

The basic idea for entropy exposed above applies to any feedback

control problem on [0, ∞):

A controller will generate control actions such that the desired behavior is

achieved for all initial values in a given set K. If continual measurement of

the output is not possible due to data rate constraints, the controller only

has a …nite amount of information available on any …nite interval [0, T ].

Hence it can only generate a …nite number of time-dependent controls

u(t), t 2 [0, T ].

The number of required controls on [0, T ] determines the minimal bits and

the growth rate as time tends to in…nity is the minimal bit rate on [0, ∞).

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 28 / 35

Invariance entropy for compact sets (with C. Kawan)

˙x(t) = Ax(t) + Bu(t), x(0) 2 K

where K R n is compact with nonvoid interior. What is the minimal bit

rate to keep the system in K?

Let T , ε > 0. R is (T , ε)-spanning, if for every x0 2 K there is u 2 R with

Let #R =: r(T , ε) be minimal.

Theorem

The invariance entropy satis…es

dist(x(t, x0, u), K ) < ε for all t 2 [0, T ].

hinv (K ); := lim

ε!0 lim

T !∞

1

T log r(T , ε) = ∑ Re λ.

Re λ>0

Far reaching generalizations involving Lyapunov exponents: Kawan ’11; for

random systems: da Silva ’12.

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 29 / 35

Controlled invariant subspaces (with Uwe Helmke)

A subspace V is controlled invariant, if

- there is a feedback F s.t. the solutions of ˙x = (A + BF )x remain in V ;

- for all x 2 V there is a control u with x(t, x0, u) 2 V for all t 0.

For a compact set K V with nonvoid interior and T , ε > 0 a set R of

control functions is (T , ε)-spanning if for every x0 2 K there is u 2 R

with

dist(x(t, x0, u), V ) := inf

v 2V kx(t, x0, u) vk < ε for all t 2 [0, T ].

Let #R =: r(T , ε) be minimal. The invariance entropy of V is

hinv (V ) := lim

ε!0 lim

T !∞

1

log r(T , ε).

T

This again is related to an entropy property of the uncontrolled system

and certain eigenvalues of A.

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 30 / 35

The Suboptimal H-In…nity Control Problem

Consider

˙x = Ax + Bu + Ew, z = Cx + Du,

Let γ > 0. Find a feedback u = Fx such that for x0 = 0

kzk L2

γ kwk L2

for all disturbances w 2 L2([0, ∞), R ` ) and stability holds.

Question: If the state x(t) is not available, what is the minimal bit rate

from the system to the controller?

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 31 / 35

De…nition of Entropy for H-In…nity Control I

We always keep a compact subset K R n with nonvoid interior …xed.

The controls u and the perturbations w are in

U = L2(0, ∞; R m ) and W = L2(0, ∞; R ` ),

If the problem is solvable, one …nds for every x0 2 K and w a control u

(given by Fx) such that

and

kzk L2(0,∞)

M kx0k + γ kwk L2(0,∞) .

Cx(t, x0, u, 0) ! 0 for w = 0.

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 32 / 35

Entropy for H-In…nity

Fix a perturbation w and let T , ε > 0.

Call R U a (T , ε)-spanning set of controls, if for every x0 2 K there

exists u 2 R such that

and

kzk = kCx( , x0, u, w) + Duk L2(0,T ) < M kx0k + γ kwk L2(0,∞)

kCx( , x0, u, 0)k < e αt (ε + M kx0k), t 0

(alternatively: kx( , x0, u, 0)k < e αt (ε + M kx0k), t 0.)

Denote by r(T , w) the minimal number of elements in a set R and de…ne

the entropy for the H∞-problem

h∞(A, B, C, D) := sup inf

w 2W α>0 lim

ε!0 lim

T !∞

1

log r(T , w).

T

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 33 / 35

A formula for the entropy

Theorem

Suppose that for every perturbation w 2 W there are α > 0 and M > 0

s.t. for every x0 2 R n and every T > 0 there is u 2 U with

and

kCx( , x0, u, w) + Duk L2(0,∞) < M kx0k + γ kwk L2(0,∞)

kCx( , x0, u, 0)k < Me αt kx0k , t 0.

Then the entropy for the H∞-problem is given by

h∞(A, B, C, D) = ∑ max(0, Re λ),

where summation is over the observable eigenvalues λ of A.

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 34 / 35

Conclusions and some open areas

We have considered an approach to determine minimal bit rates for

performing control tasks. It is based on ideas around topological entropy

from the theory of dynamical systems.

For linear systems, a formula for the exponential stabilization entropy has

been given, which also takes into account stable eigenvalues. If a speci…c

controller structure is given, these bounds may increase.

Several other control problems can be analyzed analogously.

Major challenges include:

- nonlinear problems (note that the tools are essentially nonlinear);

- interconnected systems and associated minimal bit rates, e.g. for

consensus problems. Here the development of new concepts will be

essential.

Fritz Colonius (Universität Augsburg) Minimal Bit Rates and Entropy July 11, **2012** 35 / 35