Numerical Methods Contents - SAM

△
Example 7.5.2 (Linear regression for stationary Markov chains).
Sequence of scalar random variables: (Y k ) k∈Z = Markov chain
Assume:
stationary (time-independent) correlation
Expectation E(Y i−j Y i−k ) = u k−j ∀i,j,k ∈ Z , u i = u −i .
7.5 Toeplitz matrix techniques
Example 7.5.1 (Parameter identification for linear time-invariant filters).
Model:
finite linear relationship
∃x = (x 1 ,...,x n ) T ∈ R n : Y k =
with unknown parameters x j , j = 1, . ..,n:
n∑
x j Y k−j ∀k ∈ Z .
j=1
for fixed i ∈ Z
• (x k ) k∈Z m-periodic discrete signal = known input
• (y k ) k∈Z m-periodic known output signal of a linear time-invariant filter, see Ex. 7.1.1.
• Known: impulse response of filter has maximal duration n∆t, n ∈ N, n ≤ m
Estimator
∣
n∑ ∣ ∣∣ 2
x = argmin E∣Y i − x j Y i−j
x∈R n j=1
cf. (7.1.1)
n−1
∃h = (h 0 ,...,h n−1 ) T ∈ R n ∑
, n ≤ m : y k = h j x k−j . (7.5.1)
x k input signal
y k output signal
time
time
j=0
Ôº½ º
E|Y i | 2 − 2
n∑
x j u k +
j=1
n∑
x k x j u k−j → min .
k,j=1
x T Ax − 2b T x → min with b = (u k ) n k=1 , A = (u i−j) n i,j=1 .
Lemma 4.1.2 ⇒ x solves Ax = b (Yule-Walker-equation, see below)
A ˆ= Covariance matrix:
s.p.d. matrix with constant diagonals.
Ôº½ º
✸
Parameter identification problem: seek h = (h 0 ,...,h n−1 ) T ∈ R n with
⎛
⎞
x 0 x −1 · · · · · · x 1−n ⎛ ⎞
x 1 x 0 x −1 .
. x 1 x . ⎛ ⎞ y 0
0
..
h ... . 0
.
.. .
‖Ah − y‖ 2 =
. . .. . .
.. x −1
x n−1 x 1 x ⎜ ⎟
0
⎝ . ⎠ − → min .
⎜ x n x n−1 x ⎜ ⎟
1 ⎟ h n−1 ⎝ . ⎠
⎝
.
. ⎠ y m−1 ∥ x m−1 · · · · · · x ∥ m−n 2
➣ Linear least squares problem, → Ch. 6 with Toeplitz matrix A: (A) ij = x i−j .
System matrix of normal equations (→ Sect. 6.1)
m∑
M := A H A , (M) ij = x k−i x k−j = z i−j due to periodicity of (x k ) k∈Z .
k=1
➣
M ∈ R n,n is a matrix with constant diagonals & s.p.d.
(“constant diagonals” ⇔ (M) i,j depends only on i − j)
Ôº½ º
✸
Matrices with constant diagonals occur frequently in mathematical models. They generalize of circulant
matrices → Def. 7.1.3.
Note: “Information content” of a matrix M ∈ K m,n with constant diagonals, that is, (M) i,j = m i−j ,
is m + n − 1 numbers ∈ K.
Definition 7.5.1 (Toeplitz matrix).
T = (t ij ) n i,j=1 ∈ Km,n is a Toeplitz matrix, if there is a
vector u = (u −m+1 ,...,u n−1 ) ∈ K m+n−1 such that
t ij = u j−i , 1 ≤ i ≤ m, 1 ≤ j ≤ n.
7.5.1 Toeplitz matrix arithmetic
⎛
⎞
u 0 u 1 · · · · · · u n−1
u −1 u 0 u 1 .
T =
. . .. ... . .. .
⎜ . ... . .. ... .
⎝ . .
⎟
.. ... u 1 ⎠
u 1−m · · · · · · u −1 u 0
T = (u j−i ) ∈ K m,n = Toeplitz matrix with generating vector u = (u −m+1 ,...,u n−1 ) ∈ C m+n−1
Ôº¾¼ º