ECE 174 Homework # 4 Solutions - UCSD DSP Lab

ECE 174 Homework # 4 Solutions
Reminder: Midterm is Tuesday, May 19, 2009
The midterm is closed notes and book. Bring paper and pen only. No Electronic devices at
all are allowed. You are encouraged to study the homework and sample midterm solutions
well.
Solutions
1. Recall that we (and Meyer) define the inner product to be linear in the second argument.
1 Also recall that 〈x1, x2〉 = 〈x2, x2〉.
(a) 〈x1, αx2〉 = α 〈x1, x2〉 = α 〈x1, x2〉 = α 〈x2, x1〉 = 〈x2, αx1〉 = 〈αx1, x2〉.
(b) 〈α1x1 + α2x2, x〉 = 〈x, α1x1 + α2x2〉 = α1 〈x, x1〉 + α2 〈x, x2〉
= α1 〈x1, x〉 + α2 〈x2, x〉
Note that in part (b) we obtain the property of additivity in the first argument as a
special case (just take α1 = α2 = 1).
2. Recall that the operators A : X → Y, B : X → Y, and C : Y → Z are linear and that
the adjoint operator, A ∗ , is defined by
〈A ∗ x1, x2〉 = 〈x1, Ax2〉 .
You also need to recall the properties of the inner product (such as linearity in the
second argument) and the additional properties proved in Problem 1 above.
(a) 〈y, αAx〉 = 〈y, Aαx〉 = 〈A ∗ y, αx〉 = 〈αA ∗ y, x〉.
(b) 〈y, (A + B)x〉 = 〈y, Ax + Bx〉 = 〈y, Ax〉 + 〈x, Bx〉 = 〈A ∗ y, x〉 + 〈B ∗ y, x〉
〈A ∗ y + B ∗ y, x〉 = 〈(A ∗ + B ∗ )y, x〉.
(c) The fact that (αA + βB) ∗ = αA ∗ + βB ∗ follows immediate from properties (b)
and (a), in that order.
(d) 〈x, A ∗ y〉 = 〈A ∗ y, x〉 = 〈y, Ax〉 = 〈Ax, y〉.
(e) 〈z, CAx〉 = 〈C ∗ z, Ax〉 = 〈A ∗ C ∗ z, a〉.
(f) 〈A ∗ αy, x〉 = 〈αy, Ax〉 = α 〈y, Ax〉 = α 〈A ∗ y, x〉 = 〈αA ∗ y, x〉.
(g) 〈A ∗ (y1 + y2), x〉 = 〈y1 + y2, Ax〉 = 〈y1, Ax〉 + 〈y2, Ax〉 = 〈A ∗ y1, x〉 + 〈A ∗ y2, x〉
= 〈A ∗ y1 + A ∗ y2, x〉.
(h) Linearity of A ∗ follows from properties (g) and (f), in that order.
1 Whereas many other authors define the inner product to be linear in the first argument. Of course, in
the real vector space case the inner product is linear in both arguments so that the distinction disappears.
1

3. Recall that the inner–products are defined as,
〈x1, x2〉 = x H 1 Ωx2 and 〈y1, y2〉 = y H 1 W y2 ,
where Ω and W are hermitian (i.e., Ω H = Ω and W H = W ) and positive–definite (and
hence both are invertible). Note that in this case their inverses are also hermitian,
positive–definite.
(a) 〈y, Ax〉 = y H W Ax = y H W AΩ −1 Ωx = (Ω −1 A H W )y H Ωx = (Ω −1 A H W )y, x .
(b) For r(A) = n, A has full column rank and it must be the case that m ≥ n. Since
A is possibly over–determined, we solve the least squares problem by enforcing
the geometric condition that y − Ax ∈ R(A) ⊥ = N (A ∗ ). This yields the normal
equations,
A ∗ Ax = A ∗ y .
Because r(A) = n, the n×n matrix A ∗ A also has rank n and is therefore invertible.
(This fact is consistent with the nullspace of A being trivial, so that the least–
squares problem must have a unique solution.) Thus, we have that
r(A) = n ⇒ x = (A ∗ A) −1 A ∗ y ,
for any value of y. It must therefore be the case that
r(A) = n ⇒ A + = (A ∗ A) −1 A ∗ ,
where A ∗ = Ω −1 A H W as determined in Part (a). With W a full rank hermitian
matrix and r(A) = n, it is the case that A H W A is invertible and as a consequence
the pseudoinverse, A ∗ , is independent of the weighting matrix Ω,
r(A) = n ⇒ A + = (A H W A) −1 A H W .
(c) For r(A) = m, A has full row–rank and therefore it must be the case that n ≥ m.
Note that A is onto, and therefore y = Ax is solvable for all y. However, the system
is possibly underdetermined, so we want to look for a minimum norm solution.
This requires that we enforce the constraint that any solution to y = Ax must
also satisfy the geometric condition that x ∈ N (A) ⊥ = R(A ∗ ). This condition is
equivalent to,
x = A ∗ λ ,
for some vector λ.
This condition, together with the requirement that x be a solution to y = Ax, yields,
AA ∗ λ = y .
2

Because r(A) = m, the m × m matrix AA ∗ also has rank n and is invertible. Thus,
which yields the result that
for all y. Thus,
λ = (AA ∗ ) −1 y
r(A) = m ⇒ x = A ∗ (AA ∗ ) −1 y ,
r(A) = m ⇒ A + = A ∗ (AA ∗ ) −1 .
With r(A) = m and Ω −1 a hermitian full–rank matrix, it is the case that the m × m
matrix AΩ −1 A H is invertible. With the fact that A ∗ = Ω −1 A H W , this yields the fact
that for r(A) = m, the pseudoinverse is independent of the weighting matrix W ,
r(A) = m ⇒ A + = Ω −1 A H (AΩ −1 A H ) −1 .
4. The minimum power–dissipation problem. We will solve part (b) first, then part (a).
(b) x = (I1, I2, I3) T , A = (1, 1, 1), and y = I. Note that the rank of A is m = 1, so
that A is onto (has full row rank) and therefore A + = A ∗ (AA ∗ ) −1 . The inner product
on the domain (“input space”) has weighting matrix Ω = diag(R1, R2, R3).
With this setup we have that the total power dissipated in the resistors is P =
x 2 . The inner product on the codomain (“output space”) can be taken to have
a weighting “matrix” W = 1. The adjoint operator is
Also,
This yields,
A ∗ = Ω −1 A T W = Ω −1 A T =
(AA ∗ ) −1 = (AΩ −1 A T ) −1 =
1
R1
1
+ 1
R2
A + = A ∗ (AA ∗ ) −1 = Ω −1 A T (AΩ −1 A T ) −1 =
1
,
R1
1
,
R2
1
R3
+ 1
R3
Thus the optimum currents through the resistors are
⎛ ⎞
I1
⎝ ⎠ = x = A + I
y =
I2
I3
=
R2R3 + R1R3 + R1R2
3
T
.
R1R2R3
R2R3 + R1R3 + R1R2
1
R2R3 + R1R3 + R1R2
⎛
⎝
R2R3
R1R3
R1R2
⎞
⎠ .
⎛
⎝
.
R2R3
R1R3
R1R2
⎞
⎠ .

Note that I1 + I2 + I3 = I as required. The vector of optimal voltages is given by
the vector Ωx and yields the answer,
V1 = V2 = V3 =
R1R2R3
R2R3 + R1R3 + R1R2
The optimum (minimum) solution power dissipation is computed as,
Note that
P opt = x 2 = x T Ωx = (A + y) T ΩA + y = y T A +T ΩA + y .
A +T ΩA + = (AΩ −1 A T ) −1 AΩ −1 A T (AΩ −1 A T ) −1 = (AΩ −1 A T ) −1 ,
which was computed above. Therefore,
P opt = I 2
which is dimensionally correct (“P = I 2 R”).
(a) x = (V1, V2, V3) T ,
R1R2R3
R2R3 + R1R3 + R1R2
1
A = ,
R1
1
,
R2
1
,
R3
and y = I. The rank of A is m = 1, so that A is onto (has full row rank) and
therefore A + = A ∗ (AA ∗ ) −1 . The inner product on the domain (“input space”)
has weighting matrix,
1
Ω = diag ,
R1
1
,
R2
1
.
R3
With this setup we again have that the total power dissipated in the resistors is
P = x 2 . And again the inner product on the codomain (“output space”) can
be taken to have a weighting “matrix” W = 1. The adjoint operator is
Also,
This yields,
(AA ∗ ) −1 = (AΩ −1 A T ) −1 =
A ∗ = Ω −1 A T W = Ω −1 A T = (1, 1, 1) T .
1
R1
1
+ 1
R2
+ 1
R3
A + = A ∗ (AA ∗ ) −1 = Ω −1 A T (AΩ −1 A T ) −1 =
4
=
,
I .
R1R2R3
R2R3 + R1R3 + R1R2
R1R2R3
R2R3 + R1R3 + R1R2
.
⎛ ⎞
1
⎝1⎠
.
1

With the optimal solution given by x = A + y this yields optimum voltages of
V1 = V2 = V3 =
R1R2R3
R2R3 + R1R3 + R1R2
exactly as before. The optimum (minimum) solution power dissipation is again
computed as,
where
P opt = x 2 = x T Ωx = (A + y) T ΩA + y = y T A +T ΩA + y ,
A +T ΩA + = (AΩ −1 A T ) −1 AΩ −1 A T (AΩ −1 A T ) −1 = (AΩ −1 A T ) −1 ,
which was computed above. Therefore,
exactly as before.
P opt = I 2
R1R2R3
R2R3 + R1R3 + R1R2
5. The two situations correspond to the mutually exclusive cases of a) b is in the range
of A, and b) nonzero b is not in the range of A.
6. Meyer Problem 4.6.7. Let the data pairs be given by (xk, yk), k = 1, · · · , m = 11.
(E.g., (x1, y1) = (−5, 2), etc.). For both the line fit and the quadratic fit, we want to
place the problem in the vector–matrix form,
y = Aα
for A m × n, where n is the dimension of the unknown parameter vector α, and solve
for α in the least-squares sense. For the linear fit we have,
⎛ ⎞
y1
⎜ y2
⎟
y = ⎜ ⎟
⎝ . ⎠
ym
=
⎛
1
⎜
⎜1
⎜
⎝.
⎞
x1
x2
⎟
⎟ α0
⎟ = Aα ,
. ⎠ α1
1 xm
with n = 2. While for the quadratic fit we have,
⎛ ⎞
y1
⎜ y2
⎟
y = ⎜ ⎟
⎝ . ⎠
ym
=
⎛
1
⎜
⎝
x1 x2 1 x2
1
x2 . .
2
.
1 xm x2 ⎞
⎛
⎟ ⎝
⎠
m
α0
α1
α2
⎞
,
⎠ = Aα ,
with n = 3. Note that in both cases we have the data to fill in numerical values of y
and A. In both cases we have that the matrix A is full rank (you can check in matlab,
5
I ,

ut it will usually be true for this type of problem). Thus the least squares solution
can be determined as,
α = (A T A) −1 A T y .
The optimal least–squares error has magnitude,
e 2 = y − Aα 2 = (y − Aα) T (y − Aα) ,
which can be computed for both the quadratic and linear fits. In this case you will
find that the quadratic fit provides a much tighter fit to the data. Once you have the
parameters at hand, you can perform the prediction by plugging in the new value for
xk.
Question:
Given measured data (xk, yk), k = 1, · · · , m can you fit the model,
Or the model,
y ≈ α0 + α3x 3 + α9x 9 ?
y ≈ αx + βe x + γ cos(12x 3 ) ?
6