MATH 434/534

MATH 434/534
Theoretical Assignment 4 Solution
Chapter 4
4.11
(a) Show that if f(x) = log(x), then the condition number, c(x) = ∣ ∣
1 ∣
∣.
log(x)
(b) Using the above result (or otherwise), show that log(x) is ill-conditioned near x = 1.
-Solution-
(a)
∣ ∣ ∣∣f
Using the definition of a condition number of a function, c(x) = |x| ′ (x) ∣∣
. we have
|f(x)|
∣ c(x) = |x| ∣∣ 1
∣
x
log(x) = |x| 1
|x|
log(x) = 1
log(x) .
(b)
Form part (a), we have c(x) = 1
log(x) . Taking lim x→1 on both sides, we have
1
lim c(x) = lim
x→1 x→1 log(x) = ∞.
Therefore, log(x) is ill-conditioned near x = 1.
4.16
(a) How are Cond 2 (A) and Cond 2 (A −1 ) related
(b) Show that
(i) Cond(A) ≥ 1 for a norm ‖·‖ such that ‖I‖ ≥ 1;
(ii) Cond 2 (A T A) = {Cond 2 (A)} 2 ;
(iii) Cond(cA) = Cond(A) for any given norm.
-Solution-
(a)
∥
Cond 2 (A) = ‖A‖ ∥A −1∥ ∥
∥ ∥∥A ∥ =
−1 ∥ ‖A‖ = Cond2 (A −1 )
Therefore, Cond 2 (A) = Cond 2 (A −1 ).
(b)
(i) Show that Cond(A) ≥ 1 for a norm ‖·‖ such that ‖I‖ ≥ 1.
Cond(A) = ‖A‖ ∥ ∥ ∥A
−1 ∥ ∥ ∥ ≥
∥ ∥∥AA −1 ∥ ∥ ∥ = ‖I‖ = 1
(ii) Show that Cond 2 (A T A) = (Cond 2 (A)) 2 .
Cond 2 (A T A) =
=
∥
∥
∥A T ∥∥∥
A∥ A ( −1 A ) ∥ T −1 ∥∥2
2
∥
∥
∥A T ∥∥∥ (
A∥ (A −1 ) ) T T ( ∥ A −1) T ∥∥2
2

Since ∥ ∥A T A ∥ ∥2 = ‖A‖ 2 2
, we have
However,
Cond 2 (A T A) =
∥
∥A T ∥ ∥ ∥2 = ‖A‖ 2
. Therefore, we have
∥
∥
∥A T ∥∥∥ (
A∥ (A −1 ) ) T T ( ∥ A −1) T ∥∥2
2
= ‖A‖ 2 ( ) ∥ 2 ∥ A
−1
T 2
∥∥
Cond 2 (A T A) = ‖A‖ 2 ∥
∥
2
∥(A −1 ) T 2
∥
2
= ‖A‖ 2 ∥
2
∥(A −1 ) ∥ 2
∥
2
= { ‖A‖ 2
∥ ∥∥(A −1 ) ∥ ∥ ∥2
} 2
= {Cond2 (A)} 2
2
(iii) Show that Cond(cA) = Cond(A) for any given norm.
Cond(cA) = ‖cA‖ ∥ ∥ ∥(cA)
−1 1
∥ = |c| ‖A‖
∥
∥A −1∥ ∥ = Cond(A)
|c|
4.17
(a) Let A be an orthogonal matrix. Then show that Cond 2 (A) = 1.
(b) Show that Cond 2 (A) = 1 if and only if A is a scalar multiple of an orthogonal matrix.
-Solution-
(a)
Since A is an orthogonal matrix, A T = A −1 . Now look at the Cond 2 (A). Letting ρ(A) be
the maximum eigenvalue of A, we have
∥ ∥∥A
Cond 2 (A) = ‖A‖ ∥ −1 2 ∥2
√ √
= ρ(A T A) ρ ((A −1 ) T (A −1 ))
Since A T = A −1 , A T A = A −1 A = I and A = (A −1 ) T . Therefore we have
Cond 2 (A) =
=
=
√ √
ρ(A T A) ρ ((A −1 ) T (A −1 ))
√ √
ρ(I) ρ (A(A −1 ))
√ √
ρ(I) ρ (I)) = 1
(b)
First we will show that Cond 2 (A) = 1 if A is a scalar multiple of an orthogonal matrix.
Let O be an orthogonal matrix and c be a scalar constant. Then, let A = cO and ρ(A) be
the maximum eigenvalue of A. Then consider the Cond 2 (A)
Cond 2 (A) = Cond 2 (cO)
= Cond 2 (O)
√
√
= ρ(O T O) = ρ(I) = 1
2

Now suppose that Cond 2 (A) = 1. Then we will show A is a scalar multiple of an orthogonal
matrix. From the singular value decomposition of A (Please read page 23 carefully), we have
orthogonal matrices U ∈ R n×n and V ∈ R n×n
A = UΣV T
where Σ = diag(σ 1 , . . . , σ n ) ∈ R n×n and σ 1 ≥ σ 2 ≥ · · · ≥ σ n where σ i for i = 1, 2, . . . , n are
called singular values of a matrix A.
Now considering that Cond 2 (A) = 1, from “Some Properties of the Condition Number
of Matrix” on page 67, we have
1 = Cond 2 (A) = σ max
σ min
.
This implies σ max = σ min . Therefore, we have σ max = σ 1 = σ 2 = · · · = σ n = σ min . Now let
σ i = c for i = 1, 2, . . . , n. From A = UΣV T , we have
⎛
⎞
c 0 · · · 0
0 c · · · 0
A = U
⎜ .
⎝ . . . ⎟
. .
V T
⎠
0 0 · · · c
= cUIV T
= cUV T .
However, U and V T are orthogonal matrices. Letting O = UV T , we have A = cUV T = cO,
where O is an orthogonal matrix because the product of orthogonal matrices is an orthogonal
matrix.
Therefore, Cond 2 (A) = 1 if and only if A is a scalar multiple of an orthogonal matrix.
4.18
Let U = (u ij ) be a nonsingular upper triangular matrix. Then show that
Cond ∞ (U) ≥ max |u ii|
min |u ii | .
Hence construct a simple example of an ill conditioned nondiagonal symmetric positive definite
matrix.
-Solution-
Let U be an n × n upper triangular matrix. First consider what U −1 looks like, especially
its diagonal entries. Since U is an upper triangular matrix, U −1 is also an upper triangular
matrix. Now letting
⎛
⎞
⎛
⎞
a 11 a 12 · · · a 1n
b 11 b 12 · · · b 1n
0 a
U =
22
.. . a2n
⎜ .
and U −1 0 b
=
22
.. . b2n
⎝ . . .. ⎟
⎜ .
,
. ⎠
⎝ . . .. ⎟ . ⎠
0 · · · 0 a nn 0 · · · 0 b nn
we have
⎛
⎞
a 11 b 11 ∗ · · · ∗
. I = UU −1 0 a
=
22 b 22 . . .
⎜ .
⎝ . .. .
.
.. ⎟ ∗ ⎠
0 · · · 0 a nn b nn
3

Therefore, we can see that b ii = 1
a ii
. By the definition of a maximum row-sum norm ‖·‖ ∞
,
that is, ‖A‖ ∞
= max 1≤i≤m
∑ nj=1
|a ij |, we have
‖U‖ ∞
≥ max |u ii |
and
∥
∥U −1∥ ∥ ∥∞ ≥
1
min |u ii | .
Take
U =
⎛
⎜
⎝
0.0001 2 3
0 1 12
0 0 12
⎞
⎟
⎠
Then Cond(U) = 4.9025 × 10 5 .
4.19
Let A = LDL T be a symmetric positive definite matrix where L is a unit lower triangular
matrix. Let D = diag(d ii ). Then show that
Cond 2 (A) ≥ max(d ii)
min(d ii ) .
Hence construct an example for an ill-conditioned nondiagonal symmetric positive definite
matrix.
-Solution-
Since A = LDL T is a symmetric positive definite matrix and L is a lower triangular matrix,
we have the diagonal entries of a diagonal matrix, d ii > 0 for i = 1, 2, . . . , n. Therefore, we
have
A = LDL ⎛
T
⎞ ⎛
1 0 0
= ⎜
⎝ ∗ . .. ⎟ ⎜
0 ⎠ ⎝
∗ ∗ 1
⎞ ⎛
d 11 0 0 1 ∗ ∗
. 0 . . ⎟ ⎜
0 ⎠ ⎝ 0 . .. ∗
0 0 d nn 0 0 1
⎞
⎟
⎠
where d ii > 0 for i = 1, 2, . . . , n.
Since d ii > 0 for i = 1, 2, . . . , n, we have
A = LDL T = ( LD 1 2
) (
D
1
2 L
T ) = ( D 1 2 L
) (
D
1
2 L
) T
where D 1 2
have
= diag( √ d 11 , √ d 22 , . . . , √ d nn ). Let a lower triangular matrix ̂L = LD 1 2 . Then we
A = LDL T
= ( D 1 2 L
) (
D
1
2 L
) T
=
̂L ̂L
T
Now consider Cond 2 (A). Using Cond 2 (A T A) = {Cond 2 (A)} 2 , we have
Cond 2 (A) = Cond 2 ( ̂L ̂L T )
= { Cond 2 ( ̂L T ) } 2
· · · (∗)
4

Now look at only Cond 2 ( ̂L T ).
Cond 2 ( ̂L T ) = ∥ ∥ ̂L ̂L ∥ ∥ T ∥∥∥ (
∥2 ̂L ̂L ) ∥ T −1 ∥∥2
(
=
∥ ̂L ) T T ( ̂L )∥ ∥ T ∥ ∥∥∥∥ { (
∥∥2 ̂L −1) T } T ( ̂L −1) T
∥ 2
= ∥ ̂L ∥ T 2
( ∥
∥ 2 ∥ ̂L −1) T 2
∥∥ = ∥ ∥ ̂L ∥ 2
∥ ∥ ̂L −1∥ ∥2
· · · (∗∗)
2 2 2
Now consider ∥ ̂L
∥ and ∥ ̂L −1∥ ∥ ∥2 .
2
Using the standard basis e i = (0, 0, . . . , 0, 1, 0, 0, . . . , 0), and the definition of matrix norms
} {{ }
i−1 zeros
‖A‖ p
= max ‖x‖p =1 ‖Ax‖ p
, we have
∥ ̂L ∥ ∥2 = max ∥ ̂Le
∥ √
∥∥2 i = max d ii
‖x‖ 2 =1
Since the diagonal entries of ̂L −1 are the reciprocal of the diagonal entries of ̂L, it can be
expressed as 1
d ii
for i = 1, 2, . . . , n. Therefore, we have
∥ ̂L −1∥ ∥ ∥2 = max ∥ ̂L
∥ −1 ∥∥2
e i = max 1 1
√ =
‖x‖ 2 =1
dii min √ d ii
From (∗∗), we have
Cond 2 ( ̂L T ) =
( ) ( ) 2 2
1
max
√d ii
min √ d ii
= max(d ii)
min(d ii )
From (∗) and (∗ ∗ ∗), since max(d ii)
min(d ii
, now we have
)
Therefore, we have
Cond 2 (A) = { Cond 2 ( ̂L T ) } 2
=
· · · (∗ ∗ ∗)
( ) 2
max(dii )
≥ max(d ii)
min(d ii ) min(d ii ) .
Cond 2 (A) ≥ max(d ii)
min(d ii ) .
The Hilbert matrix is a symmetric positive definite. However, it is known as an ill-conditioned
matrix.
4.20
Prove that for a given norm, Cond(AB) ≤ Cond(A)Cond(B)
-Solution-
Cond(AB) = ‖(AB)‖ ∥ ∥ ∥(AB)
−1 ∥
≤ ‖A‖ ‖B‖ ∥ ∥ ∥
∥A
−1 ∥ ∥∥B ∥ −1 ∥
= ‖A‖ ∥ ∥ ∥
∥A
−1 ∥ ‖B‖ ∥∥B ∥ −1 ∥ = Cond(A) · Cond(B)
Therefore, Cond(AB) ≤ Cond(A) · Cond(B).
5