Linear Algebra, Theory And Applications, 2012a

326 SELF ADJOINT OPERATORS
This part follows fairly easily from taking the ln and then setting partial derivatives equal to
0. The estimation of Σ is harder. However, it is not too hard using the theorems presented
above. I am following a nice discussion given in Wikipedia. It will make use of Theorem
7.5.3 on the trace as well as the theorem about the square root of a linear transformation
given above. First note that by Theorem 7.5.3,
(x i −m) ∗ Σ −1 (x i −m) = trace ( (x i −m) ∗ Σ −1 (x i −m) )
= trace ( (x i −m)(x i −m) ∗ Σ −1)
Therefore, the thing to maximize is
n∏
(
1
det (Σ) exp − 1 1/2 2 trace ( (x i −m)(x i −m) ∗ Σ −1))
i=1
= det ( Σ −1) n/2
exp
(
− 1 n
)
2 trace ∑
(x i −m)(x i −m) ∗ Σ −1
⎛
S
⎞
{ }} {
= det ( Σ −1) n/2 n∑
exp ⎜
⎝ −1 2 trace (x i −m)(x i −m) ∗ Σ −1 ⎟
⎠
i=1
i=1
≡ det ( Σ −1) n/2
exp
(
− 1 2 trace ( SΣ −1))
where S is the p × p matrix indicated above. Now S is symmetric and has eigenvalues which
are all nonnegative because (Sy, y) ≥ 0. Therefore, S has a unique self adjoint square root.
Using Theorem 7.5.3 again, the above equals
det ( Σ −1) (
n/2
exp − 1 (
2 trace S 1/2 Σ −1 S 1/2))
Let B = S 1/2 Σ −1 S 1/2 and assume det (S) ≠0. Then Σ −1 = S −1/2 BS −1/2 .
equals
det ( S −1) det (B) n/2 exp
(− 1 )
2 trace (B)
The above
Of course the thing to estimate is only found in B. Therefore, det ( S −1) can be discarded
in trying to maximize things. Since B is symmetric, it is similar to a diagonal matrix D
which has λ 1 , ··· ,λ n down the diagonal. Thus it is desired to maximize
( p∏
) n/2 ( )
λ i exp − 1 p∑
λ i
2
i=1
i=1
Taking ln it follows that it suffices to maximize
n
p∑
ln λ i − 1 2
2
i=1
Taking the derivative with respect to λ i ,
n
2
p∑
i=1
1
− 1 λ i 2 =0
λ i