Eigenvalues and Eigenvectors

Eigenvalues and Eigenvectors
For second-order and higher-order differential equations, it is often helpful to express
the equation in a matrix-vector form. In order to work with such systems, we need to
have a working knowledge of some facts from linear algebra.
Let A represent an n n matrix, let x represent an n 1 vector, and let represent a
scalar. An eigenvalue, i , is a solution to:
|A − I| 0,
where I denotes the identity matrix:
I ≡
1 0 0
0 1 0
0 0 1
Suppose that A is given by:
a 11 a 12 a 1n
A ≡
a 21 a 22 a 2n
a n1 a n2 a nn
The eigenvalues are therefore solutions to the polynomial obtained by evaluating the
determinant:
a 11 − a 12 a 1n
a 21 a 22 − a 2n
a n1 a n2 a nn −
0
If A is an n n matrix, there will be n values of that satisfy the characteristic equation,
|A − I| 0.
The solutions, i , come in three varieties. Each, or some of the eigenvalues may be
distinct. Each, or some of the eigenvalues may be repeated. Each, or some of the
eigenvalues may occur in complex conjugate pairs. Repeated eigenvalues occur
when the characteristic equation can be factored in a form that include terms of the
form 1 − i m . In this case the eigenvalue, i , is repeated m times. Complex
conjugate pairs occur when the characteristic equation can be factored in a form that
includes one or more terms of the form, a − 2 b 2 . Some other interesting

2
properties of the eigenvalues include that:
1 2 n |A| and 1 2 … n traceA
As an example, suppose that n is equal to 2. In this case, the characteristic equation
is given by:
a 11 − a 12
a 21
a 22 −
2 − a 11 a 22 a 11 a 22 − a 12 a 21 0
Define ≡ −a 11 a 22 and ≡ a 11 a 22 − a 12 a 21 . The eigenvalues therefore given by:
i − 2
2
2
−
For each of the n eigenvalues, i , there corresponds an eigenvector, i . Eigenvectors
satisfy the equation:
A i i i ,or:A − i I i 0
Because A − i , by construction, is singular, there will only be n − 1 independent
equations to solve for the n elements of each of the eigenvectors. Eigenvectors give a
direction in n-space, but not a distance in n-space. Consider the case of n 2. In this
case i solves:
a 11 − i a 12 i1
a 21 a 22 − i i2
0
0
This expression consists of two linear equations, only one of which is independent.
Arbitrarily choose the first of these and we find that:
a 11 − i i1 a 12 i2 0, and so:
i1
i2
−a 12
a 11 − i
A convenient normalization is to set 2 i1 2 i2 1.
Systems of Differential Equations
Facts from linear algebra are helpful in solving systems of first-order linear,
homogeneous differential equations. Such systems take the form ẋ Ax, where A
represent an n n matrix, and ẋ and x represent n 1 vectors. Before engaging in a
study of these more general cases, let us first turn our attention to the case of the
homogeneous second-order differential equation with constant coefficients. That is,
consider:
ÿ aẏ by 0

3
Now consider a change in variables in which we define x 1 ẏ and x 2 y. Therefore
ÿ ẋ 1 , ẏ x 1 , and y x 2 . We may therefore rewrite the original equation as:
ẋ 1
ẋ 2
−a −b
1 0
x 1
x 2
Let us guess a solution to the original differential equation that takes the form:
yt ce t
Now take the derivatives of this solution, ẏ ce t and ÿ c 2 e t . Now plug these
expressions back into the original differential equation to obtain:
c 2 e t ace t bce t 2 a bce t 0,
if and only if 2 a b 0, forc ≠ 0, e t ≠ 0.
The solutions to the polynomial equation, 2 a b 0, are identical to the
eigenvalues of the matrix A. That is:
|A − I|
−a − −b
1 −
2 a b 0
In this, a 2 2 case, there are two eigenvalues, 1 and 2 . The solution therefore
takes the form:
ẏ
y
x 1
x 2
c 11 e 1t c 12 e 2t
c 21 e 1t c 22 e 2t
We are primarily interested in the solution for yt, which is given by:
yt c 21 e 1t c 22 e 2t
The constants c 21 and c 22 are determined from the initial conditions. Once we find c 21
and c 22 , it is straightforward to solve for c 11 and c 12 .
Let us next consider how to deal with solutions to the generalized homogeneous
system of two differential equations.
ẋ Ax
a 11 a 12
a 21 a 22
x.
So far, we are dealing with 2 2 systems, but the techniques you are learning here
generalize to systems in higher dimensions. Systems of differential equations come
up all the time in economics; the classic example in macroeconomic theory is the
model of optimal growth in which both consumption and capital stock are determined
simultaneously by the savings decision of the representative household. Thus this
discussion is not an arid exercise in linear algebra, differential equations, and complex

4
variables. Instead it constitutes the bread and butter of a lot of modern
macroeconomics.
It should not be not surprising to you now that the cases of interest are essentially a
taxonomy of the characteristic equation p a 11 − a 22 − − a 21 a 12 . There are
four cases: (1) 1 2 0; (2) both roots are real; (3) the two roots are complex
conjugates; and (4) 1 2 ≠ 0, in which the roots are the same real number.
Let’s dispose of the first case immediately. If 1 2 0, then either A
0 0
0 0
0 a 12
0 0
or A
or A
. If any of these is true, then we are not
0 0
a 21 0
dealing with a system of differential equations because either one or both variables is
stationary and no variable’s rate of change depends upon its own value.
If A
0 0
0 0
, then xt x0.
If A
0 a 12
0 0
, then xt
x 1 0 a 12 x 2 0t
x 2 0
.
If A
0 0
a 21 0
, then xt
x 1 0
x 2 0 a 21 x 1 0t
,
where x0
x 1 0
x 2 0
is given.
The second case is not so trivial. Let 1 ≠ 2 both be real. Let M be the matrix whose
columns are the two eigenvectors for 1 , and 2 .Since
AM M M 1 0
, we know that M −1 AM. As long as M is non-singular,
0 2
M −1 exists. For distinct eigenvalues, ( 1 ≠ 2 ), M is always non-singular.
Now we do a nifty change of coordinates. Write x My and note that y M −1 x.
Then ẋ Mẏ since the M matrix is a matrix of constants. Hence we can rewrite our
system as Mẏ ẋ Ax AMy and thus
ẏ M −1 AMy y.

5
y 1 0exp 1 t
But the solution to this equation is just yt
. We are only
y 2 0exp 2 t
halfway there because this solution is in the wrong basis; we have to get rid of the
change of coordinates. In particular, we typically know x0, not y0. Recalling that
x My, we can write y0 M −1 x0. So that pins down the initial condition in the
changed coordinates. But also xt Myt M
answer.
y 10exp 1 t
y 2 0exp 2 t
, and that is our
To make things concrete, always follow these exact steps. First, calculate the
eigenvalues and some corresponding eigenvectors. Second write the matrix whose
columns are the eigenvectors and invert it. Third, use this inverse and your
knowledge about the initial conditions to figure out y0. Fourth, use y0 and the
matrix whose columns are the eigenvectors to get your answer in the original basis.
5 3
1
Here is an example. Let A
with x0 . Then 1 2 and
−6 −4
1
2 −1 are eigenvalues, and the matrix whose columns are the corresponding
eigenvectors is M
1 −1
−1 2
. The inverse of this matrix is M −1
2 1
1 1
. So
the initial condition in the changed coordinates is y0
2 1
1 1
1
1
3
2
.
Hence the solution in the changed basis is yt
to our original problem is
3exp2t
2exp−t
. Finally, the solution
xt Myt
1 −1
−1 2
3exp2t
2exp−t
3exp2t − 2exp−t
−3exp2t 4exp−t
.
Check that this is the right answer.
We may also write a simple set of Matlab commands to perform the diagonalization
procedure. This procedure also works for higher-order linear systems. The set of
commands is given below.
A[..]: Square matrix
[M,lambda]eig(A)

6
tsym(’t’)
zexp(diag(lambda)*t)
x0[..]: column vector of initial conditions
y0inv(M)*x0
yy0.*z
xM*y
The third case occurs when the two roots are complex conjugates. The standard
diagonalization procedure still works. The eigenvectors also occur in conjugate pairs.
Much of the solution process involves working with complex vectors and matrices.
Operations like computing the inverse of a matrix work with complex values as well as
simple real numbers. However, when the process concludes, we obtain a real valued
time function for the solution. Unfortunately, this procedure can give results which are
not easy to interpret. To see this, try using the suggested Matlab technique for solving
a 2 2 system with complex roots. It may appear that the time functions in the
solution are complex. In fact, this is not the case.
An alternate solution technique separates out the real and imaginary parts of the
eigenvalues and eigenvectors and performs an essentially idetical set of steps. For
complex conjugate eigenvalues, can write 1 a bi and 2 a − bi with b 0. We
are going to do a very similar process now, but we shall endeavour to write
A MT a,b M −1 a −b
where T a,b
is the canonical (anti-clockwise) ”twisting”
b a
matrix. Of course, if we find a clever change of coordinates and write x My, then
ẋ Ax will be equivalent to ẏ M −1 AMy T a,b y.
HowdowefindtheM matrix? Just solve for a complex-valued eigenvector that
corresponds to the eigenvalue 1 a bi. Such an eigenvector can always be written
in the form z i
m 11
m 21
m 12
m 22
and then the matrix we are looking for is
m 11 m 12
M
. Pay attention to the order of the columns. The reason that we
m 21 m 22
put the imaginary part in the first column is that the definition of any eigenvector is
A z 1 1 z 1
where all these numbers can be complex. Since these are
z 2 1 z 2
complex numbers. this equation is equivalent to:

7
A im 11
im 12
m 12
m 22
im 11
im 12
m 12
m 22
T a,b ,
where the first column on each side keeps track of the imaginary part and the second
column keeps track of the real part. Thus T a,b M −1 AM. The matrix M will have an
inverse because the imaginary roots of the characteristic equation come in pairs. So
this change of coordinates allows us to write the original equation ẋ Ax as
ẏ M −1 ẋ M −1 Ax M −1 AMy T a,b y. From our knowledge about complex
variables, we know that:
y 1 t
y 2 t
expat
ucostb − vsintb
usintb vcostb
,
for given initial conditions
y 1 0
y 2 0
u
v
.
Again, we can always compute y0 M −1 x0 because x0 is what we were really
given. Then
xt M
expaty 10costb − y 2 0sintb
expaty 1 0sintb y 2 0costb
is the answer to the initial value problem.
0 −2
1
Here is another example. Let A
with x0 . Then 1 1 i
1 2
1
and 2 1 − i are the eigenvalues. So we already know that our canonical twisting
1 −1
matrix will be T a,b
. Now we need to compute an eigenvector
1 1
corresponding to 1 1 i. (Always pick the one with the positive imaginary part.)
Using the definition of an eigenvector, we have
A − 1 Iz
−1 − i −2
1 1 − i
z 1
z 2
0
0
.
(It may not be obvious, but the top and bottom rows give the same information; you
can check this by computing 2/−1 − i. The bottom row tells us that an easy
eigenvector is:
z 1
z 2
1 − i
−1
i
−1
0
1
−1
.

8
Then the matrix giving the change of coordinates is M
−1 1
0 −1
. The inverse of
this matrix is M −1
−1 −1
0 −1
. This means that we wan rewrite ẋ Ax as:
ẏ
−1 −1
0 −1
0 −2
1 2
−1 1
0 −1
y
1 −1
1 1
y.
Also, the initial condition in the changed coordinates is:
y0
−1 −1
0 −1
1
1
−2
−1
.
The solution in the changed basis is:
yt expt
−2cost sint
−2sint − cost
.
Finally, the solution to our original problem is
xt Myt expt
−1 1
0 −1
−2cost sint
−2sint − cost
expt
cost − 3sint
cost 2sint
.
Again, check that this is the right answer.
We may also write a somewhat complicated Matlab routine which follows these same
steps. Try the procedure outlined as follows:
A[ ]
x0[ ]
[M lambda]eig(A)
[C I]max(imag(lambda))
bM(:,I(1))
b2real(b)
b1imag(b)
M[b1’;b2’]’
y0inv(M)*x0
areal(lambda(1,1))
bmax(max(imag(lambda)))
y1sym(’y1’)
y2sym(’y2’)

9
tsym(’t’)
uy0(1)
vy0(2)
y1exp(a*t)*(u*cos(b*t)-v*sin(b*t))
y2exp(a*t)*(v*cos(b*t)u*sin(b*t))
y[y1;y2]
xM*y
xsimplify(x)
The fourth and final case occurs when A has two identical (real) roots that are not
zero. Write 1 2 ≠ 0. There are two possibilities. First, if A I, then the
differential equation is already uncoupled, and we can solve it immediately. If A ≠ I,
then you can write N A − I where N 2 0 0
. Then the general solution to
0 0
ẋ Ax canbewrittenasxt exptI tN, and the definite solution is
x 1 0
xt exptI tN
. (Explaining why these facts are true is beyond the
x 2 0
scope of this course. They are based on a Taylor series expansion for operators on
matrices.)
1 −1
1
Our final example will be A
with x0 . The characteristic
1 3
1
equation has 1 − 3 − 1 0, and thus 2 − 4 4 − 2 2 0. Sothereare
two identical roots. Now we write N A − 2I
N 2 0.) We may conclude that:
−1 −1
1 1
. (Make sure that
xt exp2tI tN
x 1 0
x 2 0
exp2t
1 − 2t
1 2t
.
Finally, check that this is the right answer.