link to my thesis

82 CHAPTER 6. ITERATIVE EIGENVALUE SOLVERS
much sparser commuting matrices A x1 ,...,A xn in the inner loop, causing a speed
up in computation time and a decrease in required floating point operations. In this
section also the pseudocode of the JDCOMM method is given. Numerical experiments
using the approaches described in this chapter are given in Chapter 7 where the
performance of the various approaches is compared to that of conventional iterative
methods and of other methods for multivariate polynomial optimization.
6.1 Iterative eigenvalue solvers
In the previous chapters it was discussed how the global minimum of a Minkowski
dominated polynomial p λ can be found by solving an eigenvalue problem. Unless
the dimensions are small, this eigenvalue problem is preferably handled with iterative
eigenvalue solvers because these allow for computations in a matrix-free fashion using
an nD-system as described in Chapter 5. The matrix-free approach implements the
action Ax of the matrix A on a given vector x without the explicit construction of
the matrix A. This action is carried out repeatedly to approximate the eigenvectors
and eigenvalues of the matrix A.
A well known iterative procedure for finding the largest eigenvalue / eigenvector
pair is the power method [72]. This method operates as follows: let v 1 ,v 2 ,...,v N be
the N eigenvectors of an N × N matrix A, assuming that N independent eigenvectors
v i exist. Let λ 1 ,λ 2 ,...,λ N be the corresponding eigenvalues such that |λ 1 | >
|λ 2 |≥... ≥|λ N |. Then each x ∈ R N can be written as a linear combination of the
eigenvectors v i : x = α 1 v 1 + α 2 v 2 + ...+ α N v N . The action of the matrix A on the
vector x yields: Ax = α 1 λ 1 v 1 +α 1 λ 2 v 1 +...+α N λ N v N . More generally it holds that:
A k x = α 1 λ k 1v 1 + α 2 λ k 2v 2 + ...+ α N λ k N v N ≈ α 1 λ k 1v 1 when k is large enough, because
the eigenvalue λ 1 is dominant. Therefore, this method (extended with a renormalization
procedure) allows for the iterative approximate calculation of the eigenvalue
with the largest magnitude and its accompanying eigenvector.
The inverse power method is an iterative procedure which allows to compute the
eigenvalue with the smallest magnitude [72]. This method operates with the matrix
A −1 rather than A. Note that the matrices A and A −1 have the same eigenvectors
since A −1 x =(1/λ)x follows from Ax = λx. Since the eigenvalues of the matrix
A −1 are 1/λ 1 ,...,1/λ N , the inverse power method converges to the eigenvector v n
associated with the eigenvalue 1/λ N : iteratively computing the action of A −1 on a
vector x = α 1 v 1 + α 2 v 2 + ...+ α N w N , where v i are the eigenvectors of the matrix,
yields: (A −1 ) k x = α 1 (1/λ 1 ) k v 1 +α 2 (1/λ 2 ) k v 2 +...+α N (1/λ N ) k v N ≈ α N (1/λ N ) k v N
when k is large enough.
Other, more advanced, iterative eigenvalue solvers are Arnoldi, Lanczos and Davidson
methods [30] . Davidson’s method is efficient for computing a few of the smallest
eigenvalues of a real symmetric matrix. The methods by Lanczos [73] and Arnoldi
[4],[77] in their original form, tend to exhibit slow convergence and experience difficulties
in finding the so-called interior eigenvalues. Interior eigenvalues are eigenvalues