01.02.2015 Views

Chapter 33 - Applied Mathematics - Illinois Institute of Technology

Chapter 33 - Applied Mathematics - Illinois Institute of Technology

Chapter 33 - Applied Mathematics - Illinois Institute of Technology

SHOW MORE
SHOW LESS

Create successful ePaper yourself

Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.

MATH 590: Meshfree Methods<br />

<strong>Chapter</strong> <strong>33</strong>: Adaptive Iteration<br />

Greg Fasshauer<br />

Department <strong>of</strong> <strong>Applied</strong> <strong>Mathematics</strong><br />

<strong>Illinois</strong> <strong>Institute</strong> <strong>of</strong> <strong>Technology</strong><br />

Fall 2010<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 1


Outline<br />

1 A Greedy Adaptive Algorithm<br />

2 The Faul-Powell Algorithm<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 2


The two adaptive algorithms discussed in this chapter both yield an<br />

approximate solution to the RBF interpolation problem.<br />

The algorithms have some similarity with some <strong>of</strong> the omitted material<br />

from <strong>Chapter</strong>s 21, 31 and 32.<br />

The contents <strong>of</strong> this chapter are based mostly on the papers<br />

[Faul and Powell (1999), Faul and Powell (2000),<br />

Schaback and Wendland (2000a), Schaback and Wendland (2000b)]<br />

and the book [Wendland (2005a)].<br />

As always, we concentrate on systems for strictly positive definite<br />

kernels (variations for strictly conditionally positive definite kernels also<br />

exist).<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 3


A Greedy Adaptive Algorithm<br />

One <strong>of</strong> the central ingredients is the use <strong>of</strong> the native space inner<br />

product discussed in <strong>Chapter</strong> 13.<br />

As always, we assume that our data sites are X = {x 1 , . . . , x N }.<br />

We also consider a second set Y ⊆ X .<br />

Let P Y f<br />

be the interpolant to f on Y ⊆ X .<br />

Then the first orthogonality lemma from <strong>Chapter</strong> 18 (with g = f ) yields<br />

〈f − P Y f<br />

, P Y f<br />

〉 NK (Ω) = 0.<br />

This leads to the energy split (see <strong>Chapter</strong> 18)<br />

‖f ‖ 2 N K (Ω) = ‖f − PY f<br />

‖ 2 N K (Ω) + ‖PY f<br />

‖ 2 N K (Ω) .<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 5


A Greedy Adaptive Algorithm<br />

We now consider an iteration on residuals.<br />

We pretend to start with our desired interpolant r 0 = Pf<br />

X<br />

set X .<br />

on the entire<br />

We also pick an appropriate sequence <strong>of</strong> sets Y k ⊆ X , k = 0, 1, . . . (we<br />

will discuss some possible heuristics for choosing these sets later).<br />

Then we iteratively define the residual functions<br />

r k+1 = r k − P Y k<br />

r k<br />

, k = 0, 1, . . . . (1)<br />

Remark<br />

In the actual algorithm below we will only deal with discrete vectors.<br />

Thus the vector r 0 will be given by the data values (since P f is<br />

supposed to interpolate f on X ).<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 6


A Greedy Adaptive Algorithm<br />

Now, the energy splitting identity with f = r k gives us<br />

‖r k ‖ 2 N K (Ω) = ‖r k − P Y k<br />

r k<br />

‖ 2 N K (Ω) + ‖PY k<br />

r k<br />

‖ 2 N K (Ω)<br />

(2)<br />

or, using the iteration formula (1),<br />

‖r k ‖ 2 N K (Ω) = ‖r k+1‖ 2 N K (Ω) + ‖r k − r k+1 ‖ 2 N K (Ω) . (3)<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 7


A Greedy Adaptive Algorithm<br />

We have the following telescoping sum for the partial sums <strong>of</strong> the norm<br />

<strong>of</strong> the residual updates P Y k<br />

r k<br />

:<br />

M∑<br />

k=0<br />

‖P Y k<br />

r k<br />

‖ 2 N K (Ω)<br />

(1)<br />

=<br />

(3)<br />

=<br />

M∑<br />

‖r k − r k+1 ‖ 2 N K (Ω)<br />

k=0<br />

M∑<br />

k=0<br />

{<br />

}<br />

‖r k ‖ 2 N K (Ω) − ‖r k+1‖ 2 N K (Ω)<br />

= ‖r 0 ‖ 2 N K (Ω) − ‖r M+1‖ 2 N K (Ω) ≤ ‖r 0‖ 2 N K (Ω) .<br />

Remark<br />

This estimate shows that the sequence <strong>of</strong> partial sums is monotone<br />

increasing and bounded, and therefore convergent — even for a poor<br />

choice <strong>of</strong> the sets Y k .<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 8


A Greedy Adaptive Algorithm<br />

If we can show that the residuals r k converge to zero, then we would<br />

have that the iteratively computed approximation<br />

u M+1 =<br />

M∑<br />

k=0<br />

P Y k<br />

r k<br />

=<br />

M∑<br />

(r k − r k+1 ) = r 0 − r M+1 (4)<br />

k=0<br />

converges to the original interpolant r 0 = P X f<br />

.<br />

Remark<br />

The omitted chapters contain iterative methods by which we<br />

approximate the interpolant by iterating an approximation method on<br />

the full data set.<br />

Here we are approximating the interpolant by iterating an interpolation<br />

method on nested (increasing) adaptively chosen subsets <strong>of</strong> the data.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 9


A Greedy Adaptive Algorithm<br />

Remark<br />

The present method also has some similarities with the (omitted)<br />

multilevel algorithms <strong>of</strong> <strong>Chapter</strong> 32.<br />

However,<br />

here: we compute the interpolant Pf<br />

X<br />

single kernel K<br />

on the set X based on a<br />

<strong>Chapter</strong> 32: the final interpolant is given as the result <strong>of</strong> using the<br />

spaces ⋃ M<br />

k=1 N K k<br />

(Ω), where K k is an appropriately scaled<br />

version <strong>of</strong> the kernel K .<br />

Moreover, the goal in <strong>Chapter</strong> 32 is to approximate f , not P f .<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 10


A Greedy Adaptive Algorithm<br />

To prove convergence <strong>of</strong> the residual iteration, we assume that we can<br />

find sets <strong>of</strong> points Y k such that at step k at least some fixed percentage<br />

<strong>of</strong> the energy <strong>of</strong> the residual is picked up by its interpolant, i.e.,<br />

with some fixed γ ∈ (0, 1].<br />

Then (3) and the iteration formula (1) imply<br />

‖P Y k<br />

r k<br />

‖ 2 N K (Ω) ≥ γ‖r k‖ 2 N K (Ω)<br />

(5)<br />

‖r k+1 ‖ 2 N K (Ω) = ‖r k‖ 2 N K (Ω) − ‖PY k<br />

r k<br />

‖ 2 N K (Ω) ,<br />

and therefore<br />

‖r k+1 ‖ 2 N K (Ω) ≤ ‖r k‖ 2 N K (Ω) − γ‖r k‖ 2 N K (Ω) = (1 − γ)‖r k‖ 2 N K (Ω) .<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 11


Applying the bound<br />

A Greedy Adaptive Algorithm<br />

recursively yields<br />

‖r k+1 ‖ 2 N K (Ω) ≤ (1 − γ)‖r k‖ 2 N K (Ω)<br />

Theorem<br />

If the choice <strong>of</strong> sets Y k satisfies ‖P Y k<br />

r k<br />

‖ 2 N K (Ω) ≥ γ‖r k‖ 2 N K (Ω), then the<br />

residual iteration (see (4))<br />

u M =<br />

M−1<br />

∑<br />

k=0<br />

P Y k<br />

r k<br />

= r 0 − r M , r k+1 = r k − P Y k<br />

r k<br />

, k = 0, 1, . . .<br />

converges linearly in the native space norm. After M steps <strong>of</strong> iterative<br />

refinement there is an error bound<br />

‖P X f − u M ‖ 2 N K (Ω) = ‖r 0 − u M ‖ 2 N K (Ω) = ‖r M‖ 2 N K (Ω) ≤ (1 − γ)M ‖r 0 ‖ 2 N K (Ω) .<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 12


A Greedy Adaptive Algorithm<br />

Remark<br />

This theorem has various limitations:<br />

The norm involves the kernel K which makes it difficult to find sets<br />

Y k that satisfy (5).<br />

The native space norm <strong>of</strong> the initial residual r 0 is not known.<br />

A way around these problems is to use an equivalent discrete norm on<br />

the set X .<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 13


A Greedy Adaptive Algorithm<br />

Schaback and Wendland establish an estimate <strong>of</strong> the form<br />

‖r 0 − u M ‖ 2 X ≤ C c<br />

(1 − δ c2<br />

C 2 ) M/2<br />

‖r 0 ‖ 2 X ,<br />

where c and C are constants denoting the norm equivalence, i.e.,<br />

c‖u‖ X ≤ ‖u‖ NK (Ω) ≤ C‖u‖ X<br />

for any u ∈ N K (Ω), and where δ is a constant analogous to γ (but<br />

based on use <strong>of</strong> the discrete norm ‖ · ‖ X in (5)).<br />

In fact, any discrete l p norm on X can be used.<br />

In the implementation below we will use the maximum norm.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 14


A Greedy Adaptive Algorithm<br />

In [Schaback and Wendland (2000b)] a basic version <strong>of</strong> this<br />

algorithm — where the sets Y k consist <strong>of</strong> a single point — is<br />

described and tested.<br />

The resulting approximation yields the best M-term approximation<br />

to the interpolant.<br />

Remark<br />

This idea is related to the concepts <strong>of</strong><br />

greedy approximation algorithms (see, e.g., [Temlyakov (1998)])<br />

and<br />

sparse approximation (see, e.g., [Girosi (1998)]).<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 15


A Greedy Adaptive Algorithm<br />

If the set Y k consists <strong>of</strong> only a single point y k , then the partial<br />

interpolant P Y k<br />

r k<br />

is particularly simple:<br />

with<br />

P Y k<br />

r k<br />

= βK (·, y k )<br />

β = r k(y k )<br />

K (y k , y k )<br />

This follows immediately from the usual RBF expansion (which<br />

consists <strong>of</strong> only one term here) and the interpolation condition<br />

P Y k<br />

r k<br />

(y k ) = r k (y k ).<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 16


A Greedy Adaptive Algorithm<br />

The point y k is picked to be the point in X where the residual is<br />

largest, i.e.,<br />

|r k (y k )| = ‖r k ‖ ∞ .<br />

This choice <strong>of</strong> “set” Y k certainly satisfies the constraint (5):<br />

‖P Y k<br />

r k<br />

‖ 2 N K (Ω)<br />

= ‖βK (·, y k )‖ 2 N K (Ω)<br />

=<br />

r k (y k )<br />

∥K (y k , y k ) K (·, y k)<br />

∥<br />

2<br />

N K (Ω)<br />

≤ γ‖r k ‖ 2 N K (Ω) , 0 < γ ≤ 1.<br />

Here we require K (·, y k ) ≤ K (y k , y k ) which is certainly true for<br />

positive definite translation invariant kernels (cf. <strong>Chapter</strong> 3).<br />

However, in general we only know that<br />

|K (x, y)| 2 ≤ K (x, x)K (y, y) (see<br />

[Berlinet and Thomas-Agnan (2004)]).<br />

The interpolation problem is (approximately) solved without having<br />

to invert any linear systems.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 17


A Greedy Adaptive Algorithm<br />

Algorithm (Greedy one-point)<br />

Input data locations X , associated values f <strong>of</strong> f , tolerance tol > 0<br />

Set initial residual r 0 = P X f<br />

| X = f , initialize u 0 = 0, e = ∞, k = 0<br />

Choose starting point y k ∈ X<br />

While e > tol do<br />

Set β = r k(y k )<br />

K (y k , y k )<br />

For 1 ≤ i ≤ N do<br />

r k+1 (x i ) = r k (x i ) − βK (x i , y k )<br />

u k+1 (x i ) = u k (x i ) + βK (x i , y k )<br />

end<br />

Find e = max |r k+1| and the point y k+1 where it occurs<br />

X<br />

Increment k = k + 1<br />

end<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 18


A Greedy Adaptive Algorithm<br />

Remark<br />

It is important to realize that in our MATLAB implementation we<br />

never actually compute the initial residual r 0 = P X f<br />

.<br />

All we require are the values <strong>of</strong> r 0 on the grid X <strong>of</strong> data sites.<br />

However, since Pf<br />

X | X = f | X the values r 0 (x i ) are given by the<br />

interpolation data f (x i ) (see line 5 <strong>of</strong> the code).<br />

Moreover, since the sets Y k are subsets <strong>of</strong> X the value r k (y k )<br />

required to determine β is actually one <strong>of</strong> the current residual<br />

values (see line 10 <strong>of</strong> the code).<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 19


A Greedy Adaptive Algorithm<br />

Remark<br />

We use DistanceMatrix together with rbf to compute both<br />

K (y k , y k ) (on lines 9 and 10) and<br />

K (x i , y k ) needed for the updates <strong>of</strong> the residual r k+1 and the<br />

approximation u k+1 on lines 11–14.<br />

Note that the “matrices” DM_data, IM, DM_res, RM, DM_eval,<br />

EM are only column vectors since only one center, y k , is involved.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 20


A Greedy Adaptive Algorithm<br />

Remark<br />

The algorithm demands that we compute the residuals r k on the<br />

data sites.<br />

The partial approximants u k to the interpolant can be evaluated<br />

anywhere.<br />

If we do this also at the data sites, then we are required to use a<br />

plotting routine that differs from our usual one (such as trisurf<br />

built on a triangulation <strong>of</strong> the data sites obtained with the help <strong>of</strong><br />

delaunayn).<br />

We instead follow the same procedure as in all <strong>of</strong> our other<br />

programs, i.e., to evaluate u k on a 40 × 40 grid <strong>of</strong> equally spaced<br />

points. This has been implemented on lines 11–15 <strong>of</strong> the program.<br />

Note that the updating procedure has been vectorized in MATLAB<br />

allowing us to avoid the for-loop over i in the algorithm.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 21


A Greedy Adaptive Algorithm<br />

Program (RBFGreedyOnePoint2D.m)<br />

1 rbf = @(e,r) exp(-(e*r).^2); ep = 5.5;<br />

2 N = 16641; dsites = CreatePoints(N,2,’h’);<br />

3 neval = 40; epoints = CreatePoints(neval^2,2,’u’);<br />

4 tol = 1e-5; kmax = 1000;<br />

5 res = testfunctionsD(dsites); u = 0;<br />

6 k = 1; maxres(k) = 999999;<br />

7 ykidx = (N+1)/2; yk(k,:) = dsites(ykidx,:);<br />

8 while (maxres(k) > tol && k < kmax)<br />

9 DM_data = DistanceMatrix(yk(k,:),yk(k,:));<br />

10 IM = rbf(ep,DM_data); beta = res(ykidx)/IM;<br />

11 DM_res = DistanceMatrix(dsites,yk(k,:));<br />

12 RM = rbf(ep,DM_res);<br />

13 DM_eval = DistanceMatrix(epoints,yk(k,:));<br />

14 EM = rbf(ep,DM_eval);<br />

15 res = res - beta*RM; u = u + beta*EM;<br />

16 [maxres(k+1), ykidx] = max(abs(res));<br />

17 yk(k+1,:) = dsites(ykidx,:); k = k + 1;<br />

18 end<br />

19 exact = testfunctionsD(epoints);<br />

20 rms_err = norm(u-exact)/neval<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 22


A Greedy Adaptive Algorithm<br />

To illustrate the greedy one-point algorithm we perform two<br />

experiments.<br />

Both tests use data obtained by sampling Franke’s function at 16641<br />

Halton points in [0, 1] 2 .<br />

Test 1 is based on Gaussians,<br />

Test 2 uses inverse multiquadrics.<br />

For both tests we use the same shape parameter ε = 5.5.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 23


A Greedy Adaptive Algorithm<br />

Figure: 1000 selected points and residual for greedy one point algorithm with<br />

Gaussian RBFs and N = 16641 data points.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 24


A Greedy Adaptive Algorithm<br />

Figure: Fits <strong>of</strong> Franke’s function for greedy one point algorithm with Gaussian<br />

RBFs and N = 16641 data points. Top left to bottom right: 1 point, 2 points, 4<br />

points, final fit with 1000 points.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 25


A Greedy Adaptive Algorithm<br />

Remark<br />

In order to obtain our approximate interpolants we used<br />

a tolerance <strong>of</strong> 10 −5<br />

along with an additional upper limit <strong>of</strong> kmax=1000 on the number <strong>of</strong><br />

iterations.<br />

For both tests the algorithm uses up all 1000 iterations.<br />

The final maximum residual is<br />

maxres = 0.0075 for Gaussians, and<br />

maxres = 0.0035 for inverse MQs.<br />

In both cases there occurred several multiple point selections.<br />

Contrary to interpolation problems based on the solution <strong>of</strong> a<br />

linear system, multiple point selections do not pose a problem<br />

here.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 26


A Greedy Adaptive Algorithm<br />

Figure: 1000 selected points and residual for greedy one point algorithm with<br />

IMQ RBFs and N = 16641 data points.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 27


A Greedy Adaptive Algorithm<br />

Figure: Fits <strong>of</strong> Franke’s function for greedy one point algorithm with IMQ<br />

RBFs and N = 16641 data points. Top left to bottom right: 1 point, 2 points, 4<br />

points, final fit with 1000 points.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 28


A Greedy Adaptive Algorithm<br />

Remark<br />

We note that the inverse multiquadrics have a more global<br />

influence than the Gaussians (for the same shape parameter).<br />

This effect is clearly evident in the first few approximations to the<br />

interpolants in the figures.<br />

From the last figure we see that the greedy algorithm enforces<br />

interpolation <strong>of</strong> the data only on the most recent set Y k (i.e., for<br />

the one-point algorithm studied here only at a single point).<br />

If one wants to maintain the interpolation achieved in previous<br />

iterations, then the sets Y k should be nested.<br />

This, however, would have a significant effect on the execution<br />

time <strong>of</strong> the algorithm since the matrices at each step would<br />

increase in size.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 29


A Greedy Adaptive Algorithm<br />

Remark<br />

One advantage <strong>of</strong> this very simple algorithm is that no linear<br />

systems need to be solved.<br />

This allows us to approximate the interpolants for large data sets<br />

even for globally supported kernels,<br />

and also with small values <strong>of</strong> ε (and therefore an associated<br />

ill-conditioned interpolation matrix).<br />

One should not expect too much in this case, however, as the<br />

results in the following figure show where we used a value <strong>of</strong><br />

ε = 0.1 for the shape parameter.<br />

A lot <strong>of</strong> smoothing occurs so that the convergence to the RBF<br />

interpolant is very slow.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 30


A Greedy Adaptive Algorithm<br />

Figure: 1000 selected points (only 20 <strong>of</strong> them distinct) and fit <strong>of</strong> Franke’s<br />

function for greedy one point algorithm with flat Gaussian RBFs (ε = 0.1) and<br />

N = 16641 data points.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 31


A Greedy Adaptive Algorithm<br />

Remark<br />

In the pseudo-code <strong>of</strong> the algorithm matrix-vector multiplications<br />

are not required.<br />

However, MATLAB allows for a vectorization <strong>of</strong> the for-loop which<br />

does result in two matrix-vector multiplications.<br />

For practical situations, e.g., for smooth kernels and densely<br />

distributed points in X the convergence can be rather slow.<br />

The simple greedy algorithm described above is extended in<br />

[Schaback and Wendland (2000b)] to a version that adaptively<br />

uses kernels <strong>of</strong> varying scales.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 32


The Faul-Powell Algorithm<br />

Another iterative algorithm was suggested in<br />

[Faul and Powell (1999), Faul and Powell (2000)].<br />

From our earlier discussions we know that it is possible to express a<br />

kernel interpolant in terms <strong>of</strong> cardinal functions uj ∗ , j = 1, . . . , N, i.e.,<br />

P f (x) =<br />

N∑<br />

f (x j )uj ∗ (x).<br />

j=1<br />

The basic idea <strong>of</strong> the Faul-Powell algorithm is to use approximate<br />

cardinal functions Ψ j instead.<br />

Of course, this will only give an approximate value for the interpolant,<br />

and therefore an iteration on the residuals is suggested to improve the<br />

accuracy <strong>of</strong> this approximation.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 34


The Faul-Powell Algorithm<br />

The approximate cardinal functions Ψ j , j = 1, . . . , N, are determined as<br />

linear combinations <strong>of</strong> the basis functions K (·, x l ) for the interpolant,<br />

i.e.,<br />

Ψ j = ∑ l∈L j<br />

b jl K (·, x l ), (6)<br />

where L j is an index set consisting <strong>of</strong> n (n ≈ 50) indices that are used<br />

to determine the approximate cardinal function.<br />

Example<br />

The n nearest neighbors <strong>of</strong> x j will usually do.<br />

Remark<br />

The basic philosophy <strong>of</strong> this algorithm is very similar to that <strong>of</strong> the<br />

omitted fixed level iteration <strong>of</strong> <strong>Chapter</strong> 31 where approximate MLS<br />

generating functions were used as approximate cardinal functions.<br />

The Faul-Powell algorithm can be interpreted as a Krylov<br />

subspace method.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 35


The Faul-Powell Algorithm<br />

Remark<br />

In general, the choice <strong>of</strong> index sets allows much freedom, and this<br />

is the reason why we include the algorithm in this chapter on<br />

adaptive iterative methods.<br />

As pointed out at the end <strong>of</strong> this section, there is a certain duality<br />

between the Faul-Powell algorithm and the greedy algorithm <strong>of</strong> the<br />

previous section.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 36


The Faul-Powell Algorithm<br />

For every j = 1, . . . , N, the coefficients b jl are found as solution <strong>of</strong> the<br />

(relatively small) n × n linear system<br />

Ψ j (x i ) = δ jk , i ∈ L j . (7)<br />

These approximate cardinal functions are computed in a<br />

pre-processing step.<br />

In its simplest form the residual iteration can be formulated as<br />

u (0) (x) =<br />

N∑<br />

f (x j )Ψ j (x)<br />

j=1<br />

u (k+1) (x) = u (k) (x) +<br />

N∑<br />

j=1<br />

[<br />

]<br />

f (x j ) − u (k) (x j ) Ψ j (x), k = 0, 1, . . . .<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 37


The Faul-Powell Algorithm<br />

Instead <strong>of</strong> adding the contribution <strong>of</strong> all approximate cardinal functions<br />

at the same time, this is done in a three-step process in the<br />

Faul-Powell algorithm.<br />

To this end, we choose index sets L j , j = 1, . . . , N − n, such that<br />

while making sure that j ∈ L j .<br />

L j ⊆ {j, j + 1, . . . , N}<br />

Remark<br />

If one wants to use this algorithm to approximate the interpolant based<br />

on conditionally positive definite kernels <strong>of</strong> order m, then one needs to<br />

ensure that the corresponding centers form an (m − 1)-unisolvent set<br />

and append a polynomial to the local expansion (6).<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 38


Step 1<br />

The Faul-Powell Algorithm<br />

We define u (k)<br />

0<br />

= u (k) , and then iterate<br />

u (k)<br />

j<br />

= u (k)<br />

j−1 + θ(k) j<br />

Ψ j , j = 1, . . . , N − n, (8)<br />

with<br />

θ (k)<br />

j<br />

= 〈P f − u (k)<br />

j−1 , Ψ j〉 NK (Ω)<br />

. (9)<br />

〈Ψ j , Ψ j 〉 NK (Ω)<br />

Remark<br />

The stepsize θ (k)<br />

j<br />

is chosen so that the native space best<br />

approximation to the residual P f − u (k)<br />

j−1<br />

from the space spanned by the<br />

approximate cardinal functions Ψ j is added.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 39


Step 1 (cont.)<br />

The Faul-Powell Algorithm<br />

Using the representation<br />

Ψ j = ∑ l∈L j<br />

b jl K (·, x l ),<br />

the reproducing kernel property <strong>of</strong> K , and the (local) cardinality<br />

property Ψ j (x i ) = δ jk , i ∈ L j we can calculate the denominator <strong>of</strong> (9) as<br />

〈Ψ j , Ψ j 〉 NK (Ω) = 〈Ψ j , ∑ l∈L j<br />

b jl K (·, x l )〉 NK (Ω)<br />

= ∑ l∈L j<br />

b jl 〈Ψ j , K (·, x l )〉 NK (Ω)<br />

= ∑ l∈L j<br />

b jl Ψ j (x l ) = b jj<br />

since we have j ∈ L j by construction <strong>of</strong> the index set L j .<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 40


Step 1 (cont.)<br />

The Faul-Powell Algorithm<br />

Similarly, we get for the numerator<br />

〈P f − u (k)<br />

j−1 , Ψ j〉 NK (Ω) = 〈P f − u (k)<br />

j−1 , ∑ l∈L j<br />

b jl K (·, x l )〉 NK (Ω)<br />

= ∑ l∈L j<br />

b jl 〈P f − u (k)<br />

j−1 , K (·, x l)〉 NK (Ω)<br />

= ∑ ( )<br />

b jl P f − u (k) (x l )<br />

l∈L j<br />

j−1<br />

= ∑ (<br />

)<br />

b jl f (x l ) − u (k)<br />

j−1 (x l) .<br />

l∈L j<br />

Therefore (8) and (9) can be written as<br />

u (k)<br />

j<br />

= u (k)<br />

j−1 + Ψ ∑ (<br />

)<br />

j<br />

b jl f (x l ) − u (k)<br />

b<br />

j−1 (x l) , j = 1, . . . , N − n.<br />

jj<br />

l∈L j<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 41


Step 2<br />

The Faul-Powell Algorithm<br />

Next we interpolate the residual on the remaining n points (collected<br />

via the index set L ∗ ).<br />

Thus, we find a function v (k) in span{K (·, x j ) : j ∈ L ∗ } such that<br />

v (k) (x i ) = f (x i ) − u (k)<br />

N−n (x i), i ∈ L ∗ ,<br />

and the approximation is updated, i.e.,<br />

u (k+1) = u (k)<br />

N−n + v (k) .<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 42


The Faul-Powell Algorithm<br />

Step 3<br />

Finally, the residuals are updated, i.e.,<br />

r (k+1)<br />

i<br />

= f (x i ) − u (k+1) (x i ), i = 1, . . . , N. (10)<br />

Remark<br />

The outer iteration (on k) is now repeated unless the largest <strong>of</strong> these<br />

residuals is small enough.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 43


The Faul-Powell Algorithm<br />

Algorithm (Pre-processing step)<br />

Choose n<br />

For 1 ≤ j ≤ N − n do<br />

Determine the index set L j<br />

Find the coefficients b jl <strong>of</strong> the approximate cardinal function Ψ j by<br />

solving<br />

Ψ j (x i ) = δ jk , i ∈ L j<br />

end<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 44


The Faul-Powell Algorithm<br />

Algorithm (Faul-Powell)<br />

Input data locations X , associated values <strong>of</strong> f , tolerance tol > 0<br />

Perform pre-processing step<br />

Initialize: k = 0, u (k)<br />

0<br />

= 0, r (k)<br />

i<br />

= f (x i ), i = 1, . . . , N, e = max<br />

i=1,...,N<br />

While e > tol do<br />

Update<br />

u (k)<br />

j<br />

|r<br />

(k)<br />

i<br />

|<br />

= u (k)<br />

j−1 + Ψ ∑ (<br />

)<br />

j<br />

b jl f (x l ) − u (k)<br />

j−1<br />

b (x l) , 1 ≤ j ≤ N − n<br />

jj<br />

l∈L j<br />

Solve the interpolation problem<br />

v (k) (x i ) = f (x i ) − u (k)<br />

N−n (x i),<br />

Update the approximation<br />

Compute new residuals r (k+1)<br />

i<br />

Set new value for e =<br />

Increment k = k + 1<br />

u (k+1)<br />

0<br />

= u (k)<br />

N−n + v (k)<br />

max<br />

i=1,...,N<br />

i ∈ L ∗<br />

= f (x i ) − u (k+1)<br />

0<br />

(x i ), i = 1, . . . , N<br />

(k+1)<br />

|r<br />

i<br />

|<br />

end<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 45


The Faul-Powell Algorithm<br />

Remark<br />

Faul and Powell prove that this algorithm converges to the solution<br />

<strong>of</strong> the original interpolation problem.<br />

One needs to make sure that the residuals are evaluated<br />

efficiently by using, e.g.,<br />

a fast multipole expansion,<br />

fast Fourier transform, or<br />

compactly supported kernels.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 46


The Faul-Powell Algorithm<br />

Remark<br />

In its most basic form the Krylov subspace algorithm <strong>of</strong> Faul and<br />

Powell can also be explained as a dual approach to the greedy<br />

residual iteration algorithm <strong>of</strong> Schaback and Wendland.<br />

Instead <strong>of</strong> defining appropriate sets <strong>of</strong> points Y k , in the Faul and<br />

Powell algorithm one picks certain subspaces U k <strong>of</strong> the native<br />

space.<br />

In particular, if U k is the one-dimensional space U k = span{Ψ k }<br />

(where Ψ k is a local approximation to the cardinal function) we get<br />

the Schaback-Wendland algorithm described above.<br />

For more details see [Schaback and Wendland (2000b)].<br />

Implementation <strong>of</strong> this algorithm is omitted.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 47


References I<br />

Appendix<br />

References<br />

Berlinet, A., Thomas-Agnan, C. (2004).<br />

Reproducing Kernel Hilbert Spaces in Probability and Statistics.<br />

Kluwer, Dordrecht.<br />

Buhmann, M. D. (2003).<br />

Radial Basis Functions: Theory and Implementations.<br />

Cambridge University Press.<br />

Fasshauer, G. E. (2007).<br />

Meshfree Approximation Methods with MATLAB.<br />

World Scientific Publishers.<br />

Higham, D. J. and Higham, N. J. (2005).<br />

MATLAB Guide.<br />

SIAM (2nd ed.), Philadelphia.<br />

Iske, A. (2004).<br />

Multiresolution Methods in Scattered Data Modelling.<br />

Lecture Notes in Computational Science and Engineering 37, Springer Verlag<br />

(Berlin).<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 48


References II<br />

Appendix<br />

References<br />

G. Wahba (1990).<br />

Spline Models for Observational Data.<br />

CBMS-NSF Regional Conference Series in <strong>Applied</strong> <strong>Mathematics</strong> 59, SIAM<br />

(Philadelphia).<br />

Wendland, H. (2005a).<br />

Scattered Data Approximation.<br />

Cambridge University Press (Cambridge).<br />

Faul, A. C. and Powell, M. J. D. (1999).<br />

Pro<strong>of</strong> <strong>of</strong> convergence <strong>of</strong> an iterative technique for thin plate spline interpolation in<br />

two dimensions.<br />

Adv. Comput. Math. 11, pp. 183–192.<br />

Faul, A. C. and Powell, M. J. D. (2000).<br />

Krylov subspace methods for radial basis function interpolation.<br />

in Numerical Analysis 1999 (Dundee), Chapman & Hall/CRC (Boca Raton, FL),<br />

pp. 115–141.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 49


References III<br />

Appendix<br />

References<br />

Girosi, F. (1998).<br />

An equivalence between sparse approximation and support vector machines.<br />

Neural Computation 10, pp. 1455–1480.<br />

Schaback, R. and Wendland, H. (2000a).<br />

Numerical techniques based on radial basis functions.<br />

in Curve and Surface Fitting: Saint-Malo 1999, A. Cohen, C. Rabut, and<br />

L. L. Schumaker (eds.), Vanderbilt University Press (Nashville, TN), 359-374.<br />

Schaback, R. and Wendland, H. (2000b).<br />

Adaptive greedy techniques for approximate solution <strong>of</strong> large RBF systems.<br />

Numer. Algorithms 24, pp. 239–254.<br />

Temlyakov, V. N. (1998).<br />

The best m-term approximation and greedy algorithms.<br />

Adv. in Comp. Math. 8, pp. 249–265.<br />

fasshauer@iit.edu MATH 590 – <strong>Chapter</strong> <strong>33</strong> 50

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!