28.11.2014 Views

Interpolation and Curve Fitting

Interpolation and Curve Fitting

Interpolation and Curve Fitting

SHOW MORE
SHOW LESS

Create successful ePaper yourself

Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.

<strong>Interpolation</strong> <strong>and</strong><br />

<strong>Curve</strong> <strong>Fitting</strong><br />

Pierre Bézier<br />

De Casteljau construction of Bezier curve<br />

From www.wikipedia.org


Bezier <strong>Curve</strong>s <strong>and</strong> B-Spline <strong>Curve</strong>s<br />

• Not really “interpolating”curves as they<br />

normally do not pass through data points<br />

• <strong>Curve</strong>s stay within the polygon defined by the<br />

given points<br />

• Local control: effect of changing a knot is<br />

limited to a local region in Bezier <strong>and</strong> b-spline<br />

• Global control: moving a knot in a cubic spline<br />

affects the whole curve<br />

Numerical Methods © Wen-Chieh Lin 2


Global vs. Local Control<br />

• Does a small change affect the whole curve or<br />

just a small segment?<br />

• Local is usually more intuitive <strong>and</strong> provided<br />

by composite curves<br />

Global<br />

Local<br />

Numerical Methods © Wen-Chieh Lin 3


Parametric Form<br />

• Explicit function<br />

y f<br />

(x)<br />

• Parametric form<br />

x F ( u)<br />

1<br />

y F ( u)<br />

2<br />

• Example: unit circle<br />

x cos(u)<br />

y sin(u)<br />

Numerical Methods © Wen-Chieh Lin 4


<strong>Interpolation</strong> in Parametric Form<br />

• Given n+1 control points,<br />

x<br />

i <br />

pi <br />

<br />

y<br />

<br />

i<br />

, i 0 ,..., n<br />

<br />

<br />

zi<br />

<br />

• Compute interpolation functions<br />

x<br />

( u)<br />

<br />

( u)<br />

<br />

<br />

y(<br />

u)<br />

<br />

, 0 u<br />

1<br />

<br />

<br />

z(<br />

u)<br />

<br />

P j<br />

j: index for each segment of curve<br />

Numerical Methods © Wen-Chieh Lin 5


Composite Segments<br />

• Divide a curve into multiple segments<br />

• Represent each in a parametric form<br />

• Maintain continuity between segments<br />

• position<br />

• tangent<br />

• curvature<br />

P 1 (u) P 2 (u) P 3 (u) P n (u)<br />

Numerical Methods © Wen-Chieh Lin 6


Cubic 3D <strong>Curve</strong>s<br />

• Three cubic polynomials, one for each coordinate<br />

3 2<br />

x(<br />

u)<br />

a<br />

u b<br />

u c<br />

u d<br />

y(<br />

u)<br />

z(<br />

u)<br />

a<br />

a<br />

• In matrix notation<br />

x<br />

y<br />

z<br />

u<br />

u<br />

3<br />

3<br />

b<br />

x<br />

y<br />

z<br />

u<br />

b u<br />

2<br />

2<br />

c<br />

c<br />

z<br />

x<br />

y<br />

u d<br />

u d<br />

a<br />

<br />

b<br />

<br />

c<br />

<br />

d<br />

3<br />

2<br />

x(<br />

u)<br />

y(<br />

u)<br />

z(<br />

u)<br />

u<br />

u u 1<br />

Numerical Methods © Wen-Chieh Lin 7<br />

x<br />

x<br />

x<br />

x<br />

z<br />

x<br />

y<br />

a<br />

b<br />

c<br />

c<br />

y<br />

y<br />

y<br />

y<br />

a<br />

b<br />

c<br />

d<br />

z<br />

z<br />

z<br />

z


Hermite <strong>Interpolation</strong><br />

• Hermite interpolation requires<br />

• values at endpoints<br />

• derivatives at endpoints<br />

x(<br />

u)<br />

a<br />

x<br />

u<br />

3<br />

b<br />

x<br />

u<br />

c<br />

Needs 4 equations to<br />

solve 4 unknowns<br />

2<br />

x<br />

u<br />

d<br />

x<br />

• To create a composite curve, use the end of<br />

one as the beginning of the other <strong>and</strong> share the<br />

tangent vector<br />

P(0)<br />

control points/knots<br />

P(1)<br />

P '(0)<br />

P' (1)<br />

Numerical Methods © Wen-Chieh Lin 8


Hermite <strong>Curve</strong> Formation—<br />

consider x coordinate first<br />

• Cubic polynomial <strong>and</strong> its derivative<br />

3 2<br />

x(<br />

u)<br />

a<br />

u b<br />

u c<br />

u<br />

x'<br />

( u)<br />

• Given x i , x i+1 , x’ i , x’ i+1 , solve for a, b, c, d<br />

• 4 equations are given for 4 unknowns<br />

x<br />

x<br />

x<br />

2<br />

3<br />

a u 2<br />

b<br />

x<br />

x<br />

u c<br />

x<br />

d<br />

x(0)<br />

x i<br />

d x<br />

x(1)<br />

x<br />

i 1<br />

a<br />

x<br />

b<br />

x<br />

c<br />

x<br />

d<br />

x'<br />

(0) x'<br />

i<br />

c x<br />

x (1) x<br />

' <br />

3<br />

a 2<br />

b c<br />

'<br />

i 1<br />

x<br />

x<br />

Numerical Methods © Wen-Chieh Lin 9<br />

x<br />

x<br />

x


Hermite <strong>Curve</strong> Formation (cont.)<br />

• Problem: solve for a, b, c, d<br />

x(0)<br />

x i<br />

d x<br />

x(1)<br />

x<br />

i 1<br />

a<br />

x<br />

b<br />

x<br />

c<br />

x<br />

d<br />

x'<br />

(0) x'<br />

i<br />

c x<br />

x (1) x<br />

' <br />

3<br />

a 2<br />

b c<br />

'<br />

i 1<br />

x<br />

x<br />

x<br />

x<br />

• Solution:<br />

a<br />

b<br />

c<br />

d<br />

x<br />

x<br />

x<br />

x<br />

2(<br />

x<br />

3(<br />

x<br />

x<br />

'<br />

x<br />

i<br />

i<br />

i<br />

i1<br />

x<br />

i1<br />

x<br />

i<br />

) x<br />

'<br />

i<br />

) 2<br />

x'<br />

x<br />

'<br />

i<br />

i1<br />

x<br />

'<br />

i1<br />

Numerical Methods © Wen-Chieh Lin 10


Hermite <strong>Curve</strong> Formation (cont.)<br />

• The coefficients can be expressed as linear<br />

combination of geometric information<br />

a<br />

b<br />

c<br />

d<br />

x<br />

x<br />

x<br />

x<br />

2(<br />

x<br />

3(<br />

x<br />

x<br />

'<br />

x<br />

i<br />

i<br />

i<br />

i1<br />

x<br />

i1<br />

x<br />

i<br />

) x<br />

'<br />

i<br />

) 2<br />

x'<br />

a<br />

<br />

b<br />

<br />

c<br />

<br />

d<br />

x<br />

x<br />

x<br />

x<br />

x<br />

'<br />

i<br />

i1<br />

x<br />

'<br />

2<br />

<br />

3<br />

<br />

<br />

0<br />

<br />

1<br />

i1<br />

2<br />

3<br />

0<br />

0<br />

1<br />

2<br />

1<br />

0<br />

1 <br />

xi<br />

1<br />

<br />

x<br />

<br />

i<br />

0 <br />

x'<br />

<br />

0 <br />

x'<br />

i<br />

1<br />

i<br />

1<br />

<br />

<br />

<br />

<br />

<br />

<br />

Numerical Methods © Wen-Chieh Lin 11


Hermite <strong>Curve</strong> Formation (cont.)<br />

• x component of Hermite curve is then represented as<br />

x(<br />

u)<br />

<br />

3<br />

u<br />

<br />

u<br />

u<br />

2<br />

3<br />

u<br />

u<br />

2<br />

u<br />

2<br />

<br />

<br />

3<br />

1 <br />

0<br />

<br />

1<br />

a<br />

<br />

<br />

b<br />

1 <br />

c<br />

<br />

d<br />

x<br />

x<br />

x<br />

x<br />

2<br />

3<br />

0<br />

0<br />

<br />

<br />

<br />

<br />

<br />

<br />

1<br />

2<br />

1<br />

0<br />

1 <br />

xi<br />

1<br />

<br />

x<br />

<br />

i<br />

0 <br />

x'<br />

<br />

0 <br />

x'<br />

i<br />

1<br />

i<br />

1<br />

<br />

<br />

<br />

<br />

<br />

<br />

Numerical Methods © Wen-Chieh Lin 12


Hermite <strong>Curve</strong> Formation (cont.)<br />

• Including y <strong>and</strong> z components<br />

3<br />

2<br />

x(<br />

u)<br />

y(<br />

u)<br />

z(<br />

u)<br />

u<br />

u u 1<br />

<br />

3<br />

u<br />

P(u)<br />

u<br />

2<br />

u<br />

2<br />

<br />

<br />

3<br />

1 <br />

0<br />

<br />

1<br />

2<br />

3<br />

0<br />

0<br />

1<br />

2<br />

1<br />

0<br />

a<br />

<br />

b<br />

<br />

c<br />

<br />

d<br />

x<br />

x<br />

x<br />

1 <br />

xi<br />

1<br />

<br />

x<br />

<br />

i<br />

0 <br />

x'<br />

<br />

0 <br />

x'<br />

i<br />

u T M p<br />

x<br />

1<br />

i<br />

1<br />

y'<br />

i1<br />

y'<br />

i1<br />

<br />

<br />

<br />

<br />

<br />

<br />

z'<br />

i1<br />

z'<br />

i1<br />

<br />

<br />

<br />

<br />

<br />

<br />

Numerical Methods © Wen-Chieh Lin 13<br />

a<br />

b<br />

c<br />

d<br />

y<br />

y<br />

y<br />

y<br />

y<br />

y<br />

a<br />

b<br />

c<br />

d<br />

i<br />

i<br />

z<br />

z<br />

z<br />

z<br />

z<br />

z<br />

i<br />

i


Hermite <strong>Interpolation</strong> in Matrix Form<br />

P(u)<br />

u<br />

u T u [<br />

3<br />

T<br />

, u<br />

Mp<br />

2<br />

, u,1]<br />

2<br />

<br />

3<br />

M <br />

0<br />

<br />

1<br />

is<br />

2<br />

2<br />

the parameter<br />

M is the cofficient matrix<br />

p is the geometric information<br />

P(0)<br />

3<br />

0<br />

0<br />

1<br />

1<br />

0<br />

1 <br />

1<br />

<br />

<br />

0 <br />

<br />

0 <br />

control points/knots<br />

P(1)<br />

P '(0)<br />

P' (1)<br />

p<br />

i<br />

<br />

p<br />

<br />

i<br />

p<br />

p<br />

'<br />

i<br />

<br />

p<br />

'<br />

i<br />

P<br />

(0) <br />

<br />

P(1)<br />

<br />

<br />

<br />

P<br />

'(0)<br />

<br />

<br />

P<br />

'(1)<br />

<br />

Numerical Methods © Wen-Chieh Lin 14<br />

1<br />

1<br />

ith segment in<br />

composite curves


Blending Functions of Hermite Splines<br />

• Each cubic Hermite spline is a linear combination of<br />

4 blending functions<br />

P(<br />

u)<br />

u<br />

geometric information<br />

T<br />

Mp<br />

P(<br />

u)<br />

3 2<br />

2<br />

u 3<br />

u 1<br />

pi<br />

<br />

3 2<br />

<br />

2<br />

u 3<br />

u<br />

<br />

p<br />

<br />

i<br />

3 2<br />

u 2<br />

u u<br />

p'<br />

<br />

3 2<br />

<br />

u u<br />

<br />

p<br />

'<br />

i<br />

T<br />

1<br />

i<br />

1<br />

<br />

<br />

<br />

<br />

<br />

<br />

Numerical Methods © Wen-Chieh Lin 15


Bezier <strong>Curve</strong>s<br />

• Another variant of the same game<br />

• Instead of endpoints <strong>and</strong> tangents, four control<br />

points<br />

• points p 0<br />

<strong>and</strong> p 3<br />

are on the curve<br />

• points p 1<br />

<strong>and</strong> p 2<br />

are off the curve<br />

• P(0) = p 0<br />

, P(1) = p 3<br />

• P’(0) = 3(p 1<br />

- p 0<br />

), P’(1) = 3(p 3<br />

- p 2<br />

)<br />

3<br />

2<br />

P(<br />

u)<br />

<br />

u<br />

u<br />

u<br />

1<br />

<br />

<br />

2<br />

<br />

2<br />

1<br />

<br />

<br />

3<br />

3<br />

<br />

2<br />

<br />

<br />

0<br />

0<br />

1<br />

<br />

<br />

1<br />

0 0<br />

1<br />

<br />

P<br />

1(0)<br />

00<br />

<br />

1<br />

<br />

<br />

P0<br />

(1) 00<br />

<br />

<br />

0<br />

<br />

<br />

P'<br />

3(0)<br />

3<br />

0<br />

<br />

<br />

0<br />

<br />

P<br />

0'<br />

(1) 0<br />

3<br />

p 0<br />

p 1<br />

p 2<br />

p 3<br />

0<br />

p<br />

1<br />

<br />

p<br />

<br />

0<br />

p<br />

<br />

3<br />

p<br />

<br />

<br />

<br />

<br />

<br />

<br />

Numerical Methods © Wen-Chieh Lin 16<br />

0<br />

1<br />

2<br />

3


Bezier <strong>Curve</strong>s (cont.)<br />

• Variant of the Hermite spline<br />

• basis matrix derived from the Hermite basis<br />

• Gives more uniform control knobs (series of<br />

points) than Hermite<br />

• The slope at u = 0 is the slope of the secant<br />

line between p 0 <strong>and</strong> p 1<br />

P' (0) 3(<br />

p 1<br />

p<br />

0)<br />

dx du 3 ( x 1<br />

x<br />

0)<br />

dy du 3 ( y 1<br />

y<br />

0)<br />

dy dx y y<br />

) ( x<br />

(<br />

1 0 1<br />

x<br />

0<br />

)<br />

Numerical Methods © Wen-Chieh Lin 17


Blending Functions of Bezier <strong>Curve</strong>s<br />

• Also known as degree 3 Bernstein polynomials<br />

• Weighted sum of Bernstein polynomials (infinite<br />

series) converges uniformly to any continuous<br />

function on the interval [0,1]<br />

P(<br />

u)<br />

3<br />

(1<br />

u<br />

) p<br />

<br />

2<br />

3<br />

u(1<br />

u<br />

) p<br />

<br />

<br />

i<br />

2<br />

3u<br />

(1 u<br />

) p<br />

i<br />

<br />

3<br />

<br />

u <br />

p<br />

i<br />

T<br />

i<br />

1<br />

2<br />

3<br />

<br />

<br />

<br />

<br />

<br />

<br />

Numerical Methods © Wen-Chieh Lin 18


Composing Bezier <strong>Curve</strong>s<br />

• Control points at consecutive segments need<br />

be collinear to avoid a discontinuity of slope<br />

Numerical Methods © Wen-Chieh Lin 19


<strong>Interpolation</strong>, Continuity, <strong>and</strong><br />

• Hermite/Bezier splines<br />

• Interpolate control points<br />

• Local control<br />

Local Control<br />

• No C2 continuity for cubics<br />

• Cubic (Natural) splines<br />

• Interpolate control points<br />

• No local control - moving one control point affects whole<br />

curve<br />

• C2 continuity for cubics<br />

• Can’t get C2, interpolation <strong>and</strong> local control with cubics<br />

Numerical Methods © Wen-Chieh Lin 20


Cubic B-Splines<br />

• Basis-spline (B-spline)<br />

• Give up interpolation<br />

• the curve passes near the control points<br />

• best generated with interactive placement (it’s hard<br />

to guess where the curve will go)<br />

• <strong>Curve</strong> obeys the convex hull property<br />

• C2 continuity <strong>and</strong> local control are good<br />

compensation for loss of interpolation<br />

Numerical Methods © Wen-Chieh Lin 21


Cubic B-splines (cont.)<br />

• Any evaluation of conditions of B i (1) cannot use p i+3 ,<br />

because p i+3 does not appear in B i (u)<br />

• Evaluation conditions of B i+1 (0) cannot use p i-1<br />

• Two conditions satisfy this symmetric condition<br />

pi<br />

4<br />

pi<br />

Bi<br />

1) B<br />

i1<br />

(0) <br />

6<br />

p i<br />

p i1<br />

B B<br />

i<br />

i 1<br />

p<br />

1<br />

i2<br />

(<br />

p<br />

i<br />

p<br />

i2<br />

B' i<br />

(1) B<br />

'<br />

i1<br />

(0) <br />

2<br />

p i1<br />

p i2<br />

B i2<br />

p i4<br />

p i3<br />

Numerical Methods © Wen-Chieh Lin 22


• From<br />

Solving for B-spline Coefficients<br />

B ( u)<br />

• We can obtain<br />

B<br />

B<br />

i<br />

(0)<br />

<br />

i<br />

d i<br />

(1) a<br />

B'<br />

B'<br />

i<br />

(0)<br />

i<br />

<br />

i<br />

B'<br />

i<br />

c i<br />

i<br />

b<br />

( u)<br />

i<br />

a u<br />

c<br />

(1) 3<br />

a 2<br />

b<br />

i<br />

i<br />

3<br />

i<br />

b u<br />

i<br />

2<br />

c u d<br />

2<br />

3<br />

a u 2<br />

b u c<br />

i<br />

i<br />

d<br />

c<br />

i<br />

i<br />

• We can solve a set of 4 equations for a, b, c, d<br />

i<br />

i<br />

i<br />

i<br />

Then plug in the followings<br />

pi<br />

4<br />

pi<br />

1<br />

p<br />

i<br />

Bi<br />

1) B<br />

i1<br />

(0) <br />

6<br />

pi<br />

1<br />

4<br />

pi<br />

p<br />

i1<br />

Bi<br />

( 0) <br />

6<br />

p<br />

i<br />

p<br />

i2<br />

B' i<br />

(1) B<br />

'<br />

i1<br />

(0) <br />

2<br />

p<br />

i1 p<br />

i1<br />

B' i<br />

(0) <br />

2<br />

2<br />

(<br />

Numerical Methods © Wen-Chieh Lin 23


Cubic B-splines in Matrix Form<br />

B ( u)<br />

i<br />

u<br />

T<br />

Mp<br />

1<br />

<br />

6<br />

u<br />

3<br />

u<br />

2<br />

u<br />

<br />

1<br />

<br />

<br />

3<br />

1 <br />

<br />

3<br />

<br />

1<br />

3<br />

6<br />

0<br />

4<br />

3<br />

3<br />

3<br />

1<br />

1<br />

pi<br />

0<br />

<br />

p<br />

<br />

0<br />

pi<br />

<br />

0<br />

pi<br />

1<br />

i<br />

1<br />

2<br />

<br />

<br />

<br />

<br />

<br />

<br />

B B<br />

i<br />

i 1<br />

( u 0<br />

)<br />

p i<br />

p i1<br />

b<br />

)<br />

b 1<br />

( u<br />

p i1<br />

p i2<br />

B i2<br />

b <br />

( ) b ( u 2<br />

)<br />

1<br />

u<br />

p i4<br />

p i3<br />

Numerical Methods © Wen-Chieh Lin 24


Blending Functions of Cubic B-Splines<br />

• Represent cubic b-spline as<br />

B<br />

• Blending functions<br />

b<br />

b<br />

b<br />

3<br />

(1 u<br />

)<br />

6<br />

3<br />

u 2<br />

u<br />

2<br />

3<br />

u u<br />

<br />

2 2<br />

1<br />

<br />

0<br />

<br />

i( u)<br />

b<br />

1<br />

pi<br />

1<br />

b<br />

0<br />

pi<br />

b<br />

1pi1<br />

b<br />

2<br />

pi<br />

2<br />

2<br />

3<br />

u<br />

<br />

2<br />

1<br />

<br />

b<br />

1<br />

u<br />

<br />

6<br />

3<br />

1<br />

6<br />

b 0(<br />

u )<br />

b 1<br />

( u)<br />

b <br />

( ) ( )<br />

1<br />

u<br />

2 u<br />

Numerical Methods © Wen-Chieh Lin 25<br />

b<br />

2


Continuity in Cubic B-splines<br />

• Continuity conditions similar to cubic spline<br />

B<br />

pi<br />

4<br />

pi<br />

1<br />

p<br />

i<br />

1) B<br />

i1<br />

(0) <br />

6<br />

p<br />

i<br />

p<br />

i<br />

(1) B<br />

'<br />

i1<br />

(0) <br />

2<br />

(1) B<br />

" (0) p<br />

p p<br />

2<br />

i( B<br />

B<br />

2<br />

'<br />

i<br />

"<br />

i<br />

i1 i<br />

2<br />

i1<br />

i2<br />

Prove that this condition<br />

is automatically satisfied<br />

if the first two hold!<br />

p i<br />

p i1<br />

B B<br />

i i 1<br />

p i<br />

1<br />

p i2<br />

B i2<br />

p i4<br />

Numerical Methods © Wen-Chieh Lin 26<br />

p i3


Composing B-splines<br />

• Given points p 0 , …, p n , we can construct B-<br />

splines B 1 through B n-2<br />

• We need additional points outside the domain of<br />

the given points to construct B 0 <strong>and</strong> B n-1<br />

• Add fictitious points p -2 , p -1 , p n+1 , p n+2<br />

• Set p -2 = p -1 = p 0 <strong>and</strong> p n+2 = p n+1 = p n so that the<br />

curve starts <strong>and</strong> ends at the given extreme points<br />

Numerical Methods © Wen-Chieh Lin 27


Comparison of Basic Cubic Splines<br />

Type Control points Continuity <strong>Interpolation</strong><br />

Natural n points, shared C2 Y<br />

Hermite 2 points, 2 slopes C1 Y<br />

Bezier 4 points C1 Y<br />

B-Splines 4 points, shared C2 N<br />

Numerical Methods © Wen-Chieh Lin 28


Interpolating on a Surface<br />

• Bezier surfaces<br />

• B-Spline surfaces<br />

Numerical Methods © Wen-Chieh Lin 29


Bicubic Surfaces<br />

• To represent surfaces use bicubic functions<br />

• x(u,v), y(u,v), z(u,v) are cubic polyomials both in u<br />

<strong>and</strong> v<br />

• 16 terms for combination of powers of u <strong>and</strong> v<br />

x<br />

3 3 3 2<br />

( u,<br />

v)<br />

a<br />

11u<br />

v a<br />

12u<br />

v <br />

a<br />

44<br />

• A bicubic surface can be represented by a 4x4<br />

matrix A<br />

x( u,<br />

v)<br />

u T Av<br />

• Also known as tensor surfaces<br />

u<br />

T<br />

<br />

u<br />

1<br />

Numerical Methods © Wen-Chieh Lin 30<br />

3<br />

3<br />

u<br />

2<br />

u<br />

2<br />

v v v v 1<br />

T


Numerical Methods © Wen-Chieh Lin 31<br />

Cubic B-Spline Surfaces<br />

• Recall B-spline curve:<br />

v<br />

M<br />

MX<br />

u<br />

Av<br />

u<br />

T<br />

j<br />

i<br />

ij<br />

v<br />

u<br />

x ,<br />

T<br />

T<br />

)<br />

,<br />

( <br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

2<br />

1<br />

1<br />

2<br />

3<br />

0<br />

1<br />

4<br />

1<br />

0<br />

3<br />

0<br />

3<br />

0<br />

3<br />

6<br />

3<br />

1<br />

3<br />

3<br />

1<br />

1<br />

6<br />

1<br />

)<br />

(<br />

i<br />

i<br />

i<br />

i<br />

T<br />

i<br />

p<br />

p<br />

p<br />

p<br />

u<br />

u<br />

u<br />

u<br />

B<br />

Mp<br />

u<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

2<br />

2,<br />

1<br />

2,<br />

2,<br />

1<br />

2,<br />

2<br />

1,<br />

1<br />

1,<br />

1,<br />

1<br />

1,<br />

2<br />

,<br />

1<br />

,<br />

,<br />

1<br />

,<br />

2<br />

1,<br />

1<br />

1,<br />

1,<br />

1<br />

1,<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

ij<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

X


Numerical Methods © Wen-Chieh Lin 32<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

2<br />

1<br />

1<br />

2<br />

3<br />

0<br />

0<br />

0<br />

1<br />

0<br />

0<br />

3<br />

3<br />

0<br />

3<br />

6<br />

3<br />

1<br />

3<br />

3<br />

1<br />

1<br />

)<br />

(<br />

i<br />

i<br />

i<br />

i<br />

T<br />

i<br />

p<br />

p<br />

p<br />

p<br />

u<br />

u<br />

u<br />

u<br />

Mp<br />

u<br />

P<br />

Bezier Surfaces<br />

• Recall Bezier curve:<br />

v<br />

M<br />

MX<br />

u<br />

Av<br />

u<br />

T<br />

j<br />

i<br />

ij<br />

v<br />

u<br />

x ,<br />

T<br />

T<br />

)<br />

,<br />

( <br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

2<br />

2,<br />

1<br />

2,<br />

2,<br />

1<br />

2,<br />

2<br />

1,<br />

1<br />

1,<br />

1,<br />

1<br />

1,<br />

2<br />

,<br />

1<br />

,<br />

,<br />

1<br />

,<br />

2<br />

1,<br />

1<br />

1,<br />

1,<br />

1<br />

1,<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

ij<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

X


Bezier Surfaces<br />

p ( 0,0) p<br />

0,0) 3(<br />

p 10<br />

p<br />

)<br />

00<br />

p<br />

u<br />

p<br />

v<br />

(<br />

00<br />

( 0,0) 3(<br />

p 01<br />

p<br />

00)<br />

u<br />

v<br />

Numerical Methods © Wen-Chieh Lin 33


Least Square Approximation<br />

• Measurement errors are inevitable in<br />

observational <strong>and</strong> experimental sciences<br />

• Errors can be smoothed out by averaging over<br />

many cases, i.e., taking more measurements<br />

than are strictly necessary to determine<br />

parameters of system<br />

• Resulting system is overdetermined, so usually<br />

there is no exact solution<br />

Numerical Methods © Wen-Chieh Lin 34


Least Square Approximation<br />

• Higher dimensional data are projected into<br />

lower dimensional space to suppress irrelevant<br />

details<br />

• Such projection is most conveniently<br />

accomplished by least square method<br />

Linear projection from 2D to 1D space:<br />

Original data distributed in a 2D space<br />

They can be approximated as a 1D line<br />

Numerical Methods © Wen-Chieh Lin 35


Linear Least Squares<br />

• For linear problems, we obtain overdetermined<br />

linear system Aa = b with m×n matrix A, m > n<br />

• System is better written Aa ≈b, since equality<br />

is usually not exactly satisfied when m > n<br />

• Least squares solution a minimizes squared<br />

Euclidean norm of residual vector r = b - Aa<br />

min<br />

a<br />

r<br />

2<br />

min<br />

2<br />

a<br />

b<br />

Aa<br />

2<br />

2<br />

Numerical Methods © Wen-Chieh Lin 36


<strong>Curve</strong> <strong>Fitting</strong><br />

• Given m data points (x i , y i ), find n-vector a of<br />

parameters that gives “best fit”to model<br />

function y = f(x, a), m<br />

2<br />

min ( y f<br />

( x,<br />

a))<br />

<br />

• Problem is linear if function f is linear in<br />

components of a<br />

f ( x,<br />

a)<br />

a<br />

(<br />

x)<br />

a<br />

(<br />

x)<br />

<br />

a<br />

n(<br />

x)<br />

1 1 2 2<br />

n<br />

Aa≈b with A ij =φ j (x i ) <strong>and</strong> b i =y i<br />

i1<br />

• Problem can be written in matrix form as<br />

a<br />

i<br />

Numerical Methods © Wen-Chieh Lin 37


• Polynomial fitting<br />

f<br />

<strong>Curve</strong> <strong>Fitting</strong> (cont.)<br />

( x,<br />

a)<br />

a<br />

is linear, since polynomial linear in coefficients,<br />

though nonlinear in independent variable x<br />

• <strong>Fitting</strong> sum of exponentials<br />

is example of nonlinear problem<br />

• We will only consider linear least squares<br />

problems for now<br />

1<br />

a<br />

2<br />

x<br />

<br />

<br />

n<br />

a n<br />

x<br />

f ( x,<br />

a)<br />

a e<br />

a2x<br />

<br />

a<br />

1<br />

n1<br />

1<br />

Numerical Methods © Wen-Chieh Lin 38<br />

e<br />

a<br />

n<br />

x


Example: curve fitting<br />

• <strong>Fitting</strong> quadratic polynomial to five data points<br />

gives linear least squares problem<br />

Aa<br />

1<br />

<br />

<br />

1<br />

<br />

1<br />

<br />

1<br />

<br />

1<br />

x<br />

x<br />

x<br />

x<br />

x<br />

1<br />

2<br />

3<br />

4<br />

5<br />

x<br />

x<br />

x<br />

x<br />

x<br />

2<br />

1<br />

2<br />

2<br />

2<br />

3<br />

2<br />

4<br />

2<br />

5<br />

<br />

<br />

<br />

a<br />

<br />

a<br />

<br />

<br />

a<br />

<br />

<br />

1<br />

2<br />

3<br />

y<br />

<br />

y<br />

<br />

<br />

<br />

y<br />

<br />

<br />

<br />

<br />

y<br />

<br />

y<br />

1<br />

2<br />

3<br />

4<br />

5<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

Numerical Methods © Wen-Chieh Lin 39


Example: curve fitting (cont.)<br />

• For data x -1.0 -0.5 0.0 0.5 1.0<br />

y 1.0 0.5 0.0 0.5 2.0<br />

overdetermined linear system is<br />

1<br />

<br />

1<br />

<br />

Aa <br />

1<br />

<br />

<br />

1<br />

<br />

1<br />

1.0<br />

0.5<br />

0.0<br />

0.5<br />

1.0<br />

1.0 <br />

0.25<br />

<br />

a<br />

<br />

0.0 <br />

<br />

a<br />

<br />

<br />

0.25<br />

<br />

<br />

a<br />

1.0 <br />

1.0<br />

<br />

<br />

0.5<br />

<br />

<br />

<br />

0.0<br />

<br />

<br />

<br />

<br />

0.5<br />

<br />

<br />

2.0<br />

• Solution, which we will see later how to compute,<br />

is<br />

T<br />

a [0.086 0.40 1.4]<br />

1<br />

2<br />

3<br />

Numerical Methods © Wen-Chieh Lin 40


Example (cont.)<br />

• The computed approximation polynomial is<br />

p( x)<br />

0.086<br />

0.4<br />

x 1.4<br />

x<br />

• Resulting curve <strong>and</strong> original data points are<br />

shown in graph<br />

2<br />

Numerical Methods © Wen-Chieh Lin 41


Normal Equations<br />

• To minimize squared Euclidean norm of<br />

residual vector<br />

2<br />

T<br />

T<br />

r r<br />

r (<br />

b Aa<br />

) ( b Aa<br />

)<br />

2<br />

T T T T T<br />

b<br />

b 2<br />

a A b a<br />

A Aa<br />

• Take derivative with respect to a <strong>and</strong> set it to 0<br />

T<br />

T<br />

2A<br />

Aa 2<br />

A b 0<br />

which reduces to n×n linear system of normal<br />

equations<br />

T<br />

T<br />

A Aa A<br />

b<br />

Numerical Methods © Wen-Chieh Lin 42


Orthogonality<br />

• Space spanned by columns of m × n matrix A,<br />

span(<br />

A)<br />

{<br />

Aa : aR<br />

, is of dimension at most n<br />

• If m > n, b generally does not lie in span(A),<br />

so there is no exact solution to Aa = b<br />

• Vector y = Aa in span(A) closest to b in 2-<br />

norm occurs when residual r = b - Aa is<br />

orthogonal to span(A),<br />

again giving system of<br />

n<br />

}<br />

0 A<br />

T<br />

r<br />

A<br />

T<br />

(b Aa)<br />

A<br />

T<br />

Aa A<br />

Numerical Methods © Wen-Chieh Lin 43<br />

T<br />

b


Orthogonality (cont.)<br />

• Geometric relationships among b, r, <strong>and</strong><br />

span(A) are shown in diagram<br />

b<br />

r<br />

b<br />

Aa<br />

span(A)<br />

y Aa<br />

Numerical Methods © Wen-Chieh Lin 44


Pseudoinverse <strong>and</strong> Condition Number<br />

• Nonsquare m × n matrix A has no inverse in<br />

usual sense<br />

• If rank(A) = n, pseudoinverse is defined by<br />

T T<br />

A<br />

1<br />

(<br />

A A)<br />

A<br />

<strong>and</strong> condition number by<br />

cond(A) A<br />

• By convention, cond(A) = ∞ if rank(A) < n<br />

A<br />

<br />

Numerical Methods © Wen-Chieh Lin 45


Pseudoinverse <strong>and</strong> Condition Number<br />

• Just as condition number of square matrix<br />

measure closeness to singularity, condition<br />

number of rectangular matrix measures<br />

closeness to rank deficiency<br />

• Least squares solution of Aa≈b is given by<br />

a=A + b<br />

Numerical Methods © Wen-Chieh Lin 46


Sensitivity <strong>and</strong> Conditioning<br />

• Sensitivity of least squares solution to Aa≈b<br />

depends on b as well as A<br />

• Define angle θbetween b <strong>and</strong> y = Aa by<br />

y<br />

cos( )<br />

<br />

b<br />

• Bound on perturbation ∆x in solution x due to<br />

perturbation ∆b in b is given by<br />

x<br />

x<br />

2<br />

2<br />

2<br />

2<br />

cond<br />

( A)<br />

Aa<br />

<br />

b<br />

1<br />

cos(<br />

2<br />

2<br />

)<br />

b<br />

2<br />

Numerical Methods © Wen-Chieh Lin 47<br />

b<br />

2


Sensitivity <strong>and</strong> Conditioning<br />

• Similarly, for perturbation E in matrix A,<br />

x<br />

x<br />

2<br />

2<br />

cond ( A)(tan(<br />

cond ) ( A)<br />

1<br />

)<br />

E<br />

A<br />

2<br />

2<br />

• Condition number of least squares solution is<br />

about cond(A) if residual is small, but can be<br />

squared or arbitrarily worse for large residual<br />

Numerical Methods © Wen-Chieh Lin 48


Normal Equation Method<br />

• If m × n matrix A has rank n, then symmetric n ×<br />

n matrix is positive definite, so its Cholesky<br />

factorization<br />

T<br />

T<br />

A A LL<br />

can be used to obtain solution a to system of<br />

normal quations T<br />

T<br />

A Aa A<br />

b<br />

which has same solution as linear least squares<br />

problem Aa≈b<br />

• Normal equations method involves<br />

transformations:<br />

rectangular square triangular<br />

Numerical Methods © Wen-Chieh Lin 49


Example: Normal Equation Method<br />

• For polynomial data-fitting on<br />

x -1.0 -0.5 0.0 0.5 1.0<br />

y 1.0 0.5 0.0 0.5 2.0<br />

• Normal equations method gives<br />

A T A<br />

1<br />

<br />

<br />

1.0<br />

<br />

<br />

1.0<br />

1<br />

0.5<br />

0.25<br />

1<br />

0.0<br />

0.0<br />

1<br />

0.5<br />

0.25<br />

1<br />

1<br />

<br />

1<br />

<br />

1.0<br />

<br />

1<br />

<br />

<br />

1.0<br />

<br />

1<br />

<br />

1<br />

1.0<br />

0.5<br />

0.0<br />

0.5<br />

1.0<br />

1.0 <br />

0.25<br />

<br />

5.0<br />

<br />

0.0 <br />

<br />

0.0<br />

<br />

<br />

0.25<br />

<br />

<br />

2.5<br />

1.0 <br />

0.0<br />

2.5<br />

0.0<br />

2.5 <br />

0.0<br />

<br />

<br />

2.125<br />

Numerical Methods © Wen-Chieh Lin 50


Example: Normal Equation Method<br />

• For polynomial data-fitting on<br />

x -1.0 -0.5 0.0 0.5 1.0<br />

y 1.0 0.5 0.0 0.5 2.0<br />

• Normal equations method gives<br />

1<br />

A T b <br />

<br />

1.0<br />

<br />

<br />

1.0<br />

1<br />

0.5<br />

0.25<br />

1<br />

0.0<br />

0.0<br />

1<br />

0.5<br />

0.25<br />

1.0<br />

<br />

1<br />

<br />

0.5<br />

4.0<br />

<br />

<br />

1.0<br />

<br />

0.0<br />

<br />

1.0<br />

<br />

<br />

<br />

<br />

1.0<br />

<br />

0.5<br />

<br />

<br />

3.25<br />

<br />

2.0<br />

Numerical Methods © Wen-Chieh Lin 51


Example (cont.)<br />

• Choleskey factorization of symmetric positive<br />

definite matrix A T A gives<br />

5.0<br />

A T A <br />

<br />

0.0<br />

<br />

<br />

2.5<br />

0.0<br />

2.5<br />

0.0<br />

2.5 2.236<br />

0.0<br />

<br />

<br />

<br />

0.0<br />

<br />

2.125<br />

<br />

1.118<br />

1.581<br />

0.0 <br />

2.236<br />

0.0<br />

<br />

0.0<br />

<br />

0.935<br />

<br />

0.0<br />

1.581<br />

• Solving lower triangular system Lz = A T b by<br />

forward substitution gives z = [1.789 0.632 1.336] T<br />

• Solving lower triangular system La = z by forward<br />

substitution gives a = [0.086 0.400 1.429] T<br />

0.0<br />

0.0<br />

0.0<br />

0.0<br />

1.118<br />

0.0<br />

0.935 <br />

Numerical Methods © Wen-Chieh Lin 52

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!