28.11.2014 Views

Interpolation and Curve Fitting

Interpolation and Curve Fitting

Interpolation and Curve Fitting

SHOW MORE
SHOW LESS

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

<strong>Interpolation</strong> <strong>and</strong><br />

<strong>Curve</strong> <strong>Fitting</strong><br />

Pierre Bézier<br />

De Casteljau construction of Bezier curve<br />

From www.wikipedia.org


Bezier <strong>Curve</strong>s <strong>and</strong> B-Spline <strong>Curve</strong>s<br />

• Not really “interpolating”curves as they<br />

normally do not pass through data points<br />

• <strong>Curve</strong>s stay within the polygon defined by the<br />

given points<br />

• Local control: effect of changing a knot is<br />

limited to a local region in Bezier <strong>and</strong> b-spline<br />

• Global control: moving a knot in a cubic spline<br />

affects the whole curve<br />

Numerical Methods © Wen-Chieh Lin 2


Global vs. Local Control<br />

• Does a small change affect the whole curve or<br />

just a small segment?<br />

• Local is usually more intuitive <strong>and</strong> provided<br />

by composite curves<br />

Global<br />

Local<br />

Numerical Methods © Wen-Chieh Lin 3


Parametric Form<br />

• Explicit function<br />

y f<br />

(x)<br />

• Parametric form<br />

x F ( u)<br />

1<br />

y F ( u)<br />

2<br />

• Example: unit circle<br />

x cos(u)<br />

y sin(u)<br />

Numerical Methods © Wen-Chieh Lin 4


<strong>Interpolation</strong> in Parametric Form<br />

• Given n+1 control points,<br />

x<br />

i <br />

pi <br />

<br />

y<br />

<br />

i<br />

, i 0 ,..., n<br />

<br />

<br />

zi<br />

<br />

• Compute interpolation functions<br />

x<br />

( u)<br />

<br />

( u)<br />

<br />

<br />

y(<br />

u)<br />

<br />

, 0 u<br />

1<br />

<br />

<br />

z(<br />

u)<br />

<br />

P j<br />

j: index for each segment of curve<br />

Numerical Methods © Wen-Chieh Lin 5


Composite Segments<br />

• Divide a curve into multiple segments<br />

• Represent each in a parametric form<br />

• Maintain continuity between segments<br />

• position<br />

• tangent<br />

• curvature<br />

P 1 (u) P 2 (u) P 3 (u) P n (u)<br />

Numerical Methods © Wen-Chieh Lin 6


Cubic 3D <strong>Curve</strong>s<br />

• Three cubic polynomials, one for each coordinate<br />

3 2<br />

x(<br />

u)<br />

a<br />

u b<br />

u c<br />

u d<br />

y(<br />

u)<br />

z(<br />

u)<br />

a<br />

a<br />

• In matrix notation<br />

x<br />

y<br />

z<br />

u<br />

u<br />

3<br />

3<br />

b<br />

x<br />

y<br />

z<br />

u<br />

b u<br />

2<br />

2<br />

c<br />

c<br />

z<br />

x<br />

y<br />

u d<br />

u d<br />

a<br />

<br />

b<br />

<br />

c<br />

<br />

d<br />

3<br />

2<br />

x(<br />

u)<br />

y(<br />

u)<br />

z(<br />

u)<br />

u<br />

u u 1<br />

Numerical Methods © Wen-Chieh Lin 7<br />

x<br />

x<br />

x<br />

x<br />

z<br />

x<br />

y<br />

a<br />

b<br />

c<br />

c<br />

y<br />

y<br />

y<br />

y<br />

a<br />

b<br />

c<br />

d<br />

z<br />

z<br />

z<br />

z


Hermite <strong>Interpolation</strong><br />

• Hermite interpolation requires<br />

• values at endpoints<br />

• derivatives at endpoints<br />

x(<br />

u)<br />

a<br />

x<br />

u<br />

3<br />

b<br />

x<br />

u<br />

c<br />

Needs 4 equations to<br />

solve 4 unknowns<br />

2<br />

x<br />

u<br />

d<br />

x<br />

• To create a composite curve, use the end of<br />

one as the beginning of the other <strong>and</strong> share the<br />

tangent vector<br />

P(0)<br />

control points/knots<br />

P(1)<br />

P '(0)<br />

P' (1)<br />

Numerical Methods © Wen-Chieh Lin 8


Hermite <strong>Curve</strong> Formation—<br />

consider x coordinate first<br />

• Cubic polynomial <strong>and</strong> its derivative<br />

3 2<br />

x(<br />

u)<br />

a<br />

u b<br />

u c<br />

u<br />

x'<br />

( u)<br />

• Given x i , x i+1 , x’ i , x’ i+1 , solve for a, b, c, d<br />

• 4 equations are given for 4 unknowns<br />

x<br />

x<br />

x<br />

2<br />

3<br />

a u 2<br />

b<br />

x<br />

x<br />

u c<br />

x<br />

d<br />

x(0)<br />

x i<br />

d x<br />

x(1)<br />

x<br />

i 1<br />

a<br />

x<br />

b<br />

x<br />

c<br />

x<br />

d<br />

x'<br />

(0) x'<br />

i<br />

c x<br />

x (1) x<br />

' <br />

3<br />

a 2<br />

b c<br />

'<br />

i 1<br />

x<br />

x<br />

Numerical Methods © Wen-Chieh Lin 9<br />

x<br />

x<br />

x


Hermite <strong>Curve</strong> Formation (cont.)<br />

• Problem: solve for a, b, c, d<br />

x(0)<br />

x i<br />

d x<br />

x(1)<br />

x<br />

i 1<br />

a<br />

x<br />

b<br />

x<br />

c<br />

x<br />

d<br />

x'<br />

(0) x'<br />

i<br />

c x<br />

x (1) x<br />

' <br />

3<br />

a 2<br />

b c<br />

'<br />

i 1<br />

x<br />

x<br />

x<br />

x<br />

• Solution:<br />

a<br />

b<br />

c<br />

d<br />

x<br />

x<br />

x<br />

x<br />

2(<br />

x<br />

3(<br />

x<br />

x<br />

'<br />

x<br />

i<br />

i<br />

i<br />

i1<br />

x<br />

i1<br />

x<br />

i<br />

) x<br />

'<br />

i<br />

) 2<br />

x'<br />

x<br />

'<br />

i<br />

i1<br />

x<br />

'<br />

i1<br />

Numerical Methods © Wen-Chieh Lin 10


Hermite <strong>Curve</strong> Formation (cont.)<br />

• The coefficients can be expressed as linear<br />

combination of geometric information<br />

a<br />

b<br />

c<br />

d<br />

x<br />

x<br />

x<br />

x<br />

2(<br />

x<br />

3(<br />

x<br />

x<br />

'<br />

x<br />

i<br />

i<br />

i<br />

i1<br />

x<br />

i1<br />

x<br />

i<br />

) x<br />

'<br />

i<br />

) 2<br />

x'<br />

a<br />

<br />

b<br />

<br />

c<br />

<br />

d<br />

x<br />

x<br />

x<br />

x<br />

x<br />

'<br />

i<br />

i1<br />

x<br />

'<br />

2<br />

<br />

3<br />

<br />

<br />

0<br />

<br />

1<br />

i1<br />

2<br />

3<br />

0<br />

0<br />

1<br />

2<br />

1<br />

0<br />

1 <br />

xi<br />

1<br />

<br />

x<br />

<br />

i<br />

0 <br />

x'<br />

<br />

0 <br />

x'<br />

i<br />

1<br />

i<br />

1<br />

<br />

<br />

<br />

<br />

<br />

<br />

Numerical Methods © Wen-Chieh Lin 11


Hermite <strong>Curve</strong> Formation (cont.)<br />

• x component of Hermite curve is then represented as<br />

x(<br />

u)<br />

<br />

3<br />

u<br />

<br />

u<br />

u<br />

2<br />

3<br />

u<br />

u<br />

2<br />

u<br />

2<br />

<br />

<br />

3<br />

1 <br />

0<br />

<br />

1<br />

a<br />

<br />

<br />

b<br />

1 <br />

c<br />

<br />

d<br />

x<br />

x<br />

x<br />

x<br />

2<br />

3<br />

0<br />

0<br />

<br />

<br />

<br />

<br />

<br />

<br />

1<br />

2<br />

1<br />

0<br />

1 <br />

xi<br />

1<br />

<br />

x<br />

<br />

i<br />

0 <br />

x'<br />

<br />

0 <br />

x'<br />

i<br />

1<br />

i<br />

1<br />

<br />

<br />

<br />

<br />

<br />

<br />

Numerical Methods © Wen-Chieh Lin 12


Hermite <strong>Curve</strong> Formation (cont.)<br />

• Including y <strong>and</strong> z components<br />

3<br />

2<br />

x(<br />

u)<br />

y(<br />

u)<br />

z(<br />

u)<br />

u<br />

u u 1<br />

<br />

3<br />

u<br />

P(u)<br />

u<br />

2<br />

u<br />

2<br />

<br />

<br />

3<br />

1 <br />

0<br />

<br />

1<br />

2<br />

3<br />

0<br />

0<br />

1<br />

2<br />

1<br />

0<br />

a<br />

<br />

b<br />

<br />

c<br />

<br />

d<br />

x<br />

x<br />

x<br />

1 <br />

xi<br />

1<br />

<br />

x<br />

<br />

i<br />

0 <br />

x'<br />

<br />

0 <br />

x'<br />

i<br />

u T M p<br />

x<br />

1<br />

i<br />

1<br />

y'<br />

i1<br />

y'<br />

i1<br />

<br />

<br />

<br />

<br />

<br />

<br />

z'<br />

i1<br />

z'<br />

i1<br />

<br />

<br />

<br />

<br />

<br />

<br />

Numerical Methods © Wen-Chieh Lin 13<br />

a<br />

b<br />

c<br />

d<br />

y<br />

y<br />

y<br />

y<br />

y<br />

y<br />

a<br />

b<br />

c<br />

d<br />

i<br />

i<br />

z<br />

z<br />

z<br />

z<br />

z<br />

z<br />

i<br />

i


Hermite <strong>Interpolation</strong> in Matrix Form<br />

P(u)<br />

u<br />

u T u [<br />

3<br />

T<br />

, u<br />

Mp<br />

2<br />

, u,1]<br />

2<br />

<br />

3<br />

M <br />

0<br />

<br />

1<br />

is<br />

2<br />

2<br />

the parameter<br />

M is the cofficient matrix<br />

p is the geometric information<br />

P(0)<br />

3<br />

0<br />

0<br />

1<br />

1<br />

0<br />

1 <br />

1<br />

<br />

<br />

0 <br />

<br />

0 <br />

control points/knots<br />

P(1)<br />

P '(0)<br />

P' (1)<br />

p<br />

i<br />

<br />

p<br />

<br />

i<br />

p<br />

p<br />

'<br />

i<br />

<br />

p<br />

'<br />

i<br />

P<br />

(0) <br />

<br />

P(1)<br />

<br />

<br />

<br />

P<br />

'(0)<br />

<br />

<br />

P<br />

'(1)<br />

<br />

Numerical Methods © Wen-Chieh Lin 14<br />

1<br />

1<br />

ith segment in<br />

composite curves


Blending Functions of Hermite Splines<br />

• Each cubic Hermite spline is a linear combination of<br />

4 blending functions<br />

P(<br />

u)<br />

u<br />

geometric information<br />

T<br />

Mp<br />

P(<br />

u)<br />

3 2<br />

2<br />

u 3<br />

u 1<br />

pi<br />

<br />

3 2<br />

<br />

2<br />

u 3<br />

u<br />

<br />

p<br />

<br />

i<br />

3 2<br />

u 2<br />

u u<br />

p'<br />

<br />

3 2<br />

<br />

u u<br />

<br />

p<br />

'<br />

i<br />

T<br />

1<br />

i<br />

1<br />

<br />

<br />

<br />

<br />

<br />

<br />

Numerical Methods © Wen-Chieh Lin 15


Bezier <strong>Curve</strong>s<br />

• Another variant of the same game<br />

• Instead of endpoints <strong>and</strong> tangents, four control<br />

points<br />

• points p 0<br />

<strong>and</strong> p 3<br />

are on the curve<br />

• points p 1<br />

<strong>and</strong> p 2<br />

are off the curve<br />

• P(0) = p 0<br />

, P(1) = p 3<br />

• P’(0) = 3(p 1<br />

- p 0<br />

), P’(1) = 3(p 3<br />

- p 2<br />

)<br />

3<br />

2<br />

P(<br />

u)<br />

<br />

u<br />

u<br />

u<br />

1<br />

<br />

<br />

2<br />

<br />

2<br />

1<br />

<br />

<br />

3<br />

3<br />

<br />

2<br />

<br />

<br />

0<br />

0<br />

1<br />

<br />

<br />

1<br />

0 0<br />

1<br />

<br />

P<br />

1(0)<br />

00<br />

<br />

1<br />

<br />

<br />

P0<br />

(1) 00<br />

<br />

<br />

0<br />

<br />

<br />

P'<br />

3(0)<br />

3<br />

0<br />

<br />

<br />

0<br />

<br />

P<br />

0'<br />

(1) 0<br />

3<br />

p 0<br />

p 1<br />

p 2<br />

p 3<br />

0<br />

p<br />

1<br />

<br />

p<br />

<br />

0<br />

p<br />

<br />

3<br />

p<br />

<br />

<br />

<br />

<br />

<br />

<br />

Numerical Methods © Wen-Chieh Lin 16<br />

0<br />

1<br />

2<br />

3


Bezier <strong>Curve</strong>s (cont.)<br />

• Variant of the Hermite spline<br />

• basis matrix derived from the Hermite basis<br />

• Gives more uniform control knobs (series of<br />

points) than Hermite<br />

• The slope at u = 0 is the slope of the secant<br />

line between p 0 <strong>and</strong> p 1<br />

P' (0) 3(<br />

p 1<br />

p<br />

0)<br />

dx du 3 ( x 1<br />

x<br />

0)<br />

dy du 3 ( y 1<br />

y<br />

0)<br />

dy dx y y<br />

) ( x<br />

(<br />

1 0 1<br />

x<br />

0<br />

)<br />

Numerical Methods © Wen-Chieh Lin 17


Blending Functions of Bezier <strong>Curve</strong>s<br />

• Also known as degree 3 Bernstein polynomials<br />

• Weighted sum of Bernstein polynomials (infinite<br />

series) converges uniformly to any continuous<br />

function on the interval [0,1]<br />

P(<br />

u)<br />

3<br />

(1<br />

u<br />

) p<br />

<br />

2<br />

3<br />

u(1<br />

u<br />

) p<br />

<br />

<br />

i<br />

2<br />

3u<br />

(1 u<br />

) p<br />

i<br />

<br />

3<br />

<br />

u <br />

p<br />

i<br />

T<br />

i<br />

1<br />

2<br />

3<br />

<br />

<br />

<br />

<br />

<br />

<br />

Numerical Methods © Wen-Chieh Lin 18


Composing Bezier <strong>Curve</strong>s<br />

• Control points at consecutive segments need<br />

be collinear to avoid a discontinuity of slope<br />

Numerical Methods © Wen-Chieh Lin 19


<strong>Interpolation</strong>, Continuity, <strong>and</strong><br />

• Hermite/Bezier splines<br />

• Interpolate control points<br />

• Local control<br />

Local Control<br />

• No C2 continuity for cubics<br />

• Cubic (Natural) splines<br />

• Interpolate control points<br />

• No local control - moving one control point affects whole<br />

curve<br />

• C2 continuity for cubics<br />

• Can’t get C2, interpolation <strong>and</strong> local control with cubics<br />

Numerical Methods © Wen-Chieh Lin 20


Cubic B-Splines<br />

• Basis-spline (B-spline)<br />

• Give up interpolation<br />

• the curve passes near the control points<br />

• best generated with interactive placement (it’s hard<br />

to guess where the curve will go)<br />

• <strong>Curve</strong> obeys the convex hull property<br />

• C2 continuity <strong>and</strong> local control are good<br />

compensation for loss of interpolation<br />

Numerical Methods © Wen-Chieh Lin 21


Cubic B-splines (cont.)<br />

• Any evaluation of conditions of B i (1) cannot use p i+3 ,<br />

because p i+3 does not appear in B i (u)<br />

• Evaluation conditions of B i+1 (0) cannot use p i-1<br />

• Two conditions satisfy this symmetric condition<br />

pi<br />

4<br />

pi<br />

Bi<br />

1) B<br />

i1<br />

(0) <br />

6<br />

p i<br />

p i1<br />

B B<br />

i<br />

i 1<br />

p<br />

1<br />

i2<br />

(<br />

p<br />

i<br />

p<br />

i2<br />

B' i<br />

(1) B<br />

'<br />

i1<br />

(0) <br />

2<br />

p i1<br />

p i2<br />

B i2<br />

p i4<br />

p i3<br />

Numerical Methods © Wen-Chieh Lin 22


• From<br />

Solving for B-spline Coefficients<br />

B ( u)<br />

• We can obtain<br />

B<br />

B<br />

i<br />

(0)<br />

<br />

i<br />

d i<br />

(1) a<br />

B'<br />

B'<br />

i<br />

(0)<br />

i<br />

<br />

i<br />

B'<br />

i<br />

c i<br />

i<br />

b<br />

( u)<br />

i<br />

a u<br />

c<br />

(1) 3<br />

a 2<br />

b<br />

i<br />

i<br />

3<br />

i<br />

b u<br />

i<br />

2<br />

c u d<br />

2<br />

3<br />

a u 2<br />

b u c<br />

i<br />

i<br />

d<br />

c<br />

i<br />

i<br />

• We can solve a set of 4 equations for a, b, c, d<br />

i<br />

i<br />

i<br />

i<br />

Then plug in the followings<br />

pi<br />

4<br />

pi<br />

1<br />

p<br />

i<br />

Bi<br />

1) B<br />

i1<br />

(0) <br />

6<br />

pi<br />

1<br />

4<br />

pi<br />

p<br />

i1<br />

Bi<br />

( 0) <br />

6<br />

p<br />

i<br />

p<br />

i2<br />

B' i<br />

(1) B<br />

'<br />

i1<br />

(0) <br />

2<br />

p<br />

i1 p<br />

i1<br />

B' i<br />

(0) <br />

2<br />

2<br />

(<br />

Numerical Methods © Wen-Chieh Lin 23


Cubic B-splines in Matrix Form<br />

B ( u)<br />

i<br />

u<br />

T<br />

Mp<br />

1<br />

<br />

6<br />

u<br />

3<br />

u<br />

2<br />

u<br />

<br />

1<br />

<br />

<br />

3<br />

1 <br />

<br />

3<br />

<br />

1<br />

3<br />

6<br />

0<br />

4<br />

3<br />

3<br />

3<br />

1<br />

1<br />

pi<br />

0<br />

<br />

p<br />

<br />

0<br />

pi<br />

<br />

0<br />

pi<br />

1<br />

i<br />

1<br />

2<br />

<br />

<br />

<br />

<br />

<br />

<br />

B B<br />

i<br />

i 1<br />

( u 0<br />

)<br />

p i<br />

p i1<br />

b<br />

)<br />

b 1<br />

( u<br />

p i1<br />

p i2<br />

B i2<br />

b <br />

( ) b ( u 2<br />

)<br />

1<br />

u<br />

p i4<br />

p i3<br />

Numerical Methods © Wen-Chieh Lin 24


Blending Functions of Cubic B-Splines<br />

• Represent cubic b-spline as<br />

B<br />

• Blending functions<br />

b<br />

b<br />

b<br />

3<br />

(1 u<br />

)<br />

6<br />

3<br />

u 2<br />

u<br />

2<br />

3<br />

u u<br />

<br />

2 2<br />

1<br />

<br />

0<br />

<br />

i( u)<br />

b<br />

1<br />

pi<br />

1<br />

b<br />

0<br />

pi<br />

b<br />

1pi1<br />

b<br />

2<br />

pi<br />

2<br />

2<br />

3<br />

u<br />

<br />

2<br />

1<br />

<br />

b<br />

1<br />

u<br />

<br />

6<br />

3<br />

1<br />

6<br />

b 0(<br />

u )<br />

b 1<br />

( u)<br />

b <br />

( ) ( )<br />

1<br />

u<br />

2 u<br />

Numerical Methods © Wen-Chieh Lin 25<br />

b<br />

2


Continuity in Cubic B-splines<br />

• Continuity conditions similar to cubic spline<br />

B<br />

pi<br />

4<br />

pi<br />

1<br />

p<br />

i<br />

1) B<br />

i1<br />

(0) <br />

6<br />

p<br />

i<br />

p<br />

i<br />

(1) B<br />

'<br />

i1<br />

(0) <br />

2<br />

(1) B<br />

" (0) p<br />

p p<br />

2<br />

i( B<br />

B<br />

2<br />

'<br />

i<br />

"<br />

i<br />

i1 i<br />

2<br />

i1<br />

i2<br />

Prove that this condition<br />

is automatically satisfied<br />

if the first two hold!<br />

p i<br />

p i1<br />

B B<br />

i i 1<br />

p i<br />

1<br />

p i2<br />

B i2<br />

p i4<br />

Numerical Methods © Wen-Chieh Lin 26<br />

p i3


Composing B-splines<br />

• Given points p 0 , …, p n , we can construct B-<br />

splines B 1 through B n-2<br />

• We need additional points outside the domain of<br />

the given points to construct B 0 <strong>and</strong> B n-1<br />

• Add fictitious points p -2 , p -1 , p n+1 , p n+2<br />

• Set p -2 = p -1 = p 0 <strong>and</strong> p n+2 = p n+1 = p n so that the<br />

curve starts <strong>and</strong> ends at the given extreme points<br />

Numerical Methods © Wen-Chieh Lin 27


Comparison of Basic Cubic Splines<br />

Type Control points Continuity <strong>Interpolation</strong><br />

Natural n points, shared C2 Y<br />

Hermite 2 points, 2 slopes C1 Y<br />

Bezier 4 points C1 Y<br />

B-Splines 4 points, shared C2 N<br />

Numerical Methods © Wen-Chieh Lin 28


Interpolating on a Surface<br />

• Bezier surfaces<br />

• B-Spline surfaces<br />

Numerical Methods © Wen-Chieh Lin 29


Bicubic Surfaces<br />

• To represent surfaces use bicubic functions<br />

• x(u,v), y(u,v), z(u,v) are cubic polyomials both in u<br />

<strong>and</strong> v<br />

• 16 terms for combination of powers of u <strong>and</strong> v<br />

x<br />

3 3 3 2<br />

( u,<br />

v)<br />

a<br />

11u<br />

v a<br />

12u<br />

v <br />

a<br />

44<br />

• A bicubic surface can be represented by a 4x4<br />

matrix A<br />

x( u,<br />

v)<br />

u T Av<br />

• Also known as tensor surfaces<br />

u<br />

T<br />

<br />

u<br />

1<br />

Numerical Methods © Wen-Chieh Lin 30<br />

3<br />

3<br />

u<br />

2<br />

u<br />

2<br />

v v v v 1<br />

T


Numerical Methods © Wen-Chieh Lin 31<br />

Cubic B-Spline Surfaces<br />

• Recall B-spline curve:<br />

v<br />

M<br />

MX<br />

u<br />

Av<br />

u<br />

T<br />

j<br />

i<br />

ij<br />

v<br />

u<br />

x ,<br />

T<br />

T<br />

)<br />

,<br />

( <br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

2<br />

1<br />

1<br />

2<br />

3<br />

0<br />

1<br />

4<br />

1<br />

0<br />

3<br />

0<br />

3<br />

0<br />

3<br />

6<br />

3<br />

1<br />

3<br />

3<br />

1<br />

1<br />

6<br />

1<br />

)<br />

(<br />

i<br />

i<br />

i<br />

i<br />

T<br />

i<br />

p<br />

p<br />

p<br />

p<br />

u<br />

u<br />

u<br />

u<br />

B<br />

Mp<br />

u<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

2<br />

2,<br />

1<br />

2,<br />

2,<br />

1<br />

2,<br />

2<br />

1,<br />

1<br />

1,<br />

1,<br />

1<br />

1,<br />

2<br />

,<br />

1<br />

,<br />

,<br />

1<br />

,<br />

2<br />

1,<br />

1<br />

1,<br />

1,<br />

1<br />

1,<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

ij<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

X


Numerical Methods © Wen-Chieh Lin 32<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

2<br />

1<br />

1<br />

2<br />

3<br />

0<br />

0<br />

0<br />

1<br />

0<br />

0<br />

3<br />

3<br />

0<br />

3<br />

6<br />

3<br />

1<br />

3<br />

3<br />

1<br />

1<br />

)<br />

(<br />

i<br />

i<br />

i<br />

i<br />

T<br />

i<br />

p<br />

p<br />

p<br />

p<br />

u<br />

u<br />

u<br />

u<br />

Mp<br />

u<br />

P<br />

Bezier Surfaces<br />

• Recall Bezier curve:<br />

v<br />

M<br />

MX<br />

u<br />

Av<br />

u<br />

T<br />

j<br />

i<br />

ij<br />

v<br />

u<br />

x ,<br />

T<br />

T<br />

)<br />

,<br />

( <br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

2<br />

2,<br />

1<br />

2,<br />

2,<br />

1<br />

2,<br />

2<br />

1,<br />

1<br />

1,<br />

1,<br />

1<br />

1,<br />

2<br />

,<br />

1<br />

,<br />

,<br />

1<br />

,<br />

2<br />

1,<br />

1<br />

1,<br />

1,<br />

1<br />

1,<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

j<br />

i<br />

ij<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

x<br />

X


Bezier Surfaces<br />

p ( 0,0) p<br />

0,0) 3(<br />

p 10<br />

p<br />

)<br />

00<br />

p<br />

u<br />

p<br />

v<br />

(<br />

00<br />

( 0,0) 3(<br />

p 01<br />

p<br />

00)<br />

u<br />

v<br />

Numerical Methods © Wen-Chieh Lin 33


Least Square Approximation<br />

• Measurement errors are inevitable in<br />

observational <strong>and</strong> experimental sciences<br />

• Errors can be smoothed out by averaging over<br />

many cases, i.e., taking more measurements<br />

than are strictly necessary to determine<br />

parameters of system<br />

• Resulting system is overdetermined, so usually<br />

there is no exact solution<br />

Numerical Methods © Wen-Chieh Lin 34


Least Square Approximation<br />

• Higher dimensional data are projected into<br />

lower dimensional space to suppress irrelevant<br />

details<br />

• Such projection is most conveniently<br />

accomplished by least square method<br />

Linear projection from 2D to 1D space:<br />

Original data distributed in a 2D space<br />

They can be approximated as a 1D line<br />

Numerical Methods © Wen-Chieh Lin 35


Linear Least Squares<br />

• For linear problems, we obtain overdetermined<br />

linear system Aa = b with m×n matrix A, m > n<br />

• System is better written Aa ≈b, since equality<br />

is usually not exactly satisfied when m > n<br />

• Least squares solution a minimizes squared<br />

Euclidean norm of residual vector r = b - Aa<br />

min<br />

a<br />

r<br />

2<br />

min<br />

2<br />

a<br />

b<br />

Aa<br />

2<br />

2<br />

Numerical Methods © Wen-Chieh Lin 36


<strong>Curve</strong> <strong>Fitting</strong><br />

• Given m data points (x i , y i ), find n-vector a of<br />

parameters that gives “best fit”to model<br />

function y = f(x, a), m<br />

2<br />

min ( y f<br />

( x,<br />

a))<br />

<br />

• Problem is linear if function f is linear in<br />

components of a<br />

f ( x,<br />

a)<br />

a<br />

(<br />

x)<br />

a<br />

(<br />

x)<br />

<br />

a<br />

n(<br />

x)<br />

1 1 2 2<br />

n<br />

Aa≈b with A ij =φ j (x i ) <strong>and</strong> b i =y i<br />

i1<br />

• Problem can be written in matrix form as<br />

a<br />

i<br />

Numerical Methods © Wen-Chieh Lin 37


• Polynomial fitting<br />

f<br />

<strong>Curve</strong> <strong>Fitting</strong> (cont.)<br />

( x,<br />

a)<br />

a<br />

is linear, since polynomial linear in coefficients,<br />

though nonlinear in independent variable x<br />

• <strong>Fitting</strong> sum of exponentials<br />

is example of nonlinear problem<br />

• We will only consider linear least squares<br />

problems for now<br />

1<br />

a<br />

2<br />

x<br />

<br />

<br />

n<br />

a n<br />

x<br />

f ( x,<br />

a)<br />

a e<br />

a2x<br />

<br />

a<br />

1<br />

n1<br />

1<br />

Numerical Methods © Wen-Chieh Lin 38<br />

e<br />

a<br />

n<br />

x


Example: curve fitting<br />

• <strong>Fitting</strong> quadratic polynomial to five data points<br />

gives linear least squares problem<br />

Aa<br />

1<br />

<br />

<br />

1<br />

<br />

1<br />

<br />

1<br />

<br />

1<br />

x<br />

x<br />

x<br />

x<br />

x<br />

1<br />

2<br />

3<br />

4<br />

5<br />

x<br />

x<br />

x<br />

x<br />

x<br />

2<br />

1<br />

2<br />

2<br />

2<br />

3<br />

2<br />

4<br />

2<br />

5<br />

<br />

<br />

<br />

a<br />

<br />

a<br />

<br />

<br />

a<br />

<br />

<br />

1<br />

2<br />

3<br />

y<br />

<br />

y<br />

<br />

<br />

<br />

y<br />

<br />

<br />

<br />

<br />

y<br />

<br />

y<br />

1<br />

2<br />

3<br />

4<br />

5<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

Numerical Methods © Wen-Chieh Lin 39


Example: curve fitting (cont.)<br />

• For data x -1.0 -0.5 0.0 0.5 1.0<br />

y 1.0 0.5 0.0 0.5 2.0<br />

overdetermined linear system is<br />

1<br />

<br />

1<br />

<br />

Aa <br />

1<br />

<br />

<br />

1<br />

<br />

1<br />

1.0<br />

0.5<br />

0.0<br />

0.5<br />

1.0<br />

1.0 <br />

0.25<br />

<br />

a<br />

<br />

0.0 <br />

<br />

a<br />

<br />

<br />

0.25<br />

<br />

<br />

a<br />

1.0 <br />

1.0<br />

<br />

<br />

0.5<br />

<br />

<br />

<br />

0.0<br />

<br />

<br />

<br />

<br />

0.5<br />

<br />

<br />

2.0<br />

• Solution, which we will see later how to compute,<br />

is<br />

T<br />

a [0.086 0.40 1.4]<br />

1<br />

2<br />

3<br />

Numerical Methods © Wen-Chieh Lin 40


Example (cont.)<br />

• The computed approximation polynomial is<br />

p( x)<br />

0.086<br />

0.4<br />

x 1.4<br />

x<br />

• Resulting curve <strong>and</strong> original data points are<br />

shown in graph<br />

2<br />

Numerical Methods © Wen-Chieh Lin 41


Normal Equations<br />

• To minimize squared Euclidean norm of<br />

residual vector<br />

2<br />

T<br />

T<br />

r r<br />

r (<br />

b Aa<br />

) ( b Aa<br />

)<br />

2<br />

T T T T T<br />

b<br />

b 2<br />

a A b a<br />

A Aa<br />

• Take derivative with respect to a <strong>and</strong> set it to 0<br />

T<br />

T<br />

2A<br />

Aa 2<br />

A b 0<br />

which reduces to n×n linear system of normal<br />

equations<br />

T<br />

T<br />

A Aa A<br />

b<br />

Numerical Methods © Wen-Chieh Lin 42


Orthogonality<br />

• Space spanned by columns of m × n matrix A,<br />

span(<br />

A)<br />

{<br />

Aa : aR<br />

, is of dimension at most n<br />

• If m > n, b generally does not lie in span(A),<br />

so there is no exact solution to Aa = b<br />

• Vector y = Aa in span(A) closest to b in 2-<br />

norm occurs when residual r = b - Aa is<br />

orthogonal to span(A),<br />

again giving system of<br />

n<br />

}<br />

0 A<br />

T<br />

r<br />

A<br />

T<br />

(b Aa)<br />

A<br />

T<br />

Aa A<br />

Numerical Methods © Wen-Chieh Lin 43<br />

T<br />

b


Orthogonality (cont.)<br />

• Geometric relationships among b, r, <strong>and</strong><br />

span(A) are shown in diagram<br />

b<br />

r<br />

b<br />

Aa<br />

span(A)<br />

y Aa<br />

Numerical Methods © Wen-Chieh Lin 44


Pseudoinverse <strong>and</strong> Condition Number<br />

• Nonsquare m × n matrix A has no inverse in<br />

usual sense<br />

• If rank(A) = n, pseudoinverse is defined by<br />

T T<br />

A<br />

1<br />

(<br />

A A)<br />

A<br />

<strong>and</strong> condition number by<br />

cond(A) A<br />

• By convention, cond(A) = ∞ if rank(A) < n<br />

A<br />

<br />

Numerical Methods © Wen-Chieh Lin 45


Pseudoinverse <strong>and</strong> Condition Number<br />

• Just as condition number of square matrix<br />

measure closeness to singularity, condition<br />

number of rectangular matrix measures<br />

closeness to rank deficiency<br />

• Least squares solution of Aa≈b is given by<br />

a=A + b<br />

Numerical Methods © Wen-Chieh Lin 46


Sensitivity <strong>and</strong> Conditioning<br />

• Sensitivity of least squares solution to Aa≈b<br />

depends on b as well as A<br />

• Define angle θbetween b <strong>and</strong> y = Aa by<br />

y<br />

cos( )<br />

<br />

b<br />

• Bound on perturbation ∆x in solution x due to<br />

perturbation ∆b in b is given by<br />

x<br />

x<br />

2<br />

2<br />

2<br />

2<br />

cond<br />

( A)<br />

Aa<br />

<br />

b<br />

1<br />

cos(<br />

2<br />

2<br />

)<br />

b<br />

2<br />

Numerical Methods © Wen-Chieh Lin 47<br />

b<br />

2


Sensitivity <strong>and</strong> Conditioning<br />

• Similarly, for perturbation E in matrix A,<br />

x<br />

x<br />

2<br />

2<br />

cond ( A)(tan(<br />

cond ) ( A)<br />

1<br />

)<br />

E<br />

A<br />

2<br />

2<br />

• Condition number of least squares solution is<br />

about cond(A) if residual is small, but can be<br />

squared or arbitrarily worse for large residual<br />

Numerical Methods © Wen-Chieh Lin 48


Normal Equation Method<br />

• If m × n matrix A has rank n, then symmetric n ×<br />

n matrix is positive definite, so its Cholesky<br />

factorization<br />

T<br />

T<br />

A A LL<br />

can be used to obtain solution a to system of<br />

normal quations T<br />

T<br />

A Aa A<br />

b<br />

which has same solution as linear least squares<br />

problem Aa≈b<br />

• Normal equations method involves<br />

transformations:<br />

rectangular square triangular<br />

Numerical Methods © Wen-Chieh Lin 49


Example: Normal Equation Method<br />

• For polynomial data-fitting on<br />

x -1.0 -0.5 0.0 0.5 1.0<br />

y 1.0 0.5 0.0 0.5 2.0<br />

• Normal equations method gives<br />

A T A<br />

1<br />

<br />

<br />

1.0<br />

<br />

<br />

1.0<br />

1<br />

0.5<br />

0.25<br />

1<br />

0.0<br />

0.0<br />

1<br />

0.5<br />

0.25<br />

1<br />

1<br />

<br />

1<br />

<br />

1.0<br />

<br />

1<br />

<br />

<br />

1.0<br />

<br />

1<br />

<br />

1<br />

1.0<br />

0.5<br />

0.0<br />

0.5<br />

1.0<br />

1.0 <br />

0.25<br />

<br />

5.0<br />

<br />

0.0 <br />

<br />

0.0<br />

<br />

<br />

0.25<br />

<br />

<br />

2.5<br />

1.0 <br />

0.0<br />

2.5<br />

0.0<br />

2.5 <br />

0.0<br />

<br />

<br />

2.125<br />

Numerical Methods © Wen-Chieh Lin 50


Example: Normal Equation Method<br />

• For polynomial data-fitting on<br />

x -1.0 -0.5 0.0 0.5 1.0<br />

y 1.0 0.5 0.0 0.5 2.0<br />

• Normal equations method gives<br />

1<br />

A T b <br />

<br />

1.0<br />

<br />

<br />

1.0<br />

1<br />

0.5<br />

0.25<br />

1<br />

0.0<br />

0.0<br />

1<br />

0.5<br />

0.25<br />

1.0<br />

<br />

1<br />

<br />

0.5<br />

4.0<br />

<br />

<br />

1.0<br />

<br />

0.0<br />

<br />

1.0<br />

<br />

<br />

<br />

<br />

1.0<br />

<br />

0.5<br />

<br />

<br />

3.25<br />

<br />

2.0<br />

Numerical Methods © Wen-Chieh Lin 51


Example (cont.)<br />

• Choleskey factorization of symmetric positive<br />

definite matrix A T A gives<br />

5.0<br />

A T A <br />

<br />

0.0<br />

<br />

<br />

2.5<br />

0.0<br />

2.5<br />

0.0<br />

2.5 2.236<br />

0.0<br />

<br />

<br />

<br />

0.0<br />

<br />

2.125<br />

<br />

1.118<br />

1.581<br />

0.0 <br />

2.236<br />

0.0<br />

<br />

0.0<br />

<br />

0.935<br />

<br />

0.0<br />

1.581<br />

• Solving lower triangular system Lz = A T b by<br />

forward substitution gives z = [1.789 0.632 1.336] T<br />

• Solving lower triangular system La = z by forward<br />

substitution gives a = [0.086 0.400 1.429] T<br />

0.0<br />

0.0<br />

0.0<br />

0.0<br />

1.118<br />

0.0<br />

0.935 <br />

Numerical Methods © Wen-Chieh Lin 52

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!