Interpolation and Curve Fitting
Interpolation and Curve Fitting
Interpolation and Curve Fitting
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
<strong>Interpolation</strong> <strong>and</strong><br />
<strong>Curve</strong> <strong>Fitting</strong><br />
Pierre Bézier<br />
De Casteljau construction of Bezier curve<br />
From www.wikipedia.org
Bezier <strong>Curve</strong>s <strong>and</strong> B-Spline <strong>Curve</strong>s<br />
• Not really “interpolating”curves as they<br />
normally do not pass through data points<br />
• <strong>Curve</strong>s stay within the polygon defined by the<br />
given points<br />
• Local control: effect of changing a knot is<br />
limited to a local region in Bezier <strong>and</strong> b-spline<br />
• Global control: moving a knot in a cubic spline<br />
affects the whole curve<br />
Numerical Methods © Wen-Chieh Lin 2
Global vs. Local Control<br />
• Does a small change affect the whole curve or<br />
just a small segment?<br />
• Local is usually more intuitive <strong>and</strong> provided<br />
by composite curves<br />
Global<br />
Local<br />
Numerical Methods © Wen-Chieh Lin 3
Parametric Form<br />
• Explicit function<br />
y f<br />
(x)<br />
• Parametric form<br />
x F ( u)<br />
1<br />
y F ( u)<br />
2<br />
• Example: unit circle<br />
x cos(u)<br />
y sin(u)<br />
Numerical Methods © Wen-Chieh Lin 4
<strong>Interpolation</strong> in Parametric Form<br />
• Given n+1 control points,<br />
x<br />
i <br />
pi <br />
<br />
y<br />
<br />
i<br />
, i 0 ,..., n<br />
<br />
<br />
zi<br />
<br />
• Compute interpolation functions<br />
x<br />
( u)<br />
<br />
( u)<br />
<br />
<br />
y(<br />
u)<br />
<br />
, 0 u<br />
1<br />
<br />
<br />
z(<br />
u)<br />
<br />
P j<br />
j: index for each segment of curve<br />
Numerical Methods © Wen-Chieh Lin 5
Composite Segments<br />
• Divide a curve into multiple segments<br />
• Represent each in a parametric form<br />
• Maintain continuity between segments<br />
• position<br />
• tangent<br />
• curvature<br />
P 1 (u) P 2 (u) P 3 (u) P n (u)<br />
Numerical Methods © Wen-Chieh Lin 6
Cubic 3D <strong>Curve</strong>s<br />
• Three cubic polynomials, one for each coordinate<br />
3 2<br />
x(<br />
u)<br />
a<br />
u b<br />
u c<br />
u d<br />
y(<br />
u)<br />
z(<br />
u)<br />
a<br />
a<br />
• In matrix notation<br />
x<br />
y<br />
z<br />
u<br />
u<br />
3<br />
3<br />
b<br />
x<br />
y<br />
z<br />
u<br />
b u<br />
2<br />
2<br />
c<br />
c<br />
z<br />
x<br />
y<br />
u d<br />
u d<br />
a<br />
<br />
b<br />
<br />
c<br />
<br />
d<br />
3<br />
2<br />
x(<br />
u)<br />
y(<br />
u)<br />
z(<br />
u)<br />
u<br />
u u 1<br />
Numerical Methods © Wen-Chieh Lin 7<br />
x<br />
x<br />
x<br />
x<br />
z<br />
x<br />
y<br />
a<br />
b<br />
c<br />
c<br />
y<br />
y<br />
y<br />
y<br />
a<br />
b<br />
c<br />
d<br />
z<br />
z<br />
z<br />
z
Hermite <strong>Interpolation</strong><br />
• Hermite interpolation requires<br />
• values at endpoints<br />
• derivatives at endpoints<br />
x(<br />
u)<br />
a<br />
x<br />
u<br />
3<br />
b<br />
x<br />
u<br />
c<br />
Needs 4 equations to<br />
solve 4 unknowns<br />
2<br />
x<br />
u<br />
d<br />
x<br />
• To create a composite curve, use the end of<br />
one as the beginning of the other <strong>and</strong> share the<br />
tangent vector<br />
P(0)<br />
control points/knots<br />
P(1)<br />
P '(0)<br />
P' (1)<br />
Numerical Methods © Wen-Chieh Lin 8
Hermite <strong>Curve</strong> Formation—<br />
consider x coordinate first<br />
• Cubic polynomial <strong>and</strong> its derivative<br />
3 2<br />
x(<br />
u)<br />
a<br />
u b<br />
u c<br />
u<br />
x'<br />
( u)<br />
• Given x i , x i+1 , x’ i , x’ i+1 , solve for a, b, c, d<br />
• 4 equations are given for 4 unknowns<br />
x<br />
x<br />
x<br />
2<br />
3<br />
a u 2<br />
b<br />
x<br />
x<br />
u c<br />
x<br />
d<br />
x(0)<br />
x i<br />
d x<br />
x(1)<br />
x<br />
i 1<br />
a<br />
x<br />
b<br />
x<br />
c<br />
x<br />
d<br />
x'<br />
(0) x'<br />
i<br />
c x<br />
x (1) x<br />
' <br />
3<br />
a 2<br />
b c<br />
'<br />
i 1<br />
x<br />
x<br />
Numerical Methods © Wen-Chieh Lin 9<br />
x<br />
x<br />
x
Hermite <strong>Curve</strong> Formation (cont.)<br />
• Problem: solve for a, b, c, d<br />
x(0)<br />
x i<br />
d x<br />
x(1)<br />
x<br />
i 1<br />
a<br />
x<br />
b<br />
x<br />
c<br />
x<br />
d<br />
x'<br />
(0) x'<br />
i<br />
c x<br />
x (1) x<br />
' <br />
3<br />
a 2<br />
b c<br />
'<br />
i 1<br />
x<br />
x<br />
x<br />
x<br />
• Solution:<br />
a<br />
b<br />
c<br />
d<br />
x<br />
x<br />
x<br />
x<br />
2(<br />
x<br />
3(<br />
x<br />
x<br />
'<br />
x<br />
i<br />
i<br />
i<br />
i1<br />
x<br />
i1<br />
x<br />
i<br />
) x<br />
'<br />
i<br />
) 2<br />
x'<br />
x<br />
'<br />
i<br />
i1<br />
x<br />
'<br />
i1<br />
Numerical Methods © Wen-Chieh Lin 10
Hermite <strong>Curve</strong> Formation (cont.)<br />
• The coefficients can be expressed as linear<br />
combination of geometric information<br />
a<br />
b<br />
c<br />
d<br />
x<br />
x<br />
x<br />
x<br />
2(<br />
x<br />
3(<br />
x<br />
x<br />
'<br />
x<br />
i<br />
i<br />
i<br />
i1<br />
x<br />
i1<br />
x<br />
i<br />
) x<br />
'<br />
i<br />
) 2<br />
x'<br />
a<br />
<br />
b<br />
<br />
c<br />
<br />
d<br />
x<br />
x<br />
x<br />
x<br />
x<br />
'<br />
i<br />
i1<br />
x<br />
'<br />
2<br />
<br />
3<br />
<br />
<br />
0<br />
<br />
1<br />
i1<br />
2<br />
3<br />
0<br />
0<br />
1<br />
2<br />
1<br />
0<br />
1 <br />
xi<br />
1<br />
<br />
x<br />
<br />
i<br />
0 <br />
x'<br />
<br />
0 <br />
x'<br />
i<br />
1<br />
i<br />
1<br />
<br />
<br />
<br />
<br />
<br />
<br />
Numerical Methods © Wen-Chieh Lin 11
Hermite <strong>Curve</strong> Formation (cont.)<br />
• x component of Hermite curve is then represented as<br />
x(<br />
u)<br />
<br />
3<br />
u<br />
<br />
u<br />
u<br />
2<br />
3<br />
u<br />
u<br />
2<br />
u<br />
2<br />
<br />
<br />
3<br />
1 <br />
0<br />
<br />
1<br />
a<br />
<br />
<br />
b<br />
1 <br />
c<br />
<br />
d<br />
x<br />
x<br />
x<br />
x<br />
2<br />
3<br />
0<br />
0<br />
<br />
<br />
<br />
<br />
<br />
<br />
1<br />
2<br />
1<br />
0<br />
1 <br />
xi<br />
1<br />
<br />
x<br />
<br />
i<br />
0 <br />
x'<br />
<br />
0 <br />
x'<br />
i<br />
1<br />
i<br />
1<br />
<br />
<br />
<br />
<br />
<br />
<br />
Numerical Methods © Wen-Chieh Lin 12
Hermite <strong>Curve</strong> Formation (cont.)<br />
• Including y <strong>and</strong> z components<br />
3<br />
2<br />
x(<br />
u)<br />
y(<br />
u)<br />
z(<br />
u)<br />
u<br />
u u 1<br />
<br />
3<br />
u<br />
P(u)<br />
u<br />
2<br />
u<br />
2<br />
<br />
<br />
3<br />
1 <br />
0<br />
<br />
1<br />
2<br />
3<br />
0<br />
0<br />
1<br />
2<br />
1<br />
0<br />
a<br />
<br />
b<br />
<br />
c<br />
<br />
d<br />
x<br />
x<br />
x<br />
1 <br />
xi<br />
1<br />
<br />
x<br />
<br />
i<br />
0 <br />
x'<br />
<br />
0 <br />
x'<br />
i<br />
u T M p<br />
x<br />
1<br />
i<br />
1<br />
y'<br />
i1<br />
y'<br />
i1<br />
<br />
<br />
<br />
<br />
<br />
<br />
z'<br />
i1<br />
z'<br />
i1<br />
<br />
<br />
<br />
<br />
<br />
<br />
Numerical Methods © Wen-Chieh Lin 13<br />
a<br />
b<br />
c<br />
d<br />
y<br />
y<br />
y<br />
y<br />
y<br />
y<br />
a<br />
b<br />
c<br />
d<br />
i<br />
i<br />
z<br />
z<br />
z<br />
z<br />
z<br />
z<br />
i<br />
i
Hermite <strong>Interpolation</strong> in Matrix Form<br />
P(u)<br />
u<br />
u T u [<br />
3<br />
T<br />
, u<br />
Mp<br />
2<br />
, u,1]<br />
2<br />
<br />
3<br />
M <br />
0<br />
<br />
1<br />
is<br />
2<br />
2<br />
the parameter<br />
M is the cofficient matrix<br />
p is the geometric information<br />
P(0)<br />
3<br />
0<br />
0<br />
1<br />
1<br />
0<br />
1 <br />
1<br />
<br />
<br />
0 <br />
<br />
0 <br />
control points/knots<br />
P(1)<br />
P '(0)<br />
P' (1)<br />
p<br />
i<br />
<br />
p<br />
<br />
i<br />
p<br />
p<br />
'<br />
i<br />
<br />
p<br />
'<br />
i<br />
P<br />
(0) <br />
<br />
P(1)<br />
<br />
<br />
<br />
P<br />
'(0)<br />
<br />
<br />
P<br />
'(1)<br />
<br />
Numerical Methods © Wen-Chieh Lin 14<br />
1<br />
1<br />
ith segment in<br />
composite curves
Blending Functions of Hermite Splines<br />
• Each cubic Hermite spline is a linear combination of<br />
4 blending functions<br />
P(<br />
u)<br />
u<br />
geometric information<br />
T<br />
Mp<br />
P(<br />
u)<br />
3 2<br />
2<br />
u 3<br />
u 1<br />
pi<br />
<br />
3 2<br />
<br />
2<br />
u 3<br />
u<br />
<br />
p<br />
<br />
i<br />
3 2<br />
u 2<br />
u u<br />
p'<br />
<br />
3 2<br />
<br />
u u<br />
<br />
p<br />
'<br />
i<br />
T<br />
1<br />
i<br />
1<br />
<br />
<br />
<br />
<br />
<br />
<br />
Numerical Methods © Wen-Chieh Lin 15
Bezier <strong>Curve</strong>s<br />
• Another variant of the same game<br />
• Instead of endpoints <strong>and</strong> tangents, four control<br />
points<br />
• points p 0<br />
<strong>and</strong> p 3<br />
are on the curve<br />
• points p 1<br />
<strong>and</strong> p 2<br />
are off the curve<br />
• P(0) = p 0<br />
, P(1) = p 3<br />
• P’(0) = 3(p 1<br />
- p 0<br />
), P’(1) = 3(p 3<br />
- p 2<br />
)<br />
3<br />
2<br />
P(<br />
u)<br />
<br />
u<br />
u<br />
u<br />
1<br />
<br />
<br />
2<br />
<br />
2<br />
1<br />
<br />
<br />
3<br />
3<br />
<br />
2<br />
<br />
<br />
0<br />
0<br />
1<br />
<br />
<br />
1<br />
0 0<br />
1<br />
<br />
P<br />
1(0)<br />
00<br />
<br />
1<br />
<br />
<br />
P0<br />
(1) 00<br />
<br />
<br />
0<br />
<br />
<br />
P'<br />
3(0)<br />
3<br />
0<br />
<br />
<br />
0<br />
<br />
P<br />
0'<br />
(1) 0<br />
3<br />
p 0<br />
p 1<br />
p 2<br />
p 3<br />
0<br />
p<br />
1<br />
<br />
p<br />
<br />
0<br />
p<br />
<br />
3<br />
p<br />
<br />
<br />
<br />
<br />
<br />
<br />
Numerical Methods © Wen-Chieh Lin 16<br />
0<br />
1<br />
2<br />
3
Bezier <strong>Curve</strong>s (cont.)<br />
• Variant of the Hermite spline<br />
• basis matrix derived from the Hermite basis<br />
• Gives more uniform control knobs (series of<br />
points) than Hermite<br />
• The slope at u = 0 is the slope of the secant<br />
line between p 0 <strong>and</strong> p 1<br />
P' (0) 3(<br />
p 1<br />
p<br />
0)<br />
dx du 3 ( x 1<br />
x<br />
0)<br />
dy du 3 ( y 1<br />
y<br />
0)<br />
dy dx y y<br />
) ( x<br />
(<br />
1 0 1<br />
x<br />
0<br />
)<br />
Numerical Methods © Wen-Chieh Lin 17
Blending Functions of Bezier <strong>Curve</strong>s<br />
• Also known as degree 3 Bernstein polynomials<br />
• Weighted sum of Bernstein polynomials (infinite<br />
series) converges uniformly to any continuous<br />
function on the interval [0,1]<br />
P(<br />
u)<br />
3<br />
(1<br />
u<br />
) p<br />
<br />
2<br />
3<br />
u(1<br />
u<br />
) p<br />
<br />
<br />
i<br />
2<br />
3u<br />
(1 u<br />
) p<br />
i<br />
<br />
3<br />
<br />
u <br />
p<br />
i<br />
T<br />
i<br />
1<br />
2<br />
3<br />
<br />
<br />
<br />
<br />
<br />
<br />
Numerical Methods © Wen-Chieh Lin 18
Composing Bezier <strong>Curve</strong>s<br />
• Control points at consecutive segments need<br />
be collinear to avoid a discontinuity of slope<br />
Numerical Methods © Wen-Chieh Lin 19
<strong>Interpolation</strong>, Continuity, <strong>and</strong><br />
• Hermite/Bezier splines<br />
• Interpolate control points<br />
• Local control<br />
Local Control<br />
• No C2 continuity for cubics<br />
• Cubic (Natural) splines<br />
• Interpolate control points<br />
• No local control - moving one control point affects whole<br />
curve<br />
• C2 continuity for cubics<br />
• Can’t get C2, interpolation <strong>and</strong> local control with cubics<br />
Numerical Methods © Wen-Chieh Lin 20
Cubic B-Splines<br />
• Basis-spline (B-spline)<br />
• Give up interpolation<br />
• the curve passes near the control points<br />
• best generated with interactive placement (it’s hard<br />
to guess where the curve will go)<br />
• <strong>Curve</strong> obeys the convex hull property<br />
• C2 continuity <strong>and</strong> local control are good<br />
compensation for loss of interpolation<br />
Numerical Methods © Wen-Chieh Lin 21
Cubic B-splines (cont.)<br />
• Any evaluation of conditions of B i (1) cannot use p i+3 ,<br />
because p i+3 does not appear in B i (u)<br />
• Evaluation conditions of B i+1 (0) cannot use p i-1<br />
• Two conditions satisfy this symmetric condition<br />
pi<br />
4<br />
pi<br />
Bi<br />
1) B<br />
i1<br />
(0) <br />
6<br />
p i<br />
p i1<br />
B B<br />
i<br />
i 1<br />
p<br />
1<br />
i2<br />
(<br />
p<br />
i<br />
p<br />
i2<br />
B' i<br />
(1) B<br />
'<br />
i1<br />
(0) <br />
2<br />
p i1<br />
p i2<br />
B i2<br />
p i4<br />
p i3<br />
Numerical Methods © Wen-Chieh Lin 22
• From<br />
Solving for B-spline Coefficients<br />
B ( u)<br />
• We can obtain<br />
B<br />
B<br />
i<br />
(0)<br />
<br />
i<br />
d i<br />
(1) a<br />
B'<br />
B'<br />
i<br />
(0)<br />
i<br />
<br />
i<br />
B'<br />
i<br />
c i<br />
i<br />
b<br />
( u)<br />
i<br />
a u<br />
c<br />
(1) 3<br />
a 2<br />
b<br />
i<br />
i<br />
3<br />
i<br />
b u<br />
i<br />
2<br />
c u d<br />
2<br />
3<br />
a u 2<br />
b u c<br />
i<br />
i<br />
d<br />
c<br />
i<br />
i<br />
• We can solve a set of 4 equations for a, b, c, d<br />
i<br />
i<br />
i<br />
i<br />
Then plug in the followings<br />
pi<br />
4<br />
pi<br />
1<br />
p<br />
i<br />
Bi<br />
1) B<br />
i1<br />
(0) <br />
6<br />
pi<br />
1<br />
4<br />
pi<br />
p<br />
i1<br />
Bi<br />
( 0) <br />
6<br />
p<br />
i<br />
p<br />
i2<br />
B' i<br />
(1) B<br />
'<br />
i1<br />
(0) <br />
2<br />
p<br />
i1 p<br />
i1<br />
B' i<br />
(0) <br />
2<br />
2<br />
(<br />
Numerical Methods © Wen-Chieh Lin 23
Cubic B-splines in Matrix Form<br />
B ( u)<br />
i<br />
u<br />
T<br />
Mp<br />
1<br />
<br />
6<br />
u<br />
3<br />
u<br />
2<br />
u<br />
<br />
1<br />
<br />
<br />
3<br />
1 <br />
<br />
3<br />
<br />
1<br />
3<br />
6<br />
0<br />
4<br />
3<br />
3<br />
3<br />
1<br />
1<br />
pi<br />
0<br />
<br />
p<br />
<br />
0<br />
pi<br />
<br />
0<br />
pi<br />
1<br />
i<br />
1<br />
2<br />
<br />
<br />
<br />
<br />
<br />
<br />
B B<br />
i<br />
i 1<br />
( u 0<br />
)<br />
p i<br />
p i1<br />
b<br />
)<br />
b 1<br />
( u<br />
p i1<br />
p i2<br />
B i2<br />
b <br />
( ) b ( u 2<br />
)<br />
1<br />
u<br />
p i4<br />
p i3<br />
Numerical Methods © Wen-Chieh Lin 24
Blending Functions of Cubic B-Splines<br />
• Represent cubic b-spline as<br />
B<br />
• Blending functions<br />
b<br />
b<br />
b<br />
3<br />
(1 u<br />
)<br />
6<br />
3<br />
u 2<br />
u<br />
2<br />
3<br />
u u<br />
<br />
2 2<br />
1<br />
<br />
0<br />
<br />
i( u)<br />
b<br />
1<br />
pi<br />
1<br />
b<br />
0<br />
pi<br />
b<br />
1pi1<br />
b<br />
2<br />
pi<br />
2<br />
2<br />
3<br />
u<br />
<br />
2<br />
1<br />
<br />
b<br />
1<br />
u<br />
<br />
6<br />
3<br />
1<br />
6<br />
b 0(<br />
u )<br />
b 1<br />
( u)<br />
b <br />
( ) ( )<br />
1<br />
u<br />
2 u<br />
Numerical Methods © Wen-Chieh Lin 25<br />
b<br />
2
Continuity in Cubic B-splines<br />
• Continuity conditions similar to cubic spline<br />
B<br />
pi<br />
4<br />
pi<br />
1<br />
p<br />
i<br />
1) B<br />
i1<br />
(0) <br />
6<br />
p<br />
i<br />
p<br />
i<br />
(1) B<br />
'<br />
i1<br />
(0) <br />
2<br />
(1) B<br />
" (0) p<br />
p p<br />
2<br />
i( B<br />
B<br />
2<br />
'<br />
i<br />
"<br />
i<br />
i1 i<br />
2<br />
i1<br />
i2<br />
Prove that this condition<br />
is automatically satisfied<br />
if the first two hold!<br />
p i<br />
p i1<br />
B B<br />
i i 1<br />
p i<br />
1<br />
p i2<br />
B i2<br />
p i4<br />
Numerical Methods © Wen-Chieh Lin 26<br />
p i3
Composing B-splines<br />
• Given points p 0 , …, p n , we can construct B-<br />
splines B 1 through B n-2<br />
• We need additional points outside the domain of<br />
the given points to construct B 0 <strong>and</strong> B n-1<br />
• Add fictitious points p -2 , p -1 , p n+1 , p n+2<br />
• Set p -2 = p -1 = p 0 <strong>and</strong> p n+2 = p n+1 = p n so that the<br />
curve starts <strong>and</strong> ends at the given extreme points<br />
Numerical Methods © Wen-Chieh Lin 27
Comparison of Basic Cubic Splines<br />
Type Control points Continuity <strong>Interpolation</strong><br />
Natural n points, shared C2 Y<br />
Hermite 2 points, 2 slopes C1 Y<br />
Bezier 4 points C1 Y<br />
B-Splines 4 points, shared C2 N<br />
Numerical Methods © Wen-Chieh Lin 28
Interpolating on a Surface<br />
• Bezier surfaces<br />
• B-Spline surfaces<br />
Numerical Methods © Wen-Chieh Lin 29
Bicubic Surfaces<br />
• To represent surfaces use bicubic functions<br />
• x(u,v), y(u,v), z(u,v) are cubic polyomials both in u<br />
<strong>and</strong> v<br />
• 16 terms for combination of powers of u <strong>and</strong> v<br />
x<br />
3 3 3 2<br />
( u,<br />
v)<br />
a<br />
11u<br />
v a<br />
12u<br />
v <br />
a<br />
44<br />
• A bicubic surface can be represented by a 4x4<br />
matrix A<br />
x( u,<br />
v)<br />
u T Av<br />
• Also known as tensor surfaces<br />
u<br />
T<br />
<br />
u<br />
1<br />
Numerical Methods © Wen-Chieh Lin 30<br />
3<br />
3<br />
u<br />
2<br />
u<br />
2<br />
v v v v 1<br />
T
Numerical Methods © Wen-Chieh Lin 31<br />
Cubic B-Spline Surfaces<br />
• Recall B-spline curve:<br />
v<br />
M<br />
MX<br />
u<br />
Av<br />
u<br />
T<br />
j<br />
i<br />
ij<br />
v<br />
u<br />
x ,<br />
T<br />
T<br />
)<br />
,<br />
( <br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
2<br />
1<br />
1<br />
2<br />
3<br />
0<br />
1<br />
4<br />
1<br />
0<br />
3<br />
0<br />
3<br />
0<br />
3<br />
6<br />
3<br />
1<br />
3<br />
3<br />
1<br />
1<br />
6<br />
1<br />
)<br />
(<br />
i<br />
i<br />
i<br />
i<br />
T<br />
i<br />
p<br />
p<br />
p<br />
p<br />
u<br />
u<br />
u<br />
u<br />
B<br />
Mp<br />
u<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
2<br />
2,<br />
1<br />
2,<br />
2,<br />
1<br />
2,<br />
2<br />
1,<br />
1<br />
1,<br />
1,<br />
1<br />
1,<br />
2<br />
,<br />
1<br />
,<br />
,<br />
1<br />
,<br />
2<br />
1,<br />
1<br />
1,<br />
1,<br />
1<br />
1,<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
ij<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
X
Numerical Methods © Wen-Chieh Lin 32<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
2<br />
1<br />
1<br />
2<br />
3<br />
0<br />
0<br />
0<br />
1<br />
0<br />
0<br />
3<br />
3<br />
0<br />
3<br />
6<br />
3<br />
1<br />
3<br />
3<br />
1<br />
1<br />
)<br />
(<br />
i<br />
i<br />
i<br />
i<br />
T<br />
i<br />
p<br />
p<br />
p<br />
p<br />
u<br />
u<br />
u<br />
u<br />
Mp<br />
u<br />
P<br />
Bezier Surfaces<br />
• Recall Bezier curve:<br />
v<br />
M<br />
MX<br />
u<br />
Av<br />
u<br />
T<br />
j<br />
i<br />
ij<br />
v<br />
u<br />
x ,<br />
T<br />
T<br />
)<br />
,<br />
( <br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
2<br />
2,<br />
1<br />
2,<br />
2,<br />
1<br />
2,<br />
2<br />
1,<br />
1<br />
1,<br />
1,<br />
1<br />
1,<br />
2<br />
,<br />
1<br />
,<br />
,<br />
1<br />
,<br />
2<br />
1,<br />
1<br />
1,<br />
1,<br />
1<br />
1,<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
j<br />
i<br />
ij<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
x<br />
X
Bezier Surfaces<br />
p ( 0,0) p<br />
0,0) 3(<br />
p 10<br />
p<br />
)<br />
00<br />
p<br />
u<br />
p<br />
v<br />
(<br />
00<br />
( 0,0) 3(<br />
p 01<br />
p<br />
00)<br />
u<br />
v<br />
Numerical Methods © Wen-Chieh Lin 33
Least Square Approximation<br />
• Measurement errors are inevitable in<br />
observational <strong>and</strong> experimental sciences<br />
• Errors can be smoothed out by averaging over<br />
many cases, i.e., taking more measurements<br />
than are strictly necessary to determine<br />
parameters of system<br />
• Resulting system is overdetermined, so usually<br />
there is no exact solution<br />
Numerical Methods © Wen-Chieh Lin 34
Least Square Approximation<br />
• Higher dimensional data are projected into<br />
lower dimensional space to suppress irrelevant<br />
details<br />
• Such projection is most conveniently<br />
accomplished by least square method<br />
Linear projection from 2D to 1D space:<br />
Original data distributed in a 2D space<br />
They can be approximated as a 1D line<br />
Numerical Methods © Wen-Chieh Lin 35
Linear Least Squares<br />
• For linear problems, we obtain overdetermined<br />
linear system Aa = b with m×n matrix A, m > n<br />
• System is better written Aa ≈b, since equality<br />
is usually not exactly satisfied when m > n<br />
• Least squares solution a minimizes squared<br />
Euclidean norm of residual vector r = b - Aa<br />
min<br />
a<br />
r<br />
2<br />
min<br />
2<br />
a<br />
b<br />
Aa<br />
2<br />
2<br />
Numerical Methods © Wen-Chieh Lin 36
<strong>Curve</strong> <strong>Fitting</strong><br />
• Given m data points (x i , y i ), find n-vector a of<br />
parameters that gives “best fit”to model<br />
function y = f(x, a), m<br />
2<br />
min ( y f<br />
( x,<br />
a))<br />
<br />
• Problem is linear if function f is linear in<br />
components of a<br />
f ( x,<br />
a)<br />
a<br />
(<br />
x)<br />
a<br />
(<br />
x)<br />
<br />
a<br />
n(<br />
x)<br />
1 1 2 2<br />
n<br />
Aa≈b with A ij =φ j (x i ) <strong>and</strong> b i =y i<br />
i1<br />
• Problem can be written in matrix form as<br />
a<br />
i<br />
Numerical Methods © Wen-Chieh Lin 37
• Polynomial fitting<br />
f<br />
<strong>Curve</strong> <strong>Fitting</strong> (cont.)<br />
( x,<br />
a)<br />
a<br />
is linear, since polynomial linear in coefficients,<br />
though nonlinear in independent variable x<br />
• <strong>Fitting</strong> sum of exponentials<br />
is example of nonlinear problem<br />
• We will only consider linear least squares<br />
problems for now<br />
1<br />
a<br />
2<br />
x<br />
<br />
<br />
n<br />
a n<br />
x<br />
f ( x,<br />
a)<br />
a e<br />
a2x<br />
<br />
a<br />
1<br />
n1<br />
1<br />
Numerical Methods © Wen-Chieh Lin 38<br />
e<br />
a<br />
n<br />
x
Example: curve fitting<br />
• <strong>Fitting</strong> quadratic polynomial to five data points<br />
gives linear least squares problem<br />
Aa<br />
1<br />
<br />
<br />
1<br />
<br />
1<br />
<br />
1<br />
<br />
1<br />
x<br />
x<br />
x<br />
x<br />
x<br />
1<br />
2<br />
3<br />
4<br />
5<br />
x<br />
x<br />
x<br />
x<br />
x<br />
2<br />
1<br />
2<br />
2<br />
2<br />
3<br />
2<br />
4<br />
2<br />
5<br />
<br />
<br />
<br />
a<br />
<br />
a<br />
<br />
<br />
a<br />
<br />
<br />
1<br />
2<br />
3<br />
y<br />
<br />
y<br />
<br />
<br />
<br />
y<br />
<br />
<br />
<br />
<br />
y<br />
<br />
y<br />
1<br />
2<br />
3<br />
4<br />
5<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Numerical Methods © Wen-Chieh Lin 39
Example: curve fitting (cont.)<br />
• For data x -1.0 -0.5 0.0 0.5 1.0<br />
y 1.0 0.5 0.0 0.5 2.0<br />
overdetermined linear system is<br />
1<br />
<br />
1<br />
<br />
Aa <br />
1<br />
<br />
<br />
1<br />
<br />
1<br />
1.0<br />
0.5<br />
0.0<br />
0.5<br />
1.0<br />
1.0 <br />
0.25<br />
<br />
a<br />
<br />
0.0 <br />
<br />
a<br />
<br />
<br />
0.25<br />
<br />
<br />
a<br />
1.0 <br />
1.0<br />
<br />
<br />
0.5<br />
<br />
<br />
<br />
0.0<br />
<br />
<br />
<br />
<br />
0.5<br />
<br />
<br />
2.0<br />
• Solution, which we will see later how to compute,<br />
is<br />
T<br />
a [0.086 0.40 1.4]<br />
1<br />
2<br />
3<br />
Numerical Methods © Wen-Chieh Lin 40
Example (cont.)<br />
• The computed approximation polynomial is<br />
p( x)<br />
0.086<br />
0.4<br />
x 1.4<br />
x<br />
• Resulting curve <strong>and</strong> original data points are<br />
shown in graph<br />
2<br />
Numerical Methods © Wen-Chieh Lin 41
Normal Equations<br />
• To minimize squared Euclidean norm of<br />
residual vector<br />
2<br />
T<br />
T<br />
r r<br />
r (<br />
b Aa<br />
) ( b Aa<br />
)<br />
2<br />
T T T T T<br />
b<br />
b 2<br />
a A b a<br />
A Aa<br />
• Take derivative with respect to a <strong>and</strong> set it to 0<br />
T<br />
T<br />
2A<br />
Aa 2<br />
A b 0<br />
which reduces to n×n linear system of normal<br />
equations<br />
T<br />
T<br />
A Aa A<br />
b<br />
Numerical Methods © Wen-Chieh Lin 42
Orthogonality<br />
• Space spanned by columns of m × n matrix A,<br />
span(<br />
A)<br />
{<br />
Aa : aR<br />
, is of dimension at most n<br />
• If m > n, b generally does not lie in span(A),<br />
so there is no exact solution to Aa = b<br />
• Vector y = Aa in span(A) closest to b in 2-<br />
norm occurs when residual r = b - Aa is<br />
orthogonal to span(A),<br />
again giving system of<br />
n<br />
}<br />
0 A<br />
T<br />
r<br />
A<br />
T<br />
(b Aa)<br />
A<br />
T<br />
Aa A<br />
Numerical Methods © Wen-Chieh Lin 43<br />
T<br />
b
Orthogonality (cont.)<br />
• Geometric relationships among b, r, <strong>and</strong><br />
span(A) are shown in diagram<br />
b<br />
r<br />
b<br />
Aa<br />
span(A)<br />
y Aa<br />
Numerical Methods © Wen-Chieh Lin 44
Pseudoinverse <strong>and</strong> Condition Number<br />
• Nonsquare m × n matrix A has no inverse in<br />
usual sense<br />
• If rank(A) = n, pseudoinverse is defined by<br />
T T<br />
A<br />
1<br />
(<br />
A A)<br />
A<br />
<strong>and</strong> condition number by<br />
cond(A) A<br />
• By convention, cond(A) = ∞ if rank(A) < n<br />
A<br />
<br />
Numerical Methods © Wen-Chieh Lin 45
Pseudoinverse <strong>and</strong> Condition Number<br />
• Just as condition number of square matrix<br />
measure closeness to singularity, condition<br />
number of rectangular matrix measures<br />
closeness to rank deficiency<br />
• Least squares solution of Aa≈b is given by<br />
a=A + b<br />
Numerical Methods © Wen-Chieh Lin 46
Sensitivity <strong>and</strong> Conditioning<br />
• Sensitivity of least squares solution to Aa≈b<br />
depends on b as well as A<br />
• Define angle θbetween b <strong>and</strong> y = Aa by<br />
y<br />
cos( )<br />
<br />
b<br />
• Bound on perturbation ∆x in solution x due to<br />
perturbation ∆b in b is given by<br />
x<br />
x<br />
2<br />
2<br />
2<br />
2<br />
cond<br />
( A)<br />
Aa<br />
<br />
b<br />
1<br />
cos(<br />
2<br />
2<br />
)<br />
b<br />
2<br />
Numerical Methods © Wen-Chieh Lin 47<br />
b<br />
2
Sensitivity <strong>and</strong> Conditioning<br />
• Similarly, for perturbation E in matrix A,<br />
x<br />
x<br />
2<br />
2<br />
cond ( A)(tan(<br />
cond ) ( A)<br />
1<br />
)<br />
E<br />
A<br />
2<br />
2<br />
• Condition number of least squares solution is<br />
about cond(A) if residual is small, but can be<br />
squared or arbitrarily worse for large residual<br />
Numerical Methods © Wen-Chieh Lin 48
Normal Equation Method<br />
• If m × n matrix A has rank n, then symmetric n ×<br />
n matrix is positive definite, so its Cholesky<br />
factorization<br />
T<br />
T<br />
A A LL<br />
can be used to obtain solution a to system of<br />
normal quations T<br />
T<br />
A Aa A<br />
b<br />
which has same solution as linear least squares<br />
problem Aa≈b<br />
• Normal equations method involves<br />
transformations:<br />
rectangular square triangular<br />
Numerical Methods © Wen-Chieh Lin 49
Example: Normal Equation Method<br />
• For polynomial data-fitting on<br />
x -1.0 -0.5 0.0 0.5 1.0<br />
y 1.0 0.5 0.0 0.5 2.0<br />
• Normal equations method gives<br />
A T A<br />
1<br />
<br />
<br />
1.0<br />
<br />
<br />
1.0<br />
1<br />
0.5<br />
0.25<br />
1<br />
0.0<br />
0.0<br />
1<br />
0.5<br />
0.25<br />
1<br />
1<br />
<br />
1<br />
<br />
1.0<br />
<br />
1<br />
<br />
<br />
1.0<br />
<br />
1<br />
<br />
1<br />
1.0<br />
0.5<br />
0.0<br />
0.5<br />
1.0<br />
1.0 <br />
0.25<br />
<br />
5.0<br />
<br />
0.0 <br />
<br />
0.0<br />
<br />
<br />
0.25<br />
<br />
<br />
2.5<br />
1.0 <br />
0.0<br />
2.5<br />
0.0<br />
2.5 <br />
0.0<br />
<br />
<br />
2.125<br />
Numerical Methods © Wen-Chieh Lin 50
Example: Normal Equation Method<br />
• For polynomial data-fitting on<br />
x -1.0 -0.5 0.0 0.5 1.0<br />
y 1.0 0.5 0.0 0.5 2.0<br />
• Normal equations method gives<br />
1<br />
A T b <br />
<br />
1.0<br />
<br />
<br />
1.0<br />
1<br />
0.5<br />
0.25<br />
1<br />
0.0<br />
0.0<br />
1<br />
0.5<br />
0.25<br />
1.0<br />
<br />
1<br />
<br />
0.5<br />
4.0<br />
<br />
<br />
1.0<br />
<br />
0.0<br />
<br />
1.0<br />
<br />
<br />
<br />
<br />
1.0<br />
<br />
0.5<br />
<br />
<br />
3.25<br />
<br />
2.0<br />
Numerical Methods © Wen-Chieh Lin 51
Example (cont.)<br />
• Choleskey factorization of symmetric positive<br />
definite matrix A T A gives<br />
5.0<br />
A T A <br />
<br />
0.0<br />
<br />
<br />
2.5<br />
0.0<br />
2.5<br />
0.0<br />
2.5 2.236<br />
0.0<br />
<br />
<br />
<br />
0.0<br />
<br />
2.125<br />
<br />
1.118<br />
1.581<br />
0.0 <br />
2.236<br />
0.0<br />
<br />
0.0<br />
<br />
0.935<br />
<br />
0.0<br />
1.581<br />
• Solving lower triangular system Lz = A T b by<br />
forward substitution gives z = [1.789 0.632 1.336] T<br />
• Solving lower triangular system La = z by forward<br />
substitution gives a = [0.086 0.400 1.429] T<br />
0.0<br />
0.0<br />
0.0<br />
0.0<br />
1.118<br />
0.0<br />
0.935 <br />
Numerical Methods © Wen-Chieh Lin 52