Lecture3 Slide - The Department of Statistics and Applied Probability ...
Lecture3 Slide - The Department of Statistics and Applied Probability ...
Lecture3 Slide - The Department of Statistics and Applied Probability ...
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
1<br />
ST5207 Nonparametric Regression, Lecture 3<br />
Lijian Yang<br />
<strong>Department</strong> <strong>of</strong> <strong>Statistics</strong> & <strong>Probability</strong><br />
Michigan State University<br />
East Lansing, MI 48824<br />
<strong>and</strong><br />
<strong>Department</strong> <strong>of</strong> <strong>Statistics</strong> & <strong>Applied</strong> <strong>Probability</strong><br />
National University <strong>of</strong> Singapore<br />
Singapore 117546<br />
ST5207 Nonparametric Regression, 27th January 2005
2<br />
Step one, continued<br />
• E |ξ in | 2 [<br />
= n −2 E |K h (X i − x) − EK h (X i − x)| 2<br />
= n −2 E {K h (X i − x)} 2 − {EK h (X i − x)} 2]<br />
ST5207 Nonparametric Regression, 27th January 2005
3<br />
Step one, continued<br />
• E |ξ in | 2 [<br />
= n −2 E |K h (X i − x) − EK h (X i − x)| 2<br />
= n −2 E {K h (X i − x)} 2 − {EK h (X i − x)} 2]<br />
•<br />
E {K h (X i − x)} 2 =<br />
∫<br />
K 2 h (u − x) f (u) du = 1 h 2 ∫<br />
K 2 ( u − x<br />
h<br />
)<br />
f (u) du<br />
ST5207 Nonparametric Regression, 27th January 2005
4<br />
Step one, continued<br />
• E |ξ in | 2 [<br />
= n −2 E |K h (X i − x) − EK h (X i − x)| 2<br />
= n −2 E {K h (X i − x)} 2 − {EK h (X i − x)} 2]<br />
•<br />
E {K h (X i − x)} 2 =<br />
= 1 h 2 ∫<br />
∫<br />
K 2 h (u − x) f (u) du = 1 h 2 ∫<br />
K 2 (v) f (x + hv) hdv = 1 h<br />
∫<br />
K 2 ( u − x<br />
h<br />
K 2 (v) f (x + hv) dv<br />
)<br />
f (u) du<br />
ST5207 Nonparametric Regression, 27th January 2005
5<br />
Step one, continued<br />
• E |ξ in | 2 [<br />
= n −2 E |K h (X i − x) − EK h (X i − x)| 2<br />
= n −2 E {K h (X i − x)} 2 − {EK h (X i − x)} 2]<br />
•<br />
E {K h (X i − x)} 2 =<br />
= 1 h 2 ∫<br />
= f (x)<br />
h<br />
∫<br />
K 2 h (u − x) f (u) du = 1 h 2 ∫<br />
K 2 (v) f (x + hv) hdv = 1 h<br />
∫<br />
K 2 (v) dv + 1 f (x)<br />
u (h) =<br />
h h<br />
∫<br />
K 2 ( u − x<br />
h<br />
K 2 (v) f (x + hv) dv<br />
∫<br />
K 2 (v) dv + u (1)<br />
)<br />
f (u) du<br />
ST5207 Nonparametric Regression, 27th January 2005
6<br />
Step one, continued<br />
• E |ξ in | 2 [<br />
= n −2 E |K h (X i − x) − EK h (X i − x)| 2<br />
= n −2 E {K h (X i − x)} 2 − {EK h (X i − x)} 2]<br />
•<br />
E {K h (X i − x)} 2 =<br />
•<br />
= 1 h 2 ∫<br />
= f (x)<br />
h<br />
σ 2 in = f (x)<br />
n 2 h<br />
∫<br />
K 2 h (u − x) f (u) du = 1 h 2 ∫<br />
K 2 (v) f (x + hv) hdv = 1 h<br />
∫<br />
∫<br />
K 2 (v) dv + 1 f (x)<br />
u (h) =<br />
h h<br />
K 2 (v) dv {1 + u (1)} , V 2 n = f (x)<br />
nh<br />
∫<br />
K 2 ( u − x<br />
h<br />
K 2 (v) f (x + hv) dv<br />
∫<br />
K 2 (v) dv + u (1)<br />
∫<br />
)<br />
f (u) du<br />
K 2 (v) dv {1 + u (1)}<br />
ST5207 Nonparametric Regression, 27th January 2005
7<br />
Step one, continued<br />
• E |ξ in | 2+δ = n −(2+δ) E |K h (X i − x) − EK h (X i − x)| 2+δ<br />
ST5207 Nonparametric Regression, 27th January 2005
8<br />
Step one, continued<br />
• E |ξ in | 2+δ = n −(2+δ) E |K h (X i − x) − EK h (X i − x)| 2+δ<br />
• E |ξ in | 2+δ ≤ n −(2+δ) E {|K h (X i − x)| + |EK h (X i − x)|} 2+δ<br />
ST5207 Nonparametric Regression, 27th January 2005
9<br />
Step one, continued<br />
• E |ξ in | 2+δ = n −(2+δ) E |K h (X i − x) − EK h (X i − x)| 2+δ<br />
• E |ξ in | 2+δ ≤ n −(2+δ) E {|K h (X i − x)| + |EK h (X i − x)|} 2+δ<br />
• Using (a + b) p ≤ 2 p−1 (a p + b p ) , a, b > 0, p ≥ 1<br />
ST5207 Nonparametric Regression, 27th January 2005
10<br />
Step one, continued<br />
• E |ξ in | 2+δ = n −(2+δ) E |K h (X i − x) − EK h (X i − x)| 2+δ<br />
• E |ξ in | 2+δ ≤ n −(2+δ) E {|K h (X i − x)| + |EK h (X i − x)|} 2+δ<br />
• Using (a + b) p ≤ 2 p−1 (a p + b p ) , a, b > 0, p ≥ 1<br />
E |ξ in | 2+δ ≤ n −(2+δ) 2 1+δ E<br />
{|K h (X i − x)| 2+δ + |EK h (X i − x)| 2+δ}<br />
ST5207 Nonparametric Regression, 27th January 2005
11<br />
Step one, continued<br />
• E |ξ in | 2+δ = n −(2+δ) E |K h (X i − x) − EK h (X i − x)| 2+δ<br />
• E |ξ in | 2+δ ≤ n −(2+δ) E {|K h (X i − x)| + |EK h (X i − x)|} 2+δ<br />
• Using (a + b) p ≤ 2 p−1 (a p + b p ) , a, b > 0, p ≥ 1<br />
E |ξ in | 2+δ ≤ n −(2+δ) 2 1+δ E<br />
{|K h (X i − x)| 2+δ + |EK h (X i − x)| 2+δ}<br />
•<br />
E |K h (X i − x)| 2+δ =<br />
∫<br />
K 2+δ<br />
h<br />
(u − x) f (u) du<br />
ST5207 Nonparametric Regression, 27th January 2005
12<br />
Step one, continued<br />
• E |ξ in | 2+δ = n −(2+δ) E |K h (X i − x) − EK h (X i − x)| 2+δ<br />
• E |ξ in | 2+δ ≤ n −(2+δ) E {|K h (X i − x)| + |EK h (X i − x)|} 2+δ<br />
• Using (a + b) p ≤ 2 p−1 (a p + b p ) , a, b > 0, p ≥ 1<br />
E |ξ in | 2+δ ≤ n −(2+δ) 2 1+δ E<br />
{|K h (X i − x)| 2+δ + |EK h (X i − x)| 2+δ}<br />
•<br />
E |K h (X i − x)| 2+δ =<br />
= 1<br />
h 2+δ ∫<br />
∫<br />
K 2+δ<br />
h<br />
K 2+δ ( u − x<br />
h<br />
(u − x) f (u) du<br />
)<br />
f (u) du<br />
ST5207 Nonparametric Regression, 27th January 2005
13<br />
Step one, continued<br />
• E |ξ in | 2+δ = n −(2+δ) E |K h (X i − x) − EK h (X i − x)| 2+δ<br />
• E |ξ in | 2+δ ≤ n −(2+δ) E {|K h (X i − x)| + |EK h (X i − x)|} 2+δ<br />
• Using (a + b) p ≤ 2 p−1 (a p + b p ) , a, b > 0, p ≥ 1<br />
E |ξ in | 2+δ ≤ n −(2+δ) 2 1+δ E<br />
{|K h (X i − x)| 2+δ + |EK h (X i − x)| 2+δ}<br />
•<br />
E |K h (X i − x)| 2+δ =<br />
= 1<br />
h 2+δ ∫<br />
= 1<br />
h 2+δ ∫<br />
∫<br />
K 2+δ<br />
h<br />
K 2+δ ( u − x<br />
h<br />
(u − x) f (u) du<br />
)<br />
f (u) du<br />
K 2+δ (v) f (x + hv) hdv<br />
ST5207 Nonparametric Regression, 27th January 2005
14<br />
Step one, continued<br />
• E |ξ in | 2+δ = n −(2+δ) E |K h (X i − x) − EK h (X i − x)| 2+δ<br />
• E |ξ in | 2+δ ≤ n −(2+δ) E {|K h (X i − x)| + |EK h (X i − x)|} 2+δ<br />
• Using (a + b) p ≤ 2 p−1 (a p + b p ) , a, b > 0, p ≥ 1<br />
E |ξ in | 2+δ ≤ n −(2+δ) 2 1+δ E<br />
{|K h (X i − x)| 2+δ + |EK h (X i − x)| 2+δ}<br />
•<br />
= 1<br />
h 2+δ ∫<br />
E |K h (X i − x)| 2+δ =<br />
= 1<br />
h 2+δ ∫<br />
∫<br />
K 2+δ<br />
h<br />
K 2+δ ( u − x<br />
h<br />
K 2+δ (v) f (x + hv) hdv = 1<br />
h 1+δ ∫<br />
(u − x) f (u) du<br />
)<br />
f (u) du<br />
K 2+δ (v) f (x + hv) dv<br />
ST5207 Nonparametric Regression, 27th January 2005
15<br />
Step one, continued<br />
• E |K h (X i − x)| 2+δ ≤ 1<br />
h 1+δ<br />
max f (t) ∫ K 2+δ (v) dv<br />
t∈[a,b]<br />
ST5207 Nonparametric Regression, 27th January 2005
16<br />
Step one, continued<br />
• E |K h (X i − x)| 2+δ ≤ 1<br />
h 1+δ<br />
|EK h (X i − x)| 2+δ =<br />
max f (t) ∫ K 2+δ (v) dv<br />
t∈[a,b]<br />
{ }<br />
∣<br />
∣Bias ˆf (x)<br />
+ f (x) ∣ 2+δ = f (x) 2+δ +u ( h 2)<br />
ST5207 Nonparametric Regression, 27th January 2005
17<br />
Step one, continued<br />
• E |K h (X i − x)| 2+δ ≤ 1<br />
h 1+δ<br />
|EK h (X i − x)| 2+δ =<br />
max f (t) ∫ K 2+δ (v) dv<br />
t∈[a,b]<br />
{ }<br />
∣<br />
∣Bias ˆf (x)<br />
+ f (x) ∣ 2+δ = f (x) 2+δ +u ( h 2)<br />
E |ξ in | 2+δ ≤ n −(2+δ) 2 1+δ 1<br />
h 1+δ<br />
∫<br />
max f (t)<br />
t∈[a,b]<br />
K 2+δ (v) dv {1 + u (1)}<br />
ST5207 Nonparametric Regression, 27th January 2005
18<br />
Step one, continued<br />
• E |K h (X i − x)| 2+δ ≤ 1<br />
h 1+δ<br />
|EK h (X i − x)| 2+δ =<br />
max f (t) ∫ K 2+δ (v) dv<br />
t∈[a,b]<br />
{ }<br />
∣<br />
∣Bias ˆf (x)<br />
+ f (x) ∣ 2+δ = f (x) 2+δ +u ( h 2)<br />
E |ξ in | 2+δ ≤ n −(2+δ) 2 1+δ 1<br />
h 1+δ<br />
∫<br />
max f (t)<br />
t∈[a,b]<br />
K 2+δ (v) dv {1 + u (1)}<br />
•<br />
V −(2+δ)<br />
n<br />
n∑<br />
i=1<br />
E |ξ in | 2+δ = V −(2+δ)<br />
n nE |ξ in | 2+δ<br />
ST5207 Nonparametric Regression, 27th January 2005
19<br />
Step one, continued<br />
• E |K h (X i − x)| 2+δ ≤ 1<br />
h 1+δ<br />
|EK h (X i − x)| 2+δ =<br />
max f (t) ∫ K 2+δ (v) dv<br />
t∈[a,b]<br />
{ }<br />
∣<br />
∣Bias ˆf (x)<br />
+ f (x) ∣ 2+δ = f (x) 2+δ +u ( h 2)<br />
E |ξ in | 2+δ ≤ n −(2+δ) 2 1+δ 1<br />
h 1+δ<br />
∫<br />
max f (t)<br />
t∈[a,b]<br />
K 2+δ (v) dv {1 + u (1)}<br />
•<br />
=<br />
V −(2+δ)<br />
n<br />
[ f (x)<br />
nh<br />
∫<br />
n∑<br />
i=1<br />
E |ξ in | 2+δ = V −(2+δ)<br />
n nE |ξ in | 2+δ<br />
] −(2+δ)/2<br />
K 2 (v) dv {1 + u (1)} nE |ξ in | 2+δ<br />
ST5207 Nonparametric Regression, 27th January 2005
20<br />
Step one, continued<br />
•<br />
≤<br />
[ f (x)<br />
nh<br />
∫<br />
K 2 (v) dv {1 + u (1)}] −(2+δ)/2<br />
× n×<br />
ST5207 Nonparametric Regression, 27th January 2005
21<br />
Step one, continued<br />
•<br />
≤<br />
[ f (x)<br />
nh<br />
∫<br />
n −(2+δ) 2 1+δ 1<br />
h 1+δ<br />
K 2 (v) dv {1 + u (1)}] −(2+δ)/2<br />
× n×<br />
∫<br />
max f (t)<br />
t∈[a,b]<br />
K 2+δ (v) dv {1 + u (1)}<br />
ST5207 Nonparametric Regression, 27th January 2005
22<br />
Step one, continued<br />
•<br />
≤<br />
[ f (x)<br />
nh<br />
∫<br />
n −(2+δ) 2 1+δ 1<br />
h 1+δ<br />
K 2 (v) dv {1 + u (1)}] −(2+δ)/2<br />
× n×<br />
∫<br />
max f (t)<br />
t∈[a,b]<br />
K 2+δ (v) dv {1 + u (1)}<br />
≤ Cn (2+δ)/2+1−(2+δ) h (2+δ)/2−(1+δ) = Cn −δ/2 h −δ/2 = C (nh) −δ/2<br />
ST5207 Nonparametric Regression, 27th January 2005
23<br />
Step one, continued<br />
•<br />
≤<br />
[ f (x)<br />
nh<br />
∫<br />
n −(2+δ) 2 1+δ 1<br />
h 1+δ<br />
K 2 (v) dv {1 + u (1)}] −(2+δ)/2<br />
× n×<br />
∫<br />
max f (t)<br />
t∈[a,b]<br />
K 2+δ (v) dv {1 + u (1)}<br />
≤ Cn (2+δ)/2+1−(2+δ) h (2+δ)/2−(1+δ) = Cn −δ/2 h −δ/2 = C (nh) −δ/2<br />
• Lyapunov CLT shows<br />
V −1<br />
n<br />
n∑<br />
i=1<br />
ξ in<br />
D<br />
→ N (0, 1) ,<br />
{ f (x)<br />
nh<br />
∫<br />
} −1/2 ∑ n<br />
K 2 (v) dv<br />
i=1<br />
ξ in<br />
D<br />
→ N (0, 1)<br />
ST5207 Nonparametric Regression, 27th January 2005
24<br />
Step one, continued<br />
•<br />
≤<br />
[ f (x)<br />
nh<br />
∫<br />
n −(2+δ) 2 1+δ 1<br />
h 1+δ<br />
K 2 (v) dv {1 + u (1)}] −(2+δ)/2<br />
× n×<br />
∫<br />
max f (t)<br />
t∈[a,b]<br />
K 2+δ (v) dv {1 + u (1)}<br />
≤ Cn (2+δ)/2+1−(2+δ) h (2+δ)/2−(1+δ) = Cn −δ/2 h −δ/2 = C (nh) −δ/2<br />
• Lyapunov CLT shows<br />
V −1<br />
n<br />
n∑<br />
i=1<br />
{ f (x)<br />
nh<br />
∫<br />
ξ in<br />
D<br />
→ N (0, 1) ,<br />
{ f (x)<br />
nh<br />
∫<br />
} −1/2 ∑ n<br />
K 2 (v) dv<br />
i=1<br />
ξ in<br />
D<br />
→ N (0, 1)<br />
K 2 (v) dv} −1/2 [<br />
ˆf (x) − f (x) − Bias<br />
{<br />
ˆf (x)<br />
}] D→ N (0, 1)<br />
ST5207 Nonparametric Regression, 27th January 2005
25<br />
Digression to density estimation<br />
• [<br />
ˆf (x) − f (x) −<br />
f (2) (x)<br />
2!<br />
h ∫ ]<br />
2 v 2 K (v) dv<br />
D<br />
∫ } 1/2 → N (0, 1)<br />
K2 (v) dv<br />
because<br />
{ ∫ f (x)<br />
nh<br />
{<br />
f(x)<br />
nh<br />
K 2 (v) dv} −1/2<br />
u ( h 2) = u<br />
(<br />
h 2 n 1/2 h 1/2) (<br />
= u n 1/2 h 5/2) = u (1)<br />
ST5207 Nonparametric Regression, 27th January 2005
26<br />
Digression to density estimation<br />
• [<br />
ˆf (x) − f (x) −<br />
f (2) (x)<br />
2!<br />
h ∫ ]<br />
2 v 2 K (v) dv<br />
D<br />
∫ } 1/2 → N (0, 1)<br />
K2 (v) dv<br />
because<br />
{ ∫ f (x)<br />
nh<br />
{<br />
f(x)<br />
nh<br />
K 2 (v) dv} −1/2<br />
u ( h 2) = u<br />
(<br />
h 2 n 1/2 h 1/2) (<br />
= u n 1/2 h 5/2) = u (1)<br />
• Confidence interval for probability density function f (x) is<br />
constructed as<br />
ˆf (x) − f ′′ ∫<br />
{ ∫<br />
1/2<br />
(x)<br />
f (x)<br />
h 2 v 2 K (v) dv ± K 2 (v) dv}<br />
z 1−α/2<br />
2<br />
nh<br />
ST5207 Nonparametric Regression, 27th January 2005
27<br />
Digression to density estimation<br />
• [<br />
ˆf (x) − f (x) −<br />
f (2) (x)<br />
2!<br />
h ∫ ]<br />
2 v 2 K (v) dv<br />
D<br />
∫ } 1/2 → N (0, 1)<br />
K2 (v) dv<br />
because<br />
{ ∫ f (x)<br />
nh<br />
{<br />
f(x)<br />
nh<br />
K 2 (v) dv} −1/2<br />
u ( h 2) = u<br />
(<br />
h 2 n 1/2 h 1/2) (<br />
= u n 1/2 h 5/2) = u (1)<br />
• Confidence interval for probability density function f (x) is<br />
constructed as<br />
ˆf (x) − f ′′ ∫<br />
{ ∫<br />
1/2<br />
(x)<br />
f (x)<br />
h 2 v 2 K (v) dv ± K 2 (v) dv}<br />
z 1−α/2<br />
2<br />
nh<br />
• XploRe quantlets are denxci, denxest, we now explore their use<br />
ST5207 Nonparametric Regression, 27th January 2005
28<br />
Digression to density estimation<br />
From ˆf (x) − f (x) = n ∑<br />
i=1<br />
ξ in + Bias<br />
{<br />
ˆf (x)<br />
}<br />
, one gets<br />
E<br />
{<br />
{ } 2 n<br />
} 2<br />
∑<br />
{ }<br />
ˆf (x) − f (x) = E ξ in +Bias 2 ˆf (x)<br />
i=1<br />
= V 2 n +Bias 2 { ˆf (x)<br />
}<br />
ST5207 Nonparametric Regression, 27th January 2005
29<br />
Digression to density estimation<br />
From ˆf (x) − f (x) = n ∑<br />
i=1<br />
ξ in + Bias<br />
{<br />
ˆf (x)<br />
}<br />
, one gets<br />
E<br />
{<br />
{ } 2 n<br />
} 2<br />
∑<br />
{ }<br />
ˆf (x) − f (x) = E ξ in +Bias 2 ˆf (x)<br />
i=1<br />
= V 2 n +Bias 2 { ˆf (x)<br />
}<br />
E<br />
{ } { 2 f (x)<br />
f ˆf (2) (x) − f (x) =<br />
nh c (x)<br />
K {1 + u (1)}+ h 2 d K + u ( h 2)} 2<br />
2!<br />
ST5207 Nonparametric Regression, 27th January 2005
30<br />
Digression to density estimation<br />
From ˆf (x) − f (x) = n ∑<br />
i=1<br />
ξ in + Bias<br />
{<br />
ˆf (x)<br />
}<br />
, one gets<br />
E<br />
{<br />
{ } 2 n<br />
} 2<br />
∑<br />
{ }<br />
ˆf (x) − f (x) = E ξ in +Bias 2 ˆf (x)<br />
i=1<br />
= V 2 n +Bias 2 { ˆf (x)<br />
}<br />
E<br />
{ } { 2 f (x)<br />
f ˆf (2) (x) − f (x) =<br />
nh c (x)<br />
K {1 + u (1)}+ h 2 d K + u ( h 2)} 2<br />
2!<br />
E<br />
{ } 2 f (x)<br />
ˆf (x) − f (x) =<br />
nh c K + f (2) (x) 2<br />
h 4 d 2 K + u<br />
4<br />
( 1<br />
nh + h4 )<br />
ST5207 Nonparametric Regression, 27th January 2005
31<br />
Digression to density estimation<br />
From ˆf (x) − f (x) = n ∑<br />
i=1<br />
ξ in + Bias<br />
{<br />
ˆf (x)<br />
}<br />
, one gets<br />
E<br />
{<br />
{ } 2 n<br />
} 2<br />
∑<br />
{ }<br />
ˆf (x) − f (x) = E ξ in +Bias 2 ˆf (x)<br />
i=1<br />
= V 2 n +Bias 2 { ˆf (x)<br />
}<br />
E<br />
∫<br />
{ } { 2 f (x)<br />
f ˆf (2) (x) − f (x) =<br />
nh c (x)<br />
K {1 + u (1)}+ h 2 d K + u ( h 2)} 2<br />
2!<br />
( 1<br />
nh + h4 )<br />
{ } 2 f (x)<br />
E ˆf (x) − f (x) =<br />
nh c K + f (2) (x) 2<br />
h 4 d 2 K + u<br />
4<br />
{ } ∫ ∫<br />
2 f (x) dx f<br />
E ˆf ′′ (x) 2 dx<br />
(x) − f (x) dx = c K +<br />
h 4 d 2<br />
nh<br />
4<br />
K+o<br />
( 1<br />
nh + h4 )<br />
ST5207 Nonparametric Regression, 27th January 2005
Digression to density estimation<br />
From ˆf (x) − f (x) = n ∑<br />
i=1<br />
ξ in + Bias<br />
i=1<br />
{<br />
ˆf (x)<br />
}<br />
, one gets<br />
{<br />
{ } 2 n<br />
} 2<br />
∑<br />
{ }<br />
E ˆf (x) − f (x) = E ξ in +Bias 2 ˆf { }<br />
(x) = Vn 2 +Bias 2 ˆf (x)<br />
E<br />
∫<br />
{ } { 2 f (x)<br />
f ˆf (2) (x) − f (x) =<br />
nh c (x)<br />
K {1 + u (1)}+ h 2 d K + u ( h 2)} 2<br />
2!<br />
{ } 2 f (x)<br />
E ˆf (x) − f (x) =<br />
nh c K + f (2) (x) 2 ( ) 1<br />
h 4 d 2 K + u<br />
4<br />
nh + h4<br />
{ } ∫ ∫<br />
2 f (x) dx f<br />
E ˆf ′′ (x) 2 ( )<br />
dx<br />
1<br />
(x) − f (x) dx = c K +<br />
h 4 d 2<br />
nh<br />
4<br />
K+o<br />
nh + h4<br />
{ } { } ( ) 1<br />
MISE ˆf (x) ; h = AMISE ˆf (x) ; h + o<br />
nh + h4<br />
32<br />
ST5207 Nonparametric Regression, 27th January 2005
33<br />
Digression to density estimation<br />
•<br />
{ }<br />
AMISE ˆf (x) ; h<br />
= 1<br />
nh c K +<br />
∫<br />
f ′′ (x) 2 dx<br />
h 4 d 2 K<br />
4<br />
ST5207 Nonparametric Regression, 27th January 2005
34<br />
Digression to density estimation<br />
•<br />
{ }<br />
AMISE ˆf (x) ; h<br />
= 1<br />
nh c K +<br />
∫<br />
f ′′ (x) 2 dx<br />
h 4 d 2 K<br />
4<br />
• To find the optimal h, solve the following equation<br />
d<br />
{ }<br />
dh AMISE ˆf (x) ; h = − 1 ∫<br />
nh 2 c K + f ′′ (x) 2 dxh 3 d 2 K = 0<br />
ST5207 Nonparametric Regression, 27th January 2005
35<br />
Digression to density estimation<br />
•<br />
{ }<br />
AMISE ˆf (x) ; h<br />
= 1<br />
nh c K +<br />
∫<br />
f ′′ (x) 2 dx<br />
h 4 d 2 K<br />
4<br />
• To find the optimal h, solve the following equation<br />
d<br />
{ }<br />
dh AMISE ˆf (x) ; h = − 1 ∫<br />
nh 2 c K + f ′′ (x) 2 dxh 3 d 2 K = 0<br />
• <strong>The</strong> optimal b<strong>and</strong>width is<br />
h opt =<br />
{<br />
c K<br />
d 2 K<br />
} 1/5<br />
1<br />
∫<br />
f<br />
′′<br />
(x) 2 n −1/5<br />
dx<br />
ST5207 Nonparametric Regression, 27th January 2005
36<br />
Digression to density estimation<br />
•<br />
{ }<br />
AMISE ˆf (x) ; h = 1<br />
∫<br />
f ′′<br />
nh c (x) 2 dx<br />
K +<br />
h 4 d 2 K<br />
4<br />
• To find the optimal h, solve the following equation<br />
d<br />
{ }<br />
dh AMISE ˆf (x) ; h = − 1 ∫<br />
nh 2 c K + f ′′ (x) 2 dxh 3 d 2 K = 0<br />
• <strong>The</strong> optimal b<strong>and</strong>width is<br />
{<br />
h opt =<br />
c K<br />
d 2 K<br />
} 1/5<br />
1<br />
∫<br />
f<br />
′′<br />
(x) 2 n −1/5<br />
dx<br />
• <strong>The</strong> optimal b<strong>and</strong>width depends on curvature <strong>of</strong> the unknown<br />
density f, kernel K used <strong>and</strong> sample size n available. XploRe<br />
quantlets are denrot <strong>and</strong> denbwsel, we now explore their use<br />
ST5207 Nonparametric Regression, 27th January 2005