LAD Estimation for Time Series Models With Finite and Infinite ...

LAD Estimation for Time Series Models
With Finite and Infinite Variance
Richard A. Davis
Colorado State University
William Dunsmuir
University of New South Wales
NBER/NSF Workshop
1

LAD Estimation for ARMA Models
finite variance
infinite variance
A Real Data Example
LAD estimation
unit root problem
NBER/NSF Workshop
2

Αsymptotics in Non-standard Settings
(When Taylor series expansions do not work.)
Applications to:
• LAD estimation
(Pollard `91, Davis and Dunsmuir `95)
• M-estimation with infinite variance
(Davis, Knight and Liu `92, Davis `95)
• Unit root problems (AR + MA)
(Davis and Dunsmuir `95, Davis, Chen and Dunsmuir
`95)
NBER/NSF Workshop
3

Εxample (The median).
Data. Z 1 , . . . , Z n , IID median 0 and pdf f(0) > 0.
Median. m n = median(Z 1 , . . . , Z n )
Asymptotics:
Note: m^
n
^
(assume m 0 = population median = 0)
^
m n is AN(0,??)
minimizes
n
T n (m) =
Σ
(|Z t − m|− | Z t |)
t=1
NBER/NSF Workshop
4

Set
u=n 1/2 m and put
Then
S n (u ) = T n (un -1/2 )
^
u n = n 1/2 m n .
Using the key identity,
we have
Σt = 1
NBER/NSF Workshop
n
Σ t=1
= (|Z t − un -1/2 | − |Z t |).
|z-y| - |z| = -y sgn(z) + 2(y-z) (I(0

S n (u) = −u n -1/2 sgn(Z t )
n
Σ
t=1
n
Σt=1
+ 2 (n -1/2 u-Z t ){ I(0< Z t

Conclude :
• A n
• B n
d
P
-uN
u 2 f(0)
• S n (u) S(u):= −uN + u 2 f(0) [on C( R) ]
^
• u n = n 1/2 ^
m n
u := minimizer of S(u).
Solve S'(u) = −N + 2 u f(0) = 0, we obtain
^
d
d
^
u = N/(2f(0)) ~ N(0,1/(4f 2 (0)))
or
^
d
n 1/2 m n N(0,1/(4f 2 (0)))
NBER/NSF Workshop
7

The Paradigm:
• Objective function to be minimized: T n (θ)
• Reparameterize by setting u = a n (θ−θ 0 )
θ 0 = true value, a n = scaling
• Form new objective function:
S n (u) = T n (θ 0 +u/a n )
• Establish weak convergence of S n (u) to S(u) on C(R) R).
• Show
u^
n = a n (θ−θ
^ d
0 ) u:= ^ argmin S(u)
NBER/NSF Workshop
8

8
Theorem. Let {Y t } be the linear process
Y t = c j Z t-j , |c j | < ,
where {Z t }~IID(0,σ 2 ), median(Z 1 )=0, and f(0)>0 .
Then
S n := (| Z t -n -1/2 Y t-1 | - |Z t | )
d
∞
∞
∑ ∑
j= 0 j= 0
n
Σ t=1
γ f(0) + N,
where N ~ N(0, γ) (γ = Var(Y t )).
Key identity:
|z-y| - |z| = -y sgn(z) + 2(y-z) (I(0

Using the key identity,
|z-y| - |z| = -y sgn(z) + 2(y-z) (I(0

Results :
• A
d
n −uN 1 − v N 2 , (N 1 ,N 2 ) T ~ N( ⎢ ⎤ ).
⎦ ⎥ ,
⎡
⎣
0 1 EX 1
0 EX 1
EX 1
2
• EB n
• B n
P
EY 1
2
f(0) = (u,v) Σ (u,v) T f(0).
(u,v) Σ (u,v) T f(0).
Conclude :
• S n (u,v)
d
−uN 1 − v N 2 + (u,v) Σ (u,v) T f(0).
• (u ^
n ,v^
n ) T d Σ -1 (N 1 ,N 2 ) T / 2f(0) ~ N(0, Σ -1 / 4f 2 (0))
^
^
• (n 1/2 (µ−µ 0 ), n 1/2 (φ−φ 0 )) T d
N(0, Σ -1 / 4f 2 (0))
NBER/NSF Workshop
11

Εxample (AR(1)).
Data. X 1 , . . . , X n
Model. X t = µ 0 + φ 0 X t-1 +Z t ,
LAD estimation:
Minimize
| φ 0 | < 1, {Z t } ~ IID(0,σ 2 ), f(0)>0.
n
Σ t=1
n
T n (µ,φ) = (|X t −µ − φX t-1 | − |Z t |)
Σ t=1
= (|Z t − (µ−µ 0 ) −(φ −φ 0 )X t-1 | − |Z t |)
Set u=n 1/2 (µ−µ 0 ), v= n 1/2 (φ−φ 0 ),
S n (u,v) = (|Z t − un -1/2 − vn -1/2 X t-1 | −|Z t |)
NBER/NSF Workshop
n
Σ
t=1
12

S n (u,v) = (|Z t − un -1/2 − vn -1/2 X t-1 | −|Z t |)
n
Σ
t=1
n
Σ
t=1
n
Σ
t=1
= (|Z t − n -1/2 Y t-1 | − |Z t |)
(n -1/2 Y t-1 = un -1/2 + vn -1/2 X t-1 )
n
Σ
t=1
= - n -1/2 Y t-1 sgn(Z t )
+ 2 (n -1/2 Y t-1 -Z t ){ I(0< Z t

Results :
• A
d
n −uN 1 − v N 2 , (N 1 ,N 2 ) T ~ N( ⎢ ⎤ ).
⎦ ⎥ ,
⎡
⎣
0 1 EX 1
0 EX 1
EX 1
2
• EB n
• B n
P
EY 1
2
f(0) = (u,v) Σ (u,v) T f(0).
(u,v) Σ (u,v) T f(0).
Conclude :
• S n (u,v)
d
−uN 1 − v N 2 + (u,v) Σ (u,v) T f(0).
• (u ^
n ,v^
n ) T d Σ -1 (N 1 ,N 2 ) T / 2f(0) ~ N(0, Σ -1 / 4f 2 (0))
^
^
• (n 1/2 (µ−µ 0 ), n 1/2 (φ−φ 0 )) T d
N(0, Σ -1 / 4f 2 (0))
NBER/NSF Workshop
14

Εxample (MA(1)).
Data. X 1 , . . . , X n
Model. X t = Z t + θ Z t-1 ,
LAD estimation:
| θ 0 | < 1, {Z t } ~ IID(0,σ 2 ), f(0)>0.
Minimize
T n (θ) = (|Z t (θ)| - |Z t (θ 0 )| )
n
Σ t=1
n
Σ t=1
Set u=n 1/2 (θ−θ 0 ),
= (|X t −θX t-1 +θ 2 X t-2 - . . . (-θ) t-1 X 1 | - |Z t (θ 0 )| )
NBER/NSF Workshop
15

S n (u) = T n (θ 0 + un -1/2 )
n
Σ
t=1
= (|Z t (θ 0 + un -1/2 )| - |Z t (θ 0 )| )
(Not a convex function of u!)
Linearize Z t (θ 0 + un -1/2 ) to get
n
Σ
t=1
S n (u) ∼ (|Z t (θ 0 ) + un -1/2 Z t ' (θ 0 )| -|Z t (θ 0 )| )
where -Z t ' (θ 0 ) is the AR(1) process
Y t = −θ 0 Y t-1 +Z t .
Result : Same limit result as in the AR(1) case, i.e.
n 1/2 ^
(θ LAD −θ 0 ) d N / (Var(Y t )2f(0))
NBER/NSF Workshop
~ N(0, (1-θ 2 ) / (σ 2 4f 2 (0)))
16

Linearized Version :
Initial estimate :
^
θ 0 = θ 0 + O p (n -1/2 )
Objective Function:
Then
n
Σ
t=1
^ ^ ^ ^
T n (θ) = (|Z t (θ 0 ) + Z' t (θ 0 )(θ−θ 0 )| - |Z t (θ 0 )| ).
n 1/2 ^
(θ L −θ 0 ) d N(0, (1-θ 2 ) / (4f 2 (0))),
^
where θ L = argmin T n (θ)
NBER/NSF Workshop
17

Extensions :
ARMA Model : φ(B)X t = θ(B)Z t , {Z t } ~ IID(0,σ 2 ), f(0)>0.
Set β = (φ 1 , . . ., φ p , θ 1 , . . ., θ q ) T and
Then
v = n 1/2 (β − β 0 )
(i) S n (v ) := (|Z t (β 0 + vn -1/2 )| - |Z t (β 0 )| )
d
n
Σ
t=1
NBER/NSF Workshop
f(0)v T Γ Q
-1/2
v + v T N , N~N( 0, Γ Q ), (in C(R p+q ))
(ii) v
^
LAD = n 1/2 ^
(β LAD −β 0 )
d
Γ -1 Q N/(2f(0))
~ N(0, Γ Q -1 /(4f 2 (0)))
Note: Γ Q -1 σ 2 is the limiting covariance matrix in Gaussian case.
18

ARMA Model With Stable Noise:
(Davis, Knight and Liu `92 for AR case, Davis `95 for ARMA.)
Model:
φ(B)X t = θ(B)Z t , {Z t } ~ IID symmetric stable(α), 0

Linear Regression with ARMA Errors :
Model : Y t = A T t α + X t ,
where {X t } follows an ARMA process
φ(B)X t = θ(B)Z t , {Z t } ~ IID(0,σ 2 ), f(0)>0.
Assume A T t = (a 1t , . . . , a rt ) T satisfies Grenander’s conditions:
(a) |a jn |/b jn 0
(b)
(c)
B n
-1
A t A t+j
T
B n
-1
Γ A (j)
b jn
n
Σ
t=1
P
P8
where b jn
2
= ||A t || 2 and B n = diag(b 1t , . . . , b rt ) .
P
NBER/NSF Workshop
20

Estimation: Let
X t (α) =
⎧
⎨
⎩
0, if t < 1,
Y t − A t T α , if t > 0,
and for τ Τ =( α Τ , β Τ ), define
Z t (τ) =
⎧
⎨
⎩ . . .
0, if t < 1,
φ(B)X t (α) − θ 1 Z t-1 (τ)−
−θ q Z t-q (τ)
, if t > 0,
Minimize
n
Σ
t=1
S n ( u, v) = (|Z t (τ 0 + n(u,v))| - |Z t (τ 0 )| )
where n(u,v) = ( (B n -1 u) T , n -1/2 v T ) T .
NBER/NSF Workshop
21

Result :
d
• S n ( u, v) S ( u, v) on C(R p+q+r )
• B n (α
^
LAD −α 0 ), n 1/2 (β
^
LAD −β 0 )
d
Γ −1 N /(2f(0))
~ N(0, Γ -1 /(4f 2 (0))),
where
and
Γ =
⎡
⎢
⎣
Γ C 0
⎤
⎥
⎦
0 T Γ Q
Σ
j, k
Γ C = π j π k Γ A (j-k), π(B) = θ 0 (B) / φ 0 (B) .
NBER/NSF Workshop
22

An Example : (overshorts Y 1 , . . . , Y 57
storage tank.)
from underground
(gallons)
-100 -50 0 50 100
0 10 20 30 40 50
ACF
-0.5 0.0 0.5 1.0
NBER/NSF Workshop
0 5 10 15 20
Lag
23

Model. Y t = µ + Z t +θZ t-1
Problem. Estimate µ and construct a C.I.?
(Is µ < -5 gallons/day?)
Estimation.
Estimates
Asymptotic Var
µ ^ MLE = − 4.87 (1+θ) 2 σ 2 /n = (1.408) 2
θ
^
MLE = −.849 (1−θ 2 )/n = (.070) 2
µ ^ LAD = -6.01 (1+θ) 2 /(4nf 2 (0)) = (2.236) 2
θ
^
LAD = −.673 (1−θ 2 ) / (σ 2 4n(f 2 (0))) = (.106) 2
NBER/NSF Workshop
24

Unit Root. If θ=1, then
µ ^ MLE is AN(µ 0 , 12σ 2 /n 3 ) (if θ is not estimated)
µ ^ MLE is asymptotically non-normal (if θ is estimated)
Asymptotic distribution of µ ^
LAD and θ
^
LAD when θ = 1?
NBER/NSF Workshop
25