From Algorithms to Z-Scores - matloff - University of California, Davis

15.12. PARAMETRIC ESTIMATION OF LINEAR REGRESSION FUNCTIONS 299
and
Q =
⎛
⎜
⎝
β =
⎛
⎜
⎝
β1
β2
β3
⎞
1 H1 A1
1 H2 A2
...
1 H1033 A1033
⎟
⎠ (15.28)
⎞
⎟
⎠
(15.29)
Then it can be shown that, after all the partial derivatives are taken and set to 0, the solution is
ˆβ = (Q ′ Q) −1 Q ′ V (15.30)
For the general case (15.24) with n observations (n = 1033 in the baseball data), the matrix Q has
n rows and r+1 columns. Column i+1 has the sample data on predictor variable i.
Keep in mind that all of this is conditional on the X (i)
j , i.e. conditional on Q. As seen for example
in (15.1), our assumption is that
E(V |Q) = Qβ (15.31)
This is the standard approach, especially since there is the case of nonrandom X. Thus we will later
get conditional confidence intervals, which is fine. To avoid clutter, I will sometimes not show the
conditioning explicitly, and thus for instance will write, for example, Cov(V) instead of Cov(V|Q).
It turns out that ˆ β is an unbiased estimate of β: 7
E ˆ β = E[(Q ′ Q) −1 Q ′ V ] (15.30) (15.32)
= (Q ′ Q) −1 Q ′ EV (linearity of E()) (15.33)
= (Q ′ Q) −1 Q ′ · Qβ (15.31) (15.34)
= β (15.35)
In some applications, we assume there is no constant term β0 in (15.24). This means that our Q
matrix no longer has the column of 1s on the left end, but everything else above is valid.
7 Note that here we are taking the expected value of a vector. This is covered in Chapter 8.