Theory of Statistics - George Mason University

8.3 Nonparametric Estimation of Functions 563
Another approach to function estimation is to represent the function of
interest as a linear combination of basis functions, that is, to represent the
function in a series expansion. The basis functions are generally chosen to be
orthogonal over the domain of interest, and the observed data are used to
estimate the coefficients in the series.
It is often more practical to estimate the function value at a given point.
(Of course, if we can estimate the function at any given point, we can effectively
have an estimate at all points.) One way of forming an estimate of a
function at a given point is to take the average at that point of a filtering
function that is evaluated in the vicinity of each data point. The filtering
function is called a kernel, and the result of this approach is called a kernel
estimator.
In the estimation of functions, we must be concerned about the properties
of the estimators at specific points and also about properties over the full
domain. Global properties over the full domain are often defined in terms of
integrals or in terms of suprema or infima.
Function Decomposition and Estimation of the Coefficients in an
Orthogonal Expansion
We first do a PDF decomposition of the function of interest with the probability
density function, p,
f(x) = g(x)p(x). (8.3)
We have
ck = 〈f, qk〉
= qk(x)g(x)p(x)dx
D
= E(qk(X)g(X)), (8.4)
where X is a random variable whose probability density function is p.
If we can obtain a random sample, x1, . . ., xn, from the distribution with
density p, the ck can be unbiasedly estimated by
ck = 1
n
n
qk(xi)g(xi).
i=1
The series estimator of the function for all x therefore is
f(x) = 1
n
j
k=0 i=1
n
qk(xi)g(xi)qk(x) (8.5)
for some truncation point j.
The random sample, x1, . . ., xn, may be an observed dataset, or it may be
the output of a random number generator.
Theory of Statistics c○2000–2013 James E. Gentle