New Statistical Algorithms for the Analysis of Mass - FU Berlin, FB MI ...

76 CHAPTER 4. (BIO-)MEDICAL APPLICATIONS
Bootstrap (Efron, 1979): The bootstrap is a method for estimating the
variance and the distribution of a statistic Tn = g(X1, . . . , Xn) (note that
Tn needs to be Hadamard Differentiable, see e.g. (Shao and Tu, 1995)). In
principle it can also be used to estimate some parameter θ. This method first
creates an infinitely large mega data set by copying the original data set many
time. Then a large number of different samples are drawn from this mega set
and analyses are performed separately for each sample and the results averaged.
Thus, a lot of configurations (including configurations in which an item
may be represented several times or not at all) are considered and conclusion
about generalization of the results can be drawn. It is a robust alternative
to inference based on parametric assumptions when those assumptions are in
doubt, or where parametric inference is impossible. Opposed to jackknife the
bootstrap gives slightly different results when repeated on the same data.
In the real world we would sample n data points (X1 . . . , Xn) from some
CDF F and calculate a statistic Tn = g(X1 . . . , Xn). Transferred to the bootstrap
world, we sample n data points (X ∗ 1 . . . , X∗ n) from ˆ Fn and estimate a
statistic T ∗ n = g(X ∗ 1 . . . , X∗ n). Drawing n points at random from ˆ Fn is the
same as drawing a sample of size n with replacement from (X1 . . . , Xn) (the
original data). By the law of large numbers we know that vboot a.s.
−→VFn ˆ (Tn) as
B → ∞. It follows
VF (Tn)
O(1/ √ n)
����
≈ VFn ˆ (Tn)
O(1/ √ B)
����
≈ vboot
For the parameter estimation, the number of the bootstrap samples B is
usually chosen to be around 200. The algorithm for estimating the variance
of some statistic Tn is as follows:
� Given data: X = (X1, . . . , Xn)
� Repeat the following two steps i = 1 . . . B times
1. Draw X ∗ = (X ∗ 1 , . . . , X∗ n) with replacement from X
2. Calculate T ∗ n,i = g(X∗ 1 , . . . X∗ n)
� This results in B estimators (T ∗ n,1 , . . . , T ∗ n,B ) and can be used for various
purposes (for variance estimation, for interval estimation, hypothesis
testing and so on).
For example the variance estimator is computed by:
vboot = 1
B ·
B�
b=1
�
T ∗ 1
n,b −
B ·
and the estimator for the standard error by:
Jackknife vs. Bootstrap
ˆseboot = √ vboot
Since the jackknife only needs n computations it is usually easier computable
compared to about 200-300 replications needed for the bootstrap. However,
only using the n jackknife samples, the jackknife uses only limited information
about the statistic ˆ θ. It can be shown that asymptotically the estimators of
B�
i=1
T ∗ n,i
� 2