A comparison of bootstrap methods and an adjusted bootstrap ...

n> p situation, the .632+ bootstrap is very popular for having low variability and onlymoderate bias. However, the study in this paper and the work of Molinaro et al. [8]suggest that the .632+ bootstrap can run into problems in the n< p situation. Wepropose an adjusted bootstrap method, its performance is robust in various situations andit achieves a good compromise in the bias-variance tradeoff.2. A REVIEW OF METHODS FOR PREDICTION ERROR ESTIMATIONIn microarray class prediction problem, we observe x = ( t , y ) , i = 1,..., n, oni i inindependent subjects wheretiis ap -dimensional vector containing the gene expressionmeasurements andyiis the response for subject i . The observations x1,..., xncan beviewed as realizations of an underlying random variable X = ( TY , ) . With dichotomousoutcome, the response variable Y takes 0 or 1 values distinguishing the two classes. Aprediction rule (model) r(· , xlearn ) is developed based on the information in the learningsetlearnx . The true prediction error ( en= E⎡I{ Y ≠r( T, x)}prediction model built on the observed data x ( x x )following the same random mechanism as X .⎣⎤⎦ ) is the probability that the= ,..., 1 nmisclassifies a future itemWhen the prediction rule is built for the observed data, the prediction accuracy shouldideally be assessed on an independent large test set. But this often is impossible becauseof the relatively small sample sizes in microarray experiments. Methods for estimatingprediction errors rely on partitioning or resampling the observed data to construct thelearning and test sets. With a huge number of features, the prediction rules r( ⋅⋅ , ) containtwo key steps: the feature selection and the class prediction (discrimination) step. Featureselection is administered prior to the class prediction step for every learning set. Failureto include feature selection in resampling steps results in serious downward bias inestimating prediction error and overly optimistic assessment of the prediction rule [10, 11,7]. Methods for class prediction include various versions of discriminant analysis, nearestneighbor classification, classification trees, etc. A comprehensive comparison of the classdiscrimination methods was conducted by Dudoit, Fridlyand and Speed [12]. In this4