2DkcTXceO

R.J. Tibshirani 501CancerEpithelialStromalSpectrum for each pixelSpectrum sampled at 11,000 m/z valuesFIGURE 42.2Schematic of the cancer diagnosis problem. Each pixel in each of the threeregions labelled by the pathologist is analyzed by mass spectrometry. Thisgives a feature vector of 11,000 intensities (bottom panel), from which we tryto predict the class of that pixel.very recent work that suggest that l 1 penalties may have a more fundamentalrole in classical mainstream statistical inference.To begin, consider standard forward stepwise regression. This procedureenters predictors one a time, choosing the predictor that most decreases theresidual sum of squares at each stage. Defining RSS to be the residual sumof squares for the model containing j predictors and denoting by RSS null theresidual sum of squares for the model omitting the predictor k(j), we can formthe usual statisticR j =(RSS null − RSS)/σ 2(with σ assumed known for now), and compare it to a χ 2 (1) distribution.Although this test is commonly used, we all know that it is wrong. Figure42.3 shows an example. There are 100 observations and 10 predictors ina standard Gaussian linear model, in which all coefficients are actually zero.The left panel shows a quantile-quantile plot of 500 realizations of the statisticR 1 versus the quantiles of the χ 2 (1)distribution. The test is far too liberaland the reason is clear: the χ 2 (1)distribution is valid for comparing two fixednested linear models. But here we are adaptively choosing the best predictor,and comparing its model fit to the null model.In fact it is difficult to correct the chi-squared test to account for adaptiveselection: half-sample splitting methods can be used (Meinshausen et al., 2009;