2DkcTXceO

J. Fan 519where · denotes the componentwise product (a factor 2 of off-diagonal elementsis ignored). If we construct the high-confidence set based directly onthe estimation equations L ′ n(Θ) =ΘS n − I p ,thenthesparsehigh-confidenceset becomesmin‖ΘS n−I p‖ ∞≤γ n‖vec(Θ)‖ 1 .If the matrix L 1 -norm is used in (43.6) to measure the sparsity, then theresulting estimator is the CLIME estimator of Cai et al. (2011), viz.min ‖Θ‖ 1 .‖ΘS n−I p‖ ∞≤γ nIf we use the Gaussian log-likelihood, viz.L n (Θ) =− ln(|Θ|)+tr(ΘS n ),then L ′ n(Θ) =−Θ −1 + S n and C n = {‖Θ −1 − S n ‖ ∞ ≤ γ n }.Thesparsestsolution is then given bymin ‖Θ‖ 1 .‖Θ −1 −S n‖ ∞≤γ nIf the relative norm ‖A‖ ∞ = ‖Θ 1/2 AΘ 1/2 ‖ ∞ is used, the solution can bemore symmetrically written asmin ‖Θ‖ 1 .‖Θ 1/2 S nΘ 1/2 −I p‖ ∞≤γ nIn the construction of the sparse linear discriminant analysis from twoNormal distributions N (µ 0 , Σ) and N (µ 1 , Σ), the Fisher classifier is linearand of the form 1{β ⊤ (X − µ) > 0}, whereµ =(µ 0 + µ 1 )/2, δ = µ 1 − µ 0 ,and β = Σ −1 δ.Theparametersµ and δ can easily be estimated from thesample. The question is how to estimate β, whichisassumedtobesparse.Onedirect way to construct confidence interval is to base directly the estimationequations L ′ n(β) =S n β − ˆδ, whereS n is the pooled sample covariance and ˆδis the difference of the two sample means. The high-confidence set is thenC n = {β : ‖S n β − ˆδ‖ ∞ ≤ γ n }. (43.7)Again, this is a set implied by data with high confidence. The sparsest solutionis the linear programming discriminant rule by Cai et al. (2011).The above method of constructing confidence is neither unique nor thesmallest. Observe that (through personal communication with Dr Emre Barut)‖S n β − ˆδ‖ ∞ = ‖(S n − Σ)β + δ − ˆδ‖ ∞ ≤‖(S n − Σ)‖ ∞ ‖β‖ 1 + ‖δ − ˆδ‖ ∞ .Therefore, a high confidence set can be taken asC n = {‖S n β − ˆδ‖ ∞ ≤ γ n,1 ‖β‖ 1 + γ n,2 }, (43.8)where γ n,1 and γ n,2 are the high confident upper bound of ‖(S n − Σ)‖ ∞ and‖δ − ˆδ‖ ∞ . The set (43.8) is smaller than the set (43.7), since a further bound‖β‖ 1 in (43.8) by a constant γ n,3 yields (43.7).