2DkcTXceO

520 Features of Big Data43.7.3 PropertiesLet ˆβ be a solution to (43.6) and ˆ∆ = ˆβ − β 0 . As in the Danzig selection, thefeasibility of β 0 implied by (43.5) entails thatLetting S 0 =supp(β 0 ), we have‖β 0 ‖ 1 ≥‖ˆβ‖ 1 = ‖β 0 + ˆ∆‖ 1 . (43.9)‖β 0 + ˆ∆‖ 1 = ‖(β 0 + ˆ∆) S0 ‖ 1 + ‖ ˆ∆ S c0‖ 1 ≥‖β 0 ‖ 1 −‖ˆ∆ S0 ‖ 1 + ‖ ˆ∆ S c0‖ 1 .This together with (43.9) yields‖ ˆ∆ S0 ‖ 1 ≥‖ˆ∆ S c0‖ 1 , (43.10)i.e., ˆ∆ is sparse or “restricted.” In particular, with s = |S0 |,‖ ˆ∆‖ 2 ≥‖ˆ∆ S0 ‖ 2 ≥‖ˆ∆ S0 ‖ 1 / √ s ≥‖ˆ∆‖ 1 /(2 √ s), (43.11)where the last inequality uses (43.10). At the same time, since ˆβ and β 0 arein the feasible set (43.5), we have‖L ′ n( ˆβ) − L ′ n(β 0 )‖ ∞ ≤ 2γ nwith probability at least 1 − δ n . By Hölder’s inequality, we conclude that|[L ′ n( ˆβ) − L ′ n(β 0 )] ⊤ ˆ∆| ≤2γn ‖ ˆ∆‖ 1 ≤ 4 √ sγ n ‖ ˆ∆‖ 2 (43.12)with probability at least 1 − δ n ,wherethelastinequalityutilizes(43.11).Byusing the Taylor’s expansion, we can prove the existence of a point β ∗ onthe line segment between β 0 and ˆβ such that L ′ n( ˆβ) − L ′ n(β 0 )=L ′′ n(β ∗ ) ˆ∆.Therefore,| ˆ∆ ⊤ L ′′ n(β ∗ ) ˆ∆| ≤4 √ sγ n ‖ ˆ∆‖ 2 .Since C n is a convex set, β ∗ ∈C n . If we generalize the restricted eigenvaluecondition to the generalized restricted eigenvalue condition, viz.then we haveinf‖∆ S0 ‖ 1≥‖∆ S c0‖ 1inf |∆ ⊤ L ′′ n(β)∆|/‖∆‖ 2 2 ≥ a, (43.13)β∈C n‖ ˆ∆‖ 2 ≤ 4a −1√ sγ n . (43.14)The inequality (43.14) is a statement on the L 2 -convergence of ˆβ, with probabilityat least 1 − δ n . Note that each component ofL ′ n( ˆβ) − L ′ n(β 0 )=L ′ n(β 0 + ˆ∆) − L ′ n(β 0 )in (43.12) has the same sign as the corresponding component of ˆ∆. Condition(43.13) can also be replaced by the requirementinf |[L ′ n(β 0 +∆)− L ′ n(β 0 )] ⊤ ∆|≥a‖∆‖ 2 .‖∆ S0 ‖ 1≥‖∆ S c ‖ 1 0This facilitates the case where L ′′ n does not exist and is a specific case ofNegahban et al. (2012).