Model Selection Based on the Modulus of Continuity

Therefore, the upper bound of risk function can be determined byR(f n ) L1 R emp (f n ) L1 + ω(f, h) + h (exp − 1 ) n∑ 2|wσ ′ k | + σ√2δ . (25)k=1Example: Consider the following form of sigmoid functions:φ k (x) =11 + exp(−a i x + b i )for k = 1, · · · , nwhere a k and b k represent the adaptable parameters of the basis function φ k . Applyingthe mean value theorem to φ k , we getw(φ k , h) ‖φ ′ k‖ ∞ h h 4for k = 1, · · · , nsince φ ′ k has the maximum at x = b k/a k . That is,max {ω(φ k, h)} h1kn 4 .Therefore, the upper bound of the risk functions can be determined byR(f n ) L1 R emp (f n ) L1 + ω(f, h) + h 4n∑ 2|w k | + σ√δ . (26)k=1The suggested theorem of true risk bounds is derived in the sense of the L 1 measure, that is,the risk function described by (16). Other model selection criteria such as AIC and SEB arederived from the risk function using the quadratic loss function L(y, f n (x)) = (y −f n (x)) 2 . Inthis context, we can show that the optimal function f n ∗(x) determined from the risk functionof the L 1 measure is also optimal in the sense of the risk function using the quadratic lossfunction under the assumption that the estimation function is unbiased and the error termy − f n (x) has normal distribution.10