Flexible modelling using basis expansions (Chapter 5) Linear ...

Smoothing splines◮ This is an approach closer to ridge regression. Put knots ateach distinct x value, and then shrink the coefficients bypenalizing the fit . (Figure 5.6)◮subject to:argmin βΣ i(yi − Σ j β j h j (x i ) ) 2β T Ωβ < c◮ with no constraint we get usual least squares◮ Ω controls the smoothness of the final fit:∫Ω jk = h j ′′ (x)h′′ k (x)dx◮ This solves the variational problemargmin fΣ i (y i − f (x i )) 2 + λ∫ ba{f ′′ (t)} 2 dt◮ the solution is a natural cubic spline with knots at each x i