Bayesian Linear Regression - CEDAR

More documents

Recommendations

Info

Machine Learning ! ! ! ! !SrihariMotivation of Bayesian Approach• We have seen Linear Regression with M basisM −1functions:y(x,w) =∑j = 0w jφ j(x) = w T φ(x)• The maximum likelihood solution isw ML= (Φ T Φ) −1 Φ T t⎛ φ0(x⎜⎜ φ0(xΦ = ⎜⎜⎝φ0(x12N)))φ (x11)...φφM −1M −1(x(x1N) ⎞⎟⎟⎟⎟)⎠2
Machine Learning ! ! ! ! !SrihariShortcomings of MLE• M.L.E. of parameters wdoes not address– M (Model complexity: how many basis functions?– It is controlled by data size N• More data allows better fit without overfitting• Regularization also controls overfit (λ controls itseffect)E D(w) = 1 2N∑n=1{ t n−w T φ(x n) }2whereE(w) = E D(w) + λE W(w)• But M and choice of ϕ j are still important– M can be determined by holdout, but wasteful of dataE (w) =• Model complexity and over-fitting better handled usingBayesian approach 3W12wTw
Page 1: Machine Learning ! ! ! ! !SrihariBa
Page 6 and 7: Machine Learning ! ! ! ! !SrihariLi
Page 8 and 9: Machine Learning ! ! ! ! !SrihariPr
Page 10 and 11: Machine Learning ! ! ! ! !SrihariSa
Page 12 and 13: Machine Learning ! ! ! ! !SrihariBa
Page 14 and 15: Sequential Bayesian LearningMachine
Page 20 and 21: Machine Learning ! ! ! ! !SrihariEx
Page 24 and 25: Machine Learning ! ! ! ! !SrihariEq
Page 30 and 31: Machine Learning ! ! ! ! !SrihariKe
Page 35 and 36: Machine Learning ! ! ! ! !SrihariPa
Page 37 and 38: Machine Learning ! ! ! ! !SrihariGa
Page 39 and 40: Machine Learning ! ! ! ! !SrihariDe
Page 43 and 44: Machine Learning ! ! ! ! !SrihariGP
Page 45: Machine Learning ! ! ! ! !SrihariWi

Bayesian Linear Regression - CEDAR

Create successful ePaper yourself

Delete template?

Save as template?