Bayesian Linear Regression - CEDAR

More documents

Recommendations

Info

Machine Learning ! ! ! ! !SrihariProbabilistic Linear Regression is a GP!• We wish to evaluate y (x) at training points x 1 ,.., x N• Our interest is the joint distribution of values y=[y(x 1 ),..y(x N )] Since y(x)=w T φ(x) we can write y = Φw– where Φ is the design matrix with elements Φ nk =φ k (x n )N x M times M x 1 yields a N x 1• Since y is a linear combination of elements of w whichare Gaussian distributed as p(w) = N(w|0,α -1 I)!y is itself Gaussian with!!mean:E[y] =ΦE[w]=0 andco !variance: Cov[y]=E[yy T ]=ΦE[ww T ]Φ T =(1/α)ΦΦ T =K! where K is the N x N Gram Matrix with elements !K nm = k (x n ,x m ) = (1/α)φ(x n ) T φ(x m )Covarianceexpressedby Kernelfunctionand k (x,x’) is the kernel function!Thus Gaussian distributed weight vector induces a Gaussian joint distribution over training samples
Machine Learning ! ! ! ! !SrihariGeneral definition of GP!• We saw a particular example of a Gaussian process!– Linear regression using the parametric model!– With samples y= [y(x 1 ),..y(x N )] !– Assume p(w) = N(w|0,α -1 I)y(x)=w T φ(x)– Then E[y] =0 and Cov[y]=K, the Gram matrix which is equivalentto pairwise kernel values!• More generally, a Gaussian process is a probabilitydistribution over functions y (x)– Such that the set of values of y(x) evaluated at arbitrary pointsx 1 ,..,x N jointly have a multivariate Gaussian distribution!• For a single input x 1 , output y 1 is univariate Gaussian. For two inputsx 1, x 2 the output y 1, y 2 is bivariate Gaussian, etc!33
Page 1 and 2: Machine Learning ! ! ! ! !SrihariBa
Page 3: Machine Learning ! ! ! ! !SrihariSh
Page 6 and 7: Machine Learning ! ! ! ! !SrihariLi
Page 8 and 9: Machine Learning ! ! ! ! !SrihariPr
Page 10 and 11: Machine Learning ! ! ! ! !SrihariSa
Page 12 and 13: Machine Learning ! ! ! ! !SrihariBa
Page 14 and 15: Sequential Bayesian LearningMachine
Page 20 and 21: Machine Learning ! ! ! ! !SrihariEx
Page 24 and 25: Machine Learning ! ! ! ! !SrihariEq
Page 30 and 31: Machine Learning ! ! ! ! !SrihariKe
Page 35 and 36: Machine Learning ! ! ! ! !SrihariPa
Page 37 and 38: Machine Learning ! ! ! ! !SrihariGa
Page 39 and 40: Machine Learning ! ! ! ! !SrihariDe
Page 41 and 42: Machine Learning ! ! ! ! !SrihariPr
Page 43 and 44: Machine Learning ! ! ! ! !SrihariGP
Page 45: Machine Learning ! ! ! ! !SrihariWi

Bayesian Linear Regression - CEDAR

Create successful ePaper yourself

Delete template?

Save as template?