Bayesian Linear Regression - CEDAR

Machine Learning ! ! ! ! !SrihariProbabilistic Linear Regression is a GP!• We wish to evaluate y (x) at training points x 1 ,.., x N• Our interest is the joint distribution of values y=[y(x 1 ),..y(x N )] Since y(x)=w T φ(x) we can write y = Φw– where Φ is the design matrix with elements Φ nk =φ k (x n )N x M times M x 1 yields a N x 1• Since y is a linear combination of elements of w whichare Gaussian distributed as p(w) = N(w|0,α -1 I)!y is itself Gaussian with!!mean:E[y] =ΦE[w]=0 andco !variance: Cov[y]=E[yy T ]=ΦE[ww T ]Φ T =(1/α)ΦΦ T =K! where K is the N x N Gram Matrix with elements !K nm = k (x n ,x m ) = (1/α)φ(x n ) T φ(x m )Covarianceexpressedby Kernelfunctionand k (x,x’) is the kernel function!Thus Gaussian distributed weight vector induces a Gaussian joint distribution over training samples