Fundamentals of Probability and Statistics for Engineers

Linear Models and Linear Regression 337ye i(x i ,y i )Estimated regression line:∧ ∧y = α + βxTrue regression line:y = α + βxxFigure 11.1The least squares method of estimationThe least-square estimates ^ and ^ , respectively, of and are found byminimizingQ ˆ Xne 2 i ˆ Xn‰y i …^‡^x i †Š 2 : …11:6†iˆ1 iˆ1In the above, the sample-value pairs are (x 1 ,y 1 ), (x 2 ,y 2 ), . . . , (x n ,y n ), ande i ,i ˆ 1,2,...,n, are called the residuals. Figure 11.1 gives a graphical presentationof this procedure. We see that the residuals are the vertical distancesbetween the observed values of Y ,y i , and the least-square estimate ^ ‡ ^x oftrue regression line ‡ x.The estimates ^ and are easily found based on the least-square procedure.The results are stated below as Theorem 11.1.Theorem 11.1: consider the simple linear regression model defined byEquation (11.4). Let (x 1 ,y 1 ), (x 2 ,y 2 ),...,(x n ,y n ) be observed sample values of Ywith associated values of x. Then the least-square estimates of and are^ ˆ y ^x; …11:7†" #" #^ ˆXnX n1…x i x†…y i y† …x i x† 2 ; …11:8†iˆ1iˆ1TLFeBOOK