statisticalrethinkin..

270 9. BIG ENTROPY AND THE GENERALIZED LINEAR MODELgamma distribution (top-le). is isn’t the only way to produce a gamma distribution, butit is a natural and common one. Furthermore, if you add enough exponential values together,the gamma converges to the normal. Beginning again with the exponential in thecenter, the count distributions at the bottom of the figure can arise by counting the numberof exponentially-distributed events that happen in a finite interval of time. ere are otherrelationships among these distributions, and further relationships between these distributionsand information theory. But much of those details are beyond the scope of this book.I just want you to be aware of the wider context these distributions live in.I’m going to split the introduction of these distributions across this chapter and the next.In this chapter, I introduce and show you how to build models based upon the exponential(FIGURE 9.1, center) and gamma (top-le). ese are natural distributions of durations anddistances. Both durations and distances are measures of displacement, absolute length fromsome point of reference. erefore, unlike truly continuous measures, durations and distancesare strictly positive or zero. ey are never negative. A direct consequence of this factis that the pattern of variation around the mean always varies with the mean. is is quiteunlike the Gaussian distribution, in which the mean and variance are independent (unlessmodeled otherwise).e same general relationship between the mean and the variance is true of count distributions,as well. In the next chapter, I focus on the count distributions at the bottom ofFIGURE 9.1, the Poisson (bottom-le) and binomial (bottom-right) distributions. Counts arediscrete positive integers—like 1, 2 and 3—or zero. Again, they are never negative. Andagain, a consequence of this fact is that the variance of a count distribution is closely relatedto its mean. Both the Poisson and binomial can be derived from the exponential distribution,as indicated in FIGURE 9.1. And the Poisson can be derived from the binomial, by choosingspecial values for its parameters.It isn’t important in this book to grasp the mathematical details of the relationshipsamong these distributions, for the most part. Instead, I describe them in order to help thereader appreciate when each distribution is perhaps most helpful, natural and interpretable.9.1.2. Linking models and distributions. e first component of the GLM strategy is toopen up many natural probability distributions to apply to our outcome measurements. Butjust like with ordinary Gaussian models, we have to work with the parameters of these distributionsin order to build models that include predictor variables, other measurements.e choice of how to associate the basic parameters of the outcome distribution with thepredictor variables is called the link.To remind you, in the case of an ordinary Gaussian regression model with a singe predictorvariable, the linking strategy is to replace the parameter µ, the mean of the Gaussiandistribution, with a linear model that contains the predictor variable. is leads to the nowfamiliarmodel definition:y i ∼ Normal(µ i , σ),µ i = α + βx i .e mean for each case i is linked to an additive model that includes x i and the two parameters,α and β.If you choose a different outcome distribution—a non-Gaussian likelihood—then thefundamental parameters of that distribution will not be µ and σ. Importantly, most distributionsother than the Gaussian do not have means that are independent of their standarddeviations. So in most cases it isn’t possible to just replace the parameter for the mean with