Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

SmithsonComparing moderation of slopes and correlationsand tests of differences between correlation coefficients are essentially“subgroup” methods. At the time there was no way to unifythe examination of moderation of correlations and slopes. Thepresent paper describes and demonstrates such an approach forcategorical moderators, via structural equations models.In a later paper, Stone and Hollenbeck (1989) reprised thisdebate and recommended variance-stabilizing and homogenizingtransformations as a way to eliminate the apparent disagreementbetween moderation of correlations and moderation of slopes.These include not only transformations of the dependent variable,but also within-groups standardization and/or normalization.They also, again, recommended abandoning the distinctionbetween degree and form moderation and focusing solely on form(i.e., moderated regression). The usual cautions against routinelytransforming variables and objections to applying different transformationsto subsamples aside, we shall see that transforming thedependent variable is unlikely to eliminate the non-equivalencebetween moderation of slopes and correlations. Moreover, otherinvestigators of this issue do not arrive at the same recommendationas Stone and Hollenbeck when it comes to a “best”test.Apparently independently of the aforementioned work, andextending the earlier work of Dretzke et al. (1982), Alexander andDeShon (1994) demonstrated severe effects from heterogeneity oferror-variance (HeEV) on power and Type I error rates for theF-test of equality of regression slopes. In contrast to Stone andHollenbeck (1989), they concluded that for a categorical moderator,the “test of choice” is the test for equality of correlations acrossthe moderator categories, provided that the hypotheses of equalcorrelations and equal slopes are approximately identical.These hypotheses are equivalent if and only if the ratio of thevariance in X to the variance in Y is equal across moderator categories(Arnold, 1982; Alexander and DeShon, 1994). The reasonfor this is clear from the textbook equation between correlationsand unstandardized regression coefficients. For the ith category ofthe moderator,β yi = ρ iσ yiσ xi(1)For example, a simple algebraic argument shows that if theσ yi /σ xi ratio is not constant for, say, i = 1 and i = 2 thenβ 1 = β 2 ⇒ ρ 1 ̸= ρ 2 , and likewise ρ 1 = ρ 2 ⇒ β 1 ̸= β 2 . More generally,σ y1 σ x2σ x1 σ y2> ( ( β 2 but ρ 2 > ρ 1 ifwhen ρ 2 > ρ 1 , it is also true thatσ y1 σ x2σ x1 σ y2> ρ 2ρ 1.The position taken in this paper is that in multiple linear regressionthere are three distinct and valid types of moderator effects.First, in multiple regression equation (1) generalizes to a versionwhere standardized regression coefficients replace correlationcoefficients:β yi = B yiσ yiσ xi(3)where B yi is a standardized regression coefficient. Thus, we havemoderation of unstandardized versus standardized regressioncoefficients (or correlations when there is only one predictor),which are equivalent if and only if the aforementioned varianceratio is equal across moderator categories. Otherwise, the assumptionthat moderation of one implies equivalent moderation of theother is mistaken. This is a simple generalization of Arnold’s (1982)and Sharma et al.’s (1981) distinction.Second, the semi-partial correlation coefficient, ν xi , is a simplefunction of B yi and tolerance. In the ith moderator category,the tolerance of a predictor, X, is T xi = 1 − Rxi 2 , where R2 xiis thesquared multiple correlation for X regressed on the other predictorsincluded in the multiple regression model. The standardizedregression coefficient, semi-partial correlation, and tolerance arerelated byν xi = B yi√Txi .Equation (3) therefore may be rewritten asσ yiβ yi = ν xi √ . (4)σ xi TxiThus, we have a distinction between the moderation of the uniquecontribution of a predictor to the explained variance of a dependentvariable and moderation of regression coefficients (whetherstandardized or not). Equivalence with moderation of standardizedcoefficients (or simple correlations) hinges on whether toleranceis constant across moderator categories (an issue not dealtwith in this paper), while equivalence with moderation of unstandardizedcoefficients depends on both constant tolerance andconstant variance ratios.In a later paper, DeShon and Alexander (1996) proposed alternativeprocedures for testing equality of regression slopes underHeEV, but they and both earlier and subsequent researchers appearto have neglected the idea of testing for equal variance ratios (EVR)across moderator categories. This is understandable, given thatHeEV is a more general concern in some respects and the primaryobject of most regression (and ANOVA) models is prediction.Nevertheless, it is possible for HeEV to be satisfied when EVR isnot. An obvious example is when there is HoV for Y and equalityof correlations across moderator categories but HeV for X. Theseconditions entail HeEV but also imply that slopes cannot be equalacross categories. This case seems to have been largely overlookedin the literature on moderators. More generally, HeEV is ensuredwhen, for all i and j,The same implication holds if the inequalities are changed from> to