Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

Kraha et al.Interpreting multiple regressionbe considered the most important variable (RIW = 0.241), followedby X3 (RIW = 0.045) and X2 (RIW = 0.015). The RIWsoffer an additional representation of the individual effect ofeach predictor while simultaneously considering the combinationof predictors as well (Johnson, 2000). The sum of theweights (0.241 + 0.045 + 0.015 = 0.301) is equal to R 2 . PredictorX1 can be interpreted as the most important variablerelative to other predictors (Johnson, 2001). The interpretationis consistent with a full DA, because both the individualpredictor contribution with the outcome variable (r X 1·Y ),and the potential multicollinearity (r X 1·X2 and r X 1·X3) withother predictors are accounted for. While the RIWs may differslightly compared to general dominance weights (e.g., 0.015and 0.016, respectively, for X2), the conclusions are the consistentwith those from a full DA. This method rank ordersthe variables with X1 as the most important, followed by X3and X2. The suppression role of X2, however, is not identifiedby this method, which helps explain its rank as third in thisprocess.DISCUSSIONPredictor variables are more commonly correlated than not inmost practical situations, leaving researchers with the necessity toaddressing such multicollinearity when they interpret MR results.Historically, views about the impact of multicollinearity on regressionresults have ranged from challenging to highly problematic.At the extreme, avoidance of multicollinearity is sometimes evenconsidered a prerequisite assumption for conducting the analysis.These perspectives notwithstanding, the current article haspresented a set of tools that can be employed to effectively interpretthe roles various predictors have in explaining variance in acriterion variable.To be sure, traditional reliance on standardized or unstandardizedweights will often lead to poor or inaccurate interpretationswhen multicollinearity or suppression is present in the data. Ifresearchers choose to rely solely on the null hypothesis statisticalsignificance test of these weights, then the risk of interpretive erroris noteworthy. This is primarily because the weights are heavilyaffected by multicollinearity, as are their SE which directly impactthe magnitude of the corresponding p values. It is this realitythat has led many to suggest great caution when predictors arecorrelated.Advances in the literature and supporting software technologyfor their application have made the issue of multicollinearity muchless critical. Although predictor correlation can certainly complicateinterpretation, use of the methods discussed here allowfor a much broader and more accurate understanding of theMR results regarding which predictors explain how much variancein the criterion, both uniquely and in unison with otherpredictors.In data situations with a small number of predictors or very lowlevels of multicollinearity, the interpretation method used mightnot be as important as results will most often be very similar.However, when the data situation becomes more complicated (asis often the case in real-world data, or when suppression exists asexampled here), more care is needed to fully understand the natureand role of predictors.CAUSE AND EFFECT, THEORY, AND GENERALIZATIONAlthough current methods are helpful, it is very important thatresearchers remain aware that MR is ultimately a correlationalbasedanalysis, as are all analyses in the general linear model.Therefore, variable correlations should not be construed as evidencefor cause and effect relationships. The ability to claim causeand effect are predominately issues of research design rather thanstatistical analysis.Researchers must also consider the critical role of theory whentrying to make sense of their data. Statistics are mere tools tohelp understand data, and the issue of predictor importance inany given model must invoke the consideration of the theoreticalexpectations about variable relationships. In different contextsand theories, some relationships may be deemed more or lessrelevant.Finally, the pervasive impact of sampling error cannot beignored in any analytical approach. Sampling error limits the generalizabilityof our findings and can cause any of the methodsdescribed here to be more unique to our particular sample thanto future samples or the population of interest. We should notassume too easily that the predicted relationships we observe willnecessarily appear in future studies. Replication continues to be akey hallmark of good science.INTERPRETATION METHODSThe seven approaches discussed here can help researchers betterunderstand their MR models, but each has its own strengths andlimitations. In practice, these methods should be used to informeach other to yield a better representation of the data. Below wesummarize the key utility provided by each approach.Pearson r correlation coefficientPearson r is commonly employed in research. However, as illustratedin the heuristic example, r does not take into accountthe multicollinearity between variables and they do not allowdetection of suppressor effects.Beta weights and structure coefficientsInterpretations of both β weights and structure coefficients providea complementary comparison of predictor contribution to theregression equation and the variance explained in the effect. Betaweights alone should not be utilized to determine the contributionpredictor variables make to a model because a variable mightbe denied predictive credit in the presence of multicollinearity.Courville and Thompson, 2001; see also Henson, 2002) advocatedfor the interpretation of (a) both β weights and structurecoefficients or (b) both β weights and correlation coefficients.When taken together, β and structure coefficients can illuminatethe impact of multicollinearity, reflect more clearly the ability ofpredictors to explain variance in the criterion, and identify suppressoreffects. However, they do not necessarily provide detailedinformation about the nature of unique and commonly explainedvariance, nor about the magnitude of the suppression.All possible subsets regressionAll possible subsets regression is exploratory and comes withincreasing interpretive difficulty as predictors are added toFrontiers in Psychology | Quantitative Psychology and Measurement March 2012 | Volume 3 | Article 44 | 83