Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

More documents

Recommendations

Info

NimonStatistical assumptionsFor example, Osborne’s (2012) book provides an accessible treatmentof a wide variety of data transformation techniques whileBurdenski’s (2000) article review graphics procedures to evaluateunivariate, bivariate, and multivariate normality. Finally, theinformation presented in this article should be helpful to readersof scholarly reports. Readers cannot presume that just because anarticle has survived peer review, the interpretation of the findingsis methodologically sound (cf. Henson et al., 2010). Readers mustmake their own judgment as to the quality of the study if informationthat could affect the validity of the data presented is notreported. With the information presented in this article and others,I invite readers to take an active role in evaluating the findingsof quantitative research reports and become informed consumersof the data presented.REFERENCESBoneau, C. A. (1960). The effects of violationsof assumptions underlyingthe t test. Psychol. Bull. 57, 49–64.Box, G. E. P. (1954). Some theorems onquadratic forms applied in the studyof analysis of variance problems, II.Effects of inequality of variance andof correlation between errors in thetwo-way classification. Ann. Math.Statist. 25, 484–498.Bump, W. (1991). The normal curvetakes many forms: a review of skewnessand kurtosis. Paper Presentedat the Annual Meeting of the SouthwestEducational Research Association,San Antonio. [ERIC DocumentReproduction Service No. ED 342790].Burdenski, T. K. (2000). Evaluating univariate,bivariate, and multivariatenormality using graphical procedures.Mult. Linear Regression Viewp.26, 15–28.Cohen, J., Cohen, P., West, S. G., andAiken, L. S. (2003). Applied MultipleRegression/Correlation Analysisfor the Behavioral Sciences, 3rd Edn.Mahwah, NJ: Erlbaum.Conover, W. J. (1999). Practical NonparametricStatistics, 3rd Edn. NewYork: Wiley.Crocker, L., and Algina, J. (1986). Introductionto Classical and Modern TestTheory. Belmont, CA: Wadsworth.DeCarlo, L. T. (1997). On the meaningand use of kurtosis. Psychol. Methods2, 292–307.Dillon, W. R., and Bearden, W. (2001).Writing the survey question to capturethe concept. J. Consum. Psychol.10, 67–69.Fan, X. (1996). A SAS programfor assessing multivariate normality.Educ. Psychol. Meas. 56, 668–674.Gall, M. D., Gall, J. P., and Borg, W.R. (2007). Educational Research: AnIntroduction, 8th Edn. Boston, MA:Allyn and Bacon.Garson, G. D. (2012). Hierarchical LinearModeling: Guide and Applications.Thousand Oaks,CA: Sage Publications,Inc.Glass, G.V., Peckham, P. D., and Sanders,J. R. (1972). Consequences of failureto meet assumptions underlyingthe fixed effects analyses of varianceand covariance. Rev. Educ. Res. 42,237–288.Graham, J. M. (2008). The general linearmodel as structural equationmodeling. J. Educ. Behav. Stat. 33,485–506.Harris, R. J. (2001). A Primer of MultivariateStatistics, 3rd Edn. Mahwah,NJ: Erlbaum.Henson, R. K. (1999). “Multivariatenormality: what is it and how isit assessed?” in Advances in SocialScience Methodology, Vol. 5, ed.B. Thompson (Stamford, CT: JAIPress), 193–211.Henson, R. K., Hull, D. M., andWilliams, C. S. (2010). Methodologyin our education research culture:toward a stronger collective quantitativeproficiency. Educ. Res. 39,229–240.Hoekstra, R., Kiers, H. A. L., andJohnson, A. (2012). Are assumptionsof well-known statisticaltechniques checked, and why(not)? Front. Psychol. 3:137.doi:10.3389/fpsyg.2012.00137Howell, D. D. (1992). Statistical Methodsfor Psychology, 3rd Edn. Boston:PWS-Kent.Kerlinger, F. N. (1986). Foundations ofBehavioral Research, 3rd Edn. NewYork: Holt, Rinehart and Winston.Kieffer, K. M., Reese, R. J., and Thompson,B. (2001). Statistical techniquesemployed in AERJ and JCP articlesfrom 1988 to 1997: a methodologicalreview. J. Exp. Educ. 69, 280–309.Kline, R. B. (2005). Principles and Practiceof Structural Equation Modeling,2nd Edn. New York, NY: TheGuilford Press.Kreft, I. G. G., and de Leeuw, J.(1998). Introducing Multilevel Modeling.Thousand Oaks, CA: Sage.Mardia, K. V. (1985). “Mardia’s testof multinormality,” in Encyclopediaof Statistical Sciences, Vol. 5, eds S.Kotz and N. L. Johnson (New York:Wiley), 217–221.Maxwell, S. E., and Delaney, H. D.(2004). Designing Experiments andAnalyzing Data: A Model ComparisonPerspective, 2nd Edn. Mahwah,NJ: Erlbaum.Nimon, K., and Williams, C. (2009).Performance improvementthrough repeated measures: aprimer for educators consideringunivariate and multivariatedesign. Res. High. Educ. J. 2,117–136.Nimon, K., Zientek, L. R., and Henson,R. (2012). The assumption of a reliableinstrument and other pitfallsto avoid when considering the reliabilityof data. Front. Psychol. 3:102.doi:10.3389/fpsyg.2012.00102Onwuegbuzie, A. J., and Daniel, L. G.(2003). “Typology of analytical andinterpretational errors in quantitativeand qualitative educationalresearch,” in Current Issues inEducation, Vol. 6. Available at: http://cie.ed.asu.edu/volume6/number2/[accessed February 19, 2003].Osborne, J., and Waters, E. (2002). Fourassumptions of multiple regressionthat researchers should alwaystest. Pract. Assess. Res. Eval. 8.Available at: http://PAREonline.net/getvn.asp?v=8&n=2Osborne, J. W. (2000). Advantagesof hierarchical linear modeling.Pract. Assess. Res. Eval. 71. Availableat: http://pareonline.net/getvn.asp?v=7&n=1Osborne, J. W. (2012). Best Practices inData Cleaning: A Complete Guide toEverything You Need to Do Before andAfter Collecting Your Data. ThousandOaks, CA: Sage.Schumacker, R. E., and Lomax, R. G.(2004). A Beginner’s Guide to StructuralEquation Modeling. Mahwah,NJ: Erlbaum.Stevens, J. (2002). Applied MultivariateStatistics for the Social Sciences, 4thEdn. Mahwah, NJ: Erlbaum.Stevens, S. (1946). On the theory ofscales of measurement. Science 103,677–680.Stevens, S. (1951). “Mathematics, measurement,and psychophysics,” inHandbook of Experimental Psychology,ed. S. Stevens (New York: Wiley),1–49.Tabachnick, B. G., and Fidell, L. S.(2001). Using Multivariate Statistics,4th Edn. Needham Heights, MA:Allyn and Bacon.Thompson, B. (1988). Discard variance:a cardinal since in research. Meas.Eval. Couns. Dev. 21, 3–4.Thompson, B. (1990). Multinor: a Fortranprogram that assists in evaluatingmultivariate normality. Educ.Psychol. Meas. 50, 845–848.Thompson, B. (2006). Foundations ofBehavioral Statistics: An Insight-Based Approach. New York: GuilfordPress.Warner, R. M. (2008). Applied Statistics:From Bivariate through MultivariateTechniques. Thousand Oaks,CA: Sage.Wilkinson, L., and APA Task Force onStatistical Inference. (1999). Statisticalmethods in psychology journals:guidelines and explanation. Am. Psychol.54, 594–604.Zientek, L. R., Capraro, M. M., andCapraro, R. M. (2008). Reportingpractices in quantitative teachereducation research: one look at theevidence cited in the AERA panelreport. Educ. Res. 37, 208–216.Conflict of Interest Statement: Theauthor declares that the research wasconducted in the absence of any commercialor financial relationships thatcould be construed as a potential conflictof interest.Received: 21 June 2012; paper pendingpublished: 26 July 2012; accepted: 12August 2012; published online: 28 August2012.Citation: Nimon KF (2012) Statisticalassumptions of substantive analysesacross the general linear model: a minireview.Front. Psychology 3:322. doi:10.3389/fpsyg.2012.00322This article was submitted to Frontiersin Quantitative Psychology and Measurement,a specialty of Frontiers in Psychology.Copyright © 2012 Nimon. This is anopen-access article distributed under theterms of the Creative Commons AttributionLicense, which permits use, distributionand reproduction in other forums,provided the original authors and sourceare credited and subject to any copyrightnotices concerning any third-partygraphics etc.www.frontiersin.org August 2012 | Volume 3 | Article 322 | 75
METHODS ARTICLEpublished: 14 March 2012doi: 10.3389/fpsyg.2012.00044Tools to support interpreting multiple regression in the faceof multicollinearityAmanda Kraha 1 *, Heather Turner 2 , Kim Nimon 3 , Linda Reichwein Zientek 4 and Robin K. Henson 21Department of Psychology, University of North Texas, Denton, TX, USA2Department of Educational Psychology, University of North Texas, Denton, TX, USA3Department of Learning Technologies, University of North Texas, Denton, TX, USA4Department of Mathematics and Statistics, Sam Houston State University, Huntsville, TX, USAEdited by:Jason W Osborne, Old DominionUniversity, USAReviewed by:Elizabeth Stone, Educational TestingService, USAJames Stamey, Baylor University,USA*Correspondence:Amanda Kraha, Department ofPsychology, University of North Texas,1155 Union Circle No. 311280,Denton, TX 76203, USA.e-mail: amandakraha@my.unt.eduWhile multicollinearity may increase the difficulty of interpreting multiple regression (MR)results, it should not cause undue problems for the knowledgeable researcher. In thecurrent paper, we argue that rather than using one technique to investigate regressionresults, researchers should consider multiple indices to understand the contributions thatpredictors make not only to a regression model, but to each other as well. Some of thetechniques to interpret MR effects include, but are not limited to, correlation coefficients,beta weights, structure coefficients, all possible subsets regression, commonality coefficients,dominance weights, and relative importance weights. This article will review aset of techniques to interpret MR effects, identify the elements of the data on which themethods focus, and identify statistical software to support such analyses.Keywords: multicollinearity, multiple regressionMultiple regression (MR) is used to analyze the variability of adependent or criterion variable using information provided byindependent or predictor variables (Pedhazur, 1997). It is animportant component of the general linear model (Zientek andThompson, 2009). In fact, MR subsumes many of the quantitativemethods that are commonly taught in education (Hensonet al., 2010) and psychology doctoral programs (Aiken et al., 2008)and published in teacher education research (Zientek et al., 2008).One often cited assumption for conducting MR is minimal correlationamong predictor variables (cf. Stevens, 2009). As Thompson(2006) explained, “Collinearity (or multicollinearity) refers to theextent to which the predictor variables have non-zero correlationswith each other” (p. 234). In practice, however, predictor variablesare often correlated with one another (i.e., multicollinear), whichmay result in combined prediction of the dependent variable.Multicollinearity can lead to increasing complexity in theresearch results, thereby posing difficulty for researcher interpretation.This complexity, and thus the common admonitionto avoid multicollinearity, results because the combined predictionof the dependent variable can yield regression weights thatare poor reflections of variable relationships. Nimon et al. (2010)noted that correlated predictor variables can “complicate resultinterpretation...a fact that has led many to bemoan the presenceof multicollinearity among observed variables” (p. 707). Indeed,Stevens (2009) suggested “Multicollinearity poses a real problemfor the researcher using multiple regression” (p. 74).Nevertheless, Henson (2002) observed that multicollinearityshould not be seen as a problem if additional analytic informationis considered:The bottom line is that multicollinearity is not a problem inmultiple regression, and therefore not in any other [generallinear model] analysis, if the researcher invokes structurecoefficients in addition to standardized weights. In fact,in some multivariate analyses, multicollinearity is actuallyencouraged, say, for example, when multi-operationalizing adependent variable with several similar measures. (p. 13)Although multicollinearity is not a direct statistical assumptionof MR (cf. Osborne and Waters, 2002), it complicates interpretationas a function of its influence on the magnitude of regressionweights and the potential inflation of their standard error (SE),thereby negatively influencing the statistical significance tests ofthese coefficients. Unfortunately, many researchers rely heavilyon standardized (beta, β) or unstandardized (slope) regressionweights when interpreting MR results (Courville and Thompson,2001; Zientek and Thompson, 2009). In the presence of multicollineardata, focusing solely on regression weights yields atbest limited information and, in some cases, erroneous interpretation.However, it is not uncommon to see authors argue for theimportance of predictor variables to a regression model based onthe results of null hypothesis statistical significance tests of theseregression weights without consideration of the multiple complexrelationships between predictors and predictors with theiroutcome.PURPOSEThe purpose of the present article is to discuss and demonstrateseveral methods that allow researchers to fully interpret and understandthe contributions that predictors play in forming regressioneffects, even when confronted with collinear relationships amongthe predictors. When faced with multicollinearity in MR (or othergeneral linear model analyses), researchers should be aware of andjudiciously employ various techniques available for interpretation.These methods, when used correctly, allow researchers to reachbetter and more comprehensive understandings of their data thanwww.frontiersin.org March 2012 | Volume 3 | Article 44 | 76
Page 2 and 3:
FRONTIERS COPYRIGHTSTATEMENT© Copy
Page 4 and 5:
Table of Contents05 Is Data Cleanin
Page 7 and 8:
OsborneAssumptions and data cleanin
Page 9 and 10:
ORIGINAL RESEARCH ARTICLEpublished:
Page 12 and 13:
Hoekstra et al.Why assumptions are
Page 14 and 15:
Page 16 and 17:
Page 18 and 19:
ORIGINAL RESEARCH ARTICLEpublished:
Page 20 and 21:
García-PérezStatistical conclusio
Page 22 and 23:
Page 24 and 25:
Page 26 and 27: García-PérezStatistical conclusio
Page 28 and 29: García-PérezStatistical conclusio
Page 30 and 31: Sheng and ShengEffect of non-normal
Page 42 and 43: REVIEW ARTICLEpublished: 12 April 2
Page 44 and 45: Nimon et al.The assumption of relia
Page 56 and 57: TressoldiPower replication unreliab
Page 58 and 59: TressoldiPower replication unreliab
Page 60 and 61: ORIGINAL RESEARCH ARTICLEpublished:
Page 62 and 63: FinchModern methods for the detecti
Page 72 and 73: MINI REVIEW ARTICLEpublished: 28 Au
Page 74 and 75: NimonStatistical assumptionsand Del
Page 78 and 79: Kraha et al.Interpreting multiple r
Page 94 and 95: SmithsonComparing moderation of slo
Page 102 and 103: REVIEW ARTICLEpublished: 01 March 2
Page 104 and 105: Flora et al.Factor analysis assumpt
Page 124 and 125: Kasper and ÜnlüAssumptions of fac
Page 126 and 127:
Kasper and ÜnlüAssumptions of fac
Page 128 and 129:
Page 130 and 131:
Page 132 and 133:
Page 134 and 135:
Page 136 and 137:
Page 138 and 139:
Page 140 and 141:
Page 142 and 143:
Page 144 and 145:
Lages and JaworskaHow predictable a
Page 146 and 147:
Page 148 and 149:
Page 150 and 151:
Page 152 and 153:
Cummiskey et al.Testing assumptions
Page 154 and 155:
Page 156 and 157:
Page 158:
show all

Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?