Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

NimonStatistical assumptionsLinearity is important in a practical sense because Pearson’s r,which is fundamental to the vast majority of parametric statisticalprocedures (Graham, 2008), captures only the linear relationshipamong variables (Tabachnick and Fidell, 2001). Pearson’s runderestimates the true relationship between two variables that isnon-linear (i.e., curvilinear; Warner, 2008).Unless there is strong theory specifying non-linear relationships,researchers may assume linear relationships in their data(Cohen et al., 2003). However, linearity is not guaranteed andshould be validated with graphical methods (see Tabachnickand Fidell, 2001). Non-linearity reduces the power of statisticaltests such as ANCOVA, MANOVA, MANCOVA, linear regression,and canonical correlation (Tabachnick and Fidell, 2001).In the case of ANCOVA and MANCOVA, non-linearity resultsin improper adjusted means (Stevens, 2002). If non-linearity isdetected, researchers may transform data, incorporate curvilinearcomponents, eliminate the variable producing non-linearity,or conduct a non-linear analysis (cf. Tabachnick and Fidell, 2001;Osborne and Waters, 2002; Stevens, 2002; Osborne, 2012), as longas the process is clearly reported.VARIANCEAcross parametric statistical procedures commonly used in quantitativeresearch, at least five assumptions relate to variance. Theseare: homogeneity of variance, homogeneity of regression, sphericity,homoscedasticity, and homogeneity of variance-covariancematrix.Homogeneity of variance applies to univariate group analyses(independent samples t test,ANOVA,ANCOVA) and assumes thatthe variance of the DV is roughly the same at all levels of the IV(Warner, 2008). The Levene’s test validates this assumption, wheresmaller statistics indicate greater homogeneity. Research (Boneau,1960; Glass et al., 1972) indicates that univariate group analysesare generally robust to moderate violations of homogeneity ofvariance as long as the sample sizes in each group are approximatelyequal. However, with unequal sample sizes, heterogeneitymay compromise the validity of null hypothesis decisions. Largesample variances from small-group sizes increase the risk of TypeI error. Large sample variances from large-group sizes increasethe risk of Type II error. When the assumption of homogeneityof variance is violated, researchers may conduct and report nonparametrictests such as the Kruskal–Wallis. However, Maxwell andDelaney (2004) noted that the Kruskal–Wallis test also assumesequal variances and suggested that data be either transformed tomeet the assumption of homogeneity of variance or analyzed withtests such as Brown–Forsythe F ∗ or Welch’s W.Homogeneity of regression applies to group analyses withcovariates, including ANCOVA and MANCOVA, and assumes thatthe regression between covariate(s) and DV(s) in one group is thesame as the regression in other groups (Tabachnick and Fidell,2001). This assumption can be examined graphically or by conductinga statistical test on the interaction between the COV(s)and the IV(s). Violation of this assumption can lead to very misleadingresults if covariance is used (Stevens, 2002). For example,in the case of heterogeneous slopes, group means that have beenadjusted by a covariate could indicate no difference when, in fact,group differences might exist at different values of the covariate. Ifheterogeneity of regression exists, ANCOVA and MANCOVA areinappropriate analytic strategies (Tabachnick and Fidell, 2001).Sphericity applies to repeated measures analyses that involvethree or more measurement occasions (repeated measuresANOVA) and assumes that the variances of the differences for allpairs of repeated measures are equal (Stevens, 2002). Presumingthat data are multivariate normal, the Mauchly test can be usedto test this assumption, where smaller statistics indicate greaterlevels of sphericity (Tabachnick and Fidell, 2001). Violating thesphericity assumption increases the risk of Type I error (Box,1954). To adjust for this risk and provide better control for Type Ierror rate, the degrees of freedom for the repeated measures F testmay be corrected using and reporting one of three adjustments:(a) Greenhouse–Geisser, (b) Huynh–Feldt, and (c) Lower-bound(see Nimon and Williams, 2009). Alternatively, researchers mayconduct and report analyses that do not assume sphericity (e.g.,MANOVA).Homoscedasticity applies to multiple linear regression andcanonical correlation and assumes that the variability in scores forone continuous variable is roughly the same at all values of anothercontinuous variable (Tabachnick and Fidell, 2001). Scatterplotsare typically used to test homoscedasticity. Linear regression isgenerally robust to slight violations of homoscedasticity; however,marked heteroscedasticity increases the risk of Type I error(Osborne and Waters, 2002). Canonical correlation performs bestwhen relationships among pairs of variables are homoscedastic(Tabachnick and Fidell, 2001). If the homoscedasticity assumptionis violated, researchers may delete outlying cases, transform data,or conduct non-parametric tests (see Conover, 1999; Osborne,2012), as long as the process is clearly reported.Homogeneity of variance-covariance matrix is a multivariategeneralization of homogeneity of variance. It applies to multivariategroup analyses (MANOVA and MANCOVA) and assumes thatthe variance-covariance matrix is roughly the same at all levelsof the IV (Stevens, 2002). The Box M test tests this assumption,where smaller statistics indicate greater homogeneity. Tabachnickand Fidell (2001) provided the following guidelines for interpretingviolations of this assumption: if sample sizes are equal,heterogeneity is not an issue. However, with unequal sample sizes,heterogeneity may compromise the validity of null hypothesisdecisions. Large sample variances from small-group sizes increasethe risk of Type I error whereas large sample variances from largegroupsizes increase the risk of Type II error. If sample sizes areunequal and the Box M test is significant at p < 0.001, researchersshould conduct the Pillai’s test or equalize sample sizes by randomdeletion of cases if power can be retained.DISCUSSIONWith the advances in statistical software, it is easy for researchers touse point and click methods to conduct a wide variety of statisticalanalyses on their datasets. However, the output from statisticalsoftware packages typically does not fully indicate if necessary statisticalassumptions have been met. I invite editors and reviewers touse the information presented in this article as a basic checklist ofthe statistical assumptions to be reported in scholarly reports. Thereferences cited in this article should also be helpful to researcherswho are unfamiliar with a particular assumption or how to test it.Frontiers in Psychology | Quantitative Psychology and Measurement August 2012 | Volume 3 | Article 322 | 74