Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

More documents

Recommendations

Info

Flora et al.Factor analysis assumptionsFIGURE 5 | Scatterplot of “Remainders” by “Mixed Arithmetic” for perturbed sample with influential case indicated.Table4|Factorloading matrix obtained with perturbed sample data.VariableFactorη 1 η 2 η 3WrdMean 0.97 −0.13 0.09SntComp 0.70 0.29 −0.23OddWrds 0.74 −0.01 0.21MxdArit −0.07 1.01 0.03Remndrs 0.17 0.49 0.32MissNum 0.08 0.81 0.06Gloves 0.01 0.09 0.68Boots 0.05 0.12 0.44Hatchts 0.02 −0.08 0.89N = 100. WrdMean, word meaning; SntComp, sentence completion; OddWrds,odd words; MxdArit, mixed arithmetic; Remndrs, remainders; MissNum, missingnumbers; Hatchts, hatchets. Primary loadings for each observed variable are inbold.effect of an individual case on model fit with ML estimation canbe formally measured with an influence statistic known as likelihooddistance, which measures the difference in the likelihood ofthe model when a potentially influential case is deleted (Pek andMacCallum, 2011).Upon discovering unusual cases, it is important to determinetheir likely source. Often, outliers and influential cases arise fromeither researcher error (e.g., data entry error or faulty administrationof study procedures) or participant error (e.g., misunderstandingof study instructions or non-compliance with randomresponding) or they may be observations from a populationother than the population of interest (e.g., a participant with nohistory of depression included in a study of depressed individuals).In these situations, it is best to remove such cases fromthe data set. Conversely, if unusual cases are simply extremecases with otherwise legitimate values, most methodologists recommendthat they not be deleted from the data set prior tomodel fitting (e.g., Bollen and Arminger, 1991; Yuan and Zhong,2008; Pek and MacCallum, 2011). Instead, robust procedures thatminimize the excessive influence of extreme cases are recommended;in particular, case-robust methods developed by Yuanand Bentler (1998) are implemented in the EQS software package(Bentler, 2004) or one can factor analyze a minimum covariancedeterminant (MCD) estimated covariance matrix (Pisonet al., 2003), which can be calculated with SAS or the R package“MASS.”COLLINEARITYAnother potential concern for both multiple regression analysisand factor analysis is collinearity, which refers to perfect ornear-perfect linear relationships among observed variables. Withmultiple regression, the focus is on collinearity among explanatoryvariables, but with factor analysis, the concern is collinearityamong dependent variables, that is, the set of variables being factoranalyzed. When collinear variables are included, the productmomentcorrelation matrix R will be singular, ornon-positivedefinite. ML estimation cannot be used with a singular R, andalthough ULS is possible, collinearity is still indicative of conceptualissues with variable selection. Collinearity in factor analysisis relatively simple to diagnose: if any eigenvalues of a productmomentR equal zero or are negative, then R is non-positivedefinite and collinearity is present (and software will likely producewww.frontiersin.org March 2012 | Volume 3 | Article 55 | 109
Flora et al.Factor analysis assumptionsFIGURE 6 | Histograms of standardized residuals for each observed variable from three-factor model fitted to perturbed sample data (N = 100).a warning message) 7 . Eigenvalues extremely close to zero may alsobe suggestive of near-collinearity 8 . A commonly used statistic forevaluating near-collinearity in ordinary regression is the conditionindex, which equals the square root of the ratio of the largest tosmallest eigenvalue, with larger values more strongly indicativeof near-collinearity. For the current example, the condition indexequals 6.34, which is well below the value of 30 suggested by Belsleyet al. (1980) as indicative of problematic near-collinearity.In the context of factor analysis, collinearity often implies anill-conceived selection of variables for the analysis. For example,if both total scores and sub-scale scores (or just one sub-scale)from the same instrument are included in a factor analysis, thetotal score is linearly dependent on, or collinear with, the subscalescore. A more subtle example may be the inclusion of both a“positive mood”scale and a“negative mood”scale; although scoresfrom these two scales may not be perfectly related, they will likely7 Polychoric correlation matrices (defined below) are often non-positive definite.Unlike a product-moment R, a non-positive definite polychoric correlation matrixis not necessarily problematic.8 Most EFA software includes eigenvalues of R as default output. However, it isimportant not to confuse the eigenvalues of R with the eigenvalues of the “reduced”correlation matrix, R − ˆΘ. The reduced correlation matrix often has negativeeigenvalues when R does not.have a very strong (negative) correlation,which can be problematicfor factor analysis. In these situations, researchers should carefullyreconsider their choice of variables for the analysis and remove anythat is collinear with one or more of the other observed variables,which in turn redefines the overall research question addressed bythe analysis (see Fox, 2008, p. 342).NON-NORMALITYIn terms of obtaining accurate parameter estimates, the ULS estimationmethod mentioned above makes no assumption regardingobserved variable distributions whereas ML estimation is based onthe multivariate normal distribution (MacCallum, 2009). Specifically,for both estimators, parameter estimates are trustworthy aslong as the sample size is large and the model is properly specified(i.e., the estimator is consistent), even when the normality assumptionfor ML is violated (Bollen, 1989). However, SE estimates andcertain model fit statistics (i.e., the χ 2 fit statistic and statisticsbased on χ 2 such as CFI and RMSEA) are adversely affected bynon-normality, particularly excess multivariate kurtosis. Althoughthey are less commonly used with EFA, SEs and model fit statisticsare equally applicable with EFA as with CFA and are easily obtainedwith modern software (but see Cudeck and O’Dell, 1994, for cautionsand recommendations regarding SEs with EFA). With bothapproaches,SEs convey information about the sampling variabilityFrontiers in Psychology | Quantitative Psychology and Measurement March 2012 | Volume 3 | Article 55 | 110
Page 2 and 3:
FRONTIERS COPYRIGHTSTATEMENT© Copy
Page 4 and 5:
Table of Contents05 Is Data Cleanin
Page 7 and 8:
OsborneAssumptions and data cleanin
Page 9 and 10:
ORIGINAL RESEARCH ARTICLEpublished:
Page 12 and 13:
Hoekstra et al.Why assumptions are
Page 14 and 15:
Page 16 and 17:
Page 18 and 19:
ORIGINAL RESEARCH ARTICLEpublished:
Page 20 and 21:
García-PérezStatistical conclusio
Page 22 and 23:
Page 24 and 25:
Page 26 and 27:
Page 28 and 29:
Page 30 and 31:
Sheng and ShengEffect of non-normal
Page 32 and 33:
Page 34 and 35:
Page 36 and 37:
Page 38 and 39:
Page 40 and 41:
Page 42 and 43:
REVIEW ARTICLEpublished: 12 April 2
Page 44 and 45:
Nimon et al.The assumption of relia
Page 46 and 47:
Page 48 and 49:
Page 50 and 51:
Page 52 and 53:
Page 54 and 55:
Page 56 and 57:
TressoldiPower replication unreliab
Page 58 and 59:
TressoldiPower replication unreliab
Page 60 and 61: ORIGINAL RESEARCH ARTICLEpublished:
Page 62 and 63: FinchModern methods for the detecti
Page 72 and 73: MINI REVIEW ARTICLEpublished: 28 Au
Page 74 and 75: NimonStatistical assumptionsand Del
Page 76 and 77: NimonStatistical assumptionsFor exa
Page 78 and 79: Kraha et al.Interpreting multiple r
Page 94 and 95: SmithsonComparing moderation of slo
Page 102 and 103: REVIEW ARTICLEpublished: 01 March 2
Page 104 and 105: Flora et al.Factor analysis assumpt
Page 124 and 125: Kasper and ÜnlüAssumptions of fac
Page 144 and 145: Lages and JaworskaHow predictable a
Page 152 and 153: Cummiskey et al.Testing assumptions
Page 158: Cummiskey et al.Testing assumptions
show all

Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?