Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

More documents

Recommendations

Info

Flora et al.Factor analysis assumptionsFIGURE 7 | Distribution of Mahalanobis distance (MD) for perturbed sample data (N = 100).FIGURE 8 | Distribution of generalized Cook’s distance (gCD) for perturbed sample data (N = 100).of the parameter estimates (i.e., they are needed for significancetests and forming confidence intervals) while model fit statisticscan aid decisions about the number of common factors in EFA ormore subtle model misspecification in CFA.There is a large literature on ramifications of non-normalityfor SEM (in which the common factor model is imbedded), andprocedures for handling non-normal data (for reviews see Bollen,1989; West et al., 1995; Finney and DiStefano, 2006). In particular,for CFA we recommend using the Satorra–Bentler scaled χ 2 androbust SEs with non-normal continuous variables (Satorra andBentler, 1994), which is available in most SEM software. Althoughthese Satorra–Bentler procedures for non-normal data have seenlittle application in EFA, they can be obtained for EFA modelsusing Mplus software (Muthén and Muthén,2010; i.e.,using the SEestimation procedure outlined in Asparouhov and Muthén, 2009).Alternatively, one might factor analyze transformed observed variablesthat more closely approximate normal distributions (e.g.,Gorsuch, 1983, pp. 297–309).FACTOR ANALYSIS OF ITEM-LEVEL OBSERVED VARIABLESThrough its history in psychometrics, factor analysis developedprimarily in the sub-field of cognitive ability testing, whereresearchers sought to refine theories of intelligence using factoranalysis to understand patterns of covariation among differentability tests. Scores from these tests typically elicited continuouslydistributed observed variables, and thus it was natural for factoranalysis to develop as a method for analyzing Pearson productmomentcorrelations and eventually to be recognized as a linearmodel for continuous observed variables (Bartholomew, 2007).However, modern applications of factor analysis usually use individualtest items rather than sets of total test scores as observedvariables. Yet, because the most common kinds of test items, suchas Likert-type items, produce categorical (dichotomous or ordinal)rather than continuous distributions, a linear factor analysismodel using product-moment R is suboptimal, as we illustratebelow.As early as Ferguson (1941), methodologists have shown thatfactor analysis of product-moment R among dichotomous variablescan produce misleading results. Subsequent research hasfurther established that treating categorical items as continuousvariables by factor analyzing product-moment R can lead to incorrectdecisions about the number of common factors or overallmodel fit, biased parameter estimates, and biased SE estimates(Muthén and Kaplan, 1985, 1992; Babakus et al., 1987; Bernsteinand Teng, 1989; Dolan, 1994; Green et al., 1997). Despite theseissues, item-level factor analysis using product-moment R persistsin the substantive literature likely because of either naivetéabout the categorical nature of items or misinformed belief thatwww.frontiersin.org March 2012 | Volume 3 | Article 55 | 111
Flora et al.Factor analysis assumptionslinear factor analysis is “robust” to the analysis of categoricalitems.EXAMPLE DEMONSTRATIONTo illustrate these potential problems using our running example,we categorized random samples of continuous variables withN = 100 and N = 200 that conform to the three-factor populationmodel presented in Table 2 according to four separate cases of categorization.In Case 1, all observed variables were dichotomizedso that each univariate population distribution had a proportion= 0.5 in both categories. In Case 2, five-category items werecreated so that each univariate population distribution was symmetricwith proportions of 0.10, 0.20, 0.40, 0.20, and 0.10 for thelowest to highest categorical levels, or item-response categories.Next, Case 3 items were dichotomous with five variables (oddnumbereditems) having univariate response proportions of 0.80and 0.20 for the first and second categories and the other four variables(even-numbered items) having proportions of 0.20 and 0.80for the first and second categories. Finally, five-category items werecreated for Case 4 with odd-numbered items having positive skewness(response proportions of 0.51, 0.30, 0.11, 0.05, and 0.03 forthe lowest to highest categories) and even-numbered items havingnegative skewness (response proportions of 0.03, 0.05, 0.11, 0.30,and 0.51). For each of the four cases at both sample sizes, we conductedEFA using the product-moment R among the categoricalvariables and applied quartimin rotation after determining theoptimal number of common factors suggested by the sample data.If product-moment correlations are adequate representations ofthe true relationships among these items, then three-factor modelsshould be supported and the rotated factor pattern shouldapproximate the population factor loadings in Table 2. We describeanalyses for Cases 1 and 2 first, as both of these cases consisted ofitems with approximately symmetric univariate distributions.First, Figure 9 shows the simple bivariate scatterplot for thedichotomized versions of the Word Meaning and Sentence CompletionitemsfromCase1(N= 100). We readily admit that thisfigure is not a good display for these data; instead, its crudeness isintended to help illustrate that it is often not appropriate to pretendthat categorical variables are continuous. When variables arecontinuous, bivariate scatterplots (such as those in Figure 1) arevery useful, but Figure 9 shows that they are not particularly usefulfor dichotomous variables, which in turn should cast doubt on theusefulness of a product-moment correlation for such data. Morespecifically, because these two items are dichotomized (i.e., 0, 1),there are only four possible observed data patterns, or responsepatterns, for their bivariate distribution (i.e., 0, 0; 0, 1; 1, 0; and1, 1). These response patterns represent the only possible pointsin the scatterplot. Yet, depending on the strength of relationshipbetween the two variables, there is some frequency of observationsassociated with each point, as each represents potentiallyFIGURE 9 | Scatterplot of Case 1 items Word Meaning (WrdMean) by Sentence Completion (SntComp; N = 100).Frontiers in Psychology | Quantitative Psychology and Measurement March 2012 | Volume 3 | Article 55 | 112
Page 2 and 3:
FRONTIERS COPYRIGHTSTATEMENT© Copy
Page 4 and 5:
Table of Contents05 Is Data Cleanin
Page 7 and 8:
OsborneAssumptions and data cleanin
Page 9 and 10:
ORIGINAL RESEARCH ARTICLEpublished:
Page 12 and 13:
Hoekstra et al.Why assumptions are
Page 14 and 15:
Page 16 and 17:
Page 18 and 19:
Page 20 and 21:
García-PérezStatistical conclusio
Page 22 and 23:
Page 24 and 25:
Page 26 and 27:
Page 28 and 29:
Page 30 and 31:
Sheng and ShengEffect of non-normal
Page 32 and 33:
Page 34 and 35:
Page 36 and 37:
Page 38 and 39:
Page 40 and 41:
Page 42 and 43:
REVIEW ARTICLEpublished: 12 April 2
Page 44 and 45:
Nimon et al.The assumption of relia
Page 46 and 47:
Page 48 and 49:
Page 50 and 51:
Page 52 and 53:
Page 54 and 55:
Page 56 and 57:
TressoldiPower replication unreliab
Page 58 and 59:
TressoldiPower replication unreliab
Page 60 and 61:
Page 62 and 63: FinchModern methods for the detecti
Page 72 and 73: MINI REVIEW ARTICLEpublished: 28 Au
Page 74 and 75: NimonStatistical assumptionsand Del
Page 76 and 77: NimonStatistical assumptionsFor exa
Page 78 and 79: Kraha et al.Interpreting multiple r
Page 94 and 95: SmithsonComparing moderation of slo
Page 102 and 103: REVIEW ARTICLEpublished: 01 March 2
Page 104 and 105: Flora et al.Factor analysis assumpt
Page 124 and 125: Kasper and ÜnlüAssumptions of fac
Page 144 and 145: Lages and JaworskaHow predictable a
Page 152 and 153: Cummiskey et al.Testing assumptions
Page 158: Cummiskey et al.Testing assumptions
show all

Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?