Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

More documents

Recommendations

Info

Flora et al.Factor analysis assumptionsof a two-factor solution (RMSR = 0.056 for Case 3 and = 0.077for Case 4). In practice, it is wise to compare results for modelswith varying numbers of factors, and here we know that the populationmodel has three factors. Thus, we estimated both two- andthree-factor models for the Case 3 and Case 4 items. In Case 3, theestimated three-factor models obtained with both N = 100 andN = 200 were improper in that there were variables with negativeestimated residual variance (Heywood cases); thus, in practicea researcher would typically reject the three-factor model andinterpret the two-factor model. Table 11 gives the rotated factorpattern for the two-factor models estimated with Case 3 items.Here, the two factors are essentially defined by the skewness directionof the observed variables: odd-numbered items, which areall positively skewed, are predominately determined by η 1 whilethe negatively skewed even-numbered items are determined by η 2(with the exception of the Boots variable). A similar factor patternemerges for the two-factor model estimated with N = 100 for theTable8|Factorloading matrix obtained with EFA of product-momentR among Case 2 item-level variables.VariableFactorN = 100 N = 200η 1 η 2 η 3 η 1 η 2 η 3WrdMean 0.94 −0.10 0.05 0.94 −0.07 0.02SntComp 0.73 0.13 −0.10 0.82 0.09 −0.08OddWrds 0.72 0.07 0.09 0.74 0.04 0.10MxdArit 0.11 0.98 −0.03 0.04 0.96 −0.04Remndrs 0.11 0.70 0.09 0.04 0.76 0.12MissNum 0.13 0.73 0.06 0.14 0.78 −0.01Gloves −0.01 0.17 0.59 −0.02 0.10 0.60Boots 0.03 −0.05 0.81 0.04 0.07 0.67Hatchts 0.00 −0.02 0.75 0.01 −0.13 0.86WrdMean, word meaning; SntComp, sentence completion; OddWrds, odd words;MxdArit, mixed arithmetic; Remndrs, remainders; MissNum, missing numbers;Hatchts, hatchets. Primary loadings for each observed variable are in bold.Case 4 items, but not with N = 200 (see Table 12). Finally, thefactor pattern for the three-factor model estimated with Case 4itemsisinTable 13. Here, the factors have the same basic interpretationas with the population model, but some of the individualfactor loadings are quite different. For example, at both samplesizes, the estimated primary factor loadings for Sentence Completion(0.43 with N = 100) and Remainders (0.40 with N = 100) aremuch smaller than their population values of 0.77 and 0.80.In general, the problems with factoring product-moment Ramong item-level variables are less severe when there are moreresponse categories (e.g., more than five) and the observed univariateitem distributions are symmetric (Finney and DiStefano,2006). Our demonstration above is consistent with this statement.Yet, although there are situations where ordinary linear regressionof a categorical dependent variable can potentially produceuseful results, the field still universally accepts non-linear modelingmethods such as logistic regression as standard for limiteddependent variables. Similarly, although there may be situationswhere factoring product-moment R (and hence adapting a linearfactor model) produces reasonably accurate results, the fieldshould accept alternative, non-linear factor models as standard forcategorical, item-level observed variables.ALTERNATIVE METHODS FOR ITEM-LEVEL OBSERVEDVARIABLESWirth and Edwards (2007) give a comprehensive review of methodsfor factor analyzing categorical item-level variables. In general,these methods can be classified as either limited-information orfull-information. The complete data for N participants on p categorical,item-level variables form a multi-way frequency tablewith CjP cells (i.e., C 1 × C 2 × ...× C p ), or potential responsepatterns, where C j is the number of categories for item j. Fullinformationfactor models draw from multidimensional itemresponsetheory (IRT) to predict directly the probability thata given individual’s response pattern falls into a particular cellof this multi-way frequency table (Bock et al., 1988). Limitedinformationmethods instead fit the factor model to a set ofintermediate summary statistics which are calculated from theobserved frequency table. These summary statistics include theunivariate response proportions for each item and the bivariateTable9|Product-moment and polychoric correlations among Case 3 item-level variables.Variable 1 2 3 4 5 6 7 8 9WrdMean 1 0.57 0.71 0.35 0.42 0.54 0.30 0.17 0.61SntComp 0.24 1 0.54 0.57 0.50 0.47 0.26 0.45 0.08OddWrds 0.46 0.22 1 0.60 0.57 0.61 0.64 0.46 0.66MxdArit 0.16 0.35 0.26 1 0.57 0.83 0.52 0.03 0.19Remndrs 0.23 0.21 0.33 0.24 1 0.58 0.30 0.22 0.64MissNum 0.23 0.28 0.26 0.61 0.25 1 0.54 0.29 0.52Gloves 0.16 0.11 0.39 0.22 0.16 0.23 1 0.32 0.61Boots 0.08 0.26 0.18 0.02 0.09 0.17 0.14 1 0.47Hatchts 0.37 0.04 0.41 0.09 0.39 0.22 0.37 0.19 1N = 100. Product-moment correlations are below the diagonal; polychoric correlations are above the diagonal. WrdMean, word meaning; SntComp, sentencecompletion; OddWrds, odd words; MxdArit, mixed arithmetic; Remndrs, remainders; MissNum, missing numbers; Hatchts, hatchets.www.frontiersin.org March 2012 | Volume 3 | Article 55 | 115
Flora et al.Factor analysis assumptionsTable 10 | Product-moment and polychoric correlations among Case 4 item-level variables.Variable 1 2 3 4 5 6 7 8 9WrdMean 1 0.81 0.75 0.42 0.47 0.41 0.21 0.21 0.29SntComp 0.52 1 0.62 0.49 0.44 0.55 0.22 0.34 0.13OddWrds 0.66 0.42 1 0.54 0.47 0.53 0.36 0.42 0.50MxdArit 0.31 0.40 0.38 1 0.82 0.86 0.32 0.25 0.27Remndrs 0.43 0.31 0.41 0.54 1 0.76 0.41 0.36 0.42MissNum 0.32 0.46 0.38 0.79 0.51 1 0.35 0.30 0.33Gloves 0.20 0.14 0.33 0.25 0.31 0.28 1 0.35 0.51Boots 0.15 0.29 0.29 0.18 0.24 0.25 0.27 1 0.68Hatchts 0.29 0.12 0.46 0.19 0.38 0.27 0.42 0.45 1N = 100. Product-moment correlations are below the diagonal; polychoric correlations are above the diagonal. WrdMean, word meaning; SntComp, sentencecompletion; OddWrds, odd words; MxdArit, mixed arithmetic; Remndrs, remainders; MissNum, missing numbers; Hatchts, hatchets.Table 11 | Two-factor model loading matrix obtained with EFA ofproduct-moment R among Case 3 item-level variables.Table 12 | Two-factor model loading matrix obtained with EFA ofproduct-moment R among Case 4 item-level variables.VariableFactorVariableFactorN = 100 N = 200N = 100 N = 200η 1 η 2 η 1 η 2η 1 η 2 η 1 η 2WrdMean 0.53 0.02 0.60 0.00SntComp 0.12 0.37 0.24 0.22OddWrds 0.69 0.04 0.75 −0.06MxdArit −0.13 0.95 −0.01 0.76Remndrs 0.42 0.11 0.37 0.13MissNum 0.12 0.62 −0.01 0.87Gloves 0.44 0.06 0.35 0.07Boots 0.25 0.03 0.32 −0.04Hatchts 0.78 −0.19 0.43 −0.03WrdMean, word meaning; SntComp, sentence completion; OddWrds, odd words;MxdArit, mixed arithmetic; Remndrs, remainders; MissNum, missing numbers;Hatchts, hatchets. Primary loadings for each observed variable are in bold.WrdMean 0.49 0.21 0.46 0.27SntComp 0.23 0.40 0.55 0.12OddWrds 0.68 0.14 0.46 0.36MxdArit −0.11 0.96 0.87 −0.17Remndrs 0.34 0.42 0.58 0.14MissNum 0.02 0.84 0.88 −0.13Gloves 0.47 0.04 0.06 0.45Boots 0.49 −0.01 0.00 0.56Hatchts 0.77 −0.15 −0.06 0.79WrdMean, word meaning; SntComp, sentence completion; OddWrds, odd words;MxdArit, mixed arithmetic; Remndrs, remainders; MissNum, missing numbers;Hatchts, hatchets. Primary loadings for each observed variable are in bold.polychoric correlations among items. Hence, these methods arereferred to as “limited-information” because they collapse thecomplete multi-way frequency data into univariate and bivariatemarginal information. Full-information factor analysis is an activearea of methodological research, but the limited-informationmethod has become quite popular in applied settings and simulationstudies indicate that it performs well across a range ofsituations (e.g., Flora and Curran, 2004; Forero et al., 2009; alsosee Forero and Maydeu-Olivares, 2009, for a study comparing fullandlimited-information modeling). Thus, our remaining presentationfocuses on the limited-information, polychoric correlationapproach.The key idea to this approach is the assumption that anunobserved, normally distributed continuous variable, y ∗ , underlieseach categorical, ordinally scaled observed variable, y withresponse categories c = 0, 1,..., C. The latent y ∗ links to theobserved y according toy = cif τ c−1 < y ∗ < τ c ,where each τ is a threshold parameter (i.e., a Z-score) determinedfrom the univariate proportions of y with τ 0 =−∞and τ C =∞.Adopting Eq. 1, the common factor model is then a model for they ∗ variables themselves 11 ,y ∗ = Λη + ε.Like factor analysis of continuous variables, the model parametersare actually estimated from the correlation structure,where thecorrelations among the y ∗ variables are polychoric correlations 12 .Thus, the polychoric correlation between two observed, ordinal y11 The factor model can be equivalently conceptualized as a probit regression forthe categorical dependent variables. Probit regression is nearly identical to logisticregression, but the normal cumulative distribution function is used in place of thelogistic distribution (see Fox, 2008).12 Analogous to the incorporation of mean structure among continuous variables,advanced models such as multiple-group measurement models are fitted to the setof both estimated thresholds and polychoric correlations.Frontiers in Psychology | Quantitative Psychology and Measurement March 2012 | Volume 3 | Article 55 | 116
Page 2 and 3:
FRONTIERS COPYRIGHTSTATEMENT© Copy
Page 4 and 5:
Table of Contents05 Is Data Cleanin
Page 7 and 8:
OsborneAssumptions and data cleanin
Page 9 and 10:
ORIGINAL RESEARCH ARTICLEpublished:
Page 12 and 13:
Hoekstra et al.Why assumptions are
Page 14 and 15:
Page 16 and 17:
Page 18 and 19:
Page 20 and 21:
García-PérezStatistical conclusio
Page 22 and 23:
Page 24 and 25:
Page 26 and 27:
Page 28 and 29:
Page 30 and 31:
Sheng and ShengEffect of non-normal
Page 32 and 33:
Page 34 and 35:
Page 36 and 37:
Page 38 and 39:
Page 40 and 41:
Page 42 and 43:
REVIEW ARTICLEpublished: 12 April 2
Page 44 and 45:
Nimon et al.The assumption of relia
Page 46 and 47:
Page 48 and 49:
Page 50 and 51:
Page 52 and 53:
Page 54 and 55:
Page 56 and 57:
TressoldiPower replication unreliab
Page 58 and 59:
TressoldiPower replication unreliab
Page 60 and 61:
Page 62 and 63:
FinchModern methods for the detecti
Page 64 and 65:
FinchModern methods for the detecti
Page 66 and 67: FinchModern methods for the detecti
Page 72 and 73: MINI REVIEW ARTICLEpublished: 28 Au
Page 74 and 75: NimonStatistical assumptionsand Del
Page 76 and 77: NimonStatistical assumptionsFor exa
Page 78 and 79: Kraha et al.Interpreting multiple r
Page 94 and 95: SmithsonComparing moderation of slo
Page 102 and 103: REVIEW ARTICLEpublished: 01 March 2
Page 104 and 105: Flora et al.Factor analysis assumpt
Page 124 and 125: Kasper and ÜnlüAssumptions of fac
Page 144 and 145: Lages and JaworskaHow predictable a
Page 152 and 153: Cummiskey et al.Testing assumptions
Page 158: Cummiskey et al.Testing assumptions
show all

Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

Create successful ePaper yourself

Delete template?

Save as template?