Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

More documents

Recommendations

Info

Flora et al.Factor analysis assumptionsvariables is an estimate of the correlation between two unobserved,continuous y ∗ variables (Olsson, 1979) 13 . Adopting Eq. 3 leads toP ∗ = ΛΨΛ ′ + Θ,where P ∗ is the population correlation matrix among y ∗ , whichis estimated with the polychoric correlations. Next, because MLestimation provides model estimates that most likely would haveproduced observed multivariate normal data, methodologists donot recommend simply substituting polychoric R for productmomentR and proceed with ML estimation (e.g., Yang-Wallentinet al., 2010). Instead, we recommend ULS estimation, wherethe purpose is to minimize squared residual polychoric correlations(Muthén, 1978) 14 . The normality assumption for eachunobserved y ∗ is a mathematical convenience that allows estimationof P ∗ ; Flora and Curran (2004) showed that polychoriccorrelations remain reasonably accurate under moderate violationof this assumption, which then has only a very small effecton factor analysis results. Finally, this polychoric approach isimplemented in several popular SEM software packages (as wellas R, SAS, and Stata), each of which is capable of both EFAand CFA.EXAMPLE DEMONSTRATION CONTINUEDTo demonstrate limited-information item factor analysis, we conductedEFA of polychoric correlation matrices among the samecategorized variables for Cases 1–4 presented above, again using13 The tetrachoric correlation is a specific type of polychoric correlation that obtainswhen both observed variables are dichotomous.14 Alternatively,“robust weighted least-squares,” also known as “diagonally weightedleast-squares” (DWLS or WLSMV), is also commonly recommended. Simulationstudies suggest that ULS and DWLS produce very similar results, with slightly moreaccurate estimates obtained with ULS (Forero et al., 2009; Yang-Wallentin et al.,2010).ULS estimation and quartimin rotation. We begin with analysesof the approximately symmetric Case 1 and Case 2 items. First, thesample polychoric correlations are closer to the known populationcorrelations than were the product-moment correlations amongthese categorical variables (see Tables 5 and 7). For example,the population correlation between Word Meaning and SentenceCompletion is 0.75 and the sample polychoric correlation betweenthese two Case 1 items is 0.81,but the product-moment correlationwas only 0.60 (both with N = 100). As with product-moment R,the EFA of polychoric R among Case 1 variables also strongly suggestedretaining a three-factor model at both N = 100 and N = 200;the rotated factor-loading matrices are in Table 14. Here, the loadingof each observed variable on its primary factor is consistentlylarger than that obtained with the EFA of product-moment R(compare to Table 6) and much closer to the population factorloadings in Table 2. Next, with Case 2 variables, the EFA ofpolychoric R again led to a three-factor model at both N = 100and N = 200. The primary factor loadings in Table 15 were onlyslightly larger than those obtained with product-moment R (inTable 8), which were themselves reasonably close to the populationfactor loadings. Thus, in a situation where strong attenuationof product-moment correlations led to strong underestimationof factor loadings obtained from EFA of product-moment R (i.e.,Case 1), an alternate EFA of polychoric R produced relatively accuratefactor loadings. Yet, when attenuation of product-momentcorrelations is less severe (i.e., Case 2), EFA of polychoric R wasstill just as accurate.Recall that in Case 3 and 4, some items were positively skewedand some were negatively skewed. First, the polychoric correlationsamong Case 3 and 4 items are generally closer to thepopulation correlations than were the product-moment correlations,although many of the polychoric correlations among Case3 dichotomous items are quite inaccurate (see Tables 9 and 10).For example, the population correlation between Word MeaningTable 13 | Three-factor model loading matrix obtained with EFA ofproduct-moment R among Case 4 item-level variables.Table 14 | Three-factor model loading matrix obtained with EFA ofpolychoric R among Case 1 item-level variables.VariableFactorVariableFactorN = 100 N = 200N = 100 N = 200η 1 η 2 η 3 η 1 η 2 η 3η 1 η 2 η 3 η 1 η 2 η 3WrdMean 1.05 −0.10 −0.04 0.95 −0.10 −0.05SntComp 0.43 0.30 −0.05 0.50 0.23 −0.03OddWrds 0.54 0.05 0.32 0.68 0.04 0.14MxdArit 0.17 0.92 −0.04 0.16 0.94 −0.03Remndrs 0.17 0.40 0.25 0.16 0.46 0.17MissNum −0.02 0.87 0.06 0.03 0.85 0.00Gloves 0.00 0.10 0.47 0.00 0.06 0.46Boots 0.01 0.05 0.48 0.02 0.00 0.54Hatchts −0.01 −0.14 0.95 −0.03 −0.08 0.92WrdMean, word meaning; SntComp, sentence completion; OddWrds, odd words;MxdArit, mixed arithmetic; Remndrs, remainders; MissNum, missing numbers;Hatchts, hatchets. Primary loadings for each observed variable are in bold.WrdMean 0.95 −0.07 −0.05 0.94 −0.11 −0.02SntComp 0.85 0.14 −0.09 0.82 0.19 −0.05OddWrds 0.80 0.02 0.24 0.81 0.03 0.14MxdArit −0.02 0.91 0.04 −0.02 0.93 −0.02Remndrs −0.04 0.93 0.06 −0.05 0.86 0.16MissNum 0.12 0.82 −0.12 0.12 0.87 −0.10Gloves −0.01 0.22 0.51 0.04 0.03 0.59Boots 0.10 0.12 0.68 0.02 0.03 0.72Hatchts −0.01 −0.04 0.96 0.02 −0.01 0.92WrdMean, word meaning; SntComp, sentence completion; OddWrds, odd words;MxdArit, mixed arithmetic; Remndrs, remainders; MissNum, missing numbers;Hatchts, hatchets. Primary loadings for each observed variable are in bold.www.frontiersin.org March 2012 | Volume 3 | Article 55 | 117
Flora et al.Factor analysis assumptionsTable 15 | Three-factor model loading matrix obtained with EFA ofpolychoric R among Case 2 item-level variables.Table 16 | Three-factor model loading matrix obtained with EFA ofpolychoric R among Case 3 item-level variables.VariableFactorVariableFactorN = 100 N = 200η 1 η 2 η 3η 1 η 2 η 3 η 1 η 2 η 3WrdMean 0.99 −0.08 0.04 0.97 −0.07 0.02SntComp 0.75 0.17 −0.12 0.84 0.10 −0.08OddWrds 0.74 0.10 0.07 0.77 0.06 0.10MxdArit −0.06 1.00 −0.03 −0.06 1.00 −0.04Remndrs 0.11 0.73 0.10 0.04 0.80 0.12MissNum 0.13 0.77 0.06 0.15 0.81 0.00Gloves −0.01 0.20 0.61 −0.03 0.14 0.61Boots 0.04 −0.04 0.82 0.04 0.12 0.68Hatchts −0.02 0.00 0.80 0.01 −0.08 0.89WrdMean, word meaning; SntComp, sentence completion; OddWrds, odd words;MxdArit, mixed arithmetic; Remndrs, remainders; MissNum, missing numbers;Hatchts, hatchets. Primary loadings for each observed variable are in bold.WrdMean 0.52 0.27 0.18SntComp 0.96 0.03 −0.03OddWrds 0.44 0.34 0.32MxdArit 0.05 0.92 −0.07Remndrs 0.36 0.44 0.07MissNum 0.06 0.95 0.01Gloves −0.22 0.53 0.42Boots 0.22 −0.22 0.74Hatchts −0.09 0.11 0.82N = 200. WrdMean, word meaning; SntComp, sentence completion; OddWrds,dd words; MxdArit, mixed arithmetic; Remndrs, remainders; MissNum, missingnumbers; Hatchts, hatchets. Primary loadings for each observed variable are inbold.and Sentence Completion was 0.75 while the sample polychoriccorrelation between these oppositely skewed Case 3 items was0.57 with N = 100 and improved to 0.67 with N = 200; but theproduct-moment correlation between these two items was only0.24 with N = 100 and 0.25 with N = 200. With five-categoryCase 4 items, the polychoric correlation between Word Meaningand Sentence Completion is 0.81 with both sample sizes, but theproduct-moment correlations were only 0.52 with N = 100 and0.51 with N = 200.Given that the polychoric correlations were generally moreresistant to skewness in observed items than product-momentcorrelations, factor analyses of Case 3 and 4 items should also beimproved. With the dichotomous Case 3 variables and N = 100,the scree plot suggested retaining a one-factor model, but other fitstatistics did not support a one-factor model (e.g., RMSR = 0.14).Yet, the estimated two- and three-factor models were improperwith negative residual variance estimates. This outcome likelyoccurred because many of the bivariate frequency tables for itempairs had cells with zero frequency. This outcome highlights thegeneral concern of sparseness, or the tendency of highly skeweditems to produce observed bivariate distributions with cell frequenciesequaling zero or close to zero, especially with relativelysmall overall sample size. Sparseness can cause biased polychoriccorrelation estimates, which in turn leads to inaccurate factoranalysis results (Olsson, 1979; Savalei, 2011). With N = 200, athree-factor model for Case 3 variables was strongly supported;the rotated factor loading matrix is in Table 16. Excluding Gloves,each item has its strongest loading on the same factor as indicatedin the population (see Table 2), but many of these primary factorloadings are strongly biased and many items have moderate crossloadings.Thus, we see an example of the tendency for EFA ofpolychoric R to produce inaccurate results with skewed dichotomousitems. Nonetheless, recall that EFA of product-moment Rfor Case 3 items did not even lead to a model with the correctnumber of factors.Table 17 | Three-factor model loading matrix obtained with EFA ofpolychoric R among Case 4 item-level variables.VariableFactorη 1 η 2 η 3WrdMean 0.97 −0.08 −0.02SntComp 0.85 0.09 −0.04OddWrds 0.70 0.11 0.17MxdArit −0.03 1.01 −0.05Remndrs −0.02 0.83 0.12MissNum 0.12 0.81 −0.02Gloves 0.00 0.11 0.49Boots 0.07 −0.02 0.69Hatchts −0.02 −0.01 0.99N = 200. WrdMean, word meaning; SntComp, sentence completion; OddWrds,odd words; MxdArit, mixed arithmetic; Remndrs, remainders; MissNum, missingnumbers; Hatchts, hatchets. Primary loadings for each observed variable are inbold.With the five-category Case 4 items, retention of a three-factormodel was supported at both sample sizes. However, with N = 100,we again obtained improper estimates. But with N = 200, therotated factor loading matrix (Table 17) is quite accurate relativeto the population factor loadings, and certainly improvedover that obtained with EFA of product-moment R among theseitems. Thus, even though there was a mix of positively andnegatively skewed items, having a larger sample size and moreitem-response categories mitigated the potential for sparseness toproduce inaccurate results from the polychoric EFA.In sum, our demonstration illustrated that factor analysesof polychoric R among categorized variables consistently outperformedanalyses of product-moment R for the same variables.In particular, with symmetric items (Cases 1 and 2),Frontiers in Psychology | Quantitative Psychology and Measurement March 2012 | Volume 3 | Article 55 | 118
Page 2 and 3:
FRONTIERS COPYRIGHTSTATEMENT© Copy
Page 4 and 5:
Table of Contents05 Is Data Cleanin
Page 7 and 8:
OsborneAssumptions and data cleanin
Page 9 and 10:
ORIGINAL RESEARCH ARTICLEpublished:
Page 12 and 13:
Hoekstra et al.Why assumptions are
Page 14 and 15:
Page 16 and 17:
Page 18 and 19:
Page 20 and 21:
García-PérezStatistical conclusio
Page 22 and 23:
Page 24 and 25:
Page 26 and 27:
Page 28 and 29:
Page 30 and 31:
Sheng and ShengEffect of non-normal
Page 32 and 33:
Page 34 and 35:
Page 36 and 37:
Page 38 and 39:
Page 40 and 41:
Page 42 and 43:
REVIEW ARTICLEpublished: 12 April 2
Page 44 and 45:
Nimon et al.The assumption of relia
Page 46 and 47:
Page 48 and 49:
Page 50 and 51:
Page 52 and 53:
Page 54 and 55:
Page 56 and 57:
TressoldiPower replication unreliab
Page 58 and 59:
TressoldiPower replication unreliab
Page 60 and 61:
Page 62 and 63:
FinchModern methods for the detecti
Page 64 and 65:
Page 66 and 67:
Page 68 and 69: FinchModern methods for the detecti
Page 70 and 71: FinchModern methods for the detecti
Page 72 and 73: MINI REVIEW ARTICLEpublished: 28 Au
Page 74 and 75: NimonStatistical assumptionsand Del
Page 76 and 77: NimonStatistical assumptionsFor exa
Page 78 and 79: Kraha et al.Interpreting multiple r
Page 94 and 95: SmithsonComparing moderation of slo
Page 102 and 103: REVIEW ARTICLEpublished: 01 March 2
Page 104 and 105: Flora et al.Factor analysis assumpt
Page 124 and 125: Kasper and ÜnlüAssumptions of fac
Page 144 and 145: Lages and JaworskaHow predictable a
Page 152 and 153: Cummiskey et al.Testing assumptions
Page 158: Cummiskey et al.Testing assumptions
show all

Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?