Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

More documents

Recommendations

Info

Kasper and ÜnlüAssumptions of factor analytic approaches0.2 0.4 0.6 0.8number of extracted factors109876543210987654321098765432PLPCSTPCKGPC0.2 0.4 0.6 0.8PLEASTEAKGEArelative frequencyPLPASTPAKGPA0.2 0.4 0.6 0.8Normal Slightly skewed Strongly skewedFIGURE 2 | Relative frequencies of the numbers of extracted factors, for n = 200 and k = 4. Factor models are principal component analysis (PCA, or PC),exploratory factor analysis (EFA, or EA), and principal axis analysis (PAA, or PA). Kaiser-Guttman criterion (KG), scree test (ST), and parallel analysis (PL) serve asfactor extraction criteria.words, strongly negative skewed distributions may not be estimatedwithout bias based on the classical factor models. Increasingsample size, for example from n = 200 to 600, or changing thenumber of underlying factors, say from k = 4 to 8, did not alterthis observation considerably. For that reason, the correspondingplots at this point of the paper are omitted and can be found inKasper (2012).We performed Shapiro-Wilk tests for univariate normality ofthe estimated factor scores. As can be seen from Figure 7A, undernormally distributed true latent ability scores nearly all values of Ware statistically non-significant. In these cases, the null hypothesiscannot be rejected.A similar conclusion can be drawn when the true latent abilityvalues are not normally distributed but instead follow a slightlyskewed distribution (Figure 7B). Nearly all Shapiro-Wilk test statisticvalues are statistically non-significant. In other words, thenull hypothesis stating normally distributed latent ability valuesis seldom rejected although the true latent distribution is skewedand not normal. No relationship between the p-values and the usedfactor model or factor position may be apparent (disregarding theobservation that the p-values for the fourth factor are generallylower than for the other factors).The case of a strongly skewed factor score distribution isdepicted in Figure 7C. Virtually all values of W are statisticallysignificant and the null hypothesis of normality of factor scoresis rejected. Similar conclusions or observations may be drawn forincreased sample size or factor space dimension and we do omitpresenting plots thereof.Finally, Figure 8 shows the distribution of the estimated factorscores on the fourth factor (for k = 4) in comparison to the truestrongly skewed ability distribution under the exploratory factoranalysis model for a sample size of n = 1,000. The unit normaldistribution is plotted as a reference. The estimated factor scoreshave a skewness value of −0.47 compared to true skewness −2.The estimated distribution deviates from the true distribution anddoes not approximate it acceptably well.Frontiers in Psychology | Quantitative Psychology and Measurement March 2013 | Volume 4 | Article 109 | 131
Kasper and ÜnlüAssumptions of factor analytic approaches0.2 0.4 0.6 0.8number of extracted factors876543287654328765432PLPCSTPCKGPCPLEASTEAKGEAPLPASTPAKGPA0.2 0.4 0.6 0.80.2 0.4 0.6 0.8relative frequencyNormal Slightly skewed Strongly skewedFIGURE 3 | Relative frequencies of the numbers of extracted factors, for n = 600 and k = 4. Factor models are principal component analysis (PCA, or PC),exploratory factor analysis (EFA, or EA), and principal axis analysis (PAA, or PA). Kaiser-Guttman criterion (KG), scree test (ST), and parallel analysis (PL) serve asfactor extraction criteria.5.3. DISCREPANCY BETWEEN THE ESTIMATED AND THE TRUELOADING MATRIXIn Table 4, the average and standard deviation coefficients ¯D ands for the discrepancies are reported. The largest average discrepancyvalues are obtained for the condition n = 200, k = 8, and thestrongly skewed latent ability distribution: 0.173, 0.157, and 0.143for PCA, EFA, and PAA, respectively. Under this condition, the truefactor loadings are, mostly or clearly, overestimated or underestimated.Minor differences between the estimated and true factorloadings are obtained for n = 600, k = 4, and the normal latentability distribution: with average discrepancies 0.076, 0.063, and0.066 for PCA, EFA, and PAA, respectively.Deviations of the estimated loading matrix from the true loadingmatrix can also be quantified and visualized at the level of individualabsolute differences | ˆl i;xy −l xy |. In this way not only overalldiscrepancy averages can be studied but also the distribution ofabsolute differences at the individual entry level. Figure 9 showsthe distributions of the absolute differences | ˆl i;xy −l xy | for the differentsample sizes and numbers of underlying factors. In eachpanel, 100pk absolute differences are plotted.The majority of the absolute differences lies in the range from0 to circa 0.20. Larger absolute differences between the estimatedand true factor loadings occurred rather rarely. It is also apparentthat the 36 distributions hardly differ. This observation suggeststhat the effects or impacts of sample size, true number of factors,and the latent ability distribution on the accuracy of the classicalfactor models for estimating the factor loadings are rather weak.In that sense, estimation of the loading matrix seems to be robustoverall. In our simulation study, we were not able to see a clearrelationship between the distribution of the latent ability valuesand the discrepancy between the estimated and the true loadingmatrix.www.frontiersin.org March 2013 | Volume 4 | Article 109 | 132
Page 2 and 3:
FRONTIERS COPYRIGHTSTATEMENT© Copy
Page 4 and 5:
Table of Contents05 Is Data Cleanin
Page 7 and 8:
OsborneAssumptions and data cleanin
Page 9 and 10:
ORIGINAL RESEARCH ARTICLEpublished:
Page 12 and 13:
Hoekstra et al.Why assumptions are
Page 14 and 15:
Page 16 and 17:
Page 18 and 19:
Page 20 and 21:
García-PérezStatistical conclusio
Page 22 and 23:
Page 24 and 25:
Page 26 and 27:
Page 28 and 29:
Page 30 and 31:
Sheng and ShengEffect of non-normal
Page 32 and 33:
Page 34 and 35:
Page 36 and 37:
Page 38 and 39:
Page 40 and 41:
Page 42 and 43:
REVIEW ARTICLEpublished: 12 April 2
Page 44 and 45:
Nimon et al.The assumption of relia
Page 46 and 47:
Page 48 and 49:
Page 50 and 51:
Page 52 and 53:
Page 54 and 55:
Page 56 and 57:
TressoldiPower replication unreliab
Page 58 and 59:
TressoldiPower replication unreliab
Page 60 and 61:
Page 62 and 63:
FinchModern methods for the detecti
Page 64 and 65:
Page 66 and 67:
Page 68 and 69:
Page 70 and 71:
Page 72 and 73:
MINI REVIEW ARTICLEpublished: 28 Au
Page 74 and 75:
NimonStatistical assumptionsand Del
Page 76 and 77:
NimonStatistical assumptionsFor exa
Page 78 and 79:
Kraha et al.Interpreting multiple r
Page 80 and 81:
Kraha et al.Interpreting multiple r
Page 82 and 83: Kraha et al.Interpreting multiple r
Page 94 and 95: SmithsonComparing moderation of slo
Page 102 and 103: REVIEW ARTICLEpublished: 01 March 2
Page 104 and 105: Flora et al.Factor analysis assumpt
Page 124 and 125: Kasper and ÜnlüAssumptions of fac
Page 144 and 145: Lages and JaworskaHow predictable a
Page 152 and 153: Cummiskey et al.Testing assumptions
Page 158: Cummiskey et al.Testing assumptions
show all

Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

Create successful ePaper yourself

Delete template?

Save as template?