Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

Kasper and ÜnlüAssumptions of factor analytic approaches0.2 0.4 0.6 0.8number of extracted factors131211109876543213121110987654321312111098765432PLPCSTPCKGPC0.2 0.4 0.6 0.8PLEASTEAKGEArelative frequencyNormal Slightly skewed Strongly skewedPLPASTPAKGPA0.2 0.4 0.6 0.8FIGURE 4 | Relative frequencies of the numbers of extracted factors, for n = 200 and k = 8. Factor models are principal component analysis (PCA, or PC),exploratory factor analysis (EFA, or EA), and principal axis analysis (PAA, or PA). Kaiser-Guttman criterion (KG), scree test (ST), and parallel analysis (PL) serve asfactor extraction criteria.6. ANALYSIS OF PIRLS 2006 DATAIn addition to the simulation study, the classical factor analyticapproaches are also compared on the part of PIRLS 2006 data thatwe presented in Section 4.2. The booklet design in PIRLS impliesthat only a selection of the items has been administered to eachstudent, depending on booklet approximately 23–26 test items perstudent (Mullis et al., 2006). As a consequence, the covariance orcorrelation matrices required for the factor models can only becomputed for the items of a particular test booklet. Since analysisof all thirteen booklets of the PIRLS 2006 study is out of thescope of this paper, we decided to analyze booklet number 4. Thisbooklet contains 23 items, and nine of these items (circa 40% of allitems) have skewness values in the range of −0.6 to 0. This skewnessrange corresponds to the values considered in the simulationstudy, and no other test booklet had a comparably high percentageof items with skewness values in this range.Note that in the empirical application dichotomized multicategoryitems are analyzed. In practice, large scale assessmentdata are discrete and not continuous. Yet, the metric scale indicatorcase considered in the simulation study can serve as an informativebaseline; for instance (issue of polychoric approximation) tothe extent that a product-moment correlation is a valid representationof bivariate relationships among interval-scaled variables(e.g., Flora et al., 2012). In our paper, the simulation resultsand the results obtained for the empirical large scale assessmentapplication are, more or less, comparable.In PIRLS 2006, four sorts of items were constructed and usedfor assigning “plausible values” to students (for details, see Martinet al., 2007). Any item loads on exactly one of the two dimensions“Literacy Experience” (L) and “Acquire and Use Information” (A)and also measures either the dimension “Retrieving and StraightforwardInferencing”(R) or the dimension“Interpreting, Integrating,and Evaluating”(I). Moreover, all of these items are assumed tobe indicators for the postulated higher dimension “Overall Reading.”In other words, PIRLS 2006 items may be assumed to beone-dimensional if the “uncorrelated” factor “Overall Reading” isFrontiers in Psychology | Quantitative Psychology and Measurement March 2013 | Volume 4 | Article 109 | 133