Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

More documents

Recommendations

Info

Flora et al.Factor analysis assumptionsFIGURE 1 | Scatterplot matrix for multivariate normal random sample consistent with Holzinger data (N = 100; no unusual cases).foolproof. Cases that appear to be outliers in a scatterplot mightnot actually be influential in that they produce distorted or otherwisemisleading factor analysis results; conversely, certain influentialcases might not easily reveal themselves in a basic bivariatescatterplot. Here, concepts from regression diagnostics come intoplay. Given that regression (and hence factor analysis) is a procedurefor modeling a dependent variable conditional on one ormore explanatory variables, a regression outlier is a case whosedependent variable value is unusual relative to its predicted, ormodeled, value given its scores on the explanatory variables (Fox,2008). In other words, regression outliers are cases with largeresiduals.A regression outlier will have an impact on the estimated regressionline (i.e., its slope) to the extent that it has high leverage.Leverage refers to the extent that a case has an unusual combinationof values on the set of explanatory variables. Thus, ifa regression outlier has low leverage (i.e., it is near the centerof the multivariate distribution of explanatory variables), itshould have relatively little influence on the estimated regressionslope; that is, the estimated value of the regression slopeshould not change substantially if such a case is deleted. Conversely,a case with high leverage but a small residual also haslittle influence on the estimated slope value. Such a high leverage,small residual case is often called a “good leverage” case becauseits inclusion in the analysis leads to a more precise estimate ofthe regression slope (i.e., the SE of the slope is smaller). Visualinspection of a bivariate scatterplot might reveal such a case, anda naïve researcher might be tempted to call it an “outlier” anddelete it. But doing so would be unwise because of the loss ofstatistical precision. Hence, although visual inspection of raw,observed data with univariate plots and bivariate scatterplots isalways good practice, more sophisticated procedures are neededto gain a full understanding of whether unusual cases are likely tohave detrimental impact on modeling results. Next, we describehow these concepts from regression diagnostics extend to factoranalysis.FACTOR MODEL OUTLIERSThe factor analysis analog to a regression outlier is a case whosevalue for a particular observed variable is extremely different fromits predicted value given its scores on the factors. In other words,cases with large (absolute) values for one or more unique factors,that is, scores on residual terms in ε, are factor model outliers. Eq.1 obviously defines the residuals ε asε = y − Λη. (4)www.frontiersin.org March 2012 | Volume 3 | Article 55 | 105
Flora et al.Factor analysis assumptionsBut because the factor scores (i.e., scores on latent variables inη) are unobserved and cannot be calculated precisely, so too theresiduals cannot be calculated precisely, even with known populationfactor loadings, Λ. Thus, to obtain estimates of the residuals,ˆε, it is necessary first to estimate the factor scores, ˆη, which themselvesmust be based on sample estimates of ˆΛ and ˆΘ. Bollenand Arminger (1991) show how two well-known approaches toestimating factor scores, the least-squares regression method andBartlett’s method, can be applied to obtain ˆε. As implied by Eq.4, the estimated residuals ˆε are unstandardized in that they arein the metric of the observed variables, y. Bollen and Armingerthus present formulas to convert unstandardized residuals intostandardized residuals. Simulation results by Bollen and Armingershow good performance of their standardized residuals for revealingoutliers, with little difference according to whether they areestimated from regression-based factor scores or Bartlett-basedfactor scores.Returning to our example sample data, we estimated the standardizedresiduals from the three-factor EFA model for each ofthe N = 100 cases; Figure 2 illustrates these residuals for all nineobserved variables (recall that in this example, y consists of ninevariables and thus there are nine unique factors in ˆε). Because thedata were drawn from a standard normal distribution conformingto a known population model, these residuals themselves shouldbe approximately normally distributed with no extreme outliers.Any deviation from normality in Figure 2 is thus only due to samplingerror and error due to the approximation of factor scores.Later, we introduce an extreme outlier in the data set to illustrateits effects.LEVERAGEAs mentioned above,in regression it is important to consider leveragein addition to outlying residuals. The same concept applies infactor analysis (see Yuan and Zhong, 2008, for an extensive discussion).Leverage is most commonly quantified using “hat values”in multiple regression analysis, but a related statistic, Mahalonobisdistance (MD), also can be used to measure leverage (e.g.,Fox, 2008). MD helps measure the extent to which an observationis a multivariate outlier with respect to the set of explanatoryvariables 6 . Here, our use of MD draws from Pek and MacCallum(2011), who recommend its use for uncovering multivariate outliersin the context of SEM. In a factor analysis, the MD for a given6 Weisberg (1985) showed that (n − 1)h ∗ is the MD for a given case from the centroidof the explanatory variables, where h ∗ is the regression hat value calculated usingmean-deviated independent variables.FIGURE 2 | Histograms of standardized residuals for each observed variable from three-factor model fitted to random sample data (N = 100; nounusual cases).Frontiers in Psychology | Quantitative Psychology and Measurement March 2012 | Volume 3 | Article 55 | 106
Page 2 and 3:
FRONTIERS COPYRIGHTSTATEMENT© Copy
Page 4 and 5:
Table of Contents05 Is Data Cleanin
Page 7 and 8:
OsborneAssumptions and data cleanin
Page 9 and 10:
ORIGINAL RESEARCH ARTICLEpublished:
Page 12 and 13:
Hoekstra et al.Why assumptions are
Page 14 and 15:
Page 16 and 17:
Page 18 and 19:
ORIGINAL RESEARCH ARTICLEpublished:
Page 20 and 21:
García-PérezStatistical conclusio
Page 22 and 23:
Page 24 and 25:
Page 26 and 27:
Page 28 and 29:
Page 30 and 31:
Sheng and ShengEffect of non-normal
Page 32 and 33:
Page 34 and 35:
Page 36 and 37:
Page 38 and 39:
Page 40 and 41:
Page 42 and 43:
REVIEW ARTICLEpublished: 12 April 2
Page 44 and 45:
Nimon et al.The assumption of relia
Page 46 and 47:
Page 48 and 49:
Page 50 and 51:
Page 52 and 53:
Page 54 and 55:
Page 56 and 57: TressoldiPower replication unreliab
Page 58 and 59: TressoldiPower replication unreliab
Page 60 and 61: ORIGINAL RESEARCH ARTICLEpublished:
Page 62 and 63: FinchModern methods for the detecti
Page 72 and 73: MINI REVIEW ARTICLEpublished: 28 Au
Page 74 and 75: NimonStatistical assumptionsand Del
Page 76 and 77: NimonStatistical assumptionsFor exa
Page 78 and 79: Kraha et al.Interpreting multiple r
Page 94 and 95: SmithsonComparing moderation of slo
Page 102 and 103: REVIEW ARTICLEpublished: 01 March 2
Page 104 and 105: Flora et al.Factor analysis assumpt
Page 124 and 125: Kasper and ÜnlüAssumptions of fac
Page 144 and 145: Lages and JaworskaHow predictable a
Page 152 and 153: Cummiskey et al.Testing assumptions
Page 154 and 155: Cummiskey et al.Testing assumptions
Page 156 and 157:
Cummiskey et al.Testing assumptions
Page 158:
Cummiskey et al.Testing assumptions
show all

Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

Create successful ePaper yourself

Delete template?

Save as template?