Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

More documents

Recommendations

Info

SmithsonComparing moderation of slopes and correlationsTable 1 | Type I error: two-groups simulations.Skew N 1 N 2 σ xi = 1,1 σ xi = 1,2 σ xi = 1,4σ yi = 2,2 σ yi = 2,4 σ yi = 2,8NORMAL0 70 70 0.0518 0.0532 0.05110 45 155 0.0578 0.0553 0.05470 90 180 0.0513 0.0531 0.0503SKEWED2 70 70 0.0710 0.0704 0.06804 70 70 0.0768 0.0767 0.07132 45 155 0.0681 0.0686 0.06874 45 155 0.0778 0.0776 0.07742 90 180 0.0679 0.0708 0.07144 90 180 0.0755 0.0728 0.0723sizes have little impact on rejection rates, with the effect appearingto diminish in the larger-sample (90–180) condition. The ratesare slightly higher than 0.05, but are unaffected by the sizes of thevariances.The lower half of Table 1 shows simulations under the sameconditions, but this time with X and Y sampled from the Azzaliniskew-normal distribution (Azzalini, 1985). The standardskew-normal pdf isf (x, λ) = e−x2/ 2[ ]) λx√(1 + Erf √ .2π 2The simulations had the skew parameter λ set to 2 and 4, the pdfsfor which are shown in Figure 1. Skew increased the rejection ratesto 0.068–0.078, rendering the test liberal but not dramatically so.We now turn to investigating the power of the EVR test. Simulationstesting its power were conducted for two situations:moderated slopes but unmoderated correlations, and moderatedcorrelations but unmoderated slopes. Both batches of simulationswere run with four combinations of sample sizes (70–70, 40–140,140–140, and 80–280) and three variance ratio combinations (1–1.5, 1–2, 1–4). In the unmoderated correlations setup ρ = 0.5 forall conditions, and in the unmoderated slopes setup β y = 0.5 forall conditions. These tests also require modeling the moderationof correlations. The correlation submodel uses the Fisher link, i.e.( ) 1 + ρilog = ∑ 1 − ρ iiw i δ ri . (11)Note that we allow a different set of predictors for the correlationfrom those in equation (7). However, in this paper we will imposethe restriction w i = z i .Table 2 shows the simulation results for unequal variance ratioswith unmoderated correlations. The table contains rejection ratesof the EVR and moderation of correlation null hypotheses. Theresultant moderated slopes and error-variances are displayed foreach condition. Note that HeV and HeEV do not have discernibleeffects on either of the rejection rates. As in the preceding simulations,the rejection rates for the unmoderated correlations areonly slightly above the 0.05 criterion. The rejection rates for theEVR test l and in the 0.85–1.0 range in the conditions where thecombined sample sizes are 280 and the ratio of the variance ratiosis 2:1 or for both combined sizes when the ratio is 4:1.Table 3 shows the rejection rates of the EVR and moderation ofcorrelation null hypotheses when there are unequal variance ratiosand moderated correlations. The resultant moderated correlationsand error-variances are displayed for each condition. As before,HeV and HeEv do not affect either of the rejection rates. Likewise,as expected, the EVR rejection rates are very similar to those inTable 2. It is noteworthy that rejection rates for the unmoderatedcorrelations hypothesis are considerably smaller than those forthe EVR hypothesis, even though the correlations differ fairly substantially.It is well-known that tests for moderation of slopes andcorrelations have rather low power. These results, and the fact thatthe ratios of the variance ratios do not exceed Howell’s benchmarkof 4:1, suggest that the EVR test has relatively high power.STRUCTURAL EQUATIONS MODEL APPROACHWhen the moderator variable is categorical, the EVR test can beincorporated in a structural equations model (SEM) approach thatpermits researchers not only to compare an EVR model againstone that relaxes this assumption, but also to test simultaneouslyfor HeV, HeEV, moderation of correlations and moderation ofslopes. Figure 2 shows the regression (left-hand side) and correlation(right-hand side) versions of this model. The latter followsPreacher’s (2006) strategy for a multi-group SEM for correlations.The regression version models the error-variances σei 2 rather thanthe variances σyi 2 . Instead, σ yi 2 is modeled in the correlation version.The only addition to the correlation SEM required for incorporatingEVR tests is to explicitly model variance ratios for eachof the moderator variable categories. Two SEM package that cando so are lavaan (Rosseel, 2012) in R and MPlus (Muthén andMuthén, 2010). Examples in lavaan and MPlus are available athttp://dl.dropbox.com/u/1857674/EVR_moderator/EVR.html, asare EVR test scripts in SPSS and SAS.Simulations were run using lavaan in model comparisons forsamples with moderated slopes but unmoderated correlations, andsamples with moderated correlations but unmoderated slopes. Asbefore, each simulation had 20,000 runs.Frontiers in Psychology | Quantitative Psychology and Measurement July 2012 | Volume 3 | Article 231 | 95
SmithsonComparing moderation of slopes and correlationsf0.640.520.40.300.20.12 1 0 1 2 3xFIGURE 1 | Azzalini Skew-normal distributions with λ = 0,2,4.Table 2 | Power: moderated slopes and unmoderated correlations.N 1 N 2 σx 2 = 2 σ x 2 = 2 σ x 2 = 2 σ x 2 = 2 σ x 2 = 2 σ x 2 = 2σy 2 = 2 σ y 2 = 3 σ y 2 = 2 σ y 2 = 4 σ y 2 = 2 σ y 2 = 8σ xy = 1 σ xy = √ 3/2 σ xy = 1 σ xy = √ 2 σ xy = 1 σ xy = 2σe 2 = 1.5 σ e 2 = 2.25 σ e 2 = 1.5 σ e 2 = 3 σ e 2 = 1.5 σ e 2 = √ 8/2β y = 0.5 β y = √ 3/8 β y = 0.5 β y = √ 2/2 β y = 0.5 β y = 1δ r θ δ r θ δ r θ70 70 0.0556 0.2810 0.0603 0.6321 0.0576 0.993940 100 0.0566 0.2478 0.0569 0.5706 0.0566 0.9875140 140 0.0549 0.4841 0.0532 0.9032 0.0537 1.00080 200 0.0529 0.4311 0.0497 0.8524 0.0522 0.9999Table 3 | Power: unmoderated slopes and moderated correlations.N 1 N 2 σx 2 = 2 σ x 2 = 2 σ x 2 = 2 σ x 2 = 2 σ x 2 = 2 σ x 2 = 2σy 2 = 2 σ y 2 = 3 σ y 2 = 2 σ y 2 = 4 σ y 2 = 2 σ y 2 = 8σ xy = 1 σ xy = 1 σ xy = 1 σ xy = 1 σ xy = 1 σ xy = 1σe 2 = 1.5 σ e 2 = 2.5 σ e 2 = 1.5 σ e 2 = 3.5 σ e 2 = 1.5 σ e 2 = 6ρ xy = 0.5 ρ xy = 1/√6 ρxy = 0.5 ρ xy = 1/√8 ρxy = 0.5 ρ xy = 0.25δ r θ δ r θ δ r θ70 70 0.0944 0.2635 0.1525 0.5925 0.3426 0.986440 100 0.0992 0.2394 0.1700 0.5476 0.3558 0.9826140 140 0.1296 0.4575 0.2483 0.8771 0.5844 1.00080 200 0.1444 0.4201 0.2878 0.8326 0.6036 0.9999Simulations from bivariate normal distributions withρ xy = 0.05 for both groups (Table 4) indicated that moderatelylarge samples and slope differences are needed for reasonablepower. However, there was little impact on power from unequalgroup sizes. Rejection rates for the unmoderated correlationshypothesis were at appropriate levels, 0.0493–0.0559.www.frontiersin.org July 2012 | Volume 3 | Article 231 | 96
Page 2 and 3:
FRONTIERS COPYRIGHTSTATEMENT© Copy
Page 4 and 5:
Table of Contents05 Is Data Cleanin
Page 7 and 8:
OsborneAssumptions and data cleanin
Page 9 and 10:
ORIGINAL RESEARCH ARTICLEpublished:
Page 12 and 13:
Hoekstra et al.Why assumptions are
Page 14 and 15:
Page 16 and 17:
Page 18 and 19:
ORIGINAL RESEARCH ARTICLEpublished:
Page 20 and 21:
García-PérezStatistical conclusio
Page 22 and 23:
Page 24 and 25:
Page 26 and 27:
Page 28 and 29:
Page 30 and 31:
Sheng and ShengEffect of non-normal
Page 32 and 33:
Page 34 and 35:
Page 36 and 37:
Page 38 and 39:
Page 40 and 41:
Page 42 and 43:
REVIEW ARTICLEpublished: 12 April 2
Page 44 and 45:
Nimon et al.The assumption of relia
Page 46 and 47: Nimon et al.The assumption of relia
Page 56 and 57: TressoldiPower replication unreliab
Page 58 and 59: TressoldiPower replication unreliab
Page 60 and 61: ORIGINAL RESEARCH ARTICLEpublished:
Page 62 and 63: FinchModern methods for the detecti
Page 72 and 73: MINI REVIEW ARTICLEpublished: 28 Au
Page 74 and 75: NimonStatistical assumptionsand Del
Page 76 and 77: NimonStatistical assumptionsFor exa
Page 78 and 79: Kraha et al.Interpreting multiple r
Page 94 and 95: SmithsonComparing moderation of slo
Page 102 and 103: REVIEW ARTICLEpublished: 01 March 2
Page 104 and 105: Flora et al.Factor analysis assumpt
Page 124 and 125: Kasper and ÜnlüAssumptions of fac
Page 144 and 145: Lages and JaworskaHow predictable a
Page 146 and 147:
Lages and JaworskaHow predictable a
Page 148 and 149:
Page 150 and 151:
Page 152 and 153:
Cummiskey et al.Testing assumptions
Page 154 and 155:
Page 156 and 157:
Page 158:
show all

Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

Create successful ePaper yourself

Delete template?

Save as template?