Criterion-related Validity “Is the test valid?” Criterion-related Validity ...

Now let’s consider the relationship between reliability & validity…“reliability is a precursor for validity”• conceptually -- how can a measure be consistently accurate(valid), unless it is consistent ??• internal consistency -- all items reflect the same construct• test-retest consistency -- scale yields repeatable scores• statistically -- limited reliability means that some of the variabilityin the measure is systematic, but part isunsystematic (unreliable)• low reliability will “attenuate” the validity correlation• much like range restriction -- but this is a restriction of the“systematic variance”, not the overall varianceVarious “correction for attenuation” formulas…Note: y=criterion x=measure being assessed ‘ = estimate• estimates what would be the validitycoefficient if both the criterion and themeasure were perfectly reliable(α=1.00)• estimates what would be the validity if thecriterion were perfectly reliable• a more useful formula estimates the validitycoefficient if each measure’s reliability“improved” to a specific valuer YXr YX ’ = --------------√α Y * √α Xr YXr YX ’ = -----√α Y“improved” αsit is possible to statistically “correct” for this attenuation-- like all “statistical correction”, this must be carefully applied!Measured validity√α Y ’ * √α X ’r YX ’= r YX --------------√α Y * √α Xmeasured αsExample …You have constructed an interview which is designed to predictemployee performance• scores on this interview (X) correlate .40 with supervisor’s ratings (Y)• the interview has an α Y = .50• the supervisor rating scale (the criterion) has α X= .70Correcting both the interviewand criterion to perfectreliability...Correcting just the to perfectreliability ...r YX.40r YX ’ = ----------- = ------------ = .68√α Y * √α X √.70 * √50r YX.40r YX ’ = ------------ = ------------ = .48√α Y √.70Correcting the interview toα=.7 to and criterion to α=.9...√α Y’ *√α X’ √.90 *√.70r YX’= r YX------------- = .40 ------------- = .53√α Y * √α X √.70 * √.50