The Journal of the World Council for Gifted and Talented Children

invariance is in fact systemic regarding low-income and non low-income students, the instrument
would not be appropriate for making across-group (low-SES to non-low SES) comparisons. Put
another way, if non-invariance was found using the NNAT with students from low-income families,
the use of this measure in identifying students from low-income backgrounds for gifted education
programs would be inappropriate given the fact that the instrument does not measure what its
supposed to measure in the same fashion for both groups of students. As it stands, the invariance of
the NNAT with regard to SES groups, as is true with many other instruments, remains unknown.
Implications
Test authors should evaluate measurement invariance before an instrument is ever put into
widespread use and the results from such analyses should be presented in the instrument’s
technical manual. If such information is not available, school practitioners and academic researchers
should be wary of using an instrument with diverse groups, as they cannot be sure the instrument
measures the same thing for people in various groups. However, a secondary analysis of a test, as
presented with the NNAT example, should also be conducted in order to assure previous validity
evidence is still supported over time and across new groups.
A look at the test manuals of larger, more established instruments such as individual
intelligence tests or computer-adaptive achievement tests will reveal detailed sections on how MI/E,
bias, or differential item functioning were evaluated in the development phase or in instrument
revision. Interested readers do not have to be too concerned with how to evaluate such information
when presented, because it is so rarely presented with regard to instruments commonly used in
gifted and talented research. However, even today some measures do include this information. For
example, the technical manual of the Scales for Identifying Gifted Students (Ryser & McConnell,
2004) does present DIF validity evidence across racial/ ethnic and gender groups, although no
comparisons were made for income groups (Matthews, 2007). Still, this type of reporting should
become standard in any field where across-group comparisons are common.
Researchers interested in conducting MI/ E from the SEM perspective should consult Brown
(2006), who includes a step-by-step process as well as statistical code for several computer
programs. A CFA can be run in most statistical programs. LISREL and MPlus allow for the widest
range of estimation procedures and examination of partial (sub-item) MI/E. EQS, AMOS, and SAS
will also run CFA analyses, but have a limited variety of estimation procedures (although this should
not cause a problem for most research). Brown (2006) is an excellent source for example code for all
of these procedures. The MPlus technical manual (Muthén & Muthén, 2007) also presents code for a
variety of different CFA situations.
References
American Education Research Association. (1999). Standards for educational and psychological testing.
Washington, DC: Author.
Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York: The Guilford Press.
Carman, C. A., & Taylor, D. K. (2010). Socioeconomic status effects on using the Naglieri Nonverbal Ability Test
(NNAT) to identify the gifted/talented. Gifted Child Quarterly, 54, 75-84.
French, B. F. (2005). [Review of Naglieri Nonverbal Ability Test-Individual Administration]. The Sixteenth Mental
Measurements Yearbook. Lincoln, NE: Buros Institute.
French, B. F., & Finch, W. H. (2006). Confirmatory factor analytic procedures for the determination of
measurement invariance. Structural Equation Modeling, 13, 378-402.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indices in covariance structure analysis: Conventional criteria
versus new alternatives. Structural Equation Modeling, 6, 1-55.
Joint Committee on Testing Practices. (2005). Code of fair testing practices in education. Washington, DC:
American Psychological Association.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Mahwah, NJ: Erlbaum.
Matthews, M. S. (2007). The Scales for Identifying Gifted Students by G. R. Ryser & K. McConnell [Test review].
Pp. 729-731 in K. F. Geisinger, R. A. Spies, J. F. Calrson, & B. S. Plake (Eds.), The Seventeenth Mental
Measurements Yearbook. Lincoln, NE: The Buros Institute of Mental Measurements.
Muthén, L. K., & Muthén, B. O. (2007). Mplus User’s Guide. Fifth Edition. Los Angeles, CA: Muthén and Muthén.
Naglieri, J. A. (2003). Naglieri Nonverbal Ability Test. San Antonio, TX: Psychological Corporation.
Naglieri, J. A., & Ford, D. Y. (2003). Addressing underrepresentation of gifted minority children using the Naglieri
Nonverbal Ability Test (NNAT). Gifted Child Quarterly, 47, 155-160.
Gifted and Talented International – 26(1), August, 2011; and 26(2), December, 2011. 103