Adult Literacy in America - National Center for Education Statistics ...

performance of a sample of examinees can be summarized on a series of
subscales even when different respondents have been administered different
items. Conventional scoring methods are not suited for assessments like the
national survey. Statistics based on the number of correct responses, such as
proportion of correct responses, are inappropriate for examinees who receive
different sets of items. Moreover, item-by-item reporting ignores similarities of
subgroup comparisons that are common across items. Finally, using average
percent correct to estimate means of proficiencies of examinees within
subpopulations does not provide any other information about the distribution of
skills among the examinees.
The limitations of conventional scoring methods can be overcome by the
use of item response theory (IRT) scaling. When several items require similar
skills, the response patterns should have some uniformity. Such uniformity can
be used to characterize both examinees and items in terms of a common scale
attached to the skills, even when all examinees do not take identical sets of items.
Comparisons of items and examinees can then be made in reference to a scale,
rather than to percent correct. IRT scaling also allows distributions of groups of
examinees to be compared.
Scaling was carried out separately for each of the three domains of literacy
(prose, document, and quantitative). The NAEP reading scale, used in the young
adult survey, was dropped because of its lack of relevance to the current NAEP
reading scale. The scaling model used for the national survey is the threeparameter
logistic (3PL) model from item response theory. 2 It is a mathematical
model for estimating the probability that a particular person will respond
correctly to a particular item from a single domain of items. This probability is
given as a function of a parameter characterizing the proficiency of that person,
and three parameters characterizing the properties of that item.
Overview of Linking the National Adult Literacy Survey (NALS)
Scales to the Young Adult Literacy Survey (YALS) Scales
Prose, document, and quantitative literacy results for the National Adult
Literacy Survey are reported on scales that were established in the Young Adult
Literacy Survey. For each scale, a number of new items unique to the national
survey were added to the item pool that was administered in the original young
adult survey. The NALS scales are linked to the YALS scales based upon the
commonality of the two assessments, namely, the original young adult survey
2 A. Birnbaum. (1968). “ Some Latent Trait Models.” In F.M. Lord and M.R. Novick, Statistical Theories of
Mental Test Scores. Reading, MA: Addison-Wesley. F.M. Lord. (1980). Applications of Item Response Theory
to Practical Testing Problems. Hillsdale, NJ: Erlbaum.
Appendix A ......127