people and planet

15
The assessment would need to have enough test items
to accurately locate assessed individuals on a particular
level of the scale. It should also be possible to identify
whether individuals with very low literacy have problems
mastering the basic components of reading. One option
would be to assess the basic building blocks for reading
comprehension, including knowledge of vocabulary,
understanding of sentence logic and fluency in reading
text passages. Considerable research would be required
to ensure cross-cultural validity. As the section below
on numeracy points out, care should be taken to adopt
a model that is both relevant and feasible for countries
with few resources.
NUMERACY
While the concept of numeracy connotes different things
to different people, common themes are found in the
definitions used in international assessments. First, they
capture a continuum of skills. Second, they focus on
competencies to effectively engage with quantitative
tasks in everyday life, rather than on what people should
have learned in school. Third, they refer to a broader set
of knowledge and skills than just the ability to count,
estimate and measure.
In PIAAC, numeracy is defined as ‘the ability to
access, use, interpret, and communicate mathematical
information and ideas, in order to engage in and
manage the mathematical demands of a range of
situations in adult life’ (PIAAC Numeracy Expert Group,
2009; Gal and Tout, 2014; Tout and Gal, 2015). National
and international assessments share aspects of this
definition and other operational aspects to a lesser or
greater extent (Box 15.2).
Numeracy assessment is feasible in low and
middle income countries, though issues of validity,
comparability and policy relevance need to be addressed
given the diverse needs and technical capacities of
stakeholders. One assessment system is unlikely to
address the information needs of all stakeholders even in
a single high income country, let alone in a large number
of countries. To provide useful monitoring data by 2030,
a successful approach has to balance the ability of a
country to proceed on its own against the need for its
assessment to meet global quality standards. For poorer
countries, a model is needed that demands modest
resources and time, and fits local technical capacity and
policy needs (Gal, 2016).
BOX 15.2
Differences between national and international
numeracy assessments
A review of three waves of OECD numeracy surveys (IALS, ALL
and PIAAC), LAMP (which carried out surveys in Jordan, Mongolia,
Palestine and Paraguay) and four national surveys (Bangladesh,
Brazil, France and Kenya) suggests several areas for comparison.
Brazil, Kenya and the LAMP countries adopted a broad definition
of adult numeracy that is similar to that of PIAAC. Bangladesh and
France focused on a narrower set of arithmetic operations and basic
functional numeracy skills; for both countries, the assessment
was designed to identify low ability adults who could benefit from
public programmes.
The LAMP countries and France measured and reported numeracy
results separately from those for literacy. In contrast, Bangladesh,
Brazil and Kenya considered numeracy an integral part of literacy
and did not report separate results, instead providing a combined
result on functional literacy.
The potential influence of literacy skills on performance of
numeracy tasks was acknowledged. In LAMP and PIAAC, the design
of items was changed to reduce literacy demands or text density.
In France, an oral assessment was used. There were also differing
levels of text usage. For example, in Bangladesh more items looked
like school arithmetic exercises, using symbols and no text.
Survey sessions per household ranged on average from 60 to
90 minutes and numeracy was only one of several domains
assessed alongside literacy, with additional items in a background
questionnaire. Given the time constraint, it is not surprising that the
numeracy tests were short: 12 items in Bangladesh, 18 in France and
Kenya, and 25 (out of a pool of 57) in PIAAC.
Other differences in key aspects of the assessment framework
included the difficulty of items (how they were spread across a
continuum of levels from ‘easy’ to ‘difficult’), their contextualization
(context-free computational items or items embedded in a
particular cultural and functional context) and an estimation of
proficiency (a simple total score or psychometric modelling).
The levels of reporting used further complicate comparisons. LAMP
reports on three numeracy levels, some countries used four levels
ranging from non-literate to advanced literacy, and PIAAC used
a five-level system plus the additional category ‘below level 1’.
Consequently, it is not possible to benchmark results from one
country against another.
Source: Gal (2016).
2016 • GLOBAL EDUCATION MONITORING REPORT 283