Assessing child development in the context of ... - The iLiNS Project

Assessing child development in the context of nutrition trials in developing
countries: Principles for test selection, adaptation, and evaluation
Investigators: Elizabeth L. Prado 1 , Katie Alcock 2 , Michael T. Ullman 3 , John Phuka 4 , Beth Yakes 5 , Mary Arimond 1 , Per
Ashorn 6 , and Kathryn Dewey 1
Institutions: 1 University of California at Davis, 2 Lancaster University, 3 Georgetown University, 4 University of Malawi College of Medicine, 5 Institut de
Recherche en Sciences de la Santé, Burkina Faso, 6 University of Tampere
Background: This study is part of the International Lipid‐Based Nutrient Supplements (iLiNS) Project. We report
preliminary data from the iLiNS‐DOSE project that we are currently conducting in rural Malawi and from the iLiNS‐ZINC
project that we are currently conducting in rural Burkina Faso.
Infancy and early childhood are crucial periods for the development of motor, cognitive, and socio‐emotional skills. Many
children in developing countries fail to reach their developmental potential in these areas, affecting later academic
achievement and adult life. 1 Lipid‐based nutrient supplements (LNS) during pregnancy and infancy may improve these
aspects of development, since brain development depends on adequate nutrition. For example, during fetal and infant
development, lipids and micronutrients are necessary for neuron proliferation and myelination. 2,3 The effect of LNS
during pregnancy and infancy on motor, cognitive, and socio‐emotional development is not yet clear.
In order to accurately evaluate the effect of various formulations of LNS on these abilities, child development must be
assessed in a way that is reliable and valid in the areas where LNS interventions are conducted in developing countries.
The assessment of child development in developing countries can be challenging since the most well‐established and
widely‐used tests have been produced and standardized in developed countries. Tests produced and standardized in one
language, culture, and setting cannot be assumed to be valid in a setting that is different from that of the original target
population. For example, children’s test performance depends on their familiarity with the test format and materials. 4
Such familiarity levels may be quite different for children who grow up in different cultures and contexts. Therefore, test
items, materials, and procedures must be adapted to the local setting of the LNS intervention, and the adapted tests
must be evaluated for reliability in the local population.
To assess child development in developing countries, other issues must also be considered. For example, it can be
difficult to find and hire personnel with expertise in child assessment. Therefore, it is necessary to select assessments that
can be administered without prior experience or extensive training. In addition, assessment by direct observation of the
child may be appropriate in certain contexts, while assessment by parent interview may be appropriate in others.
Objectives: To systematically address the issues related to the assessment of child development in the iLiNS‐DOSE and
iLiNS‐ZINC trials and to ensure that the assessments were appropriate and reliable for the local settings.
Methods: We developed a set of principles for selecting, adapting, and evaluating the child development assessments for
the iLiNS‐DOSE and iLiNS‐ZINC trials. The issues that we faced and the principles that we developed are presented in
Table 1. We selected several assessments developed in Kenya, including the Kilifi Developmental Inventory (KDI) 5 , the
Profile of Socio‐Emotional Development (PSED), and an adapted version of the Developmental Milestones Checklist
(DMC) 6 . We also adapted the MacArthur‐Bates Communicative Development Inventory (CDI) 7 , originally produced in the
USA, based in part on previous adaptations of this tool in Bangladesh 8 and Kenya 9 . We evaluated the discriminatory
power, inter‐scorer agreement, internal reliability, and test‐retest reliability of the tests in the local contexts.
Results. The selected and adapted assessments proved to be appropriate in the local contexts and to yield substantial
variance in scores among 18‐month‐old children. The available data demonstrated sufficient inter‐scorer agreement,
internal reliability, and test‐retest reliability (see Table 1).
References.
1. Grantham‐McGregor S, Cheung YB, Cueto S, et al. Developmental potential in the first 5 years for children in developing countries. Lancet. 2007;369:60‐70.
2. Uauy R, Mena P, DeSantiago S, Hernandez‐Rodriguez J. Lipids and neurodevelopment. Nutrition Reviews. 2001;59(8):S34‐S48.
3. McArdle HJ, Ashworth CJ. Micronutrients in fetal growth and development. British Medical Bulletin. 1999;55(3):499‐510.
4. Greenfield PM. You can't take it with you: Why ability assessments don't cross cultures. American Psychologist. 1997;52:1115‐1124.
5. Abubakar A, Holding PA, Van Baar A, Newton CRJC, Van de Vijver FJR. Monitoring psychomotor development in a resource‐limited setting: An evaluation of the Kilifi
Developmental Inventory. Annals of Tropical Pediatrics. 2008;28:217‐226.
6. Abubakar A, Holding P, van de Vijver FJ, Bomu G, Van Baar A. Developmental monitoring using caregiver reports in a resource‐limited settting: the case of Kilifi, Kenya. Acta
Pediatr. 2010;99:291‐297.
7. Fenson L, Marchman VA, Thal D, Dale PS, Reznick JS, Bates E. The MacArthur‐Bates Communicative Development Inventories, Second Edition: User's Guide and Technical
Manual. Baltimore: Paul H. Brookes Publishing Co; 2007.
8. Hamadani JD, Tofail F, Hilaly A, Huda SN, Engle P, Grantham‐McGregor SM. Use of family care indicators and their relationship with child development in Bangladesh. J
Health Popul Nutr. 2010;28(1):23‐33.
9. Alcock KJ, Prado EL, Rimba K, Kalu R, Newton CRJC, Holding P. Parent report of language development in illiterate families ‐ the CDI in two developing country settings.
Paper presented at: 21st Congress of the International Society for the Study of Behavioral Development; 18‐22 July, 2010; Lusaka, Zambia.
10. Epsy KA, Kaufmann PM, McDiarmid MD, Glisky ML. Executive functioning in preschool children: Performance on A‐Not‐B and other delayed response format tasks. Brain
and Cognition. 1999;41:178‐199.
1

Table 1. The issues that we considered and the principles that we developed for the assessment of child development
in the iLiNS‐DOSE and iLiNS‐ZINC trials.
Issue Principle iLiNS‐DOSE, Malawi iLiNS‐ZINC, Burkina Faso
Many different abilities
develop during infancy
and early childhood and
are likely to be
influenced by nutrition.
Few or no
developmental tests
have been produced in
the local setting.
The data collectors may
not have a very high
level of education or
previous experience
with developmental
assessment.
In order to detect any
effect of LNS, test
scores should be able to
differentiate between
children of the same
age (18 months) with
different abilities.
The location may be
crowded and noisy
therefore not conducive
to direct assessment of
the child. It may be
difficult to get a clear
picture of certain skills
when spending a limited
amount of time with the
child.
Limited time and
resources.
Select tests that assess a
wide range of abilities
that may be influenced
by LNS.
Select tests that have
been produced in a
similar setting or adapt
tests that have been
produced in a dissimilar
setting. In both cases,
evaluate the tests for
reliability in the local
context.
The testers should be
able to be trained in 2‐3
weeks without prior
experience. Scoring
should be relatively clear
and straightforward,
which can be evaluated
by inter‐scorer
agreement.
Test scores should show
a substantial amount of
variance in children at
the target age (18
months).
When direct assessment
is problematic, evaluate
the child based on
parent interview.
The assessments should
be relatively brief to
administer.
The KDI assesses fine and gross motor
development. The PSED assesses socioemotional
development. The MacArthur‐
Bates CDI assesses language development.
The A not B task assesses working memory
and executive function. 10
The KDI and PSED were produced in Kenya, in
a setting similar to the iLiNS‐DOSE study area.
The A not B task was previously adapted for
Kenya. The MacArthur‐Bates CDI has been
successfully adapted to similar settings (Kenya
and Bangladesh). Data from the first 71 iLiNS‐
DOSE children evaluated at age 18 months:
Internal Test‐retest
reliability reliability
Cronbach’s Alpha r
KDI: .67 .57
PSED: .69 .66
CDI: .98 .94
A not B: Not yet available
For each test, the testers were trained in 2‐3
weeks. Inter‐scorer agreement was evaluated
at the completion of training.
Inter‐scorer agreement (n = 18):
KDI: 95%
PSED: 89%
CDI: 96%
A not B: 98%
Data from the first 71 iLiNS‐DOSE children
evaluated at age 18 months:
KDI: n = 45, range = 35‐47, mean=41, SD=3
PSED: n = 71, range = 4‐34, mean=16, SD=6
CDI: n = 65, range = 0‐90, mean=34, SD=25
A not B: n = 54, range = 0‐10, mean=6, SD=2
It is difficult to obtain a complete picture of a
child’s language ability and socio‐emotional
ability in a short (~ 1 hour) test session.
However, parents spend substantial amounts
of time with their children and observe them
in a wide variety of situations and contexts.
Therefore, we used parent interviews to
assess these skills (the CDI and PSED).
KDI and A not B: ~45 minutes
PSED: ~20 minutes
CDI: ~10 minutes
The DMC assesses Motor
development, Language
development, and
Personal‐social
development.
The DMC was produced in
Kenya, in a setting similar to
the iLiNS‐ZINC study area.
Data from the first 25 iLiNS‐
ZINC children evaluated on
the adapted DMC at age 18
months:
Internal reliability
Cronbach’s Alpha
Motor: .87
Language: .72
Personal‐social: .55
Test‐retest reliability:
Not yet available
The testers were trained in
3 weeks.
Inter‐scorer agreement:
Not yet available
Data from the first 25 iLiNS‐
ZINC children evaluated on
the adapted DMC at age 18
months:
Motor: n = 18, range = 36‐
60, mean=48, SD=8
Language: n = 24, range =
15‐28, mean=21, SD=3
Personal‐social: n = 20,
range = 38‐52, mean=46,
SD=4
The developmental
assessments were
conducted at clinic visits
where the atmosphere was
not conducive to extensive
observation of the child.
Therefore, we used a
parent interview (the
adapted DMC) in
combination with direct
observation of certain skills.
DMC: ~20 minutes
2