Why is it important and how to participate? Mark Keese, OECD - CMI

PIAAC
Programme for the International Assessment of Adult Competencies
PIAAC: a new window on adult skills
Mark Keese
(mark.keese@oecd.org)
World Bank Seminar on “Enhancing Qualifications Frameworks and Quality Assurance in MENA”,
Marseille, 14-15 June 2010

Outline
What is PIAAC and what progress has
been made?
Why do we need PIAAC?
How can MENA be involved?

What is PIAAC and what progress has been made?

PIAAC’s scope:
What is PIAAC?
The most comprehensive international survey of adult skills ever
undertaken
Measures the skills and competencies needed for individuals to
participate in society and for economies to prosper
Will help governments better understand how education and training
systems and other labour market policies can nurture these skills
How will it be carried out?
A collaboration between governments, an international consortium of
organisations and the OECD
It will take place in 28 OECD and partner countries in 2011-2012 with
results published in 2013
Based on a household-based survey of adults (minimum of 5000
respondents per country)

Measures of literacy,
numeracy & problemsolving
skills
o Direct assessment
What does PIAAC measure?
Measures of generic
skills used at work
o Job Requirements
Approach (JRA)
Measures of skill
formation & outcomes
o Background questions
on adult learning, labourmarket
status, earnings,
health …

Direct assessment module
Objective: To provide assessment of adult literacy in the
information age, covering 4 broad domains of competency
Literacy Numeracy Literacy
component skills
Problem solving in
an IT context
Literacy defined as “interest, attitude and ability of individuals to
appropriately use socio-cultural tools, including digital technology and
communication tools, to access, manage, integrate and evaluate
information, construct new knowledge, and communicate with others in
order to participate effectively in society”
Involves a 60-minute test

Generic workplace skills covered by PIAAC
Cognitive skills
Reading, Writing,
Maths, IT,
Problem Solving
Key
workplace
skills
Interaction/social skills
Physical skills Learning skills
Stamina, Strength,
Dexterity
Influence, Selfdirection,
Horizontal
interaction, Client
interaction
Required to learn
things, Keep up to
date, Help others
learn

Progress with PIAAC
Survey instruments have been developed
Field trial now underway in most participating countries
A large scale exercise involving 1500 respondents in each country
Will be used to validate instruments and test survey procedures
May-July
2010
July 2011-
Feb 2012
Sep 2013
PIAAC Timeline
• Field trial
• Full survey to take place
• First PIAAC international
report to be released

Why do we need PIAAC?

10
Key areas where PIAAC will tell us more
Outcomes
of skills
International
benchmarking
PIAAC
results
Skill
acquisition
Distribution
of skills

International benchmarking
Key policy issues:
Are there clear differences in country performance in the cognitive
assessment (even allowing for differences in GDP per capita, etc.)
which point to better performing education and training systems?
How well is each country doing in adapting its skill development policies
to a technology-rich environment and increasing international trade?
Have literacy and numeracy skills improved over time in line with
changes in education and training systems?

PISA reading score (mean)
600
550
500
450
400
350
300
250
Benchmarking
KOR
FIN
CAN HKG
NZL AUS
POL
SWE IRL
EST
JPN BEL
NLD CHE
SVN DEU DNK
HRV HUN CZE
GBR USA*
FRA
LVA AUT
GRC
ISL
LTU
SVK
PRT ITA
TUR
ESP
CHL RUS
ISR
THA
URY
JOR
SRB BGR MEX
BRA MNE
IDN COL ROM
TUN ARG
KGZ
PISA mean reading scores and GDP per capita, 2006*
AZE
ISL
LUX
y = 63.015ln(x) - 157.58
R² = 0.7178
0 10000 20000 30000 40000 50000 60000 70000
GDP per capita ($US in PPPs)
* 2003 score for the United States.
Source: OECD for PISA scores and World Bank for GDP per capita.

Key policy issues:
Distribution of skills
Are education and training institutions failing to give some groups
adequate skills for economic and social success?
What is the relative importance of education versus socioeconomic
background in determining skills that adults have and use? Do some
countries do better than others in terms of leveling the playing field?

Skill acquisition (and loss)
Key policy issues:
How well are countries performing in imparting youths with essential
cognitive skills and ensuring smooth labour-market entry?
What is the role of education/training, work experience and work
organisation?

Skill acquisition (and loss)
What is the contribution of further education and work experience to
skill acquisition?
Share of adults with high literacy levels by age
Source: Cascio, Clark and Gordon (2008) based on IALS data.

Key policy issues:
Outcomes of skills
Is there a pay-off to investments in education and training, and in which
skills and for whom? What is the role of broader institutional settings?
PIAAC will be able to go much further at looking at returns to education and
skills by including analysis of filed of education, assessed skills and a broad
range of skills used at work
In our modern IT world, how important are computer and problemsolving
skills? What other skills matter and why?
Is over-education important and why do countries differ? Does
education mismatch mean a skills mismatch?

100%
Skills mismatch
Provides measures of both education and skills mismatch
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Proportion of workers over-qualified highest in Korea but relatively fewer
over-skilled workers
In Australia and Korea, over-qualifications (under-qualifications) are
associated with a large earnings penalty (premium)
Source: OECD JRA Pilot Study (general sample).
Education and skill match
A. Education match B. Skill match
% of workers by match category % of workers by match category
Over-qualified Matched Under-qualified
Australia France Greece Korea
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Over-skilled Matched Under-skilled
Australia France Greece Korea

How can MENA be involved?

A special assessment round
A special assessment round could be carried out for MENA and
other additional countries:
Based on the PIAAC methodology and giving comparable results with those from
the first round
The cognitive assessment (of literacy, numeracy, etc.) would be carried out by
“pencil and paper” and include additional items as necessary
Some adaptation of the background questionnaire to individual country
circumstances would be possible
Infrastructure to support implementation of the survey (e.g. engage international
contractor to develop survey and oversee its implementation) would be
established in partnership with the OECD and World Bank

Roles and responsibilities
International level by project contractor:
Prepare survey instruments and survey design
Carry out centralised translation of survey instruments and training manuals
Prepare and provide training
Survey support and quality control
Analysis, cleaning and scaling of survey results
Provide data products for analysis and dissemination of results
National level by participating countries:
Adapting survey instruments to national context and verifying translation
Appointment of National Project Manager (NPM) and project team
Attending NPM meetings
Administration of field trial and main survey
Preparation and cleaning of national data files

Possible timeline for MENA participation in PIAAC
End of 2010
2011
2012
2013
2014
End 2015
• Finalise country participation and funding
arrangements
• Adaptation and finalisation of instruments
• Preparation for field trial
• Undertake field trial
• Analysis of field trial results
• Revision of instruments and procedures
• Preparation for main data collection
• Main data collection
• Data preparation and analysis
• Release of report on results

How much will it cost?
For international costs, initial estimate of 6 million euros for 6
countries plus 390K euros for additional countries, spread over 5
years
For national costs, the main element would be the cost of
carrying out a household survey of around 3500-4000 individuals

Conclusions
PIAAC will open a new window on adult skills
Participation in PIAAC would give each MENA country valuable
insights into where more can be done to improve skills based on
internationally comparable data
It will help identify which skills are essential, who has them and
where there are skill gaps
Investing in training and skills is costly and PIAAC can help
ensure that this money is well spent
For further details on PIAAC, see:
www.oecd.org/piaac