measuring the outputs and outcomes of vocational training

Institute for the Protection and Security of the Citizen
Unit for Econometrics and Statistical Support to Anti-fraud
CRELL Research Paper 2
May 2006
MEASURING THE OUTPUTS AND OUTCOMES OF
VOCATIONAL TRAINING – TOWARDS A COHERENT
FRAMEWORK FOR INDICATORS
MIRCEA BADESCU
2006 EUR 22305 EN

2
European Commission
Directorate-General Joint Research Centre
Institute for the Protection and Security of the Citizen
http://farmweb.jrc.cec.eu.int/crell/
http://www.jrc.cec.eu.int/
Legal Notice
Neither the European Commission nor any person acting on behalf of
the Commission is responsible for the use which might be made of this
publication.
ISSN 1018-5593
EUR 22305 EN
Luxembourg: Office for Official Publications of the European Communities
© European Communities, 2006
Reproduction is authorised provided the source is acknowledged

3
Abstract
This paper proposes a new analytical framework for monitoring the outcomes and outputs of vocational
training. It includes a critical review of the main statistical sources used to measure vocational
training (both initial and continuing) and to generate indicators. It also presents some country or
cross-country comparative results of empirical analysis. The annex outlines the three dimensions of
the framework (analytical, policy and statistical), using some examples of output and outcome indicators
that could be developed.
Keywords
Vocational training, framework for indicators, indicators and benchmarks
Contents
• Introduction – the political dimension
• The state of the art in reporting on outputs and outcomes of vocational training in Europe
• Pitfalls in comparing indicators generated from different data collections on vocational training
• Towards a coherent framework for indicators on outputs and outcomes of vocational training
• Bibliography
• Annex – Proposal for a framework for indicators on outputs and outcomes of vocational training

4
I. Introduction - the political dimension
The Maastricht Communiqué 1 on the future priorities of enhanced European cooperation in vocational
education and training calls for the improvement of the scope, precision and reliability of VET statistics
in order to allow for the evaluation of progress in making VET efficient, effective and attractive.
The document also stresses the need for adequate data and indicators, which are considered crucial to
understanding what is happening in VET, and describes which additional interventions and decisions
are required by all parties involved. The Communiqué also calls for the assessment of progress towards
the Lisbon goals and sets new priorities.
The Council Conclusions of 21 February 2005 on Education and Training in the framework of the
mid-term review of the Lisbon Strategy recommend the identification of indicators in various new
fields, and their eventual employment, as envisaged in the Joint Interim Report. New indicators
should be developed, in particular on the distribution of VET participants according to place of tuition,
participation in apprenticeship programmes and access to higher levels of education for VET
graduates. The Council also recommends using the European Qualifications Framework as a common
reference instrument covering both vocational and general education (at secondary and tertiary levels),
based on competencies and learning outcomes.
There is therefore scope to construct a new statistical framework for reporting the outputs and
outcomes of vocational training that will better target data needs, thereby making reporting more
explicit and customised for different users.
II. The state of the art in reporting on outputs and outcomes of vocational training in
Europe
This section deals briefly with the limits of the existing array of statistics and indicators on outputs
and outcomes of vocational training, in order to gain some insight into the way in which these outputs
have been mapped and with what scope. It looks into the different statistical approaches used to report
on vocational training (both initial and continuing).
Vocational education and training is permanently on the political agenda of the European Union. To
inform the policy debate and to reinforce the accountability of education systems, the Standing Group
on Indicators and Benchmarks (SGIB) has considered various types of measures making use of available
information. Currently the following indicators are being used to monitor performance and progress
in vocational training systems in Europe:
1 http://europa.eu.int/comm/education/news/ip/docs/maastricht_com_en.pdf

5
1. Hours in continuing vocational training (CVT) courses per 1000 working hours (in enterprises
offering CVT courses by NACE)
2. Hours in continuing vocational training (CVT) courses per 1000 working hours in all enterprises
by NACE
3. Enterprise expenditure on continuing vocational training courses as a percentage of total
labour costs
4. Participation rates in education by age and by level of education (i.e. proportion of students
in upper secondary education enrolled in vocational streams at ISCED 3)
Several other indicators have been proposed during SGIB meetings, but the lack of consensus has
kept the indicators used in the first and second edition of the report Progress towards the Lisbon Objectives
in Education and Training limited to those listed above. A decision has been taken towards
computation, in the medium term, of new indicators on VET, 2 especially on participation rates. With
the new round of Continuing Vocational Training Survey (CVTS 3), there is scope to construct composite
measures on participation in and expenditure on VET.
Internationally comparable data on vocational training focus on compulsory education and only to a
limited extent cover VET at tertiary level, therefore we cannot really speak of “reporting on the VET
sector.” Most data collections do not include a separate category for VET. 3 It should be noted, however,
that the education and training landscape in the European Union has evolved in past decades and
the distinctions between traditionally differentiated educational pathways of general-higher-vocational
training have started to become blurred as a result of changing social, economic and political priorities
(Deane and Watters, 2004). This also affects the statistical frameworks used by data providers, which
are now under constant pressure to assess new learners’ profiles and needs.
Three categories of indicators are currently being reported at an international comparable level:
Output indicators (e.g. achievements in related subjects or performance in selected skill domains,
based on literacy scales)
Outcome indicators (e.g. attainment levels, self-reported skills levels, etc.)
Impact indicators (e.g. labour market position, returns to education, societal effects, social
participation, etc.)
As a result of reporting practices, only a few simple statistics on VET are available, such as the distribution
of students enrolled in vocational programmes at different ISCED levels, the theoretical dura-
2 New Indicators on Education and Training, Commission Staff Working Paper, SEC (2004) 1524
3 In some cases VET data is reported as part of upper secondary education, in other cases as part of postsecondary
non-tertiary education, depending on the level of the course studied.

6
tion of vocational programmes, etc. Simple indicators on participation in (or graduation from) vocational
programmes are not currently reported, although the raw data are easily accessible in the databases.
There is a lack of comparable data on the volume of VET provision and the links to national
qualification frameworks, to transition processes, etc. Direct (internationally comparable) output indicators
(i.e. student achievements in basic subjects and competencies) are not available for upper secondary
vocational education, with the exception of TIMSS data (“final year”). The PISA database
does not contain results of students’ assessment by type of programme. The situation is similar for
outcome indicators. Cohort data are missing and limited information is available on effectiveness and
success rates in VET.
As a result, many proxies are used for measurement of skills and competencies, even though they are
poor in this sense. Some inconsistencies in reported results still exist across indicators covering participation
in or completion of a certain level of education, as defined internationally. In some cases
these inconsistencies are considerable; for instance, the results for upper secondary completion can
differ by over 15 percentage points from one source to another (Behringer and Pfeifer, 2004). When
data is reported by the same data provider this makes the use of data even less reliable.
In conclusion, at present there are considerable variations in the results available from different data
providers in:
i) the way in which vocational training complies with the existing statistical frameworks;
ii) the extent to which other aspects of vocational training are captured, such as the volume
of training, the cost of training, the provision of training;
iii) the way in which vocational training is delivered and, more importantly, how it is linked
to a national framework of qualifications.
These issues need to be clarified and appropriate methodological work carried out in order to match
the Commission’s needs for statistics and indicators with the proposed (and largely agreed) data collection
exercises, as well as to identify the areas where additional data will be have to be gathered
through a separate data collection.
The next section is a critical overview of the data sources (existing or forthcoming) that could be exploited
to generate new indicators on outcomes of vocational training in Europe. It should be noted
that all the data providers are making efforts to improve the collection instruments and to capture as
much information as possible in this field. New data collections have been launched in past years, especially
by Eurostat, and the results are expected to improve the availability of statistics on education
and training.

7
1. Data gathered from administrative sources
VET data collection is the standardised collection of information on vocational programmes in the
EU member states. Data on vocational programmes is gathered from administrative sources. The units
responsible for reporting are the ministerial departments/offices in charge of statistics or the National
Statistical Institute of each country. The data collection started in 1995 (with reference year 1993/94)
and was organised jointly by Eurostat and Cedefop, in the framework of the LdV programme. The
exercise was initially designed to respond to policy requests (in particular to seek alternatives to academic
routes), as a tool for policy-makers which could help them to design, monitor and evaluate national
training policies. VET data collection is the basic source of information for attainment in VET
and one of its advantages is the possibility to generate indicators by type of programme (or strands),
allowing different reporting for pre-vocational and vocational programmes. In VET data collection, a
vocational programme is defined as “any activity whose aim is to promote the acquisition of the necessary
knowledge, skills and attitudes for the exercise of an occupation or group of occupations.” In
the first data collection (1993/94) four categories of vocational programmes were defined, according
to the target population, the type of training, the place of training, the working status of participants,
as follows:
i) programmes for initial vocational education and training (target group: young people of
school age);
ii) training programmes for adults (target group: adult population);
iii) pre-vocational programmes (identified as such by the names and descriptions given at national
level);
iv) education and training programmes for disadvantaged groups/groups at risk, such as
(youth) unemployed, disabled people, migrants.
The above coverage has proved to be complete only for the first category, and has led to a revision of
the three other categories. In addition, for the first two rounds of data collection (reference years
1994/95 and 1995/96) the distinction between the two categories (initial VET and training for adults)
was eliminated and the two categories were merged into a single one called “vocational training.” At
the same time the coverage of pre-vocational programmes was improved. The last category (education
and training programmes for disadvantaged groups) was dropped, as it was felt that data were not
available and the questionnaire not designed well enough to collect this information. Vocational programmes
offered at tertiary level (ISCED 5A and 6) were excluded. Data collection was revised
again in 1998 by redefining its scope and by adding new criteria. In 2000 VET data collection was
suspended by Eurostat due to lack of resources. Several attempts were made to revive the exercise,
including a meeting called by Cedefop in November 2002 in which representatives from Member
States, Acceding and Candidate Countries, Eurostat and DG EAC concluded that the data collection
should be re-started as soon as possible (minutes of the meeting are available on Circa). VET data

8
collection is likely to be the most effective vehicle to gather information on VET and to generate
indicators in this area.
The UOE data collection is another secondary collection of data gathered from administrative
sources (registers of educational institutions), run jointly by the UNESCO Institute for Statistics
(UIS), the OECD and Eurostat. The units responsible for reporting are the ministerial departments/offices
in charge of statistics or the National Statistical Institute of each country. UOE is the
basic source of information for attainment in VET, allowing separate reporting on pre-vocational and
vocational programmes; output indicators by type of programme (strand) can be made available.
One problem with the UOE data is the ‘netting out’ of graduates from different years or earning multiple
qualifications during the reference school year. Another problem of the UOE data is that reporting
is based on an internationally comparable classification of educational programmes (ISCED)
which is often inappropriately used as a proxy for qualifications (see also the following sections in
this paper). The mapping of qualifications by level of education is rather subject to political negotiations
than underpinned by research, leading to several inconsistencies across countries as to what
should fall under the category of an ISCED-3 qualification (Behringer and Pfeifer, 2004). Eurostat is
currently working on a revision of the UOE data collection aiming to ‘net out’ the adults participating
in adult education programmes. In January 2006 Eurostat put forward to the two other partners (UIS
and the OECD) a proposal for a revision policy of the UOE, looking at improving the comparability,
relevance and coherence of the data. In the absence of a dedicated data collection on VET, UOE remains
the main source for data on vocational education and training.
2. Data gathered from surveys
The Continuing Vocational Training Survey (CVTS) is an enterprise survey carried out by Eurostat
in companies with at least ten employees. There have been two rounds of CVTS, one conducted in
1994 for twelve EU Member States and another round conducted in 2000/01 in all fifteen EU Member
States, Norway and nine Candidate Countries (of which seven became new Member States in 2004).
The survey provides, on a comparable basis for the two rounds, information on employer-sponsored
training taken during the year prior to the survey, for employed persons, excluding apprentices and
trainees. A large set of variables for the enterprises (such as costs of training, incidence of training by
sector, etc.) is available from the survey but only a limited number of variables relate to employees
(total training hours and training participation). The coverage of the different forms of training for
employees refers to training courses which take place away from the place of work (e.g. in a classroom
or a training centre, at which a group of people receive instruction from teachers/tutors/lecturers
for a period of time specified in advance by those organising the course (i.e. intended training). Initial
vocational training (pre-vocational and vocational programmes) undertaken by a person when hired in
order to gain the competencies required for the job is excluded. Post-graduate education is included in
the definition of training. CVTS will be carried out in 2006 in all EU Member States and Acceding

9
Countries on a legal basis. While keeping the results as comparable as possible with the previous two
rounds of survey, CVTS 3 will provide additional information related to the companies (e.g. training
strategies) and, more importantly, new information related to employees (e.g. reasons for nonparticipation).
The coverage will be enlarged to include some initial vocational training (formal learning
leading to a formal qualification).
The Adult Education Survey (AES) is a new initiative of Eurostat, already carried out in two Member
States in 2005 (Sweden and the United Kingdom). AES will be carried out in 2006/07 in all other
Member States and the two Acceding Countries. The majority of countries will implement AES as a
stand-alone survey; some will include AES variables within existing surveys (mostly as an ad-hoc
module of the LFS) and only two participating countries will adapt national surveys on adult education.
Some AES variables could be used to generate indicators on outputs of training, especially
on patterns of participation in training (including attitudes towards and obstacles to learning), incidence
of training at the work place, use of ICT and self-reported language skills. Crossed with background
variables (e.g. self-reported educational attainment level, working status), these could be useful
in generating composite measures on participation in adult education. AES will be implemented
under a legal basis and will have a good level of comparability as a result of using the international
classifications for education status, working status and occupational status, as well as the European
definitions of the labour force variables. The survey will use the Classification of Learning Activities
(CLA), which is a very useful tool, agreed upon by European statisticians for the classification and
comparison of learning. AES is expected to represent an important source of information on individuals’
and employers’ investments in training.
The Labour Force Survey (LFS) remains the main vehicle to gather information on participation in
training over a period of time. At present, three of the five European benchmarks used to monitor progress
in education and training at the European level are constructed using LFS variables, but LFS
variables for monitoring purposes are being used on a much larger scale throughout the EU’s Structural
Indicators. LFS has been carried out since 1993 and represents a good source of information on
participation on education and training, offering complete coverage at EU level in a timely manner.
Another advantage of LFS is the possibility to capture additional information on a specific topic by
running a so-called ‘ad-hoc module.’ The modules are technically totally compliant with the coresurvey
and offer the possibility to construct additional variables other than the LFS core variables.
Two ad-hoc modules have been carried out so far, one module on the transition from education to
working life (reference year 2000) and another module on participation in lifelong learning (reference
year 2003). A new module is planned to be carried out in 2009 on the entry of young people into the
labour market. The results of the latest ad-hoc module on lifelong learning have been extensively analysed
in the latest edition of the Progress Report (2006), showing interesting results in patterns of participation
in lifelong learning, but no specific indicators on vocational training have been made

10
available. There is still scope for further analysis and for generating new indicators, especially on
outputs of learning and attitudes towards learning. The coverage of data will be comparable with the
forthcoming surveys (e.g. AES) and proxies for participation by field of study could become available
following the revision of the LFS coding of ISCED 3C.
The European Community Household Panel (ECHP) is the EU survey that provides information
on issues like a household’s income and poverty level. The survey was carried out between 1994 and
2001. Some of the ECHP variables have been used to generate statistics and indicators on outputs of
vocational training (e.g. participation in vocational training by type of programme or by objective of
training) which, crossed with the socio-demographic background variables, have shown interesting
results. One weakness of the survey is that it follows the ISCED 76 levels of education which restricts
the comparability with the following surveys using ISCED 97 and different fields of study. ECHP
data show firstly that training and investment in research and development are complementary, but the
degree of complementarity is lower for graduates from college, and secondly that when schooling is
more comprehensive, graduates from high-schools require less training to adapt to technical progress
(Bassanini et al., 2005). From 2003 ECHP was replaced by a survey on income and living conditions
(EU-SILC). This survey will provide information on an annual basis on outputs of vocational training
(such as educational attainment levels) which could again be crossed with background variables.
There are at least two important strengths of EU-SILC; the first related to the possibility of getting
cross-sectional and longitudinal variables related to current activity status (i.e. current main job, including
information on the last main job for the unemployed). The second advantage of EU-SILC is
the possibility of running ad-hoc modules (as in the LFS) in order to gather additional information on
a specific topic. Three ad-hoc modules are already planned in the EU-SILC: intergenerational transfer
of poverty (reference year 2005), social protection (reference year 2006), and housing inadequacy
(reference year 2007). There is scope to run a specific ad-hoc module on household expenditure on
vocational training in 2009 which, along with the LFS ad-hoc module that will be carried out in the
same year on entry of young people into the labour market, would be a rich source of information on
outcomes of vocational training.
The International Adult Literacy Survey (IALS) is an individual survey which was carried out by
the OECD and Statistics Canada in the 1990s. Data is available only for some EU Member States:
Ireland, the Netherlands and Poland (reference year 1994); Belgium (Flanders only) and the United
Kingdom (reference year 1998); the Czech Republic, Denmark, Finland and Hungary (reference year
1998). The coverage of training includes training offered to employees in the past twelve months and
includes details on the three most recent courses. These details refer to duration, purpose, financing,
training provider, etc.). In the survey a distinction was made between job- or career-related training
and training for other purposes. Furthermore, training courses were divided into seven mutually exclusive
categories, based on level of education and type of programme. IALS and its follow-up (Adult

11
Literacy and Life Skills Survey with data available only for Italy, 2003) are a rich source of output,
and especially outcome, indicators on vocational training (including social outcomes). Several measures
could be used to measure the outcomes of vocational training, both job-related and training for
other purposes. IALS data can be extensively used for analysis with a view to gaining policy insights.
Eurobarometer is an opinion poll run by the Directorate General for Press and Communication (DG
PRESS) and conducted by opinion poll companies to gather information on a certain topic (public
opinion and attitudes towards it). At present results relevant to education and training are available
from two Eurobarometer polls on: experiences and attitudes towards learning (reference year 2003)
and continuing and initial education and training (reference year 2004). The questionnaires were developed
by Cedefop in cooperation with the Directorate General for Education and Culture (DG EAC)
and the European Opinion Research Group (EORG), working on behalf of DG PRESS. In all Eurobarometer
surveys, the following socio-demographic variables are collected on a standard basis: gender,
age, civil status, political opinion (left-right scale), age when finishing full-time education and
training, source of household income, range of income, current and last occupation and size of residential
community. This analysis focuses mainly on sex, age, level of education and occupation.
Eurobarometer categories use four age-groups: 15-24, 25-39, 40-54 and 55+. Eurobarometer categories
do not correspond to ISCED levels of education, but are based instead on the question “How old
were you when you stopped full-time education” Answers are allocated to one of four categories: up
to 15 years old; between 16-19 years old; at age 20 or older and those still studying. Respondents are
also allocated to one of the following occupational categories: self-employed, managers, other white
collar employees, manual workers, full-time homemakers, unemployed and retired. The retired include
not only those retiring in the usual way on reaching a certain age, but also those retiring early on
health grounds. Education and occupation are combined to produce a proxy that enables breakdowns
comparing three contrasting groups of people: highly educated with a high-level job, less-well educated
with a low-level job, less-well educated who are not active in the labour force.
III. Pitfalls in comparing indicators generated 4 from different collections on vocational
training
Through the collection of information on vocational training at each different level (individual/employee,
employer, company, system) on a cross-nationally comparable basis, every new survey
4 There are differences between data generation and data provision. A data provider is an organisation which
runs specific data collection exercises such as (inter)national surveys, in order to obtain information which is

12
adds to the knowledge base. However such new insights come at a cost, as every new survey is both a
resource-intensive and a methodologically complex undertaking.
For a cross-country survey, the most challenging stage in the development of a data collection exercise
is to build a consensus among the participating countries on the issues that will be investigated
and then to define and operationalise these in ways that are cross-nationally valid. Even after
agreement is reached, there is scope for various interpretations of the results among the participants.
For instance, in CVTS 2, Italy scored badly in the European context on training activities for the employed
population, with only one company in four carrying out any form of training activity (as opposed
to a 62% average of European companies). However, this could be explained by the peculiarities
of Italian industries, which are traditionally composed of small to medium-sized companies in
sectors such as: clothing, footwear, furnishing, etc. (Bulgarelli, 2005). This was largely explained by a
lower propensity of small companies to provide training to their employees and consequently affected
the results of Italy in the survey (which covered establishments with ten or more employees).
Another issue to be considered is the use of national surveys to measure trends on an internationally
comparable basis. It is largely agreed that changing the definitions and the coverage will always
break a time series. This happened in 2003 with a couple of indicators (on participation in lifelong
learning and early school leaving) calculated from the EU LFS variables. Many countries decided to
change the definition in the national surveys (by reporting separately participation in formal education
and other forms of learning), which has inevitably led to a break in series for these countries. In the
case of participation in lifelong learning, the changes in national surveys have had a broader impact
(Behringer and Pfeifer, 2004). With the implementation of the new definitions (which were designed
to improve the coverage of the survey) an increase in the overall participation has been recorded for
all educational attainment levels (as they are defined in LFS: low, medium, high), but by widening the
gaps between groups, which in previous years had been constant.
Another constraint concerns geographical coverage of a data collection. Most of the indicators used
for monitoring purposes are generated from EU surveys. When trying to incorporate non-European
countries, there is not much information available apart from UOE data. Even if information were
available at national level, it is very difficult to adapt it to the coverage of each individual data collection
(definitions, methodology, timeliness, etc.). Geographical coverage was dealt with by OECD for
analytical purposes for CVTS 2 and IALS data, to increase the coverage of countries. Data on both
participation rates in training and the log of training hours per employee were merged and a crosssubsequently
published. A data generator is an organization that makes use of statistical information which
already exists by putting it into a usable format for comparisons and analysis.

13
survey training index constructed for each country; final cross-survey measures were then reconstructed
based on this index (for details see OECD Employment Outlook 2004).
Timing of the reference period in a data collection exercise is another particularly important issue to
consider when comparing results. This was the case for IALS, which was carried out in several cycles
between 1994 and 2001. This has introduced an element of uncertainty in reporting the results, although
one might expect that adult literacy as it is measured in IALS would not change significantly
over a couple of years.
IV. Towards a coherent framework for indicators on outputs and outcomes of vocational
training
The initial discussion of a coherent framework of indicators and benchmarks to be used for monitoring
progress in the field of education and training took place in November 2003 during an international
seminar organised under the Italian presidency. The issues tabled and addressed at that time
have recently re-entered discussion, since the need to establish a coherent framework of indicators has
been revived with the Council Conclusions of May 2005 on new indicators in education and training.
Several discussions have taken place within the Standing Group on Indicators and Benchmarks over
the past years, beginning with input from Frans Kaiser (University of Twente), who proposed two
possible approaches (the “productive” and the “mapping process”) to the development of a coherent
framework of indicators and benchmarks for the follow-up of the Lisbon objectives in education and
training. A discussion paper distributed for the latest SGIB meeting (Ispra, April 2006) stated that the
approach requested by the Council for a coherent framework of indicators and benchmarks could be
named “the policy approach,” in which the dominant criterion is political coherence. From this perspective,
other forms of coherence (input/process/output/outcome/impact or technical/scientific) are
consequently complementary criteria.
On a more general level, a discussion of a coherent framework for indicators should presumably take
into account the multi-faceted relationships between the data generated and the expected policy insights
which an analysis of the data would yield. The management of the needs of different users and
the limits of the data collections should also be part of the discussion.
There are several approaches to indicators and their frameworks. The OECD uses a matrix that organises
the indicators to distinguish between the actors in education systems while grouping the indica-

14
tors according to whether they speak to learning outcomes, circumstances that shape these outcomes
or constraints that set policy choices into context. Policy issues (such as equity in educational outcomes
and educational opportunities, the quality of educational outcomes and educational provision,
effectiveness of resource management) are dealt with in the same framework. With the strong support
of new statistical tools, the OECD moves from the well-known, classical approach of a framework of
Input-Process-Output-Outcome towards a more policy-oriented, empirical approach, where new concepts
such as ‘policy levers’ or ‘policy space’ are introduced to outline the multi-faceted relationships
between data generated with the new tools and the policy analysis of various educational themes. One
can distinguish in this approach a combination of three different dimensions: statistical, analytical and
policy-oriented.
As the main data provider on education and training for EU member states, Eurostat has recently embraced
a linear approach to describe the education process, as shown in the figure (from a recent
document on “the way forward” for education statistics).
Inputs
Expenditure
Funding
Entrants
Students
Teachers
Process
Schools
Classes
Educ. Methods
ICT usage
Teacher Training
Working time
Violence
…
Outputs
Graduates
Early leavers
Dropouts
Outcomes
Skills
Indiv. Returns
Eco. Returns
Social returns
A combination of the two approaches will be used to organise the indicators on outputs and outcomes
of vocational training. It is hoped that this will better target data needs, thereby making reporting
more explicit and customised for different users. The framework does not aim to reflect a consensus
among professionals on how to map the measures of the current state of vocational training internationally,
but rather will try to advance the debate on indicators and their framework.

In defining the components of such a framework, a first step will be to define the education
and training variables that will form the analytical component of the framework. The variables
will need to respond to issues that are high on policy agendas (i.e. Council Conclusions
or Commission Communications) and where the comparative perspective can offer important
added value to what can be achieved through national evaluation exercises. This analysis
should also shed light on national circumstances or constraints that contextualise the policy
choices. The association of the background variables with the survey variables have been
successfully used in PISA with the help of two composites. 5
Beginning the process of definition at the analytical level will also help to identify topics
which are present on the political agenda or in key political messages but which have not yet
been (sufficiently) examined or empirically supported; the analytical component could be
strengthened with new data or through new combinations of existing statistics (i.e. with the
help of indexes). Additional statistical techniques (such as sensitivity analysis) could be of
help in identifying the variables that have been successfully used in the analytical work and
should therefore be refined and included in the new data collection exercises.
Finally, a well-defined analytical dimension would allow for a better choice of proxy measures
for outputs and outcomes of vocational training. In the example of educational attainment
level as a proxy for skills, doubts about the most appropriate interpretation of the figures
are, to some extent, borne out by the PISA results. Four new Member States participated in
IALS between 1994/94 and 1998/99 (the Czech Republic, Hungary, Poland and Slovenia) and
PISA was carried out in four countries (the Czech Republic, Hungary, Poland and Latvia) in
2000. Of these countries, only one participant in both surveys (the Czech Republic) had results
close to (and, in some skills areas, above) the average of participating countries. In Hungary,
Poland and Slovenia very high proportions of the populations aged 16-65 had low functional
literacy skills (at Level 2 or below on a five- point scale) as measured in IALS – especially
for prose and document literacy. Similarly, in both Hungary and Poland nearly one in
two (and in Latvia three in five) of all 15-year-olds had low reading literacy skills (at Level 2
or below on a five-point scale) as measured in PISA. Science literacy skills (and to a lesser
extent maths literacy) among 15-year-olds and quantitative literacy skills among adults aged
16-65 were slightly better (Kennedy and Badescu, 2004).
5 The PISA index of economic, social and cultural status was derived from a set of family background
variables such as: the highest international socio-economic index of occupational status of the
father or mother, the highest level of education of the father or mother converted into years of schooling,
the access to home educational and cultural resources (e.g. books, computers, internet connection,
other learning tools). The PISA index of instrumental motivation was derived from the frequency
with which students study for the following reasons: to increase my job opportunities; to ensure that
my future will financially secure; to get a good job. In a framework on outcomes of vocational training
a variable such as the social background of students should also include, besides the socio-economic
index (which could be used as a composite measure of parental status). measures of antecedents such
as: participation of parents in vocational training, unemployed persons in the same family or occupational
career of parents.

References
European Commission documents
Maastricht Communiqué on the future priorities of enhanced European cooperation in
vocational education and training, 6 Brussels, December 2004
Communication from the Commission: Modernising education and training: a vital contribution
to prosperity and social cohesion in Europe, COM (2005) 549 F, Brussels, 10
November 2005
Commission Staff Working Paper: Modernising education and training: a vital contribution
to prosperity and social cohesion in Europe, SEC (2005) 1415, Brussels, 10 November
2005
Commission Staff Working Paper: Progress towards the Common Objectives in Education
and Training. Indicators and Benchmarks, SEC (2005) 419, Brussels, 22 March 2005
Commission Staff Working Paper: New Indicators on Education and Training, SEC (2004)
1524, Brussels, 29 November 2004
Other documents
Leney, T. et al, “Achieving the Lisbon goal: the contribution of VET,” Final report to the
European Commission, Brussels, November 2004
Behringer, F., Pfeifer, H., “Indicators and Data for VET,” in Achieving the Lisbon Goal: the
contribution of VET, QCA, London, 2004
Deane, C., Watters, E., “Towards 2010 – Common Themes and Approaches across Higher
Education and VET in Europe,” Background Research Paper, Irish Presidency Conference,
March 2004
Bassanini, A. et al., “Workplace Training in Europe,” IZA Discussion Paper, June 2005
Bulgarelli, A., “Indicators and Benchmarking as a support to the decision-making process: the
Italian experience in active employment policies” in Statistics, Knowledge and Policy: Key
Indicators to Inform Decision Making, OECD, Paris, 2005
Descy, P. et al., “Internationally comparable statistics on education, training and skills: stateof-the-art
and prospects,” Draft article for the European Journal Vocational Training, Cedefop,
Thessaloniki, 2005
Eurostat (2000), VET data collection., Methods and definitions, Eurostat, Luxembourg, 2000
Kennedy, A., Badescu, M., A Comparative Review of Employment and Training Statistics, in
ETF Yearbook 2004, European Training Foundation, Turin, 2004
OECD (2005), Education at a Glance – OECD Indicators 2005, OECD, Paris, 2005
OECD (2004), “Improving Skills for More and Better Jobs: Does Training Make a Difference,”
in OECD Employment Outlook, OECD, Paris, 2004

Measuring the outputs and outcomes of vocational training – towards a coherent framework for indicators
Analytical
Dimensions of the framework
Policy
Statistical
Data generated
What data can show
Availability
(if YES, source)
Statistical
unit
Individual
Employer
System
Data issues
Frequency of data
generated
Remarks on data
generated
No solid evidence for
under-provision of workplace
training
Combination of vocational
schools with on-the-job
training apprenticeship
better suited to facilitate
transition from school to
work
Participation patterns
significantly different for
graduates of higher education
as opposed to VET
graduates
Training increase with
education and skillintensity
of occupations
Highly educated people
more likely to participate in
the labour market
Significant differences
between training provision
(public vs. private)
Workplace training seems
much more associated to
private returns than public
training
Significant differences in
the access to training for
younger and older workers
depending on the intensity
of training
Training provision goes
mostly to highly educated
people
Improving the image
and attractiveness of
vocational route for
employers and individuals
in order to increase
participation in VET
Flexible and open
frameworks for VET in
order to increase progression
between initial
and continuing training
and higher education
Equipping young
people with key competences
they will require
throughout life
Linking VET with the
labour market requirements
of the knowledge
economy for a highly
skilled workforce
Improving public and/or
private investment in
VET by the training
incentive effects of tax
and benefit systems
Renewing and updating
skills of an ageing
population
Development of VET
systems to meet the
needs of people or
groups at risk of labourmarket
and social
exclusion (i.e. early
school leavers, low
skilled, migrants, etc.)
Continuing competence
development of teachers
and trainers in VET,
reflecting their specific
learning needs
To better capture
participation in VT
strand at typical
age/age groups
Unduplicated
number of graduates
by netting out
graduates with
more than one
qualification during
the reference period
(‘typical’ VT
students)
Direct assessment
of skills have
proved to be better
measures than selfreporting
Use of LFS variables
to capture the
transition patterns
for young people
Age-training profiles
are downwardsloping
in all countries
Participation in
initial VT
programmes
Participation in
continuing VT
courses
Survival rate in
VT strand
Assessment of
job specific
skills for
youths
Incidence of
mismatching of
jobs
Re-scaling of
age-training
gaps between
younger and
older workers
Training gaps
for groups at
risk
Participation
patterns and
changes in
patterns over
time
Current output
of VT
How job-related
competencies
relate to outcomes
of youth
and adult agegroup
Employment
patterns of
school leavers
How some
measures of
participation in
training can be
further used to
explain differences
among
countries
How the issue
of equity in
training provision
can be
measured and
assessed
Teaching and
working time of
teachers and
trainers
YES,
UOE
YES,
CVTS 3
YES
UOE
YES
X Annual Scope for a composite measure
on participation in IVT
and CVT
X X Annual Scope for better coverage
and reducing data gaps
NO X X 5-yearly New data collection needed
or integration of data needs
into another survey
YES
LFS-
AHM
YES
EU-
SILC
YES
UOE/Fi
nance
and
CVTS3
YES
CVTS
3,
LFS-E
YES
LFS-E
Partially
yes for
teachers
NO for
trainers
X X X 5-yearly,
subject to
agreement
in
Eurostat
X X Annual
for IVT
5-yearly
for CVT
X X X 5-yearly
for CVT
Annual
for LLL
X X Quarterly
Annual
Scope for analytical work
using social background
variables
Scope for work with longitudinal
data
Scope for constructing
composite measures on costs
of vocational training.
Scope for study of efficiency
of investment in VT by
source of funds
Scope for study of companies’
expenditure on IVT
Better coverage for participation
in any kind of learning
through LFS
Scope for further breakdowns
for low-skilled in
LFS-E, incl. ESL
Better coverage for participation
in any kind of learning
through LFS
X Annual Weak coverage of trainers
New data collection instrument
needed, possible integration
in other surveys
Further development of
learning-conducive
environments in training
institutions and at the
workplace
Attitudes towards
learning,
reasons for nonparticipation
Differences
between attitudes
and
actual
participation in
VT
UOE-
Teachers
YES
Euro
barometer
and
AES
X X X Each time
with Euro
barometer
5-yearly
with AES
Weak comparability between
the two tools

European Commission
EUR 22305 EN – DG Joint Research Centre, Institute for the Protection and Security of the Citizen
Measuring the Outputs and Outcomes of Vocational Training - Towards a Coherent Framework for Indicators
Author: Mircea Badescu
Luxembourg: Office for Official Publications of the European Communities
2006 – 20 pp. – 21 x 29.7 cm
EUR - Scientific and Technical Research series; ISSN 1018-5593
Abstract
This paper proposes a new analytical framework for monitoring the outcomes and outputs of voca-tional training.
It includes a critical review of the main statistical sources used to measure vocational training (both initial and continuing)
and to generate indicators. It also presents some country or cross-country comparative results of empirical analysis.
The annex outlines the three dimensions of the framework (analytical, policy and statistical), using some examples of output
and outcome indica-tors that could be developed.

The mission of the Joint Research Centre is to provide customer-driven scientific and technical support for the conception,
development, implementation and monitoring of European Union policies. As a service of the European Commission,
the JRC functions as a reference centre of science and technology for the Union. Close to the policy-making
process, it serves the common interest of the Member States, while being independent of special interests, whether
private or national.