measuring the outputs and outcomes of vocational training

In defining the components of such a framework, a first step will be to define the education
and training variables that will form the analytical component of the framework. The variables
will need to respond to issues that are high on policy agendas (i.e. Council Conclusions
or Commission Communications) and where the comparative perspective can offer important
added value to what can be achieved through national evaluation exercises. This analysis
should also shed light on national circumstances or constraints that contextualise the policy
choices. The association of the background variables with the survey variables have been
successfully used in PISA with the help of two composites. 5
Beginning the process of definition at the analytical level will also help to identify topics
which are present on the political agenda or in key political messages but which have not yet
been (sufficiently) examined or empirically supported; the analytical component could be
strengthened with new data or through new combinations of existing statistics (i.e. with the
help of indexes). Additional statistical techniques (such as sensitivity analysis) could be of
help in identifying the variables that have been successfully used in the analytical work and
should therefore be refined and included in the new data collection exercises.
Finally, a well-defined analytical dimension would allow for a better choice of proxy measures
for outputs and outcomes of vocational training. In the example of educational attainment
level as a proxy for skills, doubts about the most appropriate interpretation of the figures
are, to some extent, borne out by the PISA results. Four new Member States participated in
IALS between 1994/94 and 1998/99 (the Czech Republic, Hungary, Poland and Slovenia) and
PISA was carried out in four countries (the Czech Republic, Hungary, Poland and Latvia) in
2000. Of these countries, only one participant in both surveys (the Czech Republic) had results
close to (and, in some skills areas, above) the average of participating countries. In Hungary,
Poland and Slovenia very high proportions of the populations aged 16-65 had low functional
literacy skills (at Level 2 or below on a five- point scale) as measured in IALS – especially
for prose and document literacy. Similarly, in both Hungary and Poland nearly one in
two (and in Latvia three in five) of all 15-year-olds had low reading literacy skills (at Level 2
or below on a five-point scale) as measured in PISA. Science literacy skills (and to a lesser
extent maths literacy) among 15-year-olds and quantitative literacy skills among adults aged
16-65 were slightly better (Kennedy and Badescu, 2004).
5 The PISA index of economic, social and cultural status was derived from a set of family background
variables such as: the highest international socio-economic index of occupational status of the
father or mother, the highest level of education of the father or mother converted into years of schooling,
the access to home educational and cultural resources (e.g. books, computers, internet connection,
other learning tools). The PISA index of instrumental motivation was derived from the frequency
with which students study for the following reasons: to increase my job opportunities; to ensure that
my future will financially secure; to get a good job. In a framework on outcomes of vocational training
a variable such as the social background of students should also include, besides the socio-economic
index (which could be used as a composite measure of parental status). measures of antecedents such
as: participation of parents in vocational training, unemployed persons in the same family or occupational
career of parents.