measuring the outputs and outcomes of vocational training

More documents

Recommendations

Info

12 adds to the knowledge base. However such new insights come at a cost, as every new survey is both a resource-intensive and a methodologically complex undertaking. For a cross-country survey, the most challenging stage in the development of a data collection exercise is to build a consensus among the participating countries on the issues that will be investigated and then to define and operationalise these in ways that are cross-nationally valid. Even after agreement is reached, there is scope for various interpretations of the results among the participants. For instance, in CVTS 2, Italy scored badly in the European context on training activities for the employed population, with only one company in four carrying out any form of training activity (as opposed to a 62% average of European companies). However, this could be explained by the peculiarities of Italian industries, which are traditionally composed of small to medium-sized companies in sectors such as: clothing, footwear, furnishing, etc. (Bulgarelli, 2005). This was largely explained by a lower propensity of small companies to provide training to their employees and consequently affected the results of Italy in the survey (which covered establishments with ten or more employees). Another issue to be considered is the use of national surveys to measure trends on an internationally comparable basis. It is largely agreed that changing the definitions and the coverage will always break a time series. This happened in 2003 with a couple of indicators (on participation in lifelong learning and early school leaving) calculated from the EU LFS variables. Many countries decided to change the definition in the national surveys (by reporting separately participation in formal education and other forms of learning), which has inevitably led to a break in series for these countries. In the case of participation in lifelong learning, the changes in national surveys have had a broader impact (Behringer and Pfeifer, 2004). With the implementation of the new definitions (which were designed to improve the coverage of the survey) an increase in the overall participation has been recorded for all educational attainment levels (as they are defined in LFS: low, medium, high), but by widening the gaps between groups, which in previous years had been constant. Another constraint concerns geographical coverage of a data collection. Most of the indicators used for monitoring purposes are generated from EU surveys. When trying to incorporate non-European countries, there is not much information available apart from UOE data. Even if information were available at national level, it is very difficult to adapt it to the coverage of each individual data collection (definitions, methodology, timeliness, etc.). Geographical coverage was dealt with by OECD for analytical purposes for CVTS 2 and IALS data, to increase the coverage of countries. Data on both participation rates in training and the log of training hours per employee were merged and a crosssubsequently published. A data generator is an organization that makes use of statistical information which already exists by putting it into a usable format for comparisons and analysis.
13 survey training index constructed for each country; final cross-survey measures were then reconstructed based on this index (for details see OECD Employment Outlook 2004). Timing of the reference period in a data collection exercise is another particularly important issue to consider when comparing results. This was the case for IALS, which was carried out in several cycles between 1994 and 2001. This has introduced an element of uncertainty in reporting the results, although one might expect that adult literacy as it is measured in IALS would not change significantly over a couple of years. IV. Towards a coherent framework for indicators on outputs and outcomes of vocational training The initial discussion of a coherent framework of indicators and benchmarks to be used for monitoring progress in the field of education and training took place in November 2003 during an international seminar organised under the Italian presidency. The issues tabled and addressed at that time have recently re-entered discussion, since the need to establish a coherent framework of indicators has been revived with the Council Conclusions of May 2005 on new indicators in education and training. Several discussions have taken place within the Standing Group on Indicators and Benchmarks over the past years, beginning with input from Frans Kaiser (University of Twente), who proposed two possible approaches (the “productive” and the “mapping process”) to the development of a coherent framework of indicators and benchmarks for the follow-up of the Lisbon objectives in education and training. A discussion paper distributed for the latest SGIB meeting (Ispra, April 2006) stated that the approach requested by the Council for a coherent framework of indicators and benchmarks could be named “the policy approach,” in which the dominant criterion is political coherence. From this perspective, other forms of coherence (input/process/output/outcome/impact or technical/scientific) are consequently complementary criteria. On a more general level, a discussion of a coherent framework for indicators should presumably take into account the multi-faceted relationships between the data generated and the expected policy insights which an analysis of the data would yield. The management of the needs of different users and the limits of the data collections should also be part of the discussion. There are several approaches to indicators and their frameworks. The OECD uses a matrix that organises the indicators to distinguish between the actors in education systems while grouping the indica-
Page 1 and 2: Institute for the Protection and Se
Page 3 and 4: 3 Abstract This paper proposes a ne
Page 5 and 6: 5 1. Hours in continuing vocational
Page 7 and 8: 7 1. Data gathered from administrat
Page 9 and 10: 9 Countries on a legal basis. While
Page 11: 11 Literacy and Life Skills Survey
Page 15: In defining the components of such
Page 18 and 19: Measuring the outputs and outcomes
Page 20: The mission of the Joint Research C

measuring the outputs and outcomes of vocational training

Create successful ePaper yourself

Delete template?

Save as template?