Projects on Competencies in the OECD Context - Analysis - DeSeCo

Definition and Selection of CompetenciesIALS was specifically designed to allow empirical tests of theoriesabout the relationship between literacy skills and educational attainmentand qualifications, and the relationship between literacy skillsand earnings – independent of educational attainment. Previously,educational attainment was the only measure available and had servedas a proxy for actual skills in research relating worker characteristics tolabor market outcomes. These analyses were key to the interest ofpolicymakers in the study.In addition, IALS provided an opportunity to further refine the theoriesdeveloped for NALS. The data were used to investigate the extentto which the constructs of prose, document, and quantitative literacyrepresent distinct dimensions in IALS countries, as proposed by thetheory guiding the work. The analyses found high correlations amongthe three scales, but concluded that there was “sufficient separationamong the three literacy scales to justify reporting these scales separately”(U.S. Department of Education 1998: 143).When the results of the first round of IALS were reported to participatingcountries, the extent to which the survey actually attained internationalcomparability was questioned by the French authorities. Theirconcerns included the appropriateness of the assessment instruments,the validity of the sampling procedures, and the reliability of the populationestimates (U.S. Department of Education 1998) 6 . In response, StatisticsCanada commissioned a review of the survey procedures by threestatisticians. Their review, which has been published in the IALS technicalreport, pointed out a number of problems mainly related to the surveydesign and procedures (U.S. Department of Education 1998).The review concluded that there is a serious need for more standardizedprocedures in future surveys, but that despite the methodologicalweaknesses, the survey results could be published. The authorsstated that all results should be interpreted with caution, and recommendedthat the reported results focus on analyses of the correlates ofliteracy and how these compare across countries rather than on overallliteracy levels. (Their position is that the former type of analysis is less6France had been an early participant in IALS and its representatives participated throughout thedevelopment of the survey.28 ESSI OECD SFSO