10.07.2015 Views

adult literacy and life skills survey adult literacy and life skills ... - ISEA

adult literacy and life skills survey adult literacy and life skills ... - ISEA

adult literacy and life skills survey adult literacy and life skills ... - ISEA

SHOW MORE
SHOW LESS

Create successful ePaper yourself

Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.

ADULT LITERACYAND LIFE SKILLSSURVEYProspectus


2004


WHAT IS ALL?The Adult Literacy <strong>and</strong> Life Skills Survey (ALL) is a large-scale,comparative <strong>survey</strong> that seeks to profile the <strong>skills</strong> of <strong>adult</strong>s in multiplecountries through direct assessment in households. The <strong>skills</strong> to bemeasured directly are:! Prose ose Literacy: the knowledge <strong>and</strong> <strong>skills</strong> needed to underst<strong>and</strong> <strong>and</strong> useinformation from texts such as editorials, news stories, poems, <strong>and</strong> fiction;! Document Literacy: the knowledge <strong>and</strong> <strong>skills</strong> required to locate <strong>and</strong> useinformation contained in various formats such as tables, forms, graphs, <strong>and</strong>diagrams;! Numeracy: the knowledge <strong>and</strong> <strong>skills</strong> required to effectively manage themathematical dem<strong>and</strong>s of diverse situations;! Problem Solving: the ability to solve problems by clarifying the nature of theproblem <strong>and</strong> developing <strong>and</strong> applying appropriate solution strategies;! Health Literacy: the knowledge <strong>and</strong> <strong>skills</strong> needed to underst<strong>and</strong> <strong>and</strong> useinformation from various health documents pertaining to promotion,protection, prevention, care & maintenance <strong>and</strong> navigation;Measurement in these domains is based upon explicit theory that predictsthe relative difficulty of tasks, a fact that improves the efficiency of assessment <strong>and</strong>that brings meaning to the proficiency scales <strong>and</strong> levels that are the key output ofthe study.The ALL study also seeks to profile the <strong>skills</strong> of <strong>adult</strong>s in one additionaldomain using indirect behavioural methods. The skill to be measured indirectly is:! Information Communication Technology:the knowledge <strong>and</strong> <strong>skills</strong> to useinformation technology to accomplish job-related tasks; <strong>and</strong>! ALL also profiles participation in episodes of <strong>adult</strong> education <strong>and</strong> training i.e.formal learning after the initial cycle of formal education has been completed.In addition to direct measures of skill the ALL study provides information onthe determinants of the observed skill profiles, the social economic, <strong>and</strong> healthoutcomes that are associated with various levels of proficiency <strong>and</strong> a broad range ofdemographic variables.WHY IS ALL IMPORTANT?There is strong evidence that developed countries have entered an erawhere full participation in modern society <strong>and</strong> in the labour market willbe linked to the capacity to accumulate knowledge <strong>and</strong> to develop <strong>and</strong>maintain a broad range of <strong>skills</strong>. Many <strong>adult</strong>s are already in a position to excel in theknowledge society, but others will find it difficult to adapt to the new conditions <strong>and</strong>will face reduced employment opportunities, increased risk of unemployment,redundancy, social exclusion, <strong>and</strong> a reduction in their overall quality of <strong>life</strong>. Toremain competitive, to counter exclusion, <strong>and</strong> to prevent growing inequalities inwages <strong>and</strong> learning opportunities, it is likely that governments, in co-ordinationwith the private sector <strong>and</strong> the social partners, will need to develop programs <strong>and</strong>policies to enhance the quantity, quality <strong>and</strong> accessibility of <strong>adult</strong> learning.3


Reliable information on the available supply of skill, the likely economicdem<strong>and</strong> for skill, current levels of <strong>adult</strong> education <strong>and</strong> training <strong>and</strong> the socialdistribution of both skill <strong>and</strong> participation in learning is an important input to thedesign <strong>and</strong> evaluation of such policies <strong>and</strong> programs.The Adult Literacy <strong>and</strong> Life Skills Survey (ALL) is being developed inresponse to this need.WHAT ARE THE USES OF ALL DATA AND RESULTS?The ALL study is designed to inform public policy in the economic, social,educational <strong>and</strong> health domains.At the broadest level the study seeks to profile the level <strong>and</strong> distribution ofproficiency in multiple skill domains at the national level <strong>and</strong> for particularpopulation subgroups, including those with low skill levels. The comparativedimension of the study allows for the comparison of these skill profiles acrosscountries.The study is also designed to shed light on the determinants of the observedskill profiles, information that is crucial to judging how rapidly government policymight improve the situation. Data from the International Adult Literacy Survey(IALS) suggests that processes of skill loss play an important role in many countries,reducing the available supply of skill.ALL also provides a wealth of information on the social, health <strong>and</strong>economic outcomes that are associated with skill at different levels. Data from theIALS study confirmed the important role that <strong>literacy</strong> <strong>and</strong> numeracy play in theallocation of economic opportunity in many labour markets <strong>and</strong> hints that these<strong>skills</strong> will become increasingly important as economies become more informationintense.Finally, provided sample sizes are large enough, the study offers an efficientmeans of judging the quality of recent educational output.HOW DOES ALL RELATE TO OTHER ADULTLITERACY ASSESSMENTS AND SURVEYS?4ALL is built on the assessment model <strong>and</strong> success of the IALS study thatmeasured prose, document, <strong>and</strong> quantitative <strong>literacy</strong> <strong>skills</strong> in <strong>adult</strong>populations in 24 countries. ALL goes far beyond <strong>adult</strong> <strong>literacy</strong> <strong>skills</strong>,however, to measure a broader set of basic <strong>life</strong> <strong>skills</strong> that appear to be directlylinked to the social <strong>and</strong> economic success of <strong>adult</strong> populations. For those countriesthat fielded IALS, ALL will provide a first glimpse of how skill profiles have evolvedover time, information crucial to judging the potential impact of public policy.Another large-scale, international <strong>survey</strong> closely related to ALL is theOECD Program for International Student Assessment (PISA), which measureslearning outcomes for 15-year-old students in all OECD member countries. PISAprovides a basis for assessing <strong>and</strong> monitoring the effectiveness of educationalsystems at national levels <strong>and</strong> for making international comparisons. While PISA isconcerned with measuring the <strong>skills</strong> of a single cohort of 15 year olds in reading,mathematics <strong>and</strong> science near the point of secondary completion, ALL seeks toassess the <strong>skills</strong> of <strong>adult</strong>s aged 16-65 irrespective of their student status including alarge number of recent graduates aged 16-24.


ALL is also closely related to the work on the Definition <strong>and</strong> Selection ofCompetencies (DeSeCo), which is led by Switzerl<strong>and</strong>, with the support of the UnitedStates <strong>and</strong> Canada, under the OECD Indicators of Education Systems (INES)project. The DeSeCo project seeks to establish a clear <strong>and</strong> definitive theoreticalconceptual framework for the assessment of <strong>skills</strong>. This framework is meant toinform priorities for measurement <strong>and</strong> to aid in the interpretation of what can becurrently measured. Through its links to <strong>and</strong> close parallels with these <strong>and</strong> otherinternational <strong>survey</strong> projects, ALL will ensure broad consistency of approach <strong>and</strong>conceptualisation of skill domains <strong>and</strong> capitalise on the strengths of similar <strong>survey</strong><strong>and</strong> instrument development efforts.WHAT ARE THE ASSESSMENT DESIGN AND TIMELINES?ALL constructed well-defined scales of performance that describe thecapabilities of <strong>adult</strong>s at different levels for each of the five skill domains.The <strong>survey</strong> uses a Balanced Incomplete Block (BIB) assessment design, inwhich a large set of tasks for each scale is broken into smaller sets, or blocks.Individual respondents do not have to take the entire set of assessment tasks.Instead, to reduce respondent burden <strong>and</strong> increase content coverage, r<strong>and</strong>omlyselected, representative samples of <strong>adult</strong>s is being administered systematicallyselected subsets of assessment tasks in specific skill domains. This design ensuresthat the results can be reported on the full scales for each of the skill domains bothwithin <strong>and</strong> between countries (<strong>and</strong> for key sub-populations when the nationalsample of respondents is large enough).ALL is implemented in three phases:! Planning PhaseEach country choosing to participate in the ALL study are obliged to draft anational planning report setting out their national study objectives <strong>and</strong> design,estimated costs <strong>and</strong> membership of the national study team. These plansare then vetted by the international study team to ensure adherence toprescribed methods <strong>and</strong> that national teams have the technical, operational <strong>and</strong>financial means to implement the study successfully. The ALL study offers twooptions for participation.! the st<strong>and</strong>ard ALL design: This design includes assessements in the field ofProse, Document <strong>and</strong> Numeracy.! an enhanced ALL design: This design includes assessments in the field ofProse, Document Numeracy <strong>and</strong> Problem Solving.! Pilot Test est PhaseEach country choosing to participate is obliged to field a pilot <strong>survey</strong>. The ALLpilot has two objectives – to ensure that the assessment will yield valid, reliable<strong>and</strong> comparable proficiency estimates; <strong>and</strong>, to provide national study teams withfirst h<strong>and</strong> experience in implementing all stages of a household-basedassessment of <strong>adult</strong> proficiency: i.e. item adaptation <strong>and</strong> printing, datacollection, scoring <strong>and</strong> data processing.Countries choosing to field the st<strong>and</strong>ard ALL design are required to field a pilot<strong>survey</strong> designed to yield 300 completed cases.Participants choosing to field the enhanced ALL design will field theproblem solving domain with an extended number of booklets. In this casethe pilot <strong>survey</strong> must yield 400 completed cases.5


! Main Survey AdministrationCountries choosing to implement the st<strong>and</strong>ard ALL design are required tofield a sample large enough to yield 3,420 completed cases. For countrieschoosing to implement the enhanced design, the required net sample yieldis 5,400 cases.WHO IS RESPONSIBLE FOR ALL?The ALL study is a joint undertaking of Statistics Canada, the NationalCentre for Education Statistics (NCES) of the U.S. Department ofEducation, <strong>and</strong> the Organisation for Economic Co-operation <strong>and</strong>Development (OECD). Statistics Canada is the key executing agency responsible forproject management, quality assurance, <strong>and</strong> analysis <strong>and</strong> reporting at theinternational level. Each country participating in the study assumes responsibilityfor implementing the study. The Educational Testing Service has been retained toprovide design <strong>and</strong> psychometric analysis services. In addition, a partnership hasbeen forged with UNESCO’s Regional office for Latin America <strong>and</strong> the Caribbean(OREALC) to assist with implementation <strong>and</strong> analysis in the Latin American region.The overall policy for the project is set by a Project Advisory Groupconsisting of representatives of countries participating in the Survey, the OECDSecretariat, <strong>and</strong> the members of the International Management Team.Participating countries are involved in the international aspects of theproject through the involvement of their National Project Managers in internationalmeetings to get training on detailed project procedures.WHAT ARE THE REQUIREMENTS FORPARTICIPATING COUNTRIES?ALL is to be conducted according to the st<strong>and</strong>ards <strong>and</strong> guidelines establishedby the ALL International Management Group. It consists of theadministration of a psychometric test of approximately 60 minutes duration,<strong>and</strong> an interviewer-administered questionnaire requiring about 30-40 minutes tocomplete. Each participating country must recruit:! a representative to serve on the Project Advisory Group;! a National Steering Committee made up of experts <strong>and</strong> representatives of keystakeholders;! a National Project Manager experienced in fielding household <strong>survey</strong>s;! a senior <strong>survey</strong> methodologist, responsible for the participating country’s sampledesign <strong>and</strong> implementation, weighting <strong>and</strong> estimation, regular data analysis,<strong>and</strong> quality assurance at all steps of the <strong>survey</strong>;! a data collection manager, responsible for the hiring, training, monitoring <strong>and</strong>control of data collection staff such as interviewers <strong>and</strong> senior interviewers, theimplementation of data collection procedures to minimise both item nonresponse<strong>and</strong> complete non-response;! a data processing manager, responsible for data capture, coding, editing<strong>and</strong> imputation, <strong>and</strong> tabulation of <strong>survey</strong> results;6


! a <strong>survey</strong> team with a proven record of excellence in the administration ofhousehold <strong>survey</strong>s <strong>and</strong> familiar with processing household <strong>survey</strong> data, coding,<strong>and</strong> editing! one or more linguists to adapt the test items in a way that respects both thetheory <strong>and</strong> culture specificity, <strong>and</strong>,! a team of researchers to analyse data from the <strong>survey</strong> at the national level.WHAT ARE THE BUDGET AND COSTS IMPLICATIONS?Direct measurement of a comprehensive set of <strong>life</strong> <strong>skills</strong> on an internationallevel requires considerable financial resources for the design <strong>and</strong> validationof the assessment material, as well as for <strong>survey</strong> administration, datacollection, <strong>and</strong> data analysis. Substantial funding for the initial design <strong>and</strong>development of the assessment materials has already been provided by Canadathrough Statistics Canada <strong>and</strong> by the United States through the National Centre forEducation Statistics. Other countries will not be asked to contribute to those costsalready incurred. To participate in ALL, therefore, each country will be responsiblefor the expenses of:! design <strong>and</strong> implementation of the national <strong>survey</strong>! contribution, over a three year period, of $150,000 USD to help defray theinternational <strong>survey</strong> overhead costs associated with implementation, includinggeneral project management, quality assurance, data scaling, analysis,<strong>and</strong> reporting.WHERE TO GET ADDITIONAL INFORMATION?For further information about ALL, please visit the ALL Web site athttp://www.ets.org/all/, or visit our interactive data search tool athttp://www27.statcan.ca/ialdata/search.asp?lang=1033 or contact:T. . Scott MurrayrayPatrick WererquinBarry McGaw(613) 951-9035 +33 145 24 97 58 +33 145 24 92 10scotmur@statcan.ca Patrick.WERQUIN@oecd.org barry.mcgaw@oecd.orgStatistics Canada Principal Administrator Deputy Director (Education)Room 2001, Main Building OECD OECDTunney’s Pasture 2, rue André Pascal 2, rue André PascalOttawa, Ontario K1A OT6 75775 Paris Cedex 16 75775 Paris Cedex 16Canada France France7


ALLADULT LITERACY AND LIFE SKILLS SURVEY2004

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!