The Effectiveness of Problem Based Learning 1 - Teaching and ...

More documents

Recommendations

Info

The researchBackgroundProblem Based Learning is a majordevelopment in educational practicethat has a large impact across subjectsand disciplines worldwide. It is notalways clear what exactly has beendone in the name of Problem BasedLearning. There is also a growingnumber of references in the literature toadapted or “Hybrid” Problem BasedLearning courses and to coursesdescribed as 'Enquiry' or 'Inquiry'Based learning. Problem BasedLearning has arguably been one of themost scrutinised innovations inprofessional education. However,despite the volume of literature onProblem Based Learning it is not at allclear that we have safe knowledgeabout the effects of Problem BasedLearning in different learning contextsand in different modes of operation.Whilst the principle of research reviewsis well established in education, theappropriate process and purpose forthem is not. Systematic reviews can bea valid and reliable means of avoidingthe bias that comes from the fact thatsingle studies are specific to a time,sample and contexts and may be ofquestionable methodological quality.They attempt to discover theconsistencies and account for thevariability in similar appearing studies.There is a consensus emerging aboutthe need for systematic reviewscovering selected topics in education.Such reviews will identify the existingevidence, provide at least someanswers to the review questions andprovide directions for future primaryresearch.Objectives of thepilot review• To establish the level and quality ofexisting ‘safe’ knowledge about theeffectiveness of Problem BasedLearning based on previouslypublished reviews• To confirm the need for a fullsystematic review of theeffectiveness of Problem BasedLearning• To establish the value of the methodof systematic review used• To identify and clarify any problemswith the review protocol, processand instrumentsReview questionsThe initial review questions were…When compared to other approachesdoes Problem Based Learning result inincreased participant performance at:• adapting to and participating inchange;• dealing with problems and makingreasoned decisions in unfamiliarsituations;• reasoning critically and creatively;• adopting a more universal or holisticapproach;• practising empathy, appreciating theother person’s point of view;• collaborating productively in groupsor teams;• identifying own strengths andweaknesses and undertakingappropriate remediation (self-directedlearning)?MethodsReview DesignThe review followed the approach usedby The Cochrane Collaboration, aninternational effort to identify evidenceof effectiveness in the field of healthcare. The design of the review protocol,data extraction tools and overall reviewprocess is based on the guidance forreviewers produced by The CochraneEffective Practice and Organisation ofCare Review Group and the NHSCentre for Reviews and Dissemination.This approach has been developedfrom the emerging ‘science’ ofsystematic reviewing in order minimisebias and errors. The review protocolwas registered with the CampbellCollaboration. Copies of the reviewprotocol and instrumentation areincluded in the full project report.Inclusion criteria• The review included participants inpost-school education programmes.• Study designs included wereRandomised Controlled Trials (RCT),Controlled Clinical Trials (CCT) ,Interrupted Time Series (ITS), andControlled Before & After studies(CBA). Qualitative data collectedwithin such studies e.g. researchers’observations of events, wereincorporated in reporting. Studiesthat use solely qualitative approacheswere not included in the review.• The minimum methodologicalinclusion criteria across all studydesigns were the objectivemeasurement of studentperformance/behaviour or otheroutcome(s) and relevant andinterpretable data presented orobtainable.Minimum inclusion criteria forinterventions (i.e. what is ProblemBased Learning) were set out in thereview protocol. However, reporting wasinsufficient to establish whether thesecriteria were met in virtually all theprimary studies considered.Review ProcessThe review coordinator examined thereviews and identified the relevantcitations. For each citation either anabstract or full text copy was obtained.Two members of the review team thenscreened the citations to eliminatethose that obviously did not meet theminimum inclusion criteria. Theincluded papers were then distributedamongst the reviewers for qualityappraisal and data extraction. Eachpaper was reviewed independently bytwo reviewers. Where there weredifferences of opinion betweenreviewers the review coordinator alsoreviewed the paper. At all stages, theprocess used and outcomes areexplicit. A full list of included andexcluded studies is provided in the fullreport. The completed qualityassessment and data extraction toolswere returned to the review co-ordinatorwho lead the process of producing areport of the review synthesising andanalysing the results where appropriate.Results91 citations were identified from the fivereviews. Of these 15 were judged tomeet the inclusion criteria. Of the 15only 12 reported extractable data. Thediscussion below refers only to these 12studies. The studies all reported onProblem Based Learning used in highereducation programmes for healthwww.tlrp.orgTeaching and Learning Research Programme
professional education at both pre andpost registration levels. The majority ofstudents were in medicine and themajority of these studies reported onpre-registration medication education.The majority of reported effects were forwhat appeared to be tests of‘knowledge’ using multiple choiceformats. There were also a smallnumber of effects reported on 'impacton practice' and on 'approaches tolearning'. Very little information wasgiven in the papers from which datawas extracted about the design,preparation or delivery processes ofeither the Problem Based Learningintervention or the control to whichProblem Based Learning was beingcompared. Four of the studies used arandomised experimental design, two aquasi-randomised experimental designand the remainder were controlledbefore and after studies. Only onestudy reported standard deviated effectsizes. Effect sizes were calculated forall the other extracted effects with theexception of three studies whereinsufficient data were provided in thestudy report.The studies that reported 'effects onpractice' all used different outcomesand measurement instruments. In onlyone of the studies that reported 'effectson practice' was sufficient data providedto calculate effect sizes. This makes itdifficult to synthesise the study results.One study reported 'attitudes topractice' and found effect sizes thatfavoured Problem Based Learning.Another measured 'nursing processskills' and of the seven effects reportedfive favoured the control group. Thethird study reported 'consultation skills'and on all the effects reported theresults favoured the control group.However, in this particular study, whichused a quasi – experimental design, thenature of the Control group interventionand the outcome measures used wouldappear to have put the control group ata distinct advantage.Two studies reported effects on'approaches to learning'. The twostudies used different instruments andreported five effects. In both studies,the results favoured Problem BasedLearning on all the scales. The ProblemBased Learning groups had fewer of theundesirable and more of the desirable'approaches to learning' after theintervention. However, the overallMajor implicationsThe review found comparatively fewstudies cited in the five sample reviewpapers whose design and methodologywere good enough to eliminate bias andprovide precise estimates of effect.Studies that achieved this wereconcentrated largely in medicaleducation in North America and Europeand focussed mainly on measuring'accumulation of knowledge' usingmultiple choice assessments.However, an outcome of 'accumulationof knowledge' and in particular itsassessment through multiple choiceinstruments is not regarded by manyadvocates of Problem Based Learningas congruent with the approach itself oras a valid indicator of success. Thesecond issue is how generalisable theseresults are to other disciplines and toother cultural contexts.Figure 1: Effect size for reported study outcomes focussing on 'knowledge'picture was of deterioration in theapproaches to learning of both theProblem Based Learning and controlgroups, which Problem Based Learningappears to have mitigated.Figure 1 illustrates the effect sizes forreported outcomes that could usefullybe grouped under a heading‘knowledge’. Effect sizes ranged from d=-4.9 to 2.0. There were sufficienteffects reported to justify an attemptedmeta analysis. The meta analysis of theoutcome ‘knowledge’ included 14effects reported in 8 different studies.Concerning the review questions, theresults of the meta analysis are largelyinconclusive. The mean effect size wasd= -0.3 but the 95% confidence intervaldid not exclude an effect size of d = 0.Sensitivity analysis suggested that studydesign, randomisation, level ofeducation and assessment format areall potential moderating variables but theresults were not conclusive. Importantlythe 95% confidence intervals do notexclude potentially ‘large effect sizes ofd = + 1.0 or – 1.0. An effect size of d= 1.0 would mean that 84% of studentsin the control group were below thelevel of the average person in theProblem Based Learning group. Aneffect of this size would appear to haveimportant practical significance.Only one study reported effects on'satisfaction with the learningenvironment' that met the reviewinclusion criteria. The study in anundergraduate medical educationprogramme required students to ratetheir experience on a series of scales.On all except two of the nine effectsreported, the effect size favoured theProblem Based Learning group. Thelargest effect size in favour of ProblemBased Learning was the students ratingof innovation (d=1.6). The largest effectsize in favour of the Control group wasstudents’ rating of clarity (d= -1.0).Teaching and Learning Research Programmewww.tlrp.org
Page 1: TeachingandLearningRESEARCHBRIEFING

The Effectiveness of Problem Based Learning 1 - Teaching and ...

Create successful ePaper yourself

Delete template?

Save as template?