Complete thesis - Murdoch University

Educating Requirements Engineers in Australia:effective learning for professional practiceJocelyn ArmaregoBA (UWA); DipLib (WAIT); DipComp (WAIT); MAppSci(InfoSci) (CSU)PhD Information TechnologySchool of Computer and Information ScienceUniversity of South AustraliaFebruary 22, 2007

Contents1 The problem with RE education 11.1 The nature of RE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 The nature of the study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101.2.1 Frameworks for the study . . . . . . . . . . . . . . . . . . . . . . . . . 111.2.2 Research approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.3 Summary structure of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . . 162 What is taught in RE 192.1 Practitioner perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.1.1 Studies addressing the global perspective . . . . . . . . . . . . . . . . . 222.1.2 CS/Engineering practitioner perspectives . . . . . . . . . . . . . . . . . 252.1.3 The Australian perspective . . . . . . . . . . . . . . . . . . . . . . . . . 292.1.4 Practitioner views of RE . . . . . . . . . . . . . . . . . . . . . . . . . . 322.1.5 Summarising the RE practitioner perspective . . . . . . . . . . . . . . 402.2 How RE is viewed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432.2.1 A positivist perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . 452.2.2 The non-positivist perspective . . . . . . . . . . . . . . . . . . . . . . . 472.2.3 Positivism and non-positivism in RE . . . . . . . . . . . . . . . . . . . 482.2.4 The non-alignment of theory to practice . . . . . . . . . . . . . . . . . 552.2.5 Addressing wicked RE education . . . . . . . . . . . . . . . . . . . . . 572.3 What mastery of which knowledge? . . . . . . . . . . . . . . . . . . . . . . . . 612.4 Bodies of Knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 682.5 Model curricula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 772.5.1 RE in model curricula and texts . . . . . . . . . . . . . . . . . . . . . . 78i

2.5.2 Towards a REBoK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 962.6 Factors against success . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 973 A framework for learning RE 1023.1 The process of knowing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1043.1.1 The nature of knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . 1053.1.2 The nature of model making . . . . . . . . . . . . . . . . . . . . . . . . 1073.1.3 The development of shared meaning . . . . . . . . . . . . . . . . . . . . 1093.1.4 The nature of problem solving . . . . . . . . . . . . . . . . . . . . . . . 1113.1.5 The development of expertise . . . . . . . . . . . . . . . . . . . . . . . 1163.1.6 Metacognition and reflection . . . . . . . . . . . . . . . . . . . . . . . . 1183.1.7 The place of creativity . . . . . . . . . . . . . . . . . . . . . . . . . . . 1213.2 Theories of learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1263.2.1 Learning as transfer . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1303.2.2 Acquiring membership of a discipline . . . . . . . . . . . . . . . . . . . 1323.3 Models of learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1333.3.1 Learning as a social or cultural activity . . . . . . . . . . . . . . . . . . 1353.3.2 Learning as authentic . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1393.3.3 Situated cognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1413.4 Formal learning of RE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1463.4.1 Traditional learning and RE . . . . . . . . . . . . . . . . . . . . . . . . 1473.4.2 Factors against success . . . . . . . . . . . . . . . . . . . . . . . . . . . 1513.5 Environments for learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1543.5.1 Instructional design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1543.5.2 Learning as student-centred . . . . . . . . . . . . . . . . . . . . . . . . 1603.5.3 PBL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1623.5.4 Experiential learning as reflective . . . . . . . . . . . . . . . . . . . . . 1663.6 A framework for RE education . . . . . . . . . . . . . . . . . . . . . . . . . . . 1693.6.1 RE as a cycle of teaching and learning . . . . . . . . . . . . . . . . . . 1694 The research design 176ii

4.1 Research in Education and IT . . . . . . . . . . . . . . . . . . . . . . . . . . . 1784.1.1 Design Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1854.1.2 Action Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1884.1.3 Educational research as Design or Action . . . . . . . . . . . . . . . . . 1944.1.4 Strengths and weaknesses of a research design . . . . . . . . . . . . . . 1964.2 Data acquisition and evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . 1994.2.1 Quantitative versus qualitative methods . . . . . . . . . . . . . . . . . 2004.2.2 Mixed methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2024.2.3 Strategies for data acquisition . . . . . . . . . . . . . . . . . . . . . . . 2034.2.4 Strategies for data evaluation . . . . . . . . . . . . . . . . . . . . . . . 2044.3 Reporting on the research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2074.4 Summary of approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2074.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2095 Developing an Action Research model for RE education 2125.1 Framework for the study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2135.1.1 Action Research for Education . . . . . . . . . . . . . . . . . . . . . . . 2145.1.2 Tools for educational change . . . . . . . . . . . . . . . . . . . . . . . . 2165.1.3 Reflective activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2185.1.4 The integrated framework . . . . . . . . . . . . . . . . . . . . . . . . . 2195.2 The Action Research cycles . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2195.2.1 Initial reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2205.2.2 Cycle 1: apprenticeship . . . . . . . . . . . . . . . . . . . . . . . . . . . 2215.2.3 Cycle 2: PBL for creativity . . . . . . . . . . . . . . . . . . . . . . . . 2235.2.4 Cycle 3: studio learning . . . . . . . . . . . . . . . . . . . . . . . . . . 2265.3 Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2285.3.1 Diagnostic devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2305.3.2 Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2395.3.3 Student feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2405.3.4 Teaching style . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240iii

5.4 The Murdoch context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2445.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2456 Apprenticing the RE student 2002 2476.1 Context for Cycle 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2496.1.1 Curriculum components . . . . . . . . . . . . . . . . . . . . . . . . . . 2496.1.2 Characteristics of teaching and learning . . . . . . . . . . . . . . . . . . 2496.2 Cycle 1 – Apprenticing in RE . . . . . . . . . . . . . . . . . . . . . . . . . . . 2526.2.1 The learning environment . . . . . . . . . . . . . . . . . . . . . . . . . 2526.2.2 What actually happened . . . . . . . . . . . . . . . . . . . . . . . . . . 2556.2.3 Interpreting what happened . . . . . . . . . . . . . . . . . . . . . . . . 2616.3 Reflection on findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2686.4 Conclusions drawn from Cycle 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 2707 Implementing a model for creative RE education 2003 2727.1 Context for Cycle 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2767.1.1 Curriculum components . . . . . . . . . . . . . . . . . . . . . . . . . . 2767.1.2 Characteristics of teaching and learning . . . . . . . . . . . . . . . . . . 2777.2 Cycle 2 – Creative RE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2817.2.1 Decoding the discipline . . . . . . . . . . . . . . . . . . . . . . . . . . . 2847.2.2 Developing the learning environment . . . . . . . . . . . . . . . . . . . 2917.2.3 What actually happened . . . . . . . . . . . . . . . . . . . . . . . . . . 3047.2.4 Interpreting what happened . . . . . . . . . . . . . . . . . . . . . . . . 3087.2.5 Reflection on findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3207.3 Conclusions drawn from Cycle 2 . . . . . . . . . . . . . . . . . . . . . . . . . . 3278 RE as Studio Learning 2005 3298.1 Context for Cycle 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3338.1.1 Curriculum components . . . . . . . . . . . . . . . . . . . . . . . . . . 3338.1.2 Staff development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3358.1.3 Orientation to Studio Learning – Design Week . . . . . . . . . . . . . . 3368.1.4 Characteristics of teaching and learning . . . . . . . . . . . . . . . . . . 341iv

8.2 Cycle 3 – Studio Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3448.2.1 Mapping the curriculum . . . . . . . . . . . . . . . . . . . . . . . . . . 3448.2.2 What was planned . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3488.2.3 What actually happened . . . . . . . . . . . . . . . . . . . . . . . . . . 3508.2.4 Interpreting what happened . . . . . . . . . . . . . . . . . . . . . . . . 3578.2.5 Reflection on findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3658.3 Beyond RE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3678.3.1 ENG302: students as advanced learners . . . . . . . . . . . . . . . . . . 3688.3.2 Approaches to studying . . . . . . . . . . . . . . . . . . . . . . . . . . 3778.3.3 Reflection on findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3859 Conclusions 3889.1 Summary of the research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3899.2 Modelling RE education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3919.2.1 Modelling RE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3919.2.2 Practitioner perspective on competency . . . . . . . . . . . . . . . . . . 3939.2.3 Developing an education-practitioner alignment . . . . . . . . . . . . . 3959.2.4 Implementing a model for RE education . . . . . . . . . . . . . . . . . 3959.3 Validity of the research approach . . . . . . . . . . . . . . . . . . . . . . . . . 3999.3.1 Criteria for quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4019.3.2 Limitations of the research . . . . . . . . . . . . . . . . . . . . . . . . . 4059.4 Implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4079.4.1 Contributions to knowledge . . . . . . . . . . . . . . . . . . . . . . . . 4079.4.2 Implications for the discipline of RE and RE education . . . . . . . . . 4089.4.3 Implications for Engineering education . . . . . . . . . . . . . . . . . . 4119.5 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4139.6 In conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416Bibliography 418Appendices 454A The Murdoch context 454v

A.1 Curriculum components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455A.2 Characteristics of teaching and learning . . . . . . . . . . . . . . . . . . . . . . 457A.2.1 Learning styles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457A.3 The learning environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460A.4 Student cohort for ENG260 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464B Instruments 466B.1 Education Relevance Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466B.2 School of Engineering Year Survey . . . . . . . . . . . . . . . . . . . . . . . . 470B.3 Student Surveys of Units . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472B.4 Attachment 1: Educational Relevance Survey instrument . . . . . . . . . . . . 474C Student Reflective Journals 496vi

List of Figures1.1 Influences on the learning environment for RE . . . . . . . . . . . . . . . . . . 111.2 Thesis chapters in support of the main proposition . . . . . . . . . . . . . . . 162.1 Influences on the Learning Environment for RE (2) . . . . . . . . . . . . . . . 192.2 Professional Capability Framework (Scott and Wilson, 2002) . . . . . . . . . . . . 392.3 Opportunism in requirements discovery (Guindon, 1990) . . . . . . . . . . . . . . 512.4 The catastrophe-cycle RE process(Nguyen and Swatman, 2000a) . . . . . . . . . . 522.5 The revised catastrophe-cycle process (as described in Raisey et al (2006)) . . . . . 532.6 Spiral model of the RE process (Sawyer and Kotonya, 2000) . . . . . . . . . . . . . 692.7 INCOSE RBoK static model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 762.8 The computing space occupied by CS, IS and SE (Shackelford, 2005) . . . . . . . 802.9 The computing space occupied by RE (after Shackelford (2005)) . . . . . . . . . . 973.1 Influences on the learning environment for RE (3) . . . . . . . . . . . . . . . . 1023.2 Model-making in learning (Norman, 1983) . . . . . . . . . . . . . . . . . . . . . . 1283.3 Topography of approaches to active learning (Horvath et al, 2004) . . . . . . . . . 1343.4 Learning as a discourse (Laurillard, 1993) . . . . . . . . . . . . . . . . . . . . . . 1373.5 Derived education model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1524.1 Lewin’s spiral of Action Research (Kemmis and McTaggert, 1988) . . . . . . . . . . 1925.1 Action Research in an educational context (based on Borg et al (1993) and Jacksonand Borden (n.d.)) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2145.2 Model for cognitive change (adapted from Rogers (2002)) . . . . . . . . . . . . . . 2175.3 A framework for double-loop learning (adapted from Hatten (1997)) . . . . . . . . 2185.4 A conceptual framework for Action Research in RE education . . . . . . . . . 219vii

5.5 Education for RE – Action Research Cycle 1 . . . . . . . . . . . . . . . . . . . 2225.6 Education for RE – Action Research Cycle 2 . . . . . . . . . . . . . . . . . . . 2245.7 Education for RE – Action Research Cycle 3 . . . . . . . . . . . . . . . . . . . 2275.8 Onion model of learning styles (Curry, 1983) . . . . . . . . . . . . . . . . . . . . 2315.9 Pedagogical dimensions of a learning environment (partial) (Reeves, 1997b) . . . 2425.10 Education for RE – Action Research Cycles . . . . . . . . . . . . . . . . . . . 2456.1 Education for RE – Action Research Cycle 1 . . . . . . . . . . . . . . . . . . . 2476.2 Kolb Learning Style Inventory 1st year Engineering students 2001 . . . . . . . 2516.3 Student mind map for topic examining teams . . . . . . . . . . . . . . . . . . 2536.4 Student mind map for topic examining UML . . . . . . . . . . . . . . . . . . . 2536.5 Raw exam marks for ENG260 1999-2002 . . . . . . . . . . . . . . . . . . . . . 2656.6 Final mark for ENG260 1999-2002 . . . . . . . . . . . . . . . . . . . . . . . . . 2666.7 Pedagogical dimensions of RE as at 2002 . . . . . . . . . . . . . . . . . . . . . . 2677.1 Education for RE – Action Research Cycle 2 . . . . . . . . . . . . . . . . . . . 2737.2 BE(SE) Murdoch University: curriculum components post2002 . . . . . . . . . 2767.3 Requirements Engineering class cohort 1999 - 2003 (Armarego, 2004b) . . . . . . 2797.4 Results of the TSI applied to the teacher . . . . . . . . . . . . . . . . . . . . . 2817.5 Excerpt of RE content categorised for mapping . . . . . . . . . . . . . . . . . 2887.6 ENG260 learning objectives mapped to topics, graduate attributes and problems2907.7 A process model for CreativePBL . . . . . . . . . . . . . . . . . . . . . . . . . 2927.8 Learning environment interaction (adapted from an idea by McLaughlan and Kirkpatrick(2004)) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2977.9 Setting the scene MurSoft . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2987.10 Setting the scene TerColl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2987.11 Setting the scene PBL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3007.12 Part of the trigger sequence for the games environment . . . . . . . . . . . . . 3017.13 Self assessment support (adapted from (Zimitat and Alexander, 1999) . . . . . . . . . 3027.14 Final mark for ENG260 2002-2003 . . . . . . . . . . . . . . . . . . . . . . . . . 3137.15 Raw exam marks for ENG260 2002-2003 . . . . . . . . . . . . . . . . . . . . . 314viii

7.16 Good/bad things about a unit structured this way . . . . . . . . . . . . . . . . 3157.17 Things to add/change/delete in a unit structured this way . . . . . . . . . . . 3167.18 Learning in a unit structured this way . . . . . . . . . . . . . . . . . . . . . . 3177.19 Results of the ASI applied to the 2003 cohort of ENG260 . . . . . . . . . . . . 3198.1 Education for RE – Action Research Cycle 3 . . . . . . . . . . . . . . . . . . . 3318.2 ASI Results for students about to undertake the Design Week . . . . . . . . . 3368.3 ASI scores for 2003 ENG260 students in 2004 . . . . . . . . . . . . . . . . . . 3378.4 Positive comments made about the Design Week . . . . . . . . . . . . . . . . . 3408.5 Negative comments made about the Design Week . . . . . . . . . . . . . . . . 3408.6 Requirements Engineering class cohort 2002 - 2005 . . . . . . . . . . . . . . . 3418.7 ATI Scales for conceptual change/student focus and information processing/teacherfocus with their respective strategies (Prosser and Trigwell, 1999) . . . . . . . . . . 3428.8 Alignment between outcomes and assessment (adapted from Learning and TeachingSupport Network (LTSN) (2002)) . . . . . . . . . . . . . . . . . . . . . . . . . . . 3458.9 Mapping of ENG260 topics to Engineers Australia graduate outcomes . . . . . 3468.10 Memo to students to trigger investigation of estimation techniques . . . . . . . 3488.11 Comparative exam marks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3628.12 Interaction schedule ENG302 session 1 . . . . . . . . . . . . . . . . . . . . . . 3688.13 Interaction breakdown ENG302 session 1 . . . . . . . . . . . . . . . . . . . . . 3698.14 Interaction schedule ENG302 session 2 . . . . . . . . . . . . . . . . . . . . . . 3698.15 Interaction breakdown ENG302 session 2 . . . . . . . . . . . . . . . . . . . . . 3698.16 Interaction schedule ENG302 session 3 . . . . . . . . . . . . . . . . . . . . . . 3708.17 Interaction breakdown ENG302 session 3 . . . . . . . . . . . . . . . . . . . . . 3708.18 Cumulative student hours for ENG302 . . . . . . . . . . . . . . . . . . . . . . 3738.19 Cumulative activity log for week 3 . . . . . . . . . . . . . . . . . . . . . . . . . 3748.20 ASI Results for students in ENG302 in 2005 . . . . . . . . . . . . . . . . . . . 3788.21 RoLI results for Markus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3798.22 RoLI results for Alaina . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3808.23 RoLI results for Dermot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380ix

8.24 RoLI results for Vaughn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3818.25 ASI results for Markus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3838.26 ASI results for Alaina . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3838.27 ASI results for Dermot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3848.28 ASI results for Vaughn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3859.1 Education for RE – the Action Research cycles . . . . . . . . . . . . . . . . . . 3909.2 A conceptual framework for Action Research in RE education . . . . . . . . . 4009.3 A conceptual model of alignment for RE education . . . . . . . . . . . . . . . 409A.1 BE(SE) Murdoch University: curriculum components pre2002 (Armarego et al,2001) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455A.2 Kolb Learning Style Inventory 1999 - 2003 cumulative results 1st year Engineeringstudents plus the researcher (Armarego, 2004b) . . . . . . . . . . . . . . 458A.3 The SENavigator for the Software Factory . . . . . . . . . . . . . . . . . . . . 461A.4 The production line for ENG260 in the Software Factory . . . . . . . . . . . . 462A.5 The MCQ environment in the Software Factory . . . . . . . . . . . . . . . . . 463A.6 Student cohort for Eng260 1999-2002 . . . . . . . . . . . . . . . . . . . . . . . 465B.1 Participant responses to questions 1-4 applied to topic: RE process . . . . . . 469x

List of Tables2.1 Minor: Bloom’s levels across model curricula . . . . . . . . . . . . . . . . . . . 212.2 Lee: Comparison of IS formal education and industry requirements . . . . . . 232.3 Noll: core IS skills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.4 Turley rankings: competencies by participant category . . . . . . . . . . . . . 252.5 Lethbridge rankings: most and least learnt formally . . . . . . . . . . . . . . . 262.6 Lethbridge rankings: lag between formal learning and importance . . . . . . . 272.7 Lethbridge rankings: most important . . . . . . . . . . . . . . . . . . . . . . . 282.8 Lethbridge rankings: most important for managers . . . . . . . . . . . . . . . 282.9 Snoke: top IS generic competencies . . . . . . . . . . . . . . . . . . . . . . . . 302.10 RE topics learnt in formal education . . . . . . . . . . . . . . . . . . . . . . . 352.11 Other software topics learnt in formal education . . . . . . . . . . . . . . . . . 352.12 Scott & Yates: top Engineering capabilities . . . . . . . . . . . . . . . . . . . . 382.13 Scott & Wilson: top IS capabilities . . . . . . . . . . . . . . . . . . . . . . . . 402.14 Levels of competence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 662.15 BoK matches for RE-related topics . . . . . . . . . . . . . . . . . . . . . . . . 682.16 SEEK topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 722.17 CSBOK: RE related topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 752.18 Minor: curricula match to perceived industry needs . . . . . . . . . . . . . . . 782.19 RE component of CC-SE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 822.20 SE component of CC-CS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 842.21 Learning objectives of SE5 in CC-CS . . . . . . . . . . . . . . . . . . . . . . . 842.22 IS 2002.7 analysis and logical design . . . . . . . . . . . . . . . . . . . . . . . 862.23 Analytical and critical thinking . . . . . . . . . . . . . . . . . . . . . . . . . . 872.24 Industry perception of graduate attributes . . . . . . . . . . . . . . . . . . . . 91xi

2.25 RE specific competencies for systems engineers . . . . . . . . . . . . . . . . . . 953.1 Comparison of teacher- and student-centred learning environments (based onHirumi (2002)) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1614.1 Criteria for the validity of Action Research (based on Krefting (1991) and Guba andLincoln (1994)) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1984.2 Principles for the evaluation of Action Research (Anderson et al, 1994) . . . . . . 1984.3 Addressing possible weaknesses of non-positivist research (based on Krefting(1991) and Anderson et al (1994)) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1994.4 Reflection in the Scholarship of Teaching model (Kreber, 1999) . . . . . . . . . . 2065.1 Tooling up for the Action Research project: instruments applied cycles (basedon Kember and Kelly (1993)) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2295.2 ASI Scales for Reproduction and Meaning Orientation (Richardson, 1990) . . . . 2376.1 Phases of Cognitive Apprenticeship model as implemented in ENG260 . . . . . 2566.2 Instruments applied to Cycle 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 2626.3 Reflection in the Scholarship of Teaching model (Kreber, 1999) . . . . . . . . . . 2687.1 Learning style of undergraduate engineering students (percentages) . . . . . . 2777.2 RE 2003 students compared with the Murdoch profile (percentages) (based onKolb Learning Style Inventory) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2787.3 TSI categories (adapted from Dunn and Dunn (1993) by University of Toronto) . . . . . 2807.4 Positive influences for enhancing creative potential (Amabile, 1996) . . . . . . . . 2927.5 Creativity activities (Edmonds and Candy, 2002) . . . . . . . . . . . . . . . . . . . 2937.6 Instruments applied to Cycle 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 3087.7 Issues in flexibility and creativity . . . . . . . . . . . . . . . . . . . . . . . . . 3127.8 Reflection in the Scholarship of Teaching model (Kreber, 1999) . . . . . . . . . . 3218.1 ATI Scales for Conceptual Change/Student Focus and Information Processing/TeacherFocus with their respective strategies (Prosser and Trigwell, 1999) . . 3438.2 Engineers Australia graduate attributes . . . . . . . . . . . . . . . . . . . . . . 3478.3 Excerpts from group minutes 2005 . . . . . . . . . . . . . . . . . . . . . . . . . 351xii

8.4 Key for activity logs ENG301 . . . . . . . . . . . . . . . . . . . . . . . . . . . 3528.5 Excerpts from activity logs ENG301 . . . . . . . . . . . . . . . . . . . . . . . . 3538.6 Instruments applied to Cycle 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . 3578.7 Summary of interaction schedule data . . . . . . . . . . . . . . . . . . . . . . . 3718.8 Tasks and keys applied in activity log ENG302 . . . . . . . . . . . . . . . . . . 3728.9 Total student hours for ENG302 . . . . . . . . . . . . . . . . . . . . . . . . . . 3728.10 Summary of RoLI subScales (adapted from Lindblom-Ylänne (2004)) . . . . . . . . 3778.11 Summary of ASI subScales expanded . . . . . . . . . . . . . . . . . . . . . . . 3828.12 Mapping of subScales . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3829.1 Summary of issues addressed by the Action Research cycles of this study . . . 3919.2 Model for professional action (Savin-Baden, 2000) . . . . . . . . . . . . . . . . . . 3969.3 Model for interdisciplinary understanding (Savin-Baden, 2000) . . . . . . . . . . . 3989.4 Principles for the evaluation of Action Research (based on Anderson et al (1994)and Krefting (1991)) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402A.1 Kolb Learning Style Inventory 1999 - 2003 cumulative results (percentages)(extracted from figures presented in Armarego (2004b)) . . . . . . . . . . . . . . . . . 458A.2 Soloman & Felder Index of Learning Styles 1999 - 2003 cumulative results(percentages) (extracted from figures presented in Armarego (2004b) . . . . . . . . . . 459A.3 Reported learning style preferences (percentages) (Felder and Brent, 2005) . . . . 460xiii

.AbstractIt is acknowledged within the systems development literature, and by practitioners, thatRequirements Engineering/Analysis is fundamental to the delivery of high quality software.It is also widely agreed that many of the problems identified in software-intensive systemscan be traced back to this component of the project. Practitioner studies suggest that someof these problems can be traced to formal education – skills and knowledge needed in dailywork are not being well developed through their formal IT education.The implications for graduates entering the discipline of Requirements Engineering include:• their conceptualisation does not match the reality of the profession: the educationthey have received does not prepare them for the real-world – the discipline is seen asneeding conceptual knowledge in several overlapping domains at a competency levelthat approaches expertise, while the approach advocated in much of the literaturehampers practice• the appropriate soft skills valued by practitioners are not well developed through formaleducation. These include affective and cognitive skills as well as understanding of the‘context’ in which the task being addressed exists• generic attributes, such as life-long learning, which addresses the adaptability and flexibilityrequired by practitioners, are also not well developed through formal education.The purpose of this research is to address the issue of education for Requirements Engineers.This thesis argues that a framework can be developed that more closely models the experiencesof practitioners, and addresses their expectations of novice Requirements Engineers.By examining the gap between the competency expectations of practitioners and traditionalformal education for the discipline, and then matching these gaps with strategies drawn fromthe literature of learning theory and models that purport to focus on these, this study showsthat non-traditional approaches provide leverage for a graduate entering the profession ofRequirements Engineering.xiv

.The research acknowledges the social nature not only of education, but also the domain intowhich the students expect to enter, so that the action of the research can only be understoodby an ongoing act of interpretation and reflection. Guided by the concept of complimentarity,this Action Research project adopts a mixed method approach to data collection and analysis:qualitative data are predominantly those provided by the participants – the students enrolledin the Requirements Engineering unit over the duration of the study, and the practitionerssurveyed in the lead up; quantitative data focus on the assessment elements of the unit, andon the responses to diagnostic instruments undertaken by students. Thematic analysis anddescriptive statistics provide the broad perspectives most appropriate for the development ofthe multiple interpretations required to negotiate understanding.Evaluation of three Action Research cycles shows that a shift in focus from technical competencyto the soft and metacognitive skills enables the competent practice of RequirementsEngineering. However, an incorrect learning environment can still exist between professionalpractice and non-traditional education - what is needed is tuning at a finer granularity so thatthe characteristics of professional practice are mapped to and reflected in the learning modelthat is applied. The research also shows that, as well as alignment between discipline andlearning model, a relationship exists between learner and learning model, and suggests thatthis relationship should be exploited in the development of competent discipline practitioners.xv

.DeclarationI declare that this thesis does not incorporate without acknowledgment any material previouslysubmitted for a degree or diploma in any university and that to the best of my knowledgeit does not contain any materials previously published or written by another person exceptwhere due reference is made in the textJocelyn Armaregoxvi

.AcknowledgementsAction Research, by definition, requires the participation of others besides the researcher inthe research undertaken.I would like to acknowledge the many different types of support I have been fortunate toreceive throughout the duration of this work.Again implicit in the definition, Action Research is undertaken in the context of an organisationand its culture. For the purpose of this work, that was Murdoch University’s Schoolof Engineering.I arrived at Murdoch with very strong ideas about how not to teach Software Engineering.Two were most influential in this work:• I had problems teaching students SE principles in one unit, knowing full well theywould discard them in the next• I was not really interested in my students knowing about Software Engineering – Iwanted them to learn to be Software Engineers.Murdoch Engineering provided a wonderful environment for exploring these ideas. An undergraduateSE programme had been initiated and my first tasks were to write curriculum forit, and to review other curriculum being developed to ensure good coverage and consistencyacross all core SE units. Our students would graduate as Engineers specialising in software:Software Engineers.However, this environment would not have been sufficient for the work I would undertake –initiating change requires the support of a champion. Professor Geoffrey G Roy fulfilled thisrole. As Professor of SE in the School of Engineering, and, later, Head of Engineering, hewas willing to listen to my proposals (the first being that I would not ‘lecture’ in any unitI took) and allowed me to space and resources to try them out. Ultimately this led to themodel of learning I was developing being applied for all Engineering teaching at the thirdand fourth year level.xvii

This leap, from one lecturer to all Engineering academics, would not have taken place withoutthe support provided by the rest of the SE team. In particular, Lynne Fowler was instrumentalin ensuring that a seamless approach to the learning of SE was maintained. What we didn’twant was to see-saw the students back and forth between the learning models I was applying,and more traditional approaches. Lynne was willing to follow through with student-centredlearning in the units she co-ordinated, so a holistic approach was generally maintained. Thesuccess of this was demonstrated when students complained that other units ‘treated themlike school students’. Lynne’s collaboration was also invaluable in the learning styles researchundertaken within the School. This was her initiation, that I wanted to use and extend.Action Research requires participants willing to change. To all the SE students who allowedme to scrutinise their learning, to place them in learning environments which both challengedtheir perceptions of how they should be taught and took them outside their comfort zone,thank you. The fact that so many of you have kept in touch since graduating, and are willingto share the joys of your successes is wonderful. Obviously your undergraduate experiencewas not that bad, although the grumbles were pretty extensive at the time.Outside of Murdoch support was strongest from two different directions.To my partner, Andrew Marriott, thank you for allowing me the space to do this work – thatoften meant pretending I was working full-time, rather than part-time.Considering everyone was undertaking some form of study during this period - Andrew wasundertaking a PhD part-time as well as being in full time work; our daughters were completingtheir secondary schooling and then undertaking tertiary study - our household has inevitablybeen chaotic. Our teenagers learnt to shy away from mum in front of the computer. Onlythe cats saw it as an opportunity to snuggle on my lap. So thank you to Ariel and Tegan(the daughters, not the cats) – I hope you get all the support you need as you further yourstudies.Finally I must acknowledge the encouragement provided by Professor Paul Swatman. Asupervisor who enables his research students rather than constraining them has a rare gift.Thank you Paul for helping me maintain a balance between this freedom and the disciplinenecessary to complete good research. I hope I do as excellent a job supervising my researchstudents.Jocelynxviii

Chapter 1The problem with RE educationThe term Requirements Engineering (RE) denotes both a discipline area and an early phasein the development of software. Within the IT specialisation of Software Engineering (SE)Requirements Engineering is the term of choice 1 . Elsewhere, it is called system or softwareanalysis or may even be encompassed within initial design considerations. Although a commonlyaccepted definition of RE has yet to be developed, Loucopoulos and Karakostas (1995,p vii) suggest it is widely agreed thatRequirements Engineering deals with activities which attempt to understand theexact needs of the users of a software intensive system and to translate such needsinto precise and unambiguous statements which will subsequently be used in thedevelopment of the system.Others (LeBlanc and Sobel, 2004, p 25) also define RE in relation to the system that willresult:Requirements represent the real-world needs of users, customers, and other stakeholdersaffected by the system. The construction of requirements includes ananalysis of the feasibility of the desired system, elicitation and analysis of stakeholders’needs, the creation of a precise description of what the system should andshould not do along with any constraints on its operation and implementation,and the validation of this description or specification by the stakeholderswhile an alternate perspective defines RE through its goals:The goal of requirements engineering is to establish and maintain the communicationand understanding between users, developers, customers, etc. of softwaresystems. Eliciting and defining requirements is a creative and iterative process.All participants of this process have to create a complete and consistent systemspecification – often in a textual representation – from vague and often contradictoryinformation.(Pohl, 2003)These definitions exemplify the conflicting perspectives that exist in the discipline.dominant views on the nature of Requirements Engineering in systems development is based1 throughout this document the term RE will be utilised to refer to the discipline. This is in acknowledgementthat a large component of the study was undertaken within a Software Engineering environment. Inother contexts,the term of choice may be systems/software analysis1The

on a positivist perspective – through an iterative process comprising complex, tightly coupledactivities user requirements are captured, structured and accurately represented so that theycan be correctly embodied in systems which are of good quality. The research literature of REpractice, on the other hand, exposes a different, interpretivist view. Here knowledge discoveryfacilitated by opportunistic behaviour and creativity dominate, with an acknowledgement ofboth the wickedness of the domain and the cognitive load of becoming competent in it.The perspectives taken on the nature of RE supports Leite (2000)’s assertion that formaleducation for RE is a major challenge of the next decade. This education has also beendominated by the positivist perspective – in general it is based on a model of professionaleducation applied in science/engineering contexts, with an aim to gain proficiency in applyingknowledge. Here science provides a rational foundation for practice [original emphasis], withpractical work at the last stage of the curriculum, where students are expected to apply sciencelearned earlier to real-life problems (Waks, 2001). However, studies of software practitionersindicate that what is taught in their formal education does not match the knowledge neededin daily work, where conceptual understanding in order to act as agents of change is of primeimportance. This gap, usually referred to being in ‘soft skills’, has been identified throughpractitioner studies as comprising affective skills, cognitive skills related to higher orderlearning and metalearning/metacognition as well as strategies to enable life-long learning.The purpose of this research is to develop and implements learning strategies which addressthe issue of aligning the competency expectations of practitioners of Requirements Engineeringwith formal education for RE, so that, by addressing the practitioner gaps, graduatesgain leverage in becoming competent professional practitioners.1.1 The nature of REA consistent assumption exists within the literature discussing systems development: that itis a cognitively intensive activity (Brooks, 1986), and that the early phases are problematicin the attempt to ensure the development of quality software (Boehm, 1981). In much ofthe literature, these problems can be traced back to issues in the requirements capture andanalysis component of the project. Contributors to problems with software developmentprojects include:• requirements issues (incompleteness, ambiguity etc) (Bell and Thayer, 1998; Johnson,1994)• expertise: lack of skill and training of RE practitioners (Senn, 1978; Lubars et al, 1993;James, 1994; Johnson, 1994; Sommerville and Sawyer, 1997; Kamsties et al, 1998; Morris2

et al, 1998; Nuseibeh and Easterbrook, 2000)• complexity of RE and RE process, including contingency issues (Kamsties et al, 1998;Morris et al, 1998; Carroll and Swatman, 1999; Houdek and Pohl, 2000; Hofmann andLehner, 2001; Zowghi et al, 2001)• (lack of) sensitivity to the environment of the system (Senn, 1978; Lubars et al, 1993;Emam and Madhavji, 1995; Sommerville and Sawyer, 1997; Kamsties et al, 1998; Morriset al, 1998; Nuseibeh and Easterbrook, 2000)• communications (Al-Rawas and Easterbrook, 1996; Zowghi et al, 2001).However, despite this body of work examining issues with software development, Conn (2002)found that, although there is an expectation that new graduates should know about basicissues in RE, it is a ‘surprise’ to them that requirements is a major cause for softwaredeficiencies. Nikula et al (2000) also conclude that general knowledge of RE in industry maybe seen to be ‘quite weak’.One explanation for the problems occurring in this early phase of software development looksat the nature of the RE discipline. It is viewed as one of conceptual complexity (Batra andDavis, 1992; Pohl, 1994; Sawyer and Kotonya, 2000). This is characterised as involving multiple,wide-application conceptual structures (schemas, perspectives, organisational principles,etc) – each of which is individually complex (Spiro et al, 1991) – interacting simultaneously.In the late 1960s those involved in the development of software agreed that one mechanismfor dealing with the range of intrinsic difficulties with software (eg complexity, visibility,and changeability (Brooks, 1986)) was to embed its production within an applied scienceenvironment: an engineering approach (Royce, 1970) was advocated as the best means tosolve such problems, whether they were scientifically/mathematically solvable optimisationones or not (Mulder, 2006).However, despite the accepted view within the majority of the software development literature(eg Ghezzi et al (1991); Jackson (1995); Sommerville and Sawyer (1997); Pfleeger(1999); Loucopoulos and Karakostas (1995); Banks (2003)) that RE is an iterative processcomprising complex, tightly coupled activities (Sawyer and Kotonya, 2000), its characteristicssuggest that problems in the discipline cannot be successfully addressed from the perspectivesadvocated by a scientific/engineering approach.Indeed, some researchers and practitioners (eg Maiden and Gizikis (2001) and Nguyen andSwatman (2000a), amongst others) argue that the accepted view is based on fundamentallywrong ideas regarding the successful development of software. RE as a smoothly evolution-3

ary process should also be regarded as flawed. Rather, it can be demonstrated to be one ofknowledge discovery (Guindon, 1989) facilitated by opportunistic behaviour (Guindon, 1990;Visser, 1992), and creativity (Lubars et al, 1993; Maiden and Sutcliffe, 1992; Maiden andGizikis, 2001; Thomas et al, 2002). These add richness to the mental model of the problemsituation (Batra and Davis, 1992) developed through an exploratory and self-correctingdialogue (Bach, 1999).Exponents of this view of RE assert it is hampered by strict adherence to an engineering andscience perspective. This is seen to restrict the essential characteristics of the process (such asopportunism) (Guindon, 1989); assists in adding accidental complexity by restricting naturalproblem-solving (Sutcliffe and Maiden, 1992); interferes with the management of knowledge(Visser, 1990) and inhibits the necessary creative thinking required by superimposing goalstoo early (Boden, 1997).The issues noted above characterise the discipline of RE as ‘wicked’ (Bubenko, 1995). Conklin(2005) provides a summary of criteria for wicked problems, based on the work of Rittel (1969),who coined the term:• the problem is ill structured, an evolving set of interlocking issues and constraints, sothat every solution that is offered exposes new aspects of the problem, requiring furtheradjustments of the potential solutions. What the problem is depends on who you ask• wicked problems have no stopping rule – a solution satisfices (Simon, 1981)• solutions to wicked problems are not right or wrong – the determination of solutionquality is not objective and is assessed in a social context in which judgement is likelyto vary widely and depends on the stakeholders values and goals• every wicked problem is essentially unique and novel – the solution will always becustom designed and fitted. Over time one acquires wisdom and experience about theapproach to wicked problems, but one is always a beginner in the specifics of a newwicked problem• every solution to a wicked problem is likely to spawn new wicked problems• wicked problems have no given alternative solutions – it is a matter of creativity todevise potential solutions, and a matter of judgement to determine which are valid,which should be pursued and implemented.Conklin (2005, p 10) concludes:The point is not so much to be able to determine if a given problem is wicked ornot as to have a sense of what contributes to the ‘wickedness’ of a problem.4

Practitioners, and students, are seen to need conceptual knowledge in several overlapping domainsin order to grapple with this wickedness and perform RE tasks successfully. Nuseibehand Easterbrook (2000) suggest Requirements Engineering draws on the cognitive and socialsciences to provide, not only theoretical grounding, but also practical techniques, while philosophy(epistemology, phenomenology and ontology), logic and theoretical computer sciencealso are elements within the profession. In particular, characteristics of expertise become fundamentalskills for Requirements Engineers. These include the ability to manipulate multiplerepresentations of problem components, and to effectively utilise the notations and symbolsystems that are shared knowledge within the communities in question; the ability to collect,manipulate and analyse many different forms of data and then present them in meaningfuland useful ways to any of the many different discourse communities (Gordon, 1996) andthe ability to manage knowledge structures (to plan) and exploit opportunistic and creativecognitive behaviour.However, as Lethbridge (2000) notes from the results of his survey, knowledge taught tosoftware practitioners and managers in their formal education does not always match theknowledge needed to be applied in daily work. This survey is supported by both anecdotalevidence (Senn, 1978) and a number of studies of RE practitioners, the activities theyundertake and technology transfer:• Bell and Thayer (1998) observed that inadequate, inconsistent, incomplete, or ambiguousrequirements have a critical impact on the quality of the resulting software.Almost twenty years later, the Standish Group conducted a survey over 8000 projectsin 350 US companies (Johnson, 1994). It found that requirements issues (eg pooruser input(12.8%), incomplete (12.3%) or changing requirements(11.8%)) are the singlelargest contributor to problems with software development projects (over 35%). Theremaining significant factors (13%) were poor technical skills and poor staffing• Houdek and Pohl (2000), in their analysis of RE process in industry, concluded thatRE activities are heavily intertwined (and may not even be perceived as separate) andthat defining a comprehensive model of an RE process was impossible. Rather,‘process chunks’ may be defined. This is supported by Hofmann and Lehner (2001) whofound that RE teams in industry saw RE as an adhoc process, in need of processimprovement (Sommerville and Sawyer, 1997; Nikula et al, 2000)• Senn (1978) describes the failures and shortcomings of systems analysts and bases theseon lack of skill, training and sensitivity, while training and inherent complexitywere among the key problems in transferring RE technology to industrial practice5

(Kamsties et al, 1998; Morris et al, 1998). The transfer and adoption of research findingsis also difficult (Nikula et al, 2000)• knowledge management during the requirements phases is a major problem (Zowghiet al, 2001), while Lubars et al (1993) concluded from an RE study involving 10 corporations,over 20 projects and approximately 90 developers, that as well as stafftraining not being well supported, many of the problems encountered were organisationalrather than technical. Other studies highlight the importance of non-technicalissues (Emam and Madhavji, 1995), including cultural differences (Zowghi et al, 2001),characteristics of the project managers (Carroll and Swatman, 1999) and communicationsbetween developers and users. Al-Rawas and Easterbrook (1996), in their studyof the problems associated with the process of RE, concur• issues resulting from a lack of expertise in RE activities is noted by James (1994)while discussions of both process improvement for RE (Sommerville and Sawyer, 1997)and important developments in the discipline. Nuseibeh and Easterbrook (2000) acknowledgethe value of ability and experience of the personnel and the importanceof the ‘environment’ of the system• And finally, Leite (2000)’s assertion, that formal education for RE is a major challengefor the next decade, is supported by Nikula et al (2000)’s conclusion (cited earlier)that general knowledge of RE in industry may be seen to be ‘quite weak’.Practitioners suggest that some of the issues confronting the discipline can be traced toformal education, in not addressing the gaps identified by them in education for softwaredevelopers in general, and Requirements Engineers specifically – most of the books andclasses are impractical (Bach, 1997). Although the technical competency of graduates can,in general, be assumed, other, softer, skills are considered by practitioners as lacking (see,for example Macauley and Mylopoulos (1995a); Lee (1999b); Lethbridge (2000); Lee (2004);Minor (2004)). These skills include• an understanding of business functions and organisational knowledge (Doke and Williams,1999; Noll and Wilikens, 2002)• the ability to teach themselves what they need to know to perform the task successfully(Lee, 1999b; Scott and Yates, 2002)• career resilience (Waterman et al, 1994)• interpersonal skills and personal attributes (Turley, 1991; Snoke and Underwood, 1999;Scott and Wilson, 2002; Macauley and Mylopoulos, 1995a; Minor, 2004)6

• generic attributes (Lethbridge, 2000).The value of these skills is confirmed in other studies, both general (eg Scott and Yates(2002); Scott and Wilson (2002)) and specifically in RE (Kozar, 1989; Zowghi, 2004).However, despite a clear indication that there are issues in the education of RequirementsEngineers, very little research exists that addresses these.Learning for Requirements EngineeringAny software development project is seen as knowledge-intensive, with many conceptsdeveloped to ease or guide the processing of knowing (Robillard, 1999), and learning (Klemolaand Rilling, 2002). This complexity, in addition to case-to-case irregularity in the domainsof application (their ill-structured nature (Simon, 1973)), poses a serious challenge for thelearning of Requirements Engineering practice.However, the accepted view that a science/engineering approach to software developmentwould ensure quality influences the learning of RE: by implication a scientific/engineeringeducation was seen as the mechanism to train students to be competent practitioners. Outsidethe science/engineering academic faculties, the education of software developers alsomodelled scientific and engineering methodologies. Benson (2003) notes that within theemerging Information Systems (IS) discipline of the 1970s, academics were migrants to thediscipline, with an overwhelming majority having qualifications in other areas, most oftencomputer science. Practitioners also relied heavily on scientific, mathematic and engineeringdisciplines, with many migrating from engineering and manufacturing. In general, thereforethe normative professional education curriculum continues to be the basis for IT education.This challenge posed by the learning of RE may be described more generally as the natureof learning in complex and ill-structured domains (Spiro et al, 1991). Here an expert levelof cognitive flexibility is required: conceptual oversimplification leads to failures that takecommon, predictable forms and to an inability to apply knowledge to new cases (failures oftransfer). Cognitive flexibility includes• the ability to represent knowledge from different conceptual and case perspectives and,later, the ability to construct from these a knowledge ensemble tailored to the needsof the understanding or problem-solving at hand – the same items of knowledge needto be presented and learned in a variety of different ways and for a variety of differentpurposes• the development of cognitively flexible processing skills and the acquisition of knowledgestructures which can support this. Flexible environments are required to permit the7

same items of knowledge to be presented and learned in a variety of different ways andfor a variety of different purposes, at different levels of granularity.Progress is being made in understanding the way knowledge is structured and how expertsthink in specific disciplines. Donald (2002), in particular, represents higher-order thinkingin several disciplines to show how teachers and students perceive the learning process, whileothers links the scientific study of thinking and learning to classroom practices. Both approachesreach the conclusion that disciplines need to be more involved in the research onhow people think and how students learn. Donald (2002, p 299) states:There is a substantial convergence in the need for deeper understanding of thedisciplines. The continuing challenge is how to draw on the expertise of scholarsto improve post-secondary education.However, there is a dearth of literature examining the education of Requirements Engineers.Numerous studies examine the learning of specific technical skills (such as programming),while student performance in capstone projects, usually involving groupwork, is also an aspectof IT education that has had some exposure in the literature. Although several studies haveexamined the practice of software development in novices, this has rarely isolated the REprocess. In general, these studies look at IT student performance in problem-solving anddesign tasks as a mechanism for either comparing expert performance against novice (egstudies by Curtis et al (1987); Guindon (1990); Visser and Hoc (1990); Adelson and Soloway(1985)), or in order to isolate a characteristic of task performance (such as opportunism(Visser, 1990)). How the students learnt the skills to perform the task has rarely beendiscussed, while there appears to be no research published that addresses the specific REpractitioner concerns described here in an educational context.The approach taken for examining education for IT specialisations in general (and henceto some extent taking up this challenge) has been the revision of both the various modelcurricula which embrace aspects of education for software development (Engel and Roberts,2001; Gorgone et al, 2002a; LeBlanc and Sobel, 2004), and the Bodies of Knowledge whichunderpin them. While a Body of Knowledge for Requirements Engineering has not yet beenidentified and defined, some components are included in other BoKs and curriculum models.These texts describe Guiding Principles – the foundational ideas and beliefs that guidedtheir development, and the desired student outcomes for an undergraduate curriculum in theappropriate specialisation. This approach, and its effectiveness, are discussed in some detailin Chapters 2 and 3. In summary the guidelines could be described as overarching – theoutcomes discussed in terms of pedagogical strategies that support them. The quote belowis one example of this approach:8

Guideline – Students should be trained in certain personal skills thattranscend the subject matter. The skills below tend to be required for almostall activities that students will encounter in the workforce. These skills must beacquired primarily through practice:*Exercising critical judgment: Making a judgment among competing solutions isa key part of what it means to be an engineer. Curriculum design and deliveryshould therefore help students build the knowledge, analysis skills, and methodsthey need to make sound judgments. Of particular importance is a willingness tothink critically. Students should also be taught to judge the reliability of varioussources of information.*Evaluating and challenging received wisdom: Students should be trained to notimmediately accept everything they are taught or read. They should also gain anunderstanding of the limitations of current SE knowledge, and how SE knowledgeseems to be developing.*Recognizing their own limitations: Students should be taught that professionalsconsult other professionals and that there is great strength in teamwork.*Communicating effectively: Students should learn to communicate well in allcontexts: in writing, when giving presentations, when demonstrating (their ownor others) software, and when conducting discussions with others. Students shouldalso build listening, cooperation, and negotiation skills.(LeBlanc and Sobel, 2004, p 40)A review of the current state of both the conceptualisation of the discipline, and educationfor it therefore suggest a range of problems exist:• the science/engineering view of the discipline does not align with the findings from researchon practitioners. The implications of this are two-fold for graduates entering thediscipline: their conceptualisation does not match the reality of the profession – a scientific/engineeringapproach hampers RE practice; and the education they have receiveddoes not prepare them for the real-world – the discipline is seen as needing conceptualknowledge in several overlapping domains at a competency level that approachesexpertise• the appropriate soft skills, valued by practitioners are not well developed through formaleducation. These include affective and cognitive skills as well as understanding of the‘context’ in which the task being addressed exists• the adaptability and flexibility required by practitioners is also not well developedthrough formal education. These are included in generic attributes such as life-longlearning• existing literature is lacking in two respects: it does not report research that attemptsto explore what might be done, educationally, to address practitioner concerns, orhow it might be done better, nor does it report research that trials new strategies9

in RE education. In particular, insights from learning theory and models, such asconstructionism and learner-centring, which appear to address the concerns raised bypractitioners, are reported only as isolated case studies.1.2 The nature of the studyThe objective of this research is to make a contribution to the discipline of RE and its educationby aligning practitioner needs with learning models that address these needs. Duringthis study I will examine how the discipline is perceived in the literature and by practitioners;what is taught, in terms of discipline content and competency levels; how it is taught; andwhat practitioners indicate is missing from the formal education. Using the findings fromthese investigations I will explore the gap between the competency expectations of practitionersof RE and traditional formal education for the discipline, as exemplified by the Bodiesof Knowledge and model curricula. The three IT specialisations of Software Engineering,Computer Science and Information Systems (considered the most visible computing areas(Glass, 1992)) will form the basis of this component of the study. The differing foundations(epistemological, psychological and philosophical) of these IT specialisations are also examined,as these produce differing bodies of knowledge, and different approaches to undertakingthe activities which comprise the discipline.Having identified the gaps reported by practitioners, both through published studies anda small, purposeful sample of RE professionals in industry, I will examine the literature oflearning theory and models. In this way I hope to make explicit the degree of alignment (ordissonance) between practice and education, in effect between the theory-in-practice (whatpractitioners do and what competencies they need to do what they do) and the espousedtheory (what formal education says practitioners do, and how students are taught to do it)(Argyris and Schön, 1974). This task will demonstrate that traditional formal education doesnot meet the competency expectations of discipline practitioners, and assist me in identifyingstrategies, based on non-traditional teaching, which go some way towards addressing these.Once teaching/learning pedagogies that support the development of the knowledge and skillsrequired by practitioners have been determined, I will develop intervention strategies forthe teaching of Requirements Engineering. Accepting the view that RE is a ill-structured‘wicked’ domain, the interventions undertaken in this research examine differing modes ofinquiry and exploration in order to create a learning environment that more closely modelsthe experiences of practitioners, and addresses their expectations of novice RequirementsEngineers. These interventions are planned, actioned and evaluated as cycles within theAction Research methodology. Reflection on the success of the intervention will inform the10

planning of subsequent cycles.By matching the gaps identified by practitioners and learning models that purport to focuson these, and subsequently proposing and actioning learning interventions this study showsthat non-traditional approaches provide leverage for a graduate entering the profession ofRE. However, the evaluations undertaken during this study show that an incorrect learningenvironment can still exist between professional practice and non-traditional education– what is needed is tuning to a finer granularity so that the characteristics of professionalpractice are reflected in the learning model. The research also shows that, as well as alignmentbetween discipline and learning model, a relationship is confirmed between learner andlearning model, and suggests that this relationship could be exploited in the development ofcompetent practitioners.1.2.1 Frameworks for the studyFigure 1.1 describes a conceptual framework for the context of this research, and identifiesthe elements that need to be examined.Figure 1.1: Influences on the learning environment for REThe Bodies of Knowledge (BoKs) and model curricula are a distillation of expert opinionand domain-specific texts. In RE the breakdown is seen to cover the areas discussed in REtexts and standards, either identically, or as noted in the SoftWare Engineering Body ofKnowledge (SWEBOK):11

derived from these and other sources to reflect a consensus, and mirror the matureand stable concepts in Requirements Engineering(Sawyer and Kotonya, 2000, p 21)This approach is mirrored in attempts at developing model curricula, with the occasionaladdition of guidelines addressing generic attributes. The Computer Science volume of ComputingCurriculum 2005 (CC-CS) follows the same pattern and draws on the same sources(with the addition of a guidelines document (Bagert et al, 1999)), as does the the IS curriculum(CC-IS) (Engel and Roberts, 2001), and the SE volume (CC-SE), which explicitlyacknowledges its dependence on SWEBOK (LeBlanc and Sobel, 2004).These help determine the learning situation for a discipline, which is influenced by pedagogyconsiderations (eg learning theories and models) and indirectly through practitioner feedback.Perspectives on learningHannafin (1997a) and Reeves (1994) suggests that several dimensions are relevant in thedescription of learning systems:• epistemological foundations – are concerned with theories about the nature ofknowledge, and describes the world view to be disseminated. At one extreme (objectivism),content aims to be comprehensive and accurate, and based on advice fromexperts in the field. At the other (constructivism), content reflects the spectrum ofviews in the domain, providing multiple perspectives/options for constructing knowledge• psychological foundations – represent beliefs about how individuals think and learn.In this continuum, shaping desirable behaviours via stimuli, feedback, reinforcement etcat one pole contrasts with a cognitivist emphasis on mental models and the connectionsbetween them. The type of knowledge to be constructed should drive the learningstrategy employed, with support provided for deductive and inductive learning• philosophical foundations – emphasise how to-be-learned domains are representedand affordances provided to support learning. An instructivist foundation stresses theimportance of goals and objectives drawn from the domain. Constructivist foundations,on the other hand, stress the primacy of learner intentions, experience and metacognitivestrategies through a rich environment that can be tailored to individual needs.These dimensions describe the nature of learning, the methods and strategies employed, andthe ways in which the to-be-learned domain should be organised and made available to thelearner.12

Perspectives on REWithin the IT disciplines, the complexity of the systems development process has also led tomultiple approaches to its definition and study: the work of Iivari (1991) and Glass (1992)identify and categorise these, based on epistemological and ontological positions taken, whileShackelford (2005) provides an overview of what might be considered computing today. Ata fundamental level, the assumptions made on, for example, the nature of the system or theimportance of its context, and the nature of knowledge, influence the perspective taken andhow the work is undertaken.The dominant views as to the nature of Requirements Engineering in systems developmentspan across the dimensions positivist – non-positivist and hard – soft. Traditionallythese perspectives have led to three broad categories - hard (objective, positivist, scientific),soft (subjective, interpretivist) and hard/soft (exemplified by the socio-technical approach)(Loucopoulos and Karakostas, 1995; Checkland and Holwell, 1998). Whether requirementsexist in some objective sense, and how methods and tools used impose and embody differentworldviews and create different worlds for the system under consideration. The beliefs heldregarding these aspects affect how the RE student learns to become a competent practitioner,and help determine what skills must be acquired.Despite an increasing emphasis on the relevance of unique factors specific to each situation(and the implied less general regularities), RE has traditionally reflected heavily the positivistepistemology. For the education of Requirements Engineers, what is disseminated to the nextgeneration is based on what is in the textbooks (Iivari, 1991; Checkland and Holwell, 1998;Sawyer and Kotonya, 2000) as well as the ideological stance adopted, and hence, in general,also align with a positivist (hard or soft) perspective. However, partly due to the diversityof the problems tackled, characterised by across-case irregularity, there is no single correctway of undertaking Requirement Engineering. Robinson (2001) suggests RE is situated inSchön’s swampy lowlands (Schön, 1983) and therefore fails to yield to a technically rationalsolution. This suggests the challenges of teaching and learning Requirements Engineeringtranscend the epistemological stance taken.1.2.2 Research approachThis research is undertaken with an acceptance of the view that not only is education a socialdiscipline, but also the (knowledge/discipline) domain into which the students expect to enter.These social system define the roles, values and rewards of their members, and their expectationsregarding participation within them. The action of the research can only be understoodby an ongoing act of interpretation (Mansell, 1991) and reflection (Schön, 1987). However,13

this interpretation can never be complete, but rather always include elements of uncertaintyand open-endedness. This demands an iterative interweaving of multiple interpretations untila sophisticated understanding is negotiated (Merleau-Ponty, 1962; Polkinghorne, 1988).I have adopted a mixed method approach to data collection and analysis as the most appropriatefor the development of the multiple interpretations required. This approach isguided by the concept of complimentarity that reflects the intention to use the results of onestrand to elaborate, enhance, and illustrate the results from the other strand. In this study,the predominant approach is qualitative but containing smaller quantitative data collectionphases that are considered interpretively. As Creswell (2003) indicates, the value of a nestedconcurrent approach is that it provides broader perspectives than by using the predominantmethod in isolation.Qualitative data are predominantly those provided by the participants – the students enrolledin the Requirements Engineering unit over the duration of the study, and the practitionerssurveyed in the lead up. This data was freely available to the researcher: as co-ordinator of theunit, the School- and University-level data are required to be considered in administering theunit. Where data collection was more difficult or intrusive, additional feedback mechanismswere built into the components of the unit. Practitioner data was solicited through a formalstudy conducted as part of this research.Quantitative data focus on the assessment elements of the unit, and on the responses todiagnostic instruments undertaken by students, either in the normal course of their studies,or specifically as part of this research. Information from the student database was alsoavailable.Other resources were also easily available: as a member of the academic staff I was involvedwith the development of the unit from its conception, and therefore had access to policy,design and review documentation whenever it existed. As academic staff I was also able toaccess the appropriate policies (eg for assessment) of Murdoch University, and to draw onthe expertise of the Teaching Learning Centre (TLC) for guidance where required.The data collected was interpreted primarily by means of thematic analysis, or throughdescriptive statistics where these were considered appropriate. However, in accepting themetaphor of this research as a journey, annotated ‘postcards’ are provided from the datathemselves wherever possible.Issues to be consideredEducational research within a discipline is cross-discipline: the discipline-specific perspectivesmay align or be dissonant to the perspectives accepted within an educational research14

environment. The tension caused is heightened when the discipline itself is cross-discipline.The broad discipline of IT has been described as such (Pham et al, 2005), accompanied, ashas already been noted, by different world views and understandings of the research domain.Studies by Brew (1999, 2001) regarding research in an educational, cross-disciplinary environmentare of interest in that she identifies a journey variation to understanding research.Here research is a transformational voyage of discovery in the researcher’s experience, withthe researchers’ presence (awareness) influencing their understanding of research issues orimpact on broader life. This metaphor aligns with the narrative nature of the reporting ofthis research. First discussed by Connelly and Clandinin (1990), narrative research reportingin education places increased emphasis on the reflective component of the reporting, acknowledgesteacher knowledge (and therefore the ‘insider’ status) and attempts to bring the teachervoice to the forefront (Creswell, 2004). Amongst the defining characteristics of narrative arechronology (unfolding over time); emplotment (the literary juxtaposing of actions and eventsin an implicitly causal sequence); and embeddedness (the personal story nests within a particularsocial, historical and organisational context, which is explicitly stated) (Ochs andCapp, 2001). This view aligns well with the research environment that is the context of thisstudy.However, a requirement of such cross-disciplinary environment is that the diverse intereststhat impact on the study are made explicit – the assumption that these are well understoodby all who examine this research cannot be made. The implication of this is a more exhaustivetreatment of several aspects of this research:• the development of the research design needs to consider concerns that could be raisedfrom each perspective• the literature of both the discipline being impacted on, and of the strategies beingapplied, need to be elaborated so that the argument made is understandable from eachperspective• the interventions themselves must be described and interpreted from each perspective.These considerations also impact on the structure of the thesis, so that some elements ofbackground or detail that more traditionally appear in one place (generally the review of theliterature) are interspersed throughout the discussion of the interventions, as required, forclarity. The purpose of these strategies is to facilitate the exposition of this work: readersfrom the domain addressed in this research may explore areas of unfamiliarity, or skim thoseof which they are knowledgable.15

The outcome of this study is an understanding of the educational dimension for RequirementsEngineering, in terms of both policy – what are the specific aims and outcomes ofRequirements Engineering education, and practice – how are these aims currently achieved;what should be changed.While these outcomes are based on Requirements Engineering education and practice inAustralia, it will be feasible to draw broader, indicative conclusions as the basis for futureresearch.1.3 Summary structure of the thesisFigure 1.2 provides a schematic view of the material covered in this thesis, and illustrateshow each chapter supports the thesis argument.Figure 1.2: Thesis chapters in support of the main propositionAs noted above, the cross-disciplinary nature of this work warrants a detailed treatment ofthe background to the research. Consequently, Chapter two focusses on the discipline ofRequirements Engineering, its nature and the perspectives taken by its exponents. Theseinclude the Bodies of Knowledge and model curricula developed as indicators of the knowledgeand skills required to practice as competent professionals. This is compared to informationof the practitioner perspective, and the perception they have of competency gaps in formaleducation. Chapter three examines the theory of learning, the foundations on which it arebuilt and the learning models that apply these theories.Together, these chapters provide the context for the research, and address the two primary16

components examinedQuestion 1 What degree of alignment or dissonance is there between practice and educationfor Requirements Engineering?Question 2 What (educational) strategies will address the missing competencies practitionershave identified?As has been noted, while the bulk of the background literature is located in these two chapters,as a mechanism for ease of reading, in reality the literature, as with the methodology, wasemergent. For this reason, some detail that could be considered background is still locatedwithin the appropriate chapter.Chapter four discusses the research design and approaches this from the perspectives of researchin education and research in IT disciplines. The methodology chosen, Action Research,is informed by the style described as the ‘Deakin’ (Carr and Kemmis, 1986; Kemmis and Mc-Taggert, 1988) approach. This has merit in being adopted for studies in educational contexts(Zuber-Skerritt, 1982, 1995). Strategies for data collection and analysis are emergent, thereforea variety is discussed, and a summary of the approaches taken provided.Chapter five synthesises the knowledge gained from the previous chapters in order to developand present a framework for the Action Research study undertaken. The framework is basedon the integration of several models, each addressing a specific aspect of the study: a modelfor Action Research in an educational setting (Borg et al, 1993), a model of organisationalculture that reflects the educational context of the study (Rogers, 2002), and a model ofreflection that incorporates the necessity for engaging in double-loop learning in order toachieve professional development (Hatten, 1997).The next three chapters describe the individual cycles undertaken, following the narrativetradition of a chronological account (hence the journey metaphor). Chapter six looks atthe Cognitive Apprenticeship model for RE education as a mechanism for enabling authenticlearning and facilitating knowledge transfer. Chapter seven explores Problem-based Learning(PBL) as a model that focusses on students dealing with ill-structured problems in ‘wicked’domains by taking control of their learning. The model developed and applied in this cyclealso addresses issues of enabling creativity within a supportive learning environment. Chaptereight examines a hybrid model developed on the basis of reflection on the interventions ofthe previous two cycles. This model is applied, and students followed into the subsequentunit. The focus here is on the longer term success of the learning strategies identified asappropriate for RE education.The final chapter, Chapter nine, provides an overview of the findings of each cycle, both ata conceptual level and in practical terms, discusses the value of these and indicates what17

possibilities arise as outcomes of the research. This final chapter also addresses issues ofevaluation of this Action Research project, based on the framework established in Chapterfive. The thesis concludes with a suggestion for the future of education in the discipline ofRE.18

Chapter 2What is taught in REThe hard part of teaching is not getting students to learn content: the hard partis getting them to learn how to learn and generate creative solutions(Wankat and Oreovicz, 1998, p 1)What is actually taught within a discipline is a complex synthesis deriving from the ideologyof the discipline, the context of the learning and the ‘tools’ used to facilitate that learning,all in theory influenced by the needs of practitioners in the discipline.The focus of this chapter is on establishing the context for this research into education forRequirements Engineering by providing an overview of the interrelationship noted above, andidentifying the depth of mismatch between the requirements of professional practitioners inthe discipline and the models of the discipline developed for its learning.Figure 2.1: Influences on the Learning Environment for RE (2)19

The first section examines the reality of education in the discipline of RE, viewed from theperspective of practitioner studies. Although several concerns regarding the fit of educationfor practice are raised, of particular interest to this work is the identification of specific gapsin education for RE.The second section looks at the tools used to facilitate learning, specifically the knowledgeof (what) content to a (how) specified level of mastery. The content (as a component of thetools used) may be taken from the appropriate Body of Knowledge (BoK), and the contextbased on coincidence between the ideology and the pedagogy applied.The subsequent section examines this aspect – the ideology and pedagogy most often ascribedto in education for software development.The shaded area in Figure 2.1 describes the elements of the conceptual framework presentedin Chapter 1 that are discussed in this chapter.Overview of FindingsThe practitioner perspective provided is based on studies reported in the literature and asmall case study undertaken locally. Practitioner studies of RE professionals indicate thatsignificant gaps exist between what is learnt through formal education, and the knowledgerequired in professional practice, where conceptual understanding, rather than applicationof technical knowledge is of prime importance. These gaps, usually referred to being in‘soft skills’, have been identified as comprising affective skills, cognitive skills related tohigher order learning and metalearning/metacognition as well as strategies to enable lifelonglearning.The breakdown of RE described in model curricula and BoKs is seen to cover the areasdiscussed in Requirements Engineering texts and standards, either identically, or reflecting aconsensus. This approach is mirrored in most attempts at developing BoKs: the CS volumeof Computing Curricula (Shackelford, 2005) (CC-CS) (Engel and Roberts, 2001) follows thesame pattern and draws on the same sources (with the addition of the Guidelines document(Bagert et al, 1999)), as do the IS (Gorgone et al, 2002a) (CC-IS), and the SE volume(CC-SE), which explicitly acknowledges its dependence on SWEBOK (LeBlanc and Sobel,2004).This is confirmed by the work of Minor (2004) who examined RE in representative tertiarylevel texts in the three core IT related specialisations (CS, SE, IS) and found that the corecomponents (he calls them central elements) show parallels. However, the emphasis placedon specific components of RE by the different IT specialisations, is noteworthy so that, forexample IS sees an increased importance on organisational elements and planning activities,20

with RE characterised as an organisational improvement process (Hoffer et al, 2002) basedon an identified problem. Differences noted include focus (eg overview of process versustechniques as in Kotonya and Sommerville (1997) and Davis (1990)) and context (eg theorganisational orientation in IS-based texts (as in Hoffer et al (2002)), as well as a greaterinclination to identify a broader skill set outside the technical area. In particular, IS-basedtexts acknowledge the importance of generic attributes: analytical and interpersonal as wellas management-oriented skills to support the analysis of a system.With regards to competency levels for RE elements, in general, model curricula advocate acompetency level hovering around Bloom’s level 3 (Application) as the most appropriate forthe completion of undergraduate education (see Table 2.1). The consistency of these findingssuggest that they can be universally applied to units addressing the discipline.Table 2.1: Minor: Bloom’s levels across model curriculaTopics CC-CS CC-IS CC-SEFeasibility Study n/a c (k)Elicitation a a cAnalysis a a aDetermination a n/a cDocumentation a a a/kVerification a n/a aRequirements Management n/a k k/cLegend: k – knowledge; c – comprehension; a – application; n/a – not addressedAlso in this chapter the perspectives through which the knowledge of RE is made availableis examined. Again, the literature of the domain provides a mechanism for identifying these.And again, a discrepancy is noted: a positivist perspective makes few, if any allowances forthe nature of RE, yet is dominant in the texts. The literature of research, on the otherhand, exposes the non-positivist view of RE. At this pole, experience based on practice andcreativity dominate, with an acknowledgement of both the wickedness of the domain andthe cognitive load of becoming competent in it. The transfer from research and practice toacademic texts is hampered by the lack of congruence in the perspectives adopted as well asthe ultimate objective of each – proficiency in applying knowledge (implied in the competencylevels considered appropriate for undergraduate learning) versus conceptual understandingin order to act as agents of change (implied in the focus on generic attributes (both cognitiveand affective) noted in practitioner studies).21

2.1 Practitioner perspectivesIn his Point/Counterpoint discussion, Bach stated that one reason Software Engineering isnot more seriously studied is the common industry belief that most of the books and classesthat teach it are impractical (Bach, 1997). An overview of the studies undertaken to gain apractitioner perspective indicates that such an indictment is not too far from the mark withregards to Requirements Engineering.2.1.1 Studies addressing the global perspectiveMost of the studies described below address RE activities in relatively general terms only:they addresses skills and knowledge required for software development activities, of whichRE is acknowledged as one (perhaps core) component. However, as Minor (2004) notes, theyonly examine the general importance of specific topics, as perceived by different stakeholders.Some of these studies examine whether the skills and knowledge needed for the activitiesnecessary to perform RE in detail are reflected in the respective model curricula, otherslook to practitioner perception of their formal education to map the relationship. However,since different approaches are taken in gaining this knowledge from different target groups:surveys, focus groups, fora or interviews applied to experienced practitioners, managers,recruitment staff, students and recent graduates, as well as examination of job advertisementsover the disciplines of IS, CS and Engineering, some insight into the practitioner perspectiveis possible.IS practitioner perspectivesSummarising his work of the previous decade on the knowledge requirements and professionaldevelopment of young IS workers Lee (1999b) refers to:• Trauth et al (1993) which found significant gaps between what industry expects ISworkers to know versus what universities teach IS students• Lee et al (1995) which examined the expectations of graduates of IS, both at the timeof the study and extrapolated, based on expected future developments. Their conclusionwas that the knowledge and skills required change, so that that the ability tolearn quickly on the job was critical to IS workers. In addition to technical skills andknowledge, the study identifies a wider range of non-technical skills, such as businessfunctional knowledge, interpersonal and management skills, and technology managementskills as important to IS professionals22

• Lee (1999a) focusses on the changing role of IT within organisations and the informationseeking behaviour of recent graduates (ie those with less than five years IS workexperience). Preliminary results from this study show that IS workers need not havea technology-relevant degree (over 80% of 300 graduates employed by a large IS consultingfirm over a period of two years majored in non-computing related areas). Inaddition, IS workers draw heavily from a ‘bipolar’ knowledge structure – most currenttechnical knowledge and localised team-centric project work, but are unable to exploittacit organisational knowledge outside their specific project.In acknowledging a gap between the academic preparation offered by universities and whatindustry demands, Lee concluded that academic programmes should stress fundamental concepts,but incorporate more team projects that emphasise information searching and problemformulation (as opposed to problem solving alone) so that students can deal more effectivelywith the challenges of industry. He noted that interpersonal communication accounts for themost important means of knowledge transfer in technological work, with team members asthe most utilised interpersonal information source. At the same time, interaction with usersand clients remains weak as does use of organisational knowledge.This is supported by Doke and Williams (1999). They looked at the importance of specificknowledge and skills for IS professionals in different IS positions. Not surprisingly, they foundthat, in general, those holding entry-level positions did not value ‘organisational knowledge’and ‘general IS knowledge’ to the same extent as they did technical knowledge. The latterbecomes less important (but not unimportant) later in a career. In all of the IS positions, thehighest ranked knowledge and skills were predominantly from the ‘organisational skills’ area(eg interpersonal behaviour and communications, oral, written and multimedia communications).System development knowledge and skills (including analysing information systemsand design and implementation of systems) also were highly ranked by all IS positions, suggestinga need to place more emphasis on these areas in formal education.Table 2.2: Lee: Comparison of IS formal education and industry requirementsIS education characteristics Industry characteristicsstructured learning environment independent and self-motivatedlearningworkers tend to be more task thanrelationship orientedworkers must interact closely andbuild relationships with colleaguesLee expands his discussion of this gap in a later study (Lee, 2004) (summarised in Table2.2). This latest work, of transition from study to workplace, also found that one of the‘reality shock’ involved in the socialisation of new graduates to work was the onus of teaching23

themselves what they needed to know in order to perform the task successfully. He concludes...educators should also help students to develop their initiatives and abilities todeal with ill-structured problems. This would require approaches which emphasizeindependent learning and collaborative teamwork.(Lee, 2004, p 135)Full integration into the organisation was seen to take up to two years, during which timethey were not considered an ‘insider’ and ‘working professional’.These results also verify Noll and Wilikens (2002)’s work. They identified the need for ISworkers to comprehend business functions as well as know how to develop technical solutions.Importance is given to team work, communication and organisational skills: these are fundamentalto IS professionals regardless of their main tasks. In turn, this confirms earlier workby Parker et al (1999), who conclude that people who fail to develop skills other than masteryover current technological tools are much more likely to become technologically stagnated inan interactive, social setting. Table 2.3 lists the core skills identified for all IS professionalsby Noll and Wilikens’ study.Table 2.3: Noll: core IS skillsSkillKnowledge of business functional areasAbility to interpret business problems and develop appropriate technical solutionAbility to understand the business environmentKnowledge of specific industryAbility to work collaboratively in a team project environmentAbility to develop and deliver effective, informative and persuasive presentationsAbility to plan, organise, and lead projectsAbility to plan, organise, and write technical manuals, documentation, and reportsSimilar issues arise throughout practitioner studies of IS since the early 1990s – a long termshift from programming and other technical subjects to business analysis and people-orientedskills – a change in emphasis to both generic attributes and managerial knowledge. Meanwhile,from the student perspective, awareness of the need for ‘career resilience’ has surfaced(Waterman et al, 1994).Gupta and Wachter (1998) notes that, often, a capstone project is recommended as themeans of achieving the needs expressed by industry in IS. This aspect of formal education isdiscussed in Chapter 3.24

2.1.2 CS/Engineering practitioner perspectivesFewer studies address the skills and knowledge needed in SE and CS. Turley and Bieman(1995) examined professional Software Engineers in an attempt to identify the competenciesand demographics that contribute to ‘excellence’ in performance. They provides a set of thirtyeight competencies that express a broad range of behaviours required of an SE engaged in‘the creation of software products’ (as opposed to maintenance, management etc). Theseare derived from multiple sources: analysis of interviews, self-described and suggested bymanagers as differentiating exceptional and non-exceptional performers. From a participationgroup with backgrounds in CS (75%) Engineering (40%) or Mathematics (21%) (allowingfor multiple degrees), Turley and Bieman identify four categories of competencies whichare statistically significant in differentiating between exceptional (XP) and non-exceptional(NXP) performers. A summary is provided as Table 2.4.Table 2.4: Turley rankings: competencies by participant categoryRank* Competency XP Rank! NXP Rank!Task Accomplishment3 Mastery of Skills and Techniques 4Personal Attributes1 Driven by Desire to Contribute 35 Perseverance NXP8 Maintains Big Picture View 59 Driven by a Bias for Action and a Sense XPof Urgency11 Driven by a Sense of Mission XP12 Exhibits and Articulates Strong Beliefs 3and Convictions14 Proactive Role with Management 2Situational Skills8 Responds to Schedule Pressure by Sacrificing2Parts of the Design ProcessInterpersonal Skills1 Seeks Help from Others 13 Helps Others 14 Willingness to Confront Others 4* Rank is based on the mean competency score for the entire sample: the top 5/4 for each category isindicatedOf the significant competencies associated with exceptional performance most are seen tocluster around the theme of external focus, with only Mastery of Skills and Techniques asa self-directed (internal) skill. Although it is possible to view these results as differentiatingbetween experience and inexperienced engineers, Turley and Bieman note that not allexperienced engineers become exceptional.Earlier Turley (1991) suggested a significant area for research was to explore how compe-25

tencies are reinforced. He concluded that education needs to support the development ofdifferential skills (namely interpersonal skills and personal attributes) through the creationof learning situations which stress these.Lethbridge (2000) also examined the industry perception: his aim was to gain a practitionerranking of the usefulness of specific topics compiled from the curricula of (emerging)SCE (Software and Computer Engineering) and CS, the influence of these on respondents’career and how much they had learned formally compared to what was required as a professional.Lethbridge grouped Engineering graduates (electrical, computer and other) andCS/SE (including IS graduates). Unremarkably, respondents learnt more mathematical andcore computing specific (content) knowledge, and less generic and SE knowledge (see Table2.5).Table 2.5: Lethbridge rankings: most and least learnt formallyRankTopicMost Learnt1 Specific Programming Languages2 Differential and Integral Calculus3 Linear Algebra and Matrices4 Probability and Statistics5 Data Structures6 Physics7 Differential Equations8 Set Theory9 Design of Algorithms10 Operating SystemsLeast Learnt66 Data Acquisition67 Maintenance, Re engineering and Reverse Engineering68 Marketing69 VLSI70 Robotics71 Software Cost Estimation72 Configuration & Release Management73 Entrepreneurship74 Process Standards (eg CMM / ISO 9000)75 NegotiationThis table (in its complete state) is notable in that the topics Technical Writing, and Analysisand Design Methods rank as having the 5th and 6th most pronounced bi-polar distribution(Lethbridge, 1999, p 14), an indication of differing educational focus. Although he foundfew surprises in the data: software topics are clearly learned far more by the CS/SE graduates,while the Engineers learn more about traditional Engineering topics, it is interestingto note that Engineers have more background in entrepreneurship and also ethics and professionalismthan Computer Scientists. The most pronounced bi-polar distribution regardingwhat respondents knew most at the time of the study focussed on both soft and RE skills26

(Leadership and Negotiation ranked third and fourth, and Data Acquisition ranked seventh).Unfortunately, many of these appear to have been learnt on the job (eg Requirements Gathering& Analysis ranks 6th for mostly learnt on the job, and 4th for very high on-the-joblearning). This topic is also highlighted as having a gap in training (inadequate knowledgebased on importance), with a 60 % difference noted between formal education and overallimportance (Lethbridge, 1999) (see Table 2.6). Although the relative youth of undergraduateSE programmes is potentially one explanation for the high differences exhibited in professionspecifictopics, model curricula and BoKs indicate such RE-related topics have usually beenincluded. Therefore, at least in this case it would seem that teaching does not reflect theneeds of the practice.Table 2.6: Lethbridge rankings: lag between formal learning and importanceRank Topic % Lag1 Negotiation 842 Configuration & Release Management 833 Leadership 734 Maintenance, Re engineering & Reverse Engineering 725 HCI / User Interfaces 676 Software Reliability & Fault Tolerance 647 Ethics and Professionalism 63Project Management 639 Management 6110 Requirements Gathering & Analysis 6011 Testing, Verification & Quality Assurance 5912 Object Oriented Concepts & Tech 5813 Giving Presentations to an Audience 5214 Software Design and Patterns 48Technical Writing 4816 Analysis & Design Methods 4417 Software Architecture 4318 Systems Programming 4219 Data Transmission & Networks 4220 Network Architecture & Data Transmission 40Of relevance to our consideration, Lethbridge computed overall importance of topics, based onthe average of both importance of details and influence. Table 2.7 provides the top rankings.Again RE topics rate highly (in italics).Meanwhile, managers rate both RE topics and more generic skills higher than developersat large (see Table 2.8). This is consistent with other studies that show a broader focus atmanagerial level.Lee also looked at the long term professional development of young engineers as technolo-27

Table 2.7: Lethbridge rankings: most importantRank Topic1 Specific Programming Languages2 Data Structures3 Software Design & Patterns4 Software Architecture5 Requirements Gathering & Analysis6 HCI / User Interfaces7 Object Oriented Concepts & Techniques8 Ethics and Professionalism9 Analysis & Design Methods10 Giving Presentations to an Audience11 Project Management12 Testing, Verification & Quality Assurance13 Design of Algorithms14 Technical Writing15 Operating Systems16 Databases17 Leadership18 Configuration & Release Management19 Data Transmission & Networks20 ManagementTable 2.8: Lethbridge rankings: most important for managersRank Topic1 Project Management2 Requirements Gathering & Analysis3 Giving Presentations to an Audience4 Management5 Ethics and Professionalism6 Analysis & Design Methods7 Software Architecture8 Leadership9 Testing, Verification & Quality Assurance10 Technical Writing11 Negotiation12 Network Architecture & Data Transmission13 Data Transmission & Networks14 Software Cost Estimation15 Process Standards (eg CMM / ISO 9000)16 Software Metrics17 File Management18 Performance Measurement & Analysis19 Maintenance, Reengineering & Reverse Engineering20 Probability and Statistics28

gists, in studies reported in the late 1980s and early 1990s (Lee, 1986, 1989, 1992, 1994).Starting from the premise that the primary focus of university education and industrialwork requirements were different, these studies explored on-the-job learning and informationseeking behaviours and found no correlation between academic achievement and job performance.Instead, what was found to have significance was that the work was challenging andthe approach taken to information seeking in order to keep up with the relevant changes inknowledge and information requirements. The success of the transition from an academicenvironment and the formation of social ties with veteran colleagues was also significant,while initial work experience influenced future career development. These results indicatethat the effective preparation of young technology workers involves far more than just a fixedset of academic subjects.2.1.3 The Australian perspectiveOther studies look at the situation in an Australian context. Again, these tend to be generic,rather than specific to RE.The aim of Turner and Lowry (1999)’s study was to achieve a better fit between universitystudy and the professional practice of IS. They based their survey on topics commonly foundin undergraduate university curricula, focussing on intellectual skills and personal attributesdesired by employers. The survey confirmed the high perception of the importance of analysisand design skills noted in northern hemisphere studies. They also found that, with theexception of programming languages, employers lay more emphasis on personal attributessuch as team skills or the ability to follow directions rather than specific academic skills.During 2001 a follow-up survey to explore the ‘other skills’ aspect of IS curriculum wasconducted (Turner and Lowry, 2003). This shows that, in general, both respondents withhigh and low levels of people contact rate soft skills higher than ‘hard’ academic skills. Thehighest ranking academic skill is Analysis and Design, even though it ranked below all butseven (of 24) of the soft skills. It may, however, be argued that several of the soft skills ratedabove this formal skill are in fact integral to it (such as problem definition skills and thinkingcreatively).Snoke and Underwood (1999) undertook a study to identify and examine the generic attributesrequired of entry-level employees from IS programmes of study. The study wassignificant in that it sampled a wide cross section of the IS academics in Australia, with thepopulation surveyed including representatives from all universities offering an undergraduatedegree in IS or with a major in IS as of July 1998. This study also showed that the per-29

sonal and group attributes are consistently more highly valued than the technical knowledgecompetencies. The high ranking of oral communications and team participation in Table 2.9suggested to the researchers that more group work and oral presentations should form partof the IS curriculum as this is a required skill in industry.Table 2.9: Snoke: top IS generic competenciesAttributeMeanRankWork as part of a team in a productive and cooperative manner 6.30Be able to participate in continued learning and intellectual development 6.28and develop critical, reflective and creative thinkingRetrieve, evaluate and use relevant information 6.22Oral communication skills 6.17Define problems in a systematic way 6.11Written communication skills 6.09Interpersonal skills 6.02Analyse, synthesise and evaluate the various solutions 5.95Self motivation 5.87With respect to the IS discipline possess coherent, extensive, theoretical 5.80and practical knowledgeConsider the quality of the solution and its timeliness 5.72With respect to the IS discipline be technologically competent (the person 5.71is able to use the current technology competently)Value the ethics of the Information Technology profession 5.70Embrace change and be obliged to engage in incremental improvement to 5.67keep up with the rapid change in technologyConfidence about their ability to learn independently 5.61With respect to the IS discipline possess theoretical and practical knowledge5.58in at least one reference discipline which include behavioral science,computer science, decision theory, information theory, organizational theory,management theoryTime management skills 5.51Work independently 5.47Participate in on-going professional development 5.38Demonstrate practical knowledge and understanding in at least one computer5.37languageThe largest Australian studies to date appear to be part of the series being undertakenthrough an Australian government supported Evaluations and Investigations Programme (Departmentof Education, Science and Training , DEST). These analyse some 200,000 commentsrecorded on the CEQ (Course Experience Questionnaire) since it was first administered in themid 1990s, from graduates of all 38 publicly funded Australian universities. A set of parallelstudies develops a tracking project that taps the experience of successful recent graduates,and operates from the assumption that graduates who have been working in professionalpractice for between two to six years are well positioned to identify what is likely to be most30

elevant for those currently studying at university. Scott and Wilson (2002) suggest thatsuch people have sufficient experience to know what counts in the real world of the professionwhilst not being too far away from their university course to have forgotten what it covered.They are also better positioned to make valid comment on the quality of their tertiary coursethan those who have just completed (the target group for the CEQ).Scott and Yates (2002) report on the experience with Engineering graduates as one of theparallel series being undertaken in various professions across Australia and New Zealand.The study sought to identify:• the capabilities which are seen to be most important for successful professional practicein Engineering during the first few years after graduation• the extent to which the universities at which the participating graduates had studiedfocused on these capabilities• key ways of improving the content, delivery, support and assessment of the undergraduateprograms in Engineering in the light of the study findings.Respondents noted that learning profession-specific content provides the ‘scaffold’ for theimportant task of career-long professional learning: the skills to undertake this are of greatimportance, with the ability to know when and when not to deploy technical expertise, andhow to continuously update it, the keys to successful professional practice. The supervisors inthe study acknowledged that a high level of technical expertise is necessary but not sufficientfor successful practice, giving emphasis to the individual’s ability to diagnose what is reallycausing a problem and to testing solutions in action.Scott and Wilson (2002) report an exploratory study of IT graduates that confirmed thesefindings. What is also important in the results of this study is the interrelationship betweencapabilities identified by respondents. While the successful professional must possess a highlevel of profession-specific technical expertise, such skills have little value without the abilityto handle uncertainty, deploy appropriate components of one’s repertoire of generic andprofession-specific expertise, accurately diagnose the unexpected and work productively withpeople from a wide variety of backgrounds:...when the unexpected occurs, what is most telling is being able to tolerate theuncertainty and ambiguity of the situation, having well developed reciprocal networksupon which to call to identify potentially relevant solutions, being able to‘read’ the total technical and social components of a troubling situation, and then31

eing able to apply a high level of appropriate technical skill in partnership withother team members to resolve the situation(Scott and Wilson, 2002, p 6)2.1.4 Practitioner views of REFewer studies have examined practitioner views of the relevance of education in RE in thesame way.Green (1989), surveyed over 800 systems analysts and users to identify which skills, job roles,and ‘nonsalary incentives’ each group perceived to be most important for systems analysts.Curiously, he found that users tended to focus on the importance of systems analysts’ technicalskills and the technical roles they performed, whereas the analysts themselves focusedmuch more on importance of interpersonal skills and their nontechnical roles (eg diplomat,change agent, communicator and salesman). These findings may be seen to support Nikulaet al (2000)’s conclusion that general knowledge of RE in industry may be seen to be ‘quiteweak’.Leitheiser (1992) discusses the industry-education gap from a management perspective, and inthe context of a projected growth rate in demand for systems analysts in the US (1990-2000)of 87%. He quotes several early studies (eg Henry et al (1974), Benbasat et al (1980), as wellas Green (1989)) in which participants rated ‘people’ skills above technical in importance forsystems analysts. Another study (Watson et al, 1990) ranked analysis/design skills as mostimportant for MIS (Management Information Systems) graduates entering the workforce,with interpersonal skills second in importance. Of interest was Benbasat et al (1980)’s findingsthat organisational characteristics affect skills requirements, with organisational maturityrelated to perceived importance of generalist skills in systems analysts. Leitheiser (1992)’sown study rates interpersonal skills as of prime importance over the decade under study.Todd et al (1995) evaluated newspaper job advertisements over a 20-year period (1970-1990)at five-year intervals. They examined the trends in job skills demanded for three job types:programmers, systems analysts, and IT managers, and noted the most dramatic changes inthe skills required for systems analysts, compared to only minor changes for programmers andIT managers. In findings that differed from other previous studies, they found that, by 1990,the systems analyst job advertisements demanded nearly as many technical skills as those forprogrammers, while also continuing to emphasise non-technical skills. They concluded thattechnical skills were more important than non-technical skills (such as business, interpersonal,and communication skills) for systems analysts, and that systems analysts would face a toughset of job demands in the future, given the apparent trend toward greater demand for technical32

skills, accompanied by ongoing demand for the non-technical skills noted above.More personal studies of RE practitioners may provide some insight into the conflicting resultsof Todd et al (1995)’s work with advertisements. Macauley and Mylopoulos (1995a)undertook a brief study of practitioner REs during 1995. Their purpose was to elicit industryrequirements of formal RE education. As will be confirmed by the discussion of BoKs andmodel curricula later in this chapter, there were no surprises in the list of technical knowledgeexpected of a graduate. This included CASE tools, modelling (process and O-O), interviewing,workshopping, analysis (information, structured, viewpoint), prototyping, requirementstraceability tools, appreciation of configuration management with respect to requirements,formal specification languages, HCI issues as well as the need for knowledge of a range oftechniques and the ‘pros and cons’ of their use.Personal qualities appropriate for an RE include:make himself or herself understood, listen, stay calm and assured under fire,quickly assimilate information, talent for sorting and analysing information, writeclear, well structured documents, make presentations, chair meetings, run a group.Also patience, perseverance, be able to live comfortably in a constant state of ambiguity,both independence and team working skills, negotiation skills, flexibility,open-mindedness, sense of humour; good interpersonal skills, analytical, logicaland open-minded.(Macauley and Mylopoulos, 1995a, p 347)However, asked what additional training would be given to a new graduate, a number ofrespondents expressed doubts as to whether this was a job for a new graduate, who wouldtake 12 to 18 months to be able to be ‘effective in the job’ (Macauley and Mylopoulos,1995a). Nevertheless, training would include interpersonal communication skills, shadowingexperienced analysts and involvement in difficult projects.This perspective suggests one interpretation of the job advertisements findings noted above– new graduates would rely on their technical skills until such time as they were ‘effectivein the job’. This interpretation could, perhaps, be linked to the findings of other studies (egDalton et al (1977)) that new graduates are assigned ‘apprenticeship’ type jobs until a levelof ‘organisational’ learning has been achieved.Another interpretation is based on the work of Litecky et al (2004). They suggest technicalskills are used a ‘filtration’ in the hiring process – screening by an employer is used to eliminatesome of the possible candidates from the pool of potential candidates and to pass others onto the second stage. They argue that the information used in the second stage can reasonablybe expected to consist of information on a different set of skills – the second stage is based33

on the perceived soft skills of the candidate, obtained through face-to-face assessments. Thissecond set of skill data has often been stated to be the most important for the final hiring ofthe candidate.Confirming the quote above from Macauley and Mylopoulos (1995a)’s work, Minor (2004)’sinterviewees also placed considerable emphasis on more generic abilities:Personality – confident and faithful, strong work ethic, be proactive or self-started, inquisitivenature, have the ability to ask people questions and accept to appear stupid,perseverance, be teachable and willing to learnInterpersonal skills – communications (inherent and improvable but not necessarily teachable),dealing with people of different background, written language, coherent writingand generally good documentation skillsManagement – of self (personal work organisation) and of large amounts of information.While the latter two items may be incorporated into learning models, and, in fact, coversome of the attributes often noted as exit criteria in tertiary education, Personality may bemore difficult to address. Minor’s interviewees unanimously noted that it was very rare thatnewly hired graduates are involved in requirements activities. He states:the interviewees mention that almost exclusively more senior people do requirementsactivities. Some interviewees argue that experience is necessary. One intervieweeexpects credibility and presence from somebody doing requirements activities.These characteristics are considered to be reserved to people more maturethan most graduates are. In businesses where requirements activities do not involvedirect contact with customers the argument is that people must have insightand knowledge about the existing software, which is described as experience.(Minor, 2004, p 83)This also confirms the results of the Macauley and Mylopoulos (1995a) study, with somerespondents even expressing doubts about RE being a suitable topic for university study.Macauley and Mylopoulos (1995a) conclude that efficient requirements activities require acertain level of knowledge and maturity which can only be gained through experience indealing with practical problems, and acknowledge that a standard university lecture cannotachieve what industry requires, while the work of Lee (specifically Lee (2004)) suggests thereis an underlying ‘socialisation’ requirement for a graduate to achieve ‘working professional’status. Minor (2004)’s study adds personality traits to the mix of characteristics necessaryin competent REs.34

Table 2.10: RE topics learnt in formal educationRank RE Topic Mean % respondents ratingmore than 01 Software Requirements Specification 2.31 802 Group Dynamics 2.13 403 RE Process 2.08 604 Knowledge of Software Packages 2.08 605 Systems Theory and Practice 2.00 506 Formal Methods and Specification techniques2.00 607 Requirements Analysis 1.92 708 Requirements Elicitation 1.70 509 Requirements Validation 1.67 6010 Requirements Management 1.56 50Minor’s work was undertaken as a mechanism to validate the results of studies done elsewhere.After the completion of his study, the participants were asked (by this researcher)to undertake a Lethbridge-style survey, both for triangulation purposes, and to facilitatefurther investigation of practitioner views of RE. The survey instrument and the data acqusition/analysisare described in Appendix B.Applying Lethbridge’s rankings to survey responses it can be seen from Table 2.10 that noRE topic rated highly (the highest individual responses being ‘3’). In line with Lethbridge’sresults, the least learnt formally by the most respondents was Requirements Analysis. Bycomparison, the top ten Other Software Topics rated as indicated in Table 2.11.Table 2.11: Other software topics learnt in formal educationRank Software Topic Mean % respondentsratingmore than01 Simulation 3.50 302 Security and Cryptography 3.16 303 Computational Complexity and Algorithm Analysis 3.00 704 Programming Language Theory 2.90 805 Data Structures and Algorithms 2.88 906 Operating Systems 2.87 807 Object Oriented Concepts & Technology 2.86 708 Specific Programming Languages 2.84 909 Data Transmission and Network 2.71 7010 Systems Programming 2.66 6011 Computer Graphics 2.64 7012 Databases 2.62 8035

The first two items indicate a bi-polar distribution: a few respondents learnt a lot about thesetopics, and can be excluded from our consideration. An additional five topics in this list wouldrank higher than the first RE topic (which would therefore rank 18th) in a composite list.Therefore, we can come to the conclusion, albeit indirectly, that there is a flaw in the educationof REs. Despite acknowledgement that it is a ‘surprise’ to graduates that requirementsis a major cause for software deficiencies (Conn, 2002), it has been well documented thatproblems with software development projects include issues relating to both the nature ofRE (eg incompleteness, ambiguity etc (Bell and Thayer, 1998; Johnson, 1994); complexity ofRE and RE process, including contingency issues (Kamsties et al, 1998; Morris et al, 1998;Carroll and Swatman, 1999; Houdek and Pohl, 2000; Hofmann and Lehner, 2001; Zowghiet al, 2001)) and the non-technical skills needed of its practitioners (eg expertise: lack ofskill and training of RE practitioners (Senn, 1978; Lubars et al, 1993; James, 1994; Johnson,1994; Sommerville and Sawyer, 1997; Kamsties et al, 1998; Morris et al, 1998; Nuseibehand Easterbrook, 2000); sensitivity to the environment of the system (Senn, 1978; Lubarset al, 1993; Emam and Madhavji, 1995; Sommerville and Sawyer, 1997; Kamsties et al, 1998;Morris et al, 1998; Nuseibeh and Easterbrook, 2000), and communications (Al-Rawas andEasterbrook, 1996; Zowghi et al, 2001)).Studies also show that while RE topics are included in formal education, practitioners do notconsider them well learnt. This supports Leite (2000)’s assertion that formal education onRE is a major challenge for the next decade.Exploring Affective AttributesThe value of the softer, more personal attributes has been explored through several studieswithin our target IT specialisations, some minor, others major.Bentley et al (1999) suggest a developmental process in which personal attributes, whichinfluence intellectual abilities and skills, are applied to the acquisition of knowledge to enablethe development of higher cognitive activities. They note that, at the end of the educationalprocess, students must be able to apply knowledge to new situations and problems. Thisrequires certain generic intellectual abilities and skills, which, they suggest, although highlyvalued by employers of IS graduates, are sometimes given only ‘lip service’ in tertiary educationcurricula. The personal attributes identified as important in the model proposed includeattributes like curiosity, risk taking, personal discipline and persistence, which can influencein important ways the successful application of intellectual skills and abilities to knowledgeto support the higher orders of thinking.36

Scott and Yates (2002) and Scott and Wilson (2002) confirm the value of these attributes.They discuss their findings in relation to a framework that identifies professional capabilityin relation to five dimensions consisting of:Emotional Intelligence – Personal (EI-P) including: being willing to face and learnfrom errors and listen openly to feedback, understanding personal strengths and limitations,being confident to take calculated risks and take on new projects, being ableto remain calm under pressure or when things go wrong, having the ability to deferjudgement and not to jump in too quickly to resolve a problemEmotional Intelligence - Interpersonal (EI-I) including: the ability to empathise withand work productively with people from a wide range of backgrounds, a willingness tolisten to different points of view before coming to a decision, being able to develop anduse networks of colleagues to help solve key workplace problems, understanding howthe different groups that make up the organisation operate and how much influencethey have in different situations, being able to work with senior staff without beingintimidated, being able to give constructive feedback to work colleagues and otherswithout engaging in personal blame, being able to motivate others to achieve greatthings, being able to develop and contribute positively to team-based projects relevantto the work areaIntellectual Capability (IC) including: knowing that there is never a fixed set of stepsfor solving workplace problems or carrying out a project, being able to identify from amass of detail the core issue in any situation, the ability to use previous experience tofigure out what is going on when a current situation takes an unexpected turn, beingable to diagnose what is really causing a problemProfession-specific skills and knowledge (Prof) including: having a high level of currenttechnical expertise relevant to the work area, understanding the role of risk managementand litigation in current professional work, understanding how organisationsoperateGeneric Skills and Knowledge (Gen) including: being able to use IT effectively to communicateand perform key work functions, being able to manage ongoing professionallearning and development, an ability to chair and participate constructively in meetings,being able to make effective presentations to clients, knowing how to manageprojects to successful implementation, an ability to help others learn in the workplace,being able to organise work and manage time effectively37

Table 2.12: Scott & Yates: top Engineering capabilitiesAttributeCapability ScaleBeing able to develop and contribute positively to team - E I – Ibased projectsBeing able to organise my work and manage time effectively GenBeing able to set and justify prioritiesI CA willingness to persevere when things are not working out E I – Pas anticipatedBeing willing to take responsibility for projects, including E I – Phow they turn outWanting to produce as good a job as possibleE I – PBeing able to develop and use networks of colleagues to help E I – Ime solve key workplace problems operateHaving a sense of humour and being able to keep work in E I – PperspectiveKnowing that there is never a fixed set of steps for solving I Cworkplace problems or carrying out a projectThe ability to use previous experience to figure out what is I Cgoing on when a current situation takes an unexpected turnBeing able to identify from a mass of detail the core issue in I Cany situationBeing able to use IT effectively to communicate and perform Genkey work functionsBeing able to diagnose what is really causing a problem and I Cthen to test this out in actionThe ability to empathise with and work productively with E I – Ipeople from a wide range of backgrounds relevant to my workareaHaving a high level of current technical expertise relevant to Profmy work areaAn ability to recognise patterns in a complex situation I CBeing able to make effective presentations to clients GenA willingness to listen to different points of view before comingto a decision in current professional workE I – IAn ability to chair and participate constructively in meetings Genand anEducational Quality (EQ) scale, which addresses issues of appropriateness, authenticityof tasks and assessment.As indicated in Table 2.12 the results of their study show that Emotional Intelligence rankshighest in importance, dominating the factors identified by graduates as important to theirprofessional careers, closely followed by Intellectual Capability, addressing generic issues suchas abstraction and contingency, while profession-specific knowledge ranks relatively low. Theability to work in teams, particularly crossdisciplinary teams that are common in the ITworkplace, is also considered vital.38

Figure 2.2: Professional Capability Framework (Scott and Wilson, 2002)In Scott and Wilson (2002) the Professional Capability Framework is refined: EmotionalIntelligence (personal and interpersonal (now social)) becomes Stance and Intellectual Capabilityis now defined by two components, Way of Thinking (incorporating cognitive intelligenceand creativity) and Diagnostic Maps (developed through reflection on experience).The framework is illustrated by Figure 2.2. Respondents were asked to rate items from thecapability scales on their importance for successful performance in their current professionalwork and then to rate the extent to which the University they attended focused on them.Table 2.13 shows the top five capabilities. Again, Emotional Intelligence (Stance) rates highly,with Intellectual Capability a close second.The ramifications of these attributes, specifically for RE, are addressed in an early study(Kozar, 1989), which surveyed the participants of a course on requirement analysis methods39

Table 2.13: Scott & Wilson: top IS capabilitiesAttributeBeing willing to take responsibility for projects, includinghow they turn outBeing willing to face and learn from my errors and listenopenly to feedbackBeing able to develop and contribute positively to teambasedprojectsBeing able to diagnose what is really causing a problemand then test this out in actionThe ability to use previous experience to figure out whatis going on when a current situation takes an unexpectedturnCapability ScalePersonal AbilitiesPersonal AbilitiesInterpersonal AbilitiesIntellectual AbilitiesIntellectual Abilitiesthree months after the classes. It found that the training itself had little influence on adoptionof methods, but personal characteristics and organisational factors including managementsupport seemed to be of more importance. In addition, Gardner (1983, p 62)’s claima human intellectual competence must entail a set of skills of problem solving –enabling the individual to resolve genuine problems or difficulties that he or sheencounters and, when appropriate, to create an effective product – and must alsoentail the potential for finding or creating problems – thereby laying the groundworkfor the acquisition of new knowledgehas bearing for the discipline of RE.2.1.5 Summarising the RE practitioner perspectiveThe practitioner perspective may be summarised by this excerpt from RE-online (a discussionforum hosted at University of Technology Sydney) (Zowghi, 2004) by Firesmith, currentlywith the Software Engineering Institute (SEI) [my italics]30 May 2004 ...in factextremely few software engineers are qualified to perform requirements engineering.Almost everywhere I go in industry, whereas software engineershave significant training in programming languages and designmethods,they typically have no training (or very little training) in requirements engineering,40

architecting, and testing beyond unit testing, and even that istypically quite weak.They are rarely given training in requirements engineering because most of the requirementsthey see are unanalyzed but merely poorly specified requirements written in theirnative languages.And since software engineers are obviously literate (even if feware good technical writers), managers and many software enginersfeel that they are therefore automatically qualified to‘‘write’’ requirements. To them,requirements elicitation, requirements analysis, and requirements management are notcritically important or can be done informally and intuitively.Professional requirements engineers and academics who specializein requirements engineering know better, but are vastly outnumbered bythe rank and file who think they understand requirements engineering because they canread textual requirements and have only been associated with RE that has been performedinformally(by this I do not mean not using a formal requirementsspecification language such as Object-Z, state charts, ordecision trees, but rather in the common non-technical meaningof being performed in a casual, non-technical way withoutprocess, method, or techniques).So if you want to discover a major reason why bad requirementsare such a major cause of project failure, you need look nofurther than the common observation thatthe vast majority of requirements are not engineered and not developed by anyone whohas learned any significant amount of RE, and the reason for that is that most peopleon the project do not see any need for RE because writing requirements to them is justfinding out what the application should do and writing it down.41

If marketing can tell you what they want and you can write downwhat they say (or better yet, have them write down what theywant), the you’re done.And the situation will never get better, no matter how many RE books and articleswe write and now matter how many classes are taught until a significant part of theindustry realizes that there is more to RE than that.And even when a RE technique like use case modeling becomespopular enough to get on the radar screen of many developers andtheir management, there is little progress because there islittle consistency in practice (or even in the literature) as towhat the technique really is or how it should be used. Progressis very slow when everyone is so busy on real projects doingtheir own job that they have little time, energy, or incentiveto even keep up with their own discipline, and even less tolearn another discipline such as RE. Besides, to the softwareengineer and manager, RE is not as exciting, interesting, orprestegious as their own fields. RE is a little like qualityengineering, someone has got to do it, but few want to do it.Also,RE requires a difficult mixture of social and technical skills and abilities that few have.So, is it any wonder that RE in practice is in such terrible shape.It is not that we need major academic advances in RE that won’t be put into practiceanyway, at least not in the future. What we really need is to put into practice whatwe’ve already known for years and years. And that is a social, psychological, political,financial, business, and educational problem, not a technical one. But the technicalproblems are the most interesting and fun problems to solve, so that is where most ofthe effort has and will be placed. And we will all be complaining about the terrible stateof requirements engineering in practice ten years from now.So what can we do? What we’ve always done. We have an ethical42

and moral duty to try to fix the biggest problems on projectsand they lie with requirements. We will write our books andarticles, teach our classes and train our fellow developers whenwe get the chance, trade advice on discussion groups like these,write master’s and doctorial theses, and do our very best toimprove the situation. Just don’t expect to see majorimprovements in the field because until most software engineersand their managers (and university professors) understand theroots of the problem and join us REs in solving it, we aredoomed to only very limited success. But until that happens,those of us who understand the value of RE will continue ourefforts, and be grateful for any little successes that we andothers like us have. We may be ‘‘lone wolves howling in thewilderness’’ but it is in our nature to howl. ;-)Donald Firesmith (USA)2.2 How RE is viewedOur profession suffers under an enormous burden of myths and half-truths(Weinberg, 1971, p vii)In the late 1960s those involved in the development of software agreed that one mechanismfor dealing with the intrinsic difficulties (eg complexity, visibility, and changeability (Brooks,1986)) of software was to embed its production within an applied science environment. Royce(1970) was the first to note explicitly that an engineering approach was required. The implicationof this alignment was that, like other engineering endeavours, methods, tools andprocedures must be applied in a systematic way to contribute to the overall purpose of theprocess, control it and enable the development of a quality product.This alignment with science and engineering was seen as a means to leverage from the ‘status’of these domains: the profession of scientifically trained engineer came into existence in the18th and 19th centuries as a product of the Enlightenment. As the Enlightenment impliedrearranging political and administrative structures in a rationalist way in order to abandonsuperstition and injustice, for engineers it meant rethinking traditional technologies in orderto rationalise and optimise them. In various debates, engineers generally took the viewthat their rational scientific methods were the best means to solve a problem, an approach43

to technical problems that resembled the rationalist approach to socio-economic problemspropagated by the labour movement. However, Mulder (2006) notes that engineers sometimesfailed to recognise that the issue at stake was not always a scientifically/mathematicallysolvable optimisation problem, but a choice between irreconcilable norms and values.By the late 1960s philosophers such as Habermas (1972) criticised the ideological characterof science-based technology – successful technologies were seen to challenge society and affectit as a whole, and a deep understanding of the motives and desires of people that would berelating to the new technology, developed through interaction, was critical. However, thebasic features of most engineering training programmes have hardly been challenged sinceEngineering Schools were established (Mulder, 2006). In general this education is based on anormative professional education curriculum, in which students first study basic science, thenthe relevant applied science (Waks, 2001), so that learning may be viewed as a progression toexpertise through task analysis, strategy selection, try-out and repetition (Winn and Snyder,1996).As Waks (2001) explains, in this normative model of professional education science provides arational foundation for practice [original emphasis], with practical work at the last stage ofthe curriculum, where students are expected to apply science learned earlier in the curriculumto real-life problems.Other approaches to educating software developers also modelled scientific and engineeringmethodologies, with their focus on process and repeatability. Benson (2003) notes that,within the emerging IS discipline of the 1970s, while the research and publications areasowed much to the social sciences, its practice relied heavily on scientific, mathematic andengineering disciplines, with many of the practitioners migrating from engineering and manufacturing.IS academics were also migrants to the discipline, with an overwhelming majorityhaving qualifications in other disciplines, most often computer science. He suggests that currentIS curricula in practice (as opposed to emerging model curricula) continue to show aheavy dependency on the thinking of the 1980s and 1990s.Later sections in this chapter will examine how RE-related components of BoKs and modelcurricula are defined in various IT specialisations (in particular SE, IS and CS). How theseare made available to the learner may be determined from introductory, tertiary studies leveltexts. Iivari (1991) and Checkland and Holwell (1998) suggest these are an embodiment ofthe current wisdom in a field or discipline, its common conceptualisations, and provide bothan account of the field in a straightforward way and a condensed presentation of the bodyof accepted theory. These are discussed in the next section. Although there are importantassumptions made in this approach (discussed in Chapter 3), for simplicity’s sake it is the44

approach taken here.2.2.1 A positivist perspectiveThe complexity of the systems development process has led to multiple approaches to its definitionand study: based on the work of Iivari (1991) (who from a focus on IS describes sevenmajor schools of thought in systems development), it is possible to identify and categorisedifferent approaches, based on the positions taken in the dimensions listed below. These areadapted from the work of Hannafin (1997a) and Reeves (1994) who suggest these are relevantin the description of learning systems (see Chapter 4):• epistemological foundations – the theory of knowledge or reality held describes theworld view to be disseminated. This is described as a continuum from objectivism toconstructivism• psychological foundations – represent beliefs about how individuals think and learn.This can also be described as a continuum from behaviourism to approaches emphasison mental models and the connections between them (eg constructivism, situatedcognition)• philosophical foundations – an instructivist foundation stresses the importance ofgoals and objectives drawn from the domain. Alternatively, the primacy of experienceand metacognitive strategies is stressed to provide contextualised outcomes.At a fundamental level, the base assumptions made on, for example, the nature of the systemor the importance of its context, as well as the dimensions noted above, have enormousinfluence. The differing views produce differing bodies of knowledge, and different approachesto undertaking the activities which comprise the discipline. Traditionally these concernshave led to three broad classes of approaches – hard (objective, positivist, instructivist), soft(subjective, interpretivist, constructivist) and hard/soft (exemplified by the socio-technicalapproach). Each of these embraces one or more schools of thought (and/or IT specialisations)and may be considered a continuum, with exponents positioned on it depending on the flavourof their approach within the class.Within the broad IT specialisations some schools of thought may be categorised as nonpositivist1 , and a very few in the gap between the two, in general most would cluster around1 as opposed to post-positivist, a metatheoretical stance following positivism in philosophy, as well as usedto refer to a group within political theory in the social sciences who do not believe it is possible to view lifefrom an objective point of view45

positivism, situated along the continuum based on the ‘hardness’ or ‘softness’ of the ontologicalassumptions made. The underlying assumption is that the world works rationallyand that therefore ‘good’ software development is achieved by applying (from a choice of)scientific investigative techniques. In this positivist approach, borrowing from the physicalsciences, software developers build models based on• theoretical and scientific knowledge• engineering knowledge – experiential and including what skills are needed, how toolswork together, what has/has not worked in the past• biomedical and epidemiological knowledge – experiential, this captures evidence aboutcausation• social, economic and institutional knowledge – who and what are involved in what weare observing(Pfleeger, 1999).By these means the ‘scientific’ software developer seek relationships that add to an understandingof what makes software good. These are applied to increase the number of timesgood software is produced, based on a cause-effect search: if s/he can find out what processactivities, tools, measurements cause good software s/he can build an effective software processthat will produce good software every time (Pfleeger, 1999). Within this perspective,Requirements Engineering is a type of problem-solving: reasoning in order to synthesise asolution for a system to satisfy the users’ needs (Loucopoulos and Karakostas, 1995).Also applied within IS education (Banks, 2003), such approaches lean towards projectmanagement-basedmethods, techniques and tools, and, while successful in creating a rangeof artefacts, do not succeed in the development of management information systems. Banksconcludes that the weakness inherent in approaches which lend themselves to ‘cookbooks’with clearly defined problems, rigid method and limited range of outcomes but tangible skillsin students is the lesser regard for real-world influences and pressures.In general, Software Engineering texts view the software development lifecycle from thesame perspective although a few (eg Floyd et al (1992)) are seen to accept the notion ofthe developer’s active role in constituting what is real – in acknowledging the richness ofexperience in the process. Within the SE literature, Ghezzi et al (1991); Jackson (1995)(as examples) assert that Software Engineering (and by default, Requirements Engineering)must be practiced as an engineering discipline:46

to develop software is to build a MACHINE, simply by describing it. ... Theproblem is in the application domain. The machine is the solution.(Jackson, 1995, p 1)Application of the (scientific) principles of separation of concerns, abstraction and modularisationis essential, as they enable the Software Engineer to evaluate notations and methodologies,etc and apply as appropriate (Ghezzi et al, 1991).2.2.2 The non-positivist perspectiveWithin the IS specialisation, the dominant underlying influence is that of Simon. His view ofmanagement is that its nature is that of problem solving through decision making, and thathuman behaviour, individual and corporate is goal seeking. The decision maker, with the aimof making “good enough” decisions, moves towards a goal which may be adjusted as a resultof such decisions. This satisficing is part of problem solving (Simon, 1960). Objective causeeffectrelationships can therefore be partially discovered by structured observation (Walsham,1993).Simon’s approach dominates the ‘hard’ view of non-positivist systems development and is seento shape the functional approach. The socio-technical school (as exemplified by Mumford(1983)’s ETHICS approach) argues that a technological system cannot be separated fromthe social system in which it is based. Potential users therefore become the designers of thesystem for their use.As an important example of an non-positivist approach, Checkland sees a system as a set ofactivities linked together so that the whole, as an entity, can pursue a purpose. Building andnaming models of such systems requires a declaration of the worldview (Weltanschauung)which makes a particular model meaningful – the set of values, outlook – a given-as-takenimage of the world. They are not seen as descriptions of reality, but rather concepts relevantto the exploration of what is perceived as reality. The models are used to both give structureto debate about the problem situation and to learn a way to possible changes which aredesirable and feasible. This requires accommodation between conflicting interests.Checkland asserts that, in the area of system development, what is difficult is that thereis no taken-as-given structure-with-content: all aspects are problematic, with a need firstto establish a Weltanschauung within practitioners and researchers, who expound differingviews of its structure, reference disciplines and perceptions of its core concerns. Hence systemdevelopment is a learning system, a learning cycle that is ideally never-ending (Checkland47

and Scholes, 1990; Bulow, 1989). These views place Checkland somewhat centrally betweenthe hard and soft poles of this continuum.Alternative views focus on the ‘soft’ orientation, and acknowledge the social constructionof organisations. Vickers posits that goals are replaced by relationship management. Theprevious history of a system, and its interactions with its environment, generate the possiblecourses of future actions, tempered by criteria by which they are judged (Vickers, 1984).Vickers’ work is placed in the interpretive tradition, which sees social action as based onpersonal and collective sense making (Checkland and Holwell, 1998) by a group engagedin dialogue. This view underlies other significant work, in which organisations are seen asnetworks of conversations (Winograd and Flores, 1986), based on Pask (1976)’s concept thatlanguage does not reflect the ‘world out there’ but constitutes it as ‘public knowledge’ in thesocial process of interaction.A non-positivist systems approach therefore explores the way in which people in organisationsattribute meaning to their world (Checkland and Holwell, 1998). The use of language in theanalyst/stakeholder dialogue does not reflect any participant’s view, but helps constitute it(Boland, 1985): different worlds are created for the system under consideration.2.2.3 Positivism and non-positivism in REThe dominant views as to the nature of Requirements Engineering in systems developmentalso span across the dimensions positivist – non-positivist and hard – soft.Within the positivist software development literature it is suggested that the purpose of theRE process is to accurately acquire and ‘represent’ the problem solution in a form appropriatefor development and implementation. Philosophically this may be seen to equate to theobjectivist view of knowledge. Just as knowledge exists independent of and external tothe individual – it is a fixed commodity with attributes, relationships and structure whichcan be known objectively – so too does a solution to an RE problem exist, independent ofthe socio-organisational context within which it is embedded. The problem becomes one ofcommunication – the fundamental goal of the objectivist paradigm is accurate transmissionand reception of knowledge (Miller and Miller, 1999). Implied within this epistemologicalstance is an acceptance that requirements exist:• hard – the software engineer creates models of physical situations in software (Fairley,1985); requirements describe the relationships among the phenomena of the problemcontext (Jackson, 1995). Pohl (1994) suggests that opaque personal views on the systemexist in the people involved in the Requirements Engineering process, and that these48

may have some common aspects• soft – Bach (1999) states that the requirements, as the reasons why we decide to buildwhat we build (the drivers of design choices), exist inside our minds. Hence the full setof requirements exists in the shared mind space among all the product stakeholders,with the requirements document a model of the information in that mind space. Thisargues for multiple viewpoints from multiple stakeholders which must be accommodated,enabling the construction of a complete set of requirements.An alternate non-positivist view accepts the subjective nature of the system developmentprocess within a socio-technical context, and further exploits the constructivist nature ofknowledge as a model of RE. Within this stance, software development may be viewed:• hard – a diagnostic (Dewey, 1910) approach to determining aspects of the problem(which suggests there is no ‘typical’ problem) is coupled with a functional approachto problem-solving, exemplified by the work of Simon (1960), where requirements areconstructed as goals through a process of structuring problems• soft – the requirements are truly constructed outside the mindspace of both stakeholdersand Requirements Engineer. Language and other representational schemes in thedialogue help constitute the participants’ views (Boland, 1985). In the same vein, Hocking(1996) argues that in reality, requirements are actually affected by the elicitationprocess and hence are dynamic and fluid. This view highlights the potential importanceof the social and cultural structures (which may be personal or organisational) held bythe Requirements Engineer and the stakeholders, amongst others.The terms used in Requirements Engineering also create perceptions about the nature ofrequirements. Hocking (1996, p 320) suggests terms such as ‘analysis’ and ‘elicitation ofinformation’ suggest that REinvolves the extraction of information from a static situation in which the usersare passive subjects of the proposed changes.In this positivist view, requirements are developed or derived or defined. (Ghezzi et al, 1991,p 364) note that the purpose of requirements analysis and specification isto identify the qualities required of the application, in terms of functionality, performance,ease of use, portability and so on49

and to explore and describe requirements located in the application domain (Jackson, 1995).This is achieved through application of the principles of separation of concerns, abstractionsand modularisation.At the ‘soft’ end of the positivist continuum (as exemplified by Sawyer and Kotonya (2000);Loucopoulos and Karakostas (1995)), the socio-technical nature of the software developmentprocess is acknowledged. However, the focus remains on a problem that can be ultimatelysolved. Within the ETHICS approach, a requirement is an outcome of the interaction betweenusers and Requirements Engineer. The Requirements Engineer, (cast here in the roleof therapist) inaugurates a learning system where different views and perspectives may beintegrated (Bloomfield, 1992). Elicitation cannot, therefore, be practiced at the level of theindividual users, in isolation from their environment and interactions. As a consequence,requirements have meaning only within the context (time, place, situation) in which theyare observed (Loucopoulos and Karakostas, 1995). This concern with participation of userswithin the development process has led to the favouring of techniques from the social sciences:ethnography as a mechanism of capturing social practice is seen to promise higherquality requirements by addressing the the meaning-making activities within the ‘society’under study (Sommerville et al, 1993; Hughes et al, 1995; Ford and Wood, 1996).While RE as a smoothly evolutionary process is the accepted view within the positivist softwaredevelopment literature, Maiden and Gizikis (2001) and Nguyen and Swatman (2000a)amongst others, argue it should be regarded as flawed in that it is based on fundamentallywrong ideas regarding the successful development of software. Unlike the engineering andmanufacturing metaphors used to drive the established view, software development is dominatedby human cognition. Software is a collaborative invention: software development is anexploratory and self-correcting dialogue (Bach, 1999).This alternate perspective suggests that Requirements Engineers are not given problems,rather they construct them (Visser, 1992) through negotiation and interaction (Hocking,1996). This construction is thought to be insight-driven and fundamentally opportunistic(Guindon, 1989; Carroll and Swatman, 1999) and based on a spiral model rather than onethat is hierarchical (which assumes a process of decomposition to a set of sub-problems whichmay be solved (Simon, 1973, 1981; Jeffries et al, 1981)), evolutionary (incrementally addingto the knowledge of the problem and evolving towards a solution (Malhotra et al, 1980))or even parallel (by which separation and then partial description of aspects and parts ofthe problem is undertaken, gradually narrowing the ‘world’ described until the final resultdescribes exactly the world of interest (Jackson, 1995)).From this perspective, the process of RE is seen as one of knowledge discovery facilitated by50

opportunistic behaviour (Guindon, 1990; Visser, 1992).Figure 2.3: Opportunism in requirements discovery (Guindon, 1990)As we see in Figure 2.3, the Requirements Engineer builds fragments of understanding ofthe problem (chunks) validated and consolidated through the traversal of layers, collectingmore areas and information at each, adding detail and richness to the mental model of theproblem situation (Batra and Davis, 1992).The creativity of this process (Lubars et al, 1993; Maiden and Sutcliffe, 1992; Maiden andGizikis, 2001; Thomas et al, 2002) is hampered by strict adherence to engineering and sciencemethodologies. These:• restrict the essential characteristics of the process (such as opportunism)• assist in adding accidental complexity through their attempts to control the RE’s professionalpractice. Sutcliffe and Maiden (1992) suggest strict adherence to methodprocedures may restrict natural problem-solving• impose a plan at odds to the RE’s cognitive planning mechanisms and hence interferingwith the management of knowledge. Visser (1990) suggests in practice, a plan is followedonly as it is cognitively cost-effective, and• inhibits the necessary creative thinking required by superimposing a goal too early inthe process. Boden (1997) makes a case for the claim that changing the goals andadding constraints during the design process might be at the core of creative thinking.As an extension to this non-scientific perspective, work initially from Deakin University (for51

Figure 2.4: The catastrophe-cycle RE process(Nguyen and Swatman, 2000a)example, Carroll and Swatman (1999); Nguyen and Swatman (2000b)) proposes a catastrophecyclemodel of RE (see Figure 2.4) whereby each cycle consists of structuring the problemspace; developing a mature understanding; building complexity and then simplifying thisthrough insight or critical thought. This behaviour has been observed in other domainswhere ill-structure and/or creative work features strongly, (examples include design (Lawson,1997; Suwa et al, 2000) and architecture (Hadamard, 1945)).Increased understanding of the problem (the essential complexity) leads to simplification ofthe model and hence reduction of the incidental complexity (Nguyen and Swatman, 2000b)that is the result of the poor fit between the structure of the model and the structure of theworld the model is representing.More recently this model has been revised, to acknowledge a plateau state of mature understandingbefore the next cycle of complexity-building. Raisey et al (2006) suggest this periodof stability occurs when the current complexity measure is lower than or approximately equalto the point that preceded the peak (see Figure 2.5).The results of this knowledge discovery leads, not to a problem-solution (which assumes aproblem can be defined), but to an ‘evolved fit’ between all stakeholders within the problemspace. This argument supports Carroll and Swatman (1997), who suggest that the term‘developing requirements’ acknowledges that there is no definitive set of requirements to bedetermined, discovered or uncovered.52

Figure 2.5: The revised catastrophe-cycle process (as described in Raisey et al (2006))As noted above, an indication of the view adhered to can be provided by the terms usedwithin the discipline. As another example, requirements elicitation implies requirements are‘out there’ to be elicited. Classic requirements analysis techniques are seen as an attemptto isolate requirements from their context (Loucopoulos and Karakostas, 1995) and focus onthe automated system itself, rather than the people and processes that system serves.The distance requirements are seen to have from the user is reinforced by the requirementselicitation techniques advocated. As an example, forms analysis does not regard the useras the prime source of knowledge about the problem domain. Rather the form, widelyused in organisational communications, is seen as the most promising source of knowledgein that it is formal, based on data, easily acquired, often accompanied by instructions andeasily automated. In a similar vein, textually based natural language as a source of problemknowledge has a vocabulary (the language of the domain and the stakeholders) to helpestablish organisational meaning; syntax that is understood and informality as a techniquefor dealing with complexity (Loucopoulos and Karakostas, 1995). However, natural languagerequirements capture is based on identifying constructs to map to requirements modellingformalisms through the application of rules and heuristics. The application of heuristics atleast acknowledges the importance of the experience of the Requirements Engineer in theprocess. Verbally based natural language, on the other hand, does imply interaction with theuser. As an example the ‘story telling’ of scenario-based RE focusses on the creation of social53

meaning and a shared sense of participation (Crowley, 1982). Scenario techniques also addressthe issue of incorporating tacit knowledge and expertise and organisational experience.For the ‘softest’ of the positivists, Loucopoulos and Karakostas (1995) Requirements Engineeringis aboutestablishing the “connection” between the need for some change within an organisationalframework and the technology that could bring about that change. Inother words, requirements engineering can be considered as a way of managingchange(Loucopoulos and Karakostas, 1995, p 5).While Loucopoulos and Karakostas (1995)’s definition of RE reflects the view that requirementsspecification involves the interplay of concerns between representation, social andcognitive aspects (Pohl, 1994), and is itself a reflective process (Loucopoulos and Karakostas,1995) it is still firmly rooted in a positivist tradition. The skills needed to address theseconcerns emanate from disciplines other than computer science and engineering, but are tobe applied within those disciplines.The positivist conceptualisation of the development process is also exemplified by the processmodels adopted. These, in the main, assume that the requirements of the system are alreadyestablished, and hence only need to be collated and documented, as noted above (see Downeset al (1988)) for a critique of the SSADM). Newer models, (eg the Win-Win Spiral model(Boehm et al, 1995)) accept the need for accommodation, but view the organisation asachieving predetermined goals via rational decision making. Goals are expressed as specifictargets to be achieved at a particular time, with lower level goals deriving from those at higherlevel. Organisational performance requirements relate to ‘critical success factors’ – thosewhich have to go right if defined business goals are to be achieved. While this teleologicalview acknowledges the need to start from organisational processes, any deep exploration ofmeaning and purpose in context is not allowed for. Contrast this with the non-positivistsystems approach briefly described above.The views of Iivari (1991), that an orthodoxy based on a positivism epistemology may beidentified within the majority of software development schools of thought is supported inthe textbook literature. This perspective has major influence, not only on the ‘world’ inwhich the system is to be developed, but also on the underlying knowledge structures, skills(physical and cognitive) and techniques the Requirements Engineer has recourse to in orderto achieve a successful RE task.54

2.2.4 The non-alignment of theory to practiceFrom the mid-1970s, Argyris and Schön (1974) worked on the concept of reasoning processes.They assert that whilst people describe the process by which they take action, this is verydifferent from how the action was planned, implemented and reviewed. This is not merely thedifference between saying and doing: the theory of action takes the form of espoused theoryand theory-in-use. The ‘espoused theories’ are officially accredited or agreed ways of reactingto certain situations as opposed to ‘theories in use’ which denote the rules and hypothesiswhich are actually applied, inferred from behaviour. The theories in use may or may not becompatible with espoused theories; furthermore, the individual using these theories may ormay not be aware of the incompatibility of the two (Argyris and Schön, 1978). Hence, whatis intended to be done (espoused) may be very different from what is done (in use), governedby inherent beliefs and feelings.A recent discussion in RE-online (Zowghi, 2004) provides a snapshot of this discrepancybetween stated and practised process, and hints at the educational issues to be tackled inthis thesis. Quotes are excerpted for simplicity – the discussion ranged over several topics,and relevant elements are highlighted. The thread subject is Bottom-up RE methods (Zowghi,2004).26 April 2004 I have a feeling that any approach that gathers stakeholdergoals, viewpoints and stories in a people-centred way is goingto be mixed: perhaps only purely analyst-devised (and hence,not surprisingly, analytic) methods will ever be top-down.Actual meetings with stakeholders tend to be chaotic from thepoint of view of neat and tidy top-down analysis! but quiteorderly and logical from the point of view of people who areexperiencing the pain of a problematic work process.Stories/scenarios/use cases may seem to offer a top-downapproach, but people can offer stories at any level. I suggestthat,rather as mathematicians do with published proofs, engineers tend to try to impose topdownorder on sets of stories, but this is post-hoc explanation rather than a method ofgathering.There is a natural small-scale way in which use case work is topdown: from any story, you ask what can go wrong and elicitexception sub-stories from that. But these mini-trees have to be55

assembled into a model bit by bit. So I see stories asend-to-end interaction sequences; you work both up and down fromthem. I note that I and the other people who’ve replied to yourquestion have written mostly about ‘user requirements’.Specifications are more likely to work in a relatively top-downway simply because it’s safer to do so once the user reqts arereasonably well-known. However I concur with Suzanne Robertson’sview that you have to work ‘top-down, bottom-up, andmiddle-every-which-way’Ian Alexander (UK)27 April 2004 There are at least two very good reasons whysome form of top-down approach is normally seen in successful projects, regardless ofhow the actual process happened in practice.What I was suggesting tends to be something like: workingupwards from middle/bottom fragments (including scenarios), thenworking top-down, then validating (partial) against thefragments.Andrew Gabb (Australia)28 April 2004 It occurs to me that most tools intentionally support top-down,but unintentionally support bottom-up as well, sincemost analysts operate opportunistically.Scott Overmyer (New Zealand)28 April 2004 This reminds me of David Parnas and Paul Clements’s classicpaper ‘A Rational Design Process: How and Why to Fake It’.Although I suspect this wasn’t the intent.Even if the information is gathered bottom-up, it is better presented top-down as thatis the easiest way for us to navigate around.Keith Collyer (USA)56

30 April 2004 And here we have a big problem!People who read our presentations, especially inexperienced ones, believe that this ishow the results have been obtained (top-down). They try to follow the example and fail.Ilia Bider (Sweden)1 May 2004 ... and others use the fact that even experienced REs do *some*bottom-up RE/RA as an excuse for *always* doing bottom-up orbottom-only (the real danger). They also use it as an excuse forbottom-up design, which is something rather different.I’ve seen internationally respected systems engineers at work,including those that give some of the more popular industrialstrength (as opposed to academic) SyE courses.They almost always use a mixture, but some strenuously deny that they do *any* bottomup. Somehow this part of their analysis/design is invisible to them.If you read practical SyE texts, you would be be forgiven for believing that SyE is areductionist approach. In practice, it rarely is, but there is always some reductionism(otherwise you’d never finish the job, obviously). That doesn’t mean the texts or processmodels are necessarily wrong, just incomplete like all models.Andrew Gabb (Australia)The comment by Collier, above, is pertinent. Parnas and Clements (1986) suggest that,given an irrational design process (ie all design processes), the documentation should makeit appear as though it were. This faking of the appearance of rationality is justified throughthe need to make the eventual maintenance task easier, as well as enabling new members ofthe design team to absorb knowledge about the project more easily. However, the processto such simplification (as described by Nguyen and Swatman (2000b)) is hidden and, asnoted by Bider above, leads to unreal expectations in novice undertakings. The danger ofsimplification in a learning environment is discussed in Chapter 3.2.2.5 Addressing wicked RE educationContinuing the discussion in RE-Online (Zowghi, 2004):57

4 May ...I doubt that they’re strongly covered in design literature,which tends to assume that you have requirements before youstart, or that RE is relatively straightforward.I don’t think I’ve seen much on this in RE literature either,although it’s almost certainly covered in cognitive sciencestuff on problem solving and decision making...There are probably a number of reasons why it isn’t commonlydiscussed in the techo stuff (apart from the underwhelming psychcredentials for most REs).One of the most likely is that it’s not about what you *do* but how you *think*. I alsodon’t know how you would teach it....The user talks about something he does or needs and weautomatically see semi-analogieswith other work we’ve done, even though the actual work area maybe totally different. This allows us to understand and‘populate’ the larger domain much faster, and helps withcompleteness. It also allows us to jump to incorrectconclusions, which can remain uncorrected for the life of theproject - I’ve had one or two of these myself....I feel that it’s a very powerful technique (ortalent, because I’ve noticed only some folks can’t do it effectively no matter how hardthey try).Without it, we can take forever to understand the user domain.It also allows us to make useful comments and ask penetratingquestions even when our understanding is far from complete,which means the users are less frustrated with our obviousstupidity. I think this is one reason why REs with *broad*58

experience can offer much more to a project.If you think about it, the dangers are just an extension ofthose that are inherent in developers doing RE, in that they aretoo ready to assume that they understand what the system shoulddo, and leap into design too soon. They obviously get therequirements from *somewhere*. OTOH, good REs tend to beparanoid about their understanding of the users’ needs andactivities, simply because they know how easy it is to guesswrong.As a final aside, slightly related,I think that it’s very important for an RE to be able to have several models (or severalvariants) of the user domain in his/her head at any one time, and usually there areconflicts between and within these mental models.Gradually the models disappear or merge as the elicitationproceeds. If we tried to ensure that we were always working withone perfect model, I don’t think we could workefficiently.Andrew Gabb (Australia)5 May As for now, I do not know any other way of teaching the “art of analysis” than the oneused by craftsmen when an underling just starts by helping his master. I do not expectthat the universities could produce ready-made system/business analysts or requirementengineers. What I do expect, though, is that the graduates in these disciplines know“what and how much they do not know”, and “dangers” should be counted to this issue.From my practice, this is not the case, the graduates tend to think they know quiteenough to do analysis, requirements engineering, etc. right away....As far as ‘dangerous’ is concerned, the term has been used ina very special meaning,namely:‘when used unconsciously or/and by an unexperienced or unqualifiedanalyst’.Ilia Bider (Sweden)59

The characteristics noted above pose problems for Requirements Engineering, summarised as‘wicked’ (Bubenko, 1995). Based on the nature of RE problems, the view is taken that manyof the software practices and tools are developed as attempts to deal with the conceptualcomplexity of the tasks undertaken. Practitioners, and students, are seen to need conceptualknowledge in several overlapping domains in order to perform Requirements Engineeringtasks successfully.The physical domain encountered by the RE is modelled, and then manipulated by meansof this model. The implication of this process is that RE may be classed as a second orderdomain – practitioners work with concepts that are abstracted. Such domains require greatemphasis on cognitive skills. Research has shown that some learning is detrimental to theability to deal with characteristics of second order domains: Lee and Truex (2000) show thatlearning that simplifies the cognitive structures of the novice (for example the learning ofdevelopment methods) is detrimental in enabling the learner to deal with complex conditionsand a high level of uncertainty. Where the domain is difficult to grasp, frequently learners intraditional educational settings are found to underachieve (Patel et al, 2000).Therefore characteristics of RE suggest that student REs require enhanced understanding oflearning processes, including reflection and critical thinking in order to model the behaviourof practitioners.Hence the problem faced by RE education can be summarised as one of grappling with a‘wicked’ problem as well, where:• complexity is augmented rather than reduced with increased understanding of the initialproblem• metacognitive strategies are fundamental to the process• problem-solving needs a rich background of knowledge and intuition to operate effectively• a breadth of experience is necessary so that similarities and differences with past strategiesare used to deal with new situations.The nature of RE problems supports the view that a great deal of specialised knowledge isbrought to structure and tame the complexity of tasks undertaken (without overly simplifyingthem), while creativity and judgement are critical for the solution of wicked problems. Thus,although the work of Argyris and Schön cited earlier is directed at organisational change, thefocus of the approach, and the double loop learning theory that aims at bringing espousedtheory and theory-in-use into congruence, has value in a discipline that deals with complex,60

ill-structured problems. Solving problems that are complex and ill-structured and whichchange as problem-solving advances requires skills and strategies may not be addressed bytraditional learning models.2.3 What mastery of which knowledge?As Firesmith asserts, and other practitioners confirm, the problem with RE education is notthe lack of technical knowledge. This section examines the approaches taken by the relevantBodies of Knowledge and model curricula to describe the components of RE so that theymay be learnt, both in formal education programmes and in the few years after graduationthat the BoKs address.The BoK within a profession acts as a compilation and distillation of the information of adiscipline pertinent to practitioners of that discipline. Achieving consensus by professionalson a core body of knowledge is a key milestone in many disciplines, including that of softwaredevelopment. The BoK may come into being through a formal or industry based process(eg the Project Management Institute (2000)’s Guide to the Project Management Body ofKnowledge is an ANSI Standard (ANSI PMI 99-001-2000), while the information securityprofession (ISC 2 , 2001) has established their BoK), with the aim to identify, aggregate,standardise and maintain a description of the knowledge of the discipline.The purpose of the development of a BoK is many-fold:• licensing and/or certification within the profession may be based on working knowledgeof core or common domains within the BoK (eg CISSP Certification for InformationSecurity professionals (ISC 2 , 2001)• accreditation of curricula both within the formal or profession-based education structuremay be based on the recognition of a core BoK (eg ACS IT Professional level courseaccreditation based on relevant tertiary qualifications (Underwood, 1997) and its ownexaminations which serve as an alternative entry (Australian Computer Society (ACS),2002). The BCS ISEB ‘Diploma’ (BCS Information Systems Examination Board, 2002)also falls in this category)• continuing professional development may be based on increased competency within theareas of the BoK (eg Certified Medical Practice Executives American College of MedicalPractice Executives (ACMPE), 2002)• the structure of training courses and publications may be based on the BoK (eg Instituteof Quality Assurance, 2002)61

• the evolution of a discipline towards professional status may be aided by the developmentof a BoK (eg the Software Engineering BoK (SWEBOK) project (Bourque andDupuis, 2001)).The expectation in each of these is that the BoK is not an inventory of all that must beknown by a practitioner within the discipline, but that the core knowledge will be known toall, probably by the end of some four years of professional practice.As one example of this, and related to curriculum development, Computing Curriculum 2001statesone of our goals in proposing curricular recommendations is to keep the requiredcomponent of the body of knowledge as small as possible. To implement this principle,the CC2001 Task Force has defined a minimal core consisting of those unitsfor which there is a broad consensus that the corresponding material is essentialto anyone obtaining an undergraduate degree in this field.(Engel and Roberts, 2001, p 14)Martin et al (2005) suggests that the implication of this ‘minimalist’ philosophy is that thecompetency sought should be addressed from two perspectives, that of• science – the set of scientific and mathematical tools used to solve (engineering) problems• practice – the recognition and formulation of a problem (not necessarily engineering)and its solution.However, as the latter requires experience and improves over time, it may not be assumedon graduation (Scott and Yates, 2002). The focus is therefore placed purely on technicalaspects of the discipline, and it follows that competency levels are determined by theseaspects. Non-technical competencies (eg communications, team-work, lifelong learning), arealso not an assumption on graduation. Yet, increasingly, professional accreditation placessome emphasis on this aspect within formal education, while practitioner studies indicatethese as keys to practitioner competency, suggesting some BoKs may be out of step with therealities of discipline practice.Levels of competenceCompetence, or mastery may be defined in a variety of ways. When used in a narrow sense,it may refer to a demonstrated capacity to do a specific task by detailed specification of62

the conditions under which performance of the task is to be demonstrated (Mayer, 1993).When used in this sense, it is usually distinguished from knowledge and understanding.Alternatively, and at the other pole in terms of definitions, competence involves both theability to perform a given task in a given context and the capacity to transfer knowledge andskills to new tasks and situations – the skilled application of understanding (Mayer, 1993).Within education competence or mastery are often linked to the concept of learning objectivesor outcomes, and usually are seen as positions on a continuum which spans pre-novice toexpert. Jonassen and Grabowski (1993) describe such a continuum as leading from ignoranceto expertise, encompassing three learning phases:introductory – very little directly transferable prior knowledge about a skill or content areaexists. It represents the initial stages of schema assembly and integrationadvanced – more advanced knowledge is acquired in order to solve more complex domainor context dependent problemsexpertise – results from extensive experience that requires broad transfer of the knowledgeacquired during the previous phases.Dreyfus and Dreyfus (1986) developed a richer skills acquisition model that focusses on adulteducation (as the stage beyond secondary schooling). This model is based on five stages:novice – rules in a context-free task environment determine novice actions. No previousexperience is required for the beginner to recognise these features, but performance ispooradvanced beginner – additional aspects of the problem situation can be recognised andaid the advanced beginner in applying instructional maxims (which, unlike a rule,requires some understanding of the domain in which the maxim applies (Polanyi, 1958)).Decisions are still based mainly on rule application. A goal of this stage is to developmore generalised understandings of the rules and when to apply themcompetence – having a deep enough understanding of the rules to know when they areapplicable and how to apply them in novel situations. Decision making about whichperspective or plan to adopt is applied to deal with overwhelming possibilities within asituation. At this point, incidental complexity may seem overwhelming, but a holisticapproach can be identified in the conscious problem-solving undertakenproficient – pattern recognition arising from extensive experience is used to identify theproblem as intuitive reaction. Decisions still need to be made, based on rules andconscious analysis of alternatives based on experience63

expertise – intuitive situational response is automatic in the expert. Fine grained discriminationbetween situations is based on abstract representations formed through reflectionon experience. Decomposition of the situation into discrete elements is not undertaken(nor required). While the situation remains stable, expertise does not require constantlearning.In later writings, Dreyfus (2001) added a sixth stagemastery – sense that there is no one right thing to do and that improving is always possible.Such continually concerned experts are never complacent. By ‘brooding’ over successesand failures, the expert can progress from responding immediately to specific situationsto responding immediately to the whole meaningful context: reaching a new level ofskillful coping beyond expertise and developing a ‘style’.In relation to education, Dreyfus also suggests that the higher modes of functioning – intuitiveexpertise and mastery – require risk taking (and hence emotional engagement), directexperience and active involvement in the company of experts.It is possible to map the depth of knowledge attained during the various phases expressed aslevels in a taxonomy. The most widely applied and understood is that of Bloom et al (1956).Bloom attempted to classify forms of learning into three domains, cognitive, affective andpsychomotor. The taxonomy of learning objectives addressing the cognitive domain representincreasing levels of cognitive complexity, each level encompassing those below:knowledge - remembering and recalling facts, dates, events, places. Learning objectives atthis level include: know common terms, specific facts, methods and procedures, basicconcepts, principlescomprehension - perceiving and understanding what is learnt, interpretation of informationin one’s own words, grasping meaning. Learning objectives at this level include:understand facts and principles, interpret verbal material, charts and graphs, translatematerial from one form to another (eg verbal to visual/mathematical), estimate futureconsequences implied in data, justify methods and proceduresapplication - using knowledge in a specified manner, application of methods, theories, conceptsto new situations. Learning objectives at this level include: solve mathematicalproblems, construct graphs and charts, demonstrate correct usage of a method or procedureanalysis - separating a concept into its elements, and determining their relationships, identificationof patterns, requiring an understanding of both the content and structural form64

of the material. Learning objectives at this level include: recognise unstated assumptions,logical fallacies in reasoning, distinguish between facts and inferences, evaluaterelevance of data, analyse organisational structure of a worksynthesis - combining elements into something new, generalising from given knowledge,organising and relating knowledge from several areas, predicting, drawing conclusions.Learning outcomes stress creative behaviours with major emphasis on the formulationof new patterns or structureevaluation - exercise judgement about value, compare and discriminate between ideas, evaluatedata. Learning outcomes are the highest in the cognitive hierarchy because theycontain elements of all other categories plus conscious value judgements based on clearlydefined internal or external criteria.(Bloom et al, 1956; Carneson et al, 2000; McKenna and Bull, 1999)Research over the succeeding years has generally confirmed that the first four levels area true hierarchy, with the lowest three levels considered as foundational thinking (Ryanand Frangenheim, 2000), used as the basis for higher learning levels. However, research ismixed on the relationship of the highest two outcomes: they may be reversed or equallydifficult activities. One example of the former is the revision to the taxonomy undertaken byAnderson and Krathwohl (2001). This revision maps a specific verb tag to each level (so that‘Knowledge’ becomes ‘Remember’) and swaps the positions of levels five and six (so that‘Evaluate’ (formally Bloom’s level 6 ‘Evaluation’ precedes ‘Create’ (formally ‘Synthesis’).The argument presented for this is that evaluation is a necessary step which precedes anygenerative process.An alternative approach to Bloom’s Taxonomy proposes ‘Synthesis’ and ‘Evaluation’ as twotypes of thinking with much in common (Bloom’s lower four levels) but undertaken for adifferent purpose. Huitt (1998) summarises these as equating to creative (synthesis) andcritical (evaluation) thinking. This equivalent-but-different view does not subsume creativethinking under critical thinking (as Bloom’s original hierarchy does) but identifies them asseparate, with their own standards of excellence. Table 2.14 provides a comparison betweenthe most common competency taxonomies.While Bloom describes learning outcomes in a single dimension, other researchers acknowledgethe difference between to content and the actions that may be performed on it. This isaddresses either by the development of an alternate taxonomy (Merrill, 1973) or an extensionto an established one.65

Table 2.14: Levels of competenceJonassen Dreyfus Bloomintroductory novice knowledge(schema assembly) (apply rules)comprehensionadvanced beginner(apply maxims)applicationadvanced(domain & context dependent)competent(process-driven)analysisexpertise(experience & transfer)proficient(intuitive & decision-)making)expert(automatic, fine graineddiscrimination)synthesisevaluationmaster(develop style)Merrill (1973) classifies levels of content in terms of facts, concepts, procedures/rules andprinciples. Performance is at:remember - similar to Bloom’s ‘knowledge’, Merrill however distinguishes the type of contentbeing remembereduse - similar to Bloom’s ‘comprehension’ and ‘application’ levels, Merrill suggests facts cannotbe used or found, only rememberedfind - similar to Bloom’s ‘synthesis’, this level refers to the derivation, discovery or inventionof new concepts, principles or applications.Anderson and Krathwohl (2001) add a ‘knowledge dimension’ to Bloom’s categories to distinguishwhat is taught. This dimension encompassesfactual knowledge – the basic details of the content which students must know to makesense of the discipline, subdivided into knowledge of terminology and specific detailsand elements66

conceptual knowledge – knowledge of classifications and categories, principles and generalisations,theories, models, structuresprocedural knowledge – subject-specific skills, algorithms, techniques and methods plusknowledge of the criteria used in determining when to use specific procedures - ‘howto’metacognitive knowledge – how students know how they learn. Includes conscious applicationof cognitive strategies and self-knowledge of learning strengths and styles,with the most appropriate learning experience different for each component. As an example,procedural knowledge may be based on modelling and personal experience, while declarativeknowledge (encompassing factual and conceptual) may lend itself to didactic, explicit ordirect instruction. Metacognitive knowledge is learnt through socialisation and co-operativelearning, all within a specific domain of knowledge.Nevertheless, despite the existence of other learning outcomes classification schemes, Bloom’staxonomy is accepted as sufficiently detailed to allow mapping between outcomes and learningactivities, and is widely applied to knowledge area descriptions within the various BoKs (egRE within the SWEBOK project (Sawyer and Kotonya, 2000)) and to model curricula (egthe Computing Curricula volumes), albeit often with some adaptations.Competency levels are also assigned by professional bodies, either based on a mapping betweenthe relevant BoKs and the profession’s agreed competency level (generally throughaccreditation of specific educational programmes), or through a ‘testing’ of an individual’sperformance against the levels. As a less well-known example of the latter, the ECDL (EuropeanComputer Driving Licence) is supported by the European Union Commission as aninternationally recognised standard of competence. The ECDL Foundation was establishedin 1997 by the Council of European Professional Informatics Societies (CEPIS). The ECDLis awarded on the successful completion of one test assessing theoretical competence, and sixtests assessing the candidate’s competence in using the computer (Mulder and Weert, 2000).Finally, competency levels may be assigned by industry bodies, again based on a mappingor individual testing. Certification such as that provided through vendors such as Microsoftfalls into this category.The most common assignment of competency levels is that undertaken by formal educationalinstitutions. This approach is a key element of the discussion in this document, and isdescribed later in this chapter.The purpose of the next subsection is to survey relevant BoKs and current model curriculain order to identify and establish a model which may define the Body of Knowledge required67

to undertake Requirements Engineering (the REBOK) competently.2.4 Bodies of KnowledgeThe nature of RE problems described in Chapter 1 supports the view that a great deal ofspecialised knowledge is brought to structure and tame the complexity of tasks undertaken.Practitioners, and students, are seen to need conceptual knowledge in several overlappingdomains in order to perform Requirements Engineering tasks successfully.Several effortshave been made to identify knowledge required to undertake either RE in its own right, oras a component of IT related specialisations. A discussion of these follows. As the numberof topics pertinent to RE knowledge is large in some BoKs and model curricula, RE topiccoverage in these is only summarised here.Table 2.15: BoK matches for RE-related topicsTopics SWEBOK SEEK ISBOK ACSBOK INCOSERE Process + + o - -Feasibility Study - - + - -Elicitation + + + + +Analysis + + + - +Specification + + + + +Verification/validation + + + + +Management + + o - -Other Software TopicsQuality Standards o - + - -(process/product)Project Management o + + + -Programming Languageso o o - -Applications - - + o -Generic SkillsModelling (conceptual, o + o + +formal, etc)System thinking o o + + +Communication Skills o o + - -Team Skills o - + - -Ethics - - + - -Legend: + extensive coverage; o partial coverage; - minimal or no coverageAs Table 2.15, which summarises the topic coverage of these BoKs, shows, the SE, IS andCS efforts may be considered to have developed the most comprehensive BoKs, due to boththe attempt to be as wide-encompassing as possible within the range of IT education andthe perceived prestige of their publishing bodies.However, this table indicates the disparity of coverage of even the core activities of RE. Onceother skills are included, the patchiness of coverage increases. Possible reasons for this are68

discussed later in this chapter. The next sections provide some insight into why this is sothrough a brief examination of these BoKs.The Software Engineering Book of KnowledgeIn conjunction with the development of the SoftWare Engineering Body Of Knowledge (SWE-BOK), Requirements Engineering has been defined as a Knowledge Area, and a descriptionprepared (Sawyer and Kotonya, 2000). That document provides both a high level overviewof the RE process and a breakdown of activities it comprises.The process-based structure of the knowledge area is conceptualised as a spiral (see Figure2.6) which terminates either on successful completion of the deliverables (a requirementsspecification of some sort) or due to external forces (such as schedule pressure).Figure 2.6: Spiral model of the RE process (Sawyer and Kotonya, 2000)The breakdown of the six topics is traditional, and can be summarised as follows:Requirements Engineering process – identified as an emerging area, which introducesthe process, places it in an SE context and describes project organisation and contractualissuesrequirements elicitation – concerned with where requirements come from and how theymay be collected. This is seen as a fundamentally human activity69

equirements analysis – the process of analysing requirements to resolve conflicts, determinethe bounds of the system and elaborate software requirements from systemrequirementssoftware requirements specification – the structure, quality and verification of the requirementsdocument(s) as a precondition to successful requirements handling. Documentquality assessment is seen as an important aspectrequirements validation – through the use of reviews, prototyping, model validation andacceptance testingrequirements management – through change management, requirements attributes andrequirements tracing.This breakdown is seen to coverthe areas discussed in most requirements engineering texts and standards, eitheridentically, or derived from these and other sources to reflect a consensus compatiblewith industry, the literature and standards, and mirrors the mature andstable concepts in Requirements Engineering.(Sawyer and Kotonya, 2000, p 21)The value of this document is two-fold:• an acknowledgement that the Knowledge Area of Requirements Engineering is notconstrained to software issues solely. A need to understand the context of the problem,in terms of technical, social/organisational and environmental issues is emphasised• the interdisciplinary nature of the activities undertaken throughout the RequirementsEngineering process is also acknowledged. Requirements Engineering is seen to be anactive, human activity, relying on inter-personal skills as well as technical competence.The SWEBOK is a comprehensive description of the knowledge needed for the practice of softwareengineering, but intentionally does not cover non-software engineering knowledge thata software engineer must know. Although one of the objectives was to provide a foundationfor curriculum development, the SWEBOK is intended to cover knowledge after four yearsof practice (Carrington, 2000). Additionally, while the document assigns Bloom levels to allsubtopics up to Analysis level, the milestone at which these competencies are to be attainedis beyond the scope of this study. Further, no rationale for the competency levels assignedis provided in the document (so, for example, while ‘Elicitation techniques’ are required at‘Application’ level (presumably after four years of experience), ‘Requirements tracing’ needonly be ‘Comprehended’, but ‘Requirements negotiation’ developed to ‘Analysis’ stage).70

Software Engineering Education KnowledgeThe Software Engineering Education Knowledge (SEEK) defines and documents an SE educationBoK appropriate for guiding the development of undergraduate SE curricula. Itdescribes a set of desired curriculum outcomes and a statement of what every SE graduateshould know. Originally based on the SWEBOK knowledge areas and multiple discussionswith dozens of SEEK area volunteers, SEEK extended that work through its emphasis on theacademic discipline of SE. Once the content of the education knowledge areas were stabilised,topics were identified to be core or elective, and labelled with one of Bloom’s taxonomy levelsas educational objectives. However, only the lower three (ie foundational) levels of learningwere chosen, since they representwhat knowledge may be reasonably learned during an undergraduate education.(Sobel, 2003, p 6)This document was specially designed to support the development of undergraduate SoftwareEngineering curricula, and therefore, is considered a subset of the more generalised BoKrepresentation. One of the ten SEEK knowledge areas is focussed on RE. All except onetopic in this knowledge area are considered either essential or desirable inclusions in curricula,emphasising the importance given to the requirements stage of the software developmentprocess, with 53 core hours from a total of 494 (10.7%) dedicated to it (see Table 2.16).However, the SEEK does not represent the curriculum, but rather provides the foundationfor the design, implementation and delivery of the educational units that make up a softwareengineering curriculum (LeBlanc and Sobel, 2004).Of interest to this discussion is the section describing Software Modeling and Analysis(MAA). The following is quoted from the SEEK:Modeling and analysis can be considered core concepts in any engineering disciplinesince they are essential to documenting and evaluating design decisionsand alternatives. Modeling and analysis is first applied to the analysis, specification,and validation of requirements. Requirements represent the real world needsof users, customers and other stakeholders affected by the system and the capabilitiesand opportunities afforded by software and computing technologies. Theconstruction of requirements includes an analysis of the feasibility of the desiredsystem, elicitation and analysis of stakeholders’ needs, the creation of a precisedescription of what the system should and should not do along with any constraints71

Table 2.16: SEEK topicsTopic Bloom Category HoursSoftware Modelling and Analysis 53Modelling foundations 19Modelling principles (e.g. decomposition,a Eabstraction, generalization, pro-jection/views, explicitness, use of formalapproaches, etc.)Pre and post conditions, invariants c EIntroduction to mathematical models c Eand specification languages (Z, VDM,etc.)Properties of modelling languages k ESyntax vs. semantics (understanding c Emodel representations)Explicitness (make no assumptions, or k Estate all assumptions)Types of models 12Information modelling (e.g. entityrelationshipa Emodelling, class diagrams,etc.)Behavioural modelling (e.g. structureda Eanalysis, state diagrams, use caseanalysis,interaction diagrams, failuremodes and effects analysis, fault treeanalysis etc.)Structure modelling (e.g. architectural, c Eetc.)Domain modelling (e.g. domain engineeringk Eapproaches, etc.)Functional modelling (e.g. component c Ediagrams, etc.)Enterprise modelling (e.g. businessDprocesses, organizations, goals, etc.)Modelling embedded systems (e.g.Dreal-time schedulability analysis, externalinterface analysis, etc.)Requirements interaction analysis (e.g.Dfeature interaction, house of quality,viewpoint analysis, etc.)Analysis Patterns (e.g. problemDframes, specification re-use, etc.)Analysis fundamentals 6Analysing well-formedness (e.g. completeness,a Econsistency, robustness,etc.)72

Topic Bloom Category HoursAnalysing correctness (e.g. static analysis,a Esimulation, model checking, etc.)Analysing quality (non-functional) requirementsa E(e.g. safety, security, us-ability, performance, root cause analysis,etc.)Prioritisation, trade-off analysis, risk c Eanalysis , and impact analysisTraceability c EFormal analysis k ERequirements fundamentals 3Definition of requirements (e.g. product,c Eproject, constraints, systemboundary, external, internal, etc.)Requirements process c ELayers/levels of requirements (e.g. c Eneeds, goals, user requirements, systemrequirements, software requirements,etc.)Requirements characteristics (e.g. c Etestable, non-ambiguous, consistent,correct, traceable, priority, etc.)Managing changing requirements c E k EMGT.ctl.1 MAA.rfd.6 Requirementsmanagement (e.g. consistency management,release planning, reuse, etc.)Interaction between requirements and k EarchitectureRelationship of requirements to systemsDengineering, humancentred de-sign, etc.Wicked problems (e.g. ill-structuredDproblems; problems with many solutions;etc.)COTS as a constraintDEliciting requirements 4Elicitation Sources (e.g. stakeholders, c Edomain experts, operational and organizationenvironments, etc.)Elicitation Techniques (e.g. interviews,c Equestionnaires/surveys, proto-types, use cases, observation, participatorytechniques, etc.)Advanced techniques (e.g. ethnographic,Oknowledge elicitation,etc.)73

Topic Bloom Category HoursRequirements specification and documentation6Requirements documentation basics k E(e.g. types, audience, structure, quality,attributes, standards, etc.)Software requirements specification a ESpecification languages (e.g. structuredk EEnglish, UML, formal languagessuch as Z, VDM, SCR, RSML, etc.)Requirements validation 3Reviews and inspection a EPrototyping to validate requirements k E(Summative prototyping)Acceptance test design c EValidating product quality attributes c EFormal requirements analysisDLegend: k-knowledge; c-comprehension; a-application; E-essential; D-desirable; O-optionalon its operation and implementation, and the validation of this description orspecification by the stakeholders.(Sobel, 2003, p 13)The long list of the MAA knowledge area (see Table 2.16) is an indication of what every SEgraduate must know about RE.The IS Body of KnowledgeAppendices five and six of Gorgone et al (2002a) describe the Information Systems BoK indetail. This consists of three major subject areas:• Information Technology• Organisational and Management Concepts• Theory and Development of Systems.Each subject area contains major topics and each major topic contains subtopics whichare the lowest level curriculum elements of the BoK. As well as being derived from surveysof practitioners and academics, the ISBOK draws on the mapping of relevant topics fromcurricula for CS and other computer related disciplines (Davis et al, 1997). Adding a fourthlevel made it possible to include the more than 100 elements from the CS knowledge body(Turner and Tucker, 1991) and 120 elements comprising a defacto SE BoK, compiled from avariety of sources (Gorgone et al, 2002a).74

Of the ten courses which encompass the ISBOK, IS 2002.7 Analysis and Logical Designfocusses on the RE area:students with information technology skills will learn to analyse and design informationsystems. Students will practice project management during team orientedanalysis and design of a departmental level system.(Gorgone et al, 2002a, p 29)Only a high level breakdown and a list of learning unit goals are provided within the BoK.No attempt to assign competency levels is made – this is left to the model curricula.The CS Body of KnowledgeAppendix A of Engel and Roberts (2001) describes the Computer Science BoK in detail.It is organised hierarchically into three levels: an area represents a disciplinary subfieldbroken into units which represent individual thematic modules. Each of these is furthersubdivided into topics. The key unit which addresses RE is SE5 Software requirementsand specifications (see Table 2.17).Table 2.17: CSBOK: RE related topicsSE5 Software requirements and specifications [core]minimum core coverage time: 4 hoursRequirements elicitationRequirements analysis modeling techniquesFunctional and nonfunctional requirementsPrototypingBasic concepts of formal specification techniquesThis is designated a core element of any curriculum, requiring a minimum coverage time of4 in-class hours to address the following learning objectives (Engel and Roberts, 2001):1. apply key elements and common methods for elicitation and analysis to produce a setof software requirements for a medium-sized software system2. discuss the challenges of maintaining legacy software3. use a common, non-formal method to model and specify (in the form of a requirementsspecification document) the requirements for a medium-size software system4. conduct a review of a software requirements document using best practices to determinethe quality of the document75

5. translate into natural language a software requirements specification written in a commonlyused formal specification language.This would appear a big ask of 4 in-class hours, but is a strong indication of the (lack of)importance placed on RE topics.INCOSEFigure 2.7: INCOSE RBoK static modelThe International Council on Systems Engineering (INCOSE) supports RE research throughits Requirements Work Group (RWG). A current project of the group is a Requirements BoKModel (RBoKM) using UML notation (see Figure 2.7), of the types of entities (classes (egrequirement, stakeholder, elicitation method)) and activities (use cases (eg capture requirement,analyse requirement)) associated with RE. This model is related to other RWG andINCOSE work by means of a framework diagram. A draft of the model and the frameworkwere completed at the INCOSE 2002 International Workshop. The Model is a componentof the Systems Engineering BoK, to which is mapped a Systems Engineering Taxonomy ofskills. As a static model, the RBoKM addresses many of the elements identified in otherBoKs as a web of relationships centred around the concept of a requirement.76

ACS Core BoK for ITUntil the early 1990s, model curricula for undergraduate computing programmes were notdefined in Australia. Consequently the overseas programmes, particularly the ACM recommendations,precursor to the Computing Curricula efforts, were adopted and recommendedby the ACS in the design of tertiary computing courses in Australia.From 1992 onwards, however, increasing effort was directed to identifying a core BoK applicableto the Australian computing context. A report by Underwood (1997, bokpt1.htm),prepared for the Australian Computer Society (ACS),identifies the ‘Core Body of Knowledge’ in Information Technology which all I.T.professionals practising in Information Systems, Computer Science and ComputerSystems Engineering should be expected to have.Although the focus of the report is course accreditation, it attempted to establish knowledgeand skills core to the IT professions in Australia. This was defined as fourteen Areas ofKnowledge of which one, Systems Analysis and Design, relates to RE activities:This area develops basic systems analysis and design skills by examining commonlyused techniques and system development methodologies. A range of lifecyclemodels are considered including the classical waterfall approach and morerecent approaches such as prototyping and evolutionary development. The aimis to present a balanced overview of the process of analysing user requirements,designing computerised information systems to meet these requirements and atthe same time developing the necessary skills to apply the techniques to simpleproblems(Underwood, 1997, bokpt4.htm)2.5 Model curriculaAnecdotally, it has been suggested (Nagasamy, 2002) that most units in systems analysisand design (or similar) focus on design onwards, while RE (or software/systems analysis) hasreceived less treatment because of its ‘artistic’ nature. As a second pass, a discussion of thenotable model curricula that embrace aspects of education for software development providessome basis for the identification of elements which might comprise a REBoK.77

2.5.1 RE in model curricula and textsThe work of Iivari (1991) and Checkland and Holwell (1998) suggest that introductory, tertiarystudies level texts provide both an account of the field in a straightforward way anda condensed presentation of the body of accepted theory. The texts, therefore, as a primevehicle for disseminating the wisdom of a discipline, should mirror these competencies.The breakdown of RE described in model curricula is seen to cover the areas discussed inRequirements Engineering texts and standards:derived from these and other sources to reflect a consensus, and mirror the matureand stable concepts in Requirements Engineering(Sawyer and Kotonya, 2000, p 21)This is confirmed by the work of Minor (2004) who, in a comparison of the major modelcurricula, and an examination of representative tertiary level texts in the three core ITrelated disciplines shows that parallels exist, and, in general terms, the base case of REknowledge assumed by practitioners is covered in models used in university programmes toan ‘application’ level of competence. Table 2.18 provides a summary of his findings. Whatis implied by this cycle of curriculum development/textbook publishing is a process of reengineering:the discipline as defined in the curricula is to some extent a composite of thecontent of prescribed texts.Table 2.18: Minor: curricula match to perceived industry needsTopics CC-CS CC-IS CC-SERE Process o - oFeasibility Study - o -Elicitation + + +Analysis + + +Documentation + + +Verification + - +Requirements Management - o oOther Software TopicsProcess Standards + + +Project Management + + +Programming Languages + + +Generic SkillsCommunication Skills + + +Team Skills + + +Legend: + extensive coverage; o partial coverage; - minimal or no coverageAn alternate view of such texts is that they are enforce arbitrary boundaries: Watson et al(2002) suggest that the scope and depth of learner engagement in a discipline are determined,78

in current practice, through the ‘levels of text’ assigned by publishers (and reviewers) of whatbecome the prescribed texts of the discipline. The influence of these is seen as significant,further suggesting that the alliance between academic authors and publishing editors appearsseminal in influencing the boundary decisions, at least at a competency level, withina discipline (Watson et al, 2002). The implication is that, in a nutshell, for the purpose ofcurriculum development (model or otherwise), levels of competence may be arbitrarily basedon such pragmatic considerations as the choice of prescribed text.Watson et al (2002) attempt a mapping between these disciplinary levels and Bloom’s taxonomyand, although neither generally acknowledged nor universally accepted, at least givessome indications (at a macro level) of an implicit correspondence:Level 1 – the expectation at this level is that students will engage with this introductorymaterial in a manner that asks them to comprehend the new knowledgeand be able to describe and explain itLevel 2 – the expectation at this level is that students will engage with this moreadvanced material in a manner that asks them to analyse the interrelationshipsbetween models, tools and contexts in particular situations and applythis understanding in more challenging situationsLevel 3 – the expectation at this level is that students will engage with this advancedmaterial in a manner that asks them to evaluate the interrelationships ofmodels, tools and contexts in particular situations and synthesis and designoriginal ways via their adapted models and tools to address such situations.[original emphasis](Watson et al, 2002, p 710)This mapping indicates some mismatch between what is espoused and what is assumedin the practice of teaching. While, almost unanimously, the model curricula and (wherecompetency levels are indicated) BoKs define undergraduate competency up to Bloom’s level3 - Application, Watson suggests this is within (his) level 2 material attainable in the secondyear of a (3 year) undergraduate programme. By the final year, students have engaged inhigher order thinking and competency levels, not mirrored in the Model Curricula and BoKs.Computing Curricula modelsThis first section looks at the classic model curricula for IT, all developed under the umbrellaof the Computing Curricula (CC) project (Shackelford, 2005). The subsequent section examines,albeit briefly, alternate approaches advocated. What is important is that these addressa perceived need to move away from CC.79

Figure 2.8: The computing space occupied by CS, IS and SE (Shackelford, 2005)The ACM Education Board and the IEEE-Computer Society Educational Activities Boardoriginally commissioned a Joint Task Force on Computing Curricula to create curriculumrecommendations in several computing specialisations: CS, Computer Engineering, SE andIS (Engel and Roberts, 2001; Gorgone et al, 2002a; LeBlanc and Sobel, 2004). Shackelford(2005) provides an interesting representation of the problem space of computing (and henceaddress only computing topics within each), and suggests how each of the IT specialisationsconceptually occupies that space, focussing on what students typically do after graduation.The horizontal dimension ranges from Theory, Principles, Innovation to Application, Deployment,Configuration. The vertical dimension ranges from Computer Hardware and Architectureto Organisational Issues and Information Systems. As can be seen in Figure 2.8, each ofthe IT specialisations occupies space that overlaps as well as is distinct, with CS occupyingthe centre and left, IS the top and right and SE spanning the centre.CC-SEIn addition to effort expended to develop the SEEK, a second effort was the constructionof a set of curriculum recommendations, describing how an SE curriculum incorporating thematerial from the SEEK can be structured in various contexts.80

The description in this section is taken from CC-SE, the Computing Curricula – SoftwareEngineering (LeBlanc and Sobel, 2004). This volume does not provide prescriptive modelcurricula – rather it identifies a minimal coreconsisting of the essential material that professionals teaching software engineeringagree is necessary for anyone to obtain an undergraduate degree in this field.By insisting on a broad consensus in the definition of the core, it is hoped the corewill be as small as possible, giving institutions the freedom to tailor the electivecomponents of the curriculum in ways that meet their individual needs. [...] Everyundergraduate program will include additional units, both within and outsidethe software engineering body of knowledge, which this document does not attemptaddress.(LeBlanc and Sobel, 2004, p 18)Therefore, in a move away from a prescriptive approach to model curricula, a set of studentoutcomes has been identified, intended as a generic list that could be adapted to a variety ofsoftware engineering programme implementations. Those relevant to RE include:• show mastery of the software engineering knowledge and skills necessary to begin practiceas a software engineer• work as an individual and as part of a team to develop and deliver executable artefacts• reconcile conflicting project objectives, finding acceptable compromises within limitationsof cost, time, knowledge, existing systems, and organizations• understand the process of determining client needs and translating them to softwarerequirements• demonstrate an understanding and appreciation for the importance of negotiation, effectivework habits, leadership, and good communication with stakeholders in a typicalsoftware development environment• learn new models, techniques, and technologies as they emerge and appreciate thenecessity of such continuing professional development.(LeBlanc and Sobel, 2004)Curriculum design and delivery is based on a set of eighteen guidelines that should be consideredby those developing an undergraduate SE curriculum, and by those teaching individualSE units. These guidelines, in general, transcend the content if any specific curriculum. Theyprovide an underpinning on how the content should be taught. As some examples:81

Guideline 11 the underlying and enduring principles of software engineering should beemphasised, rather than details of the latest or specific toolsGuideline 14 the curriculum should have a significant real-world basisGuideline 17 software engineering education in the 21st century needs to move beyondthe lecture format: it is therefore important to encourage consideration of a variety ofteaching and learning approaches.(LeBlanc and Sobel, 2004)A set of example curricula and courses that can be used to teach the knowledge described inthe SEEK (taking into consideration the guidelines) is also provided. The expectation is thatcourses from other sources (such as CC-CS) are also included. SE322-req Requirementsis the core SE course about software requirements, with a coverage of 18 SEEK hours. Theseare made up as follows:Table 2.19: RE component of CC-SESEEK Topic:HoursTypes of models 9 core hours of 12Requirements fundamentals 1 core hour of 3Eliciting requirements 2 core hours of 4Requirements specification anddocumentation 4 core hours of 6Requirements validation 1 core hour of 3Requirements management 1 core hour of 3In addition, and as acknowledgement that SE curricula must not only teach facts but alsoensure that students achieve a level of skill at doing particular tasks required of the practicingSE, earlier version of CC-SE described exercises that will enable them to build up the requisitelevel of skill (Diaz-Herrera and Hilburn, 2003), with the additional acknowledgement thatmost will be problem-solving in nature and with the constraint that these are provided onlyfor higher level topics (those that have a Bloom’s taxonomy category of ‘a’ (Application) inSEEK). Unfortunately, most sample exercises focus on technical competence (eg create classdiagrams of a variety of domains and systems; create state diagrams and other behaviouralmodels of a variety of systems; elicit requirements for a variety of problems; write goodquality requirements documents).The value of this document is that it describes sample curricula in terms of patterns for introductorycourses, intermediate software engineering courses, and other courses, respectively.While acknowledging the infancy of undergraduate SE education in the US, and thereforetheir increased value in this context, the approach taken addresses issues of adaptation, not82

only to differing education systems (an international context, a computer science or engineeringschool context, an SE start in the first year or second year, a two semesters or threequarters academic year) but also to alternate environments (teaching and institutional).CC-CSThe CC-SE described above draws heavily on the Computer Science volume (CC-CS) of theComputing Curricula (Engel and Roberts, 2001). This section describes the final report forComputer Science, endorsed by both organisations at the end of 2001.Core topics within the curriculum are drawn from the CSBoK, and define a minimal, essentialcomponent of an undergraduate CS curriculum, to be extended by electives from the BoK.While a number of core topics are introductory, others assume prior knowledge.The units described are divided into three categories according to the level at which theyoccur in the curriculum. Those designated as introductory are typically entry-level unitsoffered in the first or second year. Units listed as intermediate are usually second- or thirdyearand build a foundation for further study in the field. Those designated as advanced aretaken in later years and focus on those topics that require significant preparation in termsof earlier coursework. The point of establishing the distinction between units is to providenatural boundaries for selecting implementation strategies.Several implementations of the model curricula are provided, based on the approach (egimperative-first, algorithm-first, breadth-first etc) advocated. In each of these, the RE componentis considered at intermediate level (though the introductory-level curriculum notesthe importance of specification in the software process). Of the 280 hours of core materialin the CSBoK, the cognitive capability in requirements expected of a CS graduate (establishedthrough completion of the RE component of the SE area) consumes 4, although someapproaches including an additional hour in an introductory unit.As can be seen from Table 2.20, CC-CS addresses the needs of RE only minimally within thetechnical component of the curriculum.While the unit SE5 includes the bulk of RE-related topics, others are included in SE1 (ObjectorientedAnalysis), SE3 (Requirements Analysis and Design Modelling tools), SE8 (SoftwareMeasurement and Estimation Techniques) and SE10 (Formal Methods). Table 2.21 indicatesthe learning objectives identified for SE5, and the competency levels applied to these.Despite the paucity of RE-related topics, the list of capabilities and skills for CS graduatesaddresses some which could be considered transferable to RE. These include:Modelling use technical knowledge and understanding in the modelling and design of sys-83

Table 2.20: SE component of CC-CSSE. Software EngineeringSE1. Software designSE2. Using APIsSE3. Software tools and environmentsSE4. Software processesSE5. Software requirements and specificationsSE6. Software validationSE7. Software evolutionSE8. Software project managementSE9. Component-based computingSE10. Formal methodsSE11. Software reliabilitySE12. Specialised systems development31 core hours8 (core)5 (core)3 (core)2 (core)4 (core)3 (core)3 (core)3 (core)Table 2.21: Learning objectives of SE5 in CC-CSSE5 Software Requirements and SpecificationsApply key elements and common methods for elicitationand analysis to produce a set of softwarerequirements for a medium-sized software systemDiscuss the challenges of maintaining legacy softwareUse a common, non-formal method to model andspecify (in the form of a requirements specificationdocument) the requirements for a mediumsizesoftware system.Conduct a review of a software requirements documentusing best practices to determine the qualityof the documentTranslate into natural language a software requirementsspecification written in a commonlyused formal specification languageCompetencyApplicationComprehensionApplicationApplicationApplicationtems in a way that demonstrates comprehension of the tradeoff involved in designchoicesRequirements identify and analyse criteria and specifications appropriate to specific problems,and plan strategies for their solutionCritical evaluation and testing analyse the extent to which a computer-based systemmeets the criteria defined for its current use and future developmentMethods and tools deploy appropriate theory, practices, and tools for the specification,design, implementation, and evaluation of computer-based systemsProfessional responsibility recognise and be guided by the social, professional, and ethical84

issues involved in the use of computer technology.Other, practical skills include information management while more generic skills includecommunications, teamwork, numeracy, self-management and professional development.One could conclude that CC-CS acknowledges that activities in RE rely on a broader skilland capability basis than the SE5 unit would suggest.CC-ISIn late 2002, CC-IS – Model Curriculum and Guidelines for Undergraduate Degree Programsin Information Systems was approved and published (Gorgone et al, 2002a,b). As a collaborationbetween the ACM, AIS (Association for Information Systems) and AITP (Associationof Information Technology Professionals), the document provides a model curriculum andguidelines for undergraduate degree programmes in Information Systems. CC-IS is a minorrevision of IS97 (Davis et al, 1997), which had been widely accepted and become the basisfor accreditation of undergraduate programmes of Information Systems. As such CC-IS islargely based on the IS97 material: beyond the addition of an internet-based commerce unit,and the merging of two units that focused on exposure to packages and improving studentskills with these, the IS97 units have been retained with appropriate updating of scope andtopic descriptions.The curriculum comprises five presentation areas consisting of ten units. IS2002.7 Analysisand Logical Design addresses the RE area:students with information technology skills will learn to analyse and design informationsystems. Students will practice project management during team orientedanalysis and design of a departmental level system(Gorgone et al, 2002a, p 29)and is summarised in Table 2.22. The courses are based on 127 learning units derived fromelements in a Body of IS Knowledge. The set of topics within each learning unit is assigneda depth of knowledge level:1. awareness2. literacy/strong knowledge3. usage/comprehension/skill4. detailed understanding/application/ability(the highest in an undergraduate program)85

Table 2.22: IS 2002.7 analysis and logical designDescriptionSCOPE This course examines the system development and modification process. Itemphasizes the factors for effective communication and integration with usersand user systems. It encourages interpersonal skill development with clients,users, team members, and others associated with development, operation, andmaintenance of the system. Structured and object oriented analysis and design,use of modelling tools, adherence to methodological life cycle and projectmanagement standardsTOPICS Life cycle phases: requirements determination, logical design, physical design,and implementation planning; interpersonal skills, interviewing, presentationskills; group dynamics; risk and feasibility analysis; group-based approaches:project management, joint application development (JAD), and structuredwalkthroughs; structured versus object oriented methodologies; RAD, prototyping;database design; software package evaluation, acquisition, and integration;global and inter-organizational issues and system integration; professionalcode of ethicsDISCUSSIONStudents with the basic skills of information technology will learn to gatherinformation in order to identify problems to be solved. They will determinesystem requirements and a logical design for an information system. A projectof limited scope will be designed during this course. Students will investigatealternative solutions, and will determine feasibility of solutions. They willidentify value added by the completion of the system. Students will be exposedto methods to support each stage of the development process. While automatedtools are not a substitute for understanding of the processes involved, theymay be used to ensure that a particular methodology is used rigorously. Ifmanual methods are used, it is important to define the methodology thoroughly.Project management will be taught and used to control the team project.Team concepts including personal and interpersonal skills will be discussed andmonitored. Empowerment concepts will be used and measured. Schedulingand completing individual and group actions will be used to ensure projectmilestone completion5. advanced.For CC-IS, a survey of computing faculty in the United States was conducted to obtain dataon two areas related to the curriculum update. The first was to determine their current viewof the appropriate depth of mastery for each of the elements in the IS97 BoK. The secondwas to gather similar information for key skill areas identified within IS97. The primaryconclusions are summarised as:1. IS analysts have specific skills at approximately IS97 skill depth level 3 (the ability toUSE knowledge) in areas of ‘Interpersonal and Team Skills’, ‘Business Knowledge’,‘Organizational Process Development’ (including IS Systems Analysis and Design),‘Project Management’, ‘Database’, ‘Software Development’, ‘Web Programming’, and‘Systems Integration’86

2. skills identified in IS97 as ‘Exit Curriculum Areas’ match expectations of the computingindustry as well as IS faculty3. skill areas produced by programmes of Information Technology match expectation ofInformation Systems faculty4. the model courses of IS97 are acceptable to both IS and IT faculty. Interestingly, bothCS and SE faculty also feel IS97 courses are relevant.Table 2.23: Analytical and critical thinkingOrganisational Ethics and CreativityProblem Solving ProfessionalismProblem solving models,Codes of conduct Creativity conceptstechniques, and ap-proachesPersonal decision making Ethical theory Creativity techniquesCritical thinkingMethods to collect,summarise, and interpretdataStatistical and mathematicalmethodsLeadershipLegal and regulatorystandardsProfessionalism: selfdirected, leadership,time managementProfessionalism: commitmentto and completionof workThe systems approachWhat is fundamental to the approach initially taken in IS97 and subsequently in CC-IS isthat while Bloom level 1 is mapped to CC-IS levels 1 and 2, Bloom levels 4, 5, and 6 aremapped to IS97/CC-IS level 5 (and therefore outside the scope of undergraduate education).Appendix 4 (Depth of Knowledge Metrics and Related Pedagogy) of Gorgone et al (2002a)describes the rationale for this. Expectations of the depth of knowledge to be attained isbased on the view that graduates should be able to accept direction and complete tasksassigned as well as be able to apply their knowledge without direction (Davis et al, 1997).Classroom activity or participative learning strategies are seen as sufficient in transferring lowlevels (1 and 2) of knowledge, although a level 2 activity can be enhanced in the laboratory.The more complex CC-IS level 3 requires considerable practice and creative repetition, whilelevel 4 requires unsupervised practice. Team work, project work, and other participativelearning facilitate the achievement of these levels (Gorgone et al, 2002a).87

Representative exit characteristics of IS graduates are shown in Table 2.23.Non Computing Curricula modelsACS‘97As has been noted previously, prior to 1992 the ACS adopted and recommended overseasprogrammes, particularly the ACM recommendations, in the design of tertiary computingcourses in Australia. From that time, efforts to develop curriculum guidelines led to thetwo publications (Maynard and Underwood, 1996; Underwood, 1997) to address the issue ofundergraduate education in IT.The focus of the report by Underwood is accreditation. To facilitate a practical implementationof a course of study complying with the recommended ACS BoK, the areas of knowledgeare separated into two groups. Group 1 material relates to generic attributes relating to InterpersonalCommunications; Ethics/Social Implications/Professional Practice and ProjectManagement and Quality Assurance, and is mandatory. Group 2 addresses IT topics, generallyfrom the ACS BoK. Although Systems Analysis and Design is included as oneof the 14 Knowledge Areas, the curriculum guidelines do not mandate which areas mustbe addressed within the IT curriculum. The expectation is that the various Australian ITtertiary courses would implement and emphasise different aspects of the material in accordwith their academic and equipment strengths.IRMA/DAMA‘00In 2000 the Information Resource Management Association and the Data AdministrationManagers Association published a generic curriculum framework to address the needs ofmanagers to understand information resources management. This model curriculum (Cohen,2000), is seen as complementary to IS97 in that the focus is shifted fromthe narrow definition of the computer-based information system as a collection ofhardware and softwareto ‘human concerns’:(Cohen, 2000, p 12)the organizational impact of information systems, how these systems fit into theorganizational structure of the firm, and how managers can utilize the productsof information systems across the company(Cohen, 2000, p 12)88

to be applied to students in at least their junior year (ie after the second year of study).The framework is divided into 10 modules, the initial six of which are tagged as ‘required’.IRM6 – Design and Implementation provides students with hands-on applications of thedesign and implementation of information systems in organisations. The topics that comprisethis module are Information Management and Information Systems allocated 15% of thecourse; Systems Analysis of Information Systems (15%); Systems Definition of InformationSystems (15%) and Information Resource Management and Behaviour (10%).In addition IRM2 – Information Systems Technology focuses on learning about informationtechnology components and their applications in the management of resources inorganisations.One of the six topics of this course, Systems Professionals and Information in Organizations(20%), has as one of five subtopics, systems analysts and information management.Both these courses are included in the six required components. However, of ten coursesproposed within the contents of this curriculum model, the recommendation is that a total ofseven courses be taken by the student, with five courses required and two taken as electives(which implies one required course is not taken).The curriculum model concludes:We are particularly critical of curricula that are exclusively comprised of coursesin quantitative analysis and hardware/software-oriented courses, and offer littleon the organizational implications of information systems(Cohen, 2000, p 25)Cohen (2000) is an attempt to address this shortcoming.ISCC‘99An alternate approach to curriculum guidelines, the Information Systems-Centric Curriculum1999, funded by the US National Science Foundation, is based on collaboration betweenindustry and academia and attempts to address what:is amiss in the nation’s ability to generate well prepared new graduates in theinformation systems-centric disciplines(Lidtke et al, 1999, section2.shtml)The ISCC‘99 Task Force:89

• focused on the needs of the workplace, as seen collectively by the academic and industryTask Force – requirements for the curriculum were developed by the industry membersas a profile of the graduate that specified the technical and personal attributes neededto function effectively as an information systems specialist; enterprise collaborationis required to deliver and maintain the curriculum by providing meaningful projectactivities, site experiences, case studies, and assistance in updating the curriculum• examined the process by which students obtain the knowledge and skills that are neededto practice in large information systems-centric domains – pedagogical aspects such asteaming, just-in-time learning, and coach and mentor roles for the instructor, wereincorporated into the curriculum• embedded personal and interpersonal skill development in the curriculum – interpersonalskills, systemic thinking, and problem solving techniques were explicitly integratedinto the technical components of the curriculum• explored new approaches to learning and teaching – the curriculum is designed aroundpractical experiences that result in the incremental development of portfolios thatdemonstrate students’ preparation to function effectively as IS specialists, and appliesan inverted model, in which students experience first the context of information systems,later master details, and then return to the systems view to complete their experience.This curriculum prepares graduates to work in teams with process owners andusers. It prepares graduates to identify information systems solutions to largeproblems, and communicate their concepts to others. The graduate of the curriculumwill decompose problems, develop alternative solutions, evaluate alternatives,conceptualize designs, build, test, validate, and deliver large or complex informationsystems in a team environment. Graduates also will understand the socialimplications of their actions.(Lidtke et al, 1999)The outcome is a recommended set of courses accompanied by defined learning activities andpedagogy for both student learning and instructor teaching to achieve the attributes of agraduate noted in Table 2.24.It is notable that RE activities are very sparse in the 13 courses defined, and generally havea lower expected proficiency level (hovering around 3). The focus of this model is on the ‘softskills’, which are expected to be mastered generally to a level 4 competency over a maximumof 4 years. Enabling concepts must be understood, exercised and utilised in the curriculum.These include what is classically the content knowledge of BoKs.90

Table 2.24: Industry perception of graduate attributesPersonal Skills Interpersonal Skills Technical Knowledge& SkillsSystemic-thinkingskillsCollaborative skills Information abstraction, representation,and organisationProblem-solving skillsCritical-thinking skillsCommunication skills(oral, written, listening,and group)Enterprise computing architecturesand delivery systemsConcepts of information and systemsdistributionRisk-taking skills Conflict resolutionskillsPersonal-disciplineskillsPersistenceCuriosityHuman behavior and computerinteractionDynamics of changeProcess management and systemsdevelopmentSome Information Systems domainknowledgeUse of computing tools to applyknowledgeICF-2000The IFIP/UNESCO Informatics Curriculum Framework 2000 (ICF-2000) offers an informaticscurriculum framework at tertiary level, from which various curriculum implementationscan be constructed. The International Federation for Information Processing (IFIP) was requestedby UNESCO to undertake this project on its behalf. IFIP’s Technical Committee 3(on Education) developed the framework, which includes of the following entities:categories of professionals ICF-2000 recognises eight different professional categories (ofwhich three are professionals with a major in informatics) and four different graduateprofiles (BIP (Basic Instrumental I-Profile), BCP (Basic Conceptual I-Profile), MIP(MInor I-Profile) and MAP (MAjor I-Profile) .The size of the graduate profiles is‘measured’ in terms of credit points: 1 credit point equals one working day (= about 8hours of study). Informatics-major I-Workers are categorised as• operational (C1) – have a thorough understanding of and well-developed skills ininformatics as a broad discipline, more specifically in the area of exploitation, con-91

trol and maintenance of available I-technology and I-applications. This categorycontains a large portion of lower level I-professionals, for example computer operators,network operators, application administrators, database administrators,helpdesk employees, etc. Also university-level I-professionals will be required indirecting, supervising and managing roles with respect to this area• engineering (C2) – have a thorough understanding of and well-developed skills ininformatics as a broad discipline, more specifically in the area of analysis, designand implementation of I-systems. Examples are the information systems analyst,the software engineer, the knowledge engineer, the scientific programmer, thedatabase developer, the IC designer, etc. With the expansion of I-technology andI-applications in all kinds of processes and the increasing complexity and interactionof I-systems one would expect a continuing growth of this category• researching (C3) – have a thorough understanding of and well-developed skills ininformatics as a broad discipline, more specifically in research. They are supposedto further develop the I-discipline and its concepts, both on its own and in relationwith other disciplines.(Mulder and Weert, 2000, p 26-27)implementation factors and strategies implementation factors lead to strategies for full(or partial) implementation and controlled development, taking account of cultural,institutional and regional aspectscurriculum units a set of 12 generic I-curriculum units describe a coherent area of content(knowledge and skills) to be acquired. Targeted competencies are noted, withappropriate learning approaches (eg theoretic, practical, exercises, real life etc). Sourcecurriculum references are made (eg IS97) and appropriate classifications assigned (egUCSI (Unified Classification Scheme for Informatics (1997)) and ACM’s CCS (ComputingClassification System (1998))).A curriculum specification is given that ‘fingerprints’ a graduate profile for the specific professionalcategory with respect to:• themes – content (12 core themes: representation of information; formalism in informationprocessing; information modelling; algorithmics; system design (including requirements);software development; potentials and limitations of computing and relatedtechnologies; computer systems and architectures; computer-based communication; socialand ethical implications; personal and interpersonal skills; broader perspectives and92

context (including links with other disciplines)). Themes of interest in this discussioninclude the following, with descriptions from Mulder and Weert (2000):◦ Information Modelling – manipulating information (or its representation) isimportant precisely because we obtain understanding of situations or phenomena.Whenever you write a program or design a system to solve a problem, you are modellingthe world (or a specific domain). Critical to the discussion of informationmodelling is an appreciation of the complexity of the phenomena to be modelled.This will reveal certain inadequacies and difficulties in the very paradigm of informatics.Relevant topics are: abstract data types, object orientation, informationsystems, and databases. But also: data collection, ambiguity, effects of policy andsocial parameters, information loss, etc◦ System Design – the various fields of informatics have always been involved withthe construction of systems. Perhaps due to increasing awareness of engineeringmethodologies, more recent attention has been given to elements of good design(e.g. the emergence of the discipline of software engineering). Relevant topics are:requirements analysis, specification, design, implementation, testing and validation,maintenance, documentation, human-computer interaction, security, quality,prototyping, rapid application development, method engineering◦ Personal and Interpersonal Skills – it has been stated that the era of thesolo asocial programmer has come to an end. Through a maturing of the field,as well as the awesome complexity of the problems to be solved, effective teamworkhas become crucial in the construction of the resulting extremely complexsystems. Examples of skills required are: communication, team work, criticalthinking, leadership, working with users, interdisciplinary environments, writtenspecifications and documentation, dealing with ambiguity.• orientation – competency level. These are defined asAW AWareness (know or use) aiming at developing basic knowledge as well as skillsthat allow students to act basically literate with respect to informatics in generaland to perform standard operations using computer technology or softwarepackagesAP APplication aiming at developing a basic conceptual understanding of informaticsand of some more advanced informatics skills which allow students to applybasic informatics to other disciplines or areasDM Design and Modelling aiming at developing a general understanding and broad93

overview of informatics, especially with respect to the modelling and the designof informatics applicationsCA Conceptualisation and Abstraction aiming at developing a thorough understandingof and well-developed skills in informatics as a broad discipline, theessence being to further develop the capability of students to abstract and toconceptualise.The curriculum specification for all categories of informatics-major I-Workers allows for 21credit points in the theme of Information Modelling, 20 credit points for System Design andand 32 credit points in the theme of Personal and Interpersonal Skills, from a total of 300credit points.INCOSE‘99In 1998, the International Council on Systems Engineering (INCOSE) convened an EducationalMeasurements Working Group (EMWG) to look at measuring the academic educationcapabilities of various universities. At the first meeting, the EMWG identified one problemthat exacerbated the measuring of academic curriculum. That problem was the lack of acompetency profile as an occupancy standard.The EMWG developed a Systems Engineering Competency Profile to serve as a guide to theminimum level of competency that graduating students should achieve prior to entering thework force, and at other stages of their career, based on defined engineering levels:I Systems Engineering Practitioner (Bachelor’s Degree)II Fully Qualified Systems EngineerIII Senior Systems Engineer (Master’s Degree)IV Certified Systems Engineer.Proficiency levels (again modified Bloom et al (1956)) were used to determine a desiredindustry proficiency level for each sub-topic of the skill set:0. no competency required1. basic knowledge, recalling of specific bits of information2. comprehension of the subject matter but without application3. applying the information in new situations94

Table 2.25: RE specific competencies for systems engineersEngineering Levels: I II III IVManage Requirements 2 4 5 6Diagram the typical Requirements Management 2 4 5 6Process and explain the basic activitiesassociated with each stepDescribe the basic format for writing high quality 3 5 6 6requirements statementsList the most common techniques for identifying 3 5 6 6requirements creepExplain the use of version control for tracking 2 4 5 6requirements within the same specificationsdocumentsDefine the following documents and how requirements 2 5 5 6differ/are the same among them:Systems Specification (SS),Sub-System Specifications (SSS),Operations Concept Document(OPSCON),Functional Specification (FS),Interface Control Documents (ICDs)Explain how to track requirements among the 2 5 5 6various system specifications documents soa change to a requirements in one part of thesystem documentation will flow to (ripple through)the other companion specification documentsIdentify the most common COTS packages for 3 5 5 6tracking requirements and provide a trade-offmatrix comparing the advantages and disadvantagesamong them4. analysing, evaluating or breaking down the information5. ability to synthesize the information to form an original result6. applying the information in both expected and unforeseen situations.and conforms with other model curricula in assessing graduates to a maximum of Bloom’sLevel 3 competency. Table 2.25 describes RE-specific competencies throughout the stages ofan Engineer’s career.EIA/IS-731.1This final look at RE activities is not part of a model curriculum, but was developed in conjunctionwith concurrent attempts to develop a Systems Engineering Capability AssessmentModel (SECAM), supported by INCOSE, a Capability Maturity Model (CMM) for SystemsEngineering (SE-CMM) and an appraisal method, generated by EPIC (Enterprise Process95

Improvement Collaboration). In 1996 a merge of the current versions was initiated under theauspices of the EIA (Electronic Industries Association) G-47 (Systems Engineering) Committee.The result is EIA Systems Engineering Capability Model (SECM) and SECM AppraisalMethod, proposed as standard EIA/IS-731.1.Within the Technical Category three focus areas are most closely relate to RE activities:FA 1.1: Define Stakeholder and System Level Requirements; FA 1.2 Define Technical Problem;FA 1.3 Define Solution. Other, non-technical activities are defined in the Managementor Environment Categories (eg Management Category: FA 2.1 Plan and Organise; FA 2.2Monitor and Control; FA 2.3 Integrate Disciplines; ; FA 2.5 Manage Risk; 2.7 Manage Configurations;FA 2.8 Ensure Quality. Environment Category: FA 3.1 Define and Improve theSystems Engineering Process).Specific practices are generally defined up to level 3 (Defined: where activities are significantlyeffective and work products are of significant utility).The value of this document is the wide variety of industry input and the depth of decompositionof focus areas identified.2.5.2 Towards a REBoKWhat does a review of the BoKs and model curricula indicate with regards to technicalcompetence in RE?It would seem that no one source provides the coverage hinted at by practitioner studies.Hinted at because technical components are implicitly considered as ‘covered’. Practitionersare more concerned about the variety of techniques and strategies learnt than the particularsof a specific approach. This is in keeping with the acceptance of technical skills as a ‘filter’for employment, noted previously. Attachment 1 of Appendix B provides a summary listingof topics (with brief explanation) which may be considered fundamental to the practice ofRE technically. This listing is compiled for the purpose of surveying IT professional engagedin RE practice, and is drawn from the BoKs and model curricula discussed in this chapter.If we were to present a schematic of the computing space of RE (following Shackelford (2005)),the landscape may look like that shown in Figure 2.9. What this indicates is an extractionof components of the computing space of IS, CS and SE (at the least). The basis of thelandscape is the SE space, providing coverage from theory to applications. Some of themore theoretical components of CS are important (formal methods are one example). Whilethere is perhaps less need for theoretical aspects of application technologies, the IS focus onorganisational issues is vital.96

Figure 2.9: The computing space occupied by RE (after Shackelford (2005))Exploration of the validity of this model should be attempted in the future. The spaceshould also be ‘expanded’ to encompass the soft and cognitive skills identified in this chapteras requisite for competent RE practice.2.6 Factors against successmy perception of someone who is successful is not someone that knows the most,it is someone who can use the knowledge they do have the bestengineer in study by Turley (1991, p 119)The review of model curricula described in this chapter indicates that, in general terms, thebase case of RE knowledge identified in practitioner studies is covered in models used byformal tertiary education. A look at generic IT (as opposed to specific RE) suggests Minor(2004)’s conclusion can be extrapolated - most BoKs and model curricula address specialisationcontent comprehensively (though obviously how the discipline is viewed ideologicallyhas some bearing). It can be stated that, in many cases, the emphasis placed by differentmodel curricula reflects, at a fundamental level, the base assumptions made on, for example,the nature of the system, the importance of its context, and the nature of knowledge required97

to undertake these activities. In general terms, the approaches taken have subscribed to apositivist view of software development, and model scientific and engineering methodologies,with their focus on process and repeatability. Benson (2003) suggests this is true of the ISspecialisation as well as CS and SE, so that current IS curricula in practice (as opposed toemerging model curricula) continue to show a heavy dependency on positivist thinking.The competency assigned to RE-relevant elements is also influenced by ideological factors,and, in general, only foundational levels of competence (ie Bloom levels of knowledge, comprehensionand application) are considered appropriate for graduate attainment. Yet Leeand Truex (2000) suggests mature use of methods etc (we can generalise to categorise theseas profession-specific knowledge and skills) is based on the development/derivation of an individual‘methodology’ at an appropriate level of granularity and tailored to the profile ofthe context. Although this level of ‘mastery’ (Dreyfus, 2001) is ambitious for undergraduateeducation, as we will see in the next chapter, these processes, their granularity and, inparticular, the ability to adapt them, are an indication of higher order thinking, interrelatedwith cognitive complexity. This can be addressed in formal education.Other, non-technical skills are usually addressed at a more abstract level and often in associationwith ethics, management or social concern. Despite the effort placed in the developmentof engineering approaches to software development, the overwhelming determiner of softwareproduction productivity is personnel and team capability – up to twice as important as thenext highest productivity factor (Boehm, 1981). This finding, supported by practitionerstudies, suggest non-technical elements should be dominant (or at least be equally considered)within a formal education environment. However, Lowry and Turner (2005) suggestthat tradition and inertia act as some of the formidable barriers to substantive revisions tocurricula in line with the findings of practitioner-based studies.Therefore, while profession-specific knowledge and skills and their initial competence aregenerally considered in model curricula, though perhaps not to an appropriate level, howuseful the knowledge generally included in tertiary institution curricula is for the practicalitiesof being a professional RE can be questioned. Practitioners emphasise attributes other thantechnical skills – among others, both Bentley et al (1999) (personal attributes) and Scottand Wilson (2002) (stance) develop models which address these affective (as opposed tocognitive) qualities in practitioners. The importance of the most obvious of these, personaland interpersonal communications is also confirmed in the literature of expertise: it acts asthe vehicle by which experience and expertise is transferred from expert to novice, as well asshared among the community of experts. This and other learning models is further examinedin Chapter 3.98

Given the nature of RE problems, the discipline has been characterised as a ‘wicked’ (Bubenko,1995). In a similar vein, RE education has been referred to as an ‘educational dilemma’(Macauley and Mylopoulos, 1995a). The dilemma is to provide the student with a solidfoundation in subject matter while at the same time exposing the student to the inherentcharacteristics associated with real requirements problems and the knowledge required tosolve them.In order to address both this wickedness and educational dilemma, the Requirements Engineeringstudent must acquire cognitive flexibility – including the ability to represent knowledgefrom different conceptual and case perspectives and, later, the ability to construct from thesea knowledge ensemble tailored to the needs of the understanding or problem-solving at hand– the same items of knowledge need to be presented and learned in a variety of different waysand for a variety of different purposes.In summary, practitioners of RE should be cognitively complex. The virtue of a cognitivelycomplex person is evident only when the environment requires them to deal with complexconditions and a high level of uncertainty (Lee and Truex, 2000): characteristics of the REenvironment.Bieri et al (1966) suggestsa more cognitively complex person has available a more differentiated system ofdimensions for perceiving others’ behaviour than does a less cognitively complexindividual(Bieri et al, 1966, p 185)and hence the capacity to construe social behaviour in a multidimensional way. This alsopoints to a requirement for cognitive complexity in an RE.However, both practitioners (Macauley and Mylopoulos, 1995b; Minor, 2004) and academics(Bentley et al, 1999; Banks, 2003) question whether attaining these attributes could/shouldbe taught and learned (and hence included in university education). Theories of learning,and models that apply these do suggest it is feasible to address concerns about RE educationformally.The theories of learning highlight aspects of knowledge and skill development that are pertinentto Requirements Engineering: competence in an area is based on a rich framework ofunderstanding and a commensurate reduction of effort in further learning (Craik and Lockhart,1972; Craik and Tulving, 1975), achieved through:99

• problem-solving practice• correspondence with the learner’s intuitive model of the phenomenon (White and Frederiksen,1986) (and all that implies about previous experience, belief systems etc.)• ease of transfer (with its focus on strategic thinking and metacognitive skills) (Larkin,1989)• facility with multiple representations – translation between different models facilitatesthe understanding of concepts, whilst the models themselves support differing insights,reasoning and problem-solving (Wood, 1999)occurring within a social and cultural context (and all that implies about dialogue, selfexplanationand shared meaning (Vygotsky, 1978; Chi and Bassock, 1989; Laurillard, 1993;Jonassen et al, 1995)) where certain activities are seen as authentic (Mayes, 1992).These theories are, in turn, influenced by cognitive theories of• knowledge representation◦ an assumption that schemata can be effectively built and activated in contextsthat do not match closely the environment in which they were constructed◦ information mapping as a means of imposing structure on knowledge◦ the impact of external knowledge representations (eg diagrams, text, numbers andabstract systems of symbols) on the processes of learning, discovery and reasoning(Winn and Snyder, 1996).• knowledge construction◦ knowledge and meaning is organised by modifying mental representations basedon previous experience, either through accommodation, assimilation or discontinuity(Vygotsky, 1978; Piaget, 1980; White and Frederiksen, 1986). Reciprocalrelationship exist between learning and memory (what we learn is affected by itsmeaningfulness, that meaning is determined by what is remembered, and thatmemory affected by what we learn (Winn and Snyder, 1996)) and between knowledgeand environment◦ the focus is deep understanding rather than skill as the goal of instruction, withstages the result of the necessary re organisation of knowledge rather than theresult of maturation100

• metacognition◦ involving awareness of one’s own cognitive processes (rather than the content ofthose processes) and the use of that self awareness in controlling and improvingcognitive processes (Biggs and Moore, 1993). The learner monitors his ownprogress as learning occurs, and is able to plan, choose between and change strategiesbased on the analysis of intermediate results (Ridley et al, 1992; Biggs andMoore, 1993).◦ the most efficient way to find or construct a model which is adequate for a givenproblem is by reasoning on a metacognitive level, where a class of possible modelscan be analysed and compared. This requires a metasystem transition with respectto the variety of individual models. Gaining flexibility with reasoning goes handin hand with enhanced self regulation and sharper metacognitive understanding.The implication of a metacognitive decision-making process is twofold:◦ - that there is a wide range of skills from which a decision may be made – hencewide experience◦ - that a process of reflection exists.Learning models that address wicked domains propose that a foundation in the contentneeds to be balanced with elements of creativity and experience based on practice. In general,these models are based around constructivist principles and more specifically on experientiallearning tradition. Flexible environments are required to permit the same items of knowledgeto be presented and learned in a variety of different ways and for a variety of differentpurposes, at different levels of granularity (Spiro et al, 1991).These conclusions suggest BoKs and model curricula cannot produce the graduates requiredby practitioners, though Budgen (2004) suggests they are a useful resource, and agrees theyfulfil the requirements for a good grounding in the discipline. Therefore, an investigationof learning pedagogy and current RE learning in order to address the issue of cognitivecomplexity, which appears to be lacking in RE education, is warranted.The next chapter examines the nature of learning and how it can be applied to RequirementsEngineering learning.101

Chapter 3A framework for learning REThe purpose of this chapter is to examine the theories of learning, the foundations on whichthey are built and the learning models that apply these theories. With Chapter 2 thisprovides the context for the research. This survey is not exhaustive, but tries to delineatethe aspects of learning theory and learning models that inform the research undertaken. Fromthis background, a framework for RE learning is developed. This may be seen as a synthesisof the salient points of these two chapters.Figure 3.1: Influences on the learning environment for RE (3)It should be reiterated that while the bulk of the background literature on learning is locatedhere, as a mechanism for ease of reading, in reality the literature, as with the methodology,102

was emergent. For this reason, some detail that could be considered background is stilllocated within the appropriate chapter.The shaded area in Figure 3.1 describes the elements of the conceptual framework discussedin this chapter.Overview of FindingsIn his counterpoint to Bach (1997)’s industry-focus perspective on SE education, McCracken(1997) suggests that formal education provides the entrée into the discipline. A principledacademic education provides the practitioner with:• adaptability – a good education teaches you how to adapt, place new knowledge incontext and use knowledge in multiple contexts• experimentation – provides a sheltered environment for learning from your mistakes• breadth – formal education provides a breadth-first approach; with experience comesdepth• transfer skills – to future technologies.These enable the graduate to become, over a period of years after graduation, a member ofthe profession (hence assuming a level of proficiency).The objective of RE education may be stated simply: to reduce the amount of time taken tobecome a competent member of the profession.In Chapter 2 RE is revealed as a complex, creative activity, where ill-structure and opportunismfeature. This chapter examines the learning models that address the criteria notedabove. The first section of this chapter is an overview of the approaches taken to explainingcognitive processing, especially that for complex problems. The findings of this section area set of declarations which summarise the cognitive requirements for learning generally, andlearning in ill-structured domains, where higher order learning is seen as essential in orderto acquire the skills to deal with the characteristics of the domain. The issues that mustbe addressed in order to enable complex problem-solving (such as fear of failure) are alsoidentified.The next section discussed the frameworks developed as learning theories to explain howthis knowing occurs. An examination of formal learning for RE suggest insights from theliterature of learning have not had much impact. The characteristics identified: the normativeprofessional education model on which it is based, and the texts that mirror a positivist103

worldview do not align with the characteristics of the discipline, or the needs of practitioners,both discussed in Chapter 2.We can conclude that too simplistic an educational process is seen as detrimental to thedevelopment of competent REs: just as the creativity of the RE process is hampered, so toois the education of its proponents hampered by adherence to traditional learning models.Given that creativity is a component of RE, then it seems reasonable, as Aurum et al (2003)suggest, that students studying RE should be well versed in the importance of creativitywithin the software development process, and also be skilled in applying creativity-enhancingtechniques. The rationalisation of the domain and its education is justified:• smoothly incremental or evolutionary approach to the RE process equates well withtraditional learning theories and models the accepted convergent approach to problemsolving• learning approaches that involve critical thinking and reflection are a challenge to students:they expect to be taught formulaic and recognised methods that will allowthem to build successful systems. Banks (2003) even suggests an approach that requiresreflective and active questioning that challenges previously held beliefs may beinappropriate for undergraduate students• a reductionist approach is considered easier to understand/navigate.The RE-online discussions by practitioners, reported in Chapter 2, confirms that this argumentis used as justification, and that it is problematic for (novice) RE practitioners.However, the poor fit between the characteristics of the domain and those of the learningmodel (which produces an ‘incorrect’ learning environment) impacts on further learning and isespecially relevant in light of the noted inadequacy of formal education in training competentanalysts/designers (eg by Robillard (1999)).3.1 The process of knowingKnowing...is a term that delineates a person’s potential to act in a certain fashion(Barab et al, 2001, p 66)Theories of human cognition can be addressed at several levels: that of neural processes, thatof elementary information processes (eg, retrieval from memory, comparing simple symbols)or at the level of higher mental processes (such as problem solving, concept attainment).104

Research on complex cognitive processes is approached on two fronts: extension to work onproblem-solving and the study of expertise and secondly, examination of the selection andmanagement of strategies for higher order processes. Both converge in the study of cognitivefunctioning in general, to which the term metacognition has been applied. This sectionprovides an overview of this work, and leads to a discussion of the theories of learning whichare underpinned by the study of cognitive functioning, and which, in their turn, underpinthe research reported in this thesis.3.1.1 The nature of knowledgeA clear trend may be seen, in the history of epistemology, from viewing knowledge as staticand passive to its relativity, or situation dependence – knowledge as adaptive, actively interferingin the world. The dominant philosophical positions have been:empiricism this posits that knowledge results from a ‘mapping’ or reflection of externalobjects. Although developed by observation, it is still absolute – any piece of proposedknowledge either truly corresponds to external reality, or not (Heylighen, 1993). Theunderlying premise is that knowledge results from the organisation of perceptual dataon the basis of inborn cognitive structures. While the subjectivity of basic concepts(such as space and time) are accepted, the a priori categories are static and given. Thisimplies a process of instruction in order to obtain an image, an encoding of the reality‘out there’pragmatism sees knowledge as consisting of ‘models’ that attempt to represent the environmentin such a way as to maximally simplify problem-solving (Dewey, 1910). Theassumption is made that no model can ever hope to capture all relevant information –if it did it would be too complex to use. Therefore the parallel existence of differentmodels, even if contradictory, is accepted – the model appropriate to the problem to besolved is chosen, as long as it produces approximate (if not correct) predictions. Therole of the outside world is limited to reinforcing some of the models while eliminatingothers in a process of selection. The ultimate reality behind the model is meaningless(Heylighen, 1993).A radical departure from these positions, constructivism assumes that all knowledge is builtby the subject – neither objective, empirical data, nor inborn categories or cognitive structuresare available for this construction, with the concept of correspondence with or reflection ofan external reality rejected. Knowledge is seen as largely independent of a hypotheticalexternal reality. Knowledge construction is a means of obtaining control over perception,105

in order to eliminate deviations or perturbations (Dewey, 1960) yet allowing adaptation tochanged circumstances to take place. The issue of absolute relativism, by which any modelconstructed is adequate, is addressed through either individual coherence or social consensus.Piaget (1968) was one of the earliest psychologists to describe constructivism through thedevelopment of a ‘stages of growth’ theory.Constructivism is based on three broad principles underpinned by the premise that there isno single correct mental model of knowledge. This may be summarised as:• each person forms their own representation of knowledge, based on previous experience(adopted by Dewey (1960) and implicit in the work of Vygotsky (1978) and Piaget(1980) although Piaget assumes the existence of an external reality)• knowledge construction occurs when an inconsistency between current knowledge representationand experience occurs (accommodation to deal with disequilibrium (Bruner,1962; Piaget, 1980). This accommodation may be internal (radical constructivism) orexternal (social))• behaviour is a function of state of the person and the psychological environment, implyingthe importance of social dynamics in the analysis of learning (Lewin, 1952; Bandura,1977). Thus knowledge construction occurs within a social context, acknowledging theplace of cultural symbols (Vygotsky, 1978), and that in naturalistic settings, individualslearn through observation of models (Bandura, 1977).Vygotsky (1978), in particular, emphasised active participation by learners in order to achieveunderstanding. He stressed the social bases of the mind – cognition should be understoodin a social context and human development treated as a process of acquiring culture, whilesociocultural theory emphasises the importance of a common frame of reference in socialinteraction.Definitions of the nature of knowledge vary, dependent on the specific viewpoint adopted.An approach widely accepted is to view knowledge in terms of its memory load (Robillard,1999). This views two types of knowledge:procedural knowledge is dynamic and involves all information related to the skills developedto interact with the environment, including psychomotor skills. Knowledgeacquisition is through practice and experience, and once learnt is rarely forgotten. Keyconcepts here are• skills – people can do these without thinking too much about them106

• know-how – problem solving capability based on experience rather than conceptuallearning (Hill et al, 1998)declarative knowledge is static and based on facts. It is concerned with properties ofobjects, persons and events and their relationships, involving all information that canbe consciously and directly accessed. Declarative memory deals with two subtypes ofknowledge:• topic or semantic – this comprises all the cultural structures of the environment(social, personal, professional, technical) and supports its knowledge organisation• episodic – experience with knowledge – the heuristics developed through experiencewith particular topic knowledge.Another approach views knowledge as an expanding multi-layered network of interconnectedknowledge entities (Patel and Kinshuk, 1996). Here size is indicative of the extent of knowledgeand intensity of interconnections the richness and depth of knowledge. This approachdefines the following as constituents of knowledge:• know-how – (operational) predominantly experiential, action driven• know-why – (causal) based on abstraction, reflection driven• know-when/where – (contextual) temporal and spacial context for know-how and knowwhy• know-about – (awareness) environmental context to allow similarities to be perceivedand an extension of understanding to the unknown to be undertaken. Also containspartial and imperfect knowledge of the constituents above.The negation aspects of these, and in particular of the first three constituents is also relevant,so, for example the know-how-not represents learning from mistakes.3.1.2 The nature of model makingModels are valuable for understanding complex information. The concept of model makingor schemata in psychology derives from the work of Bartlett in the 1920s and ultimately Kantin the 18th century. Bartlett (1932) showed how individuals, instead of merely reproducingideas, reworked them in the light of past experience. These observations led to the notionof schema or conceptual model as a mechanism to interact with, explain or make predictions107

egarding new information. Mental models are proposed as the basic structure of cognition(Johnson-Laird, 1983).Both cognitive and constructivist theory assert that internal, mental models and schemataare used to interpret and incorporate experience. While there are many descriptions of whata schema is, all concur on the following characteristics:• it is an organised structure that resides in memory and in aggregate with all otherschemata contains the sum of our knowledge of the world• it exists at a higher level of abstraction than our immediate experiences of the world– recall or recognition allows the placeholders within a schema to be instantiated asrequired• it contains concepts linked together in propositions – this network enables relationshipsto determine schema structure• it is dynamic, amenable to change by general experience or through instruction• it provides a context for interpreting new knowledge as well as a structure to hold it.(Winn and Snyder, 1996)A schema provides a context that affects the interpretation of new experiences within anindividual’s mental model. After Norman (1983), factors that apply to mental modellinginclude:belief system – a person’s mental model reflects their beliefs about the physical system,acquired through observation, instruction or inference. Thus a conceptual model of themental model should contain a model of the relevant parts of the person’s belief systemobservability – there should be a correspondence between the parameters and states ofthe mental model that are accessible to the person and the aspects and states of thephysical system that the person can observepredictive power – the purpose of the mental model is to enable the person to anticipateand understand the behaviour of the physical system. The model must have predictivepowers, either by applying rules of inference or by procedural derivation. The implicationof this is that the conceptual mental model must also include a model of therelevant human information processing and knowledge structures that make it possiblefor the model to be run mentally.108

Mental models are naturally evolving – through interaction with a target system, mentalmodels of that system are formulated. In addition, mental models are rarely self-sufficient(Allen and Otto, 1996) – as the basis for transposed and elaborated structures they arerequired to accommodate novel situations (Gott et al, 1993) in order to operate.This conceptualisation is supported by the early work of Adelson and Soloway (1985): experiencedpractitioners ‘execute’ (run) their designs in order to explore their ideas, but not beingnecessarily good programmers (ie coders) (Curtis et al, 1988), they make use of intermediarymental models as abstractions of the problem elements.As well as being incomplete, Norman (1983) suggests that mental models exhibit the followingcharacteristics:• they are unstable – details are forgotten especially if the model has not been ‘run’ forsome period• they do not have firm boundaries – similar devices and operations are confused• they are unscientific – people maintain superstitious behaviour patterns if it savesmental effort• they are parsimonious – extra physical action is traded off for reduced mental complexity.The value of modelling can be demonstrated through a brief description of an experimentconducted by Chase and Simon (1973). Shown a set-up chessboard for five seconds, experiencedchess players (a chess Master and Class A player) could recreate the positions frommemory better that a beginner when the pieces simulated game play. On the other hand,the beginner performed as well or better when pieces were placed randomly – when, in factthe experienced players’ internalised model of the game could not be used. The experimentillustrated how a model helps organise information and improve efficiency of informationrecall and recognition. Although questioned in later research, this view of model-making isfundamental to the information process perspective of learning.3.1.3 The development of shared meaningCognitive psychology surmises that a person’s response to stimuli is individual, and dependenton the person’s cognitive state and on the mental processes occurring (Dalgarno, 1996). Howknowledge is constructed, therefore, depends on: what is already known; previous experience;how those experiences have been organised into knowledge structures such as schemata or109

mental models and beliefs that the individual uses to interpret the reality of objects andevents encountered.All cultures represent meaning in some way. The constructive process of representing experienceand ideas by symbols allows a creation of ‘semiotic space’ where meaning may benegotiated (Wertsch, 1991). This decentring from first hand experience enables the gainingof new knowledge through an encounter with multiple perspectives gained through social interaction.The attempt to generalise meaning across experience enables reflective abstraction– which in turn may generate new insights, constructions and transformations. At the sametime an attempt is made to coordinate perspectives for dissemination – these, shared, arealso interpreted and transformed and in turn construct further reflections and generalisedmeanings. The more differences there are between interpretations, the more contesting thereis between likely meanings, the more conscious will be the construction process (Greene,1996). Limits are pushed, features formulated and new meanings negotiated.The social context of knowledge construction (and hence the complex activity of dialogue)is seen as necessary to achieve one of the goals of constructivism: the development of sharedmeaning (Novak, 1998), though this social context is viewed differently by radical and socialconstructivists.Social constructivist theories posit that it is through communication with others (and notthe radical view of internal conversation/reflection alone) that meaning is constructed fromexperience (Jonassen et al, 1995; Cronin, 1997). It isnecessarily a social dialogical process in which communities of practitioners sociallynegotiate the meaning of phenomena(Jonassen et al, 1995, p 9).and is based on Pask (1976)’s Conversation Theory. Stakeholders of the knowledge constructionprocess together probe the connections between new information and previous experiencewith the aim of developing a mutual understanding. This mutual understanding can neverbe assumed, rather compatibility achieved.Therefore, from a social constructivist perspective, the viability of our understanding isculturally determined: the culture defines and is defined by what it agrees is ‘known’ and thisconsensus of beliefs is open to continual negotiation (Rorty, 1991). The process of knowledgeconstruction that assumes group interaction makes some assumptions regarding commonknowledge: that a convention has been established so that discourse may be understood andagreement made achievable. While consensually accepted norms are also acknowledged byradicalists, at the other end of this spectrum, distributed knowledge requires the pooling ofknowledge from many sources in order to ‘know’, acceptable to the socialist philosophy.110

In summary:Declaration 1 knowledge is consciously constructed where social interaction highlights differencesin interpretations and contesting between meaning within a sharedsemiotic space.3.1.4 The nature of problem solvingA problem exists if there is a mismatch between the current state and some desired stateand no pre-existing algorithm is available for transforming the current state to the desiredstate. From a positivist perspective, problem solving may be defined as the process of movingfrom a problem state through a series of transformations to a final state of satisfaction oralleviated dissatisfaction (Malhotra et al, 1980). It may also be viewed as a process of selectivegoal-oriented search (Simon, 1979).Problems can be categorised according to whether starting state, ending state, and allowabletransformations are well-defined or ill-defined. They may also include obstacles (Davidsonand Sternberg, 1998) in the problem or the learner/solver which hinder problem-solution.Proving a Euclidean Theorem is an example of a completely well-defined problem whiledesigning a house, a useful software system, a new business process, and writing a teachingstory are examples of completely ill-defined problems (Thomas et al, 2002).The period from the late 1950s to the early 1970s saw the development of a theory of problemsolving (Newell and Simon, 1972). Simon and his colleagues see problem solving as a closingof the gap between performance and goals (March and Simon, 1958), the goal taken asalready known. This focussed on the solving of well-structured problems, where typically theproblem was conceptualised by the solver in terms of a problem space that could be searchedselectively for a solution, using heuristics such as means-end-analysis (Simon, 1979). Thelate 1970s saw the development of these ideas. Even in simple problem domains, severalalternative strategies may be efficacious for finding solutions. Some of these strategies areseen to depend strongly on attention to perceptual cues, others on the structure of goals andsubgoals, yet others on the discovery of sequential patterns of correct moves. The researchdescribes the range of strategies, the circumstances of adoption and their modelling. Researchon the nature of expertise, its storage and evocation from long-term memory has led to themodelling of expert problem solving in specific domains (such as chess), and its extension toother semantically rich domains (Simon, 1979).However, prior to solving a problem, a description must be assimilated or understood. BeforeSimon, Dewey (1910) alluded to the discrepancies between what is and what should be as an111

element of the definition of a problem.problem-solving process (cited in Guilford (1967)):Dewey provides the following formulation of the1. a difficulty is felt2. the difficulty is located and defined3. possible solutions are suggested4. consequences are considered5. a solution is accepted.In the same vein Gigch (2000) provides an interesting and pertinent perspective on problemsolving by focussing on the problem definition or diagnosis stage of the process whileacknowledging that this diagnosis is related to the nature of social reality which affects howthe problem is configured and defined. This diagnosis is seen as a process whose output isa statement outlining the discrepancies (or lack of congruence) between what is and whatshould be. He asserts that to diagnose a problem entails the acquisition of knowledge basedon a hierarchy of cognitive functions:• observation followed by description, explanation, interpretation (relating and comparing)- at this point the problem is reframed and the differing interests of stakeholdersacknowledged• theory formulation – how the problem (now problems) arose• supposition/hypothesis, modelling/representation, experimentation/testing/proof, justification/falsification/proof– problem formulation still evolves, but is reformulated atthis point• evaluation/quantification/measurement, generalisation/laws, understanding• establishing the truth, prediction – on solution• value setting – risks, priorities may be set and acknowledged at any point where a’selection’ is made. These are elaborated through ever higher levels of abstraction andlogic• creativity and imagination, epistemology/ethics/aesthetics – the reasoning methodsused to formulate the problem, the effects of the proposed solution and the tradeoffspossible.112

To solve a problem produces a series of statements of ever-increasing complexity as theproblem is processed through the same cognitive functions. At first the problem is seen tohave narrow scope and low complexity. By its second reformulation, and based on enhancedmeaning and understanding, the problem has medium scope and complexity. Finally thelatest version reveals a problem of wide scope and high complexity as well as a higher levelof abstraction and logic. In the process, different types of knowledge are generated, allowingfor, amongst other factors, the following caveats: the problem definition is emergent; novicesand experts hold different world views (Gigch, 2000).He states that participants must remain sensitive to these progressive modifications. Althoughpositivist in nature, this view of problem diagnosis would seem to be relevant to theconsideration of the RE process, in that ill-structured problems require problem-solvers toengage firstly in problem-finding (Covington, 1987).Research during the 1990s placed primary importance on the thinking strategies of problemsolvers, and identified the following subprocesses (Davidson and Sternberg, 1998):representing the problem – identifying the most relevant features and creating a mentalmap of the components based on related information in long term memory, tacit andmetacognitive knowledgeplanning strategies – developed through review and selection in order to anticipate consequencesand risksovercoming obstacles – especially stereotyping or the need to enable incubation. Thisoften requires combining previously unnoticed elements in new ways and discoveringnew relationships between knowledge and problemexecuting plans – in order to monitor and evaluate strategy application.An alternate process breakdown, which stems from a specific type of problem solving that focusseson diagnosis and repair in systems (not necessarily computer systems) is troubleshooting.Engineering, computing and medicine involve troubleshooting.Troubleshooting can be divided into two major stages:hypothesis generation – identifying one or more potential faults. During this stage theway information is perceived and organised is critical; representation of the problem(schemata, mental models) determine pattern matching. Theories of perception areimportant here113

hypothesis evaluation – testing and making corrections. Procedural skills and metacognitionplay an important part at this stage, since individuals must make decisions aboutwhat strategies to use as well as monitor progress.Morris and Rouse (1985) review the research on troubleshooting and conclude that troubleshootingperformance degrades as the system increases in complexity or time constraintsare imposed; instruction in theoretical principles is not an effective way to train good troubleshooters;proceduralisation is the most effective way of ensuring a strategy is employed;explicit guidance on applying system knowledge or problem solving strategies improves performance,and task-related knowledge is more relevant than aptitude.These findings are supported by the work of Gott et al (1993). They describe an advancedlevel of problem solving performance that is at a premium in knowledge-rich domains whereill-structured problems are plentiful. This is based on a growing body of evidence thatsuggests adaptiveness in generalising knowledge – particularly in the context of complexproblem solving tasks – is strongly influenced by the quality of knowledge representation.Adaptive expertise is characterised by principled representations of knowledge and skillsrather than representation dominated by surface features. This concept is linked to Norman(1993)’s ‘runnable’ mental model – the main causal connections of the components of asituation are specified so that running a mental model invokes an explanatory theory to use ininstantiating a given problem situation. Gott et al (1993) posit that this adaptive/generativecapability suggests the performer not only knows the procedural steps for problem solvingbut understands when to deploy them and why they work.Declaration 2 system complexity degrades some types of problem solving performance,but task-related knowledge, with guidance on applying domain and strategicknowledge, improves performanceIssues of transfer are also highlighted in the problem solving literature. Research duringthe late 1970s and into the 1990s found (through the use of isomorphs of common problems(eg, tic-tac-toe, tower of hanoi)) that skill gained on solving one form of the same problemdid not always transfer. Little transfer from harder to easier isomorph was demonstrated,while changes in the cover story (such as a change from active to passive voice) was enough toalter the problem solving time. In addition, the problem representations assimilated from theinstructions caused changes in problem difficulty (and hence ease of solution). The discussionby Reed (1993) address issues of transfer.Research even in the seemingly well-structured domain of mathematics shows that the relationshipsbetween task performance and conceptual understanding are neither direct nor114

simple (Fuson (1992) provides a review of this area). Even when the same procedural knowledgewould seem to be called for, the mastering of different classes of the same problemdoes not occur at the same time. As a consequence, success in the application of a givenprocedure, or its use in a given problem context, does not ensure the recognition of the procedure’srelevance to a problem when the construction of different mental models in responseto different types of problems is called for (Gobet and Wood, 1999). A consequence of thisis that the theoretical interpretation and explanation for both success and errors is not simplewhere there are one-to-many mappings between procedural performance and conceptualunderstanding.Problem solving skills appear to be related to many other aspects of cognition (Frederiksen,1984), such as schemata, pattern recognition and creativity (developing new solutions).Problem solving itself may also be viewed as a creative process, as well as requiring creativity.Grundy (1987), in describing a Creative Problem Solving Model identifies five major steps:fact-finding; problem finding; idea finding; solution finding; acceptance finding. Mumfordet al (1998) point to evidence that the ability to solve problems creatively is based on thefollowing key processes: definition and structuring of the problem situation (e.g., restatementof the problem); information acquisition or encoding in order to select and organise relevantinformation; and combination and reorganisation of knowledge to address the problem.In all of this, the issue of transfer is highly relevant, while the effectiveness of even generalproblem-solving skills appears to improve when students learn self-regulation metacognitiveskills (Hannafin et al, 1996):Declaration 3 metacognitive self-regulation training helps students transfer learning tomore difficult problems within the same domainIn describing a computational representation for problem solving, Goldstein (1980) suggeststhe problem solving task requires multiple perspectives on its subject matter, achievedthrough playing the following roles:mathematician a set of mathematical and probabilitistic skills (eg argument by elimination,sets to represent hypotheses, sequential organisation of a set of heuristics) are appliedto theorems of a domain (the rules that apply). Problem solving has an element oftheorem provinghistorian the use of analogies, generalisations, corrections and refinements, to construct newknowledge based on the relationships between knowledge elements already possessedepistemologist data and hypothesis representation: insight into the breadth of knowledge115

equired by the task is required. Ability to develop a representation of the problem andthe problem solving process is a component of problem solvingpsychologist ability to estimate the cognitive load of a problem. An appreciation of therelative complexity of the material is requiredmanager organisational skills are required to manage large numbers of individual skills.This management includes: selecting appropriate skills sets to apply; ordering theirapplication; removing inappropriate setslearner learning skills need to be applied to evaluate the appropriateness of the knowledgeconstructedscholar the factual (declarative) knowledge used to justify a problem solving approach. Theunderlying governing principles are definedbookkeeper the episodic structure of memory determines whether diverse sets of experiencecan be organised for recall and application.Current perspectives on problem-solving suggest that the priority is to enable learners toacquire the habits and dispositions of expert problem solvers in the domain, rather thanfocus on mastering, in a preordained sequence, sets of facts and procedures that representthe domain content. This shifts the focus from moves or changes in conditions to reach asolution to emphasis on individual conceptualisation of the problem and internal subprocessesand addresses issues of problem organisation and understanding.Declaration 4 effective problem solving currently focusses on expert behaviour and resultsfrom the development of rich knowledge structures and metacognitivestrategies.3.1.5 The development of expertiseUnderstanding expertise is important because it provides insights into the nature of thinkingas well as problem solving – experts have acquired extensive knowledge that affects whatthey notice and how they organise, represent, and interpret information in their environment.This, in turn, affects their abilities to remember, reason, and solve problems (Bransford et al,2000).The issue is that there is marked difference between the fragmentary knowledge structures ofnovice understanding and the integrated knowledge structures which underpin more robustknowledge and flexibility exhibited by the expert knower (Wood, 1999).116

Knowledge construction may be viewed as a advancement towards expertise, with step-likeprogressions via experiences accelerated by connecting to the conceptual understanding (mentalmodels) of experts (Bruner, 1985). The richness of expert knowledge, in terms of knowledgestructures and their management, is explored through extensions to the chunking theory(Chase, 1973) 1 . Experts are seen to store into long-term memory faster than proposed bythis theory. This is explained through the development of templates, created when a chunkexhibits variation in content (Gobet and Simon, 1996). It has also been suggested that relational(semantic) links are constructed between the chunks, providing mechanisms for thegrowth of associative memory (Gobet, 1996), combining low-level with high-level aspects ofcognition. Thus the richness of the ‘indexing’ and the density of the relationships to otherchunks help determine the level of conceptual understanding attained.Alternatively, Norman (1993) argues that human cognition lies in our ability to constructexternal cognitive artefacts through the use of symbols designed to maintain, display oroperate on information in order to serve representational functions. Without the supportof external representations, internal representations (schemata, mental models) are typicallysimplistic, incomplete, fragmentary, unstable, difficult to manipulate or run, and lacking infirm boundaries (Allen and Otto, 1996). This supports the work of Chi et al (1982) whichconcludes that experts relate phenomena in a domain to higher-order principles and encodethese along with solution procedures. Their work also suggests that if a concept is categorisedinappropriately (eg relating a concept to incorrect mental models), it cannot be subsequentlyused. This emphasises the importance of ‘correct’ learning.The external representation developed may confer gains in efficiency – simple stimuli andsmall amounts of energy expended on those aspects of the environment that can yield largereturns trigger complex responses. Efficiency is further increased through offloading informationand its processing into the environment itself, thus exploiting information reflected inthe structure of the environment. This model assumes representation is distributed betweenthe environment and the brain rather than placing an emphasis on relatively complete mentalmodels and schemata. Higher order learning may then be defined by the degree to which itpermits individuals to benefit from the externalisation of information storage and processing.A higher level of perception is seen to characterise some models of expert behaviour – theexpert is said to leave information in the environment if it remains easily accessed. Thisperception involves selecting and attending to some sources of information at the expense ofothers. The selection is based on ‘the price paid’ – the risk – for non-selection (Norman andRumelhart, 1975).1 chunks are a familiar pattern that can be used as a unit, linked to suggestions for plans, moves, and othertypes of information, with about 50 000 chunks needed for a (chess) master’s memory117

An additional aspect of expertise relates to the cognitive ability to plan, defined as themanagement of knowledge structures. Expert planning is seen to be based on the identificationand exploitation of past situations and basic schema recognition. Experts followwell-structured plans towards a solution until knowledge is no longer readily available, atwhich time opportunistic behaviour takes over. Three principal characteristics of plans are:their heuristic nature – based on knowledge available; the optimal use of memory – throughthe process of abstraction, and a high level of control – plans enable the emergence of anactivity that cannot be derived from the detail of the activity being processed (Robillard,1999).In summary, experts:• construct representations of the problem based on higher-order principles and test solutionsbased on these• work from problem schema forwards to solution, based on known or recognised elements• plan solution strategy application and self-monitor progress.Declaration 5 adaptable chunking of knowledge structure and their management underpinsexpertise.3.1.6 Metacognition and reflectionCognitive psychology emphasises that effective knowledge construction is related to an awarenessand ability to control one’s thought processes. This acknowledges the place of metacognitiveand reflective strategies in knowledge construction – the mind does not just consist ofa collection of skills: a decision is required on which strategy or skill to employ in a particularsituation. This is higher order thinking (Flavell, 1979) to actively control the cognitiveprocesses engaged in thinking and acquiring knowing.Metacognition may be defined as awareness of one’s own cognitive processes (rather than thecontent of those processes) together with the use of that self awareness in controlling andimproving cognitive processes (Biggs and Moore, 1993). According to Flavell (1979) (one ofthe first to introduce the concept) metacognition consists of both metacognitive knowledgeand metacognitive experiences of regulation. Metacognitive knowledge refers to acquiredknowledge about cognitive processes, knowledge that can then be used to control cognitiveprocesses:knowledge of person variables – refers to knowledge about how human beings learn andprocess information, as well as individual knowledge of one’s own learning processes118

knowledge of task variables – include knowledge about the nature of particular tasks ormore generalised knowledge about types of task as well as the processing demands thatwill be placed upon the individualknowledge about strategy – as well as conditional (contextual) knowledge about whenand where it is appropriate to use such strategies. The basic metacognitive strategiesinclude:• connecting new information to existing (personal) knowledge• taking conscious control of learning – demonstrating awareness of the specificobjectives of a particular task• planning and selecting strategies – and weighting the cost of each against thebenefits of the goals to be achieved• monitoring the progress of learning – using available resources and adapting strategiesas required• correcting errors – using fixit strategies to address difficulties• analysing the effectiveness of learning strategies• changing learning behaviours and strategies as required – flexibility across contextsis spontaneous(Ridley et al, 1992)and are seen to interact with domain expertise: enhancing learning where domainknowledge is weak, and task motivation: learners with a mastery goal orientation (ieseeking to demonstrate ability through development of new skills) develop sophisticatedmetacognitive skills.Metacognitive strategies include the development of cognitively flexible processing skills andthe acquisition of knowledge structures that can support them. Brown (1980) suggests thatmetacognitive deficiencies are the problem of the novice: a function of inexperience in a new(and difficult) problem situation.The implication of a metacognitive decision-making process include:• that there is acquired knowledge about cognitive processes; knowledge that can thenbe used to control cognitive processes• it is actively used in a strategic manner to ensure a (present or future) goal is met• a wide range of skills from which a decision may be made exist – hence wide experience119

• that a process of reflection (both in and on action) exists.The extensive literature on metacognition within cognitive psychology points to the developmentof self-monitoring and self-control of the learning process. Many models of metacognitionstress the development of strategies of efficient processing (eg Sternberg (1987)’s metacomponents),others emphasise the process of perceptual tuning, in the sense of Schön (1987)’sreflective practitioner developing the ability to ‘see as’. Yet others refer to reflexitivity – directingreflection back on the learner’s own efforts to know (Duffy and Cunningham, 1996).The essence of reflexivity is abduction through which beliefs are created or revised to caterfor surprising experiences. Individual awareness of the state of knowing is enhanced throughthis process. Further awareness of the cultural origin and mediated nature of beliefs allowsa reconsideration of the metaphors/models that constitute knowledge. When concepts ormodels are introduced, their definition often requires more than the simple description ofthe world in terms of these concepts. It further requires an analysis of how the concept wasused, introduced, and possibly modified to be more adequate. Reasoning in this process ismetacognitive – not about the world as it is but about the relation of knowledge to the worldand to the goals pursued. Wright (1992) suggests that self-reflection involves the abstractionof meaning and is an interpretive process aimed at the understanding of reality. Humanreflection is seen as the key to understanding and creating a world in which we coexist withothers – and in which many perspectives are valid.Flavell (1979)’s definition of metacognition involves thinking about thinking and by thislogic it must include learning about learning, associated with the idea of metalearning. Metalearningprovides an essential capacity for people to change. Thus metalearning (Biggs,1985) requires:• a framework for evaluating and incorporating new experiences and information basedon a fluid mixture of experience, values, contextualised information (ie learning)• imagination as a means of viewing and anticipating the future – a necessity if one is totake control of one’s own learning and create plans and strategies in order to achievedesired goals.The acquisition of metalearning capacity reflects an empowering skill in learning that isquite different from, and superordinate to, the acquisition of other complementary learningor study skills. Meyer and Shanahan (2004) suggests it resonates with other operationalisedconcepts such as study orchestration, self-regulation in learning, locus of control, learning‘style’ versatility and student-centredness.120

Opportunities for this type of learning are enhanced where the characteristics of the disciplinealign with the characteristics of the learning environment provided formally – it followsthat learning environments designed to promote capabilities, behaviours and creative habitsof thinking and learning necessary for professional practice pay particular attention to theprocesses for learning in that discipline.In summary:Declaration 6 effective knowledge construction is enhanced by metacognitive decisionmaking,requiring a wide range of skills and experience and a process ofreflection. Novices exhibit metacognitive deficienciesDeclaration 7 aligning the learning environment with characteristics of the discipline (interms of cognitive and metacognitive skills) enhances learning.3.1.7 The place of creativityThe idea of creativity embraces a multiplicity of notions, including imagined (conceptual)ideas, development of schemata (constructs, analogies, diagrams, etc) emanating from theideas, physical execution of ideas (the activity of making, performing, etc), and createdproducts resulting from the ideas (works of art, manuscripts, performances, etc) (Brophy,1998).While there are many views about the nature of creativity, there is some agreement that thecreative process involves an application of past experiences or ideas in novel ways, with thecognitive skills of fluency, flexibility, visualisation, imagination, expressiveness and opennessunderlying creative behaviour. These may be personality characteristics, learnt or situational.There are many taxonomies of creativity in the literature, many originating in philosophyand psychology (Bergquist, 1999). General broad agreement on a hierarchical arrangementof types of creativity exists, implying that creativity is generally understood to be a creativeprocess. The highest level addresses conceptualisation, with development of schemata(derived from a concept) at an intermediate level, and physical execution (derived from theschemata) at a lower level (Bergquist, 1999).Although creativity is a concept surrounded by a number of misconceptions (including thatit is limited to a few individuals), research show that creative thinking is a universal ability.The focus on creative research has been on personality traits (eg the work of Amabile (1996))and hence its innateness.Amabile presents a general theory of creativity, based on:121

• preliminary assumptions and observations – for example◦ talent, education and cognitive skills do not appear sufficient for high levels ofcreativity, nor are particular clusters of personality traits◦ a great many outstanding creative individuals have described the phenomenonof ‘incubation’: after ceasing to consciously work on a difficult problem, theysometimes experience an apparent flash of illumination, during which the solutionappears to them unexpectedly• three components:◦ domain relevant skills – the more skills the better, and the ability to imagine/playout situations◦ creativity-relevant processes – including breaking perceptual (the way you perceivea situation) and cognitive (the way you analyse) set and breaking out ofperformance ‘scripts’, suspending judgement, knowledge of heuristics (eg use ofanalogies, when all else fails try something counter-intuitive, make the familiarstrange), adopting a creativity inducing work style (eg tolerance for ambiguity,high degree of autonomy, independence of judgement)◦ task motivation – intrinsic motivation is the most important component.More recent work has turned to social and environmental factors that influence creativity.Newer definitions describe creativity as the confluence of cognitive processes, knowledge,thinking-style, personality, motivation and environment over the life span (Adams-Price,1998; Sasser-Coen, 1993).From their summary of empirical studies of the creative process, Edmonds and Candy (2002)distill the following key activities for creativity in problem solving:exploration of ideas, knowledge, and options, based on• breaking with conventional expectations, whether visual, structural, or conceptual,is a key characteristic of creative thought• immersion – the complexity of the creative process is served well by total immersionin the activity• holistic view – the full scope of a design problem is only fully embraced by takinga holistic, or systems, view. The designer needs to be able to take an overviewposition at any point and, in particular, to find multiple viewpoints of the data oremerging design important122

• parallel channels – keeping a number of different approaches and viewpoints activeat the same time is a necessary part of generating new ideas.Exploration involves accessing source data, comprising different types of knowledgethat may be examined, assessed and interpreted in terms of the primary goals of thecreative knowledge worker: for example, addressing customer requirements, problemspecifications, design briefs. This is an open process, possibly without observable directions.However, the thoroughness and selectivity of the activity is critical to thequality of the generative stage that follows immediately and to the subsequent iterationsthat take place between those stages. Having a comprehensive set of knowledgesources readily available is extremely advantageous. Knowing where to look and howto select the knowledge is even more important. There is often rapid iteration betweenthe exploration and generation activities depending on the domain or problem areaidea generation – the generation of possible solutions or approaches to the brief or problemdefinition draws upon the results of the initial exploration. Problem formulation, asdistinct from problem solving, is critical to the effectiveness of the solution space thatis defined. It draws upon a wide range of analogous cases often outside the immediatedomain. This has been characterised as an ability to make remote associations.Creativity is demonstrated by the generation of many potential solutions (divergentthinking) instead of converging quickly toward a single and (usually) familiar solutionthat is not necessarily the optimal one. The ability to consider parallel lines of thoughtand to select and transform the results to meet the demands of a different situation isa critically important aspect of solution generationevaluation – evaluation involves taking the results of the generative activity and testing thecandidate solutions against a set of constraints. This leads to modifying, reformulating,or discarding solutions depending on the feedback. Selection of the optimal solutionmay involve a number of trade-offs against the constraints that are applied especiallywhere the product is a complex one. The application of tight constraints may beconsidered conducive to creative solution finding and thus evaluation is a vital part ofthe creative process. Evaluation may be viewed as a pervasive activity that takes placefrom the exploration phase onward. The use of expert knowledge in evaluation hasbeen identified as an important aspect of successful solution finding.The development of creative potential is greatly influenced by environment. Negative influencesinclude: working under surveillance; restricting choices; working for inappropriateextrinsic rewards; fearing failure, judgement or appearing foolish; having to find the ‘right’123

answer, being evaluated; working under time pressure; competing (Amabile, 1996). Positiveinfluences include: encouraging assertion of ideas; no reliance on order and training; no fearof failure; providing time and resources; developing expertise; giving positive, constructivefeedback that is work or task focussed; encouraging a spirit of play and experimentation;providing a mix of styles and backgrounds with opportunities for group interaction; makinga safe place for risk taking; allowing free choice in task engagement; offering rewards thatrecognise achievements or enable additional performance but maintain intrinsic motivationrather than controlling behaviour.The activities identified as creativity frequently involve acquiring new methods or skills andusing expert knowledge. However, accessing the necessary skills is seen as one of the difficultiesin applying creativity. Thomas et al (2002) suggests there is a widening gap betweenthe degree of flexibility and creativity needed to adapt to a changing world and the capacityto do so. Difficulties are based on:• individuals or groups not engaging in effective and efficient processes of innovativedesign. As examples of structuring failure, people typically fail to spend sufficient timein the early stages of design: problem finding and problem formulation, then often bringcritical judgment into play too early in the idea generation phase of problem solving. Asanother example, empirical evidence shows that people’s behaviour is path-dependentand they are often unwilling to take what appears to be a step that undoes a previousaction even if that step is actually necessary for a solution (Thomas et al, 1977)• the necessary skills, talents, and knowledge sources are not brought to bear on the problem;evidence suggests individuals have a large amount of relevant implicit knowledgeand that providing appropriate strategies, knowledge sources or representations cansignificantly improve an individual’s effectiveness in problem solving and innovation.Greene (2002) suggests that existing bodies of knowledge that can be readily accessed,examined, visualised, related, and discussed, would seem to be a key component inenabling creativity• the appropriate level, type, and directionality of motivation are not brought to bear.The effect of increased motivation interacts with personality, complexity, novelty, andwhether the motivation is intrinsic or extrinsic (Amabile, 1983).The relationship between creativity and instruction has also been a focus of research: schooland work environments may be seen to inhibit the transformation of early talent (with mostyoung children under 5 years being very creative) to adult creativity (Amabile, 1996). Schoolingat the age of starting formal education emphasises logical rather than divergent thinking,124

with the value of conventional behaviour, well-defined problems and good grades emphasised(Albert, 1996). In addition, many cultures encourage respect for the past and discouragedisruptive innovations: promoting widespread creativity raises expectations that may changeemployment patterns, educational systems, and community norms.In higher education, creativity is generally accorded higher order ability status: a problemrequiring a creative solution is likely to be challenging, and therefore may create motivationalproblems. For example, some individuals may be disinclined towards creativity because of afear of failure, and a lack of confidence. Similarly, Gardner (1999) argues that creativity is acoincidence of many factors, which includes the discipline to master a domain and a lack ofhindrance from the fear of failure.While creativity is certainly demonstrated in well-defined problems such as theorem provingand chess, ill-defined problems such as design require creativity quintessentially (Akin, 1990;Thomas et al, 2002).There has been considerable research into how designers carry out design activities. Akin(1990) identifies three classic conditions in creative acts such as design:• the recognition step – the ‘aha!’ response• the restructuring step – a change in viewpoint leads to a major breakthrough• development of procedural knowledge – regarding solution strategies.These accord with Wallis (1926)’ Model of the Creative Process, which identifies preparation,incubation, illumination, and verification as occurring in all creative activity. Csikszentmihalyi(1996)’s analysis emphasised the social nature of creativity. He stresses the benefits ofconsultations with other domain experts, receiving empathic encouragement from emotionalsupporters, and the necessity for dissemination within the field.Common characteristics of creative individuals have been identified by researchers such asGuindon (1989, 1990) and Maccoby et al (1991), with similar ways of thinking and workingexhibited. Most are ‘holistic thinkers’ in the sense they look for an overall broad scope beforemoving into specific detail. Other studies indicate that designers often proposes severalcandidate solutions early on in order to better examine the problem, and impose constraintsthat reduce the number of solutions and help generate new concepts. Boden (1997) makes agood case for the claim that changing a constraint might be at the core of creative thinking.In summary:Declaration 8 ill defined problems require creativity quintessentially. However, the conditionsfor creativity, which include incubation (the ‘aha’ factor) may be125

inhibited by environments that engender fear of failure, while the potentialis enhanced by holistic thinking.3.2 Theories of learningThe process of knowing can be seen from the preceding section to be multi-faceted: modelmakingand problem diagnosis/solution are enhanced through mastery of appropriate strategies,both at a cognitive and metacognitive level. This toolkit is enriched through exploitationof creativity – and metacognition – enhancing elements within a social environment.In the wake of the cognitive revolution, learning theorists and researchers treated learningand knowing as if they were self-contained processes taking place in the confines of individualminds (eg Newell and Simon (1972)). Intelligence, giftedness, talent, ability, and cognitionwere also considered features (or possessions) of individual minds. This line of thinking,rooted in Cartesian dualism, is founded on the separation of the learner from the learningcontext, effectively isolating the body from its mind, the self from its world, the content fromits context, and ability from those situations in which one is competent.However, many contemporary thinkers from a variety of domains describe knowing not simplyas a psychological construct existing in the head but as an interaction of individuals andphysical and social situations (Brown et al, 1989; Greeno, 1998; Sternberg and Horvath,1998). These studies, in which people perform differently in different settings even whenperforming comparable or the same problems, challenge the validity of Cartesian dualism.The spectrum of learning theories consists of many approaches or ways of explaining howhumans learn. They act as a framework for both organising knowledge and conductingresearch in an attempt to explain how knowledge is acquired. If viewed as a continuum,the extremes are represented by the theories of behaviorism and constructivism, trying toaddress the same concepts, but bipolar, based on their views of knowledge acquisition andof intervention by tools of learning. The stance taken on the learning process influences thestrategies adopted in the learning environment.From an empiricist epistemology, behaviourists based their theories on experimental observationof behaviour, applying laws and principles to real-world situations. The focus of thistheory is on conditioned response to stimuli, which Skinner (1953) expanded to include theimportance of reinforcement. A powerful influence on learning, many models of learningare founded on behaviourist principles. Examples include direct instruction models and theprimacy of exams as a mechanism for measuring learning behaviour. As an effect of behaviorism,learning objectives theoretically involve some observable evidence that learning has126

occurred.A shift from the analysis of observable responses related to environmental conditions to thedevelopment of meaning from daily experience (Bruner, 1990) led to exploration of informationprocessing (cognitivism) as a means of developing meaning and problem-solving. Thismodel assumes that learners use a variety of different systems, and that both practicingin realistic situations and the ability to learn problem-solving strategies by thinking themthrough are important. The basis of schema theory is found here, as is the work of Gagné (egGagné and Glaser (1987); Gagné et al (1992)) on teaching strategies and learning outcomes.However, dissatisfaction with both information processing and transmission as models thatallow decontextualised learning has led to the dominance of constructivism as an approachthat promotes learning to reflect discipline-based practice. Several variants of constructivismexist, all stressing the importance of higher order thinking:radical – derived from Piaget’s cognitive-development theory, all knowledge construction isequally correct (though partially judged against consensually accepted norms), with theprocess internal rather than logic based (Glaserfeld, 1996). The focus is on problembasedlearning, pertinent to current learning modelssocial – learning is socially constructed and distributed within the community (Vygotsky,1978)emergent – learning may be analysed from either a social or psychological (ie individual)perspective (Cobb and Bauersfeld, 1995). This view co ordinates both radical andsocial perspectivesneo-marxist – the immersion in a community of practice is fundamental (Lave and Wenger,1991) and enables of the learner to make the transition from peripheral to full participation.The focus is on apprenticeship and situated learning, with no strict boundarybetween intra- and inter-cranial aspects of human cognitionholistic – learning is an integration of old and new experience and is transformational (ratherthan incremental) (Bruner, 1990). However, to be effective, the learning experienceneeds to be student-directed and authentic.Constructivists draw from the convergence of several learning ideas to produce an environmentamenable to the construction of knowledge, with more recent work looking at the importanceof learner engagement as a component of the learning environment. In opposition toconstructivism, learning theory in the US in particular emerged from an objectivist tradition,where the goal is a complete, correct and in-depth understanding of predetermined meanings.127

While behaviourism and cognitivism are different theories, they share some common groundin their adherence to an objectivist epistemology.Cognitivist theories of representation (such as mental models and schemata) have influencedlearning theory in a variety of ways, including the focus on information mapping as a meansof imposing structure on knowledge (Winn and Snyder, 1996).Figure 3.2: Model-making in learning (Norman, 1983)Both constructivist and cognitivist theory assert that internal, mental models and schemataare utilised to interpret and incorporate experience. As Figure 3.2 illustrates, the considerationof mental models within the learning process requires consideration of the:target system – the system the person is learning or usingconceptual model – invented by teachers, designers, scientists, engineers to provide anappropriate representation of the target system (either asserted or perceived to beaccurate, consistent and complete)the user’s mental model of the target system, constrained by technical background, experienceand cognitive stateconceptualisation – the model of the user’s mental model.(Norman, 1983)In addition, there has been much discussion on the impact of external knowledge representations(eg diagrams, text, numbers and abstract systems of symbols) on the processes oflearning, discovery and reasoning. As an example, research into image encoding has providedevidence that128

• pictures and graphics contain information that is not contained in text• information shown in pictures and graphics is easier to recall as it is encoded both aspropositions and images(Winn and Snyder, 1996).Although attempted from both behaviourist and constructivist perspectives, a generalisedtheory of learning has not been achieved. Wood (1999) proposes reasons to explain thelimitations on generalisations of learning. In effect these summarise the factors identified inthe previous section of this chapter as necessary for higher order thinking:• expertise-based abstraction within a domain of discourse: conceptual understanding ishallmarked by the ability to make effective use of multiple systems of representationas tools to support thinking, reasoning and problem solving. These systems are culturalinventions constructed to serve as a means for representing and thinking aboutthat domain. These then support the identification of patterns and regularities inthe phenomena they were designed to represent, which in turn enable more abstractrepresentations to be constructed. However, although the construction of multiple representationsplays a formative role in the process of conceptual understanding:◦ different species of representation invite different perspectives on a domain andvary in the process of reasoning they are likely to promote – although equivalentinformation may be represented, different systems differ psychologically in whatreasoning, insights and problem solving they support◦ a facility with multiple representations differentiate novice from expert behaviour.Integrated knowledge structures constitute the relations between domain-generalprinciples and the surface appearance of a given task. The expert has the means tomediate between knowledge of abstract principles and observations of phenomena◦ the translation within and between different modes of representation makes ideasmeaningful, and hence leads to conceptual understanding• the issue of transfer: exposure to their use in problem solving does not automaticallyguarantee understanding that it is possible and useful to translate across two or morerepresentations: the connection between them must be turned into a problem space.In addition, knowledge that is overly contextualised can reduce transfer while abstractrepresentations of knowledge can help promote it129

• the importance of socialisation: the discursive processes, such as negotiating, interpreting,explaining and evaluating need to be unlimited so as to not restrict the learner’sconstruction of conceptual understanding. Conversation, both internal and external,provides an essential part of the glue that binds otherwise fragmented understanding -the role for discourse in learning and understanding is vital.There is no doubt that the amount of information to acquire increases substantially wheremultiple representations are used to enhance learning. This heavier learning toll, a ‘criticalmass’ required to be useful and the potential that usefulness may occur only in the longterm, are issues considered by models which apply these theories. In addition, realising thatit is possible and useful to translate across two or more representations is not automaticallyguaranteed by exposure to their use in problem solving (Wood, 1999): the connection betweenthem must be explicit.This learning brings together both a knowledge of the external tools that are being used toreason with and, perhaps implicitly, a sense of how the use of those tools fits the constraints ofboth situations and one’s own cognition (Wood, 1999) – gaining flexibility with reasoning andproblem solving with more than one system of signs and their systematic representation goeshand in hand with enhanced self regulation and a ‘sharper’ metacognitive understanding.In summary:Declaration 9 flexibility with multiple representations enhances conceptual and metacognitiveunderstanding and models expert behaviour. One source for gainingmultiple representations is through the multiple perspectives provided insocial interaction. However, a heavier learning toll is implicit and the connectionbetween the representations must be made explicit for the learningto be enhanced.3.2.1 Learning as transferOne of the fundamental concepts in learning is transfer – the ability to apply somethinglearned in one situation to another setting (Kearsley, 2000). Almost all theories of learningaddress transfer: cognitivists discuss it in terms of the restructuring of knowledge, schemataand mental models, social constructivists, in terms of modelling or imitation. A criticalaspect of transfer is whether learning can be applied across many settings, or whether it isalways context-specific. Despite an acknowledgement of its importance in the educationalliterature, the underlying processes and basic theoretical background are little understood130

(Salomon and Perkins, 1989), while the few theories dealing explicitly with transfer have thestatus of sets of hypotheses (some discussed in Detterman and Sternberg (1993)).The empirical evidence for the transferability of knowledge and skills to new task situations isvery mixed: while not claiming tranfer is impossible, studies such as that of Thorndike (1924)showed that certain specific kinds of instruction do not produce transfer (Simon, 1980). Aslate as 1993 Detterman (1993, p 15) notedin short, from studies that claim to show transfer and that don’t show transfer,there is no evidence to contradict Thorndike’s general conclusion: transfer is rare,and its likelihood of occurrence is directly related to the similarity between the twosituations.This is partly due to the historic focus on general transfer – abstract knowledge learnt in theclass (ie in a specific context) applied to a broad variety of other tasks (in as broad a varietyof contexts). More recently researchers have focussed on the importance of socioculturalcontext as an element of transfer (Brown et al, 1989) and caution that a holistic approach(task, learner and context) is advisable (Marini and Genereux, 1995).The level of transfer that can be achieved is debated. Clark and Voogel (1985) suggesta continuum from near (a great deal of similarity exists between training conditions andspontaneous use) to far. Conditions which make transfer possible include:• it requires that some of the processes or knowledge be essentially identical. The ‘recognitionof similarity’ seems to be a crucial issue, the ‘kick’ Detterman (1993) refers toin describing cases where transfer could be seen• to secure substantial transfer of skills students need to be made explicitly aware of theseskills, abstracted from their specific task content: a focus on the underlying principlesis necessary (Patry, 1998)• transfer is strongly influenced by mental modelling – prior models become interpretivestructures that, when inadequate, are flexibly used as the basis for transposed andelaborated structures to accommodate novel situations (Gott et al, 1993)• practise with adapting the skill in response to situational context (Patry, 1998) – thisis a process of ‘re-assembling’ knowledge gained (especially relevant in domains featuringill-structured problems) based on an understanding of individual style (implyingmetacognitive approaches) and an appreciation of the authenticity of the approach(implying peer/domain evaluation).131

Given these, a moderate level of transfer can be expected, in particular if there is a learningfocus on strategic knowledge. This includes strategies for identifying and meeting subgoals,procedural steps and metacognitive strategies for directing, monitoring and evaluatinglearning (Larkin, 1989). These accord well with concepts of expertise and problem-solving,previously discussed.So, in summary:Declaration 10 transfer is fundamental to learning. A holistic approach (task, learner,context) is required to achieve transfer, which should preferably be ‘near’.The principles of the learning object (eg skill) must be made explicit to thestudent, who needs practice in adapting the specific learning to differentcontexts, thereby reassembling knowledge gained. Transfer is enhancedwith a learning focus on strategic knowledge.3.2.2 Acquiring membership of a disciplineEach discipline and domain of specialisation creates its own dialect of discourse in order tocounter the ambiguities of normal language. The relationships between the ‘discourse communities’(Swales, 1990), ‘communities of practice’ (Lave and Wenger, 1991) and disciplinary‘genres’ (Freedman and Medway, 1994) have produced an industry in the literature (Candlin,1998). To learn a discipline, one has to learn an appropriate language of technicality andabstraction (Daniels, 1995). The learner therefore must handle the representation system aswell as the ideas they represent. At an initial level this type of learning requires internalisingof elementary concepts and their associated set of assumptions, in order to determine a term’smeaning. These are later taken for granted. At an advanced level, abstraction is achievedthrough association of elementary concepts with higher order concepts. The ease with whichthis learning takes place is to some degree dependent on the ‘generic integrity’ found withinthe discipline.Constructivist theories posit that it is through communication with others that meaningis constructed from experience (Jonassen et al, 1995; Cronin, 1997). It is necessarily asocial dialogical process in which communities of practitioners socially negotiate the meaningof phenomena. Since the discourse of a discipline is central to the way its knowledge isconstructed and transmitted, the means by which learners acquire discursive knowledge isimportant. While expert members of a discipline share knowledge about discursive practicesin their community, this is mostly tacit, and, traditionally, given little emphasis within the(formal) educational environment.132

Since constructivism rejects any direct verification of knowledge by comparing the constructedmodel with the outside world, its most important issue is how to choose between differentconstructions. Without such a selection criterion, constructivism would lapse into absoluterelativism: the assumption that any model is as adequate as any other. Constructivism viewscoherence as a criterion of truth. This means that the individual will try to build models thatare coherent with the models already possessed, or which are received through the senses orthrough communication with others. Since models are only compared with other models, thelack of access to exterior reality no longer constitutes an obstacle to further development.Declaration 11 dialogue in the language of the discipline is an important mechanism foracquiring membership within a discipline.3.3 Models of learningLearning should not only take us somewhere; it should allow us later to go furthermore easily(Bruner, 1960, p 17)Models of learning describe a learning environment and address approaches to the managementof aspects of learning. Unlike theories of learning, models are dependent on thediscipline, level (eg post-secondary), age of learner and learning setting. There are severalsources of ideas regarding effective models of learning: the work of cognitive scientists andof educators are the most formal. These are exemplified by the approach subscribed to inmodel curricula and text books (described briefly in Chapter 2) and further discussed in thissection, examining the educator-based perspectives.It is possible to describe learning models as active or non-active. While the former will bediscussed in greater detail later, in summary this addresses learning approaches that are morediscursive and collaborative. Active learning approaches are seen to create situations whichengage students in such higher order thinking tasks as analysis, synthesis and evaluation(Bonwell and Eison, 1991). By contrast, non-active learning are directed to absorption andimprinting (Horvath et al, 2004).In developing a framework for advanced active learning Horvath et al (2004) propose atopography based on the focus of the approaches and the nature of the applied methods (seeFigure 3.3). These are described as a continuum from instructionist through explorative toconstructionist, with the latter assuming that learners construct knowledge for themselves bycreative activities such as planning and design. The orthogonal axis describes the focus of133

the activities: from a priority given to sensory-motor and conceptual activities of individualstudents to the development of knowledge by a group or community of learners.Figure 3.3: Topography of approaches to active learning (Horvath et al, 2004)A distinction can be made between models predicated on constructivist and situated cognitivist(eg apprenticeship) frameworks. Although both approaches share characteristics (suchas learning occurs within a context of use, learning is frequently collaborative, learning asauthentic, learning as inquiry-based not transmission-based), they also embody many importantdifferences, both theoretically and practically. It should be noted that constructivismhas become an umbrella term that encompasses many different types of learning environments(eg see Duffy and Jonassen (1992)), even when they are predicated on vastly differenttheoretical assumptions (Barab and Duffy, 1999).Constructivist learning is a theoretical framework based on the work of Papert (1980). Constructionismbuilds on constructivism in that it distinguishes itself from more traditionalinstruction, in part, by the degree of active learner engagement as well as the assumptionthat learners have the ability to create meaning, understanding, and knowledge. Studentsare not passive receptacles of the knowledge that teachers impart – learners develop theirown reasoned interpretations of their interactions with the world. Perhaps more important,constructionist learning environments allow learners to share and collaboratively reflect onthese cognitive artifacts.Friedman (2001) indicates that experiences of the pioneers show that learning by doing,problem-based learning and project-based learning are the methods that offer the largest134

potential for constructive learning, in particular, in groups and communities. The last twoapproaches can be differentiated by saying that problem-based learning concentrates on betterunderstanding and the solving of recognised problems, while project-based learning focuseson the end product. Hence, project-based learning is more or less an artefact productionprocess, while problem-based learning is a knowledge development process. They can appearin practice side-by-side or interwoven. Learning by doing places the learners in direct contactwith the subject matter and facilitates finding information through social communication,3.3.1 Learning as a social or cultural activityA number of models highlight the social and contextualised nature of learning, presenting aperspective on learning as a social process of collaborative knowledge building (Lave et al,1988; Lave and Wenger, 1991; Brown and Duguid, 1991; Greeno, 1997). These contrast withbehavioural and cognitive models that portray learning as an individual activity and as anartefact that can be easily separated from the contexts in which it takes place.Stahl (1999) presents a learning model which classifies generic aspects of the learning processas personal and group. While individuals generate personal beliefs from their own perspectives,they do so on the basis of socio-cultural knowledge, language and representations.These beliefs become knowledge through social interaction, communication, discussion, clarificationand negotiation, so that knowledge is a socially mediated product. The group enableslearners to tune the accuracy and suitability of their personal understanding.Amongst others Schön (1983) argues that learning starts on the basis of tacit pre-understanding(Polanyi, 1958). The network of personal meanings ultimately has its origin in interpersonallanguage and culture, so that interpretation takes place within language (Wittgenstein,1953), as well as history, culture and politics. Although internal thought processcapabilities and structures have origins in previous social interactions (Vygotsky, 1978), thissocial context and origin is hidden because it has been incorporated into the tacit preunderstandingsof the individual. A breakdown renders elements of this tacit understandingproblematic. However, it is not always possible to resolve the problematic character of personalunderstanding internally, particularly when it is provoked by other people. Resolutiontypically involves some feedback from the world, so that the process of interpretation thattakes place at the level of the individual mind is an essentially social process that is enteredto create new meanings collaboratively. To do this, initial belief is typically articulated inwords and expressed in public statements.135

The importance of dialogueLearning is seen as a meaning-making venture with culturally developed tools and symbols;meaning is negotiated through cooperative social activity, discourse and debate (Mayes,1992), or through individual coherence.The learning nature of conversation has been described in the literature of learning (Harri-Augstein and Thomas, 1991; Senge, 1994; Baker et al, 2002; Garrison and Rud, 1995; Tharpand Gallimore, 1991). This perspective is underpinned by the concept that conversation canevoke reflection that results in learning. The concept of teaching/learning as conversation wasformalised through Pask (1976)’s Conversation Theory. This describes learning as occurringthrough conversations about a subject matter, which serve to make knowledge explicit.Senge (1994) coined the term ‘learningful conversation’ to distinguish conversation that engagesthe process of reflection, in particular reflection on the mental models that are afoundation for personal action. The discipline of working with mental models starts withturning the mirror inward; learning to unearth internal pictures of the world, to bring themto the surface and hold them rigorously to scrutiny. It also includes the ability to carry onlearningful conversations where people expose their own thinking effectively and make thatthinking open to the influence of others. He also contends that learningful conversation ismore likely if it takes the form of dialogue rather than discussion. Discussion is defined asconversation confined to participants stating and giving reasons for their positions (advocacy),whereas dialogue also involves participants in exploration and critique of the reasonsand assumptions associated with their positions (inquiry). He proposes several reflectionactivities that can help expose such misconceptions. Some of these activities are based onconcepts and processes originally proposed by Argyris and Schön (1974) and Schön (1983).This is in line with his belief that the ability to engage in such conversation is a pre-requisiteto professional learning – conversation has been explicitly identified as a key feature of severalprofessional learning initiatives. Sinceconversation is a meeting of minds with different memories and habits. Whenminds meet, they don’t just exchange facts; they transform them, draw differentimplications from them, engage in new trains of thought. Conversation doesn’tjust reshuffle the cards; it creates new cards(Zeldin, 1998, p 14)conversation may also be seen to be an ideal context for the gestation of new ideas andtherefore conducive to creativity.136

The conversational framework developed by Laurillard and illustrated in Figure 3.4 identifiesthe activities necessary to complete the learning process and provides a model for teachingstrategy. However, its application is seen not to inevitably lead to learning (Milton andLyons, 2003). Within the framework, the teacher sets up the conditions of the world withinwhich the student can act, the learner operates at the level of action within the teacher’sworld, and both parties operate at the level of description within their conceptual knowledge.Figure 3.4: Learning as a discourse (Laurillard, 1993)The pragmaticist basis of this learning model may be seen. Of import is the ‘construction’ andadaptation of the world by the teacher. This is not the teacher’s own conceptualisations, butan adaptation of those for the learning task at hand. The teacher constructs a hypotheticalmodel of the particular conceptual worlds of the learner. Changes can be induced onlyif there is some inkling as to the domains of experience, the concepts and the conceptualrelationships that the learner possesses at the moment. Language enables the teacher to orientthe student’s conceptual construction by precluding certain pathways and making others morelikely (Glaserfeld, 1996). Laurillard’s model focuses attention on key relationships betweenforms of discourse, academic knowledge, interactions with the world and reflection.Pask’s theory included the separation of description from model-building behaviour and thedefinition of understanding as determined by a two-level agreement (Pask, 1976). The criticalmethod for learning is ‘teachback’ – the learners teach another what they have learnt.Teachback and self-explanation (Chi and Bassock, 1989) are both seen as components of alearning dialogue. They force a focus on key aspects of the domain; force deeper processingof the topic, allowing relationships to be forged, and allow failures and conflicts to emerge137

(Gobet and Wood, 1999).learning process as a dialogue which is seen to be:These are basic premises of Laurillard (1993)’s model of thediscursive – both teacher and learner conceptualisations should be accessible to each otheradaptive – the focus of the dialogue is based on the relationship between the two conceptionsinteractive – requiring learner action and teacher feedbackreflective – supportive of the process to link feedback to goal.Acquiring membership of a disciplineAs noted previously, competence in any domain requires in addition to domain knowledge,physical and cognitive skills.In addition, competence is defined, contained and developed within communities of practice.Wenger (2000) suggests competence comes about through the communities’• joined enterprise – members hold themselves accountable to the collectively developedunderstanding of what the community is about• mutual engagement – norms and relationships are established through mutual interaction• shared repertoire – communal resources are produced (eg routines, styles, tools),and refers to historical and socially defined standards in the community.There are a number of important issues to be addressed regarding communities of practicein relation to learning. If it is present, then how do learners come to know the community,how are the community members and their resources represented in the environment, whatdoes the community mean to the learners, how do learners see themselves in relation to thecommunity, and what is the learners’ potential for membership in that community (Baraband Duffy, 1999)?Since the discourse of a discipline is central to the way its knowledge is constructed andtransmitted, the means by which learners acquire discursive knowledge is important. Whileexpert members of a discipline share knowledge about discursive practices in their community,this is mostly tacit, and, as also noted previously, traditionally, given little emphasis withinthe (formal) educational environment. To the implicit model of inducting students into thediscursive practices of a discipline has been added that of integrating such practice intocurriculum design.138

More recent research into student learning of the discourse of a discipline draws on themetaphor of apprenticeship, described as some gradually mentored pathway to membership(Candlin, 1998). Through a variety of alternative processes, the student acquires the ‘uncommonsense’ways of meaning relevant to the discipline. However, learners in a formal educationenvironment are more likely to have no direct connection with, or impact on, a communityof practice, only indirect connections through access to reports on or from the communityor the ability to use community resources, artefacts, or tools. These ‘activity groups’ can bedistinguished from a community of practice in that they are a temporary coming together ofa group of learners around a shared task intentionally designed to support learning (Baraband Duffy, 1999)). They are formed to take advantage of the learning potential afforded bycollaborative interactions.Nevertheless, learners can be enculturated in the discipline through both being socialisedto the oral and written ‘forms of talk’ (Berkenkotter and Huckin, 1995) in the community,and gaining conceptual knowledge. Although Berkenkotter’s model focusses on writing inthe discipline, the student’s concept of audience, their positioning of ‘texts’ in relation toothers in the discipline and learning to secure a useful uptake (ie transfer) are relevant here.Students are seen to absorb the disciplinary practices rather than learn them through explicitteaching. However, Berkenkotter and Huckin (1995) note that discourse based on classroombasedcurriculum and activities can share only some of the features and conventions of thediscipline (they refer to disciplinary genres and genre knowledge). It is worth noting thatwhere students have prior work-related experience or the nature of the learning experiencemodels ‘work’, students do perceive themselves as apprentice professionals (Gollin, 1998).A more realistic approach may be to view the learner’s entry into a discourse community asone of acculturation (learning an additional culture) Gollin (1998).3.3.2 Learning as authenticSituated perspectives on learning have been advocated in the psychology literature by suchresearchers as Bartlett (1932); Dewey (1910) and Lewin (1952). Elmholdt (2001) categorisesthe emerging body of work into lines of research that include the following: apprenticeship,situated cognition and cognitive apprenticeship.The primary characteristic of situativity is that individual cognition is placed within the contextof social interactions and culturally constructed tools and meanings – a tight coupling ofindividual and environment. Meaning construction is tied to specific contexts and purposes,based on shared knowledge within discourse communities. Learning then is seen as par-139

ticipating in communities of authentic practice (Sfard, 1998), and results from undertakingauthentic activities guided by expert practitioners situated in a culture of practice (Billett,1994). According to Brown et al (1989) conceptual knowledge may be considered as similarto a set of tools: they can only be fully understood through use, and using them entailsboth changing the user’s view of the world and adopting the belief system of the culture onwhich they are used. Therefore, to learn to use tools as practitioners use them, learners, likeapprentices, must enter the community and its culture – activity, concept and culture areinterdependent. Hannafin et al (1996) note that learning environments that support situatedcognition resemble, as well as the apprenticeship system common to craft professions,professions that require extensive schooling (medicine, dentistry, law). In addition, learningin cognitive domains such as mathematics, physics and language arts is enabled throughcognitive apprenticeships (Choi and Hannafin, 1995).Situation action and apprenticeship studies underly later research on situated learning, inparticular the seminal work of Lave and Wenger (1991) and the development of the conceptsof participation in communities of practice, elaborated in organisational learning studiessuch as Brown and Duguid (1991). Lave and Wenger (1991) considered the importance fornewcomers of grasping knowledge and skills, achieved by means of ‘legitimate peripheralparticipation’. In their view, the notion of participation deals with the process of situatedlearning, an integral and inseparable aspect of social practice which encourages newcomersto become part of a community of practice. Therefore, instead of regarding learning andcognition as universal processes and studying learning as decontextualied, learning is viewedas situated and bound to specific settings – everyday practice that is cohesive, meaningfuland goal-directed (Brown et al, 1989), and hence authentic.Doyle (2000) suggests the term authenticity can be understood in three different ways:student-centred – in terms of the learner’s perspective, the content of learning should beexperienced as genuine and meaningful to the participantssubject orientation – the task will be associated with the topic currently at hand in thestudent’s learningsituated – learning is connected to situated, real activities, which implies participation inreal situations.Authenticity may also be linked to on-the-job learning and to lifelong learning which is oftensituated. There are increasing numbers of studies where the notion of authenticity is centralto discussions on contextualised and constructivist approaches to learning. Newmannet al (1996) were interested in constructivist learning where people within certain areas are140

challenged to construct deep and meaningful knowledge rather than to reproduce surfaceknowledge. They suggested that complex tasks requiring abstract thinking, alternative solutions,well-developed writing, and commitment beyond the general school context would bebetter solved through authentic pedagogy.Central aspects of authentic learning are to take learner perspectives and to create a learningenvironment by referring the content to the learners’ actual life experiences. However, thetarget professional domain must also be taken into consideration (Barab and Duffy, 1999).The content of learning is then assumed to become genuine and meaningful – an authenticactivity implies real world experiences, which make the content relevant and engage thelearners in their own meaning-making.3.3.3 Situated cognitionA theoretical framework rooted in the constructivist principles advanced by Piaget (1968) andVygotsky (1978), the situated cognition approach to learning emphasises the relationshipsbetween social, behavioural/psychological and neural perspectives of knowledge. Authenticpractice (and learning) take place within a complex web of social and activity systems whichmay (or may not) acknowledge the place of mental constructs (models and schemas). Thework of Lave and Wenger (1991) exemplifies a Vygotsky-inspired socio-historical school, whilea cognitive science school incorporates the work of Norman (1993); Brown et al (1989); Collinset al (1989).Apprenticeship learningCurrent theorists have been exploring the lessons learned from on-the-job learning environments,in particular apprenticeship learning. After examining five apprenticeship situations,Lave and Wenger (1991) noted that in the successful cases there is little observable teaching,yet large quantities of learning. In these examples, the practice of community created thepotential curriculum in the broadest sense. As Lave et al (1988) note, [school] curricula tendto be a specification of practice, while apprenticeships arrange opportunities for practice. InLave and Wenger (1991)’s view, learning is not simply one kind of activity, rather ‘talentdevelopment’ viewed as an aspect of all activity. The process of learning is always situated,with prime importance given to the reasons for learning. They argued that in many apprenticeshipsituations the desire to become central to a community of practice makes learninglegitimate and of value for the individual.141

Lave et al (1988) suggested that learning is more than simply receiving a body of factualknowledge; learning is a process that involves becoming a different person with respect to possibilitiesfor interacting with other people and the environment. The individual is no longerthe same individual with new skills, but is a new person who has become more enculturatedinto the practice, negotiating meanings based on experiences as a student.Several characteristics have been identified for the apprenticeship model (Jordan, 1987):• work is the driving force – the progressive mastering of tasks is appreciated, not as astep towards a distant, symbolic goal (eg a qualification), but for its immediate valuein getting the work done• apprentices start with skills that are relatively easy and where mistakes are less costly• learning is focused on (physical) performance. It involves the ability to do rather thanthe ability to talk about something• standards of performance are embedded in the work environment. What constitutesexpert execution of the task is obvious, and judgements about the learner’s competenceemerge naturally and continuously in the context of the work. The apprentice ‘ownsthe problem’ of moving on to the acquisition of the next skill• teachers and teaching are largely invisible, with learning based on observation.Although the workplace is seen to have a number of strengths as a learning environment (Billett,1996): activities are authentic and goal-directed; learners have access to guidance – bothclose assistance from experts and ‘distant’ observing and listening to other workers and thephysical environment; they are engaged in everyday problem solving, and intrinsic reinforcement(ie internal satisfaction in making sense of new stimuli) is usual, one assumption madewithin this model is that workplaces are good learning places. However, evidence on informalon-the-job training and employers’ training investment patterns suggest that workplaces arenot good learning places for the young or the less educated (Scribner and Sachs, 1990; Tan,1989). Unless work-based apprenticeships are deliberately designed for learning, they havepotentially serious deficiencies. Limitations identified include (Billett, 1996): the constructionof inappropriate knowledge; a lack of sufficient or more challenging authentic activitiesand reluctance of experts to participate, or restrictions on their assistance. An additionallimitation addresses a difference in goals, methods, ideals and strategies between businessenterprises and learning institutions, based on productivity and survival versus learning andprofessional growth (Harris et al, 2000).142

This concern with apprenticeship is supported by later work by Elmholdt (2002). He presentsan interesting study of two ‘landscapes of learning’ which focus on the differences between theapprenticeship and another on-the- job model of learning. However, it should be noted thatElmholdt does not indicate what formal education has preceded the environments describedin his cases. In the first, blacksmith apprentices in a shipyard are surrounded by an environmentthat is characterised by many stabilising elements that support the handing down ofknowledge across persons, tasks and generations. Their learning is community stabilising andreproductive. In reproductive learning the divergence between the personal experience andthe competence in the community of practice is reduced by a change in the practice forms ofthe person, whereas the practice forms of the community are stabilised. The second, in aninternet company, is characterised by many changing elements that support flexibility andreadiness to change through reconstructive and innovative learning. This landscape is communityextending and exceeding. Reconstruction implies a transformation of the originallyconstructed, so that reconstructive learning extends existing practice forms of a community.The divergence between personal experience and competence in the community is reduced bychanges in the practice forms of both the person and the community. This learning radicallyexceeds the boundaries of a community’s existing practice forms.The stabilising elements of the shipyard are based on a three-step apprenticeship model:observational and model-learning - based on the work of journeymen; guided participation -the task is carried out by the apprentice under scaffolding provided and gradually withdrawnby the journeyman; practice. In this environment, reconstructive and innovative learningare only accepted if they fall within the tolerances of the acceptable practice forms, andmay be initiated by destabilising elements (eg introduction of technology) or unexpectedcircumstances (eg a strike enabled the apprentices to invent a new system).The flexible environment in the internet company are exemplified by individual responsibilityfor learning and a collegiate approach to problem solving. New employees are required tochoose an area of specialisation and to keep up with developments in that area. Knowledgeis to be shared with the rest of the organisation through formal training days and informalassistance. Work is continually negotiated and co-ordinated, but the three step approach isnot emphasised.While the apprentice blacksmiths were restrained to observation and imitation, the webdevelopers were encouraged to ‘try ahead’ and to bring practices from the outside to theinside. This brokering (Wenger, 1998) activity is never stable. Limitations of this approachare based on the lack of support for reproductive learning. Dreyfus (2001) asserts that limitedaccess to imitation of working experts may inhibit the development of genuine expertise. His143

assumption is that a certain minimum reproductive learning is required for future innovativeand reconstructive practices to be meaningful in the community.Apprenticeships are also seen as weaker than formal learning for content – academic knowledgeand higher order cognitive skills may only be ‘partly visible’ and therefore need explicitlearning strategies. Russell (1998) cites three ways in which apprenticeship models are limited:• they have great difficulty accounting for the effects of formal education, which is a deepthough tacit influence on workplace practice. Firstly the expectation carried into theworkplace may interfere with the traditional apprenticeship learning. In addition, thespecialisation implicit in the apprenticeship model is delayed, so that new employees donot see themselves as apprentices, in terms of their motives within the work environment• they have difficulty accounting for the multiplicity of the work environment. Increasinglyflexible response to the differing environments is required. Traditional apprenticelearning is encapsulated in the social practices of a discrete community whose relationswith other social practices are much more regularised and stabilised. Flexibility, reflectivityand critique are necessary, not simply the uncritical socialisation in a singlesocial practice• they have great difficulty accounting for the dialectical contradictions when apprenticeshave greater expertise in some areas that masters. The complex division of labour andcognition is not catered for by the apprenticeship model.In addition, if all learning was to occur in the context of use, learners would need to committo a career every early and in a narrow sense.Cognitive ApprenticeshipCognitive Apprenticeship is an attempt to address the issues identified as problematic inapprenticeship models. Based on analysis of traditional apprenticeship, child learning andcognitive science research, this model is aimed at teaching the processes experts use to handlecomplex tasks, but inside a formal education context. The focus of this learning-throughguided-experience(Collins et al, 1989) is on cognitive and metacognitive skills rather than onthe physical skills and processes of traditional apprenticeship. Situated, and hence foundedon constructivist principles, the Cognitive Apprenticeship model requires the externalisationof internal processes.144

As developed by Collins et al (1989) Cognitive Apprenticeship, incorporates strategies withinthe following framework:Content should include domain knowledge and heuristic strategies as well as control (iemetacognitive) and learning (eg which model expert learning) strategiesMethod including:• modelling and explaining to include false starts, dead ends etc, so that studentsdevelop ‘conditionalised’ knowledge with its tacit components• coaching to enable exploration and reflection based on ‘correct’ learning, scaffoldingand other aids are provided at a level that changes from high to ‘fading’ aslearning takes place• articulation enables the student to add insight, deal with the issue of problemisomorphs and compare knowledge across contexts• reflection influences strategic goal-setting and intentional learning• exploration so that students try out different strategies and hypotheses and observetheir effectsSequence instruction is ordered from simple to complex, with increasing diversity. In addition,the principle of global before local allows students to develop a conceptual mapof the activity before developing specific skillsSociology the social characteristics of learning environments focus on situated learning –authentic context include both real-life settings and problem-solving situations whichenable students to be intrinsically motivated and work in co operation, at their Zoneof Proximal Development (Vygotsky, 1978)through negotiations among present and past members. Activities thus cohere in a way thatis, in theory, accessible to members who move within the social framework.The notion of Cognitive Apprenticeship includes the development of learning contexts thatmodel proficiency and provide coaching and scaffolding. As students become immersed inauthentic activities (fading scaffolding as students develop competence), the opportunity forindependent practice is provided so that students gain an appreciation of the use of domainrelatedprinciples across multiple contexts. In these contexts, the goal is not simply toapply principles successful in apprenticeships, but actually to transform the culture so thatstudents can appreciate the purposes and uses of the knowledge they are acquiring, actively145

use knowledge as opposed to passively receiving it, and learn the varying conditions in whichthe knowledge can be used.Duncan (1996, p 67) suggests Cognitive Apprenticeship learning utilises scaffolding and verbalmodelling/coaching as:the most complete description possible of [the instructor’s] cognitive activities andstrategies, while providing organizational scaffolds for the studentsUsing Bruner (1986)’s spiral approach, the ultimate goal is for students to become selfsufficientthrough increased expertise and experience. As a development from the work ofBrown et al (1989) and Collins et al (1989), Lave and Wenger (1991) describe a CognitiveApprenticeship model that decentralises the importance of the ‘master’. No longer is theimplicit core the master teaching the apprentice, rather apprentices, legitimate peripheralparticipants in a social context, begin to assume responsibilities within that environment,testing their ability to assume roles. The full cultural context, artefacts as well as experts,afford the learner scaffolding, with little direct teaching between master and apprentice occurring.What is required is access within the learning environment, to the community ofpractice, and the tools to support the learner in the assumption of a role in that practice(Duffy and Cunningham, 1996).3.4 Formal learning of RECompetence in any domain requires in addition to domain knowledge, both physical andcognitive skills. Physical skills constitute physical expertise of the procedural tasks, includingappropriate tool use, while cognitive skills are concerned with the cognitive processes ofanalysis, interpretation and decision-making required for the carrying out of procedural tasks.Physical skills, due to their external visibility, are seen as relatively easier to acquire: cognitiveskills require more sophisticated learning process. Therefore, through a process of learning,robust domain competence is facilitated (Collins, 1990):• basic domain knowledge is acquired and subsequently used as a base to integrate knowledgegained from specific situations• basic domain knowledge is applied in abstract and contextual scenarios to generalisethe knowledge and skills to be able to apply them in real world situations.Conceptual domains (eg, philosophy, economics) emphasise domain knowledge, while skillsbasedcompetence is crucial for task-oriented domains (eg medicine, engineering) (Patel et al,146

1999). In addition, domains regarded as second order (in which practitioners work withconcepts that are already abstracted to a greater or lesser degree) are seen to be moredifficult to grasp and greater emphasis on cognitive skills is required (Patel et al, 1999).This is confirmed by the work of Laurillard (1993), which emphasises the distinction betweenfirst and second order learning (the former through experience, the latter through reflectionon experience and therefore a change in the perception of that experience). Second orderlearning relies heavily on and is assessed by means of symbolic representation, which requiresinterpretation.These criteria suggest RE is a second order task-oriented domain, implying emphasis oncognitive skills, and suggest acquisition, manipulation and organisation (and hence learningas transformation) are inherent characteristics of knowledge construction.3.4.1 Traditional learning and REWe can identify two characteristics of formal education for Requirements Engineering:• it adheres to a normative professional education model which sees science as the rationalfoundation for practice• it is based on texts that mirror a positivist worldview and present the domain assmoothly evolutionary.Waks (2001) suggests that the crisis of the professions arises because real-life problems do notpresent themselves neatly as cases to which scientific generalisations apply. Learning thatstresses the retrieval of organised packets of knowledge, or schemas from memory to augmentproblem solving is inadequate – ill-structured knowledge domains often render theseinappropriate. Rather, an appropriate ensemble of information suited to the particular understandingor problem-solving at hand must be assembled. Thus the knowledge is doublyconstructed – understanding is constructed using prior knowledge which itself is constructedon a case-by-case basis. Effective professional education calls for attention to both subjectmatter knowledge and general skills (Simon, 1980).Introductory, tertiary level texts portray the RE process as smoothly incremental. As discussedearlier (see Section 2.5.1), these texts form the basis of undergraduate education, andtherefore propose, implicitly, a learning behaviour that models the accepted (as opposed toactual) behaviour of professional Requirements Engineers.Accepting a smoothly incremental or evolutionary approach to the RE process equates wellwith traditional learning theories. In their simplest form these state that learning outcomes147

in a domain may be attained through the right set of instructional stimuli. Response to astimulus is predictable and reliable – all the teacher requires is to identify the subskills tobe mastered so that the intended behaviour is learned and to select the stimuli and strategyfor its presentation that builds each subskill (Winn and Snyder, 1996). This style of explicitteaching is often based on direct instruction, asystematic method for presenting material in small steps, pausing to check forstudent understanding and eliciting active and successful participation from allstudents(Rosenshine, 1986, p 60).Classified as a ‘transmission’ model of communication, direct instruction is well groundedin behaviourist theory. This approach to learning as a progression to expertise through alearning ‘chain’ based on task analysis, strategy selection, try-out and repetition has beenregarded as useful in well-structured domains. Daily review, presenting new material, guidedpractice, correction and feedback, independent practice, weekly and monthly reviews areelements that are mirrored in scientific and engineering methodologies, with their focus onprocess and repeatability.The evaluation of relevant BoKs and model curricula (see Chapter 2 for a summary of these)has revealed a significant mismatch between the description of the nature of RE and theskills and knowledge advocated by these models, with a strong tendency towards technicalknowledge within prescribed curricula.The training of Requirements Engineers is based on traditional learning models and are basedlargely on training in notations and prescribed processes. However, despite the engineeringand manufacturing metaphors that drive the view that software development is a smoothtransformation (of input to output), it is dominated by human cognition: software developmentis an exploratory and self-correcting dialogue (Bach, 1999). RE requires insight andcreativity. This discrepancy between theory and practice (Argyris and Schön, 1974; Glass,1995) is supported in the literature on expert behaviour (Visser, 1990; Robillard, 1999). Expertsdo not do in practice what they say the do (eg follow a methodology) because their ownplans are cognitively more cost effective and flexible, allowing for creativity and opportunism.This conflict between ‘approved’ and actual behaviour in RE practice is at the root of a majordilemma in RE education, and further exacerbates the challenge of educating RequirementsEngineers. These differing perspectives have major influence on the underlying knowledgestructures, skills (physical and cognitive) and techniques the RE has recourse to. The prescribedsyllabus of formal courses, with a focus on a solid foundation of knowledge to guide148

practice and direct future learning, is seen as a major factor in the mismatch (Macauley andMylopoulos, 1995a).In addition, within the software development environment Budgen (2003) suggests severalreasons to explain a reliance on prescriptive procedures in lieu of the less prescriptive learningmodel:• this means of transferring knowledge could not cope with the escalating need for designand development knowledge (and hence increased number of novices requiring thisknowledge)• peer-to-peer knowledge transfer is required if practitioners are to remain abreast of therapid development.The focus of an overly-prescribed syllabus may lead to both:• conceptual oversimplification, which in turn leads to failures that take common, predictableforms and to an inability to apply knowledge to new cases (failures of transfer),and• a lack of experience with granularity. Learning also involves traversing the granularityof various disciplines to varying extents, from detailed to abstract and from intrinsicallysimple to complex representations of knowledge (McCalla and Greer, 1991; Pateland Kinshuk, 1997). Students reason at many grain sizes, especially in reference toproblem solving abilities – as they refine their understanding, students articulate theirknowledge to finer grain size. They also move in the opposite direction – from finegrainedknowledge of particular situations to an understanding of inclusive, generic,coarse-grained knowledge (Patel and Kinshuk, 1997).Different theories of learning propose conflicting importance of ‘incorrect learning’. Tripp(1993) has referred to ‘fossilisation’, the learning of incorrect, but understandable, syntaxand pronounciation which suffices for communications. Since this interlanguage allows satisfactorysocial interaction, the learner does not progress to a higher degree of mastery, somistakes are fossilised and become part of the learner’s permanent repertoire.The work of Chi et al (1982) and Chi and Bassock (1989) also suggests there is increased difficultyin correcting incorrect learning. while a concept that is ‘badly’ categorised or modelledis no longer available for ‘use’ in the learning process. In a poor learning environment, thelearner is not directed to the important features of the environment. This is seen to impactgreatly on the efficacy and efficiency of further learning – a poorly structured network of149

chunks, poorly indexed (and therefore managed) is seen to lead to higher chance of failure torecognise and retrieve knowledge as required. Links may be missing (or incorrect) leading toa ripple effect and propogation of errors during performance. In addition, exposure to a narrowcurriculum (defined as limits to the variety of problem classes/ situations/representationsetc) can impede the creation of well-connected and integrated knowledge and inhibit futurelearning (Gobet and Wood, 1999).This poor fit can be further explained, at a more abstract level, by a lack of ‘generic integrity’within the discipline. As noted in Candlin (1998), genres are recognisable as representativesof particular discourses and discourse worlds, and have value as evidential data of particularsocial and institutional practices and memberships. Genres display textual characteristics,and are identified as such by co-members of the discourse world. However, their recognisabilitystatus is increasingly unsure: intertextuality (what is explicit in one context may betransmuted by incorporation in another), discursively mediated participant relationships andprocesses and regulated institutional practices determine generic integrity.Computing (as a specific example drawn from the broad IT arena) is seen to both permitand encourage a laissez-faire hybridisation of generic structures (the work of Gollin (1998)and others cited in Candlin (1998)) and tests the issue of generic integrity to the full. Thedivergence into sub-disciplines, the lack of discipline-specific guidelines and manuals derive(in Gollin (1998)’s view) from confusion over the nature of the communicative event, itspurpose and especially the tenor of relationships between participants.Within an educational context, the challenge to generic integrity is seen to lie in the tensionsarising from the simultaneous framing of the students as professionals solving real-world problemsand as students being assessed on their learning. Again the issue is between approvedand actual behaviour, here pedagogy versus workplace.Learners in traditional setting predominantly constitute students preparing for a career –classroom based students. Courses are designed to provide breadth and depth of knowledge,the relevance of which may not be fully understood by students. Real-world application ofthis knowledge is seen as at a distant horizon. This leads to a perception of low relevanceand fuels low motivation. Learner focus shifts to skills that will yield higher grades as animmediate objective: cognitive skills related to ‘exam techniques’ acquire importance thoughthey do not model real life situations. The learning, in many cases, is reduced to assignmenthopping with ‘just-in-time’ and just-enough’ learning to fulfill the assessment tasks, witha lack of adequate time for reflection on what is learnt, thus defeating the objectives ofproviding a well-balanced learning experience (Patel et al, 2000).150

Attempts to deal with these issues have been made in the area of software design education,where the more traditional lecture + laboratory work + assessment tasks are augmented byeither a capstone project which simulates a start to finish development environment or anindustry-based placement (both typically towards the completion of the qualification). Theseare seen to provide opportunities for both authentic and experiential learning, with emphasisnot so much on acquisition of knowledge as on increasing students’ ability to perform tasks.While accepted as valuable, this approach is flawed in several respects. These factors arebased on the description of learning environments proposed in the work of Savin-Baden(2000):• the opportunity (project or placement) is presented as an aid to content learning ratherthan a substitute• it focuses on know-how which will allow students to gain competence to practice withingiven frameworks (but not necessarily outside of them)• students are expected to transfer skills acquired to the world of work, but without themnecessarily being rooted in cognitive content and professional judgement.Although providing experiential learning opportunities, learning from experience is not automatic:it requires transfer to be enabled. This transfer is enhanced where there is a focuson metacognitive strategies and reflection. It is this facet that is often missing from capstoneprojects and placements.3.4.2 Factors against successAn education model for RE can be derived from the discussions in the Chapter 2 and thischapter. As shown in Figure 3.5 and discussed in Chapter 1, the educational process is basedon technical knowledge and competence derived from IT specialisation-relevant BoKs andmodel curricula. These in their turn have been developed, through a consultative process(albeit usually within a narrow view of the specialisation), from expert opinion and acknowledgedtexts within the discipline. The educational process itself is influenced by the acceptedmodel for professional education (with some allowance for practitioner-influenced authenticitythrough capstone projects and placements), institution-based generic attributed (derived,albeit at several stages removed, from practitioner input), and, within the specialisation, isitself influenced by a perspective drawn from the BOKs and texts. This perspective is underpinnedby a stance taken on the underlying epistemology, psychology and philosophy ofknowledge and learning. These are also foundational influences for the educational process.151

If we summarise the findings on each element of this model, we may gain a perspective as towhy the educational dilemma exists.Figure 3.5: Derived education modelThe educational model prescribed by the BoKs, model curricula and texts is shown to be simplistic:based on traditional professional education it places emphasis on discipline-specificskills and knowledge: Aurum et al (2003)’s suggestion that traditional IT education focusseson developing analytical and systems-thinking skills rather than creative and innovativethinking skills is borne out by Waks (2001) discussion of the problems of the normativeprofessional education model.RE is revealed as a complex, creative activity, where ill-structure and opportunism feature.Higher order learning is seen as essential in order to acquire the skills to deal with thesecharacteristics. Too simplistic an educational process is seen as detrimental to the developmentof competent REs: just as the creativity of the RE process is hampered, so too is theeducation of its proponents hampered by adherence to traditional learning models. Giventhat creativity is a component of RE, then it seems reasonable, as Aurum et al (2003) suggest,that students studying RE (they refer to systems analysis) should be well versed inthe importance of creativity within the software development process, and also be skilledin applying creativity-enhancing techniques. The poor fit between the characteristics of thedomain and those of the learning model (which produces an ‘incorrect’ learning environment)impacts on further learning.152

Specifically, the normative professional education model with its strict adherence to methodologiesderived from a ‘deterministic instrumental rationality in modern science’ (Truex andKlein, 1991; Introna, 1996) hampers the creativity of the RE process (Lubars et al, 1993;Maiden and Sutcliffe, 1992; Maiden and Gizikis, 2001). It:• restricts the essential characteristics of the process (such as opportunism (Guindon,1989) and creativity (Budgen, 1999))• assists in adding accidental complexity through attempts to control the RE’s professionalpractice (Sutcliffe and Maiden (1992) suggest strict adherence to method proceduresmay restrict natural problem-solving)• imposes a plan at odds to the RE’s cognitive planning mechanisms and hence interferingwith the management of knowledge (Visser (1990) suggests in practice, a plan is followedonly as it is cognitively cost-effective), and• implies a lack of confidence in domain knowledge (Adelson and Soloway (1985) suggestsdesign method practices are used as a mechanism for transferring knowledge rather thangenerating a solution).As the name Requirements Engineering implies, one view of the system/software analysisprocess grounds it in the implications of the term engineering. While there are domainsof specialisation within a discipline, the expectation is that they are very similar and requireidentical sets of cognitive processes. For example, all engineering tasks emphasiseproblem-solving, reasoning, creativity and team playing while requiring a good knowledge ofmathematics (Kearsley, 2000). In addition, being a critical thinker (one who has an inquiringmind, knows how to ask good questions and uses their answers to make meaning (King,1994)) is a pre requisite to effective engineering design. This assumes an ability to use highercognitive skills. However, issues in these areas are raised in discussions regarding appropriateeducation for engineering, with creativity a particular focus (Cropley and Cropley, 2000).Alternatively, Requirements Engineering may be placed within knowledge work. It is describedas having the following characteristics: information and knowledge is produced andreproduced; it is cerebral and involves the manipulation of abstractions and symbols thatboth represent the world and are objects in the world; it defies routinisation and requires theuse of creativity in order to produce idiosyncratic, esoteric knowledge; and requires a formaleducation which includes abstract, technical and theoretical knowledge (a list compiled bySchultze (2000) from numerous sources). Knowledge workers form a special class of whitecollarworker and include professionals, consultants, scientists, intellectuals and managers.153

The perspective that suggests that REs are not given problems, they construct (Visser, 1992)or discover (Guindon, 1989) them suggests that learning models that adhere most closely tothe non-positivist pole of all the dimensions of underlying ideology may address the challengeof educating REs. In addition, the characteristics of the RE process, as described in Chapter2, namely:• its opportunistic behaviour• the need for model restructuring and problem reconceptualisation to deal with intrinsiccomplexity• a dependence on insight and creativitysuggest that student REs require enhanced understanding of learning processes, includingreflection and critical thinking in order to model the behaviour of practitioners.Learning environments which enable advanced learning behaviour allow the complexity ofthe domain to be acknowledged and incorporated within the learning process. Individualcomplexity also has to be exploited within RE education – cognitive complexity is necessaryto deal with complex conditions and a high level of uncertainty. Hence, the environments developedfor RE education must address both domain and cognitive complexity requirements.3.5 Environments for learning3.5.1 Instructional designA premise of modern learning theory is that different kinds of learning goals require differentapproaches to instruction, therefore new goals for education require changes in opportunitiesto learn. Bransford et al (2000, p 19) examine the history of instruction goals and note thattoday, students need to understand the current state of their knowledge and tobuild on it, improve it, and make decisions in the face of uncertainty.They conclude that to achieve this vision requires rethinking what is taught, how teachersteach, and how what students learn is assessed.Instructional design theory is based on learning theory and forms the basic foundation forguiding the development of instructional systems (Gros et al, 1997) or learning experiences(Reigeluth, 1997).154

Jonassen (2000) suggests there is a discrepancy between what learners need (complex, illstructuredproblem-solving experience) and what formal education provides. Despite thecentrality of problem-solving in contemporary learning theories, instructional design for wellstructuredproblems are rooted in the information-processing theory: the assumption is madethat problem-solving only requires the acquisition of prerequisite skills. Hence, these modelsinadequately analyse or explicate the nature of the problem to be solved.Current trends in instructional design theory do exhibit a move away from behavioural tocognitivist and constructivist approaches based on situated foundations (Wilson and Myers,1999), with learner control a prominent topic (Richey, 1997). These shifts, which haveoccurred since the mid 1990s, have important implications for instructional design models:• instruction needs to be customised rather than standardised – the conformity and compliancerequired by traditional approaches that facilitated sorting learners into standardisedcategories are no longer appropriate• the role of the teacher becomes that of facilitator• awareness of the social context of the learning experience is required• greater collaboration between all stakeholders of the learning experience (includinglearners, teachers, designers) is required• the learning experience requires authenticity.The basis of a framework for a learning environment is a ‘constructive alignment’ (Biggs,1999) of objectives, teaching context and assessment tasks. Based on the discussions ofBrown et al (1997), these components achieve the following aims:learning objective – expresses the educational expectation of studentsteaching context – encourages students to undertake learning activities likely to achievethe objectivesassessment tasks – these focus on• the student learning process (examples include◦ to provide feedback to students to improve their learning◦ to diagnose a student’s strengths and weaknesses◦ to help a student develop skills of self-assessment to provide a profile of learningundertaken)155

• the evaluation of learning and awarding of qualifications (eg◦ to pass or fail a student◦ to license to proceed or practise◦ to select for future employment)• feedback about the quality of the learning process (eg◦ to improve teaching◦ to evaluate a unit/programme’s strengths and weaknesses◦ to demonstrate the course’s creditworthiness to the ‘outside’ world),and, with a specification of content, form the basis of frameworks for instructional design.However, Wilson and Myers (1999) cautiously suggest that an integrated framework forinstructional design can be based on learning theories drawn from the behaviourist, cognitivistas well as situativist camps. Greeno (1998, p 14) is quoted as demonstrating that differentsituations call for different tools, models, methods:learning environments organized on behaviorist skill-acquisition principles encouragestudents to become adept at practices, involving receptive learning and drill,that result in efficient performance on tests, and learning environments organizedon cognitive knowledge-structure principles encourage students to become adeptat constructing understanding on the basis of general ideas and relations betweenconcepts.While maintaining the primacy of a situated framework Greeno (1998) continues by suggestingthat behaviourist skill-oriented and cognitive understanding-oriented learning are notdiametrical opposites, but rather have important strengths and values which need to beincluded (and evaluated) from a situative perspective.Wilson and Myers (1999, p 18) suggests that some way must be found to accommodatemultiple levels of scale and, to some extent, competing paradigms or theories.As a response to the tension generated by competing paradigms for instructional design,and the plurality of perspectives they engender, Hannafin et al (1997) have developed agrounded learning systems design model. Following this approach, the test for legitimacy isno longer using the right theory, but grounding practice in some theory validated by theresearch tradition, with consistency the key. As one example, the Cognitive Apprenticeshipmodel should be consistent with its grounded theory of situated cognition. However, Wilsonand Myers (1999) sees flaws in this approach – the same level of consistency based on atheory-centred approach runs the risk of putting theory in charge and hence reducing the156

opportunistic and eclectic nature of practitioner-centered (or problem-centred) instructiondevelopment. They suggests that consistent theoretical grounding is only possible or desirablewhere participants share a common ideology: even then, resulting instructional design islikely to be a compromise reflecting the diversity of participants and stakeholders. Despitethis caveat, the point Hannafin (1997) makes, that the designers goal is to understand thecontexts of the learning environment (eg, is performance accuracy and training efficiency thegoal, or does the discipline require critical thinking and creativity) well enough to grounddesign practice using complementary foundations, is very relevant.Describing educational practice Sfard (1998, p 11) notes theoretical exclusivity and didacticsingle-mindedness can be trusted to make even the best of educational ideas fail. What isworse, however, is a mismatch, not between the instructional design and its grounding, buta lack of coincidence between the actuality of practice in the discipline and the instructionaldesign it is supposed to model. Reigeluth (1996, 1997) argues that the current paradigmof education is based on standardisation, conformity and compliance, geared to the massproduction of industrial age manufacture. This does not equate with the needs of the late20th and 21st century job market which revolves around problem-solving, teamwork, communications,initiative taking and diverse perspectives. He suggests a new paradigm is needed– based on customisation, diversity and initiative, to suit the needs of the information-ageeconomy.Barbour (2005, p 32) concurs:it seems reasonable to expect, at the end of a tertiary education experience, thatevery student’s attention should have been focussed on all of the themes identifiedin discipline and level specific contexts.He maintains that domain competence includes metaknowledge (knowledge about how knowledgeand practice is organised, conveyed, advanced, and legitimated) of the specific culturalethos of the discipline. Of the elements of his taxonomy relevant to undergraduate education,he suggests the following enable acculturation [with discipline specific elements only extracted]:Level 4 (tertiary entry – ages 16+) metaknowledge about using terms and specialistvocabulary• a vocabulary of 500 discipline specific words related to discourse in target disciplinesand a vocabulary of 836 academic words (Nation and Waring, 1997)• demonstrated fluency in using the vocabulary in literary and verbal context suchas is required for engaging in simple discourse157

Level 5 (ages 17-19) metaknowledge about communication in communities of practice• roles and communities of practice: stakeholders within the discipline• academic literacy: the use of the core means of demonstrating oral and writtenliteracy in the disciplines (including mathematics or modelling/programming languages)• the creation of clear text using the language of the discipline• logic, both first order and modal logics, the structure of defensible argument• temporal structures: managing time (work, leisure and study as an individual andgroup member)Level 6 (ages 19-20) metaknowledge about academic contexts and activities• interactions, disciplines structures and process; the advantages and disadvantagesof group work in disciplines and in practice; group dynamics and interactions;planning and managing learning in groups; conflict resolution• assessment and evaluation of group work; projects and in-depth short term studies;the role of groups in completing projects• academic literacy reflected in reports and extended essays demonstrating withindiscipline use of accurate expression following instruction in the tools and techniquesLevel 7 (ages 20-23) metaknowledge about theory and trends• the role of theory in specific disciplines; the role of methods in disciplines; therelationship between theory, methods and practice; the use and role of theory andmethods in practice; the illustration of theory and method through practice• disciplinary issues and trends: the nature and role of change in disciplines; historicaloutline of disciplinary developments, turning points and pressure points• advanced tools and techniques: identification and enumeration of past and currenttools and techniques; illustrative evaluations of past tools and techniques inrelation to current tools and techniques; the discipline specific role of computerbased technologies; appropriate and inappropriate uses of technology.Therefore the learning environment should define a level of competency that can be appliedand understood within a specific discipline. However, the study by Watson et al (2002) citedin Chapter 2 has shown a gap between the practice of determining competence levels andthe educational rationale underpinning it. In a study of how levels of units were determined158

within the formal (tertiary) education sector, they found that while expectations existed ofstudent progression towards a more competent state and the achievement of generic graduateoutcomes, there was no consistent university-wide approach. Rather an implicit (though not acommon) understanding of what differences between the levels of undergraduate units acrossa university should be existed. Within a discipline the progression was often managed by theuse of pre-requisites (skills and knowledge). Where educational reasoning existed capabilityframeworks (eg Australian Qualifications Framework (2002)) and learning taxonomies (egBloom et al (1956)) featured.It is accepted that learning does not stop with the end of formal education. However, learningafter formal education is not constrained by the cultures and methodologies of formaldisciplinary learning (although many of the cognitive strategies for learning are the same).Professional learning requires learner capacity and understanding for working with many differentsorts of knowledge in order to work with complex emergent problems for which theremay be a range of possible solutions. From the learner perspective, what is necessary is to :• have developed the executive function (metacognition) to engage in self-directed learningthat is effective• be able to visualise learning problems and engage individual conceptions of these problemsin order to ask questions like: what do we need to know? and how are we goingto come to know?• to have the knowledge to know how to go about developing new factual knowledge, todevelop new conceptual frameworks to make sense this knowledge and to be able toconsolidate, organise, connect and make future use of this knowledge.As noted by Laurillard (1993), the implications for the design of learning environments toachieve these graduate aims are two-fold:• academic learning must be situated in the domain of the objective, the activities mustmatch that domain• academic teaching must address both the direct experience of the world, and the reflectionon that experience that will produce the intended way of representing it.Savery and Duffy (1995) distilled the following instructional principles as a guide to thedesign of learning environments that address these issues:• anchor all learning to a larger task or problem159

• support the learner in developing task ownership• design an authentic task• reflect the complexity of the real-world environment the learner will be functioning in• give the learner ownership of the process used to develop a solution• support and challenge the learner’s thinking• encourage alternative views and contexts as a mechanism for testing ideas• support reflection on both the learning and the learning activity.The objectives of promoting effective learning, adaptation (transfer) and life-long learninghas implied a need for new pedagogical approaches towards student-centring learning ininstructional design. Some of these approaches are discussed in the next section.3.5.2 Learning as student-centred‘Student-centred’ is a term used to refer to learning environments that pay careful attentionto the knowledge, skills, attitudes, and beliefs that learner brings to the educational setting(Bransford et al, 2000). In general such an environment gives students greater autonomy andcontrol over choice of subject matter, learning methods and pace of study (Gibbs, 1992). Animportant implication of this definition is the need for students to assume a high level ofresponsibility in the learning situation and be actively choosing their goals and managingtheir learning (a characteristic of life-long learning). This involves considerable delegation ofpower by the teacher, who is required to be aware that learners construct their own meanings,beginning with the beliefs, understandings, and cultural practices they bring to the classroom.Accomplished teachers assume that these can serve as a foundation on which to build bridgesto new understandings (Duckworth, 1987).Grow (1991/1996) describes an approach to modelling learning from the learner’s ‘growth’towards life-long learning. This model reflects the principles advocated in student-centredlearning environments: the learner determines the need for some education, decides on apreferred approach to learning, identifies and accesses learning resources and draws on theassistance of educators as a part of that overall strategy rather than as a central element.His stages of growth are:Stage 1 learners need an authority figure to give them explicit directions on what to do,how to do it, and when. They either treat teachers as experts who know what the160

Table 3.1: Comparison of teacher- and student-centred learning environments (based on Hirumi(2002))Learning OutcomesGoals and ObjectivesInstructionalStrategyAssessmentTeacher’s RoleStudent’s RoleOrganises and presents informationto groups of studentsAct as gatekeeper of knowledge,controlling students’ accessto informationDirects learningExpects teachers to teach themwhat’s required to pass the testPassive recipients of informationReconstructs knowledge andinformationStudents sit in rowsLearning EnvironmentTeacher-centredDiscipline-specific verbal informationLower order thinking skills (egrecall, identify, define)Memorisation of abstract andisolated facts, figures and formulasTeacher prescribes learninggoals and objectives basedon prior experiences, pastpractices, and state and/orlocally mandated standardsPrescribed by teacherGroup-paced, designed for ’average’studentInformation organised and presentedprimarily by teacherAssessment used to sort studentsPaper and pencil exams usedto assess students acquisitionof informationTeacher sets performance criteriafor studentsStudents left to find out whatteacher wantsInformation presented via lectures,books and filmsStudent-centredInterdisciplinary informationand knowledgeHigher order thinking skills (egproblem-solvingInformation processing skills(eg access, organise, interpret,communicate information)Students works with teachersto select learning goals andobjectives based on authenticproblems and students’ priorknowledge, interests and experienceTeacher works with students todetermine learning strategySelf-paced, designed to meetneeds of individual studentStudent given direct access tomultiple sources of informationAssessment is integral part oflearningPerformance based, used to assessstudents ability to applyknowledgeStudents work with teachers todefine performance criteriaStudents develop selfassessmentand peer assessmentskillsProvides multiple means of accessinginformationActs as facilitator, helps studentsaccess and process informationFacilitates learningTakes responsibility for learningActive knowledge seekersConstruct knowledge andmeaningStudents work at stations withaccess to multiple resourcesStudents work individually attimes but also need to collaboratein small groups161

student needs to do, or they passively slide through the educational system, respondingmainly to teachers who ‘make’ them learn. The teacher acts as authority, coachStage 2 learners are ‘available’. They are interested or interestable. They respond to motivationaltechniques. They are willing to do assignments they can see the purposeof. They are confident but may be largely ignorant of the subject of instruction. Theteacher acts as motivator, guideStage 3 learners have skill and knowledge, and they see themselves as participants in theirown education. They are ready to explore a subject with a good guide. They will evenexplore some of it on their own. But they may need to develop a deeper self-concept,more confidence, more sense of direction, and a greater ability to work with (and learnfrom) others. These learners benefit from learning more about how they learn, such asmaking conscious use of learning strategies. The teacher acts as facilitatorStage 4 learners set their own goals and standards – with or without help from experts.They use experts, institutions, and other resources to pursue these goals. Learners atthis stage are both able and willing to take responsibility for their learning, direction,and productivity. They exercise skills in time management, project management, goalsetting,self-evaluation, peer critique, information gathering, and use of educationalresources. The teacher acts as consultant, delegator.More recent research into student approaches to learning (eg Entwistle et al (1974); Biggs(1970); Prosser and Trigwell (1999); Meyer and Shanahan (2000), amongst others) indicatesincreased important of this aspect – that what learners know and believe at the time affectshow they interpret new information, so that sometimes current knowledge supports newlearning, while at other times it hampers learning. In addition, student-centring can bedisturbing for learners who are used to a teacher-directed approach. Therefore, how readythe students may be for this approach and whether they are likely to need induction ortraining for it are questions that need to be addressed.Table 9.4 provides a summary of the differences between teacher- and student-centred learningenvironment, based on the work of Hirumi (2002).3.5.3 PBLProblem-based Learning (PBL) emerged in the 1960s to enable medical students to apply andsynthesise knowledge through using ‘real life’ case studies (Boud, 1985) in an environmentwhere an expanding knowledge base made it impossible to include all the knowledge required162

for the beginning practitioner in the undergraduate curriculum. It has since gained in popularityacross diverse disciplines where ‘meaningful’ learning requires students to deal withcomplex, ill-structured problems. The framework for theoretical assumptions about learningin PBL are drawn from several theoretical traditions: pragmatism (Dewey, 1916), cognitivepsychology (Piaget, 1968) and, most recently, social constructivism (Vygotsky, 1978). PBLinvolves teaching both a method of approaching and an attitude towards problem-solving. Itis an approach that is characterised by its flexibility and diversity since it can be implementedin a variety of ways in different subjects and disciplines (Savin-Baden, 2000).The starting point for learning in PBL is a ‘real life’ problems that the learner wishes to solve.Construction of knowledge and skills is through a staged sequence of problems presented incontext, together with associated learning materials and support from facilitators. PBLtherefore starts with the problem rather than the explicit learning of disciplinary knowledge– it creates a point at which new learning or critical thinking can be applied and reapplieduntil understanding is achieved. PBL is also designed to integrate the subject knowledgestudents require in order to solve a particular problem and therefore study issues at a deeprather than surface level (Entwistle and Ramsden, 1983). In this environment, the teacherbecomes a facilitator for learners who are actively involved in the learning process.Other characteristics of a PBL environment include (Boud, 1985; de Graaff and Kolmos,2003)• the problem space, domain and context have to be analysed, and problem definitionand requirements need to be defined• the learning/design process is iterative, and a structured process is necessary in orderto deliver in due time• learning is collaborative with a focus on communication and interpersonal skills. AsPBL tends to take place in small groups, students have to work cooperatively to achievetheir collective learning outcomes, with their level of independence measured by theirability to work with others. Consequently, communication skills, collaborative skillsand reflective/self-evaluation skills can also be developed• theory and practice are inextricably intertwined, with a focus on the process of knowledgeacquisition (rather than on the products of such processes)• students are self-directed – they are required to take responsibility for their own learning,with a leaning towards self- and peer-assessment.As with any instructional model, there are many strategies for implementing PBL. One163

taxonomy (Barrows and Tamblyn, 1980) proposes several varieties of PBL in use, describinga continuum from lecture-based cases where case material is used to demonstrate lectureinformation to problem-based (students meet with a client in a simulated environment thatallows free enquiry) and closed-loop problem-based where an evaluation of the reasoning usedbased on the resources utilised is incorporated. The type of scenarios offered, the assessmentmethods, learner autonomy and the way in which teaching and learning occur determinewhich variation is most appropriate for any given environment. Generally the problem ispresented in a way that is relevant to professional practice, and usually transcends disciplinaryboundaries so that an interdisciplinary approach is required to tackle the problem. A PBLmethodology, which expands or contracts the recursive stages described in Koschmann et al(1994):problem analysis the rich context is mined for important facts, sub-problem(s) and alternatesolution paths generated. With the guidance of the teacher, students explore theproblem, identifying strengths and weaknesses in learning and using these as a guideto individualised studyself-directed learning the learning agenda is determined by the information needed toevaluate the alternatives proposed.problem re-examination based on findings, solution paths are added, deleted or revisedabstraction an articulation process to increase the utility of the knowledge gained in specificcontextsreflection a debriefing of the experience to identify improvement in the learning process.Students reflect on the learning process and the content gained through working on theproblemis used to structure the learning experience.Although exponents of PBL note many advantages of the approach (including its emphasison student-centred learning activities that are long term, interdisciplinary and integratedwith real-world concerns and approaches, enforcing a holistic approach and thinking; itspreparation of students for flexible lifelong learning; problem-solving in unfamiliar situationsthrough reasoning critically and creatively) problems have also been reported (Boud andFeletti, 2001). Obstacles to be overcome include164

attitudinal risk• a difficulty in transitioning may be experienced by both staff and students. Bridges(1992) suggests academic staff are uncomfortable withholding information as theywatch students struggle with problems, and need training to develop facilitatorskills or they may be unsuccessful in PBL. Students may be uncomfortable withthe extensive collaboration required or with the lack of teacher-direction giventime constraints• for project development, Bridges (1992) suggests that each PBL project requires120 - 160 hours to construct, field-test, and revise. To this figure should be addedtechnical effort when the problem is developed in an online environment• for teaching, Albanese and Mitchell (1993) suggest 22% more time is requiredto teach in PBL mode, despite the reduction in content usually advocated. Intheir study, when academic staff consider the time per week in preparation toteach problems in comparison to presenting lectures, instead of 8.6 hours/weekprimarily preparing lectures, staff spend 20.6 hours/week primarily in groups withstudentsresource intensiveness• PBL is economical for classes of less than 40 students (Albanese and Mitchell,1993). It is considered not to scale well to large student numbers without greatincrease in staffing resourcescontent• guidelines for implementing PBL indicate that success is partly based on a reductionto the content covered: assuming too much content is a pitfall in a PBLenvironment (Albanese and Mitchell, 1993). This also is useful for modelling expertise– research suggests that a superficial coverage of many topics in the domainmay be a poor way to help students develop the competencies that will preparethem for future learning and work.Research supports the perspective that PBL promotes more in-depth understanding of contentthan traditional methods (eg Newble and Clarke (1986)), possibly explained by increasedstudent interest in the content being studied (evidenced by increased student attendance),higher motivation from the sense of ‘class community’ and problem ‘relevance’. Other researchsupports the view that PBL can enhance student problem solving abilities – PBL165

students are seen to be able to articulate the problem scope (Newell and Simon, 1972), usecritical-thinking skills to evaluate information (Chrispeels and Martin, 1998), use metacognitiveskills to reflect on their own problem-finding as well as solving process (Barrows andMyers, 1993) and transfer skills from the classroom to real-world situations (Woods, 1996a).3.5.4 Experiential learning as reflectiveCowan (1998) notes that there is as yet no authoritative educational explanation of teachingand learning which is centred on reflection. However, there are models of learning that describereflection as part of the process (eg Kolb (1984); Schön (1995)). Consistent with theconstructivist, cognitive and social psychology perspectives of learning, Kolb’s work presentslearning as a process in which knowledge is created by the learner through some transformationof experience. Schön refers to reflection-in-action as the responses that skillful practitionersbring to their practice. This reflection consists of strategies of action, understandingof phenomena, ways of framing the situations encountered in day-to-day experience, and maytake the form of problem solving, theory building, or re-appreciation of the situation (Schön,1985).Central to the idea of reflection is the identification of discrepancies between beliefs andactions. Interest in the reflective practice dates back to Dewey (1938), and his work withexperiential learning theories. He concluded that the experience the individual lives throughcan be described as a dynamic continuum – and that each experience influences the quality offuture experiences. Schön reformulated Dewey’s inquiry process to one of ‘design’. The resultsof such inquiry processes are knowledge that is generated in and for particular situations ofaction or practice:We should think about practice as a setting not only for the application of knowledgebut for its generation. We should ask not only how practitioners can betterapply the results of academic research, but what kinds of knowing are alreadyembedded in competent practice(Schön, 1995, p 29)but it may be put in a form that allows it to be generalised. He argues that the problemand the strategies of action can be framed in such a way that both can be carried overto new situations perceived as being similar to the first. Wisdom can therefore be learntby reflection on dilemmas that are encountered in practice and that by using reflection-onactionpractitioners can continue to develop their practice. This reflective transfer enables166

the learner/practitioner to carry some verbally explicit theory to new situations where it canbe put to work and tested, found valid and interesting, and also reinvented.Greenwood (1993) identifies weaknesses and inconsistencies in this approach, which, sheargues, has resulted in the implementation and prescription of dubious strategies for thepromotion of what Schön refers to as ‘enlightened professional artistry’. Often formal educationcannot answer the complex questions of clinical practice and there remains a gap inknowledge gained. However, the ideas expounded in the work of Argyris and Schön (1974)and the many writings of Schön (eg Schön (1983, 1985, 1987, 1995)) have triggered a rethinkingabout professional education – a slowly emerging recognition that courses for theprofessions need to seriously engage with professional practice: part of the major enterpriseof learning-to-learn to become professional. Further discussion, including that of Lave andWenger (1991), address the other major concern with Schön’s approach – the lack of focuson the ‘situated’ nature of learning. This is described elsewhere (see Section 3.3.2).The success of this learning is based on factors such as the degree of learner control, degreeof correspondence of learning environment to real environment and degree of involvementof self (Boud and Pascoe, 1978) (also described in previous sections of this chapter (see, forexample, Sections 3.5.2 and 3.3.2)), and the adoption of strategies which have come to beidentified as contributing to reflection. These are conceptualised as having the purpose ofturning experience into learning (Boud et al, 1985) or offering students the opportunity toprocess their experience to generate alternative ways of viewing a situation and achieving newappreciations or understandings. These factors and strategies, including the use of journalsand learning partners, and the creation of concept maps, share the feature that studentsare encouraged to return to their own experiences in class and outside and focus on whatthese events mean to them. In addition, reflective learning models are often based on anenvironment that takes into account how students learn and the learning requirements ofthe professional practice for which they are being prepared – to act and think professionallyas an integral part of formal education. Students are being prepared to become reflectivepractitioners, therefore opportunities for students to develop reflective skills and sensibilitiesshould be embedded as a normal part of all professional courses (Boud et al, 1985).Environments for reflective learning are, in general, based on the studio approach, modelledon the architectural studio (an adaptation of the atelier-based training at the Parisian Ecoledes Beaux-Arts in 19th Century (Chafee, 1977)), which encourages a blending of the functionaland the structural, the social and the technical by a community of learners who interactto solve problems. It is an immersive approach to learning where open problems are visitediteratively. This has long been the norm in disciplines where the nature of practice is the167

development of abstract artefacts that are used by others. Studios, and the attendant teachingmode of mentor and coach, can be found wherever artistic creativity goes hand-in-handwith a grasp of multi-disciplined content. Examples include art, interior design, architecture,graphic design. The pedagogy underlying the Studio approach has its theoretical originsin social constructivism, which places the learner at the core of the teaching and learningexperience, and is influenced by the work in many strands of educational research as well asresearch in relation to the IT disciplines, (eg Vygotsky (1978); Jonassen et al (1995); Cronin(1997) and Reeves (1997a)). A number of pedagogical processes and theoretical perspectivescome together in the practices of the design studio: Kuhn (2001) describes characteristics ofthe studio environment:• student work is organised primarily into complex and open ended problem• students’ design solutions undergo multiple and rapid iterations• the ability to work quickly, and to draw effectively on past experience, are key featuresof professional expertise – students are exposed to relevant precedent and to rapiditeration of design solutions (Dreyfus and Dreyfus, 1986). This combination is essentialto the development of true expertise, and the design studio can be a powerful venue foreducating expert software practitioners• critique is frequent, and occurs in both formal and informal ways, from teachers, peers,and visiting experts. One of the hallmarks of studio education is the creation of a‘culture of critique’, in which students, who spend long hours working side by side attheir projects, give each other frequent feedback, and also get both formal and informalfeedback from the academic in charge of the studio and reflect on their learning• a characteristic feature of design discourse is its dense interweaving of heterogeneousissues, and its odd method of progression – raising topics, considering them, and thenoften moving on to other issues without clear resolution of earlier concerns. Observershave commented on the linked and contingent nature of design decisions, reflected indesign conversations (Schön, 1983), while one author has compared design conversationsto jazz improvisation (Cuff, 1991)• students study precedents (past designs) and are encouraged to think about the bigpicture• teachers mentor students to impose appropriate constraints on their design process inorder to navigate a complex and open ended problem and find a satisfactory designsolution168

• the appropriate use of a variety of design media over the course of the project significantlysupports and improves students insight and designs.Critics of the studio model, especially in the discipline of architectural design, which isoften seen as a canonical exemplar of the design process, suggest there are issues with thenarrow definition of the ‘design’ that underlies the studio. They contend it fosters ratherunrealistic perceptions and expectations about professional practice. Central to these is theperceived tension between creativity and technical fundamentals – the idea of the ‘creativegenius’, which remains a potent ideology in architectural myth-making and one which has beenidentified with many problems that face architecture (Wigglesworth, 1993). An examinationof generally accepted design studio education models not only confirms their inadequacies,but also causes some anxiety about their immediate and long range impact on all designprofessions. For example, Balfour (1981) critically views traditional design studio problemsolvingactivities as merely ‘training’ without intellectual discipline. The design pedagogythat underlies studios represents design problem-solving to students as a creative activity tobe pursued for its own sake, independent of the (practical) need which gave rise to it.3.6 A framework for RE education3.6.1 RE as a cycle of teaching and learningThis section looks at the place of learning theories discussed earlier in this chapter in understandingthe process of RE and in facilitating the education of Requirements Engineers.A summary of the learning principles examined earlier is followed by a discussion of theirapplication to the RE process. The section concludes with the implications of these for theeducation of Requirements Engineers.The RE process has also been described as a learning cycle (Checkland and Scholes, 1990)where learning may be considered at two levels (Mayes, 1992): the metalearning processes ofdeveloping skills for making sense of the world and the business of actually employing thesein the acquisition of knowledge. From this perspective, a lack of correspondence between theconceptual model that guided the designer, the system image that is presented to the user,and the mental models of the user in Norman (1983)’s model discussed in section 3.2 may beviewed as an unsuccessful teaching/learning experience.In the same way that learning is a process of knowledge discovery, the discipline of RE can beviewed in the same way (Guindon, 1989) – as has been discussed in Chapter 2 the RequirementsEngineer builds fragments of understanding of the problem validated and consolidated169

through the traversal of layers, collecting more areas and information at each, adding detailand richness to the mental model of the problem situation (Batra and Davis, 1992).As has previously been discussed, constructivism, as a psychological theory of learning, isbased on the premise that humans have no access to an objective reality. Rather it is continuallybeing constructed, being transformed in the process (Glaserfeld, 1996). Constructivismviews learning as a self-regulatory process of struggling with the conflict between existingpersonal models of the world and discrepant new insights (Fosnot, 1996a).This provides a reasonable definition of the Requirements Engineering process: a strugglebetween the models of the system world as perceived by each of the stakeholders, includingthe Requirements Engineer and development team.This chapter has noted some of the approaches have been developed to describe learning:• one approach applies levels of processing as a continuum of analysis, from sensoryanalysis (transient) to semantic analysis, seen to be more or less permanent. Oncea sufficiently rich framework of understanding is in place, additional information iseasily assimilated as it is compatible with previously existing cognitive structures (Craikand Lockhart, 1972; Craik and Tulving, 1975). Mayes (1992, p 1) summarises theseapproaches by stating:As we build a framework, or schema, for comprehension, we build a mechanismfor automatic learning. New information is simply an elaboration, or afilling in of the slots, of what is already understood. No ‘effort’ is involved,beyond attending to the information in question.• a different conceptualisation of the learner sees a progression through increasingly complexand elaborate mental models that will characterise the gradual mastery of a newdomain. This mental models approach links initial learning through correspondencewith the learner’s intuitive model of the phenomenon, gradually increasing complexitythrough a transformation process applied to the initial models presented. Thisapproach sees understanding as achieved through conceptual discontinuity (White andFrederiksen, 1986), with language the mechanism used to orient the learner’s conceptualconstruction• yet another model sees the process of learning as one of dialogue – this perspectiveis underpinned by the concept that ‘learningful conversation’ (Senge, 1994) can evokereflection on the mental models that are a foundation for personal action and resultsin learning. Senge contends that learningful conversation is more likely if it takesthe form of dialogue rather than discussion. The conversational framework developed170

y Laurillard (1993) and illustrated in Figure 3.4 identifies the activities necessary tocomplete the learning process.Within the Requirements Engineering process participants, stakeholders and analysts arerequired to interact and cooperate in achieving the goals of the undertaking. Yet very fewtexts acknowledge the notion of the Requirements Engineer’s active role in constituting whatis taken as real – in acknowledging the richness of experience in the process. The interventionof the Requirements Engineer may be viewed as one of teaching/learning.Applying Norman (1983)’s model, from one perspective, the stakeholders embodies the‘teacher’ concept - knowledge of the system to be developed lies to greater or lesser extentwithin their domain of experience. At this level, the Requirements Engineer is the ‘learner’ –the purpose of the RE task is to produce an external, assessable representation of the system.Norman’s model can provide a representation of this view of the Requirements Engineeringprocess:• target system – that which is to be learnt: ultimately represented by the requirementsspecifications produced• conceptual model of the target – the requirements in the mindspace of the stakeholders.An issue is the ability of the stakeholders to convey this to the ‘learner’• mental model of target – the Requirement Engineer’s concept of what the target is. Thisis based on a predictive characteristics of internal mental models – the RequirementsEngineer will make assumptions of the target based on these• internal mental model – the accumulated experience/knowledge in the mind of theRequirements Engineer.At a different level, the roles in the RE process are reversed: the Requirements Engineeracts as teacher in the construction of the target system. The teaching process starts with anassessment of the internal mental models of the learner. Again using Norman’s model:• target system – that which is to be learnt: ultimately represented by the (creative)artefacts produced• conceptual model of the target – the requirements in the mindspace of the RequirementsEngineer. This is based on accumulated experience/knowledge• mental model of target – the stakeholders’ concept of what the target is. This providesassumptions of what the requirements are, based on their internal mental model171

• internal mental model – the accumulated experience/knowledge in the minds of thestakeholders, based on their expertise in the domain of application.While both views shed some insight on the reasons why the Requirements Engineering processis viewed as a complex activity, neither propose a complete picture of the knowledge constructionprocess in that the context of the construction is implicitly rather than explicitlydetailed.Increasingly the Requirements Engineering process is seen as a collaborative venture, framedby• the culture of the domain in which it is undertaken. As one example, within thediscipline of IS, Checkland and Holwell (1998) argue that the systems developmentprocess consists of building and naming models (of a set of purposeful activities). Thedevelopment of such models requires a declaration of the worldview (Weltanschauung)which makes a particular model meaningful – the set of values, outlook – a given-astakenimage of the world. These models are used as a vehicle for the requirementsanalysis process• the environment in which the RE took place. This environment includes the methods,tools available and used and economic constraints imposed as well as the cognitive skillsand social environment of the Requirements Engineering team. Checkland’s model ofsystem development follows closely the premises of the learning theory outlined above.The models built are not viewed as descriptions of reality, but rather concepts relevantto the exploration of what is perceived as reality. He acknowledges that reality is toocomplex to model meaningfully - hence a pragmatic approach is preferred, as is onewhich explores the way in which people attribute meaning to their world. The modelsdeveloped are used to give structure to debate about the problem situation and providea means of exploring possible changes that are both desirable and feasible.Pohl (1994)’s three dimensions of RE described in Chapter 1 may be addressed now from theperspective of constructivist learning theory:specification the degree of specification understanding at any given time – an iterativeprocess. Norman’s conceptual model is constructed through an iterative process andwithin a framework provided by the culture of the problem domain. This is a processof both learning and teaching in that the dialogue requires both teacher and learnerconceptualisations to be both available to each other and able to be adapted throughout the learning process. Norman notes that discovering the learner/user’s mental172

model in order to conceptualise it is not easy. At the least the ‘demand structure’ ofthe situation provides a mental model that may have not existed prior to the requestto define it (as is confirmed by Boland (1985)’s case study)representation just as background, experience and belief structures determine the mentalmodels developed by the system user, background, experience and belief structurescolour the mechanisms the Requirements Engineer puts to use in order to represent thesystem being learnt/taught. Boland found that a different use of language in a dialoguebetween analyst and stakeholder was not simply reflecting the stakeholders’ world buthelping to constitute it. This is critical to the constructivist view of knowledge - not onlymay requirements not be taken as pre-existing, but how they are constructed dependson the representational mechanisms offered by and to the Requirements Engineer andthe cognitive filters employed by all participants. This issue is further coloured bydiffering emphasis placed on the importance of ‘incorrect learning’agreement the process is completed with evolved fit between all parties. Social constructivisttheory deems that models developed and the knowledge constructed is determinedthrough communications with others, with the viability of understanding culturally determined.Implications for RE educationAs discussed in Chapter 2, the failures and shortcomings of Requirements Engineers hasbeen based on lack of skill, training and sensitivity to context, amongst others. Twentyyears later, the knowledge taught to software practitioners and managers in their formalcomputing education still does not match closely the knowledge needed to be applied indaily work (Lethbridge, 2000). This suggests the nature of RE task is not mirrored in thelearning environment being provided to student Requirements Engineers, with the strongpossibility of a failure to transfer knowledge gained to new cases (Spiro et al, 1991).In order to remedy this failure, RE education requires a focus on cognitive flexibility - providingthe student with:• the ability to represent knowledge from different conceptual and case perspectives and,later, the ability to construct from these a knowledge ensemble tailored to the needsof the understanding or problem solving at hand – the same items of knowledge needto be presented and learned in a variety of different ways and for a variety of differentpurposes173

• opportunity to develop cognitively flexible processing skills and the knowledge structureswhich can support this. In order to achieve this, the learning environment is alsorequired to be flexible, to permit the same items of knowledge to be presented andlearned in a variety of different ways and for a variety of different purposes, at differentlevels of granularity.An appropriate learning environment for RE should also provide a number of opportunities:• to identify, analyse and solve a number of issues, repetitively and at different levels ofcomplexity. This acts as preparation for professional employment• to practise the art as well as science/engineering of RE• to test the understanding of theory, its connection with application, and develop theoreticalinsight• to deal with incompleteness and ambiguity• to think independently and work co operatively, fostering insight into individual strengthsand weaknessesby means of a focus on:• knowledge construction based on experience with multiple perspectives and representations• metacognitive decision making competence, implying both wide content knowledge andreflection• experience, within an authentic context.Section 3.4.2 discussed the issue of inadequacy of formal education in training competent REpractitioners. This may be partly explained by an ‘incorrect’ learning environment resultingfrom the poor fit between the characteristics of the domain and those of the learning model– within a Computer Science domain, a scientific paradigm is seen as dominant, within anEngineering domain, a process/manufacturing metaphor exists.Yet Spiro et al (1991) suggest education theories and learning models that focus on theconstruction of knowledge are likely to be more suitable for learning in domains involvingill-structured problems. Savery and Duffy (1995) also suggest that appropriate learning inill-structured domains and/or dealing with ill-structured problems should itself be problembased,with principles that emerge from constructivism encapsulated in problem-based learning.174

Applying concepts of learning from the theory and models of constructivist learning describedin this chapter highlights aspects of knowledge construction that are pertinent to the RequirementsEngineering process: Bransford et al (2000) found that to become competent in some‘area of inquiry’, students must have a solid foundation of factual knowledge; an understandingof facts and ideas via a conceptual framework, and organisation of the knowledge in waysthat facilitate future application.As well as through problem-solving experience, this may be based on correspondence with thelearner’s intuitive model of the phenomenon - and all that implies about previous experience,belief systems etc. A richer framework is also based on a facility with multiple representations- translation between different models facilitates the understanding of concepts, whilst themodels themselves support differing insights, reasoning and problem-solving within a socialand cultural context, where certain activities are seen as authentic.This suggests learning models exist which target the needs of the discipline and address thegaps in formal education identified by practitioners. However, providing such an environmentis a challenge to the RE educationalist, given the dominance in practice of existingeducational frameworks. Therefore a solution may be proposed through the development ofa new framework for RE education. This framework should:• be based on constructivist theory (as more suitable for learning in domains involvingill-structured problems (Spiro et al, 1991)) with a focus on strategic knowledge, whichenhances knowledge construction and transfer. These strategies include strategies foridentifying and meeting sub-goals, procedural steps as well as metacognitive strategiesfor directing, monitoring and evaluating learning• be placed within a situated experiential learning environment where authentic context isexploited. Learning beyond the initial stages may best be achieved through situationalcase studies with rich contextual information (Dreyfus and Dreyfus, 1986). Focussingon the solution of authentic problems as a context for learning provides students withentry to the community of practice to which they will belong• provide the student with a learning environment which emphasises modelling practiceand making tacit knowledge explicit.However, the cost of such an approach is a heavier learning toll, as a ‘critical mass’ is requiredto be useful and the potential that usefulness may occur only in the long term.The rest of this thesis described the development and implementation of learning environmentsthat attempt to apply this framework.175

Chapter 4The research designThis chapter discusses the research design and approaches this from the perspectives of researchin education and research in IT disciplines. The methodology chosen, Action Research,is consistent with the style described as the ‘Deakin’ (Carr and Kemmis, 1986; Kemmis andMcTaggert, 1988) approach. This has merit in being adopted for studies in educational contexts(Zuber-Skerritt, 1982, 1995). Strategies for data collection and analysis are emergent,therefore a variety is discussed, and a summary of the approaches taken provided. Thenext chapter synthesises the knowledge gained from this discussion in order to develop andpresent a framework for the Action Research study undertaken. That framework is basedon the integration of several models, each addressing a specific aspect of the study: a modelfor Action Research in an educational setting (Borg et al, 1993), a model of organisationalculture that reflects the educational context of the study (Rogers, 2002), and a model ofreflection that incorporates the necessity for engaging in double-loop learning in order toachieve professional development (Hatten, 1997).This research is undertaken with the understanding that it is ‘applied’. The goal of basicresearch is to contribute to abstract, theoretical understanding by deriving new knowledge(Bunge, 1967), and is only indirectly involved in how that knowledge is applied to specific,practical or real problems. Applied research, on the other hand, contributes to understandingin order to be able to more effectively act or to ‘design interventions’ into the environment,which is the context of this study.The domain of this research is education, while the vehicle is within the discipline of IT (andspecifically RE) while the culture of the environment is an engineering one. This straddling ofmultiple disciplines provides a challenge for the researcher – to address the (research design)expectations across this context. By virtue of the nature of the disciplines that providethe context for the research, the philosophical and epistemological assumptions made arecontrasting. In particular, the Action Research family of methodologies is not well developed176

within a Software Engineering environment. Therefore it is appropriate for the followingsections to elaborate on the research approaches that could be seen to the address the issuesidentified in this study. In the context of research for education and for IT disciplines, with afocus on engineering approaches to software development, several approaches are discussed.It is suggested readers of this document could skim those components familiar to them.A framework for the research designThe choice of research design is informed by the discussions of Creswell (2003). He suggeststhat recent developments in research theory and practice require a framework that addressesthe levels of decisions involved in designing research in an environment where pragmatism andmixed methods, in particular, have changed the way research design is considered. The contrastingphilosophical and epistemological assumptions implicit in natural science and socialscience research approaches (described and discussed at great length in, for example, Bunge(1984) and Guba and Lincoln (1994)) have been expanded or extended by other researchers.Zmud (1998) expands earlier work contrasting positivist and interpretivist approaches to researchby aligning problem-oriented research with applied research; Gregg et al (2001) extendby adding the meta-level assumptions of Design Research.However, for Creswell (2003) this would seem to be too narrow an approach to take. Headvocates Crotty (1998)’s framework, which addresses:• what epistemology informs the research [eg objectivism/subjectivism]• what philosophical stance lies behind the methodology in question [eg positivism/interpretivism]• what methodology governs the choice and use of methods [eg experimental/action]• what methods are appropriate [eg interviews/surveys].Because of the cross disciplinary nature of this research, and the contrasting assumptionsmade regarding research in those disciplines, it is essential that the influence of philosophyand epistemology on strategies of research are made explicit.177

4.1 Research in Education and ITEducational researchUntil the 1990s, most educational research was adapted from quantitative research methodsin the physical and biological sciences. Although having a briefer history than in thesefields, quantitative educational research has yielded useful understanding of education. Morerecently, however, methods used in educational research have increasingly been adopted fromthe social and behavioural sciences and humanistic disciplines such as philosophy. The changein focus is an acknowledgement that education involves individuals and events in a naturalstate within a specific context. The individual’s perception of inner experience and the worldaround (the phenomenological reality) is seen to be of prime importance, and best describedqualitatively.This should not be seen as a disjunctive paradigm shift: as late as 2004 educational researchwas seen to seek a comprehensive educational effectiveness theory: a search for what causespeople to become educated. This can be considered as continuing the quest to place educationalresearch more firmly on a scientific, ideology-free footing, and to remedy what is seenas its chronic other-worldly drift (Clark, 2005). Both Nisbet (2005) and Clark (2005) providean extensive review of the changing perspectives in educational research, while Lather (2004)provides a critical review of the move to legislate scientific research into education in theUnited States.A paradigm of Praxis, the art of acting on the conditions faced in order to change them, isseen to have closest affinity to research in educational contexts (Lather, 1986). Hollingsworthand Sockett (1994, p 9) characterise this impulse toward Praxis within the education researchmovement as being based on...disenchantment with the view of control as a means of improving education,a concern with teacher autonomy, and a growing understanding of knowledge asa source of power in society created an ideological convergence which provides aclearer realization of the interconnected nature of knowledge, research and practice.An emerging area based on this belief is that of Learning Sciences. This field is interdisciplinary,drawing on multiple theoretical perspectives and research paradigms to buildunderstandings of the nature and conditions of learning, cognition, and development. Whileaccepting the importance of context for understanding, learning scientists see education asa design science, as opposed to a social science, to be compared to the natural sciences.178

This view of education requires a different form of research than traditionally adopted ineducational research, in the shape of the design experiment.IT researchWithin the Information Technology domain, a similar tension between research strategiescan be seen. In each of the main subdisciplines, CS, IS and SE, research is undertaken eitherto tackle technological/scientific issues or issues focussed on organisational/communicationsproblems. What are considered the appropriate philosophical and epistemological bases forresearch into these areas is the prime area of contention.Computer Science researchWegner (1976) attributed this tension, in Computer Science, to differing definitions of thediscipline: the study of phenomena related to computers, reflecting an empirical traditionsince it asserts that CS is concerned with the study of a class of phenomena; the studyof algorithms or information structures reflecting a mathematical tradition since algorithmsand information structures are two abstractions from the phenomena of CS; the study andmanagement of complexity, reflecting an engineering approach. He continues by arguing thatComputer Science has been dominated by empirical research paradigms in the 1950s, bymathematical research paradigms in the 1960s and by engineering-oriented paradigms fromthe 1970s. Although the rapidity of this evolution in CS research has led to divergencesof opinion regarding its nature, he concludes that each of the paradigms has played animportant role in the development of the discipline as a whole. In Wegner (1976, p 322)’sview, the ‘educated’ computer scientist should have the enquiring mind of the empiricalscientist, the modelling and abstraction ability of the mathematician, and the tool buildlingand implementation ability of the engineer.In the same vein, the Computing Research Association (Patterson et al, 1999) acknowledgesthat academic research in CS (and Engineering, but this is discussed later) applies one of twobasic research paradigms: theory or experimentation. Generalising, it suggests theoreticianstend to conduct research that resembles mathematics: the phenomena are abstract, andthe intellectual contribution is usually expressed in the form of theorems with proofs. Asa second generalisation, experimentalists tend to conduct research that involves creatingcomputational artifacts, assessing them and evaluating their effectiveness at solving the targetproblem.Clarke (2000) suggests that CS research is concerned with the conceptualisation, prototyping,construction, demonstration and application of new technology. The research strategies of179

choice continue to be drawn from the different traditions described by Wegner, with theprimary techniques including:• theorem proof – this applies formal methods to mathematical abstractions, in order todemonstrate that, within a tightly defined model, a specific relationship exists amongelements of that model• simulation – the study of a simplified, formal model of a complex environment, in orderto perform experimentation not possible in a real-world setting• laboratory experimentation – involves the creation of an artificial environment, in orderto isolate and control for potentially confounding variables.Some researchers see the need to resolve this tension, and approach CS research from a morescientific perspective, which presupposes a body of theory; an explicit theoretical frameworkto guide research; a defined research question; explicit refutable hypotheses; and a researchmethod that applies well-defined research techniques in order to enable hypotheses to betested. Tichy (1993, p 40) is most persuasive in his argument for scientific CS research:Without experiments in the tradition of science, computer science is in dangerof drying up and becoming an auxiliary discipline.... As computer science leavesadolescence behind, I hope to see the experimental branch flourishing.Others revel in the paradigm shift they see occurring – from computer scientistshistorically identified either as mathematicians (ah, the purity) or physicists (prettygood purity and much better government funding); but if you look at the kinds ofproblems we are trying to solve now (bunches of different aspects of the securityproblem, privacy, usability of pervasive computers, changing business models,e-voting) it seems pretty clear that the key issues relate to people and the waythey communicate and organize themselves, rather than discovering the underlyingphysical laws of the universe – in short, the domain of social sciences. ...Could a ‘social science’ perspective be a better lens through which to view computerscience – and could this in turn clarify the ways in which software engineering,like other engineering disciplines, applies the results of underlying science?(Pincus, 2005)Information Systems researchIn Information Systems, the tension may be described, somewhat simplistically, by consideringthe philosophical underpinnings:180

• IS research is positivist if there was evidence of formal propositions, quantifiable measuresof variables, hypothesis testing, and the drawing of inferences about a phenomenonfrom the sample to a stated population (Orlikowski and Baroudi, 1991)• interpretivist research is based on hermeneutics and phenomenology (Boland, 1985), sothat access to reality is only through social constructions such as language, consciousnessand shared meanings. It focuses on the complexity of human sense making withina specific context rather than through predefined dependent and independent variables• critical research focuses on the oppositions, conflicts and contradictions in contemporarysociety, and makes the assumption that social reality is both historically constituted andconstrained by various forms of social, cultural and political domination (Habermas,1972).The view, held by Galliers (1985) Checkland and Holwell (1998) and others, that IS is a sociotechnicaldiscipline, identifies a dualism that adds another level of tension to the debate,centred about the perceived validity of qualitative versus quantitative research within IS.The research strategies adopted by advocates of each philosophical stance are argued on thebasis of alignment with an epistemological view that lies behind the methodology and governswhat methods are considered appropriate. However, purists contend that some alignments aremore valid than others. So, for example, a QPR approach (quantitative, positivist research)stresses empirical data gathering or data exploration, while the positivist philosophy dealswith problem-solving and the testing of the theories derived to test these understandings(Straub et al, 2005).However, the socio-technologists argue the misplacement of the scientific ethos in IS research– it should be undertaken with consideration of the context of the system and the individualswho action it, and ultimately have relevance within that context. In IS, this research hasgenerally been undertaken from a social constructivist and interpretivist stance that arguesfor• an acknowledgement that a problem can only be studied in its context to be fullyunderstood• the need to be satisfied with an emergent research design which includes flexibility forthe scope of the problem to unfold• the acceptance of the human as a plausible research-gathering instrument and of qualitativemethods of data gathering as most appropriate181

• the increased importance of the provision of circumstances for participants to act andreact in the context of their environment• the acceptance that changing boundaries for the study are determined by the problemunder investigation (adapted from Lincoln (1985))• the dependence of outcomes on the researcher’s perspective of the selection and definitionof the research domain; the selection and rendition of existing theory; the definitionof the research question; the design of the research framework; the selection, definitionand operationalisation of variables; and the measurement of variables (Clarke, 2000)which leads to a requirement that multiple interpretations of the same phenomena must beallowed for.Clarke (2000) suggests strategies of choice can be unequivocally interpretivist in their style:• descriptive/interpretive research – empirical observation is subjected to limited formalrigour. Controls over the researcher’s intuition include self-examination of theresearcher’s own pre-suppositions and biases, cycles of additional data collection andanalysis, and peer review• focus group research – involves the gathering of a group of people, commonly membersof the public affected by a technology or application, to discuss a topic. Its purpose isto surface aspects, impacts and implications that are of concern• action research – the researcher plays an active role in the object of study (eg by actingas a change-agent in relation to the process being researched). While accepted as a validresearch method in applied fields such as organisation development and education, ithas been largely ignored within IS (except for the work of people like Checkland (1991))• ethnographic research – applies insights from social and cultural anthropology to thedirect observation of behaviour and seeks to place the phenomena studied in their socialand cultural context. Ethnography has become widely used in the study of informationsystems in organisations and is seen as a method whereby multiple perspectives can beincorporated (Myers, 1997)• grounded theory – seeks to develop theory that is grounded in data systematically gatheredand analysed. It suggests that there should be a continuous interplay between datacollection and analysis to enable the disciplined extraction of a theory-based descriptionof behaviour, to be based on empirical observations. Myers (1997) suggests that, since182

the method is extremely useful in developing context-based, process-oriented descriptionsand explanations of the phenomenon, grounded theory approaches are becomingincreasingly common in the IS research literatureor chosen from those applied within either a positivist or interpretivist context. In theinterpretivist tradition, the focus of these techniques is on openness to alternative perspectivesand on the identification of ambiguities in the data and the setting. Such strategies include:• field study – the object of study is subjected to direct observation by the researcher• questionnaire-based survey – this involves the collection of written data from interviewees,or the collection of verbal responses to relatively structured questions• interview-based survey – involves the recording of verbal data from interviewees, whicharises in relatively unstructured interviews or meetings• case study – involves the collection of considerable detail, from multiple sources, abouta particular, contemporary phenomenon within its real-world setting. Case study researchis the most common qualitative method used in IS, and can be positivist, interpretive,or critical, depending upon the underlying philosophical assumptions of theresearcher.A general shift in IS research away from technological to managerial and organisationalissues is seen to have led to an increasing interest in the application of qualitative researchmethods (Myers, 1997), as well as a wider acknowledgement of the importance of relevanceto practitioners. In their discussion Benbasat and Zmud (1999) maintain (and it shouldbe noted that both authors acknowledge they have mainly espoused a positivist researchtradition) that the nature of IS scholarship has placed an emphasis on rigour over practicalrelevance in order to establish IS as an academic discipline and to gain the respect of moreestablished disciplines. Among other reasons, they also cite the constraints imposed byresearch-oriented institutions which stress theory-based empirical research, economics, ormathematical modelling-based work in the pursuit of rigour.Software Engineering researchClarke (2000) also identifies an engineering approach to research. Based on a pragmatistview, strategies explore:• construction – involves the conception, design and creation (or prototyping) of an informationtechnology artefact and/or technique (most commonly a computer program,but sometimes a physical device or a method). The new technology is designed to183

intervene in some setting, or to enable some function to be performed, or some aim tobe realised. The design is usually based on a body of theory, and the technology isusually subjected to some form of testing, in order to establish the extent to which it(and, by implication, the class of technologies to which it belongs) achieves its aims• destruction – new information is generated concerning the characteristics of an existingclass of technologies. This is typically achieved through testing the technology, orapplying it in new ways. The design is usually based upon a body of theory.In her review of the state of research applicable to Software Engineering in particular, Marcos(2005) also adopts a pragmatic stance. She suggests that, as the nature of engineeringknowledge is essentially different from that of the traditional sciences, it is necessary todefine methods of research that are applicable to the specific problems of SE. Her approachdefines three scientific fields related to the character of SE knowledge: Software EngineeringScience, when the object of study does not exist outside the researcher’s mind and the researchprocess consists chiefly in creating it (eg a method, a model). Here research methods shouldbe characterised as qualitative and creative; Software Science, when the object of study is anyobject previously created by a process of research in SE Science. Research in this area candraw on quantitative, empirical models; Science of Information Systems, if the object of studyis focussed on the process of implementation and use of the objects already developed. Hereresearch should draw from qualitative methods appropriate to this cultural/social context.However, Xia (1998), in describing what is wrong with Software Engineering research methodologyfocusses on the lack of scientific theory as foundation, with the consequence that SEresearch lacks elementary requirements of science. In his opinion, the use of an artisan approachto research, which has been refuted by the history of science, implies the existingtheory (of SE) will have no convincing power: which requires a scientific attitude to researchto be adhered to. He concludes:There are many papers describing lessons learnt from software research experiments.When we will learn a lesson from the history of science and from the poorstate of SE research methodology?(Xia, 1998, p 65)The subfield of ESE (Experimental SE) (Wohlin et al, 2000) seeks to address this concern.This view discounts the synergy of research in an application as well as knowledge baseddiscipline. Potts (1993), for example, discusses three major changes (a greater reliance onempirical definition of problems, an emphasis on real case studies, and a greater emphasis184

on contextual issues) that he suggests are occurring as a result of the SE industry adoptingthe industry-as-laboratory approach, in which researchers identify problems through closeinvolvement with industrial projects and create and evaluate solutions in an almost indivisibleresearch activity. This approach emphasises what people actually do or can do in practice,rather than what is possible in principle. The subfield of EBSE (Evidence-based SE) takesas its goalto provide the means by which current best evidence from research can be integratedwith practical experience and human values in the decision making processregarding the development and maintenance of software.(Kitchenham et al, 2004)In summary it can be argued that CS, IS and SE research overlap, with some aspects of eachstudied within the research paradigms adopted by a specific discipline. A greater affinityexists between IS and SE research, in that there is an alignment in the philosophical stance.This stance also aligns with that of the educational researcher, so the assumption, that cognitionand understanding is not a thing located within the individual thinker but is a processthat is distributed across the knower, the environment in which knowing occurs, and theactivity in which the learner participates, is fundamental to how research is conducted. Thusresearch paradigms which simply examine these processes as isolated variables within laboratoryor other impoverished contexts of participation will necessarily lead to an incompleteunderstanding of their relevance in more naturalistic settings.For the both the learning scientist and the IT researcher, however, simply observing phenomenaas they naturally occur in the world may not be adequate, given a frequently explicittransformative agenda: the act of research almost inevitably influences the research domain.Although grounded in differing philosophical stances, both Design Research and Action Researchhave been proposed as means of meeting this need for contextual research for action.4.1.1 Design ResearchHistorically, design experiments have been the province of design sciences (science of theartificial (Simon, 1981)), including aeronautics, architecture, engineering and medicine. Designdisciplines have a long history of building their knowledge base through making theconstruction of artefacts and the evaluation of artefact performance following construction.Design experiments entail both ‘engineering’ particular forms of design process as prototypesand systematically studying those within the context defined by the means of supportingthem.185

Owen (1997, p 37) presents a general model for generating and accumulating knowledge thatis helpful in understanding design disciplines and the Design Research process:Knowledge is generated and accumulated through action. Doing something andjudging the results is the general model . . . the process is shown as a cyclein which knowledge is used to create works, and works are evaluated to buildknowledge.However, even within Design Research communities there is lack of consensus as to the preciseobjective - and therefore the desired outputs – of Design Research. Proposed (March andSmith, 1995; Rossi and Sein, 2003) outputs include:• constructs, which arise during the conceptualisation of the problem and are refinedthroughout the design cycle• models – a set of propositions or statements expressing relationships among constructs• methods – set of steps (an algorithm or guideline) used to perform a task. Methods aregoal directed plans for manipulating constructs so that the solution statement modelis realised• instantiation – the realisation of the artifact in an environment• better theories – since the methodological construction of an artifact is an object oftheorising for many communities (eg how to build more maintainable software), theconstruction phase of a Design Research effort can be an experimental proof and/orexploration of method. In addition, a fuller understanding of relationships betweenartefact (or system) elements potentially falsifies or elaborates on previously theorisedrelationships.Thus the goal of Design Research is to expose the completed design and its implementationin a way that provides insight into the local dynamics. This is achieved not simply throughsharing the design, but providing rich descriptions of context, guiding and emerging theory,design features of the intervention, and the impact of these features over time. However,it requires more than understanding one particular context; the relevance of the findingsto other contexts must be demonstrated. Design Research strives to generate and advancea particular set of theoretical constructs that transcends the context in which they weregenerated, selected, or refined. It is this focus on advancing theory grounded in naturalisticcontexts that sets Design Research apart from laboratory experiments or evaluation research.186

The design researcher values creative manipulation and control of the environment in additionto (if not over) more traditional research values such as the pursuit of truth or understanding.A practical or functional addition to a body of knowledge, codified and transmitted to thecommunity where it can provide the basis for further exploration, may be all that is requiredof a successful project. Indeed, it is precisely in the exploration of ‘wicked problems’ forwhich conflicting or sparse theoretical bases exist that Design Research excels (March andSmith, 1995; Carroll and Kellogg, 1989).The need to address issues of usability, scalability and sustainability are characteristics ofDesign Research: a lack of adequate consideration of the larger systemic constraints in whichthe context of intervention is a part is seen to lead to both impoverished designs as well asunder-specified theories that lack generalisable power. At the same time, the design experimentitself has similarities with more established methods for combining data of differentsorts. It can, therefore, be rightly placed as a member of the mixed methods family.Design Research in EducationDesign-based educational research, as conceived by Brown (1992) and Collins (1992), andadvocated by the Learning Sciences community, was introduced with the expectation thatresearchers would systemically adjust various aspects of the designed context so that eachadjustment served as a type of experimentation. This designed context is subject to test andrevision, and the successive iterations that result play a role similar to that of systematic variationin experiment, allowing researchers to test and generate theory in naturalistic contexts.Brown’s original concept of design experimenting assumed iterations between laboratory andclassroom. The research moves beyond simply observing in ways that allow the researcherto improve and generate evidence-based claims about learning.The application of the methods and metaphors of a design science approach to education hasa fairly short history, bound up with a shift from a social science approach to experimentationin learning research to a design experimental approach to the study of learning. As notedabove, Brown assumed that work would iterate between laboratory and classroom, capturingthe advantages of both. Within a design science approach, currently accepted theory isused to develop an educational artefact or intervention that is tested, modified, retested andredesigned in both the laboratory and the classroom, until a version is developed that bothachieves the educational aims required for the classroom context, and allows reflection on theeducational processes involved in attaining those aims.187

Design Research in ITThe new millennium saw a call for a return to an exploration of the technology that underliesIS research. Vaishnavi and Kuechler (2004) provides a comprehensive overview of the place ofDesign Research in IS as well as links to numerous resources in this field. What is importantis the conclusion drawn from this overview:We propose that Design Research is distinguished from design by the productionof interesting (to a community) new knowledge. In a typical industry designeffort a new product (artifact) is produced, but in most cases, the more successfulthe project is considered to be, the less is learned. ... In fact, most productdesign efforts in industry are preceded by many meetings designed to ‘engineerthe risk out of’ the design effort. The risks that are identified in such meetingsare the ‘we dont know how to do this yet’ areas that are precisely the targetsof Design Research efforts. This is in no way meant to diminish the creativitythat is essential to any design effort. We merely wish to point out that designis readily distinguished from Design Research (within its community of interest)by the intellectual risk, the number of unknowns in the proposed design whichwhen successfully surmounted provide the new information that makes the effortresearch and assures its valueVaishnavi and Kuechler (2004).4.1.2 Action ResearchAlthough there are examples of Action Research being undertaken as early as 1913, theterm can be traced to the social psychologist Lewin, who emphasised the need to studythe conditions for promoting social change, and, in so doing constructed a theory of ActionResearch (Lewin, 1946). Somekh (1989)defines it asthe study of a social situation, involving the participants themselves as researchers,with a view to improving the quality of action within it.To the basic themes notable in this and other definitions, namely empowerment of participants;collaboration through participation; acquisition of knowledge; social change (Masters,1995) should be added systematic enquiry; critical reflection; strategic action (Wortley, 2000).In all cases action researchers acknowledge the ambiguity, interconnectedness, complexity andconstant change inherent in the phenomena under study. So while developing a conceptualframework to interpret their findings, researchers also recognise that data collected on a188

selected group in a selected environment violate the basic principle of randomness that isnecessary for generalisation to be possible, as well as failing to identify any population towhich the results can be generalised. However, individual cases of Action Research arehelpful in developing and refining generalisable concepts and frames of reference (Galliers,1991), while Antill (1985) suggests the main purpose of Action Research is to build up usefulinsights, expertise and case law. Generalisations suggested by the participant researcher mustbe tested against other situations written up within the same framework. Powell (1991), inaddition, notes that such applied research often acts as a foundation for subsequent basicresearch. However, the strength of this form of research is said to lie in the fact that theresearcher’s biases are made overt in undertaking the research (White, 1985): the ‘cognitivefilter’ that is the researcher is accepted if not exploited.Carr and Kemmis (1986) suggests three conditions as individually necessary and jointlysufficient for Action Research to be said to exist:• a project takes as its subject matter a social practice, regarding it as a strategic actionsusceptible of improvement• the implementation and interrelationship of activities through a spiral• involves those responsible for the practice in each of the moments of activity.Action Research combines theory and practice through an iterative process of change andreflection. Mansell (1991) notes that a good Action Research project will add to knowledgeeither about the area in which it is applied, or about the process of applying it, while Mauchand Birch (1989) confirm that, at least, formative evaluation (where the focus is on theprocedure) is required. The assumption is that the enquiry process must develop naturallyrather than being constrained by preconceived ideas, hence an expectation of cycles of enquiry,with each iteration a potential for adding to the theory. Because the design is emergent, itcan illuminate the unique nature and value of the context in ways less likely to occur withquantitative research methodologies. The aim is to develop researchers who are both activepractitioners and reflective professionals.Action Research has been categorised into several types, based on the underlying assumptionsand world views of the participants (Carr and Kemmis, 1986; Grundy, 1982). These cause avariation in how the methodology is applied. At one pole a consensus model of society seesa research focus, at the other, a conflict model of society focusses on action (Wortley, 2000).A project of Technical/Technical-Collaborative/Scientific-Technical/Positivist type has thefollowing characteristics: it is instigated by an ‘expert’ or authority figure, is product-directed189

(promoting more efficient and effective practice) but promotes personal participation. Theresult is accumulated predictive knowledge based on validation and refinement of existingtheories. This type of Action Research can be seen as a move towards the ‘scientific’ as wasunderstood in terms of the positivist aspirations of the social and educational science of the1950s.A project of Mutual-Collaborative/Practical-Deliberative-Interpretivist type is based on dialoguebetween researcher and practitioner and emphasises participant collaborations. Thistype of Action Research allows for flexibility in a trade-off between measurement and controland human interpretation, interactive communication deliberation and negotiation. Althoughlasting change is a result, it is connected with the individuals directly involved in thechange process, and thus promotes praxis. Much of the educational research reported in theliterature is of this type, while the work of Schön advocated this type of Action Research.The third type of Action Research Enhancement/ Critical-Emancipatory/ Critical Science isan approach for resolving conflict between espoused and applied theory, and has two primepurposes: an increase in the closeness between problems encountered by practitioners andthe theory used to explain and resolve the problem, and to assist practitioners in identifyingand and making explicit these problems by raising the collective consciousness. Habermas(1972) presents a theoretical model, based on social critique, for understanding this type ofresearch. The initiative to undertake the project is often a confrontation with the theory,and results in the expansion of both theory and practice.Carr and Kemmis (1986) suggest that the ‘essential’ nature of Action Research involvesconnection with social movement: the connection is intrinsic. Through research on the hereand now improvement is obtained now, but also in relation to the wider social structures andprocesses:[it] is always connected to social action: it always understands itself as a concreteand practical expression of the aspiration to change the social (or educational)world for the better through improving shared social practices, our shared understandingsof these social practices and the shared situations in which thesepractices are carried out. It is thus always critical...trying to understand and improvethe way things are in relation to how they could be better. But it is alsocritical in the sense that it is activist: it aims at creating a form of collaborativelearning by doing...It aims to help people understand themselves as agents, as wellas the products, of history....[It] is also committed to spreading involvement andparticipation in the research process(Kemmis, 1993, p 2)190

Rather than viewing Action Research as a unique self-contained methodology, it is preferableto consider it as a family of applied social research that borrows underpinnings and techniquesfrom other methodologies in order to address the requirements and outcomes of both ‘action’and ‘research’. Within its holistic framework, Action Research is flexible enough to adoptany methods that satisfy its raison d’être: to inform practice and enable practice to refinetheory.This supports the stance taken by Zuber-Skerritt (1982). For her, Action Research is alwayscritical, both in the initial reflection before acting, and in the role of critical change agentsof the environment and themselves.Action Research in EducationAlthough widely used in education during the 1940s and 1950s, through Lewin’s direct influence,the move to a scientific focus and associated quantitative methods saw the decline ofAction Research. It was criticised on the basis of methodology, effectiveness and practicality.However, influenced now by Stenhouse (1975), and based on work by researchers in the UKand Australia it returned to the fore during the 1970s. A strong rejection of positivist modelsof research in education, in favour of critical enquiry linked to human action by Kemmis andMcTaggert (1988) and others also led to the development of Action Research as a means ofachieving emancipation from traditional ways of educational thinking.Educational Action Research is founded on the writing of Dewey, who believed that professionaleducators should become involved in community problem-solving. For educationalresearch, Action Research provides several key advantages: it contributes to reconstructingthe theory and knowledge base needed for enhancing practice; it develops research skills inpractitioners; it builds a collegial networking system; it helps practitioners identify problemsand seek solutions in a systematic fashion; and it can be used throughout the discipline (Borget al, 1993).In addition, its emergent design can illuminate the unique nature and value of the situationunder study in ways less likely to occur within a purely quantitative research framework.Zuber-Skerritt (1982) concludes that systematic, controlled Action Research can lead topractitioner contributions to the advancement of knowledge in higher education.Several models exist for undertaking Action Research in education, all based on Lewin’sconcept of a spiral (see Figure 4.1), that incorporates a cycle of problem diagnosis, actionintervention and reflective learning, all leading to continuous improvement of practice andan extension of personal and professional knowledge (Zuber-Skerritt, 1995). Examples range191

Figure 4.1: Lewin’s spiral of Action Research (Kemmis and McTaggert, 1988)from the four-step process of Perry-Sheldon and Allain (1987) (reconnaissance, plan, act,reflect) or Wood (1988) (plan, act, observe, reflect) to a more comprehensive seven-stepprocess of Borg et al (1993).A definition of educational Action Research was formulated by the Australian National InvitationalConference on Action-Research (ERDC, 1981, p 159):Educational action-research is a term used to describe a family of activities in curriculumdevelopment, professional development, school improvement programmes,and systems planning and policy development. These activities have in commonthe identification of strategies of planned action which are implemented, and thensystematically submitted to observation, reflection and change. Participants inthe action being considered are intricately involved with all of these activities.This definition makes it explicit that a systematic discipline of strategic inquiry must befollowed, while at the same time recognising the diversity of approaches that fall under itsmethodological banner.Kemmis (1993) concludes that the task of educational researchers involves taking concreteand explicit steps towards changing the theory, policy and practice of educational research,and participating in the work of changing education theory, policy and practice. ActionResearch places the teacher in the dual position of producer of education theory/policy anduser of that theory/policy through practice.192

Action Research in ITWithin Information Systems research, Action Research is celebrated as unique in the way itassociates research and practice, so research informs practice and practice informs researchsynergistically (Avison et al, 1999). Although a survey of the literature shows that the academiccommunity almost totally ignored Action Research – Avison et al (1999) reports only29 articles on Action Research, spanning the years 1971 to 1995, by the end of the 1990sit began growing in popularity for use in scholarly investigations of information systems,spurred by the relevance of research results. Action Research was explicitly introduced tothe IS community as a purely research methodology by Wood-Harper (1985). Like Mumford(1983) and Checkland (1981) Wood-Harper also incorporated Action Research conceptsinto an action-based systems development methodology. Double-loop organisational learning(Argyris and Schön, 1978), strongly linked to Action Research in IS, also took as one of itsinspirations the work of Lewin.Although human and social factors have a very strong impact on the success of software developmentendeavours and the resulting system, much of Software Engineering research in thelast decade is technical, quantitative and de-emphasises the people aspect (John et al, 2005).The call to broaden the focus of empirical SE research to address the human role in softwaredevelopment was made in the late 1990s, with researchers exhorted to study nontechnicalissues and the intersection between the technical and nontechnical in SE (Seaman, 1999).Here the approach advocated is that of interpretive empirical research – in particular controlledexperiments that take subjects out of their work context into controlled environmentsto enable establishing cause-effect relationships. Although this approach was advocated as ameans of taking advantage of the strengths of the blend of technical and human behaviouralaspects in SE, it is flawed: longitudinal effects are hard to examine in controlled environmentsand short term effects are not extremely relevant for long-term industrial projects.An acknowledgement of the place of Action Research in SE is very much more recent. For example,John et al (2005) advocates the full range of qualitative methodology and perspectives,including Action Research, as possibly providing deeper insights into Software Engineeringthan hard-core controlled experiments where samples are drawn from student populationswhile the study implies statements about industrial settings.Mixed methodology approaches are only recently being reported as applied in SE: an exampleis CALIBRE (Fitzgerald, 2004) an FP6 project funded through the European Union.CALIBRE uses a scientific research backbone, qualitative and quantitative case studies withdata formalised and categorised for subsequent analysis, and a number of Action Researchprojects which will undergo an initial cycle of feedback and reflection. The project has two193

outcomes - an action outcome whereby intervention helps to address the real world problem,and a research outcome whereby the research lessons from the project are harvested andtheoretically integrated.4.1.3 Educational research as Design or ActionThere are many similarities between Design Research and Action Research applied to education,in particular the Critical Science type of Action Research:• both are founded on the work of Dewey• both see the context of the learning as an important component of the environment• both conceive of a joint role: designer and researcher (although for Design Researchthe researcher is very possibly not the teacher)• both involve producing demonstrable changes at the local level• both advocate a cyclic/iterative approach to research• both borrow underpinnings and techniques from other methodologies• both advocate mixed methods for conducting the research and reporting on it.However, a critical component of Design Research is that the design is conceived not just tomeet local needs, but to advance a theoretical agenda, to uncover, explore, and confirm theoreticalrelationships in developing generalised models. Although providing credible evidencefor local gains as a result of a particular design may be necessary, it is not sufficient. Theresearcher moves beyond a particular design exemplar to generate evidence-based claims thataddress contemporary theoretical issues and further the theoretical knowledge of the field.Design researchers not only recognise the importance of local contexts but also treat changesin these contexts as necessary evidence for the viability of a theory.Although design researchers are comfortable with alternative world-states – changes to thestate-of-the-world through the introduction of novel ‘artefacts’ is the primary aim of DesignResearch – there is an obvious contrast with positivist stance where a single, given, compositesocio-technical system is the typical unit of analysis; even the problem statement is subjectto revision as a Design Research effort proceeds. However, the multiple world-states of thedesign researcher are not the same as the multiple realities of the interpretive researcher:many if not most design researchers believe in a single, stable underlying physical realitythat constrains the multiplicity of world-states. Physical laws are tentatively composed into194

a configuration that will produce an artefact with the intended problem solving functionality.This virtually demands a natural-science-like belief in a single, fixed grounding reality:an artefact is constructed and its behaviour the result of interactions between components.To the degree the artefact behaves predictably, its meaning is precisely the functionality itenables in the composite system (artefact and user). The design researcher is therefore a pragmatist.Bunge (1984) implies that Design Research is most effective when its practitionersshift between pragmatic and critical realist perspectives, guided by a pragmatic assessmentof progress in the design cycle.Within Action Research there is an explicit acknowledgement that the change is not purely theresult of the intervention – change is an ongoing occurrence in the phenomenon under scrutinyand the scrutiny itself a trigger for change (hence the inappropriateness of ‘experimentation’).In addition, the value of Action Research is based on the clear picture of the place of theresearcher as participant and influence on the environment. The researcher is not just simplyobserving interactions but is interventionist within the environment.Consequently, Action Research becomes the strategy of choice for this research into educationfor SE:• it takes place in the natural setting and does not treat the classroom as a laboratory• the researcher’s involvement in the experiences of the participants is explicit• it uses multiple methods that seek to build rapport with participants and enable themto be involved in the data collection and analysis• it is emergent rather than tightly predefined• it is inherently interpretive – the data is filtered through a personal lens situated in aspecific moment• it views social phenomena holistically• it assumes reflexivity• the reasoning, largely inductive, is based on a cyclic process between data collectionand analysis and problem identification/re-formulation.These characteristics align well with the characteristics of the domain, where:• Requirements Engineering takes place in an organisational setting• the RE’s involvement is explicit195

• it applies multiple techniques that seek to build rapport with the stakeholders, whomust be involved in the requirements collection and analysis• it is emergent rather than tightly predefined• it is inherently interpretive – based on models constructed and filtered by all stakeholderperspectives• it cannot view the system as isolated• it assumes reflexivity• the reasoning, largely inductive, is based on a cyclic process between data collectionand analysis and problem identification/re-formulation.4.1.4 Strengths and weaknesses of a research designA prevailing view regarding the validity of a research approach is based on the constructs ofreliability, objectivity, and internal and external validity. Anderson and Herr (1999) arguethat while conventional research approaches, which rely on generalisability and replicabilitycan apply these validity criteria, Action Research (or any research where interpretation andPraxis are primary goals) demands different criteria for validity, reliability, and researchquality.In relation to Orlikowski and Baroudi (1991)’s list of main possible weaknesses of nonpositivistresearch, the main issues to be addressed include:Contingency of research findings – research rigour is linked to the reliability of the instrumentsfor data collection and analysis, and the internal and external validity of thefindings. While a high internal validity does not always increase the generalisability ofthe findings, high external validity is seen to do so. Models developed through ActionResearch, due to its focus on in depth and often longitudinal studies are viewed as problematicfor external validity. However, one approach to this issue is to examine howthe findings hold across iterations. These act to expand the project scope, analogousto choosing a wider ‘sample’ in statistical studiesLow control of environment – testing or producing strong theories is seen to require highlevels of environmental control. However, such control, by definition only focusses onpredefined variables and links, ignoring others that may be as relevant to the understandingof the situation under study. Action Research overcomes this by collecting196

vast amounts of data about everything. High control is also likely to lead to artificialaction on the part of the participants, adding an element of bias regardless of theimplied objectivity of the research. This lack of manipulation, a strength of ActionResearch, is an advantage in generating relevant and valid knowledge for the contextPersonal over-involvement – some researchers contend that good research is hindered bythe personal biases introduced through approaches such as Action Research. However,Action Research is seen to foster change, generally accompanied by emotionalinvolvement on the part of all participants – either resistance and apathy or supportand enthusiasm. The researcher as ‘cultural insider’ provides considerable benefits forthe study: intimacy with the subcultures and alternate realities that comprise theoverall context reality provide valuable insights for interpretation of the results of thestudy. Edwards (1999) states that the peculiar benefit of deep insider research is theknowledge the researcher brings concerning history and cultures and an awareness ofbody language, semiotics and slogan systems operating within the cultural norms ofthe organisation/group, with much of this undiscoverable to outsiders. As Eisenhart(1989) notes, discussing the use of case study research in theory building, the underlyingassumption is that the intimate interaction with actual evidence often produces theorywhich closely mirrors reality. The iteration of cycles may work towards reducing potentialdistortions in the findings caused by over-involvement. Personal identification withmethodologies, techniques, instruments etc, introduced as part of the research is also apotential source of personal bias, but can be minimised through a clear understandingthat one outcome of the research is learning (which may indicate inappropriatenessrather than the opposite)Lack of generalisability – the generalisability of studies involving humans in a social settingis problematic: the basic principle of randomness that is necessary for generalisationto be possible is missing in Action Research, which also fails to identify any populationto which the results can be generalised. Therefore, some researchers suggestconstructs that establish the ‘trustworthiness’ of the study as more appropriate. Theseprovide truth value through credibility, applicability through transferability, consistencythrough dependability and neutrality through confirmability (Erlandson et al,1993) so thatthe judgements arrived at [are] not gratuitous or the result of subjective whim.(Madison, 1988, p 28)197

In conclusion, Action Research, as a desirable alternative to approaches based on a positivistideology and scientific philosophy, is most appropriate in certain contexts. One might arguethat the strengths and weaknesses of Action Research should be considered on their ownmerit, and in the context of the research undertaken. As an example, although low controlof the environment is noted above as a weakness of Action Research, and compensatedby quantity of data, this extensive data collection facilitates an approach that draws fromthe grounded theory methodology. In effect, applied to Action Research, the data drivesan emergent intervention. This differs from stricter adherence to Action Research, whichassumes a cycle is completed before data analysis is undertaken.Table 4.1: Criteria for the validity of Action Research (based on Krefting (1991) and Guba andLincoln (1994))ValidityCriterionCredibilityTransferabilityDependabilityConfirmabilityAddressed byprolonged and varied field experience; time sampling; reflexivity(field journal); triangulation; member checking; peer examination;interview technique; establishing authority of researcher; structuralcoherence; referential adequacynominated sample; comparison of sample to demographic data;time sample; dense descriptionaudit; dense description of research methods; stepwise replication;triangulation; peer examination; code-recode procedureaudit; triangulation; reflexivityIt may be that alternative frameworks are needed to support Action Research as rigorous andhigh quality, without sacrificing its relevance. Krefting (1991) identifies several strategies asapplicable in ensuring the rigour and quality of qualitative research. These can be appliedto Guba and Lincoln (1994)’s model, which describes four general criteria for evaluation ofresearch (see Table 4.1).Anderson et al (1994) also suggest criteria for testing the validity of Action Research, fromKock et al (2000)’s Action Research perspective. These are listed in Table 4.2.Table 4.2: Principles for the evaluation of Action Research (Anderson et al, 1994)ValidityCriterionOutcomeProcessDemocraticCatalyticDialogicAddressesDid it solve the problem?Was the activity educative and informative?Was the research undertaken in collaboration with all involvedwith the problem under investigation?Did the research transform the realities of those involved?Could the research be discussed with peers in different settings?These are seen to address generalisability issues. The framework described in Table 4.3198

lists the criteria suggested by Anderson and Herr (1999) and those of Krefting (1991) byproviding strategies aimed at addressing the quality issues raised by undertaking ActionResearch. Evaluation of the research undertaken will take into consideration these criteria.Table 4.3: Addressing possible weaknesses of non-positivist research (based on Krefting (1991)and Anderson et al (1994))Validity Addressed byCriterionOutcome Did it solve the problem?Process Was the activity educative and informative?Democratic Was the research undertaken in collaboration with all involvedwith the problem under investigation?Catalytic Did the research transform the realities of those involved?Dialogic Could the research be discussed with peers in different settings?Credibility prolonged and varied field experience; time sampling; reflexivity (field journal); triangulation; member checking; peer examination;interview technique; establishing authority of researcher; structuralcoherence; referential adequacyTransferability nominated sample; comparison of sample to demographic data;time sample; dense descriptionDependability audit; dense description of research methods; stepwise replication;triangulation; peer examination; code-recode procedureConfirmability audit; triangulation; reflexivity4.2 Data acquisition and evaluationData are not interpreted after they are collected...data collection itself is an interpretiveprocess(Ezzy, 2002, p 77-78)Evaluation of learning interventions comprises the acquisition and analysis of data that providesuseful feedback on the impact of the intervention. From the moment the researcherbegins to consider and reflect on the research, data analysis and interpretation have commenced.Approaches to evaluation may be categorised on the basis of the overarching perspectivetaken for the gathering and analysis of data. These strategies also span the scientificto participatory/action continuum. Scientific models of data collection and analysis focuson the need for objectivity and the reliability and validity of the quantitative methods advocated.A formal qualitative model emphasises the importance of observation, the valueof subjective judgement and the need to retain the phenomenological quality of the contextunder study, while participatory models of research acknowledge the qualitative model but199

emphasise the importance of participants in the study and assume change will take place, ifonly because the study has been conducted.Since change is accepted as a fundamental goal of this research, an evaluation strategy thatapplies a qualitative approach to the collection and analysis of data is seen to have thepotential to provide the information required. Therefore, rather than viewing qualitativeresearch as a distinct research approach (eg Cash and Lawrence (1989)) it can be arguedthat it is primarily a perspective to be used within the chosen research methodology.4.2.1 Quantitative versus qualitative methodsBoth quantitative and qualitative research may be interpreted from many perspectives, generallybased on the epistemological and philosophical stance taken by the researcher. Puristquantitative research come from a positivist philosophy, focusing on objectivity, deduction,confirmation, theory/hypothesis testing, explanation, prediction, standardised data collectionand statistical analysis. On the other hand, purist qualitative research come from aconstructivist background, focusing on subjectivity, induction, discovery, theory/hypothesisgeneration, the researcher as the primary data collection instrument, and qualitative analysis(Johnson and Onwuegbizue, 2004). Some purists advocate the incompatibility thesis, whichmaintains that qualitative and quantitative research paradigms and methods should not bemixed.However, the two research types do have characteristics in common. Both describe data,construct explanatory arguments from data, speculate about why the results occurred as theydid, incorporate safeguards to minimise bias and invalidity, and attempt to provide warrantedassertions about humans and their environments (Johnson and Onwuegbizue, 2004).Quantitative research has some characteristics of qualitative research: subjectivity is usedwhen deciding what to study, developing instruments that are believed to measure the targetconstruct, choosing the statistical method of analysis, interpreting scores, selecting thealpha level of significance, deciding what elements of the findings should be emphasised, anddeciding which findings are significant from a practical point of view, are all methods usedin both paradigms.Conversely, qualitative research has some characteristics of quantitative research, in thatmember checking, triangulation, negative case sampling, pattern matching and external auditsare used to ensure rigorous research in both paradigms.Qualitative research is predicated on the assumption that each individual, culture and settingis unique and requires incorporation into the research. While quantitative research is200

typically associated with the process of enumerative induction (eg to infer a characteristicor relationship between variables of a parent population), in qualitative research the conceptsand categories, rather than incidence and frequency are primary: it does not survey theterrain, but rather mines it (McCracken, 1988).Qualitative research exhibits the following characteristics:• it allows continuous reflection on the research in progress, with room for ongoing alteration• the relationship between researcher and participant is important - rather than strivingfor objectivity and lack of bias, the researcher deliberately interacts in a personalway with individuals in the study, requiring data collection procedures to be open tomodification depending on the actions of the individual• researcher intuition and judgement are valid inputs• differences in frames of reference between researcher and participants are acknowledgedas value-laden and leveraged to provide insight• the researcher acknowledges that generalisations are tentative and context-dependent,liable to lose validity in different settings or from one time period to another.The focus on context in qualitative research is reflected in the encompassing of the completephenomenon, and development of broad themes and patterns as a basis for (more-or-lesssubjective)analysis. Data collection is based on the researcher’s ability to observe andinteract with participants in the environment, while analysis typically yields verbal descriptionsderived from that interaction with the effects of the researcher’s participation consideredpart of the data. In contrast, quantitative research is generally limited to providing numericaldescription and less elaborate accounts of human perception or motivation.As noted previously, the issue of generalisability does not arise in the same way as in quantitativeresearch. The concern is about the link of the findings to other similar cases or setsof conditions: establishing a theoretical link within each case in order to determine how farthe findings can be extrapolated to the theory with which the research has engaged.Qualitative research is characterised by vast amounts of data, which need to be interpretedand summarised in relation to the research questions. This may be achieved through narrativeaccounts (case study or life stories), codification, classification and thematisation that arethen grouped and compared. Although subject to potential bias that begs a disciplinedapproach to overcome, Action Researches endorse such methods as ‘appropriate’.201

4.2.2 Mixed methodsSome researchers (eg Reichardt and Cook (1979); Creswell (2003); Johnson and Onwuegbizue(2004)) believe that, rather than discounting one or other research design, social phenomenaare best studied through a combination of both, either to complement each other throughconcurrent studies or to discover patterns that can be measured later. Mixed methods researchhas its underpinnings in phenomenology, grounded theory and ethnography and helpsprovide a complete picture of a research problem, providing information about both theprocess and the outcomes of the study (Creswell, 2003).Johnson and Onwuegbizue (2004) describe mixed methods research as the natural complementto traditional qualitative and quantitative research by combining qualitative and quantitativeresearch techniques at the levels of sampling, data collection and analysis. They considermixed methods research as expansive, creative, inclusive, pluralistic, complementary andeclectic – it recognises the commonalities between the two types of research, and frequentlyresults in superior research by yielding a more complete analysis.McNeill and Chapman (2005) advocate a scale based on the size of the sample and the needfor personal involvement. At one end reliability and representative is based on large samples,at the other validity of participant observation implying small samples. In all scenarios theuse of diverse methods in tackling a research problem argues that researchers should beflexible and select a range of methods that are appropriate to the research problem beinginvestigated. However, the selection of qualitative and quantitative methods should resultin complementary strengths and non-overlapping weaknesses (Johnson and Onwuegbizue,2004), implying a robust research design as essential.A mixed method approach to research can be seen to have the following strengths (Johnsonand Onwuegbizue, 2004):• words, pictures and narrative can be used to add meaning to numbers, while numberscan be used to add precision to words, pictures and narratives. Thus insights andunderstanding gained in a mixed methods study may be missed when only a singlemethod is used• it provides the strengths of both quantitative and qualitative research, and can minimiseany bias inherent in a particular method• researchers are not restricted to a single method or approach, so studies can answer abroader and more complete range of research questions• the strengths of an additional method can overcome the weaknesses or biases of the202

primary method• can provide stronger evidence for a conclusion through convergence and corroborationof findings• a combination of qualitative and quantitative research methods produces more completeknowledge for informing theory and practice.However, weaknesses should also be noted (Johnson and Onwuegbizue, 2004):• mixed methods research tends to be more expensive, time-consuming and resourceintensivethan single method research• it can be difficult for a single researcher to carry out both qualitative and quantitativeresearch• more extensive training is required to exploit both approaches• purists may discount such research, taking the view that researchers should always workwithin a qualitative or quantitative framework, not in both• some details of mixed methods research are yet to be worked out fully.4.2.3 Strategies for data acquisitionAs described above, the holistic nature of Action Research allows for the different researchmethods, tools and instruments of the mixed method approach to be exploited. Althoughgenerally common to qualitative research, these various methods may also be drawn fromthe quantitative arena, but always with the aim of observing and recording the context ofthe study. From that perspective, the methods can be viewed as emergent – planned on thebasis of discoveries in the data, and the reflection that follows.Kember and Kelly (1993) divide observation techniques common to Action Research intothree categories:• diagnostic devices include student assessment, learning inventories, interaction schedules,diagnosis of conception (eg mind maps). Particularly with assessment items,the difficulty is to demonstrate that the qualities being investigated are those being assessed,with understanding of concepts usually assessed formally. However, an alternatemethod sees the use of concept maps, where the definition and relationship betweencore concepts is explored qualitatively, while learning inventories provide empirically203

derived/validated measures that, in general, have been developed to examine the qualityof learning. Interaction schedules provide a profile of interaction and activity withina particular context, either through time or event sampling by an observer, or analysisof taped material• records such as diary/journal, supporting documents, tapes. The journal may be consideredto apply to all participants of the Action Research environment, and is a mechanismfor recording and reflecting on action and thought. Supporting documentationrefers to the more formal background to the interventions actioned: syllabus, documentsfor course development and accreditation, student assessment planned. Tapingis considered an easy mechanism for data collection, but may have the effect of influencingparticipant behaviour. In addition, ethical issues may inhibit the taping ofcomplete action cycles• feedback from students should provide opportunities for participants to raise issues andconcerns, either through tightly structured/open questionnaires or interviews. Bothformal and informal questionnaires provide a mechanism for participants to addressareas of interest to the researcher, while interviews allow for both general impressionsto be uncovered and tight analysis to be undertaken.Together, these techniques provide mechanisms to validate the observations through a processof triangulation, a term borrowed from psychological reports and developed by Denzin(1970). The term has been taken to mean more than one method of investigation: withinmethods(same method on different occasions, as well as researcher/observers’ reflection ontheir participation) and between-method (different methods in relation to the same objectof study), as well as the use of multiple investigators and theories. The value of such anapproach is that the discrepancy between what people say and what people do can be overcome.Since this is a fundamental issue in education for RE (see Chapters 2 and 3) it is mostappropriate to apply a mixed method approach to this study.4.2.4 Strategies for data evaluationEvaluation is often referred to as being formative or summative if not diagnostic. In thisstudy, formative evaluation is undertaken on the basis of a match between the design ofthe learning environment and the ‘real world’ as described by practitioners in the discipline.The latter provide a specification for a theory-based formal modelling in terms of conceptual,learning and instructional design factors. All information provided by the participants in thestudy may also be considered as formative user-based evaluation of the learning environment.204

In contrast, summative evaluation examines the impact of the context after the fact. This maybe ascertained from formal assessment tasks as well as direct observation. As a component ofsummative assessment, impact evaluation looks at the flow on from the intervention, eitherin the short term or longer.The data collected and analysed for both of these evaluation strategies is also availablefor monitoring analysis, to assess the integration of the innovation into the overall teaching/learningprocess. It may also comprise meta-analysis, which looks at the weight of evidence:the extent to which the results are consistent across a variety of different studies/casesby integrate the results to arrive at an overall or summary judgement on an evaluation issue.The Action Research strategy adopted in this study utilises both formative and summativeevaluation techniques but also allows for monitoring analysis to take a prominent position:the integration of data collection/data analysis allows the research to be shaped and reshapedby the participants in the research, based on the themes identified through examination ofthe data. This thematic analysis aims to identify important elements within the data, withthe categories into which the themes will be sorted induced from the data itself. It is thebasis of grounded theory research (Glaser and Strauss, 1967; Glaser, 1992). According toOrona (1990), the value is in the approach’s acceptance (if not reliance) on intuition andcreativity, nuances and detail.The first stage, open coding, is a way to generate an emergent set of categories and their properties.This phase is seen as requiring effort and reflection, and rather than being straightforward,the process is seen as non-linear, somewhat chaotic and the constant comparisonrequired demanding. Ezzy (2002) suggests this as a strength as well as weakness: it leadsto new ways of understanding as new ideas are put together or insight into participants’interpretations is acquired. Once the coding is applied to the data, the next phase, axialcoding attempts to abstract from the detail of the categories identified to isolate themes.Each of these identifies the context for all categories within it. Finally theoretical codingidentifies the core category to be verified or revised based on secondary analysis of the dataor triangulation methods.Other authors name this approach framework analysis (Lacey and Luff, 2001; Richie andSpencer, 1994), template or codebook analysis (King, 1998).Coding, then, is a process of dis- and re- assembling the data to produce new understandings,with the ability to assign codes dependent, in part, on the human observer’s familiarity withthe context. As such, our data collection procedures involve the researcher engaged in directinvolvement in the learning environment.Reflection is also an aspect of evaluation, and in this study, plays the important role of205

Table 4.4: Reflection in the Scholarship of Teaching model (Kreber, 1999)ContentCurriculum Pedagogical InstructionalWhat are the goalsof my teaching?What do I knowabout how studentslearn?What instructionalstrategies should Iuse?Process How conscientioushave I been inidentifying thisgoal?How effective amI in promoting itsachievementHow effective havemy strategies been?PremiseHow does my goalmatter? What arethe alternatives?What are alternativestrategies?Why does it matterthat I use this strategyguiding the journey, and triggering pedagogical growth in the researcher. Thus another aspectof learning, and reflection on action, is that of ontology. Side-by side with epistemology,this study aims to challenge (at least, but also, hopefully transform) the researcher’s wayof being a tertiary teacher. As DallAlba (2005) notes, knowing is not simply somethingwe possess, but who we are. Contextualised, knowing is not exclusively cognitive, but iscreated, enacted and embodied. Thus as the practices described within the Action Researchcycles are questioned, analysed and evaluated, a heightened awareness of new possibilitiesfor educational practice is engendered. Cowan (1998), in describing reflection-for-action as atype of anticipatory activity involving reflecting on past experience to identify needs, establishpriorities, aspirations and objectives, addresses this role for the reflective process.Within this study, therefore, reflection is seen as both analytical and evaluative: not onlyposing ‘How do I do it?’, but also ‘How could it be better?’ (Cowan, 1998). In this way,evaluative reflection encompasses the results of doing analytical reflection while also involvingmaking a decision on what to do next on the basis of judging whether the development whichhas taken place is adequate to proceed to the next stage.The approach to reflection taken is based on Kreber (1999)’s Scholarship of Teaching model(see Table 4.4). This categorises reflection as content, process, and premise (double looplearning) within the domains of instructional (eg writing and sequencing learning objectives,choosing readings, facilitating discussions and group work, preparing syllabi, constructingand evaluating assessment), pedagogical (eg how students learn, how to respond to differentlearning styles and approaches to studying, how to facilitate critical thinking and selfmanagementin learning, or how to influence students motivation to learn) and curricular (ofthe goals, purposes and rationales for the unit, how it fits into the larger curriculum and howthe teaching contributes to the universitys societal and cultural role) knowledge. Since the206

study requires data in each of these education domains, it is appropriate that the reflectionis similarly targeted.4.3 Reporting on the researchWhile data analysis and project evaluation lead to an interpretation of the study undertaken,the process of writing about the study and its context is a mechanism for both the gestationof ideas, and validation by peer experts.As well as maintaining a rough project draft which fleshed out the themes and backgroundrequired for the research, the idea of ‘first drafts’ (Becker, 1986) worked to focus a particularaspect of the study. In this way, discussing the study within the classes both withparticipants and with past students, presenting at conferences, and engaging in discussionand research with colleagues provided a context in which the ideas became clearer, the argumentsdeveloped more structure and logic and problems raised themselves and needed to beresolved.The next chapter reports on the Action Research cycles undertaken, and uses as referencethe peer reviewed publications produced through this process.4.4 Summary of approachIn summary this overarching framework for the study may be seen to address Avison et al(1999)’s framework for Action Research, which requires four elements to be considered:• the category of the Action Research used and its focus – it is an Enhancement/ Critical-Emancipatory/ Critical Science approach, valid for resolving conflict between espousedand applied theory. How practitioners do RE is in conflict with how educators teachRE. One of the purposes of this research is to increase the closeness between problemsencountered by practitioners in novice REs and the formal education (implying theoriesof learning used), and to assist practitioners (both in professional practice andeducation) in identifying and making explicit these problems by raising the collectiveconsciousness• the tradition and beliefs implied by its assumptions – as has been discussed in precedingchapters, the discipline of RE is presented from both a positivist and non-positivistperspective. These perspectives have major influence on the underlying knowledgestructures, skills (physical and cognitive) and techniques the Requirements Engineer207

has recourse to in order to achieve a successful RE task: the beliefs held affect how theRE student learns to become a competent practitioner. The view taken in this studyis that the discipline is a social one, requiring construction and interpretation in orderto create an evolved fit between the system views of all stakeholders.The discipline of education (and its research) can also be tackled from positivist andnon-positivist perspectives. Again, the stance taken in this study is that subjectiveinterpretation and reflection provide insight to the research context. Thus this researchstudy is situated within a broadly based constructivist epistemology and the data isviewed through a variety of lenses including those of lifelong learning and adult education• the research process, including theme, level of organisational involvement, extent ofchange, role of researcher◦ the research process is based on a model for Action Research described in Borget al (1993)’s work and is discussed in the next chapter. The triggering themesfor each cycle of the Action Research process are described below, and are basedon feedback and reflection on the preceding cycle◦ the organisation involved is a university offering an undergraduate Engineering degreein Software Engineering. Resources, in the context of a unit in RequirementsEngineering offered as a core component of this degree, as well as a supportingstructure to enable the study (eg in terms of freedom to apply interventions,ethics approval for data collection, and grants for the development of some technicalcomponents of the learning environment) were all available for the durationof the study. The results of cycles of this study informed the teaching practicefor all core Software Engineering units within the degree, so that by the end of2003, no ‘traditional’ teaching was occurring. During 2005 some elements of thework undertaken during this research were applied throughout the Engineeringcurriculum◦ change is seen as a gradual process requiring, in this context, a cultural changetowards learning both for students participating and academic staff who initiallydeal with the results of this change in students and later in implementing similarchanges themselves◦ the researcher is an agent of change, initially acting, reflecting and learning at apersonal level, later triggering change in student participants during the researchand in other SE units within the programme and finally being instrumental in thechanges instituted within the learning for Engineering at Murdoch208

• style of presentation adopted – action researchers are accountable in that they aim tomake their learning process and its results public, both to each other and to otherinterested practitioners, using accessible terminology. Brew (2001) suggests one viewof research is to consider it a journey: the researcher embarks on a voyage of discovery,with a set of guidebooks and maps to enable informed decisions on which route to take.However, the journey itself is unique, influenced, as it is, by the researcher’s previousexperience, beliefs and expectations. The journey is also transformational, both for thetravellers, other more transitory participants in the journey, and ideally for the domaintraversed during the journey. In this view, an emphasis is placed on the assimilationof research into one’s own life and understanding. In the transformation conceptionintellectual activities in which the researcher engages, whether or not they appear tohave a direct bearing, are viewed as relevant to the research because they inform thelife issues which underpin the research questions.Brew (1999) suggests that this conception of the nature of research influences thetypes of projects researchers feel comfortable pursuing, the choice of methodology, thequestions, ideas and issues pursued and the ways in which the work is carried out.This research is presented as a reflective travel diary.4.5 ConclusionThe current focus on quality in education has engendered vigourous discussion on techniquesand mechanisms for the evaluation of teaching/learning. An Action Research approach cancontribute very positively to activity within the tertiary sector concerned with teaching qualityissues:Through systematic, controlled action research, higher education teachers can becomemore professional, more interested in pedagogical aspects of higher educationand more motivated to integrate their research and teaching interests in aholistic way. This, in turn, can lead to greater job satisfaction, better academicprogrammes, improvement of student learning and practitioners insights and contributionsto the advancement of knowledge in higher education.(Zuber-Skerritt, 1982, p 15)Traditionally, teachers in a tertiary environment have not been encouraged to draw upontheoretical developments as a means of improving the learning environment. Action Researchoffers a systematic approach by putting the teacher in the dual role of producer of educational209

theory and user of that theory, thereby bringing theory and practice closer together. Brew(1999)’s journey metaphor, with its transformative nature, aligns well with the focus oninterventions that are the basis of this study.This research is undertaken with an acceptance of the view that not only is education asocial discipline, but also the (knowledge/discipline) domain into which the students expectto enter. These social systems define the roles, values and rewards of the system members,and their expectations regarding their participation within them. The action of the researchcan only be understood by an ongoing act of interpretation (Mansell, 1991) and reflection(Schön, 1987). However, this interpretation can never be complete, but rather always includeelements of uncertainty and open-endedness. This demands an iterative interweaving ofmultiple interpretations until a sophisticated understanding is negotiated (Merleau-Ponty,1962; Polkinghorne, 1988).The study adopts a mixed method approach as the most appropriate for the development ofmultiple interpretations (guided by the concept of complimentarity), and appropriates techniquesas required. This reflects the intention to use the results of one strand to elaborate,enhance, and illustrate the results from the other strand. In this study the predominant approachis qualitative but containing smaller quantitative data collection phases. And, sincechange is accepted as the overarching goal, an evaluation strategy that applies a fundamentallyqualitative approach to the collection and analysis of data is seen to have the potentialto provide the information required. As Creswell (2003) indicates, the value of the nestedmodel is that it provides broader perspectives than by using the predominant method inisolation.Consequently, for the reasons noted in this chapter, Action Research is the strategy of choice.The context of this study is an institution of (formal) tertiary education, therefore requiringan acknowledgement of theories of learning (fostering cognitive change through the constructionand organisation of knowledge), the role of the researcher as facilitator/agentof change within the context, and the position of the researcher as a ‘reflective practitioner’/learnerengaged in ‘double loop learning’. Since the remit of this study is to bothinform Software/Requirements Engineering educators in order to change the way educationfor Requirements Engineering is practiced, then the role of ‘agent of change’ fits comfortablyin Kemmis’ approach for resolving conflict between espoused and applied theory, ieEnhancement/Critical-Emancipatory/Critical Science Action Research.Implicit in this alignment is the focus on a non-positivist tradition based on qualitativemethods and an acknowledgement that the results of this work are a contribution to ongoingresearch on this issue, not a final analysis of its ‘truth’ (Lather, 1993). The value of such210

an approach is that the discrepancy between what people say and what people do can beaddressed.In conclusion, accepting the view that RE is a ill-structured ‘wicked’ domain, the interventionsundertaken in this research examine differing modes of inquiry and exploration in order tocreate a learning environment that supports the gaining of competency in RE, and moreclosely aligns with practitioner requirements of novice REs.The next chapter describes the framework developed for the research and describes the contextin which it is undertaken.211

Chapter 5Developing an Action Research modelfor RE educationAs discussed in Chapter 4, Action Research is a methodology that addresses the requirementsand outcomes of both ‘action’ and ‘research’. Within an educational setting, it is valuableas a mechanism to inform practice and enable practice to refine theory. As Zuber-Skerritt(1982) concludes, systematic, controlled Action Research in education can lead to practitionercontributions to the advancement of knowledge in higher education. Action Research is alsovaluable in an environment involving distance between espoused and applied theory. Here aconfrontation between theory and practice act as trigger for the research undertaken, with agoal of enabling critical change of the environment.Action Research therefore is the most appropriate research method in the context of thestudy:• it is a field-based study of a social practice, involving the participants themselves intheir natural setting and exploiting the biases inherent in such an environment. Ineducation Action Research places the researcher in the dual position of producer ofeducation theory/policy and user of that theory through practice• it is interventionist – the change in existing practice implicit in Action Research seesthe researcher act as agent of change while acknowledging that the changes that occurare not solely due to the actions taken• it is flexible enough to adopt any methods that satisfy its raison d’être. With thepotential to provide feedback holistically, the research ideas are allowed to evolve andemerge as part of an ongoing learning and reflection process.In order to address concerns raised against Action Research (eg Orlikowski and Baroudi212

(1991): discussed in Section 4.1.4), the following approaches are included within the researchdesign, based on the discussion of Kock et al (2000):• the study is longitudinal: findings from each cycle are examined in subsequent cycles.These act to expand the project scope, analogous to choosing a wider ‘sample’ instatistical studies• data collection is broadly focussed: this lack of manipulation provides an advantage ingenerating relevant and valid knowledge for the context under investigation, minimisingbias created by artificial action on the part of the participants in response to a focuson predefined variables and links – the data ‘speaks’. This use of multiple sources alsofacilitates an effort to triangulate the data• learning is an explicit outcome: the study is conducted in the researcher’s culture. Theresearcher was involved in both in the planning and the practice of the interventions,as well as their evaluations. Thus, while I may be considered to have a stake in theapproaches taken, and hence exhibit ‘cultural blindness’, the personal involvement ofthe researcher is seen to provide valuable insight as ’cultural insider’ with regard to thecontext of the study. The iteration of cycles is also seen to assist in reducing potentialdistortions in the findings caused by over-involvement.In an environment that attempts to address the disparity between formal education in adiscipline and practitioners’ expectations of it, the characteristics of Action Research provideleverage.5.1 Framework for the studyAction Research within a discipline may be based on a choice from the variety of modelsdescribed in the literature of any of the following: the discipline, the organisational contextor research. In choosing models to apply in this study, an attempt was made to align thecomponents. The framework for this study is developed by integrating these componentmodels, and the evaluation to be undertaken:a model for Action Research Several models for undertaking Action Research in educationexist, all based on Lewin’s concept of a four-phase spiral. The model chosen isadapted from that developed in the work of Borg et al (1993). It is comprehensive inits expansion of Lewin’s phases, making more explicit purpose of each phase and thequestions to be addressed. In addition, it specifically addresses Action Research in aneducational environment213

a model of organisational culture The research design must be placed within a contextualframework that reflects the culture of the organisation in which the study is undertaken.Since this study looks at the learning situation holistically, the model chosenis that developed by Rogers (2002). It acknowledges that effecting cognitive changethrough instructional design is based on the interaction of the participants of a learningenvironmenta model of reflection The researcher’s learning is a fundamental component of the ActionResearch undertaken. The model chosen, by Hatten (1997), examines the area ofcontinuing professional development. It acknowledges that reflection becomes the basisfor decision-making and that change is transformative, but unpredictable.These models are described below, before the integrated framework is discussed.5.1.1 Action Research for EducationFigure 5.1: Action Research in an educational context (based on Borg et al (1993) and Jacksonand Borden (n.d.))Figure 5.1 illustrates the model of Action Research applied to this study, adapted from thework of Borg et al (1993). It identifies the following stages:214

1. definition of the problem – some form of initial reflection is generally the basis foridentifying a problem that relates precisely to issues perceived as interfering with theefficacy of the educational environment or inhibiting the achievement of educationalgoals2. selection of a design – action researchers plan procedures loosely, make changes freelyand acknowledge the inherent bias of the approach. Reflecting on the results of oneiteration of the action phase leads to a redesign of the study for the next. Mixedmethods may be applied to achieve a closer approximation to formal research wherethe research is seeking some generalisable results3. selection of an environment and participants – participants in the study may not beselected for randomness or representation in the population. Rather participants areself-selected through their involvement in the environment in which the study is undertaken4. selection of tools and measures – action researchers are as likely to make opportunisticuse of instruments and measurement procedures available in the environment as theyare to impose reliable and validated measures selected from the appropriate researchliterature at large5. implementation and gathering of data – interventions are informed by theoretical considerationsand require observations to be logged or documented. The strength of ActionResearch is that interventions, potentially beneficial in the context, occur within theresearch, rather than as a time-distant byproduct. There is an implicit acknowledgementthat even casual observation affects the observed, and this effect is taken withinthe scope of the action. Action Research also allows for a measure of flexibility – althoughthe intervention may be ‘fixed’ within an Action Research cycle, it need not be.In this case implementation and data gathering are emergent, based on analysis andinterpretation within the cycle, not just at the end of it6. analysis and interpretation of data – most Action Research focuses on practical significancerather than statistical significance, while researchers often present the subjectiveopinion as well as raw data. Descriptive statistics (eg mean and percentage) and generalobservations that are not quantified or grounded in theory are valid within the studycontext7. interpretation and application of findings – conclusions are based on feedback fromparticipants, experience of the researcher and reflection on the results in the given215

context. Elliott (1982) refers to this stage as one of ‘reconnaissance’. In describing thesituation and then beginning to explain it, the researcher will possibly encounter a newand different perspective on all that has been done8. reflection – having reconnoitred, the researcher may then be in a position to re-assessthe problem. The methodological tools will be refined to suit the exigencies of thecontext so that the study can proceed. The results of this reflection are used to designa new plan of action for subsequent iterations9. reporting findings – the practical (rather than theoretical) significance of the results isof foremost importance. The purpose of reporting is for clarification of the work andits implications for other practitioners rather than in concern for careful replication.In this way Action Research may be viewed as accountable in that one aim is to makethe learning process as well as its results available to other interested practitioners,using accessible terminology. To this extent, at least, Action Research is collaborativeoutside the study situation – peer review of publications is seen as one approach foraddressing accountability.The literature suggests (eg Dick et al (2000); Kember and Kelly (1993)) that cycles may existwithin cycles in an Action Research project. These may address one or more of the phasesdescribed above and provide opportunities for the assumptions underlying the plans to betested in action. The reflection inherent in these also provides chances to correct errors.5.1.2 Tools for educational changeA model for the research design is not sufficient. The context of the study suggests that theAction Research be placed within a conceptual framework that reflects the ‘culture’ of theorganisation in which the study is conducted. The context of this study is an institutionof (formal) tertiary education. This requires an acknowledgement that a strategic goal ofthe organisation is fostering cognitive change through the construction and organisation ofknowledge, and that this change is based on theories of learning.Rogers looks at how teachers take on the role of instructional designers within the classroom.The model for cognitive change described in Rogers (2002) provides a model for addressingthis duality of role: the tools available within the learning environment are fundamental tothe study, and, indeed may act as instruments for the Action Research. These tools arecategorised as:Teaching tools – teachers select and use specific strategies and methods available and con-216

sidered appropriate for the learning taskLearning tools – students have access to, for example, motivation, perceived self-efficacyand predicted success to help with learning as well as the quality and amount of knowledgeavailable. External to the student, means of accessing the knowledge and expressingunderstanding are also learner-focused toolsThinking tools – the cognitive and metacognitive tools of knowledge construction are employedby teacher and learner. An understanding of learner characteristics (eg throughlearning inventories) may influence the choice of teaching tools adopted.Figure 5.2 illustrates this model.Figure 5.2: Model for cognitive change (adapted from Rogers (2002))The aim, then of the learning environment is to engender a ‘transformation’ in the learner.Meyer (1998) suggests transformative conceptions of learning, which involve an emphasis onunderstanding and deriving meaning (Dart, 1998), are of a higher order qualitative natureand involve the learner changing as a person.The discipline under investigation is also a key component of the value of this approach.Both researchers and practitioners have come to the conclusion that disciplines need to bemore involved in the research on how people think and how students learn. Donald (2002,p 299) statesThere is a substantial convergence in the need for deeper understanding of the217

disciplines. The continuing challenge is how to draw on the expertise of scholarsto improve post-secondary education.A practitioner perspective on how the ‘work’ of the discipline is undertaken should also colourthe conceptual framework being developed. In terms of the cognitive change model above,these perspectives should influence both Thinking and Learning Tools. These have beendescribed in previous chapters (see Chapter 2).The value of incorporating this model is that it facilitates a ‘separation of concern’ betweenthe implementation of the intervention and strategies for evaluating its impact.5.1.3 Reflective activityFigure 5.3: A framework for double-loop learning (adapted from Hatten (1997))The learning taking place is not confined to the student participants in the research beingundertaken. The value of Action Research is its ability to focus on the researcher’s learningas a fundamental component of the context under investigation. The framework for doubleloop learning proposed by Hatten (1997) (see Figure 5.3) provides a basis for considerationof the researcher’s participation within each Action Research cycle – double loop learning isa theory of personal change that is oriented towards professional education. In this model,single loop learning is a characteristic of a stable context in which problem solving is patternedon proven solutions and previous experience (Argyris and Schön, 1974). Double loop learning,218

in contrast, is seen as transformative: required by a context where change is inevitable but itsdirection unpredicted. In this environment reflection becomes the basis for decision-makingthat relies on intuitive and tacit knowledge and critical analysis. Informed, directed andcommitted action (thus Praxis) requires reflective activity in order to change the frames ofreference by which action is taken.5.1.4 The integrated frameworkThe dominant characteristics of this study suggest that a conceptual framework for ActionResearch in RE education requires each of these components to be incorporated. The processis a defined Action Research model, the context an environment where the aim is learning(cognitive change). The iterations explicit in the research design require double loop learningon the part of the researcher at least, so that future action is based on varied reflection.Figure 9.2 provides a visual summary of this framework.Figure 5.4: A conceptual framework for Action Research in RE education5.2 The Action Research cyclesThe next sections briefly describe the Action Research cycles undertaken during the study.The complete study was undertaken over the period 2002-2005, and comprised three evolu-219

tionary cycles in 2002, 2003 and 2005.The overarching goals of this study were to enhance learning transfer within formaleducation in a discipline (Requirements Engineering) by attempting to model practitioneraction. The corollary to this was that major intervention from academic staff should beminimised when this knowledge was applied later in the learners’ studies.Evaluating the success of the intervention planned and actioned in the succeeding ActionResearch cycles is based on strategies in several dimensions:• was the model well implemented – this was somewhat determined by examining theteaching style of the lecturer and by modelling the stages of the learning model appliedexplicitly• was the model appropriate◦ in the short term – for this student cohort. Assessment items, learning diagnosticinstruments and student feedback assisted in the evaluation of this item◦ in the longer term – to achieve the overarching goals of the research. Studentperformance in the follow-on unit was observed in several of the cycles.The next sections describe the initial motivation for this research, and provide an overviewof the individual cycles and the phases within them.5.2.1 Initial reflectionThe researcher has been involved with both the practice of RE and the teaching of it, thelatter within different tertiary education contexts. It became apparent early on that howRE was being taught was not how it was practiced, and that students at one institution, inparticular, were not motivated to study an area they considered peripheral to their chosenprofession (in this case Computer Science). The issue of transfer was also raised at this time– students appeared to acquire the content knowledge, and be able to apply skills learnt toproblems within the context of the unit. However, when required to draw on this knowledgein a subsequent situation, students floundered. This was most noticeable during the capstoneproject. Although all students undertaking a software development project were expectedto develop a formal Requirements Specification, very few were able to do so without majorintervention from academic staff.A career move to Murdoch University, at that time in the process of developing a programmein Software Engineering, provided the opportunity to begin addressing some of these concerns.220

With the development of ENG260 Requirements Engineering, as the first of the core SEunits offered in the BE(SE) degree, the decision was made to offer the unit as a seriesof workshops rather than in the traditional lecture/tutorial/laboratory style. The overallcontext to the study is described in Appendix A.Examination and reflection on the learning environment provided in ENG260 was triggeredby several changes that occurred over the period 2000-2001. The student cohort for ENG260was comprised predominantly of SE majors, with most students having completed their firstyear at Murdoch. The implication of this was an enculturation to the environment in theSchool of Engineering, as well as a clear individual motivation to do well in this unit – itwould determine student ability to complete the degree in their chosen discipline.In 2000 University policy changed to enable students with qualifications from a near-by technicalcollege to articulate into programmes in the School of Engineering. These studentscame to a 3-year Engineering Technology degree at the commencement of second year. Consequentlythey enrolled with very different expectations of the learning environment, whichinternal evidence indicated were impacting on their success. Predictions were that these studentswould predominate from 2002/2003, so a pro-active approach to addressing articulationwas appropriate.An additional impetus for reflection and examination of the learning environment, and inparticular the teaching style, in ENG260 was a surprisingly strong bi-polar distribution ofthe final marks for the cohort of 2001. This could be directly attributed to low performancein the final exam, and warranted investigation, primarily of the literature of engineeringeducation.5.2.2 Cycle 1: apprenticeshipThe prime issue that acted as trigger for this cycle was to provide a more ‘authentic’environment for learning, in order to facilitate transfer. Since the discipline is not practicedas lectures and tutorials, it was considered possibly more appropriate to model learning assituated. The issue of students articulating into the unit was also a motivator for reflectionon the unit. Figure 6.1 provides a visual representation of the Cycle 1 of this study.Cycle 1aThe first task was to examine the perceptions identified in the initial reflection. This wasbased on the examination of several bodies of literature:• what do practitioners of RE say about graduate learning221

Figure 5.5: Education for RE – Action Research Cycle 1• what is said about how software development, specifically RE and early design, tasksare carried out• what does the literature of learning say about modelling practitioners?Practitioners indicated an apprenticeship model of on-the-job learning. As an example, anumber of participants in the Macauley and Mylopoulos (1995a) study, asked what additionaltraining in RE would be given to a new graduate, expressed doubts as to whether this wasa job for a new graduate, who would take 12 to 18 months to be able to be ‘effective in thejob’. ‘Expert shadowing’ and ‘novice participation’ were among the techniques described toachieve this. The education literature pointed to situated learning, based on constructivistprinciples as an approach to provide an ‘authentic’ learning environment. This literature isdiscussed in Chapters 2 and 3. The literature of engineering education suggested a mismatchbetween teaching and learning, so that the effectiveness of student learning was compromised.A discussion of this literature in relation to the Murdoch context is provided in AppendixA, and in the description of the student cohort participating in each cycle of this ActionResearch project.Reflection on this literature led to the development of the plan for Cycle 1, including implementationand strategies for its evaluation. In this cycle the intervention design is based on222

the Cognitive Apprenticeship model, discussed in the work of Collins et al (1989) and Brownet al (1989). They suggest such an environment models proficiency and enculturates studentsinto authentic practices through activity and social interaction in a way similarto that evident - and evidently successful - in craft apprenticeships.(Brown et al, 1989, p 37)The background pertinent to this model of learning is provided in Chapter 3.Cycle 1The Requirements Engineering unit (ENG260) was taught applying this model during semester1 2002, with a thematic analysis undertaken at the end of semester.In Chapter 6, two distinct evaluations of this cycle are presented: one analysis focuses onthe level of success of the Apprenticeship model in relation to its implementation; the otherfocuses on the effect of the intervention on the development of the student cohort – aneffect which was examined in terms of short term impact (performance of the students inassessable tasks during the intervention unit) and longer term impact (performance of thestudents in assessable tasks dependant on the learning objectives of the intervention unitduring subsequent units).In summary, the evaluation suggested the Cognitive Apprenticeship model could be appliedreasonably successfully – while students were comfortable with a ‘master’ who, towards theend of semester ‘faded’, they acknowledged that this placed the onus on them to do thelearning, and preferred to be ‘taught’. This expectation was still observed in the followingunit, where the first task was to apply what was learnt in ENG260 to a different context.The results of this cycle indicated the intervention was only achieving some part of the goalshoped for: while students appeared more confident, in subsequent units, in applying theknowledge they had gained, they still expected to be taught: that is, the master/apprenticerelationship was assumed even if no longer appropriate.5.2.3 Cycle 2: PBL for creativityThe problem is now refined: while transfer remains an issue that still must be addressed, thestudent expectation of the learning environment (ie that they must be taught) becomesthe trigger for the next cycle. Student feedback also indicated that they were not happy withthe concept of no one solution for the problems they were solving: their prior education (atleast within Engineering) had been fundamentally scientific (the background to professional223

Figure 5.6: Education for RE – Action Research Cycle 2education for engineering is provided in Chapter 3). This demand for a definitive solutionwas linked to an inability to exploit strategies for dealing with an ill-defined domain.These needed to be made explicit within the learning environment.An additional issue identified related to students’ approach to the learning environment – afocus on learning to apply the tools and pass the unit pointed to a surface approach tolearning that could inhibit student transition to the profession. Figure 5.6 provides a visualrepresentation of the Cycle 2.Cycle 2aAgain, the first task was to take the reflections and subsequent findings of Cycle 1 back tothe literature:• what strategies exist to enhance student-centred learning• can these strategies also be used to change student expectation of problem solving asconverging to one ‘correct’ solution• can strategies be applied to focus on deep learning.The perspective that RE is not about solving given problems, but is a discipline that focusseson problem construction through knowledge discovery informs this cycle. This construction224

is seen to be creative, insight-driven and fundamentally opportunistic and achieved throughnegotiation and interaction. The characteristics noted above pose problems for RequirementsEngineering, summarised as ‘wicked’ (Bubenko, 1995) (see Chapter 2 for a discussion on thenature of RE).Problem-based Learning (PBL) (Woods, 1996a) as an approach with the potential to addressthe issues raised as a result of Cycle 1 as well as embracing the characteristics of RE as adiscipline. PBL emphasises ‘learning to learn’ by placing great responsibility for learning onthe learner (Wilson and Cole, 1996). Its supporters claim PBL results in increased motivationfor learning, better integration of knowledge across disciplines and greater commitmentto continued professional learning (Boud, 1985). Without losing the benefits of a situatedcontext, PBL is seen to offer the flexibility to cater for a variety of learning styles, integratingthe learning of content and skills in a collaborative environment that reflect how learnersmight use them in real life (Oliver and McLoughlin, 1999).These characteristics strongly suggest that PBL has application in the solving of wickedproblems in wicked domains:• learning based around constructivist principles is likely to be more suitable in domainsinvolving ill-structured problems (Spiro et al, 1991). These principles are encapsulatedalmost ideally in problem-based learning (Savery and Duffy, 1995)• appropriate learning in ill-structured domains and/or dealing with ill-structured problemsshould itself be problem-based• problem-based learning best provides an effective environment for future professionalswho need to access knowledge across a range of disciplines (Boud, 1985).Cycle 2The results of Cycle 2a suggested the unit environment could be redeveloped to adhere, asclosely as feasible within the context, to a PBL approach to learning. The plan for Cycle 2 wasdeveloped, to enable PBL for the next offering of the unit. Funding was provided through twosmall grants, and support provided through Murdoch’s TLC (Teaching and Learning Centre).However, the tight time-frame required some compromises: a final exam was necessarily keptas an assessment component, and the ability to revert back to the Apprenticeship model ifthe PBL environment was not ready for the commencement of teaching. If time permitted,the PBL environment would be augmented by creativity-enhancing strategies, to enableinnovative approaches to problem-solving in students, and to redress student perception ofthe discipline.225

Students enrolled in ENG260 during semester 1 2003 (the period February to June) were theparticipants in this cycle of the study. However, as is discussed in Chapter 7, the characteristicsof this cohort were markedly different from that of previous years.As with Cycle 1, evaluating the success of this intervention is based on strategies in severaldimensions: the success of the implementation of the PBL model, and the effect of theintervention on the development of the student cohort, the latter in terms of both performancein assessable tasks and their perceptions of learning in this environment, based on feedbacksought. Triangulation was achieved by comparing the interpretation of each sub-group ofdata, and of quantitative with qualitative results. Chapter 7 presents an analysis of the datacollected in this cycle and the findings drawn from its evaluation.The results of this cycle indicated the PBL environment was also achieving some of thegoals hoped for: students appeared more confident of their skills, and more willing to beinnovative. However, they felt they had ‘lost’ the opportunity to utilise the domain expertiseof the teacher: who, in this environment acted more as facilitator that master. Althoughaccess to other academic staff was available (ie they could engage some consultant time),they felt their interactions with the teacher were less rich. Students also raised issues aboutthe relevance of individual components of the unit (in particular non-problem-related itemssuch as those included in the portfolio), suggesting a need to examine alignment between theelements of a learning environment.The reflection based on the conclusions reached indicated that a hybrid approach mightaddress both teacher and student concerns.5.2.4 Cycle 3: studio learningThe goal of this cycle was to exploit the advantages of both the apprenticeship model andthe PBL environment developed. All the material learnt in ENG260 was conceptually new.Therefore an apprenticeship model that featured ‘fading’ as students gained competencyfailed to address this adequately. The PBL environment, on the other hand, did not allowfor a ‘master’ role: here the teacher acts as facilitator instead. The Studio Learning modeldeveloped in this cycle allowed for both approaches to be integrated. The teacher acts asexpert consultant, and can be ‘engaged’ by the students to provide modelling demonstrationand domain expertise. However, the students are required to direct their own learning: developingthe learning objectives to address each problem presented within the PBL environmentpreviously developed.This cycle required the development and then application of a Studio Learning model, based226

Figure 5.7: Education for RE – Action Research Cycle 3on a review of learning models for adaptive and flexible learning (Cycle 3a). The conceptof constructive alignment, to address issues identified in the previous cycle was also appliedmore critically.The context for the learning environment was modified throughout the School’s engineeringteaching for 2005. This did not greatly affect the learning environment for ENG260, ratherit provided some advantages. These described more fully in Chapter 8.However, the study was broadened to enable observation of the student cohort in a subsequentunit, and to collect data from that unit to evaluate more rigourously the longer termimpact on the student cohort (both student approach to tackling the environment, and theirdemonstration of independent learning and performance in assessable tasks dependant on thelearning objectives of the prior unit (Cycle 3b)). Figure 5.7 provides a visual representationof the components of Cycle 3.Participants for this cycle, undertaken during Semester 1 2005, comprised third year studentswho had undergone an orientation week in applying PBL in a studio environment. Studentsundertaking the follow-on unit (which included some from semester 1) were observed for thesecond sub-phase of this cycle.Primary analysis of the data collected during both semesters was delayed until the end of the227

year. This removed some inclination on the part of the teacher to make heavy modificationsto the follow-on unit (of course this personal involvement bias could only be minimised, notremoved).The evaluations of this cycle, presented in Chapter 8, focuses on the level of success of theStudio Learning model in relation to its implementation, and on the effect of the interventionon the development of the student cohort - an effect which was examined in terms of shortterm impact (performance of the students in assessable tasks during the intervention unit)and longer term impact (performance of the students in assessable tasks dependant on thelearning objectives of the intervention unit during the subsequent unit).Evaluation of the findings suggest that Studio Learning has merit as a model that addressesthe nature of the discipline of RE, and should be continued. However, reflection on this cyclesuggests further research is required on all aspects of aligning student learning with practitionerpractice. However, the small steps (hopefully in the right direction) made through astudy such as this can be useful to both other educators and practitioners.5.3 InstrumentsAs was discussed in Section 4.2 the emergent nature of the design of an Action Researchproject facilitates an opportunistic use of instruments. The mixed methods approach can beexploited with the aim observing and recording the context of the study using a variety ofdevices. Therefore the methods can also be viewed as emergent – planned on the basis ofdiscoveries in the data, and the reflection that follows.The instruments applied in this study are described the sections below. This description isbased on the categories developed by Kember and Kelly (1993) to encompass observationtechniques common to Action Research, namely diagnostic devices; records; and feedback.Together they provide mechanisms to validate the observations through a process of triangulation.Within the framework described in Section 5.1, interventions focus on the Teaching Toolsand are observed through the Learning and Thinking Tools within a delimited learning environment(ie a specific unit in Requirements Engineering). Table 5.1 maps the instrumentsto the Action Research cycles according to Kember and Kelly (1993)’s categories. Thoseinstruments internal to Murdoch University, or developed as part of this study, are describedmore fully in Appendix B.228

Table 5.1: Tooling up for the Action Research project: instruments applied cycles (based on Kember and Kelly (1993))Cycle Diagnostic Devices Records FeedbackLearning Styles Teaching StylesCycle 0 Learning Styles Inventory Keirsey Character Sorter Assessment items1999-2001 Year Survey1999-2001Inventory of Learning StylesCycle 1 Learning Styles Inventory Pedagogical Dimensions Assessment items & results2002 Year Survey2002Inventory of Learning Styles Unit development records Student Survey of UnitPortfolio (mind maps) 2002Cycle 2 Learning Styles Inventory Teaching Style Inventory Assessment items & results2003 Year Survey2003Inventory of Learning Styles Student records1999-2002 Performance Review2003Portfolio (mind maps) Unit development records Student Survey of UnitApproaches to Study Inventory 2003Cycle 3 Learning Styles Inventory Approaches to Teaching Assessment items & results2005 Year Survey2005Inventory of Learning Styles Inventory Student records2005 Performance Review2005Approaches to Study Inventory Unit re-development records2003 Student Survey of UnitReflections on Learning Inventory Curriculum Map2003 2005Interaction Schedules Murdoch Graduate Attributes2004 Design Orientation Feed-backActivity Logs2005 Engineers Australia Graduate Attributes1996;2005229

5.3.1 Diagnostic devicesThese include components of assessment, learning inventories, interaction schedules and toolsfor examining conception. Examples of each of these are available in the study.Tools for diagnosing learningStudents bring a complex assortment of beliefs, past experiences and expectations to a learningsituation, which influence the approach to learning they take. In turn, this approach tolearning affects the quality of their learning outcomes. Their future learning intentions andbehaviours will also reflect this (Prosser and Trigwell, 1999). Additionally, while the achievementof high quality learning is important in all graduates, it has an extra relevance wherethe context of practice is continually changing and professions are continuously developing.There is a need, therefore to identify aspects of the learners’ conceptions of learning (Meyerand Shanahan, 2000) and approaches to it so that appropriate support can be provided.Early work in the UK and Australia (eg Entwistle et al (1974); Biggs (1970) identified motivationand personality as of prime importance, with the later addition of information processing(Craik and Lockhart, 1972) and intention factors (Marton and Saljo, 1976) (roteversus meaningful, deep versus surface) to the web of interrelationships. The approach tolearning that a student takes is very sensitive to the context in which learning is done, with ademonstrated correlation between more advanced conceptions (eg abstraction of meaning andunderstanding of reality (Van Rossum and Schenk, 1984)) and a deep approach to learning.This conception to a large extent determines the student’s expectation of what the learningprocess and teaching entail. Thus, for some students, the ability to take a deep approachappears to be limited by the conception they hold of learning.A variety of self-reporting questionnaires have resulted from this interest in different aspects ofstudent learning behaviour, and from the underlying requirement to demonstrate effectivenessand efficiency in teaching. In general these apply similar formats and psychometric principles(usually based on Likert scales).An informed choice on which instruments to apply may be based on Curry (1983)’s onionmodel (see Figure 5.8). This was originally described as having three layers: a central coremade up of personality-centred models specifically related to how learners prefer to acquireand integrate information (such as the LSI); a stratum of information-processing models toexamine a learners intellectual approach to assimilation of new information, based on theapproaches to studying literature (such as ASI) and an outer layer of instructional-preferencemodels of learning style relating to external factors such as physiological and environmental230

stimuli associated with learning activities. These layers assist in the choice of over 70 modelsreported in the literature (Coffield et al, 2004). It should be noted, however, that instrumentschosen, although associated with one of Curry’s levels, may, in fact address elements of morethan one layer. The next sections describe the major diagnostic devices applied over theduration of the study.Figure 5.8: Onion model of learning styles (Curry, 1983)Personality-centred ModelsThese learning styles serve as an (relatively stable) indicator of how an individual interactswith and responds to the learning environment. The study of learning style involves theinvestigation of individual differences: people perceive and gain knowledge differently, theyform ideas and think differently, and they act differently.The value of investigating personality-centred learning models is two-fold: to help teachersdesign a balanced approach that addresses the learning needs of all of their students byattempting to provide variation in teaching style; and to provide individual students withboth an understanding of the learning implications of their style and strategies to addresstheir strengths and weaknesses.Learning Styles Inventory (LSI)The Learning Style Inventory (Kolb, 1984) is a simple test to measure an individual‘s intrinsiclearning style or predisposition in any given learning situation: a preference for one of fourstages of the learning process. These are grouped by how learners take information in (con-231

crete experience (CE) / abstract conceptualisation (AC)) and how they process information(active experimentation (AE) / reflective observation (RO)). Based on experiential learningtheory, the LSI identifies four basic learning styles. These are defined as:• Accomodator: (concrete, active) What if?; efficient in carrying out plans and like gettinginvolved in new experiences; often start with what they see and feel then plunge inand seek hidden possibilities; learn by trial an error and self discovery; like applyingcourse material in new situations to solve real problems; the instructor should poseopen-ended questions and then get out of the way, maximising opportunities for thestudents to discover things for themselves. Problem-based learning is considered anideal pedagogical strategy for these students (Felder and Brent, 2005)• Converger: (abstract, active) How?; good at problem solving, decision making, and thepractical application of ideas; start with an idea and try it out, they like to find outhow things work; learn by testing theories; respond to having opportunities to workactively on well-defined tasks and to learn by trial-and-error in an environment thatallows them to fail safely; the instructor should function as a coach, providing guidedpractice and feedback in the methods being taught• Diverger: (concrete, reflective) Why or why not?; good imaginative ability and awarenessof meaning and values; study life as it is and reflect on it to seek meaning; learn bybeing involved and need to listen and share with others, respond well to explanationsof how course material relates to their experience, interests, and future careers; theinstructor should function as a motivator• Assimilator: (abstract, reflective) What?; good at inductive reasoning and creatingtheoretical models; come up with ideas and then reflect on them; like to know what theexperts think; respond to information presented in an organised, logical fashion andbenefit if they are given time for reflection; the instructor should function as an expert.This instrument has been shown to be both reliable (ie consistent results are obtained inrepeated assessments) and valid (ie the instrument measures what it is intended to measure)within an engineering context (see Felder and Brent (2005) for a discussion of the relevantstudies).Index of Learning Styles (ILS)Soloman and Felder (1999)’s Index of Learning Styles is a self-scoring instrument, developedfor use especially in the engineering and science disciplines, that assesses preferences on fourdifferent dimensions through the answer to these four questions:232

1. What type of information does the student preferentially perceive: sensory (sights,sounds, physical sensations) or intuitive (memories, thoughts, insights)?2. What type of sensory information is most effectively perceived: visual (pictures, diagrams,flow charts, demonstrations) or verbal (written and spoken explanations)?3. How does the student prefer to process information: actively (through engagement inphysical activity or discussion) or reflectively (through introspection)?4. How does the student characteristically progress toward understanding: sequentially(in a logical progression of incremental steps) or globally (in large ‘big picture’ jumps)?The dimensions (described below) are assessed as a continuum where a learner may be locatedat any point on the axis between the two extremes:• Active/Reflective This dimension refers to processing of information. Active learnerslearn better by doing something active - discussing the material, explaining it tosomeone, or using it to solve problems. Reflective learners learn better by thinkingabout the material before trying to explain or use it• Sensing/Intuitive This dimension refers to ways of receiving information – sensinglearners like to memorise facts and solve problems using well-established methods. Theytend to be concrete, practical, methodical, and oriented toward facts and hands-onprocedures. Intuitive learners prefer discovering relationships and using innovativeproblem-solving approaches. They tend to work fast and be innovative and can oftenhandle abstract and mathematical concepts well• Visual/Verbal This dimension refers to ways of perceiving sensory information. Visuallearners retain more from things they see - pictures, diagrams, flow charts, etc whileverbal learners get more out of words (written and spoken explanations)• Sequential/Global This dimension refers to progress toward understanding – sequentiallearners gain understanding in linear, logical steps, and are able to function withonly partial understanding of material they have been taught. Global learners thinkin a systems-oriented manner, and may have trouble applying new material until theyfully understand it and see how it relates to material they already know about andunderstand. They tend to learn almost random pieces of material, then suddenly ‘getit’. Then, however, their holistic perspective enables them to see innovative solutionsto problems that sequential learners might take much longer to reach, if they get thereat all.233

Myers-Briggs Type Indicator (MBTI)/Keirsey Temperament SorterDeveloped by Myers (1998), this instrument is based on the work of Jung and his theory ofpsychological type (Lawrence, 1993). He theorised that what appears to be random variationin human behaviour is actually quite orderly, logical and consistent, and is a result of fewbasic differences in mental functions and attitudes. Strictly speaking, the MBTI assessespersonality types, but MBTI profiles are known to have strong learning style implications(Felder and Brent, 2005).People are classified according to their preferences on four scales derived from Jung’s work.According to Myers, although all preference poles are used at least some of the time byevery person, individuals have an innate disposition towards one pole of each dichotomy.The figures reflect the percentages of total population in Western culture who hold thispreference:• (E/I)Extroverts (75%) try things out, focus on the outer world of people, while Introvertsthink things through, focus on the inner world of ideas• (S/N)Sensors (75%) are practical, detail-oriented, and focus on facts and procedureswhile iNtuitors are imaginative, concept-oriented, and focus on meanings and possibilities.Jung called this ‘unconscious perceiving’• (T/F)Thinkers (50%) are skeptical, and tend to make decisions based on logic andrules. Feelers are appreciative and tend to make decisions based on personal andhumanistic considerations that are value-oriented• (J/P)Judgers (50%) set and follow agendas, seek closure even with incomplete data.Perceivers adapt to changing circumstances, postpone reaching closure to obtain moredata.In summary, this instrument is based on the premise that personality can be divided intofour orthogonal areas, with a preference for one or other pole that defines the scale, enablinga total of sixteen personality types. Other researchers (eg Keirsey and Bates (1984))have correlated these personalities into temperaments, which approximately parallel the four‘Humours’ identified by Hippoctates.The work of Myers-Briggs has been criticised in recent years: they ignore the issue of genius/madness;they used a linear four factor model to characterise ‘invariant’ patterns ofbehaviour; they often describe what people have in mind, rather than observable behaviour.Keirsey and Bates (1984) is credited with legitimising personality-centred models throughthe Keirsey Temperament Sorter. Keirsey based his adaptation of the MBTI on observed234

long term behaviourial patterns and used a systems field theory model to characterise thesepatterns.In the Keirsey model, personality is divided into four temperaments, each with four subcategories.The following summaries are taken from Keirsey and Bates (1984):• Artisans – (SP) (35-40%) being concrete in communicating and utilitarian in implementinggoals, can become highly skilled in tactical variation. Thus their most practicedand developed intelligent operations are usually promoting and operating (SPTexpediting), or displaying and composing (SPF improvising). Subtypes include Composer,Performer, Crafter, Promoter. Artisans prefer jobs where they can troubleshoot,respond to crises and negotiate. They also enjoy identifying and responding to opportunities• Guardians – (SJs) (40-45%) being concrete in communicating and co operative in implementinggoals, can become highly skilled in logistics. Thus their most practiced anddeveloped intelligent operations are often supervising and inspecting (SJT administering),or supplying and protecting (SJF conserving) Subtypes include Inspector, Supervisor,Protector, Provider. Guardians prefer jobs that demand responsibility. Theyenjoy improving the efficiency of processes and setting up standardised procedures• Idealists – (NFs) (8-10%) being abstract in communicating and co-operative in implementinggoals, can become highly skilled in diplomatic integration. Thus their mostpracticed and developed intelligent operations are usually teaching and counselling(NFJ mentoring), or conferring and tutoring (NFP advocating). Subtypes includeHealer, Champion, Counsellor, Teacher. Idealists enjoy jobs that allow them to supportand encourage others. Their tendency to be enthusiastic can energise and improvethe morale of others• Rationals – (NTs) (5-7%) being abstract in communicating and utilitarian in implementinggoals, can become highly skilled in strategic analysis. Thus their most practicedand developed intelligent operations tend to be marshalling and planning (NTJorganising), or inventing and configuring (NTP engineering). Subtypes include Engineer,Inventor, Fieldmarshal and Mastermind. Rationals enjoy jobs that demand ahigh level of expertise and high standards of competence. They enjoy designing andunderstanding systems.The problem with both the Keirsey and Myers characterisation of personality for a particularindividual is both the complexity of the individual and the myriad of circumstances that235

affects the individual: it is hard to apply general descriptions to some specific examples.However, If undertaken ‘naively’ (ie not attempting to manipulate the results), personalitymodels are useful in providing some indications of behaviour that can be explored further.Information Processing ModelsWhile personality-centred models are considered a relatively stable indicator of how an individualinteracts with and responds to the learning environment, information-processingmodels capture students’ responses and adaptations to learning contexts. As well as beingdetermined, to some extent, by past experience, the choice of approach is also influenced bystudent perception of the nature of the unit they are studying. The work of Entwistle andRamsden (1983) confirms this, while that of Ramsden (1988) adds that the manifestation ofdeep and surface learning is also dependent on the discipline.Students may be inclined to approach their learning in one of several ways. Despite a widevariety of methodologies and descriptive terms, a clear consensus has emerged that studentsapproach learning with either a ‘surface’ or a ‘deep’ orientation, originally identified by Martonand Saljo (1976). These orientations are complemented by motives and strategies thatare dependant on a specific learning context.Those with a surface orientation tend to take an approach characterised as instrumental,reproductive and minimalist, relying on rote memorisation and mechanical formula substitutionand making little or no effort to understand the material being taught. Those with adeep orientation tend to adopt a meaningful approach, characterised as striving for meaningand understanding, probing and questioning and exploring the limits of applicability of newmaterial.There is a general consensus that a deep approach to learning is desirable in higher education.There is also research evidence to support an association between deep approaches and enhancedlearning outcomes: students using a deep approach appear more able to demonstratetheir understanding, develop their conceptions of material and report greater developmentof generic skills (see Wilson and Fowler (2005) for a discussion of the relevant studies).However, it is also shown that students of either orientation prefer teaching methods whichencourage those approaches to learning (Entwistle and Tait, 1990). The value of investigatinginformation-processing learning models is therefore to provide justification for educatorsseeking to influence students towards deeper approaches to learning.Approaches to Study Inventory (ASI)The Approaches to Study Inventory by Entwistle and Ramsden (1983), was developed toaddress a range of concepts, including motivation and study methods. It describes four236

study orientations, and is one of the most widely used questionnaires on student learning inhigher education. In its most commonly used version, the ASI contains 64 items in 16 scales,however, a shortened version (32-item) of this instrument has been confirmed by Richardson(1990)’s work to possess adequate internal consistency and test – retest reliability. This latterfocuses on two of the four orientations (see Table 5.2):• a reproducing orientation indicates the use of a surface approach, with an emphasis onrote memorising, and a narrow syllabus-bound attitude, associated with both extrinsicmotivation and fear of failure• a meaning orientation indicated an intention to understand for oneself – comprehensionlearning, relating ideas, and using evidence being all motivated by interest in the ideaspresented. This orientation is characterised by a holistic style and intrinsic motivation.Table 5.2: ASI Scales for Reproduction and Meaning Orientation (Richardson, 1990)ScaleMeaningMeaning OrientationDeep Approachactive questioning in learningInterrelating ideasrelating to other parts of the courseUse of evidenceComprehension learningrelating evidence to conclusionsreadiness to map out subject area andthink divergentlyReproduction OrientationSurface approachSyllabus-boundnessImprovidenceFear of failurepre occupation with memorisationrelying on staff to define learning tasksover-cautious reliance on detailspessimism and anxiety about academicoutcomesReflections on Learning Inventory (RoLI)It has been suggested that a limitation of instruments such as the ASI is a focus on relatingconceptions of learning to other broad aspects of learning, rather than to the specific learningactivities in which students engage: it provides little information about relationships betweenconceptions of learning and the specific learning strategies that students use in a particularunit of study. The Reflections on Learning Inventory (Meyer and Boulton-Lewis, 1997) is aninstrument designed to capture this variation in student engagement of learning - typically in agiven disciplinary context or topic-specific ‘episode’ – and includes statements about students’beliefs about the nature of learning, how they know that they have learned something, andhow they feel when they are learning. The RoLI can also be used as a basis for developing thecritical first stage of metalearning capacity in students by representing responses as a graphic237

personal learning ‘profile’. Although still under development, research has established thatthe RoLI is able to capture variation in conceptions of learning and that these conceptionsdo not exist in isolation from other aspects of the learning process (Meyer, 1999).Empirical evidence indicates a large measure of conceptual overlap between the scales utilisedin the differing instruments. The LSI, ILS and ASI, therefore are adopted as appropriateinstruments to describe learner orientation to study in the context of this research, witha very small application of RoLI to explore its value in comparison to other instruments.Access to RoLI itself was refused by it’s developers for this study.Instructional-preference ModelsInstructional-preference models of learning style relate to external factors such as physiologicaland environmental stimuli associated with learning activities. Although no formalinstrument was applied to this layer, components of other instruments were seen to addresssome of the aspects of affective and habitat preferences noted below.Dunn and Dunn (1978) identify five dimensions that mark various preferences:1. environmental preferences regarding sound, light, temperature, and class design2. emotional preferences addressing motivation, persistence, responsibility and structure3. sociological preferences for private, pair, peer, team, adult or varied learning relations4. psychological preference related to perception, intake, time, and mobility5. psychological preferences based on analytic mode, hemisphericity, and action.It can be seen that many sociological and psychological elements of these categories overlapwith other layers of Curry’s onion and the instruments used.Although the term environment has been used in higher education to cover different levelsof description: from the institutional level policy, administration and regulations to aspectswhich are most immediately experienced by students, Entwistle (2003) suggests the termhabitat may offer a more appropriate metaphor to include all the members of an ecosystemand the inter-relationships between them. To this are added the notions of niche, indicatingthe fit of the species to some part of the ecosystem; and umwelt, which indicates the habitatas experienced by the animal itself. Much of the work on phenomenography can be seen asan attempt to enter the world of the student and interpret the meaning of studying in moresubjective terms.The nature of the unit as perceived by students is an important determinant of the choice ofapproach, as shown clearly in the large-scale Entwistle and Ramsden (1983) study. In terms238

of disciplinary contexts, Ramsden (1988) theorised that deep and surface approaches wouldhave very different manifestations in different academic specialisations, in agreement withthe context-dependent nature of approaches as originally formulated by Marton and Saljo(1976).Other diagnosing instrumentsAdditional devices were also applied as available or appropriate. In general, most elements ofstudent assessment were available for scrutiny. These usually comprised individual and groupassignments and a formal exam. Other assessment items (not always available) included portfoliosand review documents. Student activity logs were introduced as part of the assessmentfor the follow on unit observed during Cycle 3 of the study. The self-reporting of activity leftthe instrument open to abuse (ie student logging more hours than actually worked of forgettingto log hours), but did provide some measure of categorisation of transactions/activitieswithin the class time. Mind maps (Buzan and Buzan, 1994) were also artefacts developed bystudents to demonstrate understanding of the concepts dealt with. These provide a visualrepresentation of the knowledge of main concepts and major sub-topics within a particulardomain (Hoover and Rabideau, 1995).An additional tool, only used sparsely due to its resource hunger, is the interaction schedule.An independent observer was engaged over several sessions to log the nature of transactionsundertaken within the class. As each class lasted four hours, and occurred twice a week, thishad the potential to generate a vast quantity of data.5.3.2 RecordsThe most important records used in this research are those provided by the students themselves.These take the form of items submitted for assessment, and include, in the variousunits examined, reflective components to assignments, activity logs and journal entries. Theseare included elsewhere in this section, based on the aim they fulfilled (ie as a diagnostic deviceor feedback mechanism).From the unit perspective, formal documentation in the form of unit outlines and syllabi areavailable, as are minutes and note of staff meetings during the unit development and thenadaptation process. Student information is available to academics through access to an online system. These provide data in the form of class lists, student enrolment background,programme of study and academic record.As the processes of professional re-accreditation would occur during this study, documentsrelating to graduate outcomes and attributes (both University- and profession-based) were239

also examined. Reports from this activity and the School Review undertaken in 2005 werealso included in the documents scrutinised.5.3.3 Student feedbackFeedback from students provides opportunities for participants to raise issues and concerns.Formal feedback is based on University-supported instruments through the Institutional Researchand Evaluation Service of the Teaching and Learning Centre. The Student Survey ofUnit (SSU) is the primary formal questionnaire used in this research (Murdoch, 2004b). Adescription is provided in Appendix B.In addition, the School of Engineering instituted, early in the development of it’s programmes,that every year group of students is surveyed for each unit in which they are enrolled, twicea semester. This School-based, open-question survey, is administered by another member ofstaff (ie one not teaching any unit for that student cohort). The comments resulting fromthis source are collated (by a designated Year Co-ordinator), presented and actioned at astaff meeting. Feedback from decisions made there are returned to students, again by theYear Co-ordinator, usually within two weeks of the survey administration. A sample fromthis survey is included in Appendix B.Formal feedback was also available in the form of comments and issues raised in the assessmentelements. The prime component is the Performance Review document, discussed elsewhere.Others, less formal, include Management Reports in the follow-on unit, emails and generaldiscussions in class.5.3.4 Teaching styleStudent engagement with the learning environment and potential for success in it is foundedon a net of influences made up, not just of learner style and features of the learning environment,but also of teacher characteristics and teaching methods. As a final note oninstruments, therefore a teacher-focused perspective is provided.Philosophers and educational theorists postulate that individuals will generally be guidedin their thinking about most things, including the processes of teaching and learning, bythe principles of the dominant experiences from which their knowledge has thus far beenconstructed. Smyth (2003) provides the following examples:• if a teacher’s description of a preferred teaching strategy is didactic and rigid thenit is usually premised on the teacher controlling the learning environment. This is240

characterised by transmission of concepts and the teacher’s knowledge to the students,so that the teacher retains a position of power over the students• alternatively, a teacher’s description of a preferred teaching strategy that articulateshow interaction with students might support learning is usually premised on understandingand collaboration, adaptation and greater flexibility within the learning environment.The teacher relies on a deep understanding of the discipline as part ofthe context for learning and is motivated by the desire for increasing the social goodthrough fostering student achievement.A Teaching Style Inventory (TSI) developed at University of Toronto and adapted fromDunn and Dunn (1993) was applied as a mechanism to gain insight on the orientation of theresearcher/teacher. The instrument measures teaching style across six dimensions: instructionalplanning; teaching methods; teaching environment; evaluation techniques; teachingcharacteristics and educational philosophy. The result is a continuum spanning Traditionalto Highly Individualised, with Transitional occupying a central position.Another teaching tool for assessing teaching style has been developed by Prosser and Trigwell(1999). The Approaches to Teaching Inventory (ATI) is composed of sixteen items, eight ofwhich describe an approach intended to change students’ conceptions (four items refer to themotive of the approach, and four on the strategy). The other eight items look at informationtransmission and teacher focus (four items referring to the intentions to transmit and fourto the use of teacher-focus to achieve this). Responses to the ATI are relational: specific tothe context in which they are collected. The authors have not published norms for this tool,rather they see it as a mechanism to explore associations for example, between approach toteaching and student approach to learning. In the context of this research it is a mechanismthat provides information on the teacher within the context of the learning environment.An additional tool was used for diagnosis purposes, through characteristics of the learningenvironment. This was a mechanism to gain insight on teaching style, given that the environmentwas developed from scratch by the researcher (and therefore should be based to alarge extent on teacher characteristics). As an added justification, the model was developedfor the evaluation of technology-assisted learning, in which the learning environment understudy is placed.Reeves (1997b) has described 14 dimensions for the evaluation of computer-based education.He suggests they have the potential for understanding, describing and evaluating units, witheach dimension viewed as a continuum (see Figure 5.9 for some of the dimensions).In brief these comprise:241

Figure 5.9: Pedagogical dimensions of a learning environment (partial) (Reeves, 1997b)epistemology theories about the nature of knowledge ranging from• objectivist - content of the learning environment is comprehensive and accurate,based on advice from experts in the field• constructivist – content reflects spectrum of views in domain. Multiple perspectives/optionsfor constructing knowledge. Need scaffolding/coaching, but not directionfrom teacherphilosophy• instructivist - importance of goals and objectives drawn from the domain. Directinstruction to progress through sequence, based on progression from lower tohigher learning. ICT-based tutorials, drill-and-practice programs• constructivist - primacy of learner intentions, experience and metacognitive strategies.Requires a rich environment that can be tailored to individual needs. Requireinteractive learning environmentspsychology• behavioural - shaping desirable behaviours via stimuli, feedback, reinforcement etc242

• cognitivist – emphasis on mental models and the connections between them. Thetype of knowledge to be constructed should drive the learning strategy employed.Support for deductive and inductive learning strategiesgoal orientation - from strict protocols (eg to handle emergencies) to unfocussed (eg appreciatingmodern art)experiential value - removed from real world to apprenticeship models are skills, knowledgeattitudes learnt in a context of use or via abstract lectures and books, with the studenthaving to make the connections. PBL, anchored instruction etcteacher role - sage on stage (provider of knowledge) to guide on the sideflexibility - from allowing no local adaptation to being so open and unstructured to provideno support and guidance for valid implementationvalue of errors – learners can only make correct responses - to - allowing negative outcomesand feedback on whymotivation• intrinsic – students explore in search of new knowledge. Integral to the learningenvironment - interesting, complex problems. Linked to learner experiences• extrinsic - outside the learning environmentaccommodation of individual differences – cannot assume homogenous learners in termsof aptitude, prerequisite knowledge, motivation, experience, learning style, hand-eyeco ordination etc. Require scaffolding, cognitive bootstrapping and other types ofmetacognitive supportlearner control – learners make decisions on what sections to study/what paths to follow.Individualising instructionuser activity• mathemagenic – give learners access to various representations of content to• generative – engage learners in a process of creating, elaborating or representingknowledge (aligned more closely with constructivism) – meaningful assessment(integrated with activity)co operative learning – support for group process243

cultural sensitivity – allow for cultural diversity - accommodate diverse ethnic and culturalbackgrounds among learnersHow the instruments described above are realised is elaborated in Chapters 6 to 8, whichdiscuss the Action Research cycles in detail. The next section in this chapter provides anintroduction to the context of the study.5.4 The Murdoch contextThis study is undertaken within the context of the Bachelor of Engineering (Software Engineering)(BE(SE)) offered by Murdoch University’s School of Engineering. The teachingobjectives of this programme are focused on producing engineers with a special skill in software.We expect graduates from our BE(SE) to find career opportunities in both professionalengineering industries that have a strong interest in software as well as the full range of ITdisciplines where the design and implementation of quality software is considered a priority.Pursuing these objectives has meant a gradual shift from more traditional engineering learning,in particular how the learning is undertaken, as we address characteristics specific toSE.This environment also determines many of the constraints under which the study was conducted:• major changes to unit structure and assessment were required to adhere to appropriateUniversity policy• over the duration of the study, School policy changed regarding programme structureand unit offerings• modified regulations regarding student intake also impacted on the study• other research within the School, addressing student learning styles, informed some ofthe decisions make during this study.These are described more fully in the appropriate chapters describing the Action Researchcycle in which the impact was critical. A detailed description of the context for this study isprovided as background information in Appendix A.244

5.5 ConclusionA framework has been developed for the research project described in this document. Thisframework integratesa model for Action Research that specifically addresses Action Research in an educationalenvironmenta model of organisational culture that acknowledges that effecting cognitive change isbased on the interaction of all the participants of a learning environmenta model of reflection that examines the area of continuing professional developmentin an attempt to address the dominant characteristics of research in education within an ITdiscipline undertaken in an engineering culture.Figure 5.10: Education for RE – Action Research CyclesThe context of this study is the a unit in Requirements Engineering within the undergraduateSoftware Engineering programme (BE(SE)) offered by the School of Engineering at MurdochUniversity in Western Australia. Over the duration of this longitudinal study, many changestook place within this context. Some of these were the results of specific phases of this245

esearch, others triggered by changing School and University policy. These could be accommodatedwithin the research framework, due to the emergent nature of the research design,and the opportunistic use of tools to observe the interventions and changes occurring.Figure 5.10 summarises the overall Action Research cycles of this project. Cycle 1 investigatedan Apprenticeship model for authentic learning of RE, as a mechanism for assisting transfer;Cycle 2 addressed issues raised in Cycle 1 by examining models that focussed on studentcentredapproaches to learning in ‘wicked’ domains, where problems are generally (if notinvariably) ill-structured and required creativity to tackle them. Cycle 3 developed a hybridmodel that focussed on adaptable and flexible learning.The data collected through the use of instruments described in Section 5.3 were subject tothematic analysis and evaluation. Triangulation was achieved by comparing the interpretationof each sub-group of data, and of quantitative with qualitative results. In general,this evaluation addressed several aspects of the study: the success of the implementation ofthe intervention; the appropriateness of the model for addressing the issues raised and theimpact on students learning in the short term (their level of success and perception of theenvironment) and longer term (student performance in tasks dependent on learning (in Cycle1) in the unit under investigation and (in Cycle 3) demonstration of independent learning inthe follow-on unit).The next three chapters describe the Action Research project undertaken. As noted inChapter 1, each of three chapters discusses a discrete cycle in the Action Research project.This includes the following:• a section describing the Murdoch context, and, in particular, changes to it which impacton the cycle• the planning required to implement the intervention• the implementation and the data collected• analysis and interpretation• reflection and then reporting of the intervention.In keeping with the narrative nature of the journey metaphor employed (also discussed inChapter 1), the chronological aspects of each cycle are important, as is the thematic analysisof the ‘stories’ collected from the student participants.246

Chapter 6Apprenticing the RE student 2002Even in domains that rest on elaborate conceptual and factual underpinnings,students must learn the practice or art of solving problems and carrying out tasks.And to achieve expert practice, some version of apprenticeship remains the methodof choice.(Collins et al, 1991, p 38)Figure 6.1: Education for RE – Action Research Cycle 1The initial reflection that acted as trigger to this study of education for RE was based ona perception that traditional teaching of RE was not effective. The characteristics of the247

discipline, at this point based primarily on personal experience in industry, suggested alternateapproaches should be investigated. The prime issue that acted as trigger for this cyclewas to provide a more ‘authentic’ environment for learning, in order to facilitate transfer.Consideration of issues facing students articulating into the unit was also a motivator forexamination and reflection.Figure 6.1 provides a visual representation of Cycle 1 of this study, in the context of thethesis as a whole.The first task (Cycle 1a) was to examine the perceptions identifies in the initial reflection.This was based on the examination of several bodies of literature, focussing on: what practitionersof RE say about graduate learning; how RE and early design tasks are said to becarried out, and what the literatures of learning say about modelling practitioners. Thisliterature has been discussed in Chapters 2 and 3.Cycle 1 was planned and developed after consideration of this literature. In this cycle theintervention is based on the Cognitive Apprenticeship model, described in the work of Collinset al (1989) and Brown et al (1989). As was discussed in Chapter 3, they suggest such anenvironment models proficiency and enculturates studentsinto authentic practices through activity and social interaction in a way similarto that evident - and evidently successful - in craft apprenticeships.(Brown et al, 1989, p 37)As noted in Chapter 5, the value of this model is its alignment with practitioners perceptionof learning in the discipline: they indicated an apprenticeship model of on-the-job learningfor novice Requirements Engineers. This literature has been discussed in Chapter 2.The Requirements Engineering unit (ENG260) was taught applying this model, during semester1 2002. This chapter describes the intervention in its context, the data that was collectedand discusses possible interpretations of these.Two distinct evaluations of this cycle are presented: implementation of the model and itssuccess; and the effect of the intervention on the student cohort, both short term and longerterm impact.In summary, the interpretation of the intervention suggested a model based on CognitiveApprenticeship could be applied reasonably successfully. However, while some students werecomfortable with a ‘master’ who, towards the end of semester ‘faded’, most acknowledgedthat this placed the onus on them to do the learning, and preferred to be ‘taught’. Thisexpectation was still observed in the following unit, where the first task was to apply what248

was learnt in ENG260 to a different context: while students appeared confident in applyingthe knowledge they had gained, the master/apprentice relationship was assumed – they stillwanted to be taught.The next section describes changes to the Murdoch context which impact on this cycle. Thechapter then continues by describing, evaluating and reflecting on the elements of this cycle.6.1 Context for Cycle 1This section aims to provide the context for both the description on the intervention plannedand implemented, and the evaluations that follow. In order to fully appreciate these, theplace of the unit under study within the ‘organisation’ (ie the curriculum for the BE(SE)),the characteristics of the students who undertook it during 2002 and the learning and teachingenvironment need to be described.6.1.1 Curriculum componentsIn 2002 the curriculum model that had provided the framework for all units developed inthe School of Engineering was changed. That model is described in Appendix A. Themost significant change occurred in the final year. The capstones projects (ie group andindividual) were merged and effectively outsourced – students would undertake an honourslevelinternship with appropriate local industry, under the supervision of both an academicand industry mentor. This restructuring of the curriculum provided the opportunity toexamine more closely the learning environment in the initial SE units. The internship,while anticipated to be of real benefit for students transitioning to the workplace, placedthe academic in a ‘supervisor’ relationship with individual students: the move from formalteaching to (potentially ad hoc) learning of critical discipline knowledge was considered anissue that needed to be examined.6.1.2 Characteristics of teaching and learningTeacher characteristicsOther research within the School examined learning styles, and the match to teaching style.This collaborative work acted as a prompt for this teacher to examine more closely her teachingcharacteristics. Student performance in the previous offering of ENG260 also suggestedsuch an examination was warranted. Initially the MBTI as adapted in the Keirsey Character249

Sorter (Keirsey and Bates, 1984) was applied. As was noted in Chapter 5, it is very easy tomanipulate the instrument in order to achieve desired results, therefore it should be statedthat the result described below is based on a initial, naive completion of the survey (eg priorto investigating possible results).Based on the four Temperaments described in Chapter 5, this researcher was categorised asa Teacher within the temperament Idealist (ENFJ).Adapted from Keirsey and Bates (1984), the following is a summary of this personality type:Idealists are abstract in communicating and co operative in implementing goals,leading to intelligent operations as teaching and counselling (NFJ) or conferringand tutoring (NFP). As well as being abstract in thought and speech, and co operative,the Teacher is directive and extroverted. Teachers expect the best fromthose around them, usually expressed as enthusiastic encouragement, motivatingaction in others. They take for granted that their expectations will be met, andimplicit commands obeyed. Found in less than 5% of the population, Teacherslike to have things settled and arranged, but are at home in complex situationsthat require juggling of much data with little pre-planning, and act well as leadersof groups. Teachers value harmonious relations above all else, but with an educational(rather than social) focus: interested primarily in the personal growth anddevelopment of others.This fits in with the profiles exposed through the inventories applied in the collaborativelearning styles research. Soloman & Felder’s ILS scores for the researcher herself show as verystrongly reflective (or very slightly active, given a score of 9% active), moderately intuitive(27% sensing), strongly verbal (18% visual) and slightly more global than sequential (36%sequential). Kolb’s LSI identifies her as an Assimilator: an abstract, reflective What? person,good at inductive reasoning and creating theoretical models; who likes to know what theexperts think and responds to information presented in an organised, logical fashion, withthe instructor considered as expert.This profiling has implications for the learning environment in which the teacher participates.In terms of the interventions planned, the profile highlights aspects that need to be monitored.Key concepts from this identity are abstract; co operative; directive (implicit commands);liking for things to be settled and arranged. Of particular note, the directive quality andliking for settlement, may imply a less than flexible approach where flexibility and studentcentringare overarching concerns within this research.250

Learner characteristicsThe work of Lumsdaine and Lumsdaine (1995) suggests that between 20% and 40% of studentintake to engineering is lost through not catering for students with strengths in communicationsand team work or creative problem solving, synthesis and design. In addition,traditional engineering education does little to provide students with the systemic perspectiveon individual subjects (a global perspective) they need to function effectively, and the oneswho take too long to get it by themselves are at risk academically (Felder and Brent, 2005).Practitioners studies indicate that the Lumsdaines’ ‘lost’ skills are those considered necessaryfor competent professional practice. This practitioner perspective is discussed in Chapter2. In summary, industry requires personable professionals who can function in an adaptive/creative,collaborative environment. As Turley (1991) notes, education should supportthe development of differential skills (namely interpersonal skills and personal attributes)through the creation of learning situations which stress these.Figure 6.2: Kolb Learning Style Inventory 1st year Engineering students 2001The ENG260 cohort for 2002 is within the group captured in Figure 6.2. The majority ofstudents fall into the Converger/Assimilator profiles expected of engineers (Kolb, 1984) 1 .The learning style of the Assimilator students is catered for in traditional Engineering teach-1 discussed in Appendix A; see specifically Table A.1 for the Murdoch breakdown. The results from TableA.2 in the same appendix highlight the mismatch between student learning styles and traditional engineeringteaching: showing a clear preference for active, sensory learning amongst students but strong preferences forreflective learning by staff251

ing: they respond to information presented in an organised logical fashion and expect theteacher to function as an expert. Convergers respond to having opportunities to work activelyon well-defined tasks and to learn by trial-and-error, also catered for in engineering teaching.This traditional teaching is seen by some researchers (eg Holt and Solomon (1996)) to excludeDivergent and Accommodators from effective learning, and to limit the opportunitiesof all learners to develop the skills required for proficiency in two key areas of engineering:design and invention (requiring a divergent approach), and business management (requiringaccommodative skills).In terms of the intervention implemented in this cycle, the learning styles profile has somemajor implications: the majority of students should be comfortable within the Apprenticeshipmodel. The modelling/coaching role taken by the teacher can be considered an example ofexpertise for the Assimilators, while the Converger students respond to having opportunitiesto work actively on well-defined tasks and to learn by trial-and-error in an environmentthat allows them to fail safely. They expect the instructor should function as a coach,providing guided practice and feedback in the methods being taught. On the other hand,the Accomodator students may feel constrained by the apprenticeship environment, and bemore at ease once the Fading occurs. It would seem, therefore that the learning model alignswell with the learning characteristics of this particular cohort. Its non-traditional nature,however, may challenge some of their expectations of how they should be taught while at thesame time being inclusive of the less typical engineering students.6.2 Cycle 1 – Apprenticing in REThe environment established for ENG260 in 2002 was triggered by the desire to provide amore ‘authentic’ environment for learning, in order to facilitate transfer of the disciplinespecificknowledge acquired in this unit to other software development situations, initiallywithin the formal education context but also after completion. Since the discipline is notpracticed as lectures and tutorials, it was considered more appropriate to model learning assituated. Although this approach was the basis of the workshop format of the unit sinceits inception, one issues raised in the initial reflection section of Chapter 5 (section 5.2.1)suggested a more rigourous model of situated learning had the potential to address these.6.2.1 The learning environmentAs has been noted, Requirements Engineering (ENG260) is the first of the core SE units,offered in the first semester of the second year of study, after a common first year, and is252

taught in workshop mode (two sessions of 2 hours duration twice per week).Figure 6.3: Student mind map for topic examining teamsFigure 6.4: Student mind map for topic examining UMLIn general terms, each workshop session commences with a mini-lecture/discussion of themajor elements of the topic under discussion. This acts as a means of focussing studentattention on important elements and raising awareness of any conceptual mis-conceptions.The latter is aided by the use of mindmapping (Buzan and Buzan, 1994), a technique usedto map knowledge based in individual understanding. Students are required to developconceptual relationships both within each topic and between topics covered in the unit.253

Student example mind maps is shown as Figures 6.3 and 6.4. These examples (neither bestnor worst) from early in the semester demonstrate students’ ability to come to grips withthe technique – the central idea of a topic is identified, and other concepts radiate fromthis point. These become a component of the learner’s portfolio for the unit, but must beavailable for validation by the teacher at any workshop session, and act as an indicator ofconceptual understanding throughout the unit.Students are also able to monitor their own conceptual understanding through the MCQenvironment. In ENG260, the MCQ 2 is primarily a diagnostic/formative evaluation tool.The students are not mandated to attempt the tests, but a (slight) incentive is provided inthat a few of the questions (20 from a databank of over 500) are included in the final exam.The session continues with students undertaking tasks set for each topic, describing solutionsand issues in achieving these. The class is very interactive – students are expected to havecompleted and logged the reading required, produced mind maps and attempted the tasksbefore the workshop session. The trigger for most discussion, other than the initial topicsummary, is initiated by the student: asking for a task to be worked through, providing asample solution for the class to critique, raising conceptual problems, asking for clarification,etc. The aim is to minimise teaching, while maximising learning.Formal assessment is based on two assignments and an exam as well as a portfolio completedover the semester.The first assignment (worth 10% of the total marks), tackled individually, had as its outcomedemonstration of application of the components of a (simplified) Requirements Specification.Students developed Scope statements, Context and Use Case Diagrams, Sequence Diagrams,State Transition Diagrams etc, each ‘stand-alone’ in terms of context. Feedback also focussedon appropriate use of notation, tools, and process to develop these.The second assignment (worth 15%) required the same artefacts, now placed in the contextof a problem in some domain. A cohesive (still simplified) Requirements Specification wasthe deliverable. This task was team-based, and included the requirement for a critique of thedevelopment process and the group dynamics. In general this assignment acted as an ‘eyeopener’:the need to negotiate with team members, explore alternatives and resolve conflict,co-ordinate tasks and produce a deliverable that had a unified look-and-feel, taxed students.The exam (worth 60% of the total marks) attempted to address all levels of Bloom’s taxonomy:the lowest three levels may be considered as foundational thinking (Ryan and Frangenheim,2000), used as a basis for higher learning levels. A multiple choice section (extractedfrom the MCQ environment) addressed knowledge and asked for definitions, specific facts,2 Described in Appendix A254

while comprehension and application are addressed in terms of problems to solve and translationof verbal to visual form. Higher order learning is demonstrated through a requirementto evaluate alternate strategies, identify assumptions, explain data and excerpts of specifications,draw conclusions and provide abstractions from components presented.The portfolio (worth 15%) provided a mechanism for students to provide additional informationon their learning. Completion of mandatory components ensured a pass mark – however,students were encouraged to extend their reading in the discipline, complete additional exercisesand extend the mindmapping they were required to undertake.6.2.2 What actually happenedA virtue of Action Research is its responsiveness (Dick et al, 2000). At the macro level, itenables refinement of the ‘idea’ that drives the research through the various cycles that makeup the study. At a micro level, it facilitates ‘tweaking’ of the planned intervention to caterfor the changing environment and study context as they occur. In a classroom this is animportant feature.This cycle of the research project acknowledges the place of both types of responsiveness.Although the major interpretation of the data collected occurred post-hoc, the nature of thelearning environment implied that fine-grained adjustments were made as necessary. Thissection describes what was planned and what happened in the classroom in a reasonablyfactual way. The next section provides more explicit analysis and interpretation.What was plannedThis learning environment was based on a Cognitive Apprenticeship model (Collins et al,1989). In Cognitive Apprenticeship settings, learning is considered a process of active knowledgeconstruction that is dependent on the activity, discourse, and social negotiations thatare embedded within a particular community of practice (Brown et al, 1989). The teachermodels effective practices within authentic, professionally relevant contexts. The students inENG260 are presented with tasks they would undertake as practicing professionals, requiringproficiency with notations and tools, but also an appreciation of the context in which thesemust be applied. This requires an understanding of the underlying conceptual frameworksused in the domain. Because these skills are all new to them, the students are closely coachedby the teacher, both individually and in their groups later in the semester, to apply a processfor modelling each task as they reason about the issues being raised. Whenever thestudents reach an impasse, and are unable to continue or complete the task independently255

or with assistance from group peers, the teacher can ‘take over’ by once again modelling theappropriate approach, often out loud, in a protocol analysis environment, for all students.Gradually, students are required to complete tasks more independently, with the final classassessment item requiring the development of a complete model of a problem, with critiqueand justification of the approach taken, with minimal support from the teacher.Table 6.1: Phases of Cognitive Apprenticeship model as implemented in ENG260Phase Component Class Activities & Teacher RoleSessionsI Modelling 1-6 Demonstration of a task as a process. Exampleapproaches and sample solutionsprovided as basis for comparison and critique.Teacher explains strategies appliedand use of modelling tools (eg notation)explicitlyII Coaching 7-16 Critique and whole class discussion of individualapproaches applied. Focus is onexploration of multiple perspectives andthe reasoning processIII Scaffolding 17-20 Teacher’s role is to question, prompt andencourage students to stay on taskIV Fading 21-26 Student collaboration and peer discussionlead to a negotiated solution for submissionThe curriculum can be described as a 2-cycle spiral:• the first part of the semester (8-9 weeks) is focussed on learning the use of the tools,gaining an understanding of the conceptual framework (in this case Object-Orientationprinciples) and an appreciation for the context in which professionals practice (eg,historical overview, issues in RE theory and practice, organisational involvement, groupdynamics)• the second part of the semester focusses on issues of group work and knowledge transfer– students are involved in a group project that requires them to apply the tools to modela (slightly) complex problem.In broad terms, the planned approach was that the phases (see Table 6.1) of the CognitiveApprenticeship model would be traversed throughout the semester (which comprises 26 classsessions as noted above, each of 2 hours duration). It should be noted that there is no clean256

eak between phases – however the focus of the class sessions was expected to change, alwayswith the ability to revisit any phase as required.In order to maintain some level of independence from the intervention, neither content norformal assessment tasks were changed for this offering of the unit. The content was web-basedand supported by a textbook. For both assignments, while the problem changed, what wasrequired to be undertaken was identical to assignments of the previous years. The exam alsomatched that of previous years, with questions in each section drawn from a database I hadcompiled over the years. It should also be noted that the exam needed to be submitted duringsemester – usually by the eighth or ninth week of classes. In this way, the items of formalassessment could not be said to specifically target the learning interventions implementedin this cycle – rather they addressed the overall objectives of the unit in an engineeringschool and university environment (therefore addressing components of graduate attributesat professional and university level).What actually happenedMost of the students who arrived at the first class session were aware that this unit was‘different’ in that it was not taught in the lecture/tutorial/lab style they had experienced intheir first year. The first session stressed this difference. As well as the overview of the unitmandated by university policy, students were introduced to the on-line environment (whichwas new to them), to the techniques required to undertake mind mapping (which none hadseen before), and given an overview of how each class would be conducted.Although I did not describe the model of teaching per se, my teaching philosophy was brieflytouched on – in effect to provide them with competencies to be able to be RequirementsEngineers, if they chose to, not just to know about Requirements Engineering. Studentreaction was fairly predictable – most had chosen to study SE for the thrill of implementingsoftware, not specifying it (abstract stuff; hard concepts to grasp) 3 . The rest of the sessionwas based around the context for RE. The session ended with ‘homework’ – specifically whereto find it. Each topic had reading, exercises and sometimes a diagnostic test to complete.All work tackled could be included in the portfolio, to be handed in at the end of semester.Phase I and II of the Apprenticeship model appeared to work to plan (refer back to Table 6.1)for the majority of the students – each new concept was described for the whole class, examplesworked through and students then tackled similar exercises, generally in pairs. Studentsappreciated the accessibility of their learning resources (notes are all on-line; accessibility ofinformation; useful material) and my main task was to solve individual problems (generally3 Student comments (in italics) drawn from Year surveys257

of the how do I do this? and what do I do now? types) as well as addressing conceptualqueries. An example of the latter is provided below 4 .Subject: instancesDate: Sun, 10 Mar 2002 20:34:22 +0800From:[Morgan]To: Jocelyn Armarego Hi, [...] The def. for superclass in Brown page 143 seems alittle inaccurate; I think it should say that: superclass - aclass that is a generalization of its subclasses.Especially I thought it shouldn’t say that it includes all the*instances* of its subclasses ... because I thought we aredealing with object definitions (not instaces) at the classlevel, and objects are *instanciated* when the program runs.Since each session had been allocated one or two topics (ie from the web resources) it becamefairly clear which students were falling behind – these would prefer to watch modelling (iethe teacher doing the work) rather than engage in coaching – the students providing exampleexercises for the class to comment. The Week 4 Year survey confirmed this perception:sample negative comments included: lots of work; some concepts hard to understand wherethey are useful. Some overarching issues were also evident not enough info of what ... we areexpected to achieve; too much reading for the week makes the brain squirm.These concerns needed to be addressed during the semester – time out was made to discussthe nature of RE, it’s place and importance in software development, a revisit to the UnitOutline – which described the aims and objectives of the unit (this document was alwaysavailable online – most students accessed it at some time during each session, as it alsoincluded the topic breakdown on a week-by-week basis). A compromise on the workloadissue was reached – a mandatory minimum set of elements for the Portfolio were defined.While this latter appeared to relieve some of the stress in the class, at least one student wasnot impressed: no pressure to do weekly work.On the positive side, although attendance at workshops was not compulsory, participationwas very high (generally 90 - 100%): active participation on the part of most students wasan indicator of engagement with the unit (helpful class sessions; lab teaching is good; fun4 Throughout this thesis all students are given aliases258

topic, encouraging; good workshops). Students were quite expansive in their reasons for notattending, as the email below, from a mature-aged student, indicates.Subject: Me [Markus]Date: Tue, 16 Apr 2002 16:31:08 +0800To: JocelynI am in your engineering requirements unit G260 as yourprobaly may already know as I’ve heard you’ve been asking for mein class. We’ll I hope you do not think I am trying to avoidyou class it is only I have huge demand on my time, so I do mywork as best I can.[...]I’m not trying to make excuses for not being in class, though ifI feel I can cover the topics at home I do, as it’s a 80 klmround trip to school so if I don’t have to be there I usuallydon’t. I feel I am understanding the work pretty well maybefrom my years working and some of that a supervisor/ Manager. Imostly update, and will be well update hopefully by the end ofthe non teaching week.I have completed the assignment 1 which is due this thursday soI hope I was on the right track, some feedback from you would begreatly appreciated, I will put in your pigen hole at the officeif you have one.Sorry I haven’t contacteded you sooner, as I seemed to have notime even with time management skills, too much to do in tolittle time, hopefully this makes some sence to you as I amextremely tired at the moment, if you feel it is a problem menot being at class can you let me know.At this time, the class discussion often revolved around two issues:259

• the ‘correct solution’ to exercises undertaken. Student were comfortable with samplesolutions to their mathematics and engineering subjects being supplied by the lecturer.The idea of multiple acceptable solutions was a troubling one• removing themselves from their programming experience in order to define the problemas non-deterministically as possible. You can’t do that in Java almost became the classtheme.By the end of Phase II the the first spiral of the curriculum had been completed – the bulkof the discipline content had been covered, albeit some of it relatively broadly. Students hadcompleted the first (individual) assignment and received feedback. The rest of the semesterrevisited the content from a perspective of analysis and evaluation rather than application.So, for example, instead of just being able to produce an artefact (eg a Class diagram) theywere required to produce and then analyse it’s structure, elegance, critique alternative modelsetc. The context for this was the second assignment, in a group work environment. Theassignments, however, were always considered (by me, at least) as not the work of the classsession, which was still based on topics, reading and exercises in the web-based environment.At this point students were required to ‘take over’ to some extent – they would presentsample solutions to the class for discussion and critiquing. This addressed the Articulationand Reflection components of the Apprenticeship model, and assumed a Fading on the partof the teacher.It was from this point that the class became less cohesive in their participation within thelearning environment. Some students worked well and reasonably independently. With thesestudents (approximately 50% of the cohort), my role became one of supporting (Scaffolding)and then only intervening when requested/required (Fading). However, others were not ableto operate in this way: they continually attempted to revert to Modelling and Coaching.In effect, to use my brain rather than theirs. For them the Apprenticeship model see-sawedbetween Phases I and II and Phases III and IV – progression through the phases of the modelwas not smooth.This ambivalence about the learning environment was quite noticeable: while some studentsappreciated it as co-operative and interactive (class discussions are very useful, casual workshopenvironment), others felt it shifted the burden too heavily to their shoulders (far toomuch content to read; too much workload leading to no marks). Since the classroom provideda less formal environment for student-teacher interaction, students were open in voicing theiropinions and concerns. Where these were more general (ie not related to a specific student’sproblems), adhoc mini-focus group sessions were undertaken. While, in general these did notimpact on the intervention model, the issues were considered and timely feedback presented260

to students. What this meant was that students were much more aware that any previousgroup, of what was being attempted, and why.An additional ‘distraction’ was the requirement to work more formally in groups for the secondassignment. To all intents and purposes, this meant that class work was also undertakenin the same groups – this gave them the opportunity to flip between assignment and classwork throughout the session, and was useful in particular when I was making the rounds– they could raise assignment problems at that time. As has been stated, this assignmentacted as an ‘eye-opener’: the need to negotiate with team members, explore alternatives andresolve conflict, co-ordinate tasks and produce a deliverable that had a unified look-and-feel,taxed some students (this leaves little time for other subjects; not clearly stated what we arerequired to do in terms of assessed work) while others could relate the experience to theworkplace (can see industry advantages; shows how to analyse stuff). In almost every case ofa request for an extension beyond the due date, the comment was on the lines of the work isdone, just not put together.The unit was completed with submission and return of the second assignment, submissionof the Portfolio and a final exam. The portfolio, in particular, was a contentious issue.Students had indicated, as early as Week 4 of semester, that they considered it unnecessaryeffort, despite the value of marks allocated to it (large workload; need to learn new software).Although a minimum was set at that time (in response to comments in the Year survey)negotiations continued during the rest of the semester. However, students indicated theyfound the mind maps particularly useful. For some students, these were checked periodicallythroughout the semester while for others they were completed in block mode. They appearedto be used by many students as their unit review material for the exam – discussion sessionsduring the study week were split between conceptual understanding (with mind maps asprops) or application of the notation correctly.6.2.3 Interpreting what happenedThe purpose of the evaluation conducted after the 2002 semester was to assist in determiningif the implementation was effective in facilitating student learning and transfer, and toprovide the researcher with data to reflect on improvements to be made. Table 6.2 providesa summary of the instruments used to collect data for analysis in this cycle.All elements of assessment were available to be evaluated in relation to this research. Forthis cohort, this included individual and group application of tools and methods being learnt(the assignments) and a Portfolio that included mind maps to provide some information on261

Table 6.2: Instruments applied to Cycle 1Cycle Diagnostic Devices Records FeedbackLearningStylesTeachingStylesCycle 1 LSI Pedagogical Assessment items Year SurveyDimensions & results 2002ILSSSUUnit developmentPortfolio (mindrecordsmaps)conceptual understanding, and elements to supply some information on student’s willingnessto explore outside the boundaries provided within the unit (and hence transcend the coursematerial) by means of a reading log and additional exercises attempted.In addition, all students had been mapped to a learning style during their first year, andformal University-wide feedback (Student Survey of Units (SSU)) collected. The Schoolbased,Year Survey was also a rich source of student perception of the unit undertaken.These all provided qualitative as well as quantitative data, with responses to open questionsand comments in the surveys considered of primary importance.Impact on student developmentStudents had indicated their ambivalence about the learning environment during semester.The Year surveys supported this perception – easy to understand; interesting; encouraging;seems very important for large systems; abstract stuff contrasted with too easy; abstract stuff;hard to put theory into practice.The results of the SSU feedback confirmed these. Students liked the online environmentand the scaffolding it provided. However, while some students could see the benefits ofa less traditional learning environment (...all members were able to spend more time withthe lecturer which therefore lead (sic) to a greater understanding of the units subject. The“discussion” format in the lectures also made this unit very enjoyable; good class involvement,good group activities) many just wanted to be taught (...need lecture notes, powerpoint etc;[change to] ... labs & lecture format).Unfortunately, the anonymous nature of this feedback made it not ethically acceptable torelate specific comments to particular student learning styles. An interesting insight mayhave been provided by the comparison: were particular learner types more/less comfortablewith the Apprenticeship model (as the literature suggested), or were the comments acrosstype? This question was worth keeping in mind to be addressed in the next cycle, if possible.262

Although issues raised by the Year surveys were actioned as required by the procedure setup within Engineering, after the end of semester all comments received were open-codedto produce a set of categories that might provide insight into participant perception of theenvironment in which they were learning. From these categories themes were extracted.Themes identified addressed:discipline content : comments regarded relevance (also linked to motivation), understanding,depthenvironment : including lecturer style and delivery, general class sessions formatmaterials : website, text, other sourcesworkload : assignment formats and load, unit workload throughout semester.In each case both positive and negative comments were made. The issue that appeared, atthis time, to be of primary importance revolved around student perception of the learningsituation. This encompasses the themes of discipline content and environment. The othertwo themes were considered of marginal interest in that they were ‘expected’ as a componentof any student feedback received, and were not delved into at this stage.Students come to the unit with strong preconceptions regarding the learning environmentand an awareness that ENG260 would be ‘different’. This is understandable since the focusof the previous year is to engender a scientific/engineering way of thinking, and for thosestudents articulating in, a prescriptive approach to learning. Marks for assessment components(especially the portfolio) also suggested many students were only interested in doingthe minimum required to learn the tools and techniques, and pass the unit. This confirmedthe importance of motivation as an element of a primary theme.The learning environment for ENG260 provided a contrast that some students found difficultto assimilate. Although due process and procedure has its place, the focus of the unit is ondivergent thinking and the development and evaluation of alternatives. Students come withsome competence in programming. In ENG260 they are asked to ignore the problem-solving(coding) of a situation presented, and to explore and then formulate the problem itself. Myexperience in teaching this unit has shown that students’ expectations are challenged:• they expect there to exist a definitive solution to the problems with which they arepresented (à la science/mathematics)• they expect to define the problems only in terms of the programming language withwhich they are familiar (currently Java)263

• they expect a fundamentally competitive class environment to exist (is my solutionbetter than yours)• they expect their ‘wild ideas’ to be laughed at and ultimately rejected, and thereforeare inhibited in expressing them.In summary they see software development as fundamentally scientific (where following adefined process will lead to a quality product (Pfleeger, 1999)) and well suited to their nature.This perspective is not unexpected – as Baxter-Magnola (2001) and Perry (1988) suggest,students at low intellectual (or epistemological) stages of development either believe thatevery intellectual and moral question has one correct answer and their (competent) teachersknow what it is or are transitioning to believing that some knowledge is certain, and thatmaking judgements following logical procedures prescribed by authority deserves full credit.Challenges to their belief systems within the units they take and interactions with peersare necessary for them to gradually come to believe in the validity of multiple viewpoints.However, discussions during the class suggested these perceptions were very little changedunder the Apprenticeship model for RE.It could be argued that the issues identified reveal the inappropriateness of the learningsituation for the material to be learnt. Yet the literature of the discipline (and of educationin the discipline) suggest otherwise (see Chapter 2 for a discussion of these).Alternatively, it could be argued that the learning model was inappropriate for the characteristicsof the student cohort. This is (negatively) supported by the literature – that traditionalteaching, at least in an engineering context, fails to address the needs of most students, evenif it is the dominant approach (and the one, therefore with which most students are familiar)(see for example the discussion in Felder and Brent (2005), which addresses extensivelystudent diversity in learning).Success of the learning modelThroughout the semester, the principles of the Cognitive Apprenticeship described in Collinset al (1991) were kept in mind, and applied as seemed valid. So, for example, the secondcycle of the curriculum focussed more on heuristic and control strategies than basic domainknowledge – so not just how to construct a Class diagram, but what techniques make iteasier to do so ‘elegantly’, what different approaches exist for achieving the same task. Thesocial characteristics (Sociology) were also reasonably well aligned. The tasks were realistic,situated and required co operation. Intrinsic motivation was more difficult to achieve –many students were motivated to just pass the unit. The portfolio mark, as an indicator of264

motivation, showed that just over 50% of the students expended effort to pass this component(students would pass by completing all tasks marked as mandatory) of which only 12% wentwell beyond the requirements of the material presented. However, most students completedthe mindmapping component of the portfolio (requiring a minimum of twenty six mind maps).This may be seen as student acknowledgement of its usefulness as a formative and diagnosticassessment tool, but for the purpose of passing the exam.The model suggest the Sequencing of learning activities should be global to local, increasingin complexity and increasing in diversity. Increases in complexity and diversity did not causemany problems – exercises became more complex as the unit progressed and students wererequired to integrate the skills they had learnt for each component independently to developinga whole in the second assignment. However, the strategy of global to local skills wasproblematic. The course material had been developed to teach individual skills independentlythen putting them together in the second spiral. However, sample artefacts were availablefor students to examine (eg example Requirements Specifications comprising the individualcomponents). A global perspective was provided by placing RE in the context of softwaredevelopment and the issues encountered, but some students found this confusing – why didthey need to know this abstract stuff? Often comments such as hard concepts to grasp wereaimed at the less concrete components of the material.Figure 6.5: Raw exam marks for ENG260 1999-2002Despite these comments, if success were measured by academic results, ENG260 following anexplicit Apprenticeship model could be classed as successful: Figure 6.5 shows that a larger265

percentage of students were successful in the final exam (80% pass rate, compared to thelowest pass rate of 44% in 2001), with marks generally indicating an acceptable distribution.All but one student passed the unit (Figure 6.6).Figure 6.6: Final mark for ENG260 1999-2002These results suggest that, in terms of acquiring discipline knowledge in a situated, collaborativeenvironment, the Apprenticeship model worked relatively well. The raw exam marksindicate students were individually able to master to content. The caveat to this is the initialperception that, based on the learning styles profile of the cohort, the learning model shouldnot inhibit student learning.The results for the Portfolio component, however, suggest most are not interested in exploringbeyond the set requirements of the unit – student comments regarding not being pressuredto complete weekly tasks support this perception. This indicated while the model enabledeffective learning across the learning styles, the motivational element was missing.Alternatively, this data may indicate a teacher problem rather than a learning environmentproblem. With the results of the thematic analysis in mind a review of the unit based onReeves (1997b) (see Figure 6.7) was undertaken, to answer the question:is the teacher’s style and stance supportive of student-centred learning, or is it aninhibitoras could be suggested by her personality profile (see Section 6.1.2) and the student issuesraised.266

Figure 6.7: Pedagogical dimensions of RE as at 2002This instrument focuses on the unit rather than the teacher, but provides a profile of thecharacteristics of the learning environment. This also enables insight on teaching style to begained. As has been described in Chapter 5, the instrument provides 14 dimensions for theevaluation of technology-assisted learning.The review of the unit showed that ENG260 had a reasonably high level of teacher directionand indicated a Transitional approach to teaching. This may appear confusing to thestudents: providing mixed signals on how they should learn, and therefore deserved furtherinvestigation.Long(er)-term impact on student developmentThe same student cohort undertook the follow-on unit in 2003. The RE component of the unitshowed an improvement in student success rate: a mean of 79.3%, up from 71.6% the previousyear (undertaken under very similar conditions, although the problem was a different one).Students were also able to complete the task closer to schedule (it took students 8 weeks toproduce a Requirements Specification in 2002 (3.5 weeks over schedule), 4 weeks to completethe task in 2003). These figures suggest a measure of transfer is occurring – students areable to complete the task given in an appropriate time frame and at an appropriate level ofcompetency.However, students expressed increased concerns regarding the learning situation of this followonunit, presented in a more learner-centred mode. The feedback there suggested that, while267

students were comfortable in taking some control of their learning in a capstone projectunit, a formal unit should be ‘taught’. Students expressed concern that the problem wasopen-ended and too big (the teacher should establish tight boundaries), that they had toestimate and manage the project themselves (rather than having milestones imposed), thattheir assessment was based on the motivation and performance of the rest of the class (allassessment was group-based).6.3 Reflection on findingsReflection is also an aspect of evaluation, and in this study, plays the important role ofguiding the journey, and triggering pedagogical growth in the researcher. Kreber (1999)’sscholarship of teaching model includes reflection on content (what do I know?); process (howdo I know I am effective?) and premise (double loop learning: what are the alternatives?)within the domains of Instructional, Pedagogical and Curricular knowledge. Table 6.3summarises the issues to be considered in applying this reflection model.Table 6.3: Reflection in the Scholarship of Teaching model (Kreber, 1999)ContentCurriculum Pedagogical Instructionalgoals, purposes & rationalesfor the unit; how it respond to different learn-learning objectives; choos-how students learn; how to writing & sequencingfits into the larger curriculum;how the teaching con-to studying; how to fa-discussions & group work;ing styles & approaches ing readings; facilitatingtributes to the universitys cilitate critical thinking & preparing syllabi; constructingand evaluatingsocietal & cultural role self-management in learning;how to influence studentsmotivation toassessmentlearnWhat are the goals ofmy teaching?What do I know abouthow students learn?What instructionalstrategies should Iuse?Process How conscientioushave I been in identifyingthis goal?Premise How does my goalmatter? What are thealternatives?How effective am I inpromoting its achievementWhat are alternativestrategies?How effective have mystrategies been?Why does it matterthat I use this strategy?The results on my reflection included:Curriculum the goals of this cycle were two-fold: to provide a mechanisms for students totransfer the discipline knowledge they had gained to other units; to provide a learningenvironment that was situated in authentic practice. This aligns with practitioner268

comments that graduate practitioners need to be able to adapt what they have learntto different contexts, and with concerns regarding how RE is practiced.The Apprenticeship model appeared reasonably effective in this regard: students wereable to learn the tools of the discipline they were exposed to and apply them appropriately.The grades they achieved attest to this conclusion, both within the unit, and ina follow-on where this knowledge was necessary.However, students appeared reluctant to take control of their learning: in ENG260their performance on components of the unit that were less directed (eg elements ofthe portfolio), indicated they were less inclined to devote the time required to completethese. In the follow-on unit this perception was more explicit – student concernsregarding the open nature of the learning environment supported thisPedagogy the workshop component of the model was deemed effective for some students butneeds refinement to address difficulties experienced by other students. Although I havedata on student learning styles, a deeper understanding of the assumptions studentsmake about the learning environment is necessary.My effectiveness therefore in promoting the objectives is only partial: while studentsappeared more confident, in subsequent units, in applying the knowledge they hadgained, they still expected to be taught: that is, the master/apprentice relationshipwas assumed even if no longer appropriate. Alternative strategies for situated learningneed to be explored. However, these need to enhance the gains made through theApprenticeship modelInstructional students expressed concerns about the spiral nature of the curriculum – forsome this was boring. Others commented on the amount of preparation required (egreading, completing exercises) to support attendance at class. This suggests the instructionalaspect of the curriculum should be examined to determine alignment betweenits components.Reporting findingsAn internal review of ENG260 was undertaken towards the end of 2002. In the context ofthe refinement of the SE curriculum within Engineering, the findings of the Apprenticeshipmodel were described and discussed with the Professor of Software Engineering and other SEcolleagues. The purpose of such meetings was to ensure a holistic approach to the curriculumwas maintained, to provide feedback on the success or otherwise of learning strategies appliedbased on the cohort’s performance in subsequent units, and to identify implications of such269

strategies on other units within SE. At this time changes proposed for Cycle 2 were preemptedand approved internally.Based on a report describing the findings of Cycle 1, funding was sought to resource theredevelopment required for Cycle 2. These were obtained from both the University TeachingLearning Centre (TLC) under their Innovative Teaching Development & Research Schemeand from the Division of Science and Engineering’s Teaching Quantum Funds. These specificallytarget innovations in teaching practice for wider dissemination within the Universityenvironment.This cycle was reported in the literature only as the background to future interventions.The Conference on Software Engineering Education & Training (CSEET) paper (Armarego,2002), however, preempted the PBL approach taken in the Requirements Engineering unit bydescribing a restructuring of a follow-on unit. This case study informed the work undertakenin the next cycle of the Action Research study. In this context, the CSEET paper (andthe feedback from reviewers and participants) addressed the need to expose the study toacademics and practitioners engaged in the discipline. Of particular interest, reviewers’suggested (on a 6-point Likert scale) that this work was very good to excellent in its likelihoodto be used and referenced by others, to influence them, or to stimulate their thinking.6.4 Conclusions drawn from Cycle 1Even within a constructivist framework, the relationship between teacher and learner (ormaster and apprentice) can remain unidirectional – the former modelling behaviour for thelatter (Duffy and Cunningham, 1996). In this environment, the teacher, in a coaching role,acts as an authority-figure and gives learners explicit directions on what to do, how to do it,and when (Grow, 1991/1996).Evaluation of the Cognitive Apprenticeship model in relation to practitioner characteristicsindicated that although this model addressed some components of industry needs, the fitbetween characteristics of action in the discipline and those of the learning model exhibitedelements of an ‘incorrect’ learning environment. Students learning within the Apprenticeshipmodel exhibited some of the traits of surface learning - they focussed on learning the toolsand techniques of RE at the expense of a more expansive view of the discipline: they did notsee themselves as acquiring the more generic skills valued by practitioners, with the majorityof students focussed on being able to apply the tools and techniques in order to pass the unit.Thus the conclusion reached was that the master/apprentice relation could be down-playedso that students took early control of their own learning, and that a more open approach270

to describing the design of the unit might be beneficial to students challenged by its nontraditionalnature.The next chapter looks at changes made for the following offering of ENG260, with the aimof addressing issues raised in this cycle.271

Chapter 7Implementing a model for creative REeducation 2003Today’s constraining factor is not the software, not the hardware, not the network.It is human creativity(Keegan, 1998, p 239)As noted at the conclusion of the previous chapter, reflection on the success (or otherwise) ofthe Apprenticeship model highlighted issues, both immediate and longer term, that requiredconsideration. In summary, in the Apprenticeship model students focussed on learning thetools and techniques of RE at the expense of a more expansive view of the discipline.Student perception of the learning situation suggested some of the traits of surface learning.Investigation of the literature related to this concept – surface and deep learning (eg Entwistleand Ramsden (1983) and Richardson (1990)), indicated further analysis and reflection of the2002 data was warranted. This highlighted an issue that had not previously been flaggedas overly relevant: student feedback indicated the perception of a heavy workload. Thissuggested the instructional aspect of the unit needed be examined more closely.There is some support in the literature to suggest such a perception is an indication ofsurface learning: Entwistle and Tait (1990, 1995) found that students who reported themselvesas adopting surface approaches to learning preferred teaching and assessment procedureswhich supported that approach, whereas students reporting deep approaches preferredcourses which were intellectually challenging and assessment procedures which allowed themto demonstrate their understanding. More recently Cope and Staehr (2005) confirmed thatperception of workload appeared to be a key to encouraging the use of deep learning approaches,while surface learning have been associated with perceptions of too high a workload.272

This suggested further examination of the pedagogical aspect, in terms of approaches tolearning, was necessary in order to explore this perception.Kember (2006), describing his earlier work (Kember, 2004), acknowledges that perceivedworkload is a complex construct which could be influenced by a wide range of aspects ofteaching and learning, all interrelated. However, he hypothesised that students could beencouraged to work hard, without perceiving that workload was excessive, by creating asuitable teaching and learning environment, that, among other aspects concentrates on keyconcepts and promoting understanding, includes assessment which tests understanding andpromotes a climate in which student-student relationships and class coherence can develop -particularly through group discussion, assignments and projects (Kember, 2004, p 181-182).These become aspects to be considered in relationship to workload, in this cycle.This was in addition to the issue flagged by the problem encountered during a follow-onunit. This indicated that the curriculum knowledge (ie the expectation I had, as teacher,of where the students should be in terms of their engagement with the profession and thediscipline) also needed further consideration. As a summary of this problem, while studentswere more comfortable with the idea of directing their own learning during the capstoneprojects, they felt (very strongly, at times) that within a formal unit, they should be taught.This highlighted a need to emphasise student-centred learning earlier in the curriculum.Figure 7.1: Education for RE – Action Research Cycle 2273

This reflection on the outcomes of the Apprenticeship cycle of the study informed the planningfor Cycle 2. At this point the issue of student-centred learning was of prime importance.This literature (discussed in Chapter 3) pointed to learning strategies based on inquiry, orprojects or problems as addressing some of these concerns. Figure 7.1 places this cycle in thecontext of the thesis as a whole.Additional triggers for changeWithin the Engineering Education environment, a change in focus which included equippinggraduates for lifelong learning as well as receiving a broader education with a wider rangeof backgrounds was advocated in the mid 1990s (IEAust, 1996). This is an acknowledgementthat the development of student skills and understanding in generalisable and transferableskills is a necessary dimension of professional education (McLaughlan and Kirkpatrick, 2004).Active learning approaches (which include collaborative learning, problem-based learning,case methods and combinations of roleplays and simulations) are advocated to engage studentsin higher order thinking tasks such as analysis, synthesis and evaluation (Bonwell andEison, 1991). Active learning methods attemptto develop the cognitive [knowledge, understanding and thinking] and affective[emotive] dimensions of the learning process in such a way that learners’ activeinvolvement in the learning is improved.(Learning and Teaching Support Network (LTSN), 2003)Criticism of engineering graduates’ lack in the ability to be flexible and creative (specifically inthe lateral and divergent thinking (Guilford, 1967) seen as an implicit and necessary quality ofa practical engineer (Alpay and Ireson, 2006)) also confirmed the decision to address aspectsof creativity in this cycle.Creativity is described as a balance of convergent and divergent thinking appropriate to thesituation (Nickerson, 1999), with the process of design seen to rarely be convergent, in thesense of being directed towards a single preferred solution. The issue of introducing creativityearly in the design education environment is also being tackled in that environment throughless traditional learning models (in this case Design-Led Learning (DLL) (Edwards, 2002)),with similar concerns regarding how early DLL can be introduced, the amount of content‘lost’ and the depth of learning achieved. The conclusions to date suggest that creative designcan be integrated into the curriculum from the introductory year (Lumsdaine et al, 1999),but that support and mediation is mandatory (Edwards, 2002).While RE is a problem-developing process where design is a problem-solving process, designshares the RE characteristic of being described as open-ended, unstructured, or ‘wicked’274

(Rittel and Webber, 1984). However, problem development requires enhanced creativityskills – superimposing a goal (ie a solution) too early in the process inhibits the necessarycreative thinking required (Boden, 1997). This is confirmed in the work of Thomas et al(2002) regarding structuring failure in early design.Gardner (1999) argues that creativity is a coincidence of many factors, which includes thediscipline to master a domain and a lack of hindrance from the fear of failure. However, aproblem requiring a creative solution is likely to be challenging, and therefore may createmotivational problems, particularly in an educational setting. In addition, challenging learningenvironments favour students who are deep learners and therefore could be said to fail toaddress the diversity of learning approaches in the student cohort.Establishing a creative environment, based on authentic problem scenarios in a ProblembasedLearning context which incorporates creativity-enhancing processes seemed an appropriateapproach for both challenging student preconceptions and focussing on the insightdrivenopportunistic nature of the RE process, as well as potentially tackling the issue ofworkload perceptions.The choice of PBL over other situated and action learning models was made on the basis ofsome familiarity with the model (it had been applied for two years, at this time, in a follow-onunit in SE), its seeming appropriateness for tackling issues in the discipline (despite a paucityof literature for PBL in the IT discipline generally and SE specifically) – as well as having adefined process to provide the discipline to assist students in mastering the domain, and thepotential for the model to integrate with creativity-enhancing approaches. In addition, PBLappeared to address wider engineering education issues, such as generic skills and life-longlearning. These are discussed in greater detail in later sections of this thesis.This chapter describes how the issues identified above were addressed: how the interventionwas planned, what actually occurred and possible interpretations for findings of the cycle.As for Cycle 1, two distinct evaluations are proposed: implementation of the model and itssuccess and the effect of the intervention on the student cohort.In summary, these evaluations suggested that the process-driven characteristics of PBL wereat some odds with the characteristics of the discipline, in particular the opportunistic natureof problem solving in RE. While the environment appeared to facilitate creativity-enablingactivities by embedding these within the process, the process itself acted as a deterrentto student motivation to study (and hence on deep learning), and to the creativity beingnurtured – opportunism was difficult within the process and hence flexibility inhibited. Ineffect, a focus on process detracted from the ‘authenticity’ of the environment. Some of thefindings of this cycle have been published. Section 7.2.5 provides details.275

The next section describes changes to the learning environment which impact on this cycle.The chapter then continues in its description and discussion of the components of theCreativePBL model.7.1 Context for Cycle 27.1.1 Curriculum componentsIn 2002 the curriculum model was changed so that the Design Project and Thesis (ie groupand individual capstones) were merged and effectively outsourced (see Figure 7.2)Figure 7.2: BE(SE) Murdoch University: curriculum components post2002This change had critical impact on ENG260 – as was noted in Chapter 6, for the majority ofstudents the experience of student-centred learning had been based on the capstone projects– they preferred to be ‘taught’ within formal units. The follow-on unit to ENG260 hadalready been modified to assist students to transition to the workplace by providing a learnercentredenvironment based on PBL. However, issues raised there suggested a student-centredapproach to learning should be introduced early within the student learning experience – thefinal year was too late. That work (reported in Armarego (2002)), can be summarised asproviding students with a number of opportunities:• to identify, analyse and solve a number of issues, repetitively. This acts as preparation276

for professional employment• to practise the art as well as science of SE in a controlled setting• to test the understanding of theory, its connection with application, and develop theoreticalinsight• to deal with incompleteness and ambiguity• to think independently and work co operatively, fostering insight into individual strengthsand weaknesses.The findings of that work inform the decisions made in this cycle. In particular, it appearedthat these opportunities should be made available earlier in the students’ academicexperience.7.1.2 Characteristics of teaching and learningChanges in cohort characteristicsTable 7.1 compares the 2003 RE students (ENG260) with both the cumulative first yearengineering students at Murdoch and with figures reported in Felder and Brent (2005), basedon over twenty studies of undergraduate engineering students.Table 7.1: Learning style of undergraduate engineering students (percentages)and staff (based on Soloman & Felder Index of Learning Styles)Processing Perception Input UnderstandingACT REF SEN INT VIS VER SEQ GLOFelder 64 36 63 37 82 18 60 40(2005)Eng 1st 56 44 63 37 77 23 56 44year (cum)ENG260 75 25 90 10 95 5 75 25Legend: ACT-active; REF-reflective; SEN-sensing; INT-intuitive; VIS-visual; VER-verbal; SEQ-sequential;GLO-globalWhile it can be seen that the learning styles of Murdoch Engineering students (since 1999learning styles data for each Year 1 cohort have been acquired) substantially align with thosereported by Felder – Murdoch engineering students are typical of Engineering students, theRE cohort show a strong(er) bias in each of the dimensions that is dominant for engineers.They are more active, therefore should prefer group work and dislike lectures, more sensing,therefore prefer to learn facts, and solve problems by a well-established process, very strongly277

visual where reading text is classed as verbal and fundamentally sequential, implying a preferencefor logical steps to problem-solving (Soloman and Felder, 1999).The Kolb results (Table 7.2) support these findings: the 2003 RE students are most stronglyengineering types. They are pragmatists (Convergers) who revel in active experimentation(labs, fieldwork) with a tendency to narrow technical interests, or theorists (Assimilators)with a forté in the basic sciences. The imaginative ability of the Diverger or the intuitiveproblem solving of the Accommodator is sadly limited. Yet we have seen that these are skillshighly rated in an RE context.Table 7.2: RE 2003 students compared with the Murdoch profile (percentages) (based on KolbLearning Style Inventory)Eng 1styear(cum)ENG1082001*Accomodator 8 18 10Diverger 18 3 10Assimilator 33 32 40Converger 41 47 40*The 2002 RE students were a subset of this groupENG2602003In years previous to 2003, the class predominantly comprised students expecting to completea 4-year Bachelor of Engineering (BE) degree, most probably in software. These studentscome into the programmme with a higher tertiary entrance score than Bachelor of EngineeringTechnology (BTech) students, and have completed one year of study within the School.This has implications in learning expectations, acceptance of the learning culture within theSchool, etc.As Figure 7.3 shows, in 2003 the BE students were in the minority. An additional factorin 2003 was that almost 50% of the BTech students were TAFE (Technical and FurtherEducation, ie technical college) articulation students. These students entered the programmewith advanced standing, and although a formal analysis in the Murdoch context had not beenundertaken at this stage, there was anecdotal evidence that they had additional problems.A discrepancy between the teaching/learning environment they were expecting and the onethey were experiencing was almost immediately apparent across all units they were studying.The characteristics of both articulating and Murdoch students have implications for thelearning environment of this cycle. Firstly, while the Apprenticeship model aligned reasonablywell with student learning styles, the PBL model could be expected to be a challenge – PBLis considered an ideal pedagogical strategy only for Accomodator students (Felder and Brent,2005), who make up one of the minorities in the cohort.278

Figure 7.3: Requirements Engineering class cohort 1999 - 2003 (Armarego, 2004b)PBL could also challenge articulation students with regards to their tacit preconceptionsabout university education. Ferris (2003) provides a discussion of the issues of advancedstanding in an engineering education environment. Although at this point he is specificallytargeting articulation from overseas, some issues have a parallel in the Australian context:the jump, from a diploma to the middle of a University degree, demands skills that are notdirectly taught by specific units but are conveyed through the learning environment andassessment methods, the broader aspects of curriculum and through a process of encouragingstudents to question teachers and to explore through project based learning. As Ferris (2003,p 39) notes:these capabilities, distinguishing them from the mechanistic connotation of competencies,underlie success in an engineering degree but are not necessarily developedin courses mapped by a content oriented advanced standing process.Ferris concludes that the capabilities of graduates involved in articulation programmes aredifferent than those possessed by normal entry students. This difference relates to the studentsperception of the approach to the analysis of engineering problems and the ability ofthe student to create novel solutions to problems, as well as an expectation to be taught.Since these are issues targeted in this cycle, ongoing monitoring of the intervention has increasedimportance, while interpretations of the data have to take the cohort characteristicsinto consideration.279

Teacher characteristicsThe results of the Reeves (1997b) review of ENG260 (discussed in Chapter 6) also suggesteda deeper exploration of teacher characteristics was warranted. Those described below are theresult of an instrument applied late in 2002, placing it at the transition from Cycle 1 to Cycle2 of the Action Research project. The impact of the results will be described a little later inthis chapter, in the context of the implementation of the intervention.Table 7.3: TSI categories (adapted from Dunn and Dunn (1993) by University of Toronto)Category* Number of Questions1 Instructional Planning 122 Teaching Methods 6Teaching environment3 Student Groupings 64 Room Design 65 Environment 76 Evaluation Techniques 87 Teaching Characteristics 88 Educational Philosophy 14*numbering refer to the data labels on the chart (see Figure 7.4) based on this instrumentAt the commencement of Cycle 1, the teacher is described as an Assimilator with the personalitystyle of ‘Teacher’. Strong characteristics of these styles include a tendency to bedirective, assuming expectations will be met and implicit commands obeyed, as well as beingcharacterised as reflective, creative/theoretical and verbal.This is borne out by the Teaching Style Inventory (TSI) developed at University of Torontoand adapted from Dunn and Dunn (1993). This inventory provides a mechanism to comparea teacher’s teaching philosophy with the methods applied in the classroom. It is based on 67questions, categorised into six groups. Each Likert-based response is weighted according tothe relative importance of individual items. Table 7.3 provides a summary of the groupingsand subscales where relevant.The scores obtained can be charted against predictor profiles that identify teachers as HighlyIndividualised, Somewhat Individualised, Transitional, Somewhat Traditional and Traditional.As Figure 7.4 shows, the teacher/researcher of this study has a Transitional approach toteaching.Although the authors suggest that most teachers’ philosophy is far more individualised thantheir methods – these have institutional and social constraints that bind the practices ofteaching and learning in higher education and elsewhere (Smyth, 2003), this profile indicates280

Category numbering refers to the category labels in Table 7.3Figure 7.4: Results of the TSI applied to the teachera close alignment between teaching philosophy and methods, and also aligns with the resultsof the Reeves-based review. However, Smyth’s comments on constraints do have bearing:Room Design, for example is difficult to change when the original set-up models a laboratoryand School policy/politics dictated which rooms are utilised for which classes.Again, these insights have enormous implications within this cycle. Of particular importanceis the tendency to keep control of the class. The PBL model requires the teacher to act asfacilitator in an attempt to remove teacher-direction. What strategies were applied to achievethis are described later in the chapter.7.2 Cycle 2 – Creative REThe decision to redevelop ENG260 to a Problem-Based Learning model was not a trivialone. There was potential to impact on the teacher’s academic duties during the developmentperiod (July 2002 - February 2003) and to require considerable resources to ensure the task281

was well done. The implication of this was that internal support was mandatory, and forthcomingnot only from the Professor of SE and discipline colleagues, who took on the role of‘critical friends’ throughout the development process, but also from the Division of Scienceand Engineering, and the University in the form of funding and mentoring.The process of redevelopment required reflection of many aspects of the unit: while muchhas been written regarding the value of PBL in learning, (eg Boud (1985); Wilson and Cole(1996)), undertaking such a project comes at a cost:• content - guidelines for implementing PBL indicate that success is partly based on areduction to the content covered: assuming too much content is a pitfall in a PBLenvironment (Albanese and Mitchell, 1993). This also is useful for modelling expertise– research suggests that a superficial coverage of many topics in the domain may be apoor way to help students develop the competencies that will prepare them for futurelearning and work• time to develop project - Bridges (1992) suggests that each PBL project requires 120 -160 hours to construct, field-test, and revise. To this figure should be added technicaleffort when the problem is developed in an online environment• cost – PBL is economical for classes of less than 40 students (Albanese and Mitchell,1993). It is considered not to scale well to large student numbers without great increasein staffing resources• more time to teach less content – Albanese and Mitchell (1993) suggest 22% more timeis required to teach in PBL mode, despite the reduction in content usually advocated.In their study, when academic staff consider the time per week in preparation to teachproblems in comparison to presenting lectures, instead of 8.6 hours/week primarilypreparing lectures, staff spend 20.6 hours/week primarily in groups with students• difficulty in transitioning, both for staff and students – Bridges (1992) suggests academicstaff are uncomfortable withholding information as they watch students struggle withproblems, and need training to develop facilitator skills or they may be unsuccessfulin PBL. Students may be uncomfortable with the extensive collaboration required orwith the lack of teacher-direction given.Funding was obtained from both the University Teaching Learning Centre (TLC) under theirInnovative Teaching Development & Research Scheme and from the Division of Science andEngineering’s Teaching Quantum Funds. These specifically target innovations in teachingpractice for wider dissemination within the University environment.282

A team comprising the researcher, the Academic Support Officer from Murdoch’s TLC asmentor, a third year SE student as research assistant and a CS student with expertise ininteractive graphics as technical assistant laboured over one semester plus the summer breakto redevelop the unit. Each role was critical to the success of the redevelopment:• the CS student produced all the code required for the interactive component of theenvironment – in an open format based on xml and Java scripting, so that parameterscould be adjusted as necessary throughout the semester, and the web pages modified• the SE student maintained records of the team meetings and helped source and developsupport material required by the PBL process. In addition, as the top student in theclass when she undertook ENG260, her perceptions of the learning environment werevaluable (although it should be acknowledged that the best students are less likely tobe deterred by a bad learning environment). Also, although a gender agenda was notexplicit, it was important that the scenarios being developed did not demotivate thefemale minority in the class• the Academic Support Officer’s role was to keep the researcher focussed. Most decisionsduring the process were met with a why? – the need to explain and justify required avery critical and reflective approach to the task, and was invaluable. In addition, herexperience was in applying PBL in an IT environment, enabling some commonalitiesto be exploited• the researcher acted as discipline expert, and, obviously, had initiated the project. Allthe conceptual modelling was mine, as was the development of the scenarios and the‘sequencing’ within the environment• discipline colleagues at Murdoch acted as sounding board, while my academic supervisor– an RE practitioner and researcher took on the role of devil’s advocate.Final testing was completed only days before the unit commenced: this tight schedule meantthat some decisions were deferred (eg the lead time was too short to remove the final examfrom the assessment profile). Absolute commitment to run in PBL mode was not requireduntil after Week 1 classes - the Apprenticeship model as backup was always possible. However,once the decision had been made it was very important that the technical infrastructure wasrobust: the problem scenario was dependent on online triggers being released automaticallyand available to students.Sections 7.2.1 and 7.2.2 describe the extensive planning required before the CreativePBLmodel could be implemented.283

7.2.1 Decoding the disciplineMajor ‘intellectual’ effort went into deciding on what content needed to be addressed by theproblems developed – the question became one of what did the students have to know to beeffective in further units? Although this is an issue in all PBL environments, it had addedsignificance in RE because it is foundational for the SE degree programme – many disciplinespecificconcepts are introduced here. The purpose of Cycle 2a was to address more carefullythe requirements of the discipline for formal education, in effect to decode it into educationalterminology.Aligning learning with practitioner needsThe first approach taken to adapting unit content was focussed on addressing elements identifiedby practitioners as being of prime importance, and at levels of Bloom’s taxonomy thatreflected the higher order learning required by the nature of RE.Industry is seen to require a broad perspective on RE from formal education. While thebase case of RE content knowledge assumed by practitioners is covered, in general terms, bymodels used in university units, industry requires a greater focus on ‘soft’ skills, organisationalknowledge and flexibility. The implications of this include initiative, ability to deal withcomplexity and ill-structure and organisational (self, task and information) skills (Armaregoand Minor, 2005).As discussed in Chapter3, Thomas et al (2002) suggest there is a widening gap between thedegree of flexibility and creativity needed to adapt to a changing world and the capacity todo so. These difficulties are attributed to:• structuring failure – failing to spend sufficient time in the early stages of design: problemfinding and problem formulation, and bring critical judgment into play too early. Inaddition, path-dependent behaviour is exhibited, implying an unwillingness to undo aprevious action even if that step is actually necessary for a solution (Thomas et al,1977)• unwillingness to apply implicit knowledge and• the appropriate level, type, and directionality of motivation on a problem.Although the Apprenticeship model had addressed some components of industry needs, elementsof an ‘incorrect’ learning environment were identified in the evaluation of that cycle.Patel et al (2000) argue that learners in an ill-fitting (and generally traditional) setting focuson skills that will yield higher grades as an immediate objective. With the relevance of284

domain knowledge not fully understood, cognitive skills related to exam techniques acquireimportance though they do not model real life situations. The learning, in many cases, isreduced to assignment hopping with ‘just-in-time’ and ‘just-enough’ learning to fulfill the assessmenttasks. These are also characteristics of surface learning. The Apprenticeship cycleexhibited some of these traits – students focussed on learning the tools and techniques of RErather on either higher or softer skills.Aligning domain needs with learningThe nature of RE described previously (opportunistic, exploratory, creative, emergent (Bubenko,1995; Guindon, 1989; Maiden and Gizikis, 2001; Nguyen and Swatman, 2000a)) implies a needto enable students to not only learn to use past experience on a general level, but to also beable to deal with each new problem situation in its own terms. Gott et al (1993) posit thatthis adaptive/generative capability suggests the performer not only knows the proceduralsteps for problem solving (ie, applying knowledge) but understands when to deploy themand why they work. The implication of this is effort spent on higher (metacognitive) learningskills, including abstraction and reflection.Requirements Engineering has also been described as wicked (see Chapter 2). This impliesa need to:• incorporate creativity-enhancing activities within the curriculum• foster adaptability in students by providing for divergent as well as convergent thinking• focus on metacognitive strategies and reflection as an aid to transfer of the skills andknowledge learnt.For a curriculum to address all these components, a foundation in the content needs to bebalanced with elements of creativity and experience based on practice.Albert (1996) notes that schooling at the age of starting formal education emphasises logicalrather than divergent thinking, with the value of conventional behaviour, well-defined problemsand good grades emphasised. In addition, many cultures (here we may say disciplinebasedas well as social) encourage respect for the past and discourage disruptive innovations.Promoting widespread creativity raises expectations that may change employment patterns,educational systems and community norms.The three components of Amabile (1983)’s general theory of creativity (described in Chapter3) – domain relevant skills; creativity-relevant processes and intrinsic task motivation – are285

necessary for the enhancement of creative potential. These components had a critical focusin the learning environment we developed.Investigating alignmentIt became apparent that the issues highlighted as either practitioner or domain needs offormal education could be best addressed through approaches to learning, which focus onadvanced knowledge acquisition (Spiro et al, 1991). A framework for RE education shouldexploit the learning models that provide an appropriate environment for RE practice. Toachieve this, it should be based on constructivist theory with a focus on strategic knowledge;be placed within a situated experiential environment where authentic context is exploited;provide the student with exposure to creative enhancing activities.As noted previously, authentic or situated learning attempts to place the learner within aframework that models the physical and social context of the domain. According to Brownet al (1989), conceptual knowledge can be considered a set of tools: fully understood onlythrough use as practitioners use them. Learning is seen as participating in communitiesof practice (Sfard, 1998), with authentic learning not offered as an aid to content learning,but as a superior substitute for it. Stepien et al (1993) states that authentic learning isproblem-based learning which turns instruction topsy-turvy.Focussing on the solution of authentic problems as a context for learning also accords wellwith theories of expertise – learning beyond the initial stages may best be achieved throughsituational case studies with rich contextual information (Dreyfus and Dreyfus, 1986).Its supporters claim PBL results in increased motivation for learning, better integrationof knowledge across disciplines and greater commitment to continued professional learning(Boud, 1985). As well as offering the flexibility to cater for a variety of learning styles, thefocus moves from dealing with content and information in abstract ways to using informationin ways that reflect how learners might use it in real life (Oliver and McLoughlin, 1999).As noted in Chapter 5, these characteristics strongly suggest that PBL has application inthe solving of wicked problems in wicked domains:• learning based around constructivist principles is likely to be more suitable in domainsinvolving ill-structured problems (Spiro et al, 1991). These principles are encapsulatedalmost ideally in problem-based learning (Savery and Duffy, 1995)• appropriate learning in ill-structured domains and/or dealing with ill-structured problemsshould itself be problem-based286

• problem-based learning best provides an effective environment for future professionalswho need to access knowledge across a range of disciplines (Boud, 1985).More recently, Jonassen (2002) suggests two critical properties of PBL relevant to RE:• its problem solving requires the mental representation of problematic situations – theproblem space (Newell and Simon, 1972) must be constructed, either individually or(of more relevance in RE) socially through negotiation• active manipulation of the problem space is required for PBL problem solving. Thisinvolved information gathering, model building, hypothesis generation, speculation, solutiontesting, among others. This engages conscious activity, and in successful problemsolvers, leads to more systematic manipulation of the problem space. The ‘problematic’nature of the situation enables learners to accommodate when their current experiencecannot be assimilated into the existing mental models they possess.This learning model has been applied to numerous areas of higher education (social work,architecture, law and engineering), although it is best documented with medicine. For REeducation, a PBL environment could provides opportunity for students to value their ownperspectives in the learning process and to argue their interpretations of the problem and itssolutions. However, this requires the teacher to take on the role of facilitator – the teacher’spedagogical stance influences the way PBL is enacted in practice (Savin-Baden, 2000).Mapping the curriculumThe outcome of this initial phase was to confirm the need to build into the curriculum a focuson generic and soft skills as part of the outcomes of the unit, to address both practitioner anddiscipline needs. To maximise effectiveness, these needed to be embedded into the knowledgebase constructed by the student during the unit. This has the advantage of enabling studentsto develop the requisite skills situated within the learning context but, of course, potentiallyrequired extensive adaptation of the existing learning environment.As an educational strategy PBL requires three components to be differentiated (Walton andMathews, 1989):• an integrated curriculum organised around realworld problems rather than disciplinesand with an emphasis on cognitive skills• small groups, tutorial instruction and active learning conditions to facilitate problembasedlearning287

• outcomes such as the development of skills and motivation together with the developmentof an ability to be lifelong learners.See Appendix A for a discussion of Topic numberingFigure 7.5: Excerpt of RE content categorised for mappingIn order to facilitate all the alignments required, a curriculum map was developed with thetopics in ENG260 categorised firstly by the broad area of RE syllabus (see Figure 7.5) andthen by the learning outcome to be addressed. Curriculum mapping, as an evaluative toolattributed to English (1978), has been used primarily in schools, with limited use in highereducation. English (1978) advocated the use of mapping to ensure that the declared aims ofa curriculum match those which are taught and learned, while Biggs (1999) includes mappingof assessment in order to achieve the alignment of declaration, delivery and learning as wellas assessment of individual skills.This map was based on scrutiny of documentation related to the unit, in particular syllabusand unit outline information provided to students at the commencement of the semester.These detail topics to be covered, assessment elements and criteria and expected demonstrableoutcomes of these. Because the researcher had been instrumental in the development of theunit originally, a rich chronology of development decisions was also available for analysis.This documentation (which included minutes of meetings during the development process,jottings on topics, their importance and expected learning outcomes and reflection on teachingphilosophy) were an important source of historical decision-making information.The data gleaned from all of these were initially mapped to the generic graduate outcomes of288

Murdoch University, and then, as progress was made in developing the problems to addressthe learning outcomes identified, to these as well. Figure 7.6 shows the fine grained mapping –from Learning Objectives, to RE topic (indicated as Domain) to Murdoch Graduate Attributesand finally to Problem – necessary for alignment.The curriculum map provided a mechanisms for communications between the researcher (asthe discipline expert and curriculum designer) and the mentor (as an expert in pedagogy) toalign their conceptual model of the unit: its intended outcomes could be stated and evaluatedmore clearly. The map also highlighted any discrepancies: that an element is declared as alearning outcome addressed by problem(s) and had support materials identified.It was also anticipated that, as curriculum maps were developed in the future for subsequentSE units (at least those co-ordinated by the researcher), the place of the problems developedfor ENG260 in the longitudinal skill development process could be shown. This would be anasset in the professional (re)accreditation process.One of the most difficult aspects of PBL is the development of the problem (Boud and Feletti,2001; Duch et al, 2001). The curriculum map was expected to be useful at that stage: theextent to which problems were aligned with declared objectives and assessment could bedemonstrated.Biggs (1999) argues that student appreciation that assessment aligns with learning opportunitiesis essential in order to prevent them working with the aim of passing assessment ratherthan with the aim of achieving intended learning outcomes. The curriculum map thereforealso addressed some of the elements of incorrect learning identified in Cycle 1. However,this is a bit of an issue in PBL environments, where, in theory, students determine theirown learning objectives, rather than having them provided. A compromise was chosen forENG260 – the learning objectives were available as part of the support material and couldbe ‘found’ by exploring the environment, but were not ‘given’.The final outcome of the mapping process, and the intense reflection on teaching philosophyit required, was a strong acknowledgement that a focus on creative strategies for dealingwith ill-structured problems was at least as important as technical discipline content. Whileacquiring particular domain knowledge remains one of the unit objectives, adaptiveness ingeneralising knowledge in order to enhance productive thinking as a basis for insight andtrue novelty of thinking is equally important. This became the focus of the next stage of theAction Research process.289

Figure 7.6: ENG260 learning objectives mapped to topics, graduate attributes and problems290

7.2.2 Developing the learning environmentThe prime motivation in changing the learning environment was to address the issues identifiedpreviously as an ‘ill-fit’ as early as feasible within the SE student’s programme of study,and to challenge the expectations students had about teaching, through less traditional approachesto learning.The CreativePBL model was developed to address the characteristics of RE as a domain, andto provide a learning environment that enhances the opportunity for creative and divergentthinking. There is evidence that students who have been taught to explore different waysto define problems (the prime objective of RE) engage in more creative problem solvingover the longer term (Baer, 1988), addressing flexibility and adaptability issues raised bypractitioners.Applying a PBL methodologyAs previously noted (see Section 3.5.3) one taxonomy (Barrows and Tamblyn, 1980) proposesseveral varieties of PBL in use. These describe a continuum, from lecture-based cases toclosed-loop problem-based where an evaluation of the reasoning used based on the resourcesutilised is incorporated. The type of scenarios offered, the assessment methods, learnerautonomy and the way in which teaching and learning occur determine which variation ismost appropriate for any given environment. The PBL process advocated by Koschmannet al (1994) is used in this research to anchor the student within the learning environment,and provides the discipline to assist in content learning (Gardner, 1999).Learners meet an ill-structured problem before they receive any instruction. They have toformulate the problem, determine what information they need and what will constitute asolution. Teachers ask questions such as ‘what’s going on here?’ and ‘what do we need toknow?’. As students become self-motivated learners, teachers fade into the background andbecome colleagues.Embedding creativity-enhancing activitiesThe PBL process also provides an appropriate environment for the development of creativepotential. Table 7.4 lists some of these.Divergent thinking requires motivational states such as willingness to take risks, tolerance ofambiguity, and the courage of one’s own convictions, as well as personal properties such asopenness, flexibility and nonconformity. Amabile’s positive influences for creativity, therefore,also enhance divergent thinking.291

Table 7.4: Positive influences for enhancing creative potential (Amabile, 1996)encouraging assertion of ideasno reliance on order and trainingno fear of failureproviding time and resourcesdeveloping expertisegiving positive, constructive feedback that is work or task focussedencouraging a spirit of play and experimentationproviding a mix of styles and backgrounds with opportunities for group interactionmaking a safe place for risk taking allowing free choice in task engagementoffering rewards that recognise achievements or enable additional performancebut maintain intrinsic motivation rather than controlling behaviourEdmonds and Candy (2002) describe elements of creativity (see Table 7.5). The congruencebetween these and the PBL stages of Koschmann et al (1994) enabled creative activities tobe embedded into the PBL process.Figure 7.7: A process model for CreativePBLWithin the CreativePBL framework (see Figure 7.7), the focus is firmly on examining theproblem at length rather than quickly solving it. This is ‘holistic’ in the sense that:• an overall broad scope must be established• candidate solutions can be proposed early in order to better examine the problem292

Table 7.5: Creativity activities (Edmonds and Candy, 2002)exploration of ideas, knowledge, and options, based on◦ breaking with conventional expectations, whether visual, structural, or conceptual,is a key characteristic of creative thought◦ immersion – the complexity of the creative process is served well by total immersionin the activity◦ holistic view – the full scope of a design problem is only fully embraced bytaking a holistic, or systems, view. The designer needs to be able to take anoverview position at any point and, in particular, to find multiple viewpointsof the data or emerging design is important◦ parallel channels – keeping a number of different approaches and viewpointsactive at the same time is a necessary part of generating new ideas.Exploration involves accessing source data that may be examined, assessed and interpretedin terms of the goals. This is an open process, possibly without observabledirections, but the thoroughness and selectivity of the activity is critical. Having acomprehensive set of knowledge sources readily available is extremely advantageous.Knowing where to look and how to select the knowledge is even more important.idea generation - problem formulation, as distinct from problem solving, is critical tothe effectiveness of the solution space that is defined. It draws upon a wide range ofanalogous cases often outside the immediate domain. This has been characterised asan ability to make remote associations. Creativity is demonstrated by the generationof many potential solutions instead of gravitating quickly toward a single and (usually)familiar solution that is not necessarily the optimal one. The ability to considerparallel lines of thought and to select and transform the results to meet the demandsof a different situation is a critically important aspect of solution generationevaluation involves taking the results of the generative activity and testing the candidatesolutions against a set of constraints. This leads to modifying, reformulating,or discarding solutions depending on the feedback. Selection of the optimal solutionmay involve a number of trade-offs against the constraints that are applied especiallywhere, as is usually the case, the product is a complex one. The applicationof tight constraints may be considered conducive to creative solution finding andthus evaluation is a vital part of the creative process. Evaluation may be viewedas a pervasive activity that takes place from the exploration phase onward. Theuse of expert knowledge in evaluation has been identified as an important aspect ofsuccessful solution finding293

• constraints are discovered that help generate new concepts (and can reduce the numberof solutions)before moving into specific solution detail. Boden (1997) makes a case for the claim thatchanging the goals and adding constraints during the design process might be at the core ofcreative thinking.Since this CreativePBL model was developed to focus on creativity and divergent thinking,instead of students aimed at finding the single, best, ‘correct’ answer to a standard problemin the shortest time (convergent thinking) they aimed at redefining or discovering problemsand solving them by means of branching out, making unexpected associations, applying theknown in unusual ways, or seeing unexpected implications.This approach also had the value of addressing issues identified by Thomas et al (2002) anddescribed in Section 7.2.1:• problem analysis is a critical stage: starting from the unknown and progressing to adescription of the problem itself, and the knowledge needed to deal with it. Problemsolvinghabit is challenged by the need to generate alternate solution paths. In learningRE this problem analysis is a critical outcome• the value of alternative perspectives is fostered through participation a collaborativeenvironment and the active promotion of critical friendship. Critical appraisal andself appraisal skills are developed through the use of reflection tools such as the 4SAT(Zimitat and Alexander, 1999)• although external motivation is difficult to eliminate within an undergraduate degree,PBL is seen to foster intrinsic motivation through the authenticity of the tasks undertaken(Wilson and Cole, 1996). Emphasis is placed on constructing a framework inwhich details of the problem are situated.This approach also aligns with the (revised) catastrophe-cycle RE process (Raisey et al, 2006)discussed in Chapter 2.Establishing a problem contextThe development of the problem and its context was a considerable challenge: it had tobe suitably complex and open-ended in order to instantiate the constructivist principles;it had to exhibit ill-structure so that students would engage with complexity, and presentclaims and rationale to negotiate understanding with their peers; it had to be too ‘large’294

for individual students to complete successfully on their own; it had to address the learningoutcomes identified through the curriculum mapping, and provide a mechanism for studentsto demonstrate their competency in achieving these.This latter point is relevant in light of the importance placed on a creativity-enhancingenvironment. Cowdroy and de Graaff (2005) suggest that the focus of assessment of highlycreativeability is forced onto the end-product (the painting, the performance), rather thanthe creative process behind the product (the concept, the schemata) when the originatingconcepts are not revealed. Pertinent to this discussion, they differentiate between severaltraditions of formal education for creativity, with the traditional apprenticeship model ofvocational education being categorised as oriented towards ‘followship’ while a ‘studio apprenticeship’tradition is oriented towards passing on the abilities and values of the masterand is comparable with Cognitive Apprenticeship models in higher education. The point tobe made is that both these focus on schematisation and execution, whereas the highest levelof creative ability is conceptualisation. They concludeif this is neither taught nor assessed, then it must be accepted that creative abilityas a whole is neither taught nor assessed.(Cowdroy and de Graaff, 2005, p 511)If this is true of the creative arts, how much more difficult to teach and assess these withinan engineering ethos?Therefore it was important that students were able, through the assessment activities, todemonstrate not only their conceptual understanding of the discipline, but also the creativeprocess undertaken (the conceptualisation of the product). Learners were expectedto construct a problem representation based on the triggers provided, and manipulate theproblem space so that an external representation (in this case the artefacts that were deliverables)could be created from the multiple individual representations at a fine enough grainto demonstrate group and individual achievement of the appropriate learning outcomes.As well as providing a dedicated ‘office’ space – a laboratory specifically designated for useby SE students in third and fourth year, a collaborative on line work space was enabled foreach group. Only group members and the teacher had access to this area – with studentsrequired to maintain all documentation on the site. This resource facilitated constructionof the shared understanding necessary for collaborative learning, and enabled students totune the accuracy and suitability of individual understanding through disentangling cognitiveconflicts. It also introduced technical issues of version control and change management,also authentic in the domain. From the researcher’s perspective, these versions document the295

changing shared understanding within the group. In addition, the representation tools provided(mindmapping, models through the CASE tool Rational Rose from IBM Corporation(2006), etc) act as mediators for collaborative learning byproviding learners with the means to articulate their emerging knowledge in a persistentmedium, inspectable by all participants, where the knowledge then becomespart of the shared context.(Suthers, 2000, p 2)By this means, the accuracy or suitability of (individual) personal understanding is tunedthrough the group-oriented collaborative learning process (Stahl, 1999).A roleplay-simulation environment was developed to enhance the opportunity for collaborativeexperiential learning, in so far as the simulation acts as the context and structure withinwhich the roleplay occurs. Roleplays involve participants deliberately adopting a role fora specific purpose and simulations are simplifications of reality that maintain the essentialfunctions of the simulated environment. McLaughlan and Kirkpatrick (2004)’s analysis ofstudent performance in such an environment suggests it supports student learning aboutalternative perspectives on problems and encourages transfer of learning to new contexts.The environment is set up to provide students with an opportunity to deal with a complexprofessional problem by ‘living’ it, with the use of the Internet providing a richly developedcontext for the roleplaying in a blended mode (where online interaction and resourceprovision are supported by face-to-face activities). As McLaughlan and Kirkpatrick (2004)note, Internet-mediated roleplay-simulations can be designed to maintain effectively the interactionnecessary for individuals and groups to work in a way that is truly collaborativerather than simply supporting distributed individual effort. Although Internet-mediatedroleplay-simulations more commonly support learning in the social sciences, McLaughlanand Kirkpatrick (1999) has reported the success of this approach in Engineering.The scenario developed for the CreativePBL environment focuses on the secondment of theclass to a (virtual) organisation – collaboration between a software house and the university.MurSoft requires a team to work, on short-term placement, on a project to develop gamingsoftware to be used as an educational resource within a tertiary institute. This provides anauthentic context for learning: students will have an opportunity, within their final year ofstudy, to undertake an internship with a software-based organisation.In order for the additional time required to engage with the material in PBL mode (Albaneseand Mitchell, 1993), and to address the issue of loss of motivation in the second cycle of thespiral (some students in previous years had indicated they were bored by the spiral296

Figure 7.8: Learning environment interaction (adapted from an idea by McLaughlan and Kirkpatrick(2004))297

Figure 7.9: Setting the scene MurSoftFigure 7.10: Setting the scene TerColl298

approach), the content was no longer spiralled. This meant that students worked throughthe discipline content in one pass, although there were many opportunities throughout theproblem to revisit understanding of any component – in particular, all resources were availablethroughout the unit (and in fact, throughout any subsequent unit in SE).All interaction with the client is undertaken through web-based material: a fictitious character,the Team Manager acts as go-between for the team and the client, while memos, minutesof meetings, telephone messages, ‘talking heads’, press releases etc provide the problem triggersrequired. These act as prompts to students to undertake some task identified in the PBLproblem. Support material is provided, as are expert consultants as required. The ‘lecturer’is always available as a resource. The interaction between stakeholders in this scenario is indicatedin Figure 7.8. Students are expected to undertake the learning tasks collaborativelyby interacting with the learning environment.The decision was taken to minimise technical complexity (eg in terms of advanced interactivemedia and ‘appearance’) so that students could easily access the resource off-site and withminimal software requirements (a Sun Microsystems (2006) Java runtime environment andAdobe Systems Inc. (2006) Flash reader). Pragmatically, technological capacity could neverkeep up with student expectations (in general males in their late teens, very au fait withgaming environments). Rather, ‘cognitive realism’ (Herrington et al, 2003) to the real-life task(as opposed to a technological-driven view) was seen as having (much) greater significance.Figures 7.9 and 7.10 are samples of the support material provided within the environment.Unit content is centred on the online teaching material (now viewed as a resource repositoryin the problem environment) within the Software Factory previously described, and a recommendedtext. These act as a constraint: students initially explore this material in orderto achieve the learning outcomes they have identified in a problem component, rather thanhaving unlimited access to resources on the Internet and elsewhere. This is a significant issue:RE is a relatively new discipline, with varying approaches taken in its description. It isimportant at this early concept-learning stage that students are not confused or frustrated bythe presentation of too many alternate viewpoints, tools, definitions for the same concept etc.This is likely to occur if students are to explore freely during the self-directed learning stage ofthe PBL process. On the other hand, it is important that students become aware that otherviews exist. Again, providing environment constraints adds to the authentic approach: asgraduates, students will be expected to follow the operating procedures standardised withinthe employing organisation, rather than having the freedom to choose those which suit themindividually.The development of each problem required:299

• identification of the learning objectives from the curriculum mapping to be specificallyaddressed• a cross check that domain content could be addressed by the problem (and that appropriatecontent was accessible within the environment)• identification of the graduate attributes addressed by this problem• trigger identification and subsequent construction.In addition, a trigger sequence (see Figure 7.12) was determined, as triggers were coded in anxml document and released (the memos etc previously mentioned) automatically. The webmaterial was expanded to include resources on the PBL methodology. These included supportdocuments for students on the PBL process (see Figure 7.11), self-assessment items basedon the 4SAT instrument (Zimitat and Alexander, 1999) (see Figure 7.13), trigger checklistsand the like, and were adapted from work undertaken by the academic mentor in a previousPBL environment (Searle and Clarke, 2000).Figure 7.11: Setting the scene PBL300

Figure 7.12: Part of the trigger sequence for the games environment301

Figure 7.13: Self assessment support (adapted from (Zimitat and Alexander, 1999)As learning has moved from a teacher to a student-centred focus, Boud and Falchikov (2005)suggests a shift away from assessment’s purely summative role, to one that aids in learning.Rowntree (1987) argued that it should help prepare students for life, and that modifyingthe assessment system can influence students’ approaches to studying and motivation. Thecomponents of the group work can be considered to take on this role – the assignment isintegrated into the learning process so that what is assessed is the total learning experience.As this is seen to be assessing process rather than product, it should encourage a deepapproach to learning (Trotter, 2006).However, in an environment where collaboration and group effort is emphasised, the issue ofindividual achievement is problematic. Murdoch’s assessment policy (Murdoch, 2004a) doesnot, at this stage, acknowledge the special needs of learning models such as PBL. Therefore,it was important that formal assessment components adhered to the requirements in termsof supervised/unsupervised and individual/group, especially in terms of what proportion ofeach could be so designated.As noted previously, the administrative cycle for major changes to unit offerings could notbe accommodated in the development of the CreativePBL environment, especially since thedecision to ‘run’ as PBL would be determined by robust availability of the technical components,and could be delayed into (the very commencement of) semester. Therefore the exam302

as summative assessment was retained, and administered in a form as closely matching thatof previous years as possible. In order to achieve some reasonable comparison, the formatwas identical, and components chosen from a database of questions as for previous papers.In terms of formal assessment items, the unit objectives were assessed over three groupassignments, a final exam and individual Portfolios and Performance Reviews. The groupassignments (worth 10% of the final mark each) modelled individual elements of the groupassignment of the Apprenticeship model. As assessable artefacts, the components of thegroup work defined the completion of one ‘phase’ of the overall problem – developing aRequirements Specification. Each could be considered a problem in its own right (eg problemone: developing a Requirements Model (scope, Context and Use Case Diagram with attendantdocumentation); problem two: developing an Object Model (Class Diagram) etc), problemthree: developing the Behavioural model, all for the same, increasingly complex scenario.Each problem required use of the products of the preceding. In this way, students were ableto activate (recent) prior knowledge, refine their understanding of the problem context andreflect on the implications of the feedback from their manager on the success of that priortask in order to commence the PBL process for the next.To facilitate this process, a small component of the mark was allocated to ‘rework’ addressingissues raised in the feedback and amending/updating documentation as required. It shouldbe noted that not all groups chose to undertake the rework – and paid the penalty whentheir increased understanding of the problem context was not well reflected in their submissions.In relation the the revised catastrophe model previously discussed (Raisey et al,2006), this meant that incidental complexity was rarely reduced through simplification basedon increased understanding of the problem. Although together, these encompassed all theelements of the Requirements Specification set in previous years, what was added was anindication of any re-conceptualisation undertaken by the students after feedback or whentackling a subsequent problem. These re-conceptualisations were to be submitted as well, tobe taken into account when assessing student learning.The portfolio (worth 10%) also modelled that of the previous year, although expected inclusionswere more explicitly stated (ie as had been negotiated during 2002). The final exam(worth 5% less, at 55% of final mark) also modelled that of the previous year, both forthe same reasons (ie to maintain some level of independence from the learning model beingapplied) and pragmatically due to the assessment policies of the School and the University.The one new assessment element was the Performance Review. This facilitated feedback thatcould be matched to a specific student (in addition to anonymous feedback captured throughmore formal channels). The Performance Review (worth 5%) is described in greater detail303

in the context of implementing CreativePBL.7.2.3 What actually happenedThe timetabling of the classes did not change from 2002 – sessions of two hours durationtwice a week (Tuesday morning and Thursday afternoon, allowing students time for theself-directed learning required by the process in between).In order to ensure the teams had the opportunity to integrate, the students were initiallyprovided with a very small problem to define. This problem introduced them to the MurSoftenvironment, and also served the purpose of introducing PBL, making it explicit and therefore‘open’. Web-based support material was indicated, and the class as a whole worked throughthe problem with the teacher, visiting each phase of the PBL process (how the phase was‘entered’, what was to be achieved, how it could be tackled, what needed to be producedat the end of it). Students are also given some little time to familiarise themselves withaspects on group dynamics and teamwork (since the rest of semester was to be spent oncollaborative tasks), and with the lecturer, who takes on the role of academic consultant(not the client, but a resource students have access to), providing scaffolding and othersupport for the completion of the problems. This process occupied the first two workshopsessions. Most students completed this task relatively easily – in effect what was offered wasa mini-apprenticeship, with the process modelled and discussed. However, unlike previousyears, some students had not been aware that the unit was ‘different’ – in particular thearticulation students were confronting their first semester at the University with all theirother units taught traditionally, and they were unprepared for ENG206.Session three to eight were not as smooth as the first sessions had indicated they mightbe. Once students had to grapple with the PBL mode, they floundered (don’t know whatis going on; hard to follow 1 ). Each phase of the PBL process caused them problems. Thefirst issue, encountered during session three, was to try to identify the learning objectivesin the problem (what do I need to know?). Students did not know what to look for. Toease this problem, keywords in the triggers were highlighted, so for example a memo thatstated it would be appropriate for your people to prepare a project scope for us might highlight‘project scope’. Students could use the web topics and textbook to gain an understanding of a‘scope’ (phase 2 of the PBL process). However, even these resources caused problems (Week4 comments included on-line material is too confusing; on-line material not uniform/notorganised; website is a bit messy; webpage needs to be streamlined), although no changeshad been made to those pages. Some students did manage to navigate the resources by1 Student comments (in italics) drawn from Year surveys304

Week 4 (all unit information available; well explained and resourced). Later in the semesterstudents were more comfortable with the resources (prescribed text relevant to course andeasy to understand; good web-based material), suggesting the issue was as much the learningenvironment as a whole at the commencement of semester as the resources themselves.Each member of a team was required to ‘research’ what had been identified as the learningelements required, return to the team and discuss their findings in order to re-examine theproblem (phase 3). Students expected the solutions to be in the textbook (more reference toprescribed text course content) – not assembled from the various sources available to them.Through group discussion and negotiation (phase 4) (inability to work alone was an issue), asolution is proposed and documented. Because of the requirements of the University assessmentpolicy these were quite explicitly spelt out. However, some students still had problems(very vague on assessment and what specifically needs to be completed; assessment are vague).Finally a debriefing (phase 5) allows for reflection on the learning process (boring unit).The PBL literature had indicated students required about four weeks to engage with thelearning environment. Despite negative comments about the learning model (don’t really likehow it’s structured; does not have proper course structure; no lecture or tutorial (all fromthe Week 4 Year survey)) some students acknowledged its value (makes you think; helpswith thinking about all areas of a problem (good for other units)), and considered the unitwell structured; interesting; practical; well presented; good for software industry; it’s reallygood. However, the lecturer was required to address student perceptions at a staff meetingreviewing the Year surveys.A greater understanding of teacher characteristics was achieved through applying additional‘Teaching styles’ instruments between Cycle 1 and Cycle 2 of the project (see Section 7.1.2).This had a profound impact on how the teacher ‘behaved’ within the class. The teacher’spersonal characteristics, in particular a directive tendency, was in conflict with the strategyto be applied in this cycle – in fact, a much clearer alignment can be seen between themaster/apprentice model of learning and the teacher’s personal characteristics as ‘Teacher’(see Chapter 6). Therefore, in choosing to abandon this learning model, clearly, changes werealso required to the teacher’s behaviour.This was a difficulty within the classroom, for reasons cited in the PBL literature (eg Bridges(1992)) – academic staff feel a need to intervene early when students seem to be going ‘offtrack’.To mitigate this effect, the approach decided on for ENG260 was that the teacher wouldalways be available within the class environment, but engaged in other tasks. This meantthat, to a large extent, students were required to approach her, rather than being in an305

environment where the teacher was continually looking over their shoulders. In the first fewweeks of the semester, many students found this a problem – either for cultural or personalityreasons, they were loathe to interrupt her ‘work’. Therefore a transitional approach wasapplied: while the teacher was never ‘not engaged in some activity’, she would stop, visiteach group of students (some groups chose rooms other than the classroom to work in),providing opportunity for questions and discussion, then withdraw. Gradually the length oftime between ‘visits’ was extended, but with students reminded to come and discuss issuesas often as necessary. This did not seem to cause any problems (easy to get help for unit)– most students were happy to send emails or make appointments outside of class if theirproblems had not be well covered within the class environment 2 .Subject: Re: g260 Req. Eng. classDate: Wed, 30 Apr 2003 15:06:19 +0800From: [Rey]To: "Jocelyn Armarego" Hi Jocelyn [...] and do you want all attrubutes/methods includedand if so do you want them on the class diagram or can we keepthem together with the class descriptions so that the classdiagram is more readable when printed.and will you be on rockingham campus anytime thursday, as mygroup are meeting again from 11am to 5pm. thats about all,Date: Sat, 3 May 2003 12:36:56 +0800From: [Rey]To: "Jocelyn Armarego" Hi,i hope you received my last email and can reply asap, also i’mwondering about our class diagram’s classes. All classes on ourclass diagram are persistant classes, (26 total), are theresupposed to be transient classes on our diagram? if so, itbecomes a sought of grey area were analysis ends and designbegins. [...] From the point of view of the notes we seem to be2 Throughout this thesis all students are given aliases306

going ok, but i can’t really get a clear idea from anyone of howmuch the class diagram encompasses before we’ve goneunnecessarily far. Our class diagram is going to be quiteexplanatory/well documented, we just need an idea of minimumrequirements of the "class diagram structure".Other issues raised in the literature include student discomfort with the extensive collaborationrequired or with the lack of teacher-direction given. The email below provides anexample of student desire for the unit to run ‘normally’.Date: Sat, 1 Mar 2003 16:39:20 +0800From: [Ralph]To: Jocelyn Hi Jocelyn, [...] And also can we have a week by week studyschedule presented at the class and explain what you expect andthe how the content of online teaching materials related to thetopic week by week.Thank you confused student.As a compromise solution to this issue, a problem-by-problem breakdown to the content wasprovided on the website.Most students settled quickly in the group environment – except for the one dissenting commentabove, students indicated they liked it (lots of group project; encourages group work).They also demonstrated their liking for the ‘freedom’ of the classes (free form structure; learnwhat you like at your own pace). Unlike the Apprenticeship model, where the workshop sessionhad tended to focus on class discussion, here students were free to address whicheverphase of the PBL process they had reached, in relation to the problems being tackled –which were the assessment items. Students indicated they liked that aspect as well (focus onassignments; hands-on assignments) as it modelled the ‘real world’ (more practical trainingand real time examples; probably useful). As noted, some groups found a different room towork in, and would call on the lecturer as required, or wait for her to do the rounds. In termsof location, the only constraint was that they had to be near-by.As students found their own working style, workshop sessions took on a format different tothat of previous years. The session would start with the whole class together – this wouldallow any ‘whole-of-class’ matters to be addressed. Groups would then go off and work,307

and return to the classroom for the final fifteen minutes of the session. At that time theywere required to complete the process documentation for that session (this included reflectionon their effort during the session). For one group in particular, the environment providedtoo much freedom - they inevitably spend the class session discussion other matters (eithersocial or related to other units). However, progress with the problem was always made – justnot during class time. Although this was a topic of discussion with them, there seemed noreason to change their behaviour: the group appeared to be achieving the learning outcomes.For some of members of this group, however, the raw exam mark did suggest they had notexploited the learning opportunities provided in the environment (two of the three membersrequired supplementary assessment in order to achieve a pass).Throughout the semester, the Academic Support Officer was available as mentor and consultant.Any issues that seemed to be attributable to applying PBL were referred to her.In particular, the concerns raised over the first few weeks of semester, when students didnot seem to be comfortable with the process, were discussed at length. This implies thatfine-grained adjustments were constantly made to the learning environment – triggers werebrought forward or delayed as students progressed through the material, components of thePBL process were reiterated if any group seemed ‘lost’, students were reminded of supportmaterial (all available online) if they seemed to need it.7.2.4 Interpreting what happenedAs for Cycle 1, the purpose of the evaluation conducted after the 2003 semester was to assistin determining if the implementation was effective in facilitating student learning, and toprovide the researcher with data to reflect on improvements to be made. Table 7.6 providesa summary of the instruments used to collect data for analysis in this cycle.Table 7.6: Instruments applied to Cycle 2Cycle Diagnostic Devices Records FeedbackLearningStylesTeachingStylesCycle 2 LSI TSI Assessment items Year Survey& results2003ILSPerformanceStudent records ReviewPortfolio (mindmaps) Unit development SSUrecordsASIThe data collection strategy reflected a wish to apply triangulation techniques, as quantitative308

esponses could support (or otherwise) responses to multiple qualitative instruments. Therichness of the data collected continues to provide insight into student perceptions and triggersfurther refinement in the learning environment offered.Success of the learning modelAs has been noted, strategies were put in place to try to ensure the PBL model was beingapplied reasonably well. Students were provided with a variety of PBL-based resources (suchas a guide for students and web-links to other material). The teacher attempted to facilitaterather than direct or coach (again resources were available for the teacher’s perspective, drawnfrom other work undertaken by the Academic mentor (Searle and Clarke (2000)), with thementor on hand when issues arose.Student motivation seemed to be high, once the initial transition to PBL had been made. Asan example, student willingness to complete the MCQ tests was higher than noted in previousyears – issues with the tests themselves were brought up during this semester (though theyprobably existed before then):Subject: Online Questionaire Bug ReportDate: Fri, 6 Jun 2003 03:00:46 +0800From: [Rudy]To: "Jocelyn Armarego" RM030 based on Brown Chapt 16 there seems to be some errors inthe online questionnaire concerning - Question 37 - possibleincorrect answer???? Question 28 - all selections give anincorrect Red X [no correct answer possible]Subject: Rm 210 Online test week 11Date: Sat, 31 May 2003 17:49:54 +0800From: [Clayton]To: Jocelyn Armarego Hi JocelynJust a request about the online test it only allows 10 minutesto complete 63 questions can this be modified on the webpage toa more realistic time, as You don’t get the chance to complete309

all the questions on the site (RM210)The portfolio mark also acts as an indicator of motivation. In 2003 this showed that just over65% of the students expended effort to pass this component (up from 50% in 2002). However,a better indicator was that 41% of students (up from 12% in 2002) went well beyond therequirements of the material presented.Subject: G260 Mind MapsDate: Sun, 8 Jun 2003 13:04:03 +0800From: [Pete]To: Jocelyn Armarego Hello JocelynYou probabily will have noticed the absence of myportfolio, although it is 6 days late I will still hand it in onthe Monday before the exam expecting full penalties. I have noexcuse but my other subjects had taken over all my time andbecause G260 is not required for Instrumentation and Control Ilet it slide. At the start of the year I was not sure whichdegree I wanted to undertake between Industrial Computers andICE, and it was in the last 3 weeks where I have made a decisionand choosen ICE.Subject: Requirements Engineering PortfolioDate: Tue, 3 Jun 2003 02:12:19 +0800From: [Clayton]To: Jocelyn Dear Jocelyn I am writing to you making a request that if allpossible, I could hand in the portfolio by 5pm at South Streetinternal mail at the DSE office on due date Tuesday. [...]although I have been quite ill, I have still been doing as muchwork as I can at home on the portfolio, I have completed all theweekly online weekly exercises even tough they were not allcompulsary as well as have completed the reading log. I am up toweek 12 on mind maps and have two more to do. [...] I hope thisis ok at your end.310

On another topic will you be in during class time on Thursday todiscuss revision questions for the exam. Thank you for your timeand understanding in this matter, and sorry for any troublesthis may cause.This year, however, some students considered the portfolio component as useless work (toomassive workload with mind map and summaries; there is not a lot of work to submit but theamount of boring reading).The major theme to emerge from all the data collected relates to the like/dislike of thelearning environment. Of particular note, some elements considered ‘bad’ by the students(learning by doing) are a highlight of the PBL process. This may be a reflection of studentapproach to study or preferred learning style, and deserves further investigation.However, this was not really an unexpected result. The learning style profile of the cohort hadindicated PBL would be a challenge, both for ‘old’ Murdoch students and those articulatingin. It was also very different from any other unit they had undertaken previously. Studentmotivation was higher than in the previous year, but this could be attributed to reasons otherthan the PBL environment: for example the anecdotal evidence that articulating studentsare more likely to undertake set work.The strategy applied – PBL – emphasises student responsibility for learning and requires achange in teacher role from instructor to facilitator (Savin-Baden, 2000) as well as studentrole to more pro-active learning. However, it is a process-oriented approach to learning,implying process is of greater importance then the product (Dahlgren, 2000). This raisedissues of several types:• students are very product oriented – they see the artefact (generally the code theydevelop) as the primary goal of the activities they undertake. Being made to focus onprocess to (in their perception) the detriment of the product was very frustrating, andhad some detrimental effect• the dependence on process also had some detrimental effect on the creativity enhancingenvironment that had been developed. As Edwards (2004) notes, the nature of designimplies a need for flexibility – the unexpected is expected. The PBL process was constraining– in theory students were required to follow process stages in sequence, evenif the aha! factor suggested otherwise• the student perception of the new learning style as problematic (a ‘bad’ thing about311

the course) required exploration, since perception of the learning environment has beenfound to be related to the approach to learning students adopt (Cope and Staehr, 2005)• the focus on what students were doing (ie the process) is in conflict with the ultimateaim of the intervention – to model professionals in practice. Schön (1983) notes thatin the ordinary form of practical knowledge practitioners do not think about whatthey are doing, except when puzzled or surprised. He named this reflecting-in-action,and argued that it is central to the ability to act effectively in unique, ambiguous, ordivergent situations.Table 7.7: Issues in flexibility and creativityFrom Thomas et al (2002)Addressed in this study contextindividuals or groups do not engage ineffective and efficient processes of innovativedesign:– structuring failure – problem analysis is a critical stage:starting from the unknown and progressingto a description of the problem itself,and the knowledge needed to deal with itis fundamental to RE– bring critical judgment into play too – problem-solving habit is challenged byearlythe need to generate alternate solutionpaths–behaviour is path dependent– students were encouraged to re concep-relevant implicit knowledge that alternateperspectives to a problem existthe appropriate level, type, and directionalityof motivation are not broughtto beartualise the problem– the value of alternative perspectives isfostered through participation a collaborativeenvironment and the active promotionof critical friendship– critical appraisal and self appraisalskills are developed through the use ofreflection tools such as the 4SAT (Zimitatand Alexander, 1999)– although external motivation is difficultto eliminate within an undergraduate degree,PBL is seen to foster intrinsic motivationthrough the authenticity of thetasks undertaken (Wilson and Cole,1996)Another issue identified related to the assessment components for the unit. As noted previously,the redesign of the unit was time-intensive, therefore the backup of being able torun in Apprenticeship mode was necessary. In addition, the lead time was too short forformally changing assessment within the unit: the exam could not be removed. This meanta mismatch between the espoused theory (student-centred, problem-based learning) and thefinal assessment component, an exam, and was in conflict with the concept of constructive312

alignment outlined by Biggs (1999). Student criticism of non-problem-related elements werealso made. They saw the problem as the driver for the unit (as they should) and thereforeexpressed dissatisfaction at completing ‘extraneous’ work.The need to ensure that teaching, assessment and every aspect of the teaching-learningenvironment are aligned to the main aims or intended learning outcomes of a unit has beenaccepted for many years, and were re-affirmed in the development of the curriculum mapdescribed earlier. But the term ‘constructive alignment’ goes beyond this to require that theaims of the unit are in line with constructivist principles of learning and the findings of studentlearning research. As noted in Elton (2000) we want students to learn with understandingand be assessed for it. The students did not perceive this to be true.Having said this, the framework was seen to have considerable merit – issues identified bythe literature were being tackled, although not necessarily in a way that was comfortable forstudents. In particular the more abstract issues identified by Thomas et al (2002) (issues inflexibility and creativity) were addressed (see Table 7.7) within the environment.Impact on student developmentThis section looks at the short-term impact of the intervention on students. Unlike 2002, itwas not possible to track students into subsequent SE units – as Figure 7.3 indicated, veryfew students were enrolled in the BE(SE) programme.Figure 7.14: Final mark for ENG260 2002-2003313

The formal assessment of the unit was based on group work (three components), and the examwhich modelled previous exams, based on questions that had been used before. In theory itwas possible to compare how well students performed in relation to previous cohorts.Although an exam does not align well with the learning philosophy being applied, it can bestated that the PBL environment did not appear to unduly disadvantage students: Figure7.14 indicates a marked positive shift in student final marks, though the average mark wasFigure 7.15: Raw exam marks for ENG260 2002-2003within the range established by previous cohorts. As Figure 7.15 indicates, the raw exammark shows a bimodal distribution in the number of students achieving within a specificmark range. No correlation was found between articulation status and marks – articulationsstudents did as well as students who had been at Murdoch for their first year. In addition,83% achieved a final mark of 60% or better (which is higher than the class average). However,the strongly discriminating nature of the final exam raises the following questions:1. did the students perceive this style of learning as unsuitable? and2. was the exam format unsuitable for this style of learning?These can be partially answered by looking at the qualitative data collected.One of the final components of the formal assessment was to prepare for a PerformanceReview. As well as some more technically based issues (eg how easy would it be to go to design314

Figure 7.16: Good/bad things about a unit structured this wayfrom the specification developed by your team) students were asked for their impressions ontheir team performance and asked to comment on whether they thought they learnt less ormore this way. This provided rich data on student perception of the learning environment ingeneral, and could be mapped to specific students (comments could be made anonymously,if preferred - only submission/no submission was recorded as an assessment object, but moststudents were happy to submit with name). The responses were coded to identify recurrentthemes, with frequency of comments recorded in a spreadsheet. Students were asked to reflecton the following questions:• what is good/bad about a unit structured this way. Figure 7.16 shows the results. Whilesome students could appreciate the authentic nature of the environment (like RE practice),this learning style did not sit well with many students (intimidating learning style;sudden change in study style). However, the groupwork environment was almost alwaysviewed positively (project+team improve individual learning/group lottery)• things to add/change/delete in how the virtual secondment and project were organised.Figure 7.17 shows that students felt they lacked guidance in the form of examples andexercises to use for benchmarking their performance (need examples/reference/guide).In addition, they expressed concern that such a large component of their assessmentwas based on group work (add more individual), generally a feature of capstone projectsonly. The relevance of non-problem-related elements was also raised as an issue (linksupplementary work to project)315

Figure 7.17: Things to add/change/delete in a unit structured this way• reflecting on whether they (individually) felt they had learnt more/less this way. Figure7.18 shows that the class was fairly evenly divided on the point of learning more or lessfrom this approach: comments on a lack of mastery of subjects: (less every time newcontent arrives); of only focusing on components addressed by the project, on delegatingand relying on others (slack off and rely on others to do reading; learn concepts fromothers), indicate concern about less content learning (no mastery of subject). Towardsthe end of semester, some of these students still felt lost and confused: self teaching isnot one of my fortés stated one student, perhaps with a hint of despair.Students felt they learnt more in the areas of research, communications (confidence tospeak up; need to be heard & get ideas across) and team skills. They added conceptseasier to grasp; forced to learn more for project relevant components and, finally theyhad to grapple with various perspectives from others. In summary there were ampleresources & up to us to take it.Comments on the collaborative environment were also indicative of student acknowledgementof the changing nature of the unit, although some students had issue with the random allocationof group membership. Those comments characterising the opposing perspectives areperhaps the most pertinent, and an aspect that requires further research. An understandingof the learning motivation for the students might reveal a relationship between why theylearn and how they expect to learn.The University-administered SSU for ENG260 supports results of analysis of the qualitative316

Figure 7.18: Learning in a unit structured this wayfeedback: bi-polar distribution is evident in answer to two major questions:• Q1 – It was clear what I was expected to learn in this unit (standard deviation 1.242)• Q9 – The unit resources were well organised and easy to follow (standard deviation1.063)The open-ended questions on this instrument also indicated similar perceptions:Student A teacher is good, but I like the traditional way of providingcontent, ie teachers say‘‘learning this learn that’’Student B I don’t really know[what to change]but lectures would be good instead of workshops!Student C the introduction to PBL was a good concept to grasp that will behandy in future unitsStudent D I liked how people were forced to work in a group like in theindustry317

Student E [best aspect]real world skills that would be critical in a work environment... overall I enjoyed the self learning aspectThis feedback re-enforces the conclusion drawn from the Engineering Year survey, where goodpoints included a lot of information being covered; excellent content; good lecturer versus needlecturer to use powerpoint slides on board rather than reading the book all the way; need properlectures to explain basics (even one hour per week); no good text information to study or learnfrom.Most comments could be added to the categories developed in the thematic analysis of Cycle1, with most positive comments reflecting the discipline content and its relevance, whilegroupwork could be added to environment. The majority of negative comments also fellwithin the theme of environment – some students indicated confusion, commenting doesnot have a proper course structure or provide enough guidance, and workload: studentsfound the assessment ‘vague’ (ie not pre-defined to the extent they expected)([need more]instructions on what to do). Interestingly, few comments were made on excessive workload,with one student suggesting an adequate amount of work.While these data indicate students perceived the learning environment ambivalently, howthey perceived their own learning was less clear. Also unclear was whether the final examwas a component of the assessment that was detrimental to the learning being achieved.These concerns could be partly addressed by examining student orientation to study.Approaches to studyStudents’ responses and adaptations to learning contexts are captured by information-processingmodels of learning, with a student’s choice of approach influenced by student perceptionof the nature of the course being studied, and the discipline (Entwistle and Ramsden, 1983;Ramsden, 1988). A clear consensus has emerged that students approach learning with eithera ‘surface’ or a ‘deep’ orientation, and that individual differences in the ways students approachtheir studies explain, at least in some measure, differences in study success (althoughthis is not self-evident – depending on the type of assessment, which may allow success to beachieved with a surface approach).It has also been suggested that students may adopt different approaches depending on thelearning context, with a surface approach more common where performance in exams isemphasised and approaches to teaching are teacher-centered and less so where independentlearning is stressed (Trigwell et al, 1999). Their findings give support to the contextual view318

of student learning: study approaches or orientations are formed in the interaction betweenindividuals and their environment, and are not ‘innate’.In order to gain some insight into this aspect of learning, and as a follow-up to some of theissues identified above, the shortened version of the Approaches to Study Inventory (ASI) byEntwistle and Ramsden (1983) 3 was administered to the student cohort, at the completionof ENG260 in 2003. The results are as indicated in Figure 7.19 (note that some studentsdeclined to complete the survey).Figure 7.19: Results of the ASI applied to the 2003 cohort of ENG260This post-hoc ASI showed that students were very much sitting on the fence between learningfor meaning (MO) (mean 2.53, standard deviation 0.43) and learning for reproduction (RO)(mean 2.56, standard deviation 0.41). This confirms the students’ ‘fence-sitting’ with regardsto their perception that they had learnt more or less from this approach (compare with Figure7.18).A correlation between a high(er) score for meaning orientation and final result did not exist –of the students who completed the ASI (88%), 41% showed same or higher Meaning Orientationto Reproduction Orientation. Of these, 28.5% achieved each of a H(igh) D(istinction), ora C(redit), while 14% achieved each of a D(istinction), a P(ass) or a fail (N). This latter studentindicated close orientations (MO 2.65, RO 2.38) All other students who failed exhibitedmuch higher ROs to MOs (between 0.43 and 0.81 difference). Of the students who indicatedhigher ROs, 62.5% achieved a P(ass) or lower. Raw exam results show a different picture: of3 as previously noted, this 32-item instrument has been confirmed by Richardson (1990)’s work to possessadequate internal consistency and test-retest reliability319

the 41% with same or higher MO, 28.5% achieved a D(istinction), 28.5% a C(redit) and 43%failed the exam component (N). This could suggest the exam does not target meaning as wellas it could. However, 50% of students with higher ROs also failed the exam, suggesting itdoes not focus on reproduction, either. It should be noted, perhaps as a caveat, that learningfor understanding is less reliably assessed than memory learning and learning that achievessome form of creativity will be quite radically different for different students (Elton, 2000).Some alignment could be discerned between the tenor of comments made in the PerformanceReview and results of the ASI. In effect, supporting the work of Entwistle and Tait(1990, 1995) meaning-oriented students were more likely to see their learning environment inpositive terms while reproduction orientation was associated with the view that the learningenvironment was difficult. These results were also supported by a later study by Tynjäläet al (2005), which indicates that students’ conceptions of the characteristics of their learningenvironments were related to their study orientations and strategies.From the teacher’s perspective, what this confirmed was that although a great deal of effortwent into preparing the PBL environment, more scaffolding would appear to be required.Students seem to need greater preparation in order to tackle a different learning model (ega better understanding of the PBL process), and support structures (examples, guidelines)so that they have a clear indication of the appropriateness of their learning. Drew (2001)discovered that a heavy workload tends to affect the depth at which students studied, whileChambers (1992) found that a ‘reasonable’ workload is a precondition of good studying anddeep learning. Student comments on this aspect suggested the motivation to deep learningcould be enhanced.7.2.5 Reflection on findingsThe focus of my reflection, based on Kreber (1999) (see Table 7.8, reproduced for information)is on the Instructional component.The main purpose of this cycle was to modify the instructional strategies applied to ENG260to address issues exposed in the evaluation of the Apprenticeship model applied in Cycle1. Deciding to apply PBL had implications on Curriculum, in that discipline content wasmodified (ie reduced). However, the teaching goal, the process for identifying this goaland its premise did not change. In terms of Pedagogy, effort was placed in researchingstudent learning in order to evaluate the effectiveness of the strategies put in place to promotethe achievement of the teaching goal. Under the Instructional component, the followingcomments are noted:320

Table 7.8: Reflection in the Scholarship of Teaching model (Kreber, 1999)ContentCurriculum Pedagogical InstructionalWhat are the goalsof my teaching?What do I knowabout how studentslearn?What instructionalstrategies should Iuse?Process How conscientioushave I been inidentifying thisgoal?How effective amI in promoting itsachievementHow effective havemy strategies been?PremiseHow does my goalmatter? What arethe alternatives?What are alternativestrategies?Why does it matterthat I use this strategy• Content – the goal of this cycle was to provide an underpinning for students to takeearly control of their learning. The ongoing exploration of learning models suggestedPBL could be an appropriate strategy to achieve this - through its emphasis on problemand student-centredness, PBL is seen to:◦ acknowledge the base of student experience◦ emphasise student responsibility for learning◦ cross boundaries between disciplines◦ intertwine theory with practice◦ focus on the process of knowledge acquisition rather than the products of thatprocess◦ change staff roles from instructor to facilitator◦ focus on communication and interpersonal skills so that students understand thatto relate their knowledge, skills beyond their area of technical expertise are required(Savin-Baden, 2000)• Process – the strategy was effective in part – it provided students with a process to dealwith problems within a metacognitive-rich framework that makes complexity apparentand lets students deal with it explicitly. However, while some students were able to‘work’ the PBL process into their personal learning strategies (self motivation; takeinitiative), and hence exploit the benefits, many students found the PBL methodologyan issue. Positive aspects were that students did not rely on the teacher as the ‘lecturer’– some groups were able, later on in the semester, to value her as a consultant resource -321

to set up meetings, discuss their approach to the problem or tentative findings/solutionpaths and discuss the pros/cons of the approaches they were favouring. When issueswith tools and notations arose, other consultants were ‘engaged’ to resolve these.This change in relationship with the teacher was very noticeable. While some studentswere critical of the lack of guidance provided by the learning environment (and byimplication the teacher) (more of what is expected in triggers), others could see thevalue of not relying on the teacher as heavily (requires self motivation to research whatwe were to produce; proactive learning - good chance of absorbing info better)• Premise – Glass (1995) suggests that discipline and creativity are the odd couple ofsoftware development – the discipline imposed by methodology, for example, forms aframe for the opportunistic creativity of design. The educational dilemma becomes oneof providing an educational base that enables software developers to both create andengineer the systems they build: to be adaptable to the changing environment that isinevitable in their chosen discipline. The PBL approach promised to provide studentswith a solid foundation in subject matter while at the same time exposing them toreal-world characteristics. As a strategy, PBL was also important in that it targetedissues that were raised in the evaluation of the previous master/apprentice strategy.However the requirements of the strategy, strict adherence to the PBL methodologychosen, was finally seen as a restriction on other aspects of the unit goals, namely theadvocacy of creative elements in student approach to the problem. In effect, the strategydoes not align well enough with professional practice in the discipline. There is asuggestion that efforts to help students learn at the levels of analysis, synthesis, andevaluation may be impeded by a mismatch between the kinds of thinking actually requiredin specific disciplines and generic formulas for encouraging higher-order thinking(Middendorf and Pace, 1986). In the final analysis, applying a strict PBL methodologyfor learning may run counter to an important strand in current thinking about teachingthat stresses the disciplinary nature of knowledge. As a tool for learning, it must beadapted to the discipline.An additional issue, from the literature, raises the problem of contextualisation. The researchinto case-based and PBL suggests that presenting overly contextualised knowledge as themechanism to learn leads to failure to transfer flexibly to new situations (Vanderbilt, 1997).Abstract problem representation is seen as one approach to dealing with this issue (Spiroet al, 1991).322

Reporting findingsThis work has been presented at two international conferences and a national Teaching/LearningForum. In this context, these publications addressed the need to engage with educators andresearchers in education, as well as Software or Requirements Engineers and SE/RE educators.The Higher Education Research and Development Society of Australasia (HERDSA), holdsa prestigious conference with strict refereeing criteria for the publication of education research.Armarego and Clarke (2003), presented in that forum as a refereed paper focusedon the background to the approach undertaken during this cycle. The paper describes theissues of education for software development professions and discussed the development of thePBL-based model to address the adaptability skills and knowledge of Software Engineeringstudents. A reviewer’s comment indicates this project waswell grounded in the research literature [...] for Conferencepurposes it would be appropriate to recognise this paper as anexcellent brief review concerning high level outcomes sought increativity and divergent thinking in a disciplinary area moreusually noted for prescriptive curricula and convergentthinking. It sets out very strongly a basis for the research inprogress.The Teaching Learning Forum 2004 paper (Armarego, 2004b) described the PBL-based cyclein terms of addressing student preconceptions regarding the way they should learn. Thispaper describes the PBL environment provided, looks at the student cohort undertaking theunit in 2003, and describes their reaction to this alternative way of learning.Armarego (2004c) provides a more abstract view of the development of the PBL approachtaken: issues identified in education for engineering (in particular engineering of software);triggers for the PBL approach taken; and where the approach did and did not succeed.This paper pointed to the need to attempt a hybrid approach to learning in RequirementsEngineering.This work was described within a forum that addresses issues of interest to the RE profession,both practitioners and academics (Armarego, 2004a). Reviewer comments indicate avalidation of the findings:The paper shows extensive scholarship and considerable thoughtin research program design. The paper is very well written andcontains only a few tiny typographical errors.323

WEAKNESSES of the paper:The results in the paper reflect contemporary thinking inrequirements engineering texts. That is, the result isunsurprising, particularly to anyone with industry experience.ACTIONABLE ADVICE (HOW to overcome the weaknesses and improvethe paper):The major weakness is also a useful finding and there is nothing ofsubstance that can or should be done to change the paper.I would suggest, however, that any further research shouldaddress curriculum design for postgraduate software and systemsengineering programs designed for experienced professionalsrather then undergraduate programs. The postgraduate programscould then be targeted at people who possess the unteachablecompetencies and who have had sufficient experience to integratethe university declarative knowledge with their on-the-jobprocedural knowledge. It would also be worth comparing theindustry needs analysis data with such postgraduateprograms.A second reviewer concurs:STRENGTHS of the paper:I have much admiration for the authors in attempting to bring somerealism into the teaching of software engineering. From what I’veseen in Australia, which isn’t too much, how students are taughtis generally poor compared to how students are taught in Europe,at least in my experience. So kudos for the authors for attempting this.The paper gives an honest appraisal of results thus far and isagain applauded for this.324

WEAKNESSES of the paper:There needs more detail discussion of the implications of the results.However, the similarity in grades over the years might be as a resultof the course being in its infancy, with all the typical teething troublesthese types of courses have. Perhaps a longitudinal study over the nextfive years will yield better outcomes. I hope so as I approve of the coursethe authors are implementing.In response to a comment regarding the need for creative problem-solving in RE:Yes, I agree. You might say Requirements Engineering is Not For DummiesIn response to a comment regarding student inhibition:How do you know? If this is the case then this is a damningindictment on a higher education institution and the wrongmessage is being put across entirely to students. It is nowonder that they might be afraid to think for themselves iflecturers on other courses give them no encouragement, oractively discourage them. Of course, the cultural/religiousupbringing of the students might also be responsible for this.In any case, a strong, scary point. Well done. [...] ...a puristmight argue it is critical [to] teach about ways in which theCustomer’s problems can be addressed rather than religiousadherence to any particular method. This does mean, though, thatrequirements engineers must have a toolkit to work from, inselecting the most appropriate tool for the problem. UML isinsufficient. Other approaches must be taught as well asself-taught prior to this part of the course. [...] I wonderabout the quality of feedback from students. I get this sort offeedback too so don’t question what you have [...] Theimplication from the student that he/she does not enjoyself-teaching is a worry. What calibre of students do youtypically get? What are they’re characteristics? Are students sooverly spoon-fed from school all the way to uni and beyond thatthey have no academic curiosity? Again, a damning indictment onthe education system and well done for sticking it in the325

paper!Software Engineers with an interest in education are the target audience of the CSEETconferences (Conference on Software Engineering Education & Training). Armarego (2005)was addressed to this audience. One reviewer’s comments are:Acceptance: 5 Relevance: true Length of Content: true PracticalImpact: 5 Originality: 5 Correctness: 5 Technical Depth: 5Presentation: 5 Fixable: 5 Overall: 5 Positive Aspects: Anexcellent tutorial on problem based learning that accompaniesthe discussion of using PBL in software engineering. I found thepaper refreshingly candid in acknowledging the tension betweendiscipline and creativity - a tension that is all too oftenbrushed aside in engineering education. Negative Aspects: Thebibliography might have to be cut for space - we usually don’thave papers with 53 references. Still, this does provide auseful bibliography ...Within the Murdoch context, the funding gained to undertake the re-development of the unitrequired outcomes and findings to be disseminated, both in a formal report to the TLC andthrough the presentation of in-house seminars. These were completed during the latter partof 2003.The most critical review of the findings of this Cycle occurred in-house. By the middle ofsemester 2 of 2003, the Professor of Software Engineering, who was also Head of Engineeringfor the duration of this research, proposed that the model of learning applied in ENG260(and which had, to some extent, informed changes throughout the SE curriculum) becomethe basis for teaching in the final two years of all engineering programmes at Murdoch. Thefirst step to this decision-making process was to submit the data collected during this cycleto scrutiny. The Review Team included SE academics at Murdoch and adjunct academics.These were current practitioners of SE who took on both a teaching and advisory role withinthe discipline. The purpose of this review was to explore alternate readings for the data andidentify implications for other specialisations. This resulted in a proposal, drafted by theHead of Engineering and the researcher, to the School executive by the end of September2003. The impact of this proposal, which was accepted at a 2-day Planning Meeting at theend of November 2003, on this research are described in the next chapter.326

7.3 Conclusions drawn from Cycle 2The intervention for Cycle 2 was based on a model for Problem-based Learning. PBL emphasises‘learning to learn’ in an environment that moves from dealing with content andinformation in abstract ways to using information in ways that reflect how learners might useit in real life (Oliver and McLoughlin, 1999). In pragmatic terms this means no loss of theadvantages of the Apprenticeship model – situated ‘authentic’ tasks enabling transfer. Anadvantage of PBL, however, is its focus on solving of wicked problems in wicked domains.Evaluation of the PBL model in relation to discipline characteristics indicated that the modeldoes not align well enough to the opportunistic and creative nature of professional practice:students were inhibited by strict adherence to the process. In hindsight, perhaps the strongsupport PBL enjoys in the convergent-focussed medical education arena should have suggestedthis. The conclusion reached was that the PBL methodology could be down-playedso that students could exploit creative opportunity within their own learning.In the same way that the master/apprentice model addressed some aspects of the disciplinepractitioner action and inhibited other, the facilitation aspect of the PBL model also exhibitedelements of an ‘incorrect’ learning environment. A mentor/protégé (Fosnot, 1989)relationship allows teacher and learner to seek to understand each other’s position with theaim of agreement and/or defensible deviations. However, this requires a confidence on thepart of the learner that is not often present at novice stage, and therefore needs to be fostered.The work of Laurillard (1993) develops this concept of learning as a dialogue.Further evaluation of this model indicated that student conceptions of the characteristics oftheir learning environments were related to their study orientations and strategies. Meaningoriented students were likely to see their learning environment with positive terms such ashaving good atmosphere and demanding deep learning while reproduction orientation wasassociated with the view that the learning environment demands surface learning and requiresstudents to be overworked.It would seem that criteria for evaluation of learning (specifically assessment) could go someway to addressing this issue: a study by Sambell et al (1997) suggests that from studentspoints of view, assessment has a positive effect on their learning and is ‘fair’ when it relatesto authentic tasks; represents reasonable demands; encourages them to apply knowledge torealistic contexts; emphasises the need to develop a range of skills and is perceived to havelong-term benefits. Alternative assessment (eg portfolios, demonstrations etc) are perceivedas characterised by these qualities and students report these modes help them to learn in amore in-depth way. However, in our environment students perceived the portfolio as extra-327

neous to the tasks undertaken.As we learn more about how students learn, and what they need to learn in order to practice ascompetent professionals in their chosen discipline, we move further from traditional teachingand closer to the concept of learning as a reflection on professional practice undertaken byboth teachers and learners. This view of professional education has implications for thedesign of teaching (Laurillard, 1993):• academic learning must be situated in the domain of the objective: the activities mustmatch that domain• academic teaching must address both the direct experience of the world, and the reflectionon that experience that will produce the intended way of representing it.While both the Apprenticeship model and CreativePBL addressed these to some extent,further refinement is required to make the alignment between education and practicing inthe discipline closer. The next chapter looks at changes made for the following offeringof ENG260 to address these issues, and to examine whether the students benefitted fromchanges made, in the follow-on unit.328

Chapter 8RE as Studio Learning 2005Teaching is more difficult than learning because what teaching calls for is this: tolet learn.(Heidegger, 1968, p 15)The PBL environment the students experienced during 2003 may be considered a creativeone: one of the aims of its development was to enhance divergent thinking and the creativepotential of students. Despite the measure of success of the CreativePBL model, the learningdiagnostics (eg ASI) results indicated at least as strong a bias to surface learning as therewas to deep learning in the student cohort. The literature suggests this is an outcome ofthe (different) learning environments students are exposed to in (different) units. Ultimatelyinnovation introduced a few units may be undermined if traditional approaches are maintainedelsewhere in the students’ programme - so that benefits may only be apparent or areenhanced if the innovation is introduced across the entire curriculum. While this is not somuch of a problem in the SE programme at Murdoch University – by the end of 2003 most ofthe core units in SE had been aligned to some form of task - or - problem-based environment,it is critical in the context of a strategic move away from traditional lecture/tutorial/lab-stylelearning within the Engineering area at Murdoch.This relates to a further issue to be tackled: the need to engage in life-long learning. Thepractitioner studies discussed in Chapter 2 indicated the importance of graduates’ ability tokeep up with changes in knowledge and information requirements (eg Lee (1999b); Snoke andUnderwood (1999)), while the only professional (as opposed to affective) capability cited inthe top twenty engineering capabilities in Scott and Yates (2002)’s study is current technicalexpertise relevant to the work area. Practitioners acknowledge that the speed with whichtechnology evolves, the multiplicity of its impact on society and the ramifications of thatimpact mean that metacognitive and knowledge construction skills as well as adaptability329

ecome vital. Professional practitioners with such skills become agents of change (Garlanet al, 1997).The CreativePBL environment students experienced has gone some way to addressing thequestion of preparing students for situations that are highly variable and novel (Bowden andMarton, 1998):• shifting the focus from teaching to learning: the environment is student-centred andminimises ‘teaching’• concentrating on developing (generic) capabilities and on student learning outcomes:it may be considered a creative environment that enhances divergent thinking and thecreative potential of students, thereby addressing some generic attributes• moving from highly differentiated and fragmented curricula to integrated learning programmes:the approach is somewhat holistic.However, evaluation of this model highlighted two issues that needed to be addressed:• were student conceptions of the characteristics of their learning environments relatedto their study orientations and strategies? As has been noted, the literature suggestsmeaning oriented students are likely to see their learning environment with positiveterms while reproduction orientation is associated with the view that the learning environmentdemands surface learning and requires students to be overworked. The ASIdata did not indicate such a strong correlation – students with both higher MO andhigher RO indicated positive and negative aspects of the learning environment (seeFigure 7.16). This suggests they are receiving mixed messages regarding the learningenvironment• ultimately the CreativePBL model is a process-oriented approach (implying processis of greater importance then the product (Dahlgren, 2000)), which may reinforce theperception that RE is a smooth process of sequential stages. This was interpreted as aninhibitor to student engagement with the learning environment - the effort expended inapplying and monitoring the process did not allow opportunism and heuristic insight theimportance they were warranted in discipline practice. Andresen et al (1995) describethe need for contingency measures to be available in the creative nature of design (asopposed to problem-based learning) where the unexpected is expected.This reflection on the data gathered during the CreativePBL cycle of the study informed theplanning for Cycle 3. In this cycle, focus returned to the issues of transfer and adaptability330

Figure 8.1: Education for RE – Action Research Cycle 3while maintaining a commitment to providing an authentic, situated environment. Figure8.1 places this cycle in the context of the thesis as a whole. Cycle 3a reflects the planningrequires to set up the model and implement an orientation to it, while Cycle 3b describesthe observation of students in a follow-on unit, in particular with regards to their ability totake control of their learning and transfer the knowledge they had gained during Cycle 3.Also important at this time was a School-level focus on generic attributes, triggered by theprofessional re-accreditation process.Cycle 3 therefore took a 2-pronged approach:• modifying the CreativePBL model and building into the RE curriculum a greater focuson generic skills as part of the outcomes of the learning environment. Further refinementto the model was also required in order to achieve a greater degree of alignmentbetween to components of the learning environment. Student reflection also needs tobe addressed more explicitly, so that students were able to challenge the effectivenessof their learning. One strategy (Edwards, 2004) requires the teacher to guide studentsin the nature of the processes they engage with and to help them reflect critically ontheir effectiveness• applying this model both to the RE unit and concurrently across all units and all331

programmes within Engineering, albeit for the final two years of study only. Thisapproach addressed the issue of undermining the learning ‘philosophy’ being initiatedin the RE unit and ultimately in the SE programme.These elements were addressed by exploring Schön’s ideas on reflecting-in-action (Schön,1983). However, as discussed in Chapter 3, the design studio approach, which is seen toexemplify learning for practice, also has some problems in that it may focus on the creativityof the design process to the detriment of the practical application that should be its goal.In addition, while the studio model enables processes to be exploited to support the designactivity, reflection-facilitating strategies are seen as often missing, and require scaffoldingthrough concrete activities built into the environment (Nelson, 2003).What is needed, then, is a model of education that adds, to the positive aspects of studios, afocus on metacognition, so that learning integrates evaluation of the ‘practical’ outcomes ofthe problem with the creative process. Aspects of Laurillard’s learning model, with teachback(Pask, 1976) and self explanation (Chi and Bassock, 1989) are incorporated as key phenomenain the learning dialogue, thereby:• forcing a focus on key aspects of the domain• forcing deeper processing of the topic, allowing relationships to be forged• allowing failures and conflicts to emerge (Gobet and Wood, 1999).These ideas form the basis for the Studio Learning model for RE proposed and applied in thischapter. This model explores the ‘dialogic’ nature of learning (Laurillard, 1993) and supportsthe idea that learning is defined in terms of dynamic sets of relationships whose interactionsand interdependencies create and control conditions that are supportive of specified conceptswithin the discipline.This chapter describes how the issues identified in Cycle 2 were addressed: how the interventionwas planned, what actually occurred and possible interpretations for findings of thecycle. As in previous cycles two distinct evaluations are proposed: implementation of themodel and its success and the effect of the intervention on the student cohort. This cycleplaces greater emphasis on examining longer term impacts of the intervention, through datacollected in a follow-on unit.In summary, these evaluations suggested that there was close alignment between characteristicsof the discipline and the Studio Learning model developed, although some concern shouldbe expressed regarding the pragmatic decision to retain components of assessment that didnot align with the learning model – in Elton (2000)’s terms, doing the right thing wronger.332

In this cycle it became evident that student approach to study has strong bearing on thelearning outcomes achieved, and that students with strong meaning orientation also align withthe learning model. This finding is supported by research that indicates such students seekout opportunities which challenge them, and are more likely to display increased motivation.Observation of student interaction in the follow on unit, and examination of their workpatterns and reflective journals support these findings.observing students in a subsequent unit also brought into focus the students’ developmentalgrowth towards life-long learning. In particular a transition from Stage 2 to Stage 3/4 ofGrow (1991/1996)’s model was discernable.The next section describes changes to the learning environment which impact on this cycle.The chapter then continues in its description and discussion of the components of the StudioLearning model.8.1 Context for Cycle 3In mid-2003 the decision was taken to move all third and fourth year learning within Engineeringto a PBL-based model. The aims of this move were to enable graduates to beequipped to meet the changes of a transforming industry through:• improved learning outcomes for students in areas such as project management, problemsolving, group and co-operative work skills and communication skills (thereforetargeting both generic engineering skills and University-based generic attributes)• an increased focus on design content within each discipline area• a closer match to professional requirements and the potential to transition into employmentpositions on graduation.This decision had enormous impact on the researcher and the academic staff as a whole, aswell as on the students completing their final two years within Engineering. These are brieflydescribed in the next sections.8.1.1 Curriculum componentsThe first impact was on the learning situation, which changed during 2005. The success ofthe approach taken in the SE programme where, by this stage, no core unit was provided ina lecture-tutorial format, was one of many factors that led to a revamp of all Engineering333

offerings at third and fourth year level. From 2005 both Year 1 and 2 became common for(all) Engineering students, with specialisations focussed in the final two years. In addition,all units would be taught in a Design Studio model, informed by the approaches describedin this study, in particular the CreativePBL model discussed in Chapter 7, and the StudioLearning described in this chapter. An advantage of this was that the orientation to PBL andStudios could be centralised: all third year (and fourth year, for 2005 only) students would berequired to undertake an orientation programme in the first week of semester. This ensureda common base of understanding on the part of students, and gave academic staff (some ofwhom had undertaken the PBL training described below, but had not yet implemented aPBL unit) some exposure to issues surrounding such a radical change in learning.Implementing the Studio Learning model in Engineering required considerable revision ofunits. The most influential to this study was the decision to go from eight 3-point units (hencefour units per semester) to three 6-point Studios in each specialisation. Within the SoftwareEngineering discipline the revision required was relatively minor, except that ENG260 neededto be moved to third year to fit into the revised curriculum model.Each Engineering specialisation was defined as three Design Studios, taken over two years.Within SE the RE unit (ENG260) was proposed as the initial SE Studio, ENG301. Thisunit absorbed all the material of ENG260, and integrated components of another SE unit,which addressed process within the SE development lifecycle. The main impact on theresearch being undertaken was the addition of an assessment activity that required studentsto develop a tool for estimating effort within a development project. Triggers for the problemsin ENG260 were broadened to ensure students engaged with the appropriate resources toundertake this additional task.Development of the Studio model also required changes in class timetabling. Rather thantime set aside for lectures, tutorials and labs, Studios worked in a block-teaching framework– each Studio was allocated 10 hours of class contact (teacher present)– generally on theone day and another 10 hours of additional class time. Therefore students were expected tospend a minimum of two days a week on each Studio plus any additional time required byindividual study habits.For ENG301, I did not consider this timetabling practical – students needed time to absorband reflect on the problem, as well as do the research required. Having to wait a weekto interact with the teacher I also considered too long, even with an ‘open door’ policy.Fortuitously, a colleague felt the same about his Studio. The compromise reached was toschedule two 5-hour blocks separated by at least a day (ie in practice Tuesday afternoon andThursday morning). This allowed both students and teacher the time to follow up on issues334

identified within a session. It also facilitated timely feedback – in general anything submittedon Tuesday could be back with the students by the Thursday session, and from Thursdayback on Tuesday.8.1.2 Staff developmentHitchcock (2000, p 52) suggests thesuccess of any PBL curricular initiative requires the assistance of faculty skilledin PBL.This requires academics to take the role of trained facilitators to guide the learners withoutteaching them in a traditional manner: having someone for the groups to look to for guidanceleads to a richer, more holistic level of learning (Dahlgren, 2000). Although PBL had beenapplied in the SE programme (and reported on internally through School seminars), in generalit could be said that academic staff were not deeply conversant with the approach. Due tothis unfamiliarity, a series of staff development workshops were initiated for academic staffin order to provide training and education on PBL.The staff development workshops were structured to address two separate concerns:• provide a background in PBL approaches by undertaking a PBL session – the Schoolengaged the services of Dr Sally Clarke, then in New Zealand, who had acted as mentorto the researcher during the development of the PBL version of ENG260, to facilitatea workshop on PBL. The outcome of this was an understanding of what changes wouldbe needed to enable PBL across all units, and a set of tasks to be completed by eachacademic staff member for each of the units they coordinated. Tasks included curriculummapping and problem development, amongst others. Concerns were also raised –the most critical being discipline content coverage. In addition, several staff memberswere not convinced that basic (ie foundational) knowledge could be learnt this way• an appreciation of the issues raised by such active learning – the School invited anacademic from Victoria, Dr Roger Hadgraft, who had a great deal of experience inapplying PBL in a (civil) engineering context. The result of this workshop was clearercommitment on the part of academic staff to make this work.The workshops were scheduled several months apart, so that work had commenced on therevision to units before Dr Hadgraft arrived. An advantage of this was that staff could discussissues from their experience with grappling with them.335

Although the strategy discussed and ‘learnt’ was PBL, the experience discussed in the previouschapter has some impact on the type of PBL learning to be advocated. As has beennoted, a strict adherence to the PBL methodology was seen as restrictive in some aspects.Instead, a modified approach, focussed on the Studio Learning environment described in thischapter, was proposed for adoption.8.1.3 Orientation to Studio Learning – Design WeekIn order to effect the ‘cultural change’ towards learning required by this move, studentscoming into 3rd year of Engineering (and for 2005, 4th year students) were involved inan orientation programme. The objectives of this week-long activity are described moreextensively in Armarego and Fowler (2005), and in summary were to model Studio Learning;establish the roles and responsibilities of students and academics within this model, andprovide an introduction to the necessary support services made available with the learningenvironment. These objectives were achieved through a small-scale design task as a meansof identifying and exposing the Studio Learning approach and an introduction to generictools, techniques, methods and processes that might otherwise have to be duplicated in eachStudio, and provided an opportunity to pretest student perceptions of the model.Figure 8.2: ASI Results for students about to undertake the Design WeekStudents were required to complete a set of learning styles diagnostics. These act as benchmarks,and will be one of the bases for ongoing evaluation of the learning approach. AsFigure 8.2 indicates, the results of the ASI undertaken by (most) students on Day 1 of the336

Studio Week show that this student cohort, while slightly leaning towards a deep approachto learning (as should be evident at this stage in their undergraduate progress), are notoverly oriented towards Meaning (MO mean 2.62, standard deviation 0.54) compared toReproduction (RO mean 2.06 standard deviation 0.48) 1 . Unfortunately, further analysis ofthe data shows no evidence that the 4th year students in this cohort favour Meaning aboveReproduction - the indications are that this is an individual trait.What may be significant is a comparison of the ASI score for the ENG260 2003 students whohad progressed to 4th year in 2005. Although a very few students fit the criteria, results canbe considered in a positive light (see Figure 8.3), with a marked increase in MO.Figure 8.3: ASI scores for 2003 ENG260 students in 2004All students completed the orientation programme successfully - success being measured interms of both the product (task adequately designed) and the process (group process established,PBL process applied). Students demonstrated their engagement with this learningmodel: the quality of the final presentations and diversity of solutions emphasised their abilityto be self motivated independent learners. Significantly, initial observation indicated thatstudents who were at Murdoch Engineering prior to the Design Week were better able tomake the shift to Studio Learning - understandable since it is preempted in several unitsalready running. However, student feedback showed that articulation students and (international)students joining the School on exchange programmes initially found the learningmodel disorienting and confronting.1 These figure can be compared with the ASI results of the ENG260 cohort (see Section 7.2.4)337

On reflectionA key component of the orientation was feedback on the process and outcomes by way of astudent journal/diary indicating tasks, outcomes and times spent. The following narrativesare excerpts from the reflective journal undertaken during the Studio week. The studentsspeak for themselves (Frazer (F), Konrad (K), Sam (S), Dorian (D), Morris (M)):Design Learning I am trying to keep an open mind and am going to put my besteffort in, even though it is a new style of teaching orpresentation, as you only get out of something what you put in.Went home for lunch but had ideas running through my head aboutthe design studio. I think I am going to like the idea as Ithink that I learn best this way from past experiences (M)Groups I was glad that we had a number of different types of people inthe group (K); Formed groups, good to grab people with otherareas of expertise. Very interesting, amazing what relatedinventions people have come up with (S); Tensions are high,people want to thump other people in the group, for wastingtime(D); Everyone has their own individual style and it was acase of the most liked (not necessarily the best) idea that waschosen (M)Journal Journal began. Noticed that it is a good way to watch how timeis used. The Journal provided a useful tool to relate allaspects of the project and worked to sooth my mind to ensure allis going well (K) Good reflection for how I behave, selfawareness leading to personal improvement, that sort of thing(S)Keeping Minutes We’ve actually used past minutes throughout the week toclarify decisions, documentation is time consuming but usefulimagine that!!! - Interesting to look back and have a record oftime I have spent and how my groups project has progressed andcome together throughout the week (S)Tasks Took too long, browsing the web with Gateway on is no good, asthere is always the possibility of you checking your Mail (nodiscipline on my side here) (F); Individual research spent inthe library. The most important thing to come from this338

particular exercise is that in the design studies, thisindependent research will be a crucial part of what I have toindividually use. So, I think I might see if there are anylibrary courses available to help me improve my efficiency infinding things as this will save me a lot of valuable time(M)Summary Having a project run over 1 week is very neat.There is pressure to get tasks completed however there isclosure because we don’t have to touch it again after thisweek. or do we? We could focus on the group processes involvedinstead of focussing too much on the technical detail of theproject (S)These sessions are good as I have definitely learnt that youcant take things personally. Also I realize that the project istoo big for one person to do properly and hence I have to relyon other people, which I often have a problem of doing as I liketo think (like I’m sure most people do!!!) that I can do thebest job. However, due to the nature of the project and skillsof the group members I realize that this is not the case. Thisis a bit liberating in a sense as I can now solely concentrateon my area tracking, and this is great as I have a welldefined problem and boundary to work within (M)As part of the normal feedback requested of Engineering students during the semester, the4th year students were asked to comment on the Design Week in the light of subsequentexperiences with Studio Learning, providing feedback on the value of the experience. Thestudents were asked:i) if they thought an orientation programme is required for the Design StudiosYES it was important to understand what was required to getthe most out of the Design Studio processNO we are missing out on learning materials for each subjectii) to comment on anything in general that you like in the week339

Figure 8.4: Positive comments made about the Design WeekFigure 8.5: Negative comments made about the Design Week340

good to have intro and reasons behind it (Design Studio); workingwith people outside of my disciplineiii) to comment on anything in general that you didn’t like in the weekno marks allocated for work that was being done and there was1 week less of semesteriv) to make suggestions about such an orientationmaybe could be part of 1st week alongside normal subjects,or extra week tacked in the frontAll comments were categorised and recoded to produce Figures 8.4 and 8.5, and informedthe review of the Design Studios undertaken at the end of semester.8.1.4 Characteristics of teaching and learningChanges in cohort characteristicsFigure 8.6: Requirements Engineering class cohort 2002 - 2005341

The profile of the student enrolled in ENG301 continued to be non-typical. In this offeringof the unit, only one student was enrolled in a BE(SE). Almost every other flavour of (engineering)student was present (see Figure 8.6). In addition, some students were enrolled inthe unit because other options (eg internship) had not eventuated. This had the potential toaffect student motivation.Teacher characteristicsSince from a social-constructivist perspective the context in which learning takes place isviewed as important for learning processes, it was important to take the teacher’s role intoconsideration. The Approaches to Teaching Inventory (ATI) (Prosser and Trigwell, 1999)provides a profile of the teacher but with a focus on a specific learning environment, so thatthe profile may/will differ depending on the unit for which responses are being elicited. Table8.1 provides an indication of the subscales utilised and the scoring achieved in relation to theRequirements Engineering unit (ENG260).Figure 8.7:ATI Scales for conceptual change/student focus and information processing/teacherfocus with their respective strategies (Prosser and Trigwell, 1999)342

Table 8.1: ATI Scales for Conceptual Change/Student Focus and Information Processing/TeacherFocus with their respective strategies (Prosser and Trigwell, 1999)ScaleScoreConceptual Change/Student FocusIntentionAssessment in this course should be an opportunity for students to reveal their changed 4conceptual understandingI encourage students to restructure their existing knowledge in terms of new ways of thinkingabout the subject that they will develop4I feel that it it better for students in this subject to generate their own notes rather than 5always copy mineI feel a lot of time or activity in this course should be used to question students’ ideas 3Total score from a possible maximum 20 16StrategiesIn my interactions with students in this subject, I try to develop a conversation with themabout the topics we are studyingI set aside some teaching time..so that the students can discuss, among themselves, the 5difficulties that they encounter studying this subjectIn this course, I use difficult or unexplained examples to provoke debate 4Time or space is made formally available in this course for students to discuss their changing 4understanding of the subjectTotal score from a possible maximum 20 18Information Processing/Teacher FocusIntentionI feel it is important that this course should be completely described in terms of specific 3objectives relating to what students have to know for formal assessment itemsI feel it is important to provide a lot of facts to students so that they knop what they have 2to learn for this subjectI think an important reason for running teaching sessions... in this course is to give students 1a good set of notesI feel I should know the answers to any questions that students may put to me during this 2courseTotal score from a possible minimum 4 8StrategiesI design my teaching in this course with the assumption that most of the students have 2very little prior useful knowledge of the topics to be coveredIn this course I concentrate on covering the information that might be available from a 2good textbookI structure this course to help students to pass the formal assessment items 3In this course I only provide the students with the information they will need to pass the 1formal assessmentsTotal score from a possible minimum 4 85343

What the summary chart (see Figure 8.7) shows is that from a possible score of 20 points onConceptual Change/Student Focus and its appropriate Strategy, the teacher achieved scoresof 16 and 18. From a best (lowest) possible score of 4 points for Information Transmission/TeacherFocus and its Strategy, a score of 8 for each was achieved. While this suggestssome gains in the researcher’s approach to teaching, more improvement is possible. Althoughthe authors of this instrument do not publish norms, these results provide a mechanism toenable comparison of alignment (between teacher and students and between learning modelsand teacher) – these will be described and discussed later in this chapter. One strategy tobe tried within this cycle is to apply Lepper et al (1993)’s view of scaffolding. They suggestthat the most suitable way to scaffold pupils is not necessarily to tell them what is maximallyinformative. The most helpful remarks, in their opinion, are those that are incomplete orotherwise imperfect. This can help the learner to think more about the problem being solved.Accordingly, the apparently laudable and precise patterns of feedback, correction, diagnosisand demonstration are not representative of what expert tutors actually do. This strategymay be seen as addressing the ‘direction’ tendencies in the teacher.8.2 Cycle 3 – Studio LearningThe decision to modify the CreativePBL environment developed for 2003 was based on reflectionof the findings of Cycle 2 and a revisiting of the literature of the discipline of REand of PBL. This latter suggested that, if the PBL is collaborative, learners are expected toconstruct a problem representation, transferring their internal representation into externalones. Where the problem is ill-structured, learners exchange and negotiate their perspectivesin order to tackle the intrinsic complexity of the problem. This leads to dynamic (to thedetriment of systematic) manipulation of the problem space and eventual problem solving,rather than a step-by-step process that is determined by individual emerging views.8.2.1 Mapping the curriculumAlthough curriculum mapping has had limited use in higher education (attributed to itscomplexity and static nature (Robley et al, 2005)), its value was acknowledged during theReview process of Cycle 2 (see Section 7.2.5) – the map provided a clear indication of alignmentissues, and was a useful resource for alignment of units within the discipline and acrossall engineering specialisations. Detailed maps were therefore mandated for all units to bepresented in Design Studio mode.The curriculum map developed for ENG260 was extended and updated to reflect changes to344

the curriculum. The University assessment policy (Murdoch, 2004a) was also scrutinised toensure the unit conformed. This was a challenging process, since the policy (at that time)recognised only ‘standard’ units (ie those based on lecture/tutorial/lab format) and projectbasedunits (to cover cap-stone and theses). The Design Studio were required to adherevery closely to the University Assessment policy – as a ‘new’ style of unit each Studio wasscrutinised carefully at University level before approval was given. Of particular importancewere the proportions of group and individual assessment. The outcome of adjustments madein order to conform was a more even split between group and individual tasks, and muchless importance placed on the final exam. Most problematic was the focus on groupworkand class participation, and the impact of self and peer assessment. As Elton (2000) notes,traditional teaching in practice is quite often well matched to traditional assessment, bothbiased towards memory learning.A model of alignment, based on the work of the Engineering Subject Centre of the Learningand Teaching Support Network (LTSN) (2002) was developed to assist in the mapping task.It also assisted in tackling the issue of a surface learning focus in student (or, in reality, alesser swing to deep learning than could be expected). Figure 8.8 summarises this model.Figure 8.8: Alignment between outcomes and assessment (adapted from Learning and TeachingSupport Network (LTSN) (2002))The map based on this model indicates that the learning objectives noted in the unit documentationare modified through student engagement with the tasks and activities. Theteacher identifies additional outcomes drawn from this engagement that address generic attributes.If alignment exists, the assessment is based on demonstration of the combinedoutcomes. Within the Murdoch environment, assessment criteria must be frozen in week 1 of345

Table 8.2 for a description of the outcomesseeFigure 8.9: Mapping of ENG260 topics to Engineers Australia graduate outcomessemester. However, the feedback loop ensures adaptation is facilitated for closer alignmentin the next offering of the unit.The map also highlighted any discrepancies: that an element is declared as a learning outcome,delivered as a learning objective, learned as either a new skills or an enhancement ofan existing skill, and summatively assessed.Elton (2000) suggests this mapping process assists in identifying where teachers’ theory inaction differs from their espoused theory through an evaluation of what is designed (methodsor assessment for objectives) and what is practiced. The existence of an exam indicated a misalignmentdid exist in ENG301, but the decision to retain this assessment was a pragmaticone – the School-based Teaching and Learning Committee expressed a reservation in havingno exams in any of the SE Design Studios, and it was a useful benchmarking to be able toevaluate succeeding student cohort performance in exams. Of course, this is only true if themapping can show the exam did not seek to measure how much low level information thestudent can reproduce. The raw exam mark average for the 2003 cohort was at the highend of the range established by previous cohorts. Therefore, while this assessment was notdetrimental to students undertaking a PBL unit, neither did it advantage them (which couldbe considered a negative effect).346

(a)(b)(c)(d)(e)(f)(g)(h)(i)(i)(j)(k)(l)Table 8.2: Engineers Australia graduate attributesGeneric AttributesAbility to apply knowledge of basic science and engineering fundamentalsAbility to communicate effectively, not only with engineers but alsowith the community at largeIn-depth technical competence in at least one engineering disciplineAbility to undertake problem identification, formulation and solutionAbility to utilise a systems approach to design and operational performanceAbility to function effectively as an individual and in multidisciplinaryand multi-cultural teams, with the capacity to be a leaderor manager as well as an effective team memberUnderstanding of the social, cultural, global and environmental responsibilitiesof the professional engineer, and the need for sustainabledevelopmentUnderstanding of the principles of sustainable design and developmentUnderstanding of professional and ethical responsibilities and commitmentto themExpectation of the need to undertake lifelong learning, and capacityto do soSE AttributesAble to apply project management techniques and software developmentmethodologies to deliver large scale software systems on time,within budget and to agreed performance specificationsAble to apply both procedural and O-O development paradigms toall phases of a software engineering projectAble to understand fundamental attributes and components of softwaresystemsThe curriculum map for the follow-on studio, ENG302 showed a much cleaner alignmentacross the dimensions. Peer- and self-assessment, assessment of reconceptualisations, demonstrationand presentations, portfolios and reflective journals addressed the learning moreconstructively. Struyven et al (2005) propose that the learner experience of evaluation andassessment determines the way in which the student approaches (future) learning. Whilethe experience of learning is diminished by assessment methods which are perceived to beinappropriate, alternative assessment was perceived to enable, rather than pollute, the qualityof learning achieved – studies (eg by Sambell et al (1997); Slater (1996) and Segers andDochy (2001)) found that student effort was channelled into understanding when alternativeassessment (including portfolios and peer- and self-assessment) was utilised.Data for the mapping were triangulated using document analysis of feedback from students,SE staff, focus groups and a (very few) interviews with past students. The data gleaned fromall of these activities were then used to map the unit to the generic graduate outcomes ofEngineers Australia (EA), as the accrediting body for the BE(SE) degree (Table 8.2 providesthe legend for Figure 8.9). From this point it was relatively easy to map between EA and347

Murdoch’s graduate attributes, to ensure coverage of these, as well.The curriculum map was then subject to review by the SE academics (in effect the sameparticipants of the Review Team convened late in 2003), and modified as required. Thepurpose of this exercise was to ensure the outcomes of the RE Design Studio would integratewith other Software Engineering Studios.8.2.2 What was plannedOnly minor changes were made to the RE component of the Design Studio: generally to assistin data collection, and to make the group working environment more seamless. However, theaddition of the content of another SE unit had some impact. Except for the area of estimationtechniques, that unit covered in greater depth curriculum elements touched on lightly inENG260, since as the introductory unit to the SE discipline, ENG260 had provided somecontext for RE. So, for example, the Software Development Life Cycle was dealt with briefly,as was a historical perspective to SE as a discipline. Therefore the bulk of the materialintegrated easily into the format existing for ENG260. The secondment environment wasexpanded to include an additional ‘problem’ or task (see Figure 8.10). This acted as atrigger for students to explore the major additional component – estimation techniques.Figure 8.10: Memo to students to trigger investigation of estimation techniquesThe greatest change in the unit occurred in relation to formal assessment. In 2005, this wasbased on• three group assignments (no change in format from the previous offering, but markallocation was modified to 10%, 10% and 15%). As in 2003, each group-based problemrequired use of the products of the preceding problem. In this way, students were348

encouraged to refine their understanding of the problem context and reflect on the implicationsof the feedback from their manager on the success of that prior task in orderto commence the next. Again a small component of the mark was allocated to ‘rework’addressing issues raised in the feedback and amending/updating documentation as required.And again, not all groups chose to undertake the rework – and paid the penaltywhen their increased understanding of the problem context was not well reflected intheir submissions. Where reconceptualisations did occur, these were to be submittedas well, to be taken into account in the assessment• two individual components (the Performance Review as previously described (worth5%) and an estimation planning tool (worth 10%))• a Portfolio (now 20%, acknowledging its value in encouraging deeper learning) comprising,as its major element, the concept maps. This year, however, the maps wereevaluated on a regular basis by team members. This peer review was seen as a mechanismto assist the development of a common understanding of the knowledge beingdeveloped within the unit – simply, as a view into the minds of the students (Freemanand Urbaczewski, 2001). The Portfolio was now both a group and individual effort –including a reading log compiled by the group (but initialed by whoever had written thesummary), minutes of group meetings throughout the semester (with rotating minutetaker)to indicate task allocation and individual activity logs for ‘in- and out-of-class’time spent on the unit• a final exam (now worth the minimum mark acceptable in the Murdoch context – 30%of the final mark) also acknowledging its (lack of) value in encouraging deeper learning.This also modelled that of the previous year, both for the same reasons (ie to maintainsome level of independence from the learning model being applied) and pragmaticallydue to the assessment policies of the School and the University.The Design Studio environment set up by the School also mandated a participation requirement.Although this was not allocated a mark, students were required to attend the Studiofor a minimum time. For ENG301 this was negotiated with the student cohort, and definedto be the first two hours of each 5-hour Studio session, for a minimum of 75% of the semester.No student missed more than two of the twenty six sessions, although they may have onlystayed for the mandatory two hours.In order to assist students in addressing the content, the study schedule which mappedcontent to problems (requested by students in 2003) was retained. The group-work spacewas also retained. An additional feature was the development of a Studio Library. This was,349

in effect a small collection of texts and articles that could be useful during the semester, andthat would normally have been placed into ‘Closed Reserve’ in the campus Library. Thephilosophy behind establishing a collection within the Studio Lab was that students shouldbe able to access resources outside the web environment with minimal effort. These items(twelve books and sundry photocopies) were not to be removed from the lab – they also actedas an incentive for students to spend additional time there.8.2.3 What actually happenedThe floundering that had been a characteristic of the first four weeks of semester in 2003 didnot seem to be an issue this year – students appeared comfortable with the class environment,and were willing to work towards the milestones. Minutes of meetings indicated reasonablework-breakdown (see Table 8.3 which provides sample entries), while activity logs showedmost students were willing to spend time outside of class to achieve the objectives they hadset (Tables 8.4 and 8.5 also has sample entries). Students were able to spend some of thistime within the Studio Lab – as the semester progressed, students were increasingly presentat this time, for group meetings or for individual work. The teacher could also be called inas required.Comments from the Week 4 Year survey support this perceptions – only one negative commentwas recorded (heavy workload, takes up a lot of time) while the majority of commentswere positive ([good]lecturers/facilitators; lots of learning to do; seems well structured). Itshould be noted that this is in contrast to feedback in other Studios running at that time –where comments related to common themes:workloadhaving to learn everything ourselves; heavy workload; takes uptoo much timestructureseems disorganised at times; poor unit organisation; structureis not clear; no set text; would prefer a set process to learn;no lecture notes and structure350

Table 8.3: Excerpts from group minutes 2005Date Time Description of discussion Action by28/2/2005 13.00- Review the project scope and use cases All15.00Interface for the Educational Game EnvironmentAction by Task Completed Task AllocatedAll Interface screens for most usecasesVaughnOrganise the project’s draftSimonPrepare Set Permission screenAlainaPrepare Tracking Progress screenDate Time Description of discussion Action by5/5/2005 9.30-12.30 Continue to work on Class Diagram AllShow our work to Jocelyn and asked for AllfeedbackCreate a conceptual modelDiscussion with JocelynPlan meetings that would be done beforethe assignment’s dueAction by Task Completed Task AllocatedAll Review of the Conceptual Modeland Class DiagramAll Consider possible improvement inthe two ModelsDate Description of discussion Action by22/3/2005 Review last minutes, confirm they are correct AllDivvy up the reading logRoyPlan when next meeting will take place (out of classtime)To be Thursday 24-3-2005Plan what needs to be done and when for requirementsProduce some initial use case diagrams to develop MikeDermotDevelop the use casesMikeDermotRoyProduce an initial Context DiagramRoy351

expectationstoo much learning too quickly; time seems to limit opportunitiesfor feedback; hard to gaugeexpectations; unclear re outcomes; indirect assessment questionsgroup workgroup work difficult; poor student participation (by some).These are themes that were very evident in Cycle 1 and 2 of this study, suggesting theimportance on the development of both experience (for staff) and support structures (forstudents).The staff comment from the Year 3 co-ordinator 2 confirms thisthe first assessments reflect the novelty of the Designapproach to learning for the students. Adjusting to theexpectations of the studio format seems to be the greatestissue. Setting patterns for getting work in and feedback to thestudents in a timely way seems to be the main point that couldbe addressed. I expect that by the second round of surveys manyof these issues, especially related to workloads will have beenaddressed as the students will have spent more time in theDesign Studios.Table 8.4: Key for activity logs ENG301Task KeyDiscussionConsultationResearchDesign workReview & testingPersonal ManagementGroup ManagementDescriptionTime spent discussing issues internally within groupTime spent discussing issues with facilitatorTime spent researching new areas of knowledge to assist withengineering of projectTime spent on formulating and describing design solutionTime spent on reviewing the design solution and/or testingitTime spent on individual management, documentation andrelated tasksTime spent on project management related tasks for thegroup as a whole2 this researcher is the Year 4 co-ordinator, therefore her summary is disregarded here352

Table 8.5: Excerpts from activity logs ENG301Date Student Key Description of Time ReflectionDiscussion01/03/05 Simon Disc Introduction of GroupMembers14.30-17.00InterestingGroupCon Research on SoftwareMembersFactory on Scope forStudent SystemCon Discussion on ScopeCon Writing Document28/03/05 Simon Rev Review the projectscope and use cases13.00-15.30Des Interface for the EducationalGame Environment05/04/05 Alaina Con Review of project 13.00-14.3012/05/05 Dermot GM Finished off Assignmentfinals, and finalPresentationRes Researched activitydiagrams25/04/05 Simon Res Reading PersonalPlanning Tool17/05/05 Dermot GM Time spend doing Assignmentas a groupRes Researching diagramsfor assignmentCon Talking to Jocylin abtassignment and requiredspecifications23/05/02 Alaina Res Research RequiredReading30/05/05 Vaughn GM Worked on class diagramand sequence diagramWe all need to read upon this setions to gaina better understandingof Class diagramAlso developed a listof questions to ask JocelynThese Interfacesare interesting,especially withvisioStudy the othergroup’s requirementmodel2.5hr Time wasn’tspent thatproductivelyHowever workwas done aprox50% of the time16.00- That’s goanna21.00 be a lot of work2hr Time spent well,going throughtasksSome time wasalso spent talkingabout jobsand the future13.45-14.4509:00-11:00importance ofignorance in REFound that wewere unsure on alot of aspects ofthis sectionEveryone contributeto thediscussion andthe attemptsvery good eventhough they didnot produce thedesired result353

Although the class sessions within ENG301 proceeded very smoothly, the group dynamicswere very different to those of previous years. One group worked very well together, andhad set themselves high targets to achieve (together). The following are responses from thatgroup to the request to Provide a brief comment on how well your team worked together inthe Performance Review:Simon Our team has worked together very well. We always solved theproblems together and everybody was included into the solving ofthe process. I think we also developed a team spirit during thissemester. Otherwise it would not have been possible that we hadworked on the last project till 4 o’clock in the morning. Alsoit was very interesting to work with people from differentcountries, even though there were some technical languageproblems with our computers. Of course because our differenthome countries and languages we had sometimes small difficultiesin understanding each other. But because of our team spirit italways has been possible to get to know each other of our pointof views. And additionally we could use the different strengthof each team member and use it for everyone’s advantage and thegoal of the group which was the development of ourspecificationVaughn A team environment can be seen to in two dimensions e.g.the task and the social interaction. There was good socialinteraction within our group i.e. we enjoyed working togetherand everyone made sacrifices to get the job done to the standardexpected e.g. working into the morning due to the inability towork on the project earlier due to networking failures. I notedwe where laughing a lot as we worked together, I interpretedthis as another sign we were working together well. We also ateout together during meal breaks in the project e.g. dining outat McDonalds (more times than I had been to McDonalds in mywhole life). I believe that the willingness to share food wheneating out is an indication of how well a team respects/trustseach other.One group member felt more could have been achieved with a tighter group:354

Alaina Although each member achieved allocated tasks, we did notspend time to review and convince ourselves what a member didbefore we submitted assignments. We should have had morediscussion about the reason why the person implemented his/hertask in some way. All members would have benefited from it as itrequires more knowledge and understanding to explain what wasdone in a task and why. It would have reduced the amount of theredundant diagrams or features. Furthermore, it should havereduced much mistakes and provided more consistency in ourrequirements specification.This group acknowledged the value of the ‘rework’ component of each group task:Vaughn We have to apply learning’s to a realistic problemwhich means it moves as out in the real world e.g. the lecturer(TerColl) pointed out errors in thinking and this resulted in ushaving to revise what we had completed previously in order tomove to the next step. I found this gave me a greater depth ofknowledge than the usual do an assignment get some of it wrongand move on to the next usually non relatedassignmentThe other group tended to allocate tasks to be completed individually, then integrated.These students chose not to take the opportunity to rework their models – consequently theirmarks tapered down as the semester progressed, and they did not leverage from a deeperunderstanding of the problem (which was demonstrated in class, but not in submitted work).However, members of the group acknowledged this:Roy Inspiration in the team was low. Projects were mostly driven bydeadlines rather than producing a high qualitydocumentMike I believe our team worked fairly well. We are all ratherindependent learners so most of the time we split the project upinto individual sections and then did them on our own. [Roy] hasthe highest work ethic out of us three and therefore took themost work upon himself. [Dermot] and I probably relied on himmore than we should have. This meant that Roy had the greatest355

understanding of how our assignments worked and then we wouldhave to try and work everything in with his process. Roy and Ihad a couple of minor disagreements over how a particularfunction worked but we were not able to find concrete evidencesupporting one or the other so we ended up using [roy]sideasDermot We worked well to compliment each others strengthsand weaknesses ( i.e. ROY = strengths Me and Mike = weaknesses:). Both Mike and I learned a lot from Roy over the semester andI believe that he may have learned a little bit from us (thingssuch as patients, anger management, and youthfulness). For themost part however there were little disagreements and we allworked together. I felt that I dropped the ball a little bit inthe second assignment by not reading up well enough on thesubject and therefore tried to make up for it on the finalassignment.Although the class atmosphere was generally positive, some students had problems withmotivation. As noted previously, two of the cohort had been told to enrol in the studiowhen internship opportunities did not eventuate. Mike, in particular, had motivation issuesthroughout the semester:Mike I am not a huge fan of Design Studios and do not think that thestructure works for me. My view of this unit has beendiscoloured by my disappointment in not getting an internshipand also being forced to do this unit rather than choosing to.This view has been difficult to break out of and I’ve struggledfor motivation with this unitLooking at the amount of effort Mike put into the unit confirms this attitude: the activitylogs indicated he devoted 66 hours to this unit over the whole semester – very little more thanthe mandatory two hours per session. By comparison, the most time recorded was Alaina’s210 hours. This comes close to the 260 hours expected in the Design Studio model proposedwithin the School.356

8.2.4 Interpreting what happenedAs for previous cycles, the purpose of the evaluation conducted after the 2005 semester wasto assist in determining if the implementation was effective in facilitating student learning,and to provide the researcher with data to reflect on improvements to be made. Table 8.6provides a summary of the instruments used to collect data for analysis in this cycle.Table 8.6: Instruments applied to Cycle 3Cycle Diagnostic Devices Records FeedbackLearningStylesTeachingStylesCycle 3 LSI ATI Assessment items Year Survey& results2005ILSPerformanceStudent records ReviewASIUnit redevelop SSURoLIrecordsDesignInteraction Curriculum Map OrientationSchedulesFeedbackMurdochActivityGraduateLogsAttributesEngineers AustraliaGraduateAttributes1996;2005Success of the learning modelIn 2005 success of the learning model could not be assessed without consideration of the totalenvironment in which the students were immersed.The ENG301 students themselves acknowledged this. They noted that with all their studiesundertaken within a Studio environment, they felt they were much more in control of theirefforts. Probing of this concept within a focus-group environment indicate the following:• students felt academic staff were more tolerant of the needs of other Studios• with a full-time load of only two Studios student time was not as fragmented acrossdifferent areas• except for the (negotiated) compulsory attendance, students could vary the time theyspent on each Studio in response to their total learning context. It was the team’s role357

to ensure tasks were on schedule. They concluded that this flexibility reduced stressand allowed them to focus on the learning they needed to achieve for the task.These perceptions were confirmed in student comments in the University survey (SSU), referringspecifically to ENG301:Student A This unit teaches a process that is built on knowledge but moreimportantly that knowledge is converted to a skill via practiceon the problem. I don’t believe this is achieved by the otherstyle of teaching e.g. lectures and exercise typeassignmentsStudent B These design studios are a formalisation of what isoccurring naturally i.e. we learn from and work with each otheralreadyStudent C This method of teaching has provided me with a frame workthat I can use to identify future problems and developsolutions.The approach initiated in 2003 was continued this year – as teacher, I was present duringthe class time, and could be called on as required. Again, this seemed more effective withthis group – either I was more comfortable with the facilitator role (very probable) or thecohort (being a year further in their studies) more relaxed in that atmosphere. I suspectboth perspectives to be relevant. Their ASI scores (described a little later in this section)also suggested this cohort would be more amenable to this style of learning.Vaughn The lecturer spending the agreed allocated time inthe class room has been very useful i.e. we have been able tolearn at a faster rate because we have been able to consult withthe lecturer when we where unsure i.e. the lecturer became amentor/consultant who suggested and guided rather than justgiving being a lecture/guruSimon I think structure a course this way is a very goodidea, especially for foreign students. You learn more Englishduring a project or during working in a group than in a lecture.Also in this structure it is possible to ask questions wheneveryou want. In a lecture it is not always possible. Additionally358

the students have the possibility to increase theircommunication skills and how to deal with a problem within agroup. Another advantage of this Course has been the smallgroups which have increased the benefits of the points mentionedabove.Because both groups decided to work in the Studio Lab (rather than finding other workspaces during class time), whole-of-class discussions were spontaneous and occurred morefrequently than in previous years. However, some students would have liked more:Alaina There could have been more time to discuss a topicin whole class. As we are divided into two groups, I felt I wasseparated from the other group and it would be better if therecould be more opportunity that enables to share information withthe other group’s membersDermot I would be inclined to change the way the tutorialsare conducted, to provide one day maybe for teaching theconcepts, and the other day for applying the concepts, and doingexamples.The relationship between motivation and group dynamics could clearly be seen in this cycle.With the increase in the number of assessment items, and in the percentage of marks basedon individual effort, the Portfolio ceased to be a clear indication of motivation, although itshould be said that the only student to fail this component was the explicitly noted leastmotivated student. With his fail, individual marks within that group ranged from 6.5 to 11.5(out of a possible 20 marks): the other group’s mark for this item ranged from 14.25 to 18.25.This raises the issue of group dynamics:Vaughn [...]the biggest problem I see with designstudios is the inability of groups to deal with members that arenot doing their share of the work or aligning their work withthe group.The reason I am pointing this problem out is purely selfish.don’t want to get stuck with freeloaders or some domineeringperson that wants to do it their way and only their way, inI359

future design studios. [...] While I understand that it is hardfor a lecturer to know of or act upon the problem of freeloaders or dominating personalties, if it is not identified. Itis also hard for those students doing the work to resolve theissue them selves i.e. while this University teaches technicalskills it has not taught soft skills especially the soft skillsneeded to motivate and manage peers.Currently the options open to students to deal with members notdoing their share of the work includes:Threatening them with eviction or evicting them. Evicting agroup member is whistle blowing which is counter cultural to theAustralian way i.e. we are brought up not to be dobbers. Leavethe team and work alone which results in the individual who wasdoing the work having to do even more work because they anindividual competing in a team event.Neither of these options is fair on student(s) who are doing thework and it treats the symptom not the cause of the problem i.e.the inability of members in the group to motivate all members towork together for the benefit of group members.This suggests that all Studios require this issue to be addressed – implying it should be raisedduring the Design Week, and alternatives provided for students to access.For the discipline of RE the Studio Learning model appeared to hold the most promise –students had been provided with a process during the Design Week, and were able to applyit in the unit, but also adapt or discard it as they perceived necessary. In this way, theenvironment supported both behaviours, and hence both the discipline and the opportunismnecessary for RE. Robillard (2005) notes that in a typical opportunistic problem-solvingactivity, individuals spontaneously rely on teammates to provide missing information. Thestudent data indicates this is occurring (each of us shares the ability or outcomes from theability e.g. ability to research (find information/knowledge), understanding a problem thatothers don’t and interpreting it into a context the other can understand so they can solve theproblem; some of the knowledge I have learnt has resulted from the interaction with my teammembers i.e. I don’t believe the level of understanding I now have, would have been achieved360

y working on the assignments by myself).They perceived the environment (and the teacher) as non-threatening (I personally believethe unit was excellently conducted as per how it was constructed to be), and were generallycomfortable with the environment established (suggesting changes is rather difficult as thecourse is designed for self-teaching with supervision; maybe, sometimes it would be not uselessto include a few training sessions (like a lecture) in the project. This would assure thateveryone’s has a based understanding of the actual content. Of course, this would lead to thedisadvantage that the students don’t learn things on there own. In conclusion I would liketo say that I’m not quite sure about this), though some students would have preferred moredeadlines imposed (while I kept up with the readings and assignments I ended up gettingbehind in creating the mind maps and logging my personal journal. While I acknowledgethat is my responsibility to manage my own time it would have been useful if this task waschecked more frequently). Some students also saw the value of reflection on their progress(the personal journals can be a useful tool to ensure the unit learning’s are integrated intothe students existing knowledge/experience and that the students behaviours are modified toalign with those required to be a successful engineering graduate.[...] Weekly checking of theweekly journals would be useful in ensuring they are being used as a tool to guide studentdevelopment into the higher levels of Bloom’s taxonomy and the non technical aspects of theirlearning).However, the high Meaning Orientations of some members of the cohort is a likely indicatorof their ability to exploit the environment to their benefit. So, in this cycle, the learningmodel aligned well with the discipline and with the characteristics of (most of) the studentcohort. This suggests a question for future study – should student characteristics be used asa determinant for engagement with a learning model (or in fact with a discipline)? As anaside, two students in this cohort decided to switch specialisation to SE after this semester.A Department-wide review of the Design Studios was undertaken at the end of the firstsemester offerings. Feedback was sought from all academic staff involved and student representativesacross all Design Studios, and discussed during a 1-day workshop. As the mostdeveloped of Studios, ENG301 acted as benchmark. The conclusion reached was that, whilerefinement was needed - for many staff this was the first implementation of non-traditionallearning environments, and a need for student feedback aligned to the learning environmentidentified, Design Studios had been successful. A decision was also made in that forum tocontinue the orientation programme (Design Week). Students commented positively on theprovision of an introduction and rationale to PBL and Design Studios (46% of commentsreceived (n = 33)) and to the value of working in interdisciplinary groups (30% of comments361

eceived).Impact on student developmentThe student cohort undertook the unit successfully: notwithstanding a reduction in importanceof the final exam, a statistically significant increase in marks across all components ofthe exam was noted, with the exam modelling previous offerings intentionally. The averagesfor both the raw exam marks and the final mark showed a marked improvement, as indicatedin Figure 8.11. While this could be seen to indicate the exam fulfilled its role by notmeasuring how much low level information the student can reproduce – if that was the case,Woods (1996b) suggests the end result is predictable - poorer performance when comparedwith students who have had experienced a more didactic approach, the number of studentsin each cohort has enormous impact on these figures.Figure 8.11: Comparative exam marksHowever, it should be noted that this cohort indicated higher scores in the Meaning Orientationcomponent of the ASI (MO ranging from 0.69 to 1.5 higher than their RO). Only onestudent exhibited a small (0.19) bias to RO.362

Despite this pleasing outcome, some students were not happy that an exam existed at all.Although the comment below was made before the exam was sat, Vaughn’s assessment of hislikely performance was quite accurate:Vaughn I do well in assignments and not so well in the exams.This system favours those who do well at exams i.e. they get toshare in the results from me working hard on the assignments butI don’t get to share the results of their ability to do well inthe exams. This will results in my grade average moving downwhile theirs moves up and that from my point of view is notfair! If these design studios are to be fair then all theassessment should be from group work i.e. no exams. If this isnot going to the case i.e. exams remain not then I would preferto go back to the old system i.e. working alone. Even it meantworking alone when everyone else is in a group.Effort had been made to integrate other tasks into the learning environment. This still causedsome problems – students considered these peripheral to the problems being tackled, eventhough these other tasks were part of the learning objectives of the unit:Dermot I think we should delete things like the reading journal,and some of the individual tasks that are a bit ridiculous, suchas the mind maps, 38 mind maps are a bit harshSimon Maybe, it is not necessary to have many organizing task, asfor example the reading log or the activity log, to do in theproject. I would not delete it, but maybe it would not a quitebad idea to decrease the amount of workload of doing organizingthings and, also, increasing the amount of learning new methods,issues or other facts of software engineering content. Of courseit is essential to know how to write a reading log or activitylog but is it essential doing it all the time? On the other handyou are only be able to see the advantage of a reading oractivity log at the end of a project and if you have done it allthe time during the projectMost telling are comments made in response to the question regarding individual studentperception of their learning in this environment:363

Simon In my opinion I learn more communication skills andin organizing and less in technical skills in this unit. So inmy opinion I learned neither more nor less in this unit, butdifferent things which I havent learned beforeVaughn Seriously I feel I have learnt a lot more useful things in thisunit compared to most of the other units I have taken at thisUniversity -- I am learning more, much more for reasons thatinclude:I have been working in a very good team and feel that some ofthe knowledge I have learnt has resulted from the interactionwith my team members i.e. I don’t believe the level ofunderstanding I now have, would have been achieved by working onthe assignments by myselfThe assignments being based around a problem gives a morerealistic context as opposed to some abstract exercise to testunderstanding of theoryI have found that the assignments have been an extension of theprevious one other and clearly a process that is being builtupon at every stage i.e. each additional stage in the processhas enlightened me to the relevance of the previous stage.This method of teaching has provided me with a framework that Ican use to identify future problems and develop solutions. Ihave noticed that the design studios require a lot more workfrom me than if I was working alone. For example I have to spendmore time working on problems because of the extra overhead ofworking in a team (meetings and social interaction). There isalso the need to do extra research to gain information that isnormally just handed out in a lecture. However I don’t mindputting in the extra effort because I feel the extra effort isworth it because I feel more confident that I do know thematerial (not an impostor) and can apply it to futuresituations364

Alaina I felt I have not learnt adequately because I couldnot manage my time effectively. However, the context of thisunit was very interesting and the amount of workload was notheavy, I believe I could have learnt much more if I couldorganise the study more successfullyMike I think I would have learnt more from this unit if it had ofbeen run in the normal way of units as that is how I am used toUniversity Units being run. Doing the unit I can see theimportance of planning and project specifications but it justdoesn’t interest me greatlyDermot I personally feel I learn less, I guess this is notmy style of learning. It is as good as me taking a unitexternally and just staying at home and teaching my self, and ifI have problems asking a friend, or researching further. I guesshowever teaching your self things you do tend to understandconcepts better. However I feel that I am an audio visuallearner, thus listening to someone explaining the concepts,PowerPoint’s and teaching it to us makes life easier for me. Ibelieve I gain a better understanding in this wayRoy More than last I studied this material, given that this timearound I have had some experience with an OO language, whereaslast time I did not and needed to put in a great deal of effortto understand concepts without programmingexamples.In particular Dermot’s comments support the findings of Entwistle and Tait (1990, 1995).They found that students who reported themselves as adopting surface approaches to learningpreferred teaching and assessment procedures which supported that approach, whereasstudents reporting deep approaches preferred courses which were intellectually challengingand assessment procedures which allowed them to demonstrate their understanding.8.2.5 Reflection on findingsReflection on ENG301 as a component of the Action Research reported in this chapter, basedon data collected throughout the semester, led to the following comments.365

Curriculum - changes to the RE component of the curriculum were minimal, effectivelyfine-tuning to achieve closer alignment with the curriculum map developed. One significantbenefit was achieved through the development of the Design Week – having anorientation to the learning model outside the unit allowed for the focus within the uniton learning in the context of the discipline. No changes are identified as required forthe next offering of the unit (other than usual updating of material)Pedagogy - the PBL literature (eg Albanese and Mitchell (1993)) indicated students takeup to four weeks to come to terms with the (PBL) learning environment. This hadbeen confirmed during previous cycles of this study, but was not evident here - studentswere very confident in taking control of their learning, transferring knowledge they hadgained during the Design Week in order to tackle the material in this unit. Studentsalso appeared more willing to work with their learning diagnostics results and buildlearning strategies based on these. This suggests students were able to place themselveswithin Stage 3 of Grow (1991/1996)’s model for life-long learning. An examination ofstudent interaction in subsequent units is required to confirm (or not) these impressions.Students did, however, express dissatisfaction with some of the assessment, perhapsindicating a higher than expected awareness of what was detrimental to their learningstrategies. This mismatch has been seen to lead to student schizophreniaInstructional - the block-teaching format proved very successful. Attendance was generallygreater than 95% - students exploited the group discussion and consultation times thesefacilitated. It was noticeable that group members also accommodated those studentswho preferred to spend considerable time working alone. The group negotiated worktasks so that the additional time required by the Studio could be either individual orcollaborative.Vaughn’s comments in response to the question Do you feel there are any good things abouta unit structured in this way?, quoted here in their entirety, act as a summary of studentperception:Vaughn Yes I think this design studio is very very very(Did I mention very) well run. The problem (Development of thegame environment by Mursoft for TerColl) covers all the learningoutcomes. We have to apply learning’s to a realistic problemwhich means it moves as out in the real world e.g. the lecturer(TerColl) pointed out errors in thinking and this resulted in ushaving to revise what we had completed previously in order to366

move to the next step. I found this gave me a greater depth ofknowledge than the usual do an assignment get some of it wrongand move on to the next usually non related assignment. Thelecturer spending the agreed allocated time in the class roomhas been very useful i.e. we have been able to learn at a fasterrate because we have been able to consult with the lecturer whenwe where unsure i.e. the lecturer became a mentor/consultant whosuggested and guided rather than just giving being alecturer/guru.8.3 Beyond REThree students from this cohort enrolled in the second SE Design Studio (ENG302), as did onestudent from the 2002 cohort. In terms of the learning environment, that student (Markus),while not participating in the ENG301 Studio Learning environment, had undertaken theDesign Week orientation. It should be noted that the basis for this next SE Studio was theoriginal SE unit to apply PBL (that implementation is described in Armarego (2002)).The aim of this phase of Cycle 3 (Cycle 3b) was to observe changes in student interaction,both in the group environment and with the teacher. An independent observer was engagedover several sessions to log the nature of transactions undertaken within the class. Althoughthis interaction schedule were only used sparsely due to its resource hunger, it was able toprovide significant insight of performance during the learning session. As each class lasted aminimum of four hours, and occurred twice a week, this had the potential to generate a vastquantity of data. Students were also required to maintain an activity log, and encouragedto use it as a journal as well. The entries were used as a mechanism for student voice to beheard, and provide insights into their approaches to the learning environment.In addition, students completing ENG301 were asked to complete the RoLI instrument. Thiswas applied as a trial and the results examined to see if it would enrich the informationalready collected through other learning style inventories.The context for this unit is that students work, in teams, to specify, design and develop a systemfor an external client. As well as satisfying all the criteria for Studio Learning (includingsufficient complexity to permit a dynamic design space; multiple acceptable solutions) theproblem had sufficient ‘length’ to require good project management. Interaction with clientswas also a feature, and the problem chosen was outside the student or teacher’s expertise.This meant students could not rely on the teacher to solve application domain issues, but367

equired them to either consult with the client (who was only minimally available) or findout for themselves.From the perspective of the students undertaking this unit, the work environment was familiar– the group work environment, location, teacher and learning model were all basedon ENG301. Students were required to undertake reflective activity logs, minute meetingsand rotate the role of project manager (who both set the milestones and scheduled the workfor that milestone). It was the cohort’s decision whether a final exam would be scheduled– it depended on their ability to present and demonstrate a product by the exam period,with a decision required by Week 10 of semester. An exam has not been set since this choicehas been provided to students – it indicates (and the activity logs confirm) strong intrinsicmotivation.8.3.1 ENG302: students as advanced learnersAs noted above, data in this phase was based on collection of mainly qualitative data –students completed a reflection at the end of each week, attached to a chart of activitybreakdown. In addition, an interaction schedule was completed for several sessions. Althoughnominally quantitative, in fact recording of interactions is based on the observer’sinterpretation of the action. These are discussed in the next sections.Interaction SchedulesFigures 8.12, 8.14, and 8.16 provide the data of student interaction over three separate Studiosessions (see Table8.7 for a more readable list of the interactions logged).Figure 8.12: Interaction schedule ENG302 session 1The first session logged has team members constructing their understanding of the problemenvironment. Figure 8.13 provides a visual representation of the coded interactions.In general students worked individually, accessing resources online and in texts, and ‘touchedbase’ with the lecturer and with other team members only intermittently. Of the totalinteractions logged (n = 91) 22% involved the teacher. Over 48% of the interactions hadsome students working individually, while only 4.4% of the interactions can be considered368

Figure 8.13: Interaction breakdown ENG302 session 1Figure 8.14: Interaction schedule ENG302 session 2Figure 8.15: Interaction breakdown ENG302 session 2369

‘whole of group’ work (which may include the teacher, but requires no one to be workingindependently). [Note within the group it is possible for some students to be working togetheror interacting with the teacher while others are working individually].Session 2 (see Figure 8.15) is chronologically the next Studio session. Here the change togroup interaction is noticable. Of the total logged (n = 66) 27% involved the teacher, andless than 8% was individual. The bulk of interactions involved members of the group withor without the teacher exploring and questioning the understanding they had constructed inthe previous session (over 68% of interactions are classed as ‘whole of group’).Session 3 (see Figure 8.17) occurred some time later, and is based on students coming togrips with a new area of discipline knowledge. Here the bulk of interactions were focussed onstudents questioning (16%) and discussing (19%) with the teacher, and the teacher explaining(17%). Of the total interactions logged (n = 116) 51% involved the teacher, 23% wereindividual and 47% was whole of group work. Table 8.7 provides a summary of the raw dataacross the three sessions observed.Figure 8.16: Interaction schedule ENG302 session 3Figure 8.17: Interaction breakdown ENG302 session 3370

Table 8.7: Summary of interaction schedule dataInteractionPercentagesSession 1 Session 2 Session 3Teacher explaining 3.4 13.6 17.2Teacher demonstrating 2.2 4.5 0Teacher questioning 2.2 3.1 5.2Teacher checking 5.5 1.5 9.5Students discussing with teacher 8.8 6 18.1Students working individually 48.3 7.6 23.3Students discussing together 16.5 39.4 4.3Students questioning 4.3 10.6 16.4Students explaining 8.8 10.6 5.2Taking a break 3.1What the interaction schedules indicate is a willingness on the part of the students to varytheir behaviour based on the specific needs of the learning situation, calling on the teacheronly as required. It should be noted that the value of this instrument and the data collectionit enables would be enhanced if the granularity was finer (eg identifying individual groupmembers and their specific interaction patterns). Within the context of this study ethicsapproval did not allow for video capture of any sessions.Activity logStudents were asked to maintain a record of their activities in relation to this Studio (ENG302).This included any task which could be coded against the keys provided (see Table 8.8) atany time (not just during class sessions). Each week the project manager was required tocompile a report drawn from the activity logs, providing a cumulative chart and highlightingany issues identified.The purpose of this task was to provide some empirical data against which to map studentperception of their workload. Interestingly, this issue did not appear during the unit,although, in fact, the workload was set for approximately 20 hours per week. As the cumulativegraph below (see Figure 8.9) shows, most students spent the appropriate time on thisunit. This compares favourably with the data from ENG301 – there, for the same studentsthe range was from an average of 16 hours per week to less than 10 hours. This suggestsseveral interpretations:• students were motivated to spend the time – this could be due to the project beinginteresting (not likely – some students grumbled about the problem domain. Howeverlack of interest did not appear to reduce motivation)371

Table 8.8: Tasks and keys applied in activity log ENG302Tasks KeyCorresConsultProjectReadingExercisesReviewResearchIndvManProjManOtherDescriptionTime spent discussing issues internallyTime spent discussing issues with JocelynTime spent working on specific components of theprojectTime spent reading course materialTime spent performing course exercisesTime spent reviewing past learned knowledge to assistwith engineering of projectTime spent researching new areas of knowledge toassist with engineering of project, external of coursereadingsTime spent on individual management related tasksTime spent on project management related tasksTime spent on any other task related to the unitTable 8.9: Total student hours for ENG302Student TotalHoursWeeklyAverageMarkus 399:35 23:30Alaina 344:39 20:20Dermot 186:09 10:54Vaughn 306:40 18:00• group dynamics – each team member took on the role of Project Manager. While thisaffected individual time, it did not appear to influence the group time spent• ‘ownership’ of the problem and of the assessment decisions.This last suggestion appears to be the strongest interpretation – students indicated throughoutthe semester that they would not sit an exam, and consequently needed to completethe project to demonstration stage. Excerpts from the activity log reflections support thisinterpretation.What is interesting is the comparison of Dermot’s figures – in this unit he hovered around11 hours per week – in ENG301, he peaked at 10 hours in one week. It would appear fromhis reflections, that a leadership role had a strong influence on his motivation.Figure 8.18 shows the time spent on each category of task across the semester, by student.On a weekly basis, students were able to review their work/learning patterns and comment onthem as journal entries within the activity logs. Figure 8.18 indicates the variety of studentapproaches that were catered for in the Studio Learning model. Markus, for example, spentmost of his time on project related activities, with comparatively less time on other activities,including the three Rs of reading, reviewing and researching. Alaina, on the other hand, was372

Figure 8.18: Cumulative student hours for ENG302more comfortable with the three Rs – she spend the least time of the group on the projectitself, but often acted as scribe and researcher for the group. Of course, what this chart doesnot indicate is the effectiveness of the time spent.A summary of a weekly log is shown in Figure 8.19. This confirms the different learningapproaches of the students. At this time Markus expressed concern at ‘remembering’ previouslylearnt material (ie that of ENG260), and spent time on review and exercises. Vaughnspread himself across multiple tasks – as Project Manager at this stage, he felt a need to beon top of it all.Issues raised by the Project Manager tended to be technical (eg The delay of submissionof the Requirements Specification caused further delay of starting the Architecture Models).However, some comments could be reflective:Alaina The task required individual study and research,which appeared to have been managed by every group member. Anissue was little time was spent on group discussion due to thelarge workload on individualsDermot Although most tasks had been completed by individuals,it was valuable we had time to compare work and clarify how the373

Figure 8.19: Cumulative activity log for week 3system worked before submission of the finalreportMarkus Due to issue working within the Java programmingenvironment the class development is progressing slower, thoughthe coding of the packages is still developing. This slowerdevelopment is slowing the overall progress of theimplementation stage. Also pressure of end of semester anddemands of other units is making time management difficult forgroup members.Given the importance of perception of workload on learning orientation, it was interestingthat students were eager to comment on the amount of time they were spending on thelogged activities (sometimes to justify lack of output!). It became one of the items of groupdiscussion as well as individually with the teacher.Self and peer assessmentIndividual student perception of their own work in comparison to other team members wasalso collected. At the end of each component of the project (defined as output from theproject for assessment) students were required to complete a self-/peer-assessment form.Each team member submitted a rating, which is used to calculate an adjustment factor forthe group mark assigned by the teacher. The students appeared to have no inhibitions aboutrating their team members’ contributions (or lack) higher or lower than average, with each374

student achieving at least two rating of higher than average. Interestingly, a higher thanaverage rating did not appear to correlate with either specific team roles (eg that of ProjectManager) or project phases. However, cultural backgrounds appear to influence the selfassessmentcomponent of the instrument – the usefulness of this metric therefore requiresfurther investigation.Student reflectionJournal entries were based on consideration of the following questions:what did I achieve this week?what issues did I have this week?what can I do next week to address these issuesin the areas of learning, personally and in the group. Students were also able to make generalcomments.Appendix C contains verbatim entries from these journals. The entries reflect the dynamicnature of the learning experience. As an aid to reading these narrative, it should be noted thateach student took on the role of Project Manager for a 4-week period during the semester.The order was Vaughn, Alaina, Dermot then Markus.These extracts provide significant insight to the students’ perceptions of their learning environment.Despite the numerous comments regarding time management, workload was rarely,if ever raised as an issue. Students also appeared highly motivated. Although presenting aworking demo of the problem/system required much more effort, and is group-based, eachcohort has decided to dedicate the (extra) time required to produce a working system. Theimplication of this decision is students’ ability to gauge the level of proficiency of their attemptsto master the problem and complete the task: they appear to be drawn actively intothe problem and learning environment, suggesting ‘real learning’ is occurring. This alignswith Stage 4 of Grow (1991/1996)’s model for life-long learning.Vaughn’s reflections focus on time – he often felt that it was not being used effectively,either through personal issues (need to concentrate on the important stuff) or due to lackof group cohesion (team is not working togeather well). He exhibited a tendency to ‘blame’others (either circumstances or team members) for lack of progress. His greater expertise inmanagement support was exploited at the commencement of semester. However, this wasnot reflected as confidence in his performance in the group later – he felt quilty that I wasnot able to help ... more.375

Alaina is the most focussed member of the team – both to task understanding and to thelearning environment. Her comments reflect a growing ease with the group environment – awillingness to learn from them rather than only from her own effort.Dermot’s comments are most interesting when compared with his approach in ENG301.There, he was happy to work at the minimal level required, here he expresses concern aboutbeing a good piece of work. Whereas Alaina was most fearful of taking on the role of ProjectManager (and at one stage was in tears because of her lack of confidence in her abilities),Dermot thrived on the challenge (Management seems to be easier than sitting back and waiting...).Markus started out with a lack of confidence in his own previous learning (don’t want to letthe group down), but was very soon able to draw on that (feeling more confident on workingwith requirements modeling for the project). He shone in the latter part of the project wherehis wider expertise could be used to the group’s benefit.These journals show students were highly motivated to complete the task to demonstration/presentationto the client level. Although some suggestions have been previously made,further study is needed to explore why this is so. Weiner (1992) suggests that general studentattitude towards the controllability of the learning outcome (ie externally dictated and beyondstudent control, or within the internal control of the student through effort and personalinterest) may influence their motivation and the level of achievement in the learning process.ENG302 exhibits this characteristic – students decide what form the final assessment willtake. Dweck and Elliott (1983) and Dweck (1986) also suggest that if the student has alearning goal orientation (ie views intelligence as incremental), motivation is increased towardssituations which offer opportunities for increasing competence or intelligence. ENG302provides such opportunities. If, as the studies of Jones et al (1993) indicate, a correlationexists between student theory of intelligence and study habits, then this aspect of studentlearning behaviour should be explored.More interestingly, as students rotated into the role of Project Manager, they (individually)applied what they had previously learnt with regards to learning strategies and approachesto study in order to motivate their group members. A good (observed) example of this wasVaughn’s approach to managing Dermot, who exhibited tendencies towards ADHD (AttentionDeficit Hyperactivity Disorder), by keeping him well occupied for the duration of his(Vaughn’s) management stint!376

8.3.2 Approaches to studyingSome researchers in learning styles and study orientation suggest a student’s profile shouldalways be contextualised – a student will approach study for each unit differently, dependingon the demands placed on them by the learning environment, their previous experience andpersonal stance at the time. This section examines this concept by comparing the data ofthe Approaches to Study inventory previously applied to the RoLI instrument developedspecifically to address the context of student learning.ROLIStudy orchestration looks at individual students in relation to their learning environment.The concept was introduced by Meyer (1991) as a ‘contextualised study approach adoptedby individual students or groups of students’. Research undertaken in Finland (eg Lindblom-Ylänne (2004)) has shown that coherent or dissonant study orchestration is developed throughTable 8.10: Summary of RoLI subScales (adapted from Lindblom-Ylänne (2004))SubScaleExampleIND Thinking independently I know I have learned something when I canform counter arguments of my ownRID Relating ideas In learning new concepts or ideas I relate themas far as possible to what I already knowRER Rereading a text When re-reading a text I add to the meaning ofwhat I already know about itSDI Seeing things differently I believe that learning involves seeing thingsfrom a new perspectiveRAU Repetition aids understandinatingRepetition helps me to remember things by cre-a deeper impressionMAU Memorising after understandingI need to know the meaning of something beforeI can commit it to memoryMWU Memorising with understandinganisesKnowing the meaning of something in effect or-it in my memory at the same timeMAR Memorising as rehearsal I learn things that don’t make sense to me byreading them over and over until I can rememberthemMBU Memorising before understandingI need to commit something to memory beforeI can make meaning out of itFAC Learning is fact-based Learning means collecting all the facts that needto be rememberedKDF Knowledge discrete and Knowledge really just consists of pieces of informationfactualDER Detailed related thinking I have difficulty in fitting together facts and detailsto form an overall view of somethingFRA Fragmentation Much of what I have learned seems to consistof unrelated bits and pieces of information377

interaction between students’ study orientation, factors such as their learning experiencesand regulatory skills, and the learning environment.A high level of dissonance indicated, in that study, an individual student’s study habit problemsand lack of metacognitive skills to evaluate study practices and quality of learning. Thishigh dissonance is characterised by the inclusion of elements from both the surface (consistingof scales measuring a reproduction-directed approach to learning) and deep (consisting ofscales measuring a meaning-directed approach to learning scales). Slightly dissonant studyorchestrations are characterised by elements from either a deep or surface profile, but alsocontained theoretically atypical combinations of scale scores. Table 8.10 describes the subscales.The first seven (IND to MWU) reflect the Deep Study orchestration.As noted previously, at the end of ENG301 students were asked to complete a RoLI inventory.At this time the aim was to trial the instrument and to compare the results with the ASI –in effect to test the value of introducing a new diagnostic instrument. While the literaturesuggests RoLI has a finer granularity, and may be used, as in Lindblom-Ylänne (2004), as acomponent of student counselling on study habits, issues regarding its availability make it aproblematic instrument.Figure 8.20 extracts the students enrolled in ENG302 from the ASI data presented previouslyin Figure 8.2 (Note students numbering is random in each case).Figure 8.20: ASI Results for students in ENG302 in 2005For this group, the means are MO – 2.94 and RO – 1.81. What this shows is that, while378

all students are meaning oriented (with three students above the mean), two students arealso above the mean for reproduction. Student 1 would be classed as strongly coherent inLindblom-Ylänne (2004)’s terminology – high score for MO with commensurate low scorefor RO. Student 4 indicates coherence (though less strong) in the approaches to study, whileStudent 2 exhibits as somewhat dissonant and Student 3 as strongly dissonant.The RoLI data show some anomalies. Following Lindblom-Ylänne (2004)’s lead, these profilesare based on concentrating on the subscales which exhibit the highest scores and by analysingthe theoretical linkages:• Student 1 (Markus) (see Figure 8.21) presents with high scores for Deep Study orchestration,in keeping with the ASI score. However, he also scored high on ‘theoreticallycontrasting’ subscales. In particular, KDF and FAC interfere with scores for RID, INDFigure 8.21: RoLI results for Markusand SDI, and suggest he has a didactic/reproductive belief about learning and teaching,and consequently may well struggle with tasks that ask for more than repetition ofpassively learned material (Kember, 2001). Yet his ASI scores indicare a very loworientation towards reproduction (0.88) As his journal entries indicate, Markus hasmedical and personal problems that continually interfered with his studies. In addition,he came to the unit with some anxiety – he was not convinced he had learnt the materialwhen he undertook ENG260 in 2002 – it was easier to recall than I thought. Over thesemester, he gained confidence in his proficiency, and was, in fact, the most able in thefinal phase of the project. This perhaps suggests the ASI results are more indicativeof his general approach to study, while the RoLI breakdown is a truer picture of his379

approach to the unit• Student 2 (Alaina) is the most strongly coherent of the group (see Figure 8.22). Whileher scores for Deep Study orchestration are in keeping with the ASI score for Meaning,her Surface Study scores are much lower than her ASI indicates. Her journal entriessupport these results – she is bent on understanding and working out the material inthe unit. Her ASI scores might suggest that she falls back on reproducing when herunderstanding is reducedFigure 8.22: RoLI results for Alaina• Student 3 (Dermot) (see Figure 8.23) exhibits a high level of dissonance – he scoredFigure 8.23: RoLI results for Dermot380

high on ‘theoretically contrasting’ subscales, so that FAC, in paricular, as well as KDFinterfere with scores for RID, IND and SDI, while MAR interferes with MAU andMWA. His scores indicate a significant level of learning pathology (high scores in KDF,DER and FRA). According to Lindblom-Ylänne (2004) this are characteristics of studentswho lack the metacognitive skills to reflect on their own learning approaches andconceptions. This supports the results of the ASI for Dermot – he is hedging his betsall the way• Student 4 (Vaughn) also exhibits a high level of dissonance (see Figure 8.24) – scoringvery high on ‘theoretically contrasting’ subscales, so that here, too, KDF and FACinterfere with scores for RID, IND and SDI, while MAR interferes with MAU andMWA. His scores indicate a strong level of learning pathology (very high scores inKDF, DER and FRA). This contradicts the results of the ASI significantly. Vaughn’sjournal entries support the RoLI results – throughout the unit, he felt under pressure,needed to catch up and felt guilty about not helping other members more. Again the ASIscores might suggest that he falls back on reproducing when understanding is reduced.Figure 8.24: RoLI results for VaughnIt is clear from this small sample that qualitative methods are necessary for interpretingdissonance in RoLI profiles – the journal entries fulfil that purpose to some extent. The dataalso supports Meyer’s idea that student study approach needs to be contextualised, at leastwith regards to the learning environment.In order to provide a clearer picture of the comparison between data collected by the ASIand the RoLI, the ASI results were expanded to the subscales. Table 8.11 describes these.381

As indicated previously (see Table 5.2), the first four scales determine Meaning Orientation,and the rest Reproduction.The ASI breakdown for the student participants are indicated in Figures 8.25, 8.26, 8.27 and8.28. The comparison is based on a tentative mapping (see Table 8.12). Future researchshould examine more closely the relevance of such a mapping. Informed by Norton et al(2004)’s approach, a comparison between ASI and RoLI results is based on matching orTable 8.11: Summary of ASI subScales expandedSubScaleDefinitionDA Deep approach active questioning in learningCA Comprehension learning readiness to map out subject area and think divergentlyRI Relating ideas relating to other parts of the courseUE Use of evidence relating evidence to conclusionsSA Surface approach pre occupation with memorisationIP Improvidence over-cautious reliance on detailsFF Fear of failure pessimism and anxiety about academic outcomesSB Syllabus boundedness relying on staff to define learning tasksTable 8.12: Mapping of subScalesASI RoLIDA SDICA MWURI RIDUE INDSA MAR/MBUIP FAC/DERFFSBbettering mapped scores. Higher scores than the ASI on deep elements and lower scores thansurface define ‘better’. ‘Match’ is defined to allow for a tolerance of 10% either way on thescore (ie 80%/85% or 85%/80% are both considered matches, although the first is technically‘better’ while the second is technically ‘poorer’). Of course, for Reproduction and Surface thelower score is considered as ‘better’). Students who showed four or more (of the six mappedASI subscales) ‘better’ or ‘matched’ elements were described as having a congruent profile. Astudent who has a RoLI profile which is incongruent with the ASI results shows lower scoreson the deep elements and higher scores on the surface elements.Based on this interpretation, Students 2 and 4 (Alaina and Vaughn) exhibit profiles that arecongruent, while Students 1 and 3 (Markus and Dermot) are inconsistent – this is indicatedby an equal number of better/matched scales to poorer.382

Figure 8.25: ASI results for MarkusFigure 8.26: ASI results for Alaina383

Students preconceptions predispose them to view a learning in a discipline in a certain way,and may lead to the adoption of less than coherent learning processes. This comparisonof individual student results shows that these two instruments should optimally be usedtogether – ASI and RoLI are somewhat consistent in identifying orientation and orchestration.However RoLI’s finer granularity comes to the fore in diagnosing a Reproduction orientationin it’s focus on memorisation and factual knowledge within Surface Study. On the otherhand, the ASI isolates FF (Fear of Failure) and SB (Syllabus Boundedness) – not identifiedin RoLI, but important factors in student-centred and life-long learning models.Figure 8.27: ASI results for DermotThe value of these results depends on feedback to students. Lucas and Meyer (2004) suggestthat teachers should reflect back to students their own preconceptions. They conclude that,once educators know more about their students, they can support them in developing abetter awareness and understanding of themselves as learners. Given current concerns aboutstudent performance in higher education, and their success as practitioners, the question ofhow to help students raise their metalearning awareness is of importance.Such a small cohort does not allow conclusions to be drawn, except in very general terms.Students appear to be receiving mixed messages (Norton et al, 2004) regarding the importanceof deep versus surface approaches to learning, and appear ready to hedge their bets. Thismay be explained by the transitional nature of the School environment – not only is 2005the first year of ‘across the board’ Studio Learning, but staff comfort with this approach ispatchy, with some studios experiencing major confrontations between students and teachers.In addition, Elton (2000) suggests that where the teaching methods (in theory versus inpractice) conflict with the assessment (designed versus in practice), the result is studentschizophrenia.384

Figure 8.28: ASI results for Vaughn8.3.3 Reflection on findingsThe transition from explicitly authoritative teacher to facilitator is absolutely essential,but with that change the teacher remains an authority without being in authority. Grow(1991/1996) refers to this teacher role as one of delegating: the teacher no longer teaches thediscipline content but cultivates the students’ ability to learn. Students are consulted andlearning objectives, approaches, evaluation criteria and timing are negotiated.The focus of reflection in this cycle is on the Pedagogy component of Kreber (1999)’s framework.The main purpose of this phase of the Cycle 3 was to observe (if any) changes instudent interaction, both in the group environment and with the teacher, as an indicator ofstudent-centring.In terms of the Instructional component, the following reflections can be noted: the interactionschedules show that students are able to take early control of their learning. Due toexternal, unexpected events within the School, the teacher was often absent for extendedperiods of class time – this did not appear to have major impact on the students’ ability (orwillingness) to progress with the tasks.In terms of Pedagogy, effort continued to be placed in learning more about student learning,and on examining alternate approaches for obtaining data on individual student approaches.Evaluation of these indicates that, despite the success of students within the unit, it maybe necessary to address individual student preparedness for non-traditional learning on anindividual basis.Alignment must address learner interest as well as the constructive elements previously dis-385

cussed. When the unit is not aligned with learner interests or the situation constrains thestudents approach to learning, the dependent learner mode will tend to dominate – controlof the learning process is relinquished to the teacher, while the student will demand carefullyarticulated structure, clear guidance and clearly-defined assessment. These characteristicswere most noticeable in an Apprenticeship model of learning, but continued through outboth CreativePBL and Studio Learning of RE. However, when the unit is aligned with thelearner’s interests and the situation allows them to adopt their preferred learning styles, theywill tend to display adult learning behaviour. The students will prefer to design their approachto the material and will focus on the salient points that address their needs. Thefindings discussed above suggest that a stronger alignment was developed in ENG302.The Studio Learning model is a dynamic one – students and teacher negotiate how thelearning will take place. The student who comes out of such a dynamic classroom should bemuch better prepared for professional practice. Studio Learning is designed to give confidencein decision making to the student throughout the classroom experience. As such, this studentmay perform differently from previous conventionally trained employees. They may be morewilling to make their own decisions and apply their new skills.The students themselves suggest this is occurring:From: [Markus]Sent: Wednesday, 10 May 2006 12:08 AMTo: Jocelyn ArmaregoSubject: Re: internshipsJocelyn, Sorry I have not got back to you sooner, I have beenwaiting on definitive answers regarding internshippossibilities. Earlier in the semester when you were helping uswith internship with Xxx I mentioned that I had a job for nextyear ..., well a Software Project within the Company arose thatfitted within the guidelines of the University for anInternship.The particular project is a large one and most likely I willonly get to the simulation phase. I will be redesigning acomplete operating system [...] I am confident of doing the taskwith both my background in mechanics ..., and also using themethodology of Software Design you have taught me. As I will be386

using these methods of requirements and design before attemptingto do any coding.I still stand by that the Software Design Studio you taught lastyear really has given much confidence in the process and theimportance of Software Design. You should feel good aboutyourself that you have a positive and practical approach toteaching. I remember I struggled through the originalrequirements and you just let me pass, though you did say stillspend some time on it, which I did. Even though it had been acouple of years before doing Software Design Studio I was amazedhow much easier it seemed the next time round, so you must havedefinetly pointed out the areas I needed to review. [...] Mustgo I need to still do some study, I thank you again for guideand assistance with my studies and future employment.Markus exhibits many of the attributes this research was attempting to target. He expressesconfidence in his own ability to learn and apply new knowledge as well as adapt what hehas learned. This confidence based on knowledge and metacognitive skills that have beenencouraged and developed throughout his formal education.Reporting findingsThe reporting at the end of this cycle is both more general and more focussed – addressingan engineering education audience in one case, and specifically educators of RE in the other.The background to the changes being undertaken, and in particular an evaluation of theorientation week to introduce students to Studio Learning was reported on at the GlobalColloquium for Engineering Education (Armarego and Fowler, 2005).The attempt to align the teaching with industry needs, both generally in Software Engineeringand more specifically in RE, was described at the RE Education & Training workshop (REET)attached to RE2005 (Armarego and Minor, 2005), This paper incorporated the preliminarywork undertaken by Minor validating, in the local context the practitioner perception (thisis discussed in Chapter 2). Reviewer comment suggested the paper was very likely to lead toan interesting discussion which engages many workshop participants.387

Chapter 9Conclusions26 April 2006 --- wrote:How do we know if we are getting any better at RequirementsEngineering (RE)?Tony Markatos responds:Excellent question! I often wonder the same.I feel that we are regressing - especially when considering whatis of the highest priority - requirements elicitation.The way I learned it and definetly the way I experienced it, adequatefunctional requirements elicitation is the major thing that needsto be done.And yet, we truely seem to be regressing back to the basic mistakes ofthe 60’s. Specifically, we have lost awarness of the need for anapproach that actually prods us through functional requirementsdiscovery and, instead, have opted in amass for techniques that,in large part, rubber stamp our flawed understanding.Tony Markatos (Greece)This thesis has argued that traditional formal education does not meet the competency expectationsof practitioners of Requirements Engineering. Practitioner dissatisfaction withformal education focusses on non-technical components of competency: they look for graduateswho are flexible, adaptable in the organisational environment and can continue learning.These have been identified as cognitive skills related to higher order learning and metalearn-388

ing/metacognition as well as strategies to enable opportunism and creativity.By specifically examining the match between gaps identified by practitioners and educationmodels that purport to focus on these, learning interventions have been planned, applied andrefined as part of this research.Through evaluation of these Action Research cycles, this study shows that non-traditionalapproaches can provide leverage for the student entering the profession of RE by explicitlyaddressing the gaps identified by practitioners. This is achieved through a shift in focus fromtechnical competency to the soft and metacognitive skills that enable the competent practiceof RE. However, the same evaluation shows that an incorrect learning environment can stillexist between professional practice and non-traditional education – what is needed is tuningto a finer granularity so that the characteristics of professional practice are mapped to andreflected in the learning model that is applied.The research also shows that, as well as alignment between discipline and learning model, analignment can exist between learner and learning model, and suggests that this relationshipshould be exploited in the development of competent practitioners.9.1 Summary of the researchThis final chapter provides a summary of the research, discusses the value of the findings andindicates what possibilities arise from it.The overview of the literature provided in Chapter 2 makes it is possible to identify a conceptualmodel of the Requirements Engineering discipline.Chapter 2 also examines the practitioner perspective, and the perception they have of competencygaps in formal education. These studies have suggested that, assuming appropriatetechnical knowledge, formal education does not address their needs for competent practitionersin the areas of Stance and Intellectual Capabilities (Scott and Wilson, 2002) specificto Requirements Engineering, namely cognitive skills related to higher order learning andmetalearning/metacognition as well as strategies to enable opportunism and creativity. AsGardner (1983) noted, a high level of intellectual competence, in particular the potential forfinding or creating problems as well as solving them, lays the groundwork for the acquisitionof new knowledge.However, the literature of learning, discussed in Chapter 3, indicates that the characteristicsof RE can be addressed within a formal education environment, albeit not by traditionallearning.389

The lack of alignment between the actuality of practice in the discipline and the instructionaldesign supposed to model it creates an incorrect learning environment. This poor fit betweenthe characteristics of the domain and those of the learning model, can partly explain theinadequacy of formal education in training competent RE practitioners. If this is so, asolution can be proposed through the development of a new framework for RE education.This framework is also discussed in Chapter 3.This study has proposed three intervention strategies for addressing these issues in an ActionResearch-based project. Chapter 4 discusses the methodological background for this studyand the choices made. Chapter 5 describes the research design in concrete terms.Figure 9.1: Education for RE – the Action Research cyclesEach intervention strategy is to address specific areas and, through feedback on the reflection,strategies are refined for the next cycle to address the issues identified. Figure 9.1 illustratesthe cycles of the project. These were undertaken within the context of a unit in RequirementsEngineering within the School of Engineering at Murdoch University over the period 2002to 2005. Chapters 6 to 8 describe the interactions undertaken (summarised in Table 9.1),and discuss interpretations of the data. A summary of this research as a completed ActionResearch project (and hence discussing each cycle) has been accepted for publication asChapter 8 in Lowry and Turner (in press) (Armarego, in press).390

Table 9.1: Summary of issues addressed by the Action Research cycles of this studyCycleApprenticeshipCreativePBLStudio Learningto Addressauthenticity; transferstudent-centred learning; creativity; (authenticity; transfer;adaptability)transfer; deep learning; student-centred learning; opportunism;creativity; authenticity; metalearningThe next section summarises the findings of this research, both at a conceptual level and inpractical terms. This is followed by a discussion of the evaluation of this research, and thestrategies adopted to address validity criteria, and the limitations identified.9.2 Modelling RE education9.2.1 Modelling REThe overview of the literature provided in Chapter 2 makes it is possible to identify a conceptualmodel of the Requirements Engineering discipline. This model, based on perspectivestaken by its exponents, Bodies of Knowledge and model curricula, provides indicators of theknowledge and skills required to practice as competent professionals.A knowledge intensive (Robillard, 1999) and intrinsically complex task (Brooks, 1986), RequirementsEngineering may be categorised as:a process of knowledge discovery (Guindon, 1989) – the Requirements Engineer buildsfragments of understanding of the problem validated and consolidated, adding detailand richness to the mental model of the problem situation (Batra and Davis, 1992)requiring:a facility with model-making – enabling the multiple perspectives that include theconceptual model that guides the engineer, the system image presented to the userand the mental models of the user (Norman, 1983) to be matchedexpertise – Requirements Engineering also needs both specialised composite knowledgeto structure the problem and tame the complexity of the task (Jeffries et al,1981; Guindon, 1990) as well as planning based on identification and exploitationof past situations and basic schema recognition (Robillard, 1999). This planningmay be defined as the management of knowledge structuresand involving:391

a learning dialogue (Laurillard, 1993) – in an environment of teaching and learning meaningis negotiated through a dialogue, which is seen to be:discursive – both teacher and learner conceptualisations should be accessible to eachotheradaptive – the focus of the dialogue is based on the relationship between the twoconceptionsinteractive – requiring learner action and teacher feedbackreflective – supportive of the process to link feedback to goalin order to deal with the‘wicked’ problems undertaken in RE – after Simon (1973) RE tasks may be characterisedas having the following ill-structured features: an incomplete and ambiguous specificationof the problem; a lack of stopping rules (to evaluate when a solution has beenreached); many sources of knowledge which cannot be predetermined and may needto be integrated and a lack of an exhaustive list of operations to reach a solution andabsence of a predetermined solution path.Establishing requirements for software-intensive systems is seen to involve, amongst others,the intellectual activities of analysis, specification (synthesis), evaluation and creativity, undertakenin a social setting. The first three of these imply higher order learning, while thelatter is seen to require creativity-enabling environments. These are also discussed in Chapter2.The summary provided by Loucopoulos and Karakostas (1995) supports the conceptual modeldefined by these characteristics:• analysis problems have ill-defined boundaries, structure and a sufficient degree of uncertaintyabout the nature and make-up of the solution• requirements are found in organisational contexts, with associated conflicts, expectationsand demands of the proposed system• the solutions are artificial – they are designed and therefore represent some of manypotential solutions• analysis problems are dynamic – they change while being solved• solutions require interdisciplinary knowledge and skills• the knowledge base of the analyst is constantly evolving392

• the process of analysis is primarily cognitive in nature - all the other skills facilitatethis cognitive process.9.2.2 Practitioner perspective on competencyPractitioner studies have suggested that, assuming appropriate technical knowledge, formaleducation does not address their needs for competent professionals. This is borne out in thelocal context both in discussion between practitioners on the RE-online list (Zowghi, 2004)(excerpts of which are discussed in Chapter 2), and, even more locally, in the context of theSEF (Software Engineering Forum 1 ). The work described in this thesis has been the subjectof several meetings, most recently in September 2006, discussing issues in training competentRE practitioners.Competence in any domain requires, in addition to domain knowledge,physical skills – constituting physical expertise of the procedural tasks, including appropriatetool use. These, due to their external visibility, are seen as relatively easier toacquirecognitive skills – concerned with the cognitive processes of analysis, interpretation anddecision-making required for the carrying out of procedural tasks. Cognitive skillsrequire a more sophisticated learning process.Competence also assumes the ability to apply higher order thinking – this is a function ofthe interaction between cognitive strategies – to support and regulate the constructionof knowledge; metacognition – in particular strategic knowledge and domain-specificknowledge – pertinent to the problem situation. In its turn, higher order thinking is seen asaddressed, within learning taxonomies, at a post-foundational level (eg Bloom et al (1956)’slevels 4, 5 and 6), implying advanced learning.Another aspect of competency addresses the affective and generic attributes required bya discipline, including interpersonal and organisational skills and a capability for lifelonglearning.If we apply the Professional Capability Framework of Scott and Wilson (2002) what hasbeen identified as lacking in education for RE professionals are in the dimensions of Stance1 this is a group of invited SE practitioners and academics who meet monthly to discuss topics of interest,either from the industry perspective seeking academic input, or from the academic perspective seekingindustry input. The group operates under the umbrella of the ITEE College of Engineers Australia. Thepurpose of the ‘invitation only’ is to ensure no one group (or agenda) dominates393

and Intellectual Capability. Stance incorporates Emotional Intelligence (personal andsocial), and addresses the affective and generic attributes noted above; Intellectual Capabilityincludes Way of Thinking (including cognitive intelligence and creativity) and DiagnosticMaps (model making developed through reflection on experience), addressing the cognitiveaspects. Although this framework has been developed to address practitioner needs fromEngineering graduates, the correlation with IT practitioner concerns is strong – the commontheme is evident from the majority of studies. In particular, even the early studies of REcompetence indicated the importance of personal abilities/characteristics (eg Kozar (1989))while intellectual capabilities (such as the manipulation of problem space (Thomas et al,1977)) are identified as important for competent practice. A link to Gardner (1983)’s modelof multiple intelligences is also established (see Section 2.1.4).In Requirements Engineering, the gap between formal education and practitioner needs isgreat enough for professionals to doubt RE as a discipline for new graduates to engage with(Minor, 2004) – or even for RE being a suitable topic for university study (Macauley andMylopoulos, 1995b). Academics have expressed similar concerns (Bentley et al, 1999; Banks,2003).The educational dilemma therefore is to provide RE students with a solid foundation insubject matter while at the same time• exposing them to the inherent characteristics associated with real requirements problems• addressing the knowledge required to solve them, implying a explicit focus on◦ the cognitive skills related to higher order learning and metalearning/metacognition◦ strategies to enable opportunism and creativity◦ the development of the affective skills related to emotional intelligence.One critical finding from the review of practitioner perspective relates to the importance oforganisational (as opposed to individual) maturity. An emphasis on soft and cognitive skillsin its employees has been shown to be a characteristic of a mature organisation (Benbasatet al, 1980). This is worthy of further investigation, in particular in relation to the disciplineof RE. If technical skill acts as a filter to employment (as has been discussed in Chapter 2),do only mature IT organisations engage in competent RE?394

9.2.3 Developing an education-practitioner alignmentThe literature of learning indicates that the characteristics of Requirements Engineeringcan be addressed within a formal education environment, albeit not by traditional learning.Reigeluth (1996, 1997) argues that the current paradigm of education is based on standardisation,conformity and compliance, geared to the mass production of industrial agemanufacture. This does not equate with the needs of the late 20th or early 21st century jobmarket which revolves around problem-solving, teamwork, communications, initiative takingand diverse perspectives. This implies a lack of coincidence between the actuality of practicein the discipline and the instructional design supposed to model it – suggesting the need fora new paradigm, based on customisation, diversity and initiative, to suit the needs of theinformation age.If, as has been argued, the inadequacy of formal education in training competent RE practitionersmay be partly explained by an ‘incorrect’ learning environment resulting from thepoor fit between the characteristics of the domain and those of the learning model, a solutioncan be proposed through the development of a new framework for RE education. Thisframework should:• be based on constructivist theory (as more suitable for learning in domains involvingill-structured problems (Spiro et al, 1991)) with a focus on strategic knowledge to enhanceknowledge construction and transfer. This includes metacognitive strategiesfor directing, monitoring and evaluating learning. These strategies also enablestudents to traverse the stages of growth towards life-long learning (Grow, 1991/1996)• be placed within a situated experiential learning environment where authenticcontext is exploited. Learning beyond the initial stages may best be achieved throughsituational case studies with rich contextual information (Dreyfus and Dreyfus, 1986).Focussing on the solution of authentic problems as a context for learning providesstudents with entry to the community of practice to which they will belong• provide the student with a learning environment that has an emphasis on modellingpractice, making tacit knowledge explicit and thus empowering students to thinkindependently.9.2.4 Implementing a model for RE educationThrough subsequent Action Research cycles, this research has proposed intervention strategiesto address specific areas of education-practitioner alignment.395

These feedback cycles of intervention, reflection and refinement were undertaken within thecontext of a unit in Requirements Engineering within the School of Engineering at MurdochUniversity over the period 2002 to 2005.It should be noted that each of the models takes as a given a ‘block’ timetabling for learning.The value of this (in addition to its authenticity in a practitioner context) is the abilityof students (and teacher) to leverage learning from the increased ‘flow’ time (uninterruptedproductive time) (de Marco and Lister, 1999) available to them.2002 the Apprenticeship modelCollins et al (1989) and Brown et al (1989) suggest this environment based on CognitiveApprenticeship models proficiency and enculturates studentsinto authentic practices through activity and social interaction in a way similarto that evident - and evidently successful - in craft apprenticeships.(Brown et al, 1989, p 37)The Apprenticeship model applied in ENG260 could be seen to exemplify Savin-Baden(2000)’s model for professional action as summarised in Table 9.2.Table 9.2: Model for professional action (Savin-Baden, 2000)KnowledgeLearningProblem ScenarioStudentsFacilitatorAssessmentpractical and performativeoutcome focussed acquisition of skills and knowledgefor the workplacefocussed on real-life situations that require an effectivepractical solutionpragmatists inducted into professional cultures whocan undertake practical actiona demonstrator of skills and guide to best practicetesting of skills and competencies for the work placesupported by a body of knowledgeThis addresses professional action which will allow students to gain competence to practice,so that they are expected to transfer skills acquired to the world of work. However, there isno emphasis on higher learning – either cognitive content or professional judgement. Withinthe Murdoch RE learning environment, this deficiency was epitomised – students were boundby frameworks that had been developed within the unit and encountered problems in notonly generalising their learning, but also allowing their general knowledge to enhance theirdiscipline learning. In short, they were constrained by the environment that had been built.396

A significant finding of this cycle relates to student emphasis on ‘correct’ answers to problemsolving undertaken. This suggests a lack of epistemological/intellectual maturity (see Section6.2.3, which discusses the work of Baxter-Magnola (2001) and Perry (1988)). Challenges tostudent belief systems and peer interaction are suggested as requirements for opening studentsto the validity of multiple perspectives.The nature of RE implies a need to incorporate creativity-enhancing activities within thelearning environment, to foster adaptability in students by providing for divergent as well asconvergent thinking and to focus on metacognitive strategies and reflection as an aid to thetransfer of the skills and knowledge learnt. These are characteristics of deep learning, notexplicitly addressed by learning models which focus on know-how within a given framework.Therefore, while the Apprenticeship model did address some aspects of the characteristics ofRE - specifically authenticity in a situated environment, evaluation and reflection identifiedelements of an ‘incorrect’ learning environment. The Apprenticeship model exhibited someof the traits identified by Patel et al (2000) – students focussed on learning the tools andtechniques of RE at the expense of a broader (and more abstract) understanding within thediscipline.2003 the CreativePBL modelThe CreativePBL model was developed to address the deficiencies of the Apprenticeshipmodel that were identified in Cycle 1. PBL emphasises ‘learning to learn’ in an environmentthat moves from dealing with content and information in abstract ways to using informationin ways that reflect how learners might use it in real life (Oliver and McLoughlin, 1999).This strongly suggests that PBL has application in the solving of wicked problems in wickeddomains:• learning based around constructivist principles is likely to be more suitable in domainsinvolving ill-structured problems (Spiro et al, 1991). These principles are encapsulatedalmost ideally in problem-based learning (Savery and Duffy, 1995)• appropriate learning in ill-structured domains and/or dealing with ill-structured problemsshould itself be problem-based• problem-based learning best provides an effective environment for future professionalswho need to access knowledge across a range of disciplines (Boud, 1985)• its problem solving requires the mental representation of problematic situations – theproblem space must be constructed, either individually or (of more relevance in RE)397

socially through negotiation (Jonassen, 2002).The CreativePBL model was developed to focus on creativity and divergent thinking, so that,instead of students aimed at finding the single, best, correct answer to a standard problemin the shortest time (convergent thinking) they aimed at redefining or discovering problemsand solving them by means of branching out, making unexpected associations, applying theknown in unusual ways, or seeing unexpected implications.This model focussed on student-centred learning within an ill-structured domain, so that thegap between know-how and know-that could be bridged. It could be considered an example ofSavin-Baden (2000)’s model for interdisciplinary understanding (see Table 9.3). This modelfocusses on understanding and synthesising information, and aligns more closely with thecharacteristics of the discipline – in particular the need to transcend the learnt context andintegrate multiple perspectives, which require elements of adaptability and creativity.Table 9.3: Model for interdisciplinary understanding (Savin-Baden, 2000)KnowledgeLearningProblem ScenarioStudentsFacilitatorAssessmentpropositional, practical and performativethe synthesis of skills and knowledge across disciplineboundariesacquiring knowledge to be able to do, therefore centredaround knowledge with actionintegrators across boundariesa co ordinator of knowledge and skills across boundariesof bothexamination of skills and knowledge in a context thatmay have been learnt out of contextHowever, while the CreativePBL environment appeared to facilitate creativity-enabling activitiesby embedding these within the process, the process itself acted as a deterrent to studentmotivation to study and to exploit the creativity being nurtured – opportunism was difficultwithin the process and hence flexibility inhibited; here a focus on process detracted from the‘authenticity’ of the environment.In effect, this strategy also did not align well enough with professional practice in the discipline.There is a suggestion that efforts to help students learn at the levels of analysis,synthesis, and evaluation (ie at Bloom et al (1956)’s higher levels) may be impeded by amismatch between the kinds of thinking actually required in specific disciplines and genericformulas for encouraging higher-order thinking. In the final analysis, applying a strict PBLmethodology for learning may run counter to the important strand in current thinking aboutteaching that stresses the disciplinary nature of knowledge.398

2005 Studio LearningThis model was developed to gain leverage from the positive elements of both the Apprenticeshipand CreativePBL models previously applied.Felder (1996) suggests that basic components of a learning strategy to reach all types oflearners should include ‘teaching around the cycle’: explaining the relevance of each new topic(Diverger); presenting basic information and methods associated with the topic (Assimilator);provide opportunities to practice the methods (Converger) and encourage exploration of theapplications (Accomodator). Addressing all learning styles in this manner enables studentsto develop the mental dexterity required in professional practice.Andresen et al (1995) describe the need for contingency measures to be available in the creativedomains where the unexpected is expected. Edwards (2004) reports on approaches toexplicitly provide the opportunity for students to adopt expert strategies. The teacher guidesstudents in the nature of these processes and helps them to reflect critically on their effectiveness.He notes that the best means of facilitating expertise is to provide the opportunityfor practice. However, only through encouraging students to challenge their own effectivenesscan they learn what this implies.The Studio Learning strategy appeared to be effective in addressing these issues. Not onlywas student feedback positive, and a significant improvement in their assessment mark discernable,but an observation and analysis of some of the cohort in the subsequent unit showedstrong indications of willingness to transfer knowledge gained, to take control of their learning,and indicated motivation to deeper learning. Examination of student reflective comments,in conjunction with data regarding student learning, adds another dimension to the issue ofeducation for competent practice. This examination indicated a relationship existing betweenthe learner and the learning model, so that students whose approach favoured deep learningfor understanding were advantaged by a learning environment which challenged them. Thisfinding is discussed in greater detain in Section 9.4.9.3 Validity of the research approachWithin both education and IT research a variety of research approaches are accepted, withthe norm to distinguish between positivist and non-positivist positions and the assumptionsmade in each. Because of the transformative nature of the study, an interpretive stance hasbeen adopted, and Action Research identified as the methodology of choice.A conceptual framework for Action Research in RE education was developed by adapting and399

Figure 9.2: A conceptual framework for Action Research in RE educationintegrating three models: the process is a defined Action Research model based on the work ofBorg et al (1993) which specifically addresses Action Research in an educational environment,the context an environment where the aim is learning (cognitive change): Rogers (2002)’smodel acknowledges that effecting cognitive change through instructional design is basedon the interaction of the participants of a learning environment. The iterations explicit inthe research design require double loop learning on the part of the researcher at least, sothat future action is based on varied reflection. The model proposed by Hatten (1997) isused for this aspect. The dominant characteristics of this study suggested that each of thesecomponents to be incorporated. Figure 9.2 provides a visual summary of this framework.The framework is shown to be consistent with the context of the research:• it is a field-based study of a social practice, involving the participants themselves intheir natural setting and exploiting the biases inherent in such an environment. Theresearcher acts as ‘deep insider’ (Edwards, 1999) within the context of this study, and,as such is◦ aware of the organisational history and personal relationships which are interwovenwith that history◦ more likely to possesses reasonable beliefs about the landscape, the territory, the400

unspoken agendas of groups within the organisation.Educational Action Research places the researcher in the dual position of producer ofeducation theory/policy and user of that theory through practice• it is interventionist – the change in existing practice implicit in Action Research requiresthe researcher to act as agent of change while acknowledging that the changes thatoccur are not solely due to the actions taken. However, as an insider the researcheris more likely to know about the behaviours and attitudes of individuals within theorganisation/group and has confirmation from others over years. The researcher is ina position to reasonably predict responses by individuals, which is useful for the nextpoint• it is flexible enough to adopt any methods that satisfy its raison d’être. With thepotential to provide feedback holistically, the research ideas are allowed to evolve andemerge as part of an ongoing learning and reflection process. Again the researcher asdeep insider is able to shape the research dynamically to check predictions made andtrack changes.In an environment that attempts to address the disparity between formal education in a disciplineand practitioners’ expectations of it, these characteristics of Action Research provideleverage.9.3.1 Criteria for qualityAlternative frameworks are suggested as needed to support Action Research as rigorous andhigh quality, without sacrificing its relevance. Criteria for the evaluation of interpretiveresearch were described in Chapter 4 – Table 9.4 provides a summary. These criteria havebeen borne in mind throughout the research. The rest of this section provides some examples,drawn from the thesis, to illustrate what evidence exists to address them.Outcome Did it solve the problem? This criterion requires that action emerging from thestudy leads towards resolution of the problem (and can be applied to subsequent researchcycles). The dynamic nature of the discipline (and of a learning environment thataddresses the needs of a discipline) suggest the problem cannot be ’solved’. However,closer alignment between the two allows for a ‘satisficing’ 2 . In terms of the researchitself, each subsequent cycle has been informed by the previous. Thus issues raised in2 a term introduced by Simon (1956) to indicate a decision to pursue a course of action that will satisfythe minimum requirements necessary to enable a particular goal to be achieved401

Table 9.4: Principles for the evaluation of Action Research (based on Anderson et al (1994) andKrefting (1991))ValidityCriterionOutcomeProcessDemocraticCatalyticDialogicCredibilityTransferabilityDependabilityConfirmabilityAddressed byDid it solve the problem?Was the activity educative and informative?Was the research undertaken in collaboration with all involvedwith the problem under investigation?Did the research transform the realities of those involved?Could the research be discussed with peers in different settings?prolonged and varied field experience; time sampling; reflexivity(field journal); triangulation; member checking; peer examination;interview technique; establishing authority of researcher; structuralcoherence; referential adequacynominated sample; comparison of sample to demographic data;time sample; dense descriptionaudit; dense description of research methods; stepwise replication;triangulation; peer examination; code-recode procedureaudit; triangulation; reflexivityone cycle are addressed and ‘solved’ in another. The Reflection sections of Chapters6 and 7 exemplify this process, while this chapter addresses the extent to which theproblem as a whole was ‘solved’Process Was the activity educative and informative? Reflecting on the suitability of datacollection, and modifying strategies in order to enrich the data are seen as mechanismsthat address process validity. It can be seen that the data collection for Cycle 1 wasnot very sophisticated and did not include a detailed ‘story’ from participants. Thiswas addressed in subsequent cycles, enabling the voice of the participant students to beheard, thus enriching the value of their involvement. Learning is identified as an explicitoutcome of this research. The professional growth of the researcher as educationalist isevident from the reflections on each cycle and the increased sophistication of subsequentintervention planning, implementation and interpretationDemocratic Was the research undertaken in collaboration with all involved with the problemunder investigation? The multiple perspectives of the students are represented, ingeneral, through their own words. Colleagues acted as ‘critical friends’ by offering feedbackand providing alternate perspectives from which to view the data. The studentsthemselves can be seen to be learning skills other than technical ones – almost despitethemselves. This is clearest in the (chronological) excerpts taken from the reflectivejournals (in Appendix C and discussed in Chapter 8)402

Catalytic Did the research transform the realities of those involved?The argument for catalytic validity lies not only within the recognition ofthe reality-altering impact of the research process, but also in the desire toconsciously channel this impact so that respondents (participants?) gain selfunderstandingand, ultimately self determination through research participation(Lather, 1991, p 68).Did participants gain self-understanding and self determination through this research?Feedback such as that from Markus, quoted in Section 8.3.3 suggests they do. Theresearcher also has gained enormously from the insights into her teaching. Disseminationof the findings of this research include refereed publications, and a workshop at aninternational conference. It would seem that this work has the potential to influencethe wider IT education community. A reviewer comment is valuable in this contextI found what the paper does contain to be very interesting.Personally, I had not before seen all this evidence in oneplace. It gave me cause for reflection. It made me think abouthow to change RE curricula. All in all, I liked the paper a lotfor its value to me as a fellow researcher and educatorDialogic Could the research be discussed with peers in different settings? Critical conversationsabout the research findings and practice were held at several levels – within theresearch context, as noted above, within the education profession by publishing in thatdiscipline and, most importantly, within the Software Engineering and RequirementsEngineering profession. The most valuable conversations were those with my supervisor,who, as a practicing RE as well as a researcher of RE practice asked the mostprovocative questions that helped me define my expectations and intentions, stretchedme to articulate precisely my rationale for those decisions and helped me to see importantinformation from different perspectives. Peer reviewers for publications in thatdiscipline also provided feedback that helped shape the interpretations and reflectionsmade.The challenge has been to make this work accessible from an educational perspective.In this respect, the same could be said of the Academic Support Officer who actedas mentor during the development for Cycle 2. Her persistence in requiring me toconsider educational pedagogy was instrumental in my acquisition of the vocabulary ofthat discipline, and therefore being able to articulate this work in that context403

Credibility Guba and Lincoln (1994) suggest prolonged participation, persistent observationand peer debriefing as strategies to address this criterion. These have been discussedabove. Triangulation and the collection of ‘primary’ data (ie data produced bythe participants themselves) have also been applied in this study in order to enhancethe credibility of the findingsTransferability comparison of the context of this study with other contexts is facilitatedby the inclusion of detailed descriptive data, and through rich description of the contextitself. This is provided as a background appendix (Appendix A) and throughdiscussion of the changing context at the commencement of the chapters discussing theinterventions (Chapters 6 to 8). Guba and Lincoln (1994) notes that transferability ofan Action Research account depends on the readers ‘seeing the setting’ for themselvesand identifying with it. The narrative approach taken in reporting this research is onestrategy for achieving thisDependability the stability of the data is enhanced through triangulation techniques andthe development of an ‘audit trail’ which describes each process undertake. The descriptionof research methods may be considered extensive, both in Chapter 5 as overallstrategy and in each chapter describing an Action Research cycle. There, what wasplanned as well as what eventuated are provided as a dense description of the strategyadoptedConfirmability triangulation is again a suggested strategy addressing this criterion. ‘Diary’entries to record ideas and reflections are also considered as addressing confirmability,with reflection and the process of double loop learning the most relevant here. EachAction Research cycle concludes with a reflection section which seeks to place theoutcomes of the intervention (and the interpretation of these) within the context of theresearch learning journey.By virtue of the nature of the research, the findings of this study are limited to the contextin which it was undertaken. However, in order to address concerns raised against ActionResearch (eg contingency, control and over-involvement (Orlikowski and Baroudi, 1991), discussedin Section 4.1), the following approaches are included within the research design, basedon the discussion of Kock et al (2000):• some elements of the study are longitudinal: findings from each cycle are examined insubsequent cycles, to see how well they hold, while some participants are tracked acrosssubsequent semesters. These act to expand the project scope, analogous to choosing awider ‘sample’ in statistical studies404

• data collection is broadly focussed, and facilitates an approach that draws from thegrounded theory methodology. It allows a dynamic adaptation of both the interventionand data collection based on a continuous interplay between data collection and analysis.In effect, the data drives an emergent intervention. This eschewing of manipulationenables a disciplined extraction of a description of behaviour. It also provides an advantagein generating relevant and valid knowledge for the context under investigation,minimising bias created by artificial action on the part of the participants in responseto a focus on predefined variables and links – the data ‘speaks’. This use of multiplesources also facilitates an effort to triangulate• learning is an explicit outcome: the study is conducted in the researcher’s culture. Iwas involved in both in the planning and the practice of the interventions, as well astheir interpretation. I may be considered to have a stake in the approaches taken, andhence exhibit ‘cultural blindness’. However, the personal involvement of the researcheris seen to provide valuable insight as ’cultural insider’ with regard to the context ofthe study – the concept of insider research helps in the gathering of ‘rich’ data. Theiteration of cycles is also seen to assist in reducing potential distortions in the findingscaused by over-involvement, while peer review through publication enables externalevaluation of the findings of the study.The measures taken, while not exhaustive, demonstrate that the research fulfills to a largeextent the criteria set out for its evaluation.9.3.2 Limitations of the researchA major limitation of the study is the inability to continue interacting with and monitoringthe model developed in the final cycle in RE education – as of November 2005 the programmeof study for the BE(SE) at Murdoch University was terminated. However, Design Studioscontinue to be the applied throughout third and fourth year units in Engineering, withincreased stakeholder commitment. A School review undertaken by the University, at theend of the first academic year of Design Studios, acknowledged staff and student satisfactionwith Studio Learning, and recommended that the model be applied throughout the School (ienot just to Engineering) (Lawrance et al, 2005). Longitudinal monitoring with data collectionwill go some way towards confirming these initial findings and perceptions.Within the time frame of this study, several of the Engineering programmes were professionallyre-accredited by Engineers Australia. While this may not appear significant, the reviewof the proposal (as opposed to post-implementation) for Design Studios attracted favourable405

comment from the accreditation panel. Their final report (Bradley, 2004) suggested thiscould become the leading programme in this country, albeit with some resistance to changelikely, and requested a formal report on Design Studios within Engineering by the end of2006.Another limitation of this study is the focus on participant students only during their studies.Within the SE programme, additional research has been undertaken to evaluate student abilityto transfer the skills and competencies gained to subsequent units and to the workplacelearning environment (in the context of an internship). While the majority of that work isnot discussed here, preliminary results, and in particular employer reaction within the ITdiscipline are encouraging. Student engagement with subsequent Design Studio (ENG302specifically, in this context) is reported briefly (see Section 8.3). More interestingly, as studentsrotated into the role of Project Manager, they (individually) applied what they hadpreviously learnt with regards to learning strategies and approaches to study in order tomotivate their group members. This is a pleasing outcome and hints at the catalytic natureof this study.However, further research is requires to test these findings in the context of their transitionto the workplace. At this time, reporting is only anecdotal – as an example of positiveemployer reaction, a global software development organisation with a workforce of over 30000 accepted a lone SE intern in 2003. In 2004 this was doubled to two students - in 2005the request was for ten students who had participated in Studio Learning in SE. Again in2006 the organisation was willing to offer all SE students an internship. More revealing, thisdemand was not matched at other universities in the state offering Engineering programmesfor software. A further indication of employer satisfaction is provided by graduate careerprospects. While empirical evidence is in the process of being accumulated - there are stilltoo few SE graduates to provide statistically significant results, that anecdotal evidence isalso encouraging. Where one (20%) 2004 graduate Software Engineer was employed by thesame global software development organisation, of the 2005 cohort 50% (six graduates) arenow employed there. Both 2006 graduates (100%) are also with the same organisation, whilesome of the 2007 cohort (currently undertaking internships within the organisation) have alsobeen offered employment.Methodological limitations must also be acknowledged. As has been noted, and in the contextof the Action Research itself being a learning exercise, data collection at the commencementof the study lacked sophistication – a particular issue was providing a mechanism to capturestudent perception of the learning environment as they were experiencing it (or reasonablesoon afterwards). Only snapshots, albeit quite crude, were available through responses to406

various surveys undertaken throughout the semester. This source of rich data was tappedmore effectively once assessment components were developed and integrated into the learning.The Performance Review achieved a highly desirable reflective element to the data; theActivity Log/Journal, completed weekly, proved to be richest in revealing the journey undertakenby the students during the semester. Although the possibility of inaccurate studentself-reporting of their study behaviour cannot be ruled out, the data provided are believable,have added ‘personalities’ to the account of the interventions, and added interpretive depth.This data continue to be mined for insight into the student learning process.Success is reported here in the context of the research study undertaken. A broader viewof the success addresses the level of influence the research has, both on immediate practiceof education for RE, and on RE educational research. A more robust research design wouldincorporate a longitudinal study to assess this. Although dialogic validity has been attempted,the resources required to address this issue more fully are beyond those available to theresearcher.9.4 Implications9.4.1 Contributions to knowledgeAs noted in Chapter 1, the objective of this research has been to make a contribution to thediscipline of RE and its education by aligning practitioner needs with learning models thataddress these needs.The development of a profile for a ‘virtual’ Requirements Engineer, based on characteristicsidentified as important by practitioners, informs the conceptual model for RE education.The development of a schematic of the computing space for RE (after Shackelford (2005))also informs the conceptual model. That model suggests that learning for RE should bedrawn from non-traditional models of education, which are appropriate in that they includestrategies that address more explicitly the gaps in formal education identified by practitioners.Although in its infancy within this University and in the discipline of IT, Studio Learninghas been seen to address issues raised in studies of discipline practitioners and the educationliterature. The need to:• provide students with authentic experiences which address competencies additional tospecific discipline knowledge◦ students were exposed to learning both as a ‘generic’ metacognitive activity, and407

as a skill to be continually adapted and utilised within a discipline context◦ flexibility in thinking - addressing creativity, opportunism and divergency/ convergency– was made explicit and strategies to exploit it developed• provide learners with a deep understanding of self and others in complex human activitysystems◦ in a collaborative environment, students became aware of and learnt to utilise eachothers strengths and weaknesses in achieving the learning outcomes. They learnthow to ‘jell’, what to do if they did not, and to be empathetic to the contexts ofother students◦ they learnt to value and exploit alternate perspectives brought to a problem bydifferent stakeholders (client, teacher/consultant, other team members) to enrichtheir learning◦ they became aware of the need to be self-motivated and learn independently◦ students were confident in questioning their own and others’ assumptions withinthe learning environment• allow time to explore new ideas and to reflect on possible processes and outcomes◦ students were open to discussion and feedback and willing to retrace their steps/redothe work in order to advance to a solution◦ they were willing to ‘trust’ each other’s knowledge (implicit or not, technical ornot), accepting the multi-disciplinary nature of the skills and knowledge requiredto achieve the learning objectiveswithin an environment that enabled the advantages of ‘flow time’ to be exploited• be challenged◦ students were motivated by the (increasing) complexity of the task, and were ableto focus on cognitive and interpersonal skills to adapt to the changes required.Chapters 6 to 8 provide the basis for these findings.9.4.2 Implications for the discipline of RE and RE educationIt follows, therefore that the contribution to knowledge is a consequence of the alignmentbetween practitioner expectations and formal education within that discipline:408

• the research establishes a relationship between the characteristics of RE (identifiedpartially in studies of practitioner activity) and established models of learning. Thesecharacteristics inform the development of a conceptual model for RE education. Thisresearch, through a sequence of Action Research cycles, develops a learning model thataddresses more explicitly the gaps in formal education identified by practitioners. Basedon the constructivist paradigm, this Studio Learning model exploits the reflective practitionerconcept of professional learning by incorporating some elements of CognitiveApprenticeship with components of Problem-based Learning and creativity-enhancingstrategies• the research also confirms that there is a relationship between characteristics exhibitedby learners and the learning environment provided. Students display aptitudes forspecific learning environments – those whose approaches to learning align with thelearning model appear to gain increased benefits• for the discipline of RE, this suggests that students with specific characteristics, taughtin a manner that is appropriate to the discipline, have greater potential to becomingcompetent practitioners: a case of the sum of the alignments being greater that itsparts. Figure 9.3 provides a summary of this model of alignment.Figure 9.3: A conceptual model of alignment for RE educationThe pragmatics of this research indicate that education for RE occurs in an Engineeringcontext. The computing space schematic for RE (see Figure 2.9) confirms that the basis409

for the landscape is SE 3 . As noted in Chapter 2, there is a proliferation of Engineeringdegrees of software. The suitability of this Engineering ethos for learning RE requires greaterexamination.Most Engineering instruction is oriented toward Masterminds – introverts (lecturing and individualassignments rather than active class involvement and cooperative learning), intuitors(emphasis on science and math fundamentals rather than Engineering applications and operations),thinkers (emphasis on objective analysis rather than interpersonal considerations indecision-making), and judgers (emphasis on following the syllabus and meeting assignmentdeadlines rather than on exploration of ideas and creative problem solving). In addition, traditionalEngineering education does little to provide students with the systemic perspectiveon individual subjects (a global perspective) they need to function effectively, and the oneswho take too long to get it by themselves are at risk academically (Felder and Brent, 2005).Holt and Solomon (1996) point out that, while Engineering education relies heavily on problemsolving and engineering science (Kolb’s Convergent and Assimilators), it tends to excludeDivergent and Accommodators from effective learning, and limits the opportunities of alllearners to develop the skills required for proficiency in two other key areas of Engineering:design and invention (requiring a divergent approach), and business management (requiringaccommodative skills). The work of Lumsdaine and Lumsdaine (1995) suggests that between20 and 40% of student intake to Engineering is lost through not catering for studentswith strengths in communications and team work or creative problem solving, synthesis anddesign.However, as discussed briefly in Section 9.4.3 below, the focus of Engineering education inAustralia has been changing to include equipping graduates for lifelong learning as well asproviding a broader education with a wider range of backgrounds. This change was advocatedin the mid 1990s (IEAust, 1996), and is an acknowledgement that the development of studentskills and understanding in generalisable and transferable skills is a necessary dimension ofprofessional education (McLaughlan and Kirkpatrick, 2004).The Studio Learning model suggests that the Engineering environment that was the contextfor this study (described in Appendix A) is neither appropriate nor inappropriate for REeducation. Studio Learning transcends the limitations noted above, so that the imaginativeability of the Diverger and the intuitive problem solving of the Accomodator are activelyencouraged. In addition, the cohesion provided by the professional accreditation requirements3 however it should be noted that the model is not validated. This could be considered as future work, asis the extension of the landscape to encompass the soft and cognitive skills identified as part of the landscapeof RE410

for Engineering degrees suggests an emphasis on soft skills within the learning environment.As Bentley et al (1999) suggest (discussed in Chapter 2), these are sometimes given only‘lip service’ in tertiary education curricula for IS. Of course the issue of students with thepotential for RE competence enrolling in Engineering programmes is a moot point, and alsodeserving of investigation.9.4.3 Implications for Engineering educationThroughout this study, the Engineering context has been acknowledged. Therefore, theimplications for Engineering education, although not the prime focus od this work can alsobe stated.Within Engineering education, findings from research into learning have strategic implications:in terms of ensuring academic success, in the favouring of students with particularlearning and personality styles and with the development of lifelong learning skills in thosestudents so that professionally they can adapt to the changing contexts they will encounterprofessionally.Academic successAcademic success may be measured by the rate of graduation and eventual practice as aprofessional within the discipline of graduation. Attrition, both from the curriculum andfrom the profession indicate some complex lack of matching between the individual (student)and the discipline chosen. Two studies (of over 25 000 students in over 300 institutions inthe US (Astin, 1993; Moller-Wong and Eide, 1997)) indicate that in Engineering less than50% of first-year Engineering students eventually graduate in the discipline.Other studies indicate this attrition is not based on academic ability to cope with Engineering,but rather involves a complex set of factors including students’ attitudes toward Engineering,their self-confidence levels, and the quality of their interactions with instructors and peers(Astin, 1993) along with their aptitude for Engineering. In turn, students’ attitudes towardEngineering and confidence levels are strongly related to their classroom experience, withEngineering students highly dissatisfied with the quality of instruction they receive, comparedto students in other disciplines (Astin, 1993).Additional research findings (UWA, 1996; Felder, 1996) indicate that Engineering students’motivation and success can be adversely affected if their learning styles, and the learningstyles of the staff teaching them, are not taken into account: there is considerable evidencethat a mismatch between lecturers’ expectations of the way students learn and students’ own411

individual preferred learning styles, disadvantage to students. The research suggests thatthese mismatches lead to lack of motivation and interest in students, affecting students’ success(UWA, 1996; Felder, 1996; Zywno and Waalen, 2001). This study supports these findings,and indicates the importance of additional alignment – teacher to learning environment.ProfilingStudies of Engineering students indicate a dominance of the Rationalist temperament (NT),intrigued by systemic entities (machines and organisms) and their complexity, and highlyskilled in strategic analysis. Variants are Architects (INTP) who enjoy figuring out structure,build, configuration, spatiality of things); Masterminds (INTJ) who take on directiveactivities such as contingency planning or organising role, with an inclination to take charge;Inventors (ENTP) who are functional Engineers, with a tolerance for and enjoyment of complexproblems; Fieldmarshals (ENTJ) who work best at situational organisation: able tocommunicate their vision to others.Felder and Silverman (1988, p 680) sum up the situation:Learning styles of most engineering students and teaching styles of most engineeringprofessors are incompatible in several dimensions. Many or most engineeringstudents are visual, sensing, inductive, and active and some of the most creativestudents are global; most engineering education is auditory, abstract (intuitive),deductive, passive, and sequential. These mismatches lead to poor student performance,professorial frustration, and a loss to society of many potentially excellentengineers.This results in between 20 and 40% of student intake to Engineering lost through not cateringfor students with strengths in communications and team work or creative problem solving,synthesis and design (Lumsdaine and Lumsdaine, 1995). It would seem probable that studentswith characteristics not catered for in traditional Engineering education models maybe self-selecting out of Engineering due to an ‘inhospitable learning climate’ (Lumsdaine andLumsdaine, 1995). The figures for 4th Year Engineering students in Table A.1 could be seento support this.Flexibility and creativity for lifelong learningThe focus of Engineering education has changed to include equipping graduates for lifelonglearning as well as providing a broader education with a wider range of backgrounds. Thischange was advocated in the mid 1990s (IEAust, 1996), and is an acknowledgement that the412

development of students skills and understanding in generalisable and transferable skills isa necessary dimension of professional education (McLaughlan and Kirkpatrick, 2004). Activelearning approaches (which include collaborative learning, problem-based learning, casemethods and combinations of roleplays and simulations) are advocated to engage studentsin such higher order thinking tasks as analysis, synthesis and evaluation (Bonwell and Eison,1991). Active learning methods attemptto develop the cognitive [knowledge, understanding and thinking] and affective[emotive] dimensions of the learning process in such a way that learners’ activeinvolvement in the learning is improved(Learning and Teaching Support Network (LTSN), 2003).In the UK criticism has arisen regarding Engineering graduates’ ability to improvise withthe equipment and resources at hand in dealing with day-to-day operational problems. Theneed for flexibility, fluency and originality in day-to-day dealings, which typically define thecreative effort (Guilford, 1967), is seen as a lack in training. A research focus has also emergedin developing creativity in (engineering) design (Winograd, 1996; Cropley and Cropley, 2000),with creativity described as a balance of convergent and divergent thinking appropriate tothe situation (Nickerson, 1999). The process of design is seen to rarely be convergent, in thesense of being directed towards a single preferred solution.The implication of this is that approaches such as the Studio Learning model developedthrough this research have increased importance in an Engineering context. The commentsby the accreditation panel support this (see Section 9.3.2).9.5 Future workFrom the research perspective, some issues were identified both from observation of theStudios in general, from comparison of the SE Studios with other Engineering Studios andfrom discussions with staff and students.Of prime importance was the development of staff in order to support the model of learning.The adoption of Studio Learning across the School may be considered ’transformational’,a second order change that required a fundamental shift in behaviour and resources (Levy,1986). A correlation appeared to exist between attendance at staff development sessions andthe ‘success’ of the specific Studio. The implication of this result is that no staff should teachwithin the Studio Learning environment without adequate and appropriate training (both413

in the learning theory behind the model and in facilitation techniques). While it is clearfrom the literature (eg Barrows (1986) and Woods (1996a)) that an understanding of theunderpinning pedagogical basics is necessary, as are special facilitation skills, it has resourceimplications within the tightening finance environment prevailing at many universities. Inan Australian context the implication is even greater, given University academics are notrequired to have or obtain teaching qualifications. A ‘reward’ structure that acknowledgesefforts in this area may go some way to addressing this issue. Future research should addressthis issue.Another issue revolves around the nature of the learning environment developed during thisproject. Prior to 2003 students were able to undertake ENG260 externally. They had successfullycompleted the unit from Germany, USA, Melbourne as well as from Perth with theresearcher/lecturer-in-charge in Europe. The PBL environment established assumed, however,same place same time interaction between students (as opposed to the lecturer or the‘virtual secondment’). In the development of a new learning environment this was considereda sensible (as well as less risky) approach. Lee and Kim (2005) are the latest to suggest interactingwith computer-mediated protocols can make it difficult for students in a web-basedcollaborative environment to negotiate the shared meaning necessary for effective learning.In addition, poor communications channels may make it difficult for learners to performcognitive activities when solving problems through social interaction.Although the Department review mentioned above mandated internal enrolment only for allDesign Studios, this may be considered a backward step. Future research should examinethe place for distributed Studio Learning and the infrastructure required to support it.The outcomes noted above are based on Requirements Engineering education and practicein a specific context. Nevertheless, when viewed in the context of the literature and theperspectives of practitioners, the importance of effective alignment between education andpractice should be broadly applicable. The research establishes a relationship between thecharacteristics of the discipline and established models of learning. It also goes some way toconfirming that there is a relationship between characteristics exhibited by learners and thelearning environment provided.It is feasible, therefore, to hypothesise that an alignment between discipline and the learningmodel in the first instance and between the learning model and the students engagedin it is crucial to the development of competent practitioners. This hypothesis, modelledconceptually in Figure 9.3 is deserving of testing in the wider discipline/education area.It is also feasible to hypothesise that other disciplines would benefit from an investigation ofthe alignment between their characteristics and education for its practitioners. Where pro-414

fessional education was previously based on the ‘normative’ education described in Chapters2 and 3, current trends are to apply alternative learning models. This is particularly true ofPBL.However, recent discussions of the ‘failure’ of PBL in critical disciplines (eg Glew (2003)discussing PBL in medicine) supports the findings discussed in this thesis that extensivecommitment and training are required to successfully (firstly) align the learning model withthe discipline to (ultimately) provide a discipline-appropriate learning environment.The discipline of Requirements Engineering is shown to be one of multiple perspectives. Practitionersreport a discipline of knowledge discovery facilitated by opportunistic behaviour andcreativity, requiring experience in a wide range of soft and metacognitive skills in order topractice competently. The literature reports a smooth evolutionary process of goal-directedproblem solution 4 . What deserved examination, not attempted in this research, is the relationshipbetween the reporting and the needs of the ‘client’. As an example of this dichotomy,this thesis aims to present a logical argument supported by evidence for the examiner, whichonly incidently models how the research was undertaken. For communicating this knowledgewith that client, a ‘structured’ approach is mandatory. However, for presenting the disciplinein a learning context, such an approach provides a false model of practice. In that contextthis research shows that a knowledge discovery model of the discipline addresses more closelythe competency expectations of practitioners. It is suggested that further study examine themodel of RE as a discipline of Learning and Teaching (discussed in Chapter 3) in relation tothe nature of the communications/dialogue for different stakeholders.SE education must include an element of training during and after formal education. Mc-Connell and Tripp (1999) suggest at least four years of apprenticeship for Software Engineers.The reality is that new graduates are expected to perform on the same level as their experiencedcounterparts – the idea of an apprenticeship period is foreign to software development.The best that can be hoped for is a sympathetic, experienced mentor as coach.The progression to Studio Learning in RE was a journey undertaken by all participants ofthe study: in empowering graduates to be industry-ready, the researcher benefits from adouble-loop approach as the espoused theory of teaching becomes aligned with the theoryin practice. It provides learning situations to examine and experiment with our theories ofaction (Argyris and Schön, 1974). For the student, the collaborative nature of the learningenvironment that has evolved transcends the classroom, fostering self-directed learning andreflective practice that integrates class and work experience.4 the thread from the RE online forum discussed in Chapter 2 looks at this issue from a practitionerperspective415

The Studio Learning model provides a meta-knowledge environment that informs the learnerabout the knowledge assets acquired as part of the programme of study. Knowledge aggregationthrough learning concepts and experiences from the Studio Learning environmentis expected to lead to continuous improvement and motivate students to have a life-longlearning interest in their chosen profession.Informal monitoring of students involved in this approach suggest they integrate into theworkplace easily. However, this is deserving of further study – to track these students asgraduates and practitioners. A grant proposal has been submitted within the MurdochUniversity environment for this purpose.The future will test the long-term wisdom of this approach.9.6 In conclusionAs we learn more about how students learn, and what they need to learn in order to practice ascompetent professionals in their chosen discipline, we move further from traditional teachingand closer to the concept of learning as a reflection on professional practice undertaken byboth teachers and learners. This view of professional education has implications for thedesign of teaching:• academic learning must be situated in the domain of the objective: the activities mustmatch that domain• academic teaching must address both the direct experience of the world, and the reflectionon that experience that will produce the intended way of representing it.(Laurillard, 1993)This research shows the gap between practitioner expectations of formal education for REcan be reduced through fine-grained alignment of the learning environment with the characteristicsof the discipline. While technical knowledge acquired by students is importantin that it acts as a ‘filter’ for graduate employment, of greater impact on the professionalcompetence is the focus on soft and metacognitive skills. These are learnable within a formaleducation environment, albeit through the application of non-traditional learning models.However, what both practitioner studies (especially the work of Minor (2004)) and this studyhint at is the importance of individual characteristics and abilities. Minor’s participantsindicated a Personality component to competent RE practice. This research suggests that an416

alignment between the learner and the (discipline-aligned) learning model enhances studentlearning of that discipline.At this point in the development of the IT discipline, there is no formal education programmesthat specifically address RE: as the model curricula show, this key area of knowledge forsoftware development is addressed to a lesser rather than greater extent within suggestedprogrammes of study.The suggestion can be made, drawn from the findings of this research, that this state ofaffairs should not be acceptable while activities within the RE discipline continue to bemajor issues in the development of useful software systems. What is needed is a concertedeffort to augment appropriate RE skills within the software development community.In the first instance, and acknowledging the importance of learner maturity (intellectual,epistemological and experiential) the development of post-undergraduate programmes in REcould address the gap in knowledge identifies by practitioners as existing.This research also hints that not every IT professional would make a competent RE practitioner– certain personality and learning traits indicate an enhanced potential for success.Therefore the development of diagnostic instruments that specifically address such traits, andthe application of these to select students for RE (in the first instance for the post-graduateprogramme) goes even further in addressing the practitioner gaps identified.Industry requires RE professionals who integrate into the organisational structure, and havethe ability to adapt models, skills and analytical techniques quickly. In the longer term, therefore,in the same way that students may designate themselves, at the end of an undergraduatedegree, as an graduate of CS, IS or SE, it should be possible to proclaimI am a Requirements Engineer.417

BibliographyAdams-Price C E (ed) (1998) Creativity and Successful Aging, New York: Springer-VerlagAdelson B and Soloway E (1985) “The role of domain experience in software design”, IEEETransactions on Software Engineering, 11(11), pp 1351–1360Adobe Systems Inc. (2006) “Adobe Flash Player”,flashplayer/http://www.adobe.com/products/Akin O (1990) “Necessary conditions for design expertise and creativity”, Design Studies,11(2), pp 107–113Al-Rawas R and Easterbrook S (1996) “Communication problems in Requirements Engineering:a field study”, in “First Westminster Conference on Professional Awareness inSoftware Engineering”, LondonAlbanese M and Mitchell S (1993) “Problem-based learning: a review of the literature on itsoutcomes and implementation issues”, Academic Medicine, 68(1), pp 52–81Albert R S (1996) “Some reasons why childhood creativity often fails to make it past pubertyinto the real world”, in Runco M A (ed) “Creativity from Childhood through Adulthood:new directions for child development”, San Fransisco: Jossey-Bass, pp 43–56Allen B S and Otto R G (1996) “Media as lived environments: the ecological psychology ofeducational technology”, in Jonassen (1996), pp 199–225Alpay E and Ireson J (2006) “Self-theories of intelligence of engineering students”, EuropeanJournal of Engineering Education, 31(2), pp 169–180Amabile T M (1983) The Social Psychology of Creativity, New York: Springer-VerlagAmabile T M (1996) Creativity in Context, Boulder (CO): Westview PressAmerican College of Medical Practice Executives (ACMPE) (2002) Defining the profession:A Guide to the Body of Knowledge for Medical Practice Management, Englewood (CO):Medical Group Management Association, http://www.mgma.com/acmpe/bokguide.cfmAnderson G L and Herr K (1999) “The new paradigm wars: is there room for rigorouspractitioner knowledge in schools and universities?”, Educational Researcher, 28(5), pp12–21, 40Anderson G L, Herr K and Nihien A S (1994) Studying your own school: an educators guideto qualitative practitioner research, Thousand Oaks (CA): CorwinAnderson L and Krathwohl D A (2001) Taxonomy for Learning, Teaching and Assessment:a revision on Bloom’s taxonomy of educational objectives, New York: LongmanAndresen L, Boud D and Cohen H (1995) “Experience-based learning”, in Foley G (ed)“Understanding Adult Education and Training”, Sydney: Allen and Unwin, pp 207–215Antill L (1985) “Selection of a research method”, in Mumford et al (1985), pp 203–215418

Argyris C and Schön D A (1974) Theory in practice: increasing professional effectivenes, SanFransisco: Jossey-BassArgyris C and Schön D A (1978) Organisational Learning: a theory of action perspective,Reading (MA): Addison WesleyArmarego J (2002) “Advanced Software Design: a case in problem-based learning”, in LethbridgeT, McCracken W M and Lutz M (eds) “Proceedings: CSEE&T2002 15th Conferenceon Software Engineering Education and Training, Covington, Kentucky”, Los Alamitos(CA): IEEE Computer Society Press, pp 44–54Armarego J (2004a) “Learning Requirements Engineering within an engineering ethos”, inNguyin L, Armarego J and Swatman P A (eds) “Towards effective Requirements Engineering:proceedings of AWRE’04 (9th Australian Workshop on Requirements Engineering),Adelaide”, Adelaide: University of South Australia, pp 11.1–11.11Armarego J (2004b) “Student perceptions of quality learning: evaluating PBL inSoftware Engineering”, in Hobson J (ed) “Seeking Educational Excellence: 13thTeaching Learning Forum”, Perth: Teaching Learning Centre, Murdoch University,http://www.tlc.murdoch.edu.au/tlconference/abstracts/ArmaregoJ.pdfArmarego J (2004c) “Towards achieving Software Engineering wisdom”, in Sheehy F andStrauble B (eds) “Transforming Knowledge into Wisdom: holistic approaches to teachingand learning. Proceedings of HERDSA (The Higher Education Research and DevelopmentSociety of Australasia) Conference Miri Sarawak”, HERDSA, volume 27, pp 5–15Armarego J (2005) “Educating agents of change”, in Lethbridge T C and Port D (eds)“Proceedings: CSEE&T2005 18th Conference on Software Engineering Education andTraining, Ottawa”, Los Alamitos (CA): IEEE Computer Society Press, pp 181–194Armarego J (in press) “Aligning learning with industry requirements”, in Lowry and Turner(in press), chapter 8Armarego J and Clarke S (2003) “Preparing students for the future: learning creative softwaredevelopment - setting the stage”, in Bond C and Bright P (eds) “Learning for an UnknownFuture: Proceedings of HERDSA (The Higher Education Research and Development Societyof Australasia) Conference Christchurch, New Zealand”, HERDSA, volume 26, pp17–24, http://surveys.canterbury.ac.nz/herdsa03/pdfsref/Y1003.pdfArmarego J and Fowler L (2005) “Orienting students to Studio Learning”, in Radcliffe D (ed)“Proceedings of the 2005 ASEE/AaeE 4th Global Colloquium on Engineering Education”,Sydney: Australasian Association for Engineering EducationArmarego J, Fowler L and Roy G (2001) “Constructing Software Engineering Knowledge:development of a learning environment”, in Ramsey D, Bourque P and Dupuis R (eds) “Insearch of a Software Engineering profession: Proceedings of CSEE&T2001 14th Conferenceon Software Engineering Education and Training, Charlotte (NC)”, Los Alamitos (CA):IEEE Computer Society Press, pp 258–267Armarego J and Minor O (2005) “Studio Learning of Requirements: towards aligning teachingto practitioner needs”, in Zowghi D and Huang J (eds) “Proceedings of REET 2005 1stInternational Workshop on Requirements Engineering Education and Training”, Paris:Université Paris 1 - Sorbonne, http://crinfo.univ-paris1.fr/RE05/workshop.htmlAstin A W (1993) What Matters in College: four critical years revisited, San Francisco:Jossey-Bass419

Aurum A, Handzic M and Gardiner A (2003) “Supporting creativity in software development:an application to IT education”, in McGill (2003), pp 77–87Australian Computer Society (ACS) (2002) Examinations in Information Technology, Sydney:Australian Computer Society, http://www.acs.org.au/ACS Syllabuses 2002.pdfAustralian Qualifications Framework (2002) Australian Qualifications Framework: ImplementationHandbook, Carlton South (VIC): Australian Qualifications Framework AdvisoryBoard, 3rd edAvison D E, Lau F, Myers M and Nielsen P A (1999) “Action research”, Communicationsof the ACM, 42(1), pp 94–97Bach J (1997) “SE education: we’re on our own”, IEEE Software, 14(6), pp 26,28Bach J (1999) “Reframing requirements analysis”, IEEE Computer, 32(2), pp 120–122Baer J M (1988) “Long term effects of creativity training with middle-school students”,Journal of Adolescence, 8, pp 183–193Bagert D J, Hilburn T B, Hislop G, Lutz M, McCracken M and MengelS (1999) Guidelines for Software Engineering Education, CMU/SEI-99-TR-032,Pittsburg (IL): Software Engineering Institute, Carnegie-Mellon University, version1.0 ed, http://www.sei.cmu.edu/publications/documents/99.reports/99tr032abstract.htmlBaker A, Kolb D and Jensen P (2002) Conversational learning: an experiential approach toknowledge creation, Westport (CT): QuorumBalfour A (1981) “Captive of Love and Ignorance: Architecture, Education and Practice”,in “Architecture Education Study. Volume I”, New York: Cooper Hewitt, pp 775–801Bandura A (1977) Social Learning Theory, Saddle River (NJ): Prentice HallBanks D A (2003) “Belief, inquiry, argument and reflection as significant issues in learningabout Information Systems development methodologies”, in McGill (2003), pp 1–10Barab S A and Duffy T (1999) “From practice fields to communities of practice”, in Jonassenand Land (2000), pp 25–26, http://ceo.cudenver.edu/ brent wilson/SitCog.htmlBarab S A, Hay K E and Yamagata-Lynch L C (2001) “Constructingnetworks of action-relevant episodes: an in situ researchmethodology”, Journal of the Learning Sciences, 10(1-2), pp 63–112,http://inkido.indiana.edu/research/onlinemanu/papers/barabjlscna.pdf, also inBarab, S and Kirshner, D (eds) ”Rethinking methodology in the learning sciences”Barbour R H (2005) “Metaknowledge in tertiary education”, in Barrie S and SutherlandK (eds) “Higher Education in a Changing World: Proceedings of HERDSAAnnual International Conference”, Sydneys: University of Sydney, pp 26–33,http://www.itl.usyd.edu.au/herdsa2005/introduction.phpBarrows H S (1986) “A taxonomy of PBL methods”, Journal of Medical Education, 20(6),pp 481–486Barrows H S and Myers A C (1993) Problem-Based Learning in Secondary Schools, Springfield(IL): Southern Illinois University Medical SchoolBarrows H S and Tamblyn R M (1980) Problem-based Learning, an Approach to MedicalEducation, New York: Springer-Verlag420

Bartlett C (1932) Remembering: an experimental and social study, Cambridge (MA): CambridgeUniversity PressBatra D and Davis J G (1992) “Conceptual data modelling in database design: similaritiesand differences between expert and novice designers”, International Journal of Man-Machine Studies, 37, pp 83–101Baxter-Magnola M B (2001) “A constructivist revision of the measure of epistemologicalreflection”, Journal of College Student Development, 42(6), pp 520–534BCS Information Systems Examination Board (2002) Diploma in Business Systems Development- Specialising in System Analysis and Design, Swindon, Wiltshire: British ComputerSociety, http://www.bcs.org/examBecker H (1986) Writing for Social Scientists, Chicago: University of Chicago PressBell T E and Thayer T A (1998) “Software Requirements: are they really a problem?”,in “Proceedings of the Second International Conference on Software Engineering”, SanFransisco, pp 61–68Benbasat I, Dexter A S and Mantha R W (1980) “Impact of organizational maturity oninformation systems skill needs”, MIS Quarterly, 4(1), pp 21–34Benbasat I and Zmud R W (1999) “Empirical research in Information Systems: the practiceoe relevance”, MIS Quarterly, 23(1), pp 3–16Bennett N, Dunne E and Carré C (2000) Skills Development in Higher Education and Employment,Buckingham (UK): The Society for Research into Higher Education and OpenUniversity PressBenson S (2003) “Metacognition in Information Systems education”, in McGill (2003), pp213–222Bentley J F, Lowry G R and Sandy G A (1999) “Towards the compleat information systemsgraduate: a problem based learning Approach”, in Hope B and Yoong P (eds) “Proceedingsof the 10th Australasian Conference on Information Systems”, Australian ComputerSociety/ACIS: Wellington (NZ), pp 65–75Bergquist C (1999) “A comparative view of creative theories: psychoanalytic, behaviouristand humanist”, http://www.vantagequest.org/trees/comparative.htmBerkenkotter C and Huckin T (1995) Genre Knowledge in Disciplinary Communication,Erlbaum: Hillsdale (NJ)Bieri J, Atkins A L, Briar S and et al (eds) (1966) Clinical and Social Judgement: thediscrimination of behavioural information, New York: John WileyBiggs B J and Moore P J (1993) The Process of Learning, Englewood Cliffs (NJ): PrenticeHallBiggs J (1999) Teaching for Quality Learning at University: what the student does, Buckingham(UK): Open University PressBiggs J B (1970) “Faculty pattern in study behaviour”, Australian Journal of Psychology,22, pp 161–174Biggs J B (1985) “The role of metalearning in study processes”, British Journal of EducationalPsychology, 55, pp 185–212Billett S (1994) “Searching for authenticity”, Vocational Aspect of Education, 46(2), pp 3–16421

Billett S (1996) “Towards a model of workplace learning: the learning curriculum”, Studiesin Continuing Education, 18(1), pp 43–58Bloom B S, Englehart M D, Furst E J, Hill W H and Krathwohl D R (1956) Taxonomyof Educational Objectives. The classification of educational goals, Handbook 1: cognitivedomain, new York: Longmans, Green and CoBloomfield B P (1992) “Understanding the social practices of systems developers”, Journalof Information Systems, 2, pp 198–206Boden M A (1997) The Creative Mind: myths and mechanisms, London: Weidenfeld andNicolsonBoehm B W (1981) Software Engineering Economics, Englewood Cliffs (NJ): Prentice HallBoehm B W, Bose P, Horowitz E and Lee M J (1995) “Software requirements negotiationand renegotiation aids: a Theory-W based approach”, in “International Conference onSoftware Engineering Proceedings”, Association for Computing Machinery, pp 243–253Boettcher J and Conrad R M (1999) Faculty guide for moving teaching and learning to theweb mission, Viejo (CA): League for Innovation in the Community CollegeBoland R J (1985) “Phenomenology: a preferred approach to research on information systems”,in Mumford et al (1985), pp 193–201Bonwell C C and Eison J A (1991) Active learning: creating excitement in the classroom,Technical Report ED340272, Eric Digest, Washington (DC)Borg W R, Gall J P and Gall M D (eds) (1993) Applying Educational Research: a practicalguide, White Plains (NY): Longman, 3rd edBoud D (1985) “Problem-based learning in perspective”, in Boud D (ed) “Problem-basedLearning in Education for the Professions”, Canberra: Higher Education Research Societyof Australasia, pp 13–18Boud D and Falchikov N (2005) “Redesigning assessment for learning beyond higher education”,in Barrie S and Sutherland K (eds) “Higher Education in a Changing World:Proceedings of HERDSA Annual International Conference”, Sydneys: University of Sydney,pp 34–41, http://www.itl.usyd.edu.au/herdsa2005/introduction.phpBoud D and Feletti G (eds) (2001) The Challenge of Problem-based Learning, London: KoganPage, 2nd edBoud D, Keough R and Walker D (1985) Reflection: Turning Experience into Learning,London: Kogan PageBoud D and Pascoe J (1978) “Postscript: conceptualising experiential education”, in BoudD and Pascoe J (eds) “Experiential Learning: developments in Australian post-secondaryeducation”, Sydney: Australian Consortium on Experiential Education, Sydney TeachersCollege, pp 23–34Bourque P and Dupuis R (2001) SWEBOK: guide to the Software Engineering Bodyof Knowledge: Trial Version, Los Alamitos (CA): IEEE Computer Society Press,http://www.swebok.org/Bowden J and Marton F (1998) The University of Learning: beyond quality and competencein higher education, London: Kogan PageBradley A (2004) Report of Accreditation visit: Murdoch University, Technical Report Draft3, Accreditation Board, Engineers Australia, Canberra422

Brandt D S (1997) “Constructivism: teaching for understanding of the internet”, Communicationsof the ACM, 40(10), pp 112–117Bransford J D, Brown A L and Cocking R R (eds) (2000) How People Learn: brain, mind,experience, and school, Washington (DC): National Academy PressBrew A (1999) “Conceptions of research and scholarship: implications for higher educationteaching and learning”, in “Proceedings of the AARE/NZARE Conference”, Melbourne:Australian Association for Research in EducationBrew A (2001) The Nature of Research: inquiry into academic contexts, London: RoutledgeFalmerBridges E M (1992) Problem based learning for administrators, University of Oregon, Eugene,Oregon: ERIC Clearinghouse on Educational ManagementBrooks F P (1986) “No silver bullet - essence and accidents of software engineering”, in KuglerH J (ed) “Proceedings of Information Processing 86: the IFIP 10th World Conference”,Amsterdam, pp 1069–1076Brophy K (1998) Creativity, Melbourne: Melbourne University PressBrown A L (1980) “Metacognitive development and reading”, in Spiro R J, Bruce B andBrewer W (eds) “Theoretical Issues in Reading Comprehension”, Hillsdale (NJ): ErlbaumBrown A L (1992) “Design experiments: theoretical and methodological challenges in creatingcomplex interventions in classroom settings”, The Journal of The Learning Sciences, 2(2),pp 141–178Brown G, Bull J and Pendlebury M (1997) Assessing Student Learning in Higher Education,London: RoutledgeBrown J S, Collins A and Duguid P (1989) “Situated cognition and the culture of learning”,Educational Researcher, 18, pp 32–42Brown J S and Duguid P (1991) “Organizational learning and communities-of-practice: towardsa unified view of working, learning and innovation”, Organization Science, 2(1), pp40–57Bruner J S (1960) The Process of Education, New York: VintageBruner J S (1962) On Knowing, Essays for the Left Hand, Cambridge (MA): Harvard UniversityPressBruner J S (1985) “Models of the learner”, Educational Research, 14(6), pp 5–8Bruner J S (1986) Actual Minds, Possible Worlds, Cambridge (MA): Harvard UniversityPressBruner J S (1990) Acts of Meaning, Cambridge (MA): Harvard University PressBubenko J A (1995) “Challenges in Requirements Engineering”, in “2nd IEEE InternationalSymposium on Requirements Engineering”, Los Alamitos (CA): IEEE Computer SocietyPress, pp 160–162Budgen D (1999) “Software design methods; life belt or leg iron?”, IEEE Software, 16(5), pp133–136Budgen D (2003) Software Design, Harlow (Essex): Pearson Education Ltd423

Budgen D (2004) “Computing Curricula Software Engineering: position paper”, in “Designand Assessment of Trustworthy Software-Based Systems: Proceedings of the 28th AnnualInternational Computer Software and Applications Conference (COMPSAC04)”, LosAlamitos (CA): IEEE Computer Society Press, pp 174–175Bulow I v (1989) “The bounding of a problem situation and the concept of a system’sboundary in soft systems methodoligy”, Journal of Applied Systems Analysis, 16, pp 35–41Bunge M (1967) Scientific Research, Berlin: Springer-VerlagBunge M (1984) “Philosophical inputs and outputs of technology”, in Bugliarello G andDonner D (eds) “History and Philosophy of Technology”, Urbana (IL): University of IllinoisPress, pp 263–281Buzan T and Buzan B (eds) (1994) The Mind Map Book : how to use radiant thinking tomaximize your brain’s untapped potential, DuttonCandlin C N (1998) “Reasearching writing in the academy: participants, texts, processes andpractices”, in Candlin and Plum (1998), pp 1–28Candlin C N and Plum G A (eds) (1998) Reasearching Academic Literacies, Sydney:NCELTR, Macquarie UniversityCarneson J, Delpierre G and Masters K (2000) “Designing and Managing Multiple ChoiceQuestions”, http://www.uct.ac.za/projects/cbe/mcqmanCarr W and Kemmis S (eds) (1986) Becoming Critical: education, knowledge and actionresearch, Lewes (UK): FalmerCarrington D (2000) SWEBOK: knowledge area description for Software Engineering Infrastructure,Los Alamitos (CA): IEEE Computer Society/ACM, http://www.swebok.org/,version 0.6Carroll J and Kellogg W (1989) “Artifact as theory nexus: hermeneutics meets theory-baseddesign”, in “Proceedings of the SIGCHI Conference on Human Factors in ComputingSystems: wings for the mind”, Los Alamitos (CA): ACM Press, pp 7–14Carroll J M and Swatman P A (1997) “How can the requirements engineering process beimproved?”, in J S D (ed) “Proceedings of the 8th Australasian Conference on InformationSystems”, Adelaide: University of South AustraliaCarroll J M and Swatman P A (1999) Managing the RE process: lessons from commercialpractice, School of Management Information Systems Working Paper 1999, DeakinUniversity, Australia, check all thisCash J I and Lawrence P R (eds) (1989) The Information Systems Research Challenge:qualitative research methods, Boston (MA): Harvard Business SchoolChafee R (1977) “The Teaching of Architecture at the Ecole des Beaux-Arts”, in Drexler A(ed) “The Architecture of the Ecole des Beaux-Arts”, New York: Museum of Modern Art,pp 171–202Chambers E (1992) “Workload and the quality of student learning”, Studies in Higher Education,17(2), pp 141–154Chase W G (1973) Visual Information Processing, New York: Academic PressChase W G and Simon H A (1973) “Perceptions in chess”, Cognitive Psychology, 4(1), pp55–81424

Checkland P B (1981) Systems Thinking, Systems Practice, Chichester: WileyCheckland P B (1991) “From framework through experience to learning: the essential natureof action research”, in Nissen H E, Klein H K and Hirschheim R (eds) “InformationSystems Research: contemporary approaches and emergent traditions”, Amsterdam:North-Holland, pp 397–403Checkland P B and Holwell S (1998) Information, Systems and Information Systems - makingsense of the field, Chichester: WileyCheckland P B and Scholes J (1990) Soft Systems Methodology in Action, Chichester: WileyChi M T H and Bassock M (1989) “Learning from examples via self-explanation”, in Resnick(1989), pp 251–282Chi M T H, Glaser R and Rees E (1982) “Expertise in problem solving”, in Sternberg R J (ed)“Advances in the Psychology of Human Intelligencee”, Hillsdale (NJ): Erlbaum, volume 1,pp 7–75Choi J and Hannafin M J (1995) “Situated cognition and learning environments: roles,structures and implications for design”, Educational Technology Research & Development,43(2), pp 53–69Chrispeels J H and Martin K J (1998) “Becoming problem solvers: the case of three futureadministrators”, Journal of School Leadership, 8, pp 303–331Clark C (2005) “The structure of educational research”, British Educational Research Journal,31(3), pp 289–308Clark R E and Voogel A (1985) “Transfer of training principles for instructional design”,Educational Communications and Technology Journal, 33, pp 113–123Clarke R (2000) “Appropriate Research Methods for Electronic Commerce”,http://www.anu.edu.au/people/Roger.Clarke/EC/ResMeth.htmlCobb P and Bauersfeld H (1995) “The coordination of psychological and sociological perspectivein mathematics education”, in Cobb P and Bauersfeld H (eds) “Emergence ofMathematical Meaning: interaction in classroom cultures”, Hillsdale (NJ): Erlbaum, pp1–16, in Cobb and Bauersfeld (1995)Coffield F J, Moseley D V, Hall E and Ecclestone K (2004) Learning Styles for Post 16Learners: what do we know?, London: Learning and Skills Research Centre, University ofNewcastle upon TyneCohen E (ed) (2000) Curriculum Model 2000, Information Resource ManagementAssociation and the Data Administration Managers Association,http://gise.org/IRMA-DAMA-2000.pdfCollins A (1990) “Generalising from situated knowledge to robust understanding”, AmericanEducational Research Association General MeetingCollins A (1992) “Toward a design science of education”, in Scanlon E and OShea T (eds)“New Directions in Educational Technology”, New York: Springer-Verlag, pp 15–22Collins A, Brown J S and Holum A (1991) “Cognitive apprenticeship:making thinking visible”, American Educator, 15(3), pp 6–11, 38–46,http://www.kenton.k12.ky.us/DL/General/Readings/Collins CogApp.pdfCollins A, Brown J S and Newman S E (1989) “Cognitive apprenticeship: teaching the craftsof reading, writing and mathematics”, in Resnick (1989), pp 453–494425

Conklin J (2005) “Wicked problems and social complexity”, in Conklin J (ed) “DialogueMapping: building shared understanding of wicked problems”, Wiley, pp 1–25,http://cognexus.org/wpf/wickedproblems.pdfConn R (2002) “Developing Software Engineers at the C-130J Software Factory”, IEEESoftware, 19(5), pp 25–29Connelly F M and Clandinin D J (1990) “Stories of experience and narrative inquiry”, EducationalResearcher, 19(5), pp 2–14Cope C and Staehr L (2005) “Improving students learning approaches through interventionin an information systems learning environment”, Studies in Higher Education, 30(2), pp181–197Covington M V (1987) “Instruction in problem solving planning”, New York: CambridgeUniversity Press, chapter 13Cowan J (1998) On Becoming an Innovative University Teacher: reflection in action, Oxford):Oxford University PressCowdroy R and de Graaff E (2005) “Assessing highly-creative ability”, Assessment & Evaluationin Higher Education, 30(5), pp 507–518Craik F I M and Lockhart R S (1972) “Levels of Processing: a framework for memoryresearch”, Journal of Verbal Learning and Verbal Behaviour, 11, pp 671–684Craik F I M and Tulving E (1975) “Depth of processing and the retention of words in episodicmemory”, Journal of Experimental Psychology: General, 104, pp 268–294Creswell J (2003) Research Design: qualitative, quantitative and mixed methods approaches,Thousand Oaks: Sage, 2nd edCreswell J (2004) Educational Research: planning, conducting, and evaluating quantitativeand qualitative research, Upper Saddle River (NJ): Merrill, 2nd edCronin P (1997) Learning and assessment of instruction, TechnicalReport, Centre for Cognitive Science, University of Edinburgh,http://www.cogsci.ed.ac.uk/~paulus/Work/Vranded/litconsa.html ORwww.informatics.ed.ac.uk/publicationsCropley D H and Cropley A J (2000) “Creativity and innovation in the Systems Engineeringprocess”, in “Systems Engineering: A Decade of Progress and A New Century ofOpportunity: Proceedings of INCOSE 2000”, Minneapolis, pp 807–814Crotty M (ed) (1998) The Foundations of Social Research: meaning and perspective in theresearch process, London: SageCrowley D J (1982) Understanding Communication: the signifying web, New York: Gordonand BreachCsikszentmihalyi M (1996) Creativity: flow and the psychology of discovery and invention,New York: HarperPerennialCuff D (1991) Architecture: the story of practice, Cambridge (MA): MIT PressCurry L (1983) “An organisation of learning style theory and constructs”, in Curry L (ed)“Learning Style in Continuing Education”, Halifax: Dalhousie University426

Curtis B, Guindon R, Krasner H, Walz D, Elam J and Iscoe N (1987) Empirical studies ofthe design process: papers for the second Workshop on Empirical Studies of Programmers,Technical Report STP-260-87, MCC Technical ReportCurtis B, Krasner H and Iscoe N (1988) “A field study of the software design process forlarge systems”, Communications of the ACM, 31(11), pp 1268–1287Dahlgren M A (2000) “Portraits of PBL: course objectives and students study strategies incomputer engineering”, Instructional Science, 28(4), pp 309–329Dalgarno B (1996) “Constructivist computer assisted learning: theory an techniques”, in“Making New Connections: proceedings of ASCILITE ’96”, University of Adelaide, pp127–148DallAlba G (2005) “Improving teaching: Enhancing ways of being university teachers”,Higher Education Research and Development, 24(4), pp 361–372Dalton G W, Thompson P H and Price R L (1977) “The four stages of professional careers:a new look at performance by professionals”, Organizational Dynamics, 6(1), p 23Daniels H (1995) “Pedagogic practices, tacit knowledge and discursive discrimination: Bernsteinand post-Vygotskian research”, British Journal of Sociology of Education, 16(4), pp517–532Dart B (1998) “Teaching for improved learning in small classes”, in Dart B and Boulton-Lewis G M (eds) “Teaching and Learning in Higher Education”, Melbourne: AustralianConcil for Educational Research Ltd, pp 222–249Davidson J and Sternberg R J (1998) “Smart problem-solving: how metacognition helps”,in Hacker D J, Dunlosky J and Graesser C (eds) “Metacognition in Educational Theoryand Practice”, Mahwah (NJ): Erlbaum, pp 47–68Davis A M (1990) Software Requirements: analysis and specification, Englewood Cliffs (NJ):Prentice HallDavis G B, Gorgone J T, Couger J D and Longenecker H E (1997) IS’97: model curriculumfor undergraduate degree programs in Information Systems, Park Ridge (IL): Associationof Information Technology ProfessionalsDenzin N (1970) “Strategies of multiple triangulation”, in Denzin N (ed) “The Research Actin Sociology: a theoretical introduction to sociological method”, New York: McGraw-Hill,pp 297–313Department of Education, Science and Training (DEST) (1996) “Evaluations and InvestigationsProgramme”, Http://www.dest.gov.au/sectors/higher education/policy issues reviews/reviews/evaluations and investigations programme.htmDetterman D K (1993) “The case for the procesution: transfer as an epiphenomenon”, inDetterman and Sternberg (1993), pp 1–24Detterman D K and Sternberg R J (eds) (1993) Transfer on Trial: intelligence, cognitionand instruction, Norwood (NJ): AblexDewey J (1910) How We Think, London: HeathDewey J (1916) Democracy and Education, New York: MacmillanDewey J (1938) Experience and Education, New York: Collier BooksDewey J (1960) The Quest for Certainty, New York: Capricorn427

Diaz-Herrera J L and Hilburn T B (eds) (2003) Computing Curricula – Software EngineeringVolume: Public Draft 1, Los Alamitos (CA): Joint Task Force on Computing Curricula,ACM and IEEE Computer Society, http://sites.computer.org/ccse/artifacts/Dick B, Passfield R and Wildman P (2000) “A beginner’s guideto action research”, Action Research and Evaluation OnLine,http://www.csu.edu.au/schools/gcm/ar/arp/guide.htmlDoke E R and Williams S R (1999) “Knowledge and skill requirements for information systemsprofessionals: an exploratory study”, Journal of Information Systems Education, 10(1), pp10–18Donald J G (2002) Learning to Think, San Francisco: Jossey-BassDownes E, Clare P and Coe I (1988) Structured Systems Analysis and Design (SSADM),London: Prentice HallDoyle W (2000) “Authenticity”, in “Proceedings of the American Educational ResearchAssociation (AERA) Annual Meeting”,Drew S (2001) “Perceptions of what helps learn and develop in education”, Teaching inHigher Education, 6(3), pp 309–331Dreyfus H (2001) On the Internet, London: RoutledgeDreyfus H L and Dreyfus S E (1986) Mind over Machine, New York: Free PressDuch B J, Groh S E and Allen D E (eds) (2001) The Power of Problem-based Learning,Sterling (VA): Stylus PublicationsDuckworth E (1987) The ‘Having of Wonderful Ideas’ and Other Essays on Teaching andLearning, New York: Teachers College Press, Columbia UniversityDuffy T M and Cunningham D J (1996) “Constructivism: implications for the design anddelivery of instruction”, in Jonassen (1996), pp 170–198Duffy T M and Jonassen D H (1992) Constructivism and the Technology of Instruction,Hillsdale (NJ): ErlbaumDuncan S L S (1996) “Cognitive apprenticeship in classroom instruction: implications forindustrial and technical teacher training”, Journal of Industrial Teacher Education, 33(3),pp 66–86Dunn R and Dunn K (1978) Teaching Students through their Individual Learning Styles: apractical approach, Reston (Va): Reston PublishingDunn R and Dunn K (1993) Teaching Secondary Students through Their Individual LearningStyles, Boston: Allyn & BaconDweck C S (1986) “Motivational processes affecting learning”, American Journal of Psychology,41, pp 1040–1048Dweck C S and Elliott E S (1983) “Achievement motivation”, in Mussen P and HetheringtonE M (eds) “Handbook of Child Psychology”, New York: Wiley, pp 643–691Edmonds E and Candy L (2002) “Creativity, art practice and knowledge”, Communicationsof the ACM, 45(10), pp 91–95Edwards B (1999) “Inside the whale : deep insider research”, in “Proceedings ofAARE Conference”, Association for Active Educational Researchers, pp 432–435,http://www.aare.edu.au/99pap/edw99006.htm428

Edwards N S (2002) Design in undergraduate engineering courses and case studies, CommissionedReport on the Application of Design-led Learning (DLL) in Undergraduate EngineeringCourses, PROGRESS Project, University of HullEdwards N S (2004) “Evaluations of introducing project-based design activities in the firstand second years of engineering courses”, European Journal of Engineering Education,29(4), pp 491–503Eisenhart K (1989) “Building theories from case study researh”, Academy of ManagementReview, 14(4), pp 532–550Elliott J (1982) Action-research: a framework for self-evaluation in schools, Technical ReportWorking paper No. 1, Teacher-pupil interaction and the quality of learning, SchoolsCouncil, LondonElmholdt C (2001) What’s productive in a situated perspective on learning?, Working Paper01-16, University of Technology SydneyElmholdt C (2002) “Innovative learning is not enough: landscapes of learning at a shipyardand an IT-company”, in “30th Nordic Educational Research Association Conference”,http://www.psy.an.dk/phd/claus/Publikationer.htmElton L (2000) “Matching teaching methods to learning processes: dangers of doing thewrong thing righter”, in “2nd Annual Conference of the Learning in Law Initiative”,Emam K E and Madhavji N (1995) “A field study of Requirements Engineering practices inInformation Systems development”, in IEEE (b), pp 68–80, in IEEE (1995)Engel G and Roberts E (eds) (2001) Computing Curricula 2001: Computer Science – finalreport, Los Alamitos (CA): Joint Task Force on Computing Curricula, ACM and IEEEComputer Society, http://acm.org/education/curric vols/cc2001.pdfEnglish F W (1978) Quality control in curriculum development, Arlington (VA): AmericanAssociation of School AdministratorsEntwistle N J (2003) “University teaching-learning environments and their influences onstudent learning: an introduction to the ETL project”, in Mason L, Andreuzza S, ArfeB and Del Favero L (eds) “Proceedings of the 10th European Conference for Researchon Learning and Instruction”, Padova: Department of Developmental and SocialisationPsychology, University of PadovaEntwistle N J and Ramsden P (1983) Understanding Student Learning, London: CroomHelmEntwistle N J and Tait H (1990) “Approaches to learning, evaluations of teaching, andpreferences for contrasting academic environments”, Higher Education, 19, pp 169–194Entwistle N J and Tait H (1995) “Approaches to studying and perceptions of the learningenvironment across disciplines”, New directions for teaching and learning, 64, pp 93–103Entwistle N J, Thompson J B and Wilson J D (1974) “Motivation and study methods”,Higher Education, 3, pp 379–396ERDC (1981) Joint Statement of the Australian National Invitational Conference on ActionResearch, Melbourne: Deakin UniversityErlandson D A, Harris E L, Skipper B L and Allen S D (1993) Doing Naturalistic Inquiry: aguide to methods, London: Sage429

Ezzy D (2002) Qualitative Analysis: practice and innovation, Crows Nest (NSW): Allen &UnwinFairley R (1985) Software Engineering Concepts, New York: McGraw-HillFelder R M (1996) “Matters of Style”, Prism, 6(4), pp 18–23, http://www2.ncsu.edu/unity/lockers/users/f/felder/public/Papers/LS-Prism.htmFelder R M and Brent R (2005) “Understanding student differences”, Journal of EngineeringEducation, 94(1), pp 57–72Felder R M and Silverman L K (1988) “Learning and teaching styles in engineeringeducation”, Journal of Engineering Education, 78(7), pp 674–681,http://www.ncsu.edu/felder-public/Papers/LS-1988.pdfFerris T L J (2003) “A technician is half an engineer - overseas students and advanced standingin engineering degrees”, in “Engineering Education for a sustainable future: proceedings ofthe 14th Annual AAEE Conference, Melbourne”, Australasian Association for EngineeringEducation, pp 37–44Fitzgerald B (2004) “CALIBRE Roadmap”, http://www.calibre.ie/roadmap.phpFlavell J (1979) “Metacognition and cognitive monitoring: a new area of cognitivedevelopmentalinquiry”, American Psychologist, 34(10), pp 906–911Floyd C, Zullighoven H, Budde R and Keil-Slawik R (eds) (1992) Software Development andReality Construction, Berlin: Springer-VerlagFord J and Wood L (1996) “An overview of ethnography and system design”, in Wixon Dand Ramey J (eds) “Field Methods Casebook for Software Design”, London: BlackwellScientific, pp 269–299, in Wixon and Ramey (1996)Fosnot C (ed) (1989) Enquiring Teachers Enquirying Learners: a constructivist approach toteaching, New York: Teachers College PressFosnot C (1996a) “Constructivism: a psychological theory of learning”, in Fosnot (1996b),pp 8–33, in Fosnot (1996)Fosnot C (ed) (1996b) Constructivism: theory perspectives and practice, New York: TeachersCollege PressFowler L, Allen M, Armarego J and Mackenzie J (2000) “Learning styles and CASEtools in Software Engineering”, in “Flexible Futures in Tertiary Teaching: Proceedingsof the 9th Annual Teaching Learning Forum”, Perth: Edith Cowan University,www.ecu.edu.au/conferences/herdsa/main/papers/ref/pdf/Fowler.pdfFowler L, Armarego J and M A (2001) “Learning Theory and its Application to FemaleLearner Support in Engineering”, in Megzer A (ed) “Journeys in Leadership: reflections,redirections. 10th International Women in Leadership Conference”, Fremantle,http://isp.ecu.edu.au/wil/conf 01/abstracts/Fowler.docFowler L, McGill D, Armarego J and M A (2002) “Quantitative learning conversations:constructivism and its application to learning in an engineering environment”,in Goody A, Herrington J and Northcote M (eds) “Quality Conversations:Proceedings of HERDSA Annual International Conference”, Canberra:Higher Education Research and Development Society of Australasia Inc., p xxxxx,http://www.ecu.edu.au/conferences/herdsa/main/papers/ref/pdf/Fowler.pdf,(also published in Research and Development in Higher Education Vol 25)430

Frederiksen N (1984) “Implications of cognitive theory for instruction in problem solving”,Review of Educational Research, 54(3), pp 363–407Freedman A and Medway P (eds) (1994) Genre and the New RhetoricFreeman L A and Urbaczewski A (2001) “Using concept maps to assess students’ understandingof Information Systems”, Journal of Information Systems Education, 12(1), pp3–8Friedman K (2001) “Creating design knowledge: from research into practice”, Loughborough:Loughborough University: Department of Design and Technology, pp 39–129Fuson K C (1992) “Reaseach into whole number addition and subtraction”, New York:Macmillan, pp 243–275, in Grouws (1992)Gagné R M, Briggs L and Wager W (1992) Principles of Instructional Design, Fort Worth(TX): Harcourt Brace, 4 edGagné R M and Glaser R (1987) “Foundations in learning research”, in Gagné R M (ed)“Instructional Technology Foundations”, Hillsdale (NJ): Erlbaum, pp 49–83Galliers R D (1985) “An approach to information needs analysis”, in Galliers R D (ed) “InformationAnalysis: selected readings”, Sydney: Addison-Wesley, pp 291–304, in Galliers(1987)Galliers R D (1991) “Choosing information systems research approaches”, in Galliers R D(ed) “Information Systems Research: issues, methods and practical guidelines”, London:Blackwell Scientific Publications, pp 144–162, in Galliers (1992)Gardner H (1983) Frames of Mind: theory of multiple intelligences, New York: Basic BooksGardner H (1999) Intelligence Reframed: multiple intelligences for the 21st century, NewYork: Basic BooksGarlan D, Gluch D P and Tomayko J E (1997) “Agents of Change: educating future leadersin Software Engineering”, IEEE Computer, 30(11), pp 59–65Garrison J and Rud A (eds) (1995) The educational conversation: closing the gap, Albany(NY): State UniversityPress of New YorkGhezzi C, Jazayeri M and Mandrioli D (1991) Fundamentals of Software Engineering, UpperSaddle River (NJ): Prentice HallGibbs G (1992) Assessing more students, Oxford: Oxford Brookes UniversityGigch J P v (2000) “Metamodelling and problem solving”, Journal of Applied Systems Studies,1(2), pp 327–336Glaser B G (1992) Basics of grounded theory analysis: emergence vs forcing, Mill Valley(CA): Sociology PressGlaser B G and Strauss A L (1967) The discovery of grounded theory: strategies for qualitativeresearch, Chicago: AldineGlaserfeld E v (1996) “Introduction: aspects of constructivism”, in Fosnot (1996b), pp 3–7,in Fosnot (1996)Glass R L (1992) “A comparative analysis of the topic areas of Computer Science, SoftwareEngineering, and Information Systems”, Journal of Systems and Software, 25, p xxxx431

Glass R L (1995) “A theory about software’s practice (Editor’s Corner)”, Journal of Systemsand Software, 28, pp 187–188Glew R H (2003) “The problem with Problem-based medical education: promises not kept”,Biochemistry and Molecular Biology Education, 31(1), pp 52–58Gobet F (1996) “Discrimination nets, production systems and semantic networks: elements ofa unified framework”, in “Proceedings of the 2nd International Conference on the LearningSciences”, Evanston (Il): Northwestern University, pp 398–403, in Proceedings (1996)Gobet F and Simon H A (1996) “Templates in chess memory: a mechanism for recallingseveral boards”, Cognitive Psychology, 31, pp 1–40Gobet F and Wood D (1999) “Expertise, models of learning and computer-based tutoring”,Computers & Education, 33, pp 189–207Goldstein I (1980) “Developing a computational representation for problem-solving skills”,in Tuma and Reif (1980), pp 53–79, in Tuma and Reif (1980)Gollin S (1998) “Literacy in a computing department: the invisible in search of the illdefined”,in Candlin and Plum (1998), pp 293–333Gordon J (1996) “Tracks for learning: metacognition and learning technologies”,Australian Journal of Educational Technology, 12(1), pp 46–55,http://cleo.murdoch.edu.au/gen/aset/ajet/ajet12/wi96p46.htmlGorgone J T, Davis G B, Valacich J S, Topi H, Feinstein D and LongneckerH E (2002a) IS 2002: model curriculum and guidelines for undergraduate degreeprograms in Information Systems, Atlanta: Association for Information Systems,http://www.acm.org/education/is2002.pdfGorgone J T, Topi H, Feinstein D, Valacich J S, Longnecker H E and Davis G B (2002b)“IS 2002: final report of the undergraduate Information Systems model curriculum”, in“Twenty-Third International Conference on Information Systems”, pp 923 – 931Gott S P, Hall E P, Pokorny R A, Dibble E and Glaser R (1993) “A naturalistic study oftransfer: adaptive expertise in technical domains”, in Detterman and Sternberg (1993),pp 258–288de Graaff E and Kolmos A (2003) “Characteristics of problem-based learning”, InternationalJournal of Engineering Education, 19(5)Green G I (1989) “Perceived importance of systems analysts’ job skills, roles and non-salaryincentives”, MIS Quarterly, 13(2), pp 115–133Greene M (1996) “A constructivist perspective on teaching and learning in the arts”, inFosnot (1996b), pp 120–141, in Fosnot (1996)Greene S L (2002) “Characteristics of applications that support creativity”, Communicationsof the ACM, 45(10), pp 100–104Greeno J G (1997) “On Claims That Answer the Wrong Question”, Educational Researcher,26(1), pp 5–17Greeno J G e a (1998) “The situativity of knowing, learning and research”, American Psychologist,53(1), pp 5–26Greenwood J (1993) “Reflective practice: a critique of the work of Argyris and Schön”,Journal of Advanced Nursing, 21, pp 1044–1050432

Gregg D, Kulkarni U and Vinze A (2001) “Understanding the philosophical underpinningsof Software Engineering research in information systems”, Information Systems Frontiers,3(2), pp 169–183Gros B, Elen J, Kerres M, Merriënboer J and Spector M (1997) “Instructional design andthe authoring of multimedia and hypermedia systems: does a marriage make sense?”,Educational Technology, 37(1), pp 48–56Grow G O (1991/1996) “Teaching learners to be self-directed”, Adult Education Quarterly,41(3), pp 125–149, http://www.longleaf.net/ggrowGrundy A B v (1987) Creative Problem Solving, New York: QuorumGrundy S J (1982) “Three modes of action research”, Curriculum Perspectives, 2(3), pp23–34Guba E and Lincoln Y (1994) “Competing paradigms in qualitative research”, in DenzinN and Lincoln Y (eds) “The Handbook of Qualitative Research”, Thousand Oaks (CA):Sage, pp 105–117Guilford J P (1967) The Nature of Human Intelligence, New York: McGraw-HillGuindon R (1989) “The process of knowledge discovery in system design”, in Salvendy Gand Smith M J (eds) “Designing and Using Human-Computer Interfaces and KnowledgeBased Systems”, Amsterdam: Elsevier, pp 727–734Guindon R (1990) “Designing the design process:Human-Computer Interaction, 5, pp 305–344exploiting opportunistic thoughts”,Gupta J D and Wachter R M (1998) “A capstone course in the information systems curriculum”,International Journal of Information Management, 18(6), pp 427–441Habermas J (1972) Theory and Practice, London: Heinemman, translated by J ViertalHadamard J (1945) An essay on the psychology of invention in the mathematical field, Princeton(NJ): Princeton University PressHannafin M J (1997) “Better learning with multimedia? Concepts and results from psychologyand education”, in “Multimedia und internet–neue perspektiven fr die bildung,Gesellschaft fr Padagogik und Information (GPI)”, Munich, keynote addressHannafin M J (1997a) “The case for grounded learning systems design:what the literature suggests about effective teaching learning and technology”,in “Proceedings of ASCILITE ’97”, Curtin University of Technology,http://www.curtin.edu.au/conference/ascilite97/papers/Hannafink/Hannafink.htmlHannafin M J, Hannafin K M, Hooper S R, Reiber L and Kini A S (1996) “Research on andresearch with emerging technologies”, in Jonassen (1996), pp 378–402Hannafin M J, Hannafin K M, Land S M and Oliver K (1997) “Grounded practice andthe design of constructivist learning environments”, Educational Technology Research &Development, 45(3), pp 101–117Harri-Augstein S and Thomas L (1991) Learning conversations: the self-organized learningway to personal and organizational growth, London: RoutledgeHarris R, Simons M and Bone J (2000) More than meets the eye? rethinking the role ofworkplace trainer, NREC433

Hatten R A (1997) Guidelines for Effectiveness in Continuing Professional Education, Master’sthesis, University of Sydney, SydneyHeidegger M (1968) What is called thinking?, New York: Harper and Row, translated by FD Wieck and J G GrayHenry R M, Dickson G W and LaSalle J (1974) Human resources for MIS: a report onresearch, Technical Report WP-74-01, MIS Research CenterHerrington J, Reeves T and Oliver R (2003) “Patterns of engagement in authentic onlinelearning environments”, Australian Journal of Educational Technology, 19(1), pp 59–71Heylighen F (1993) “Epistemology: introduction”, Brussels: Free University of Brussels,http://pespmc1.vub.ac.be/epistemi.htmlHill M, Bailey J D and Reed P A S (1998) “Hypermedia systems for improving knowledge,understanding and skills in engineering degree courses”, Computers & Education, 31(1),pp 69–88Hirumi A (2002) “Student-centred, technology-rich learning environments (SCenTRLE)”,Journal of Technology and Teacher Education, 10(4), pp 497–537Hitchcock M A (2000) “Teaching faculty to conduct problem-based learning”, Teaching andLearning in Medicine, 12(1), pp 52–57Hocking L (1996) “Systems analysis and early design as the negotiation of meaning andinterest”, in “Proceedings of the 7th Australasian Conference on Information Systems”,Hobart: University of Tasmania, pp 319–330Hoffer J A, George J F and Valacich J S (2002) Modern Systems Analysis and Design,Englewood Cliffs (NJ): Prentice Hall, 3rd edHofmann H F and Lehner F (2001) “Requirements Engineering as a success factor in softwareprojects”, IEEE Software, 18(4), pp 58–66Hollingsworth S and Sockett H (1994) “Positioning teacher Research in educational reform:an introduction”, in Hollingsworth S and Sockett H (eds) “Teacher Research and EducationalReform”, Chicago: University of Chicago Press, pp 1–20Holt J and Solomon F (1996) “Engineering Education - the way ahead”,Australasian Journal of Engineering Education, 7(1), pp 1–22; 83–98,http://elecpress.monash.edu.au/ajee/vol7no1/holt.htmHoover J J and Rabideau D K (1995) “Semantic webs and study skills”, Intervention inSchool and Clinic, 30(5), pp 292–296Horvath I, Wiersma M, Duhovnik J and Stroud I (2004) “Navigated active learning in aninternational academic virtual enterprise”, European Journal of Engineering Education,29(4), pp 505–519Houdek F and Pohl K (2000) “Analyzing Requirements Engineering processes: a case study”,in “DEXA’00: Proceedings of the 11th International Workshop on Database and ExpertSystems Applications”, Los Alamitos (CA): IEEE Computer Society Press, pp 983–987Hughes J, O’Brien J, Rodden T, Rouncefield M and Sommerville I (1995) “Presenting ethnographyin the requirements process”, in IEEE (b), pp 27–34, in IEEE (1995)Huitt W (1998) “Critical thinking”, Educational Psychology Interactive,http://chiron.valdosta.edu/whuitt/col/cogsys/critthnk.html434

IBM Corporation (2006) “IBM Rational Rose”, ftp://ftp.software.ibm.com/software/rational/web/datasheets/rose ds.pdfIEAust (1996) Changing the culture: engineering education into the future, Technical Report,Institution of Engineers, AustraliaIEEE (1993a) International Symposium on Requirements Engineering, San Diego: IEEEComputer Society PressIEEE (1995b) Second International Symposium on Requirements Engineering, Los Alamitos(CA): IEEE Computer SocietyIivari J (1991) “A paradigmatic analysis of contemporary schools of IS development”, EuropeanJournal of Information Systems, 1(4), pp 249–272Institute of Quality Assurance (2002) Body of Knowledge, London:Professional Policy Board, Institute of Quality Assurance,http://www.iqa.org/htdocs/trainingevents/g0-1.htmlIntrona L D (1996) “Notes on a teleological information systems development”, InformationTechnology and People, 9, pp 20–39ISC 2 (2001) Common Body of Knowledge, Dunedin (FL): International Information SystemsSecurity Certification Consortium Inc, http://www.isc2.org/Jackson M (1995) Software Requirements and Specifications: a lexicon of practice, principlesand prejudices, Wokingham (UK): ACM Press/Addison-WesleyJackson N and Borden V (n.d.) Learning about teaching - using action and enquiry researchto improve teaching practices and student outcomes, Technical Report, Indianapolis (IN)James L (1994) “Practical experiences in automatic requirements elicitation: the real issues”,in “Workshop on Requirements Elicitation for Software-based Systems”,Jeffries R, Turner A A, Polson P and Atwood M E (1981) “The processes involved in designingsoftware”, in Anderson J R (ed) “Cognitive Skills and their Acquisition”, Hillsdale (NJ):Erlbaum, pp 225–283John M, Mauer F and Tessem B (2005) “Human and social factors of Software Engineering- workshop summary”, in “Proceedings of the International Conference on SoftwareEngineering (ICSE)”, St Louis (MI): IEEE Computer Society PressJohnson J (1994) Chaos: charting the seas of information technology, Standish Group,http://www.standishgroup.com/sample research/chaos 1994 1.phpJohnson R and Onwuegbizue A (2004) “Mixed method research: a research paradigm whosetime has come”, Educational Researcher, 33(7), pp 14–26Johnson-Laird P (1983) Mental Models, Cambridge (MA): Harvard University PressJonassen D and Land S (eds) (2000) Theoretical Foundations of Learning Environments,Hillsdale (NJ): ErlbaumJonassen D H (1992) “Semantic networking as cognitive tools”, in Kommers et al (1992), pp19–21Jonassen D H (ed) (1996) Handbook of Research for Educational Communications and Technology,New York: Simon & Schuster Macmillan435

Jonassen D H (2000) “Towards a design theory of problem solving”, Educational TechnologyResearch & Development, 48(4), pp 63–85Jonassen D H (2002) “Learning to solve problems online”, in Vrasidas C and Glass V (eds)“Distance Education and Distance Learning”, Greenwich (CT): Information Age Publishing,pp 75–98Jonassen D H, Davidson M, Collins M, Campbell J and Haag B (1995) “Constructivism andcomputer-mediated communication in distance education”, American Journal of DistanceEducation, 9(2), pp 2–26Jonassen D H and Grabowski B L (1993) Handbook of individual differences, learning andinstruction, New York: Allyn & BaconJones C H, Slate J R, Marini I and DeWater B K (1993) “Academic skills and attitudestoward intelligence”, Journal of College Student Development, 34, pp 422–424Jong T d (2006) “Technological advances in inquiry learning”, Science, 312(5773), pp 532–533Jordan B (1987) Modes of Teaching and Learning: questions raised by the training of traditionalbirth attendants, Palo Alto (CA): Institute for Research on LearningKamsties E, Hormann K and Schlich M (1998) “Requirements Engineering in small andmedium enterprises”, Requirements Engineering, 3(2), pp 84–90Kearsley G (2000) Explorations in Learning & Instruction: the theoryin practice database, Washington (DC): George Washington University,http://www.gwu.edu/~tip/engineer.htmlKeegan D (1998) “The virtual countinghouse: finance transformed by electronics”, in LeebartD (ed) “The Future of Electronic Marketplace”, Cambridge (MA): MIT Press, pp 205–240Keirsey D and Bates M (eds) (1984) Please Understand Me, Prometheus Nemesis BookCompany, 3 ed, http://www.keirsey.comKember D (2001) “Beliefs about knowledge and the process of teaching and learning as afactor in adjusting to study in higher education”, Studies in Higher Education, 26(2), pp205–221Kember D (2004) “Interpreting student workload and the factors which shape students perceptionsof their workload”, Studies in Higher Education, 29(2), pp 165–184Kember D (2006) “Characterising a teaching and learning environment conducive to makingdemands on students while not making their workload excessive”, Studies in HigherEducation, 31(2), pp 185–198Kember D and Kelly M (eds) (1993) Improving Teaching through Action Research, Campbelltown(NSW): Higher Education Research and Development Society of Australasia IncKemmis S (1993) “Action research and social movement: a challenge for polical research”,Education Policy Analysis Archives, 1(1), http://epaa.asu.edu/epaa/v1n1.htmlKemmis S and McTaggert R (eds) (1988) The Action Research Reader, Geelong: DeakinUniversity Press, 3rd edKing A (1994) “Inquiry as a tool in critical thinking”, in Halpern D F (ed) “Changing CollegeClassroom”, San Fransisco: Jossey-Bass, pp 13–38436

King N (1998) “Template analysis”, in Symon G and Cassell C (eds) “Qualitative Methodsand Analysis in Organizational Research”, London: Sage, pp 118–134Kitchenham B, Dybå T and Jørgensen M (2004) “Evidence-based Software Engineering”, in“Proceedings of the International Conference on Software Engineering (ICSE)”, Edinburgh:IEEE Computer Society Press, pp 273–281Klemola T and Rilling J (2002) “Modeling comprehension processes in software development”,in “Proceedings of the first IEEE Conference on Cognitive Informatics (ICCI’02)”,Calgary (Canada): University of CalgaryKock N F, McQueen R J and Scott J L (2000) “Can Action Research be made more rigorous ina positivist sense? the contribution of an iterative Approach”, Action Research E-Reports,9, http://www.cchs.usyd.edu.au/arow/arer/009.htmKolb D A (1984) Experiential Learning: experience as the source of learning and development,Englewood Cliffs (NJ): Prentice HallKommers P A M, Jonassen D H and Mayes J T (eds) (1992) Cognitive Tools for Learning,Heidelberg: Springer-VerlagKoschmann T D, Myers A C, Barrows H S and Feltovich J (1994) “Using technology to assistin realising effective learning and instruction: a principled approach to the use of computersin collaborative learning”, The Journal of the Learning Sciences, 3(3), pp 227–264Kotonya G and Sommerville I (1997) Requirements Engineering: processes and techniques,Chichester: WileyKozar K A (1989) “Adopting systems development methods: an exploratory study”, Journalof Management Information Systems, 5(4), pp 73–86Kreber C (1999) “A course-based approach to the development of teaching scholarship: acase study”, Teaching in Higher Education, 4, pp 309–325Krefting L (1991) “Rigor in qualitative research:tThe assessment of trustworthiness”, TheAmerican Journal of Occupational Therapy, 45(3), pp 214–222Kuhn S (2001) “Learning from the architecture studio: implications for project-based pedagogy”,International Journal of Engineering Education, 17(4/5), pp 349–52Lacey A and Luff D (2001) “Qualitative Data Analysis”, http://www.trentfocus.org.uk/Resources/introduction qualitative research.htmLarkin J H (1989) “What kind of knowledge transfers?”, in Resnick (1989), pp 283–306Lather P (1986) “Research as praxis”, Harvard Educational Review, 56(3), pp 157–277Lather P (1991) Getting Smart: feminist research and pedagogy within the postmodern, NewYork: RoutledgeLather P (1993) “Fertile obsession: validity after poststructuralism”, Sociological Quarterly,34(4), pp 673–693Lather P (2004) “Scientific research in education: a critical perspective”, British EducationalResearch Journal, 30(6), pp 759–772Laurillard D (1993) Rethinking University Teaching: a framework for the effective use ofeducational technology, London: Routledge437

Lave J, Smith S and Butler M (1988) “Problem solving as an everyday practice”, in “LearningMathematical Problem Solving”, Palo Alto (CA): Institute for Research on Learning,report Number IRL88-0006Lave J and Wenger E (1991) Situated Learning: legitimate peripheral participation, Cambridge(MA): Cambridge University PressLawrance W, Kalam A, Figgis J and Rhyn D v (2005) Review of the School of Engineering,Technical Report, Murdoch UniversityLawrence G (1993) People Types and Tiger Stripes: A Practical Guide to Learning Styles,Gainesville (FL): Center for Applications of Psychological Type, 3rd edLawson B (1997) How Designers Think: the design process demystified, New York: ArchitecturalPress, 3rd edLearning and Teaching Support Network (LTSN) (2002a) “Constructivealignment and why it is important to the learner”,http://www.ltsneng.ac.uk/er/theory/constructive alignment.aspLearning and Teaching Support Network (LTSN) (2003b) “Glossaryof terms in learning and teaching in higher education”,http://www.ltsn.ac.uk/genericcentre/index.asp?id=17632LeBlanc R and Sobel A E K (eds) (2004) Software Engineering 2004: curriculum guidelinesfor undergraduate degree programs in Software Engineering, Los Alamitos (CA): IEEEComputer Society Press, http://sites.computer.org/ccse/SE2004Volume.pdfLee D M S (1986) “Academic achievement, task characteristics, and first job performance ofyoung engineers”, IEEE Transactions on Engineering Management, EM-33(3), pp 127–133Lee D M S (1989) “Educating engineers in the era of computers-integrated operations”,Engineering Education, 79(1), pp 31–35Lee D M S (1992) “Job challenge, work effort, and job performance of young engineers: acausal analysis”, IEEE Transactions on Engineering Management, 39(3), pp 214–226Lee D M S (1994) “Social ties, task-related communication and first job performance of youngengineers”, Journal of Engineering and Technology Management, 11, pp 203–228Lee D M S (1999a) “Information seeking and knowledge acquisition behaviors of younginformation systems workers: preliminary analysis”, in “1999 Americas Conference onInformation Systems”, Milwaukee (WI): Association for Information Systems, pp 856–858Lee D M S (1999b) “Knowledge/skill requirements and professional development of IS/ITworkers: a summary of empirical findings from two studies”, in “Panel on Workforce Needsin Information Technology, Computer Science and Telecommunications Board, NationalAcademy of Sciences”, Milwaukee (WI)Lee D M S (2004) “Organizational entry and transition from academic study: examininga critical step in the professional development of young IS workers”, in Igbaria M andShayo C (eds) “Strategies for Managing IS/IT Personnel”, Hershey (PA): Idea Group, pp113–141Lee D M S, Trauth E M and Farwell D (1995) “Critical skills and knowledge requirementsof IS professionals: a joint academic/industry investigation”, MIS Quarterly, 19(3), pp313–340438

Lee J and Truex D P (2000) “Exploring the impact of formal training in ISD methods onthe cognitive structure of novice information systems developers”, Information SystemsJournal, 10, pp 347–367Lee M and Kim D (2005) “The effects of the Collaborative Representation Supporting Tool onproblem-solving processes and outcomes in web-based collaborative Problem-Based Learning(PBL) environments”, Journal of Interactive Learning Research, 16(3), pp 273–293Leite J C S P (2000) “Is there a gap between RE research and RE practice?”, in “Fourth InternationalConference on Requirement Engineering”, Los Alamitos (CA): IEEE ComputerSociety, pp 73–74, in IEEE (2000)Leitheiser (1992) “MIS skills for the 1990s: a survey of MIS managers’ perceptions”, Journalof Management Information Systems, 9(1), pp 69–91Lepper M R, Woolverton M, Mumme D L and Gurtner J L (1993) “Motivational Techniquesof Expert Human Tutors: lessons for the design of computer-based tutors”, in Lajoie S Pand Derry S J (eds) “Computers as cognitive tools”, Hillsdale (NJ): Erlbaum, pp 75–105Lethbridge T C (1999) The relevance of education to software practitioners: data from the1998 survey, Computer Science Technical Report TR-99-05, School of Information Technologyand Engineering, University of Ottowa, CanadaLethbridge T C (2000) “What knowledge is important to a software professional?”, Computer,33(5), pp 44–50Levy A (1986) “Second order planned change: definition and conceptualisation”, OrganisationalDynamics, 15, pp 5–20Lewin K (1946) “Action research and minority problems”, Journal of Social Issues, 2, pp34–46Lewin K (1952) “Group decision and social change”, in Swanson G E, Newcomb T M andHartley F E (eds) “Readings in Social Psychology”, New York: Holt, pp 459–473Lidtke D, Stokes G E, Haines J and Mulder M C (eds) (1999) An Information Systems-centricCurriculum ‘99: program guideleines for educating the next generation of information systemsspecialists, in collaboration with industry, http://www.iscc.unomaha.eduLincoln Y S (1985) “The substance of the emergent paradigm: implications for researchers”,in “Organisational Theory and Inquiry: the paradigm revolution”, pp 137–157, cited inGrover and Greer (1991)Lindblom-Ylänne S (2004) “Raising students awareness of their approaches to study”, Innovationsin Education and Teaching International, 41(4), pp 405–421Litecky C R, Arnett K P and Prabhakar B (2004) “The paradox of soft skills versus technicalskills in IS hiring”, Journal of Computer Information Systems, 45(1), pp 69–76Loucopoulos P and Karakostas V (1995) System Requirements Engineering,McGraw-HillLondon:Lowry G and Turner R (2005) “Information Systems Education for the 21st Century: aligningcurriculum content and delivery with the professional workplace”, in Carbonara D (ed)“Technology Literacy Applications in Learning Environments”, Hershey (PA): IRM Press,pp 171–202Lowry G and Turner R (eds) (in press) Information Systems and Technology Education: fromthe university to the workplace, Hershey (PA): Idea Group439

Lubars M, Potts C and Richer C (1993) “A review of the state of the practice in requirementsmodeling”, in IEEE (a), pp 2–14, in IEEE (1993)Lucas U and Meyer J H F (2004) “Supporting student awareness: understanding student preconceptionsof their subject matter within introductory courses”, Innovations in Educationand Teaching International, 41(4), pp 459–471Lumsdaine E, Shelnutt J and Lumsdaine M (1999) “Integrating creative problem solving andengineering design”, http://www.succeed.ufl.edu/papers/99/00294.pdfLumsdaine M and Lumsdaine E (1995) “Thinking preferences of engineering students: implicationsfor curriculum restructuring”, Journal of Engineering Education, 84(2), pp 193–204Macauley L and Mylopoulos J (1995a) “Requirements Engineering: an educational dilemma”,Automated Software Engineering, 2(4), pp 343–351Macauley L and Mylopoulos J (1995b) “Requirements Engineering: an educationaldilemma”, Forum for Academic Software Engineering, 5(22),http://www.cs.ttu.edu/fase/v5n22.txtMaccoby M, Jurgen R K, Perry T S and Watson G F (1991) “Innovative genius: the innovativemind at work”, IEEE Spectrum, 28(12), pp 23–35Madison G B (1988) The Heurmeneutics of Postmodernity: figures and themes, Bloomfield(IN): Indiana University PressMaiden N and Gizikis A (2001) “Where do requirements come from?”, IEEE Software, 18(5),pp 10–12Maiden N A M and Sutcliffe A G (1992) “Exploiting reusable specifications through analogy”,Communications of the ACM, 34(5), pp 55–64Malhotra A, Thomas J C, Carroll J M and Miller L A (1980) “Cognitive processes in design”,International Journal of Man-Machine Studies, 12, pp 119–140Mansell G (1991) “Action research in information science development”, Journal of InformationSystems, 1, pp 29–40March J G and Simon H A (1958) Organisations, New York: WileyMarch S and Smith G (1995) “Design and natural science research on information technology”,Decision Support Systems, 15, pp 251 – 266de Marco T and Lister T (1999) Peopleware: product projects and teams, New York: DorsetHouse, 2nd edMarcos E (2005) “Software engineering research versus software development”, ACM SIG-SOFT Software Engineering Notes, 30(4), pp 1–7Marini A and Genereux R (1995) “The challenge of teaching for transfer”, Hillsdale (NJ):Erlbaum, pp 1–19Martin R, Maytham B, Case J and Fraser D (2005) “Engineering graduates’ perceptionof how well they were prepared for work in industry”, European Journal of EngineeringEducation, 30(2), pp 167–180Marton F and Saljo R (1976) “On qualitative differences in learning I - outcomes and processes”,British Journal of Educational Psychology, 46, pp 4–11440

Masters J (1995) “The history of action research”, Action Researcher E-Report, 3,http://www.cchs.usyd.edu.au/arow/arer/003.htmMauch J E and Birch J W (1989) Guide to the Successful Thesis and Dissertation: conceptionto publication: a handbook for students and faculty, New York: Marcel DekkerMayer E (1993) Putting General Education to Work: the key competencies report, TechnicalReport, Committee to advise AEC and MOVEETMayes J T (1992) “Cognitive tools: a suitable case for learning”, in Kommers et al (1992),http://led.gcal.ac.uk/clti/papers/TMPaper5.htmlMaynard G B and Underwood A (1996) Guidelines for Accreditation of Courses in Universitiesat the Professional Level, Sydney: Australian Computer Society Inc.McCalla G I and Greer J E (1991) “Granularity-based reasoning and belief revision in studentmodels”, in McCalla G I and Greer J E (eds) “Student Modelling: the key to individualisedknowledge-based instruction”, Heidelberg: Springer-Verlag, pp 39–62, in McCallaand Greer (1991a)McConnell S and Tripp L (1999) “Professional Software Engineering: fact or fiction?”, IEEESoftware, 16(6), pp 13–17McCracken D (1988) The Long Interview, Qualitative Research Methods, Vol. 13, NewburyPark: SageMcCracken W M (1997) “SE education: what academia can do”, IEEE Software, 14(6), pp27, 29McGill T (ed) (2003) Current Issues in IT Education, Hershey (PA): IRM PressMcKenna C and Bull J (1999) “Designing effective objective test questions:an introductory workshop”, in “Proceedings of the Third Annual ComputerAssisted Assessment Conference”, Loughborough: Loughborough University,www.lboro.ac.uk/service/ltd/flicaa/conf99/pdf/bullworkshop.pdfMcLaughlan R G and Kirkpatrick D (1999) “A decision making simulation using computermediated communication”, Australian Journal of Educational Technology, 15, pp 242–256McLaughlan R G and Kirkpatrick D (2004) “Online roleplay: design for active learning”,European Journal of Engineering Education, 29(4), pp 477–490McNeill P and Chapman S (2005) Research Methods, London: Routledge, 3rd edMerleau-Ponty M (1962) Phenomenology of Perception, London: Routledge, translated by CSmithMerrill M D (1973) “Content and instructional analysis for cognitive transfer tasks”, AVCommunication Review, 21, pp 111–117Meyer J H (1998) “A medley of individual differences”, in Dart B and Boulton-Lewis G M(eds) “Teaching and Learning in Higher Education”, Melbourne: Australian Concil forEducational Research Ltd, pp 42–71Meyer J H F (1991) “Study orchestration: the manifestation, interpretation and consequencesof contextualised approaches to studying”, Higher Education, 22, pp 297–316Meyer J H F (1999) “Variation and concepts of quality in student learning”, Quality in HigherEducation, 5, pp 167–180441

Meyer J H F and Boulton-Lewis G M (1997) “The association between university studentsperceived influences on their learning and their knowledge, experience, and conceptions, oflearning”, in “Proceedings of the 7th European Conference for Research on Learning andInstruction”, AthensMeyer J H F and Shanahan M P (2000) “Making teaching responsive to variation in studentlearning”, in Rust C (ed) “Proceedings of the 7th Improving Student Learning Symposium,Manchester”, Oxford: Oxford Centre for Staff and Learning Development, Oxford BrookesUniversityMeyer J H F and Shanahan M P (2004) “Developing metalearning capacity in students:actionable theory and practical lessons learned in first-year economics”, Innovations inEducation and Teaching International, 41(4), pp 443–458Middendorf J and Pace D (1986) “Decoding the disciplines: a model for helping studentslearn disciplinary ways of thinking”, New Directions for teaching and learning, 98, pp 1–12Miller S M and Miller K L (1999) “Using instructional theory to facilitatecommunications in Web-based courses”, Educational Technology & Society, 2(3),http://ifets.ieee.org/periodicalMills J, Ayre M, Hands D and Carden P (2005) “Learning about LearningStyles: can it improve engineering education?”, MountainRise, 2(1), pp 145–182,http://facctr.wcu.edu/mountainrise/archive/vol2no1/issue.htmlMilton J and Lyons J (2003) “Evaluate to improve learning: reflecting on the role of teachingand learning models”, Higher Education Research and Development, 22(3), pp 297–312Minor O (2004) Theory and Practice in Requirements Engineering: an investigation of curriculaand industry needs, Master’s thesis, University of Koblenz-Landau, Koblenz (Germany)Moller-Wong C and Eide A (1997) “An engineering student retention study”, Journal ofEngineering Education, 86(1), pp 7–15Morris N and Rouse W (1985) “Review and evaluation of empirical research in troubleshooting”,Human Factors, 27(5), pp 503–530Morris P, Masera M and Wilikens M (1998) “Requirements Engineering and industrial uptake”,in “3rd IEEE International Conference on Requirements Engineering”, Los Alamitos(CA): IEEE Computer Society Press, pp 130–137Mulder F and Weert T v (eds) (2000) Informatics Curriculum Framework 2000 for Higher Education:building effective higher education informatics curricula in a situation of change,Paris: International Federation for Information Processing (IFIP)/UNESCOMulder K F (2006) “Engineering curricula in Sustainable Development: an evaluation ofchanges at Delft University of Technology”, European Journal of Engineering Education,31(2), pp 133–144Mumford E (1983) Designing Human Systems for New Technology: the ETHICS method,Manchester: Manchester Business SchoolMumford E, Hirschheim R, Fitzgerald G and Wood-Harper A T (eds) (1985) Research Methodsin Information Systems, Amsterdam: Elsevier ScienceMumford M D, Baughman W A, Supinski E P and Anderson L E (1998) “A construct approachto skill assessment: procedures for assessing complex cognitive skills”, in Hakel M D(ed) “Beyond Multiple-Choice: evaluating alternatives to traditional testing for selection”,Hillsdale (NJ): Erlbaum, pp 75–112442

Murdoch (2004a) “Assessment Policy”, http://www.murdoch.edu.au/admin/policies/assessment.html, last amended Nov 2005Murdoch (2004b) “Student Surveys of Units”,evaluation/survey/unit.htmlhttp://www.tlc.murdoch.edu.au/eddev/Myers I B (ed) (1998) Introduction to Type, Palo Alto (CA): Consulting Psychologists PressInc.Myers M D (1997) “Qualitative Research in Information Systems”, MIS Quarterly, 21(2), pp241–242, www.qual.auckland.ac.nz, last updated 2006Nagasamy G (2002) “Requirements testing”,OldArchive/http://research.it.uts.edu.au/re/re-Nation I S P and Waring R (1997) Vocabulary size, text coverage, and word lists, New York:Cambridge University PressNelson W A (2003) “Problem solving through design”, in Knowlton D S and Sharp D C(eds) “Problem-Based Learning in the Information Age”, San Francisco: Jossey-Bass, 2nded, pp 39–44Newble D J and Clarke R M (1986) “The approaches to learning of students in a traditionaland in an innovative problem-based medical school”, Medical Education, 20, pp 267–273Newell A and Simon H A (1972) Human Problem Solving, Englewood Cliffs (NJ): PrenticeHallNewmann F M, Marks H M and Gamoran A (1996) “Authentic pedagogy and student performance”,American Journal of Education, 104(8), pp 280–312Nguyen L and Swatman P A (2000a) “Complementary Use of ad hoc and post hoc DesignRationale for Creating and Organising Process Knowledge”, in “Proceedings of the HawaiiInternational Conference on System Sciences HICSS-33”, Maui, HawaiiNguyen L and Swatman P A (2000b) “Essential and incidental complexity in requirementsmodels”, in “Fourth International Conference on Requirements Engineering, Proceedings”,Schaumburg (Il): IEEE, pp 130–139Nickerson R S (1999) “Enhancing creativity”, in Sternberg R E (ed) “Handbook of Creativity”,Cambridge (UK): Cambridge University Press, pp 392–430Nikula U, Sajaniemi J and Kälviäinen H (2000) A state-of-the-practice survey on RequirementsEngineering in small- and medium-sized enterprises, Technical ReportResearch Report 1, Telecom Business Research Center, Lappeenranta (Finland),http://www.lut.fi/TBRCNisbet J (2005) “What is educational research? changing perspectives through the 20thcentury”, Research Papers in Education, 20(1), pp 25–44Noll C L and Wilikens M (2002) “Critical skills of IS professionals: a model for curriculumdevelopment”, Journal of Information Technology Education, 1(3), pp 143–154Norman D A (1983) “Some observations on mental models”, in Gentner D and Stevens A L(eds) “Mental Models”, Hillsdale (NJ): Erlbaum, pp 7–14Norman D A (1993) Things That Make Us Smart, Reading (MA): Addison-WesleyNorman D A and Rumelhart D E (1975) Explorations in Cognition, San Fransisco: Freeman443

Norton L S, Owens T and Clark L (2004) “Analysing metalearning in first-year undergraduatesthrough their reflective discussions and writing”, Innovations in Education and TeachingInternational, 41(4), pp 423–441Novak J (1998) Learning, Creating and Using Knowledge; concept maps as tools to understandand facilitate the process in schools and corporations, Hillsdale (NJ): ErlbaumNuseibeh B and Easterbrook S (2000) “Requirements Engineering: a roadmap”, in “Proceedingsof the International Conference on Software Engineering (ICSE)”, Limerick: ACMPress, pp 35–46Ochs E and Capp L (2001) Living narrative: creating lives in everyday storytelling, Cambridge(MA): Harvard University PressOliver R and McLoughlin C (1999) “Using web and problem-based learning environmentsto support the development of key skills”, in “Responding to Diversity: Proceedingsof ASCILITE ’99”, Brisbane: Queensland University of Technology, pp 249–256,http://www.ascitite.org.au/conferences/brisbane99/papers/olivermcloughlin.pdfOrlikowski W J and Baroudi J J (1991) “Studying information technology in organizations:research approaches and assumptions”, Information Systems Research, 2, pp 1–28Orona C (1990) “Temporality and identity loss due to Alzheimer’s disease”, Social Scienceand Medicine, 30(11), pp 1247–1256Owen C (1997) “Design Research: building the knowledge base”, Journal of the JapaneseSociety for the Science of Design, 5(2), pp 36–45Papert S (1980) Mindstorms: children, computers, and powerful ideas, New York: BasicBooksParker D, Bailey J and Mitchell R B (1999) “Developing non-technical competencies of ISprofessionals through technological applications”, in “Proceedings of the Southwest AdministrativeSystems track: Southwestern Federation of Administrative Disciplines Conference”,pp 1–9Parnas D L and Clements P C (1986) “A rational design process: how and why to fake it”,IEEE Transactions on Software Engineering, 12(2), pp 251–257Pask G (1976) Conversation Theory: applications in education and epistemology, Amsterdam:ElsevierPatel A and Kinshuk (1996) “Knowledge characteristics: reconsidering the design of intelligenttutoring systems”, in Behrooz A (ed) “Knowledge Transfer”, London: Pace, pp190–197Patel A and Kinshuk (1997) “Granular interface design: decomposing learning tasks andenhancing tutoring interaction”, Amsterdam: Elsevier Science, pp 161–164Patel A, Kinshuk and Russell D (1999) “Designing life long learning systems for numericdomains”, Educational Technology, 39(5), pp 8–11Patel A, Kinshuk and Russell D (2000) “Intelligent tutoring tools for cognitive skill acquisitionin life long learning”, Educational Technology & Society, 3(1), pp 32–40Patry J L (1998) “Transfer evaluation in educational processes:models, results and problems - a theoretical approach”,http://www.sbg.ac.at/erz/salzburger beitraege/patry 98 2.htm444

Patterson D, Snyder L and Ullman J (1999) “Best Practices Memo: evaluatingcomputer scientists and engineers for promotion and tenure”,www.cra.org/reports/tenure review.html, first published in Computing ResearchNewsPerry W G (1988) Forms of Intellectual and Ethical Development in the College Years: ascheme, San Francisco: Jossey-Bass, an updated reprint of the original 1970 workPerry-Sheldon R and Allain V A (1987) Using educational research in the classroom, Bloomington(IN): Phi Delta Kappa Educational FoundationPfleeger S L (1999) “Albert Einstein and empirical software engineering”, IEEE Computer,32(10), pp 32–37Pham B, Bruce C and Stoodley I (2005) “Constituting information technology research:the experience of IT researchers”, Higher Education Research & Development, 24(3), pp215–232Piaget J (1968) La Structuralisme, London: Routledge & Kegan Paul, translated in 1971 byC MaschlerPiaget J (1980) “The psychogenesis of knowledge and its epistemological significance”, London:Routledge & Kegan Paul, pp 23–34Pincus J D (2005) “Computer Science is really a social science”,http://research.microsoft.com/users/jpincus/cs SocSci.htmlPohl K (1994) “The three dimensions of Requirements Engineering: a framework and itsapplication”, Information Systems, 19(3), pp 243–258Pohl K (2003) “Software Systems Engineering: main research areas”, http://www.sse.unidue.de/wms/en/index.php?go=108Polanyi M (1958) Personal Knowledge, London: Routledge & Kegan PaulPolkinghorne D (1988) Narrative Knowing and the Human Sciences, Albany (NY): StateUniversity of New York PressPotts C (1993) “Software-Engineering research revisited”, IEEE Software, 10(5), pp 19–28Powell R R (1991) Basic Research Methods for Librarians, Norwood NJ: Ablex, 2nd edProject Management Institute (2000) A Guide to the Project Management Bodyof Knowledge, Newtown Square, Pennsylvania: Project Management Institute,http://www.pmi.org/publictn/pmboktoc.htmProsser M and Trigwell K (1999) Understanding Learning and Teaching: the experience inhigher education, Buckingham: Oxford University PressRaisey D, Tan K, Swatman P A and Blackburn T (2006) “An empirical study of the evolvingdynamics of creative teams in action”, in “Proceedings of the International Conference onCreativity and Innovation in Decision Making and Decision Support (CIDMDS)”, London:International Federation for Information ProcessingRamsden P (1988) “Studying learning: improving teaching”, in Ramsden P (ed) “ImprovingLearning: new perspectives”, London: Kogan Page, pp 13–31Reed S K (1993) “A schema-based theory of transfer”, in Detterman and Sternberg (1993),pp 39–67445

Reeves T C (1994) “Evaluating what really matters in computer-based education”,http://www.medicine.mcgill.ca/ibroedu/review/Reeves Evaluating WhatReally Matters in Computer-Based Education.htmReeves T C (1997a) “Established and emerging evaluation paradigms for instructional design”,in Dills C R and Romiszowski A J (eds) “Instructional Development Paradigms”,Englewood Cliffs (NJ): Educational Technology Publications, pp 163–178Reeves T C (1997b) “A Model of the effective dimensions of interactive learningon the World Wide Web”, in Veteli J (ed) “Proceedings of InteraktiivinenTeknologia Koulutuksessa (ITK’97)”, Hameenlinna (Finland), pp 23–31,http://itech1.coe.uga.edu/Faculty/treeves/WebPaper.pdfReichardt C S and Cook T D (1979) “Beyond qualitative versus quantitative methods”, inCook T and Reichardt C S (eds) “Qualitative and Quantitative Methods in EvaluationResearch”, Beverly Hills (CA): Sage, pp 7–32Reigeluth C M (1996) “A new paradigm for ISD?”, Educational Technology, 36(3), pp 13–20Reigeluth C M (1997) “Instructional theory, practitioner needs and new directions: somereflections”, Educational Technology, 37(1), pp 42–47Resnick L (ed) (1989) Knowing, Learning and Instruction: essays in honour of Robert Glaser,Hillsdale (NJ): ErlbaumRichardson J T E (1990) “Reliability and replicability of the Approaches to Studying questionnaire”,Studies in Higher Education, 15, pp 155–168Richey R C (1997) “Agenda-building and its implications for theory construction in instructionaltechnology”, Educational Technology, 37(1), pp 5–11Richie J and Spencer L (1994) “Qualitative data analysis for applied policy research”, inBryman A and Burgess R G (eds) “Analysing Qualitative Data”, London: Routledge, pp173–194Ridley D S, Schultz P A, Glanz R S and Weinstein E (1992) “Self-regulated learning: theinteractive influence of metacognitive awarenss and goal-setting”, Journal of ExperimentalEducation, 60(4), pp 293–306Rittel H J (1969) Reflections on the scientific and political significance of decision theory,Technical Report Working Paper 115, Institute of Urban and Regional Development, Universityof California, Berkeley (CA)Rittel H J and Webber M M (1984) “Planning problems are wicked problems”, in Cross N(ed) “Developments in Design Methodology”, Chichester: Wiley, pp 135–144Robillard P N (1999) “The role of knowledge in software development”, Communications ofthe ACM, 42(1), pp 87–92Robillard P N (2005) “Opportunistic problem solving in Software Engineering”, IEEE Software,22(6), pp 60–67Robinson H (2001) “Reflecting on research and practice”, IEEE Software, 18(1), pp 110–112Robley W, Whittle S and Murdoch-Eaton D (2005) “Mapping generic skills curricula: outcomesand discussion”, Journal ofFurther and Higher Education, 29(4), pp 321–330Rogers P L (2002) “Teacher-Designers: how teachers use instructional design in real classrooms”,in Rogers P L (ed) “Designing Instruction for Technology-Enhanced Learning”,Hershey (PA): Idea Group, pp 1–17446

Rorty R (1991) Objectivity, Relativism and Truth: philisophical papers, volume 1, Cambridge(MA): Cambridge University PressRosenshine B (1986) “Synthesis of research on explicit learning”, Educational Leadership, pp60–69Rossi M and Sein M (2003) “Design Research Workshop: a proactiveresearch approach”, in “Proceedings of Information Systems ResearchSeminar in Scandinavia (IRIS26)”, Helsinki: IRIS Association,http://tiesrv.hkkk.fi/iris26/presentation/workshop designRes.pdfRowntree D (1987) Assessing Students, London: Harper and RowRoy G G and Armarego J (2003) “The Development of On-line Tests on Multiple ChoiceQuestions”, in Taniar D and Rahayu W (eds) “Web-Powered Databases”, Hershey (PE):Idea Group, pp 121–143Royce W W (1970) “Managing the development of large software systems: concepts andtechniques”, Proceedings of the IEEE WESCON, 1970, pp 1–9Russell D R (ed) (1998) The limits of the apprenticeship models in WAC/WID research,http://www.public.iastate.edu/ drrussel/russell4c98.htmlRyan T and Frangenheim E (2000) “How we see Bloom’s taxonomy”, The E-Learner LessonPlanner, http://www.e-learning.com.au/portal/bloom2.aspSalomon G and Perkins D N (1989) “Rocky roads to transfer: rethinking mechanisms of aneglected phenomenon”, Educational Psychologist, 24, pp 113–142Sambell K, McDowell L and Brown S (1997) “But is it fair?: an exploratory study of studentperceptions of the consequential validity of assessment”, Studies in Educational Evaluation,23(4), pp 349–371Sasser-Coen J R (1993) “Qualitative changes in creativity in the second half of life”, Journalof Creative Behaviour, 27(1), pp 18–27Savery J R and Duffy T M (1995) “Problem based learning: an instructional model and itsconstructivist framework”, Educational Technology, 35(5), pp 31–38Savin-Baden M (2000) Problem-based Learning in Higher Education: untold stories, Buckingham(UK): Society for Research into Higher Education and Open University PressSawyer P and Kotonya G (2000) SWEBOK: Software Requirements Engineeringknowledge area description, Los Alamitos (CA): IEEE Computer Society/ACM,http://www.swebok.org/, version 0.6Schön D A (1983) The Reflective Practitioner: how professionals think in action, New York:Basic BooksSchön D A (1985) The Design Studio, London: RIBA PublicationsSchön D A (1987) Educating the Reflective Practitioner, San Fransisco: Jossey-BassSchön D A (1995) “Knowing-in-action: the new scholarship requires a new epistemology”,Change, 27(7), pp 27–34Schultze U (2000) “A confessional account of an ethnography about knowledge work”, MISQuarterly, 24(1), pp 3–41447

Scott G and Wilson D N (2002) “Tracking and profiling successful IT graduates: an exploratorystudy”, in Wenn A, McGrath M and Burstein F (eds) “Enabling Organisationsand Society through Information Systems: proceedings of the 13th Australasian Conferenceon Information Systems”, Melbourne: Victoria University of Technology, pp 1185–1195Scott G and Yates W (2002) “Using successful graduates to improve the quality of undergraduateengineering programs”, European Journal of Engineering Education, 27(4), pp60–67Scribner S and Sachs P (1990) A Study of On-the-job Training, New York: Institute onEducation and the Economy, Columbia UniversitySeaman C B (1999) “Qualitative methods in empirical studies of Software Engineering”,IEEE Transactions on Software Engineering, 25(4), pp 557–572Searle N and Clarke S (2000) “Real World Software Inc (RWSI) project”, QUT Teaching andLearning Large GrantSegers M and Dochy F (2001) “New assessment forms in problem-based learning: the valueaddedof the students perspective”, Studies in Higher Education, 26(3), pp 327–343Senge P (1994) The fifth discipline fieldbook: strategies and tools for building a learningorganization, London: Nicholas Brealey PublishingSenn J A (1978) “A management view of systems analyists: failures and shortcomings”, MISQuarterly, 2(3), pp 25–32Sfard A (1998) “On two metaphors for learning and the dangers of choosing just one”,Educational Researcher, 27(2), pp 4–13Shackelford R (ed) (2005) Computing Curricula 2005: the overviewreport, The Joint Task Force for Computing Curricula 2005,http://www.acm.org/education/curric vols/CC2005-March06Final.pdfSimon H A (1956) “Rational choice and the structure of the environment”, PsychologicalReview, 63(2), pp 129–138Simon H A (1960) The New Science of Management Decisions, New York: Harper and RowSimon H A (1973) “The structure of ill structured problems”, Artificial Intelligence, 4, pp145–180Simon H A (1979) “Information processing models of cognition”, Annual Review of Psychology,30, pp 363–396Simon H A (1980) “Problem solving and education”, in Tuma and Reif (1980), pp 81–96Simon H A (1981) Sciences of the Artificial, Cambridge (MA): MIT Press, 2nd edSkinner B F (1953) Science and Human Behaviour, New York: Free PressSlater T F (1996) “Portfolio assessment strategies for grading first-year university physicsstudents in the USA”, Physics Education, 31(5), pp 329–333Smyth R (2003) “Concepts of change: enhancing the practice of academic staff developmentin higher education”, International Journal for Academic Development, 8(1/2), pp 51–60Snoke R and Underwood A (1999) “Generic attributes of IS graduates - a Queensland study”,in “Proceedings of the 10th Australasian Conference on Information Systems”, Wellington(NZ): Victoria University of Wellington, pp 817–823448

Sobel A E K (ed) (2003) Computing Curricula – Software Engineering Volume: final draft ofthe Software Engineering Education Knowledge (SEEK), Los Alamitos (CA): IEEE ComputerSociety Press, http://sites.computer.org/ccse/artifacts/FinalDraft.pdfSoloman B and Felder R M (1999) Index of Learning Styles(ILS), Raleigh, (NC): North Carolina State University,http://www2.ncsu.edu/unity/lockers/users/f/felder/public/ILSpage.htmlSomekh B (1989) “Action research and collaborative school development”, in McBride R(ed) “The Inservice Training of Teachers: some issues and perspectives”, Brighton: FalmerPress, pp 160–176Sommerville I, Rodden T, Sawyer P, Bentley R and Twidale M (1993) “Integrating ethnographyinto the requirements engineering process”, in IEEE (a), pp 165–173, in IEEE (1993)Sommerville I and Sawyer P (1997) Requirements Engineering: a good practice guide, Chichester:WileySpiro R J, Feltovich P J, Jacobson M and Coulson R (1991) “Cognitive flexibility,constructivism and hypertext: random access instruction for advanced knowledgeacquisition in ill-structured domains”, Educational Technology, 31, pp 24–33,http://www.ilt.columbia.edu/ilt/papers/Spiro.htmlStahl G (1999) “Perspectives on collaborative knowledge-building environment:towards a cognitive theory of computer support for learning”,http://d3e.open.ac.ik/cscl99/Stahl/Stahl-paper.htmlStenhouse L (ed) (1975) An Introduction to Curriculum Research and Development, London:HeinemannStepien W J, Gallagher S A and Workman D (1993) “Problem-based learning for traditionaland interdisciplinary classrooms”, journal for the Education of the Gifted, 16(4), pp 338–357Sternberg R J (1987) “Teaching intelligence: the application of cognitive psychology to theimprovement of intellectual skills”, in Baron J and Sternberg R J (eds) “Teaching ThinkingSkills”, New York: FreemanSternberg R J and Horvath J A (1998) “Cognitive conceptions of expertise and their relationsto giftedness”, in Friedman R and Rogers K (eds) “Talent in context”, Washington (DC):American Psychological Association, pp 177–191Straub D, Gefen D and Boudreau M C (2005) “The ISWorld Quantitative, Positivist ResearchMethods Website”, http://www.dstraub.cis.gsu.edu:88/quant/Struyven K, Dochy F and Janssens S (2005) “Students perceptions about evaluation andassessment in higher education: a review”, Assessment & Evaluation in Higher Education,30(4), pp 325–341Sun Microsystems (2006) “Java Technology”, http://java.sun.com/reference/docs/Sutcliffe A G and Maiden N A M (1992) “Analysing the novice analyst: cognitive models insoftware engineering”, International Journal of Man-Machine Studies, 36, pp 719–740Suthers D (2000) “Initial evidence for representational guidance of learning discourse”, in“Proceedings of the International Conference on Computers in Education”, Taipei: HigherEducation Research and Development Society of Australasia Inc.449

Suwa M, Gero J and Purcell T (2000) “Unexpected discoveries and s-inventions of designrequirements: important vehicles for a design process”, Design Studies, 21(6), pp 539–567Swales J M (1990) Genre analysis: English in academic and research settings, Cambridge:Cambridge University PressTan H (1989) Private Sector training in the United States: who gets it and why, New York:Institute on Education and the Economy, Columbia UniversityTharp R and Gallimore R (1991) The instructional conversation: teaching and learning insocial activity, Technical Report, University of California, Santa CruzThomas J C, Lee A and Danis C (2002) “Enhancing creative design vis software tools”,Communications of the ACM, 45(10), pp 112–115Thomas J C, Lyon D and Miller L (1977) Aids for Problem Solving, Technical Report RC-6468, IBM IBM T. J. Watson Research, New YorkThorndike E L (1924) “Mental discipline in high school studies”, Journal of EducationalPsychology, 15, pp 1–22; 83–98Tichy W F (1993) “Should computer scientists experiment more?”, IEEE Computer, 31(5),pp 32–40, http://wwwipd.ira.uka.de/ tichyTodd P A, McKeen J D and Gallupe R B (1995) “The evolution of IS job skills: a contentanalysis of IS job ads”, MIS Quarterly, 19(1), pp 1–37Trauth E M, Farwell D and Lee D M S (1993) “The IS expectation gap: industry expectationversus academic preparation”, MIS Quarterly, 17, pp 293–307Trigwell K and Prosser M (1997) “Towards an understanding of individual acts of teachingand learning”, Higher Education Research and Development, 16, pp 241–252Trigwell K, Prosser M and Waterhouse F (1999) “Relations between teachers approaches toteaching and students approaches to learning”, Higher Education, 37, pp 57–70Tripp S D (1993) “Theories, traditions and situated learning”, Educational Technology, pp71–77Trotter E (2006) “Student perceptions of continuous summative assessment”, Assessment &Evaluation in Higher Education, 31(5), pp 505–521Truex D P and Klein H K (1991) “A rejection of structure as a basis for information systemsdevelopment”, Amsterdam: Elsevier Science, pp 213–235Tuma D T and Reif F (eds) (1980) Problem Solving and Education: issues in teaching andresearch, Hillsdale (NJ): ErlbaumTurley R T (1991) Essential Competencies of Exceptional Professional Software Engineeris,PhD thesis, Colorado State University, Fort Collins (CO)Turley R T and Bieman J M (1995) “Competencies of exceptional and non-exceptional softwareengineers”, Journal od Systems and Software, 28(1), pp 19–38Turner A J and Tucker A (1991) “Computing Curricula 1991: Report of the ACM/IEEE-CSJoint Curriculum Task Force”, Communications of the ACM, 34(6), pp 68–84Turner R and Lowry G (1999) “Educating Information Systems professionals: towards arapprochement between new graduates and employers”, in Hope B and Yoong P (eds)“Proceedings of the 10th Australasian Conference on Information Systems”, Wellington(NZ): Australian Computer Society/ACIS, pp 1049–1058450

Turner R and Lowry G (2003) “Education for a technology-based profession: softening theInformation Systems curriculum”, in McGill (2003), pp 153–172Tynjälä P, Salminen R, Sutela T, Nuutinen A and Pitkänen S (2005) “Factors related to studysuccess in engineering education”, European Journal of Engineering Education, 30(2), pp221–231Underwood A (1997) The ACS Core Body of Knowledge for InformationTechnology Professionals, Sydney: Australian Computer Society,http://www.acs.org.au/national/pospaper/bokpt1.htmUWA (1996) Do male and female students differ in their preferred style of learning?, Perth:Institutional Research Unit, University of Western AustraliaVaishnavi V and Kuechler W (2004) “Design Research in Information Systems”,http://www.isworld.org/Researchdesign/drisISworld.htmVan Rossum E J and Schenk S M (1984) “The relationship between learning conception,study strategy and learning outcome”, British Journal of Educational Psychology, 54, pp73–83Vanderbilt C T G (ed) (1997) The Jasper Project: lessons in curriculum, instruction, assessment,and professional development, Mahwah (NJ): ErlbaumVickers G (1984) The Vickers Papers, London: Open Systems Group, Harper and RowVisser W (1990) “More or less following a plan during design: opportunistic deviations inspecification”, International Journal of Man-Machine Studies, 33, pp 247–278Visser W (1992) “Designers’ activities examined at three levels: organisation strategies andproblem-solving processes”, Knowledge-Based Systems, 5(1), pp 92–104Visser W and Hoc J (1990) “Expert software design strategies”, in Hoc J M, Green T R G,R S and J G D (eds) “Psychology of Programming”, San Diego (CA): Academic Press, pp235–247Vygotsky L S (1978) Mind and Society: the development of higher psychological processes,Cambridge (MA): Harvard University PressWaks L J (2001) “Donald Schön’s philosophy of design and design education”, InternationalJournal of Technology and Design Education, 11, pp 37–51Wallis G (1926) The Art of Thought, New York: HarcourtWalsham G (1993) Interpreting Information Systems in Organisations, Chichester: WileyWalton H J and Mathews M B (1989) “Essentials of problem-based learning”, Medical Education,23, pp 542–548Wankat P and Oreovicz F (1998) “Content tyranny”, Prism, 9(10), http://www.prism-magazine.org/october/html/october teaching.htmWaterman R H, Waterman J A and Collard B A (1994) “Toward a career resilient workforce”,Harvard Business Review, 69, pp 87–95Watson G K, Morgan C K, McKenzie T, Roberts D and Cochrane K (2002) “Meeting thechallenge of positioning undergraduate units of study at the appropriate level”, in “QualityConversations: proceedings of the HERDSA Annual International Conference”, Canberra:Higher Education Research and Development Society of Australasia Inc., pp 704–712,http://www.herdsa.org.au/publications451

Watson H J, Young D, Miranda S, Robichaux B and Seerley R (1990) “Requisite skills fornew MIS hires”, Data Base, 21(2), pp 20–29Wegner P (1976) “Research paradigms in computer science”, in “Proceedings of the InternationalConference on Software Engineering (ICSE)”, Los Alamitos (CA): IEEE ComputerSociety Press, pp 322–330, http://www.idi.ntnu.no/emner/dif8916/wegner.pdfWeinberg G M (1971) The Psychology of Computer Programming, New York: van NostrandReinholdWeiner B (1992) Human Motivation: metaphors, theories and research, Newbury Park (CA):SageWenger E (1998) Communities of Practice: learning, meaning and identity, Cambridge (MA):Cambridge University PressWenger E (2000) “Communities of practice and social learning systems”, Organization, 7(2),pp 225–246Wertsch J V (1991) Voices of the Mind: a sociocultural approach to mediate action, Cambridge(MA): Harvard University PressWhite B Y and Frederiksen J R (1986) Progressions of quantitative models as a foundationfor intelligent learning environments, Cambridge (MA): Bolt, Beranek and Newman., techReport 6722White K B (1985) “Perceptions and deceptions: issues for information systems research”, inMumford et al (1985), pp 237–242Wigglesworth S (1993) “The Crisis of Professionaization: British Architecture 1993”, Practices,2Wilson B G and Cole P (1996) “Cognitive teaching models”, in Jonassen (1996), pp 601–621Wilson B G and Myers K M (1999) “Situated cognition in theoreticaland practical context”, in Jonassen and Land (2000), pp 57–88,http://ceo.cudenver.edu/ brent wilson/SitCog.htmlWilson K and Fowler J (2005) “Assessing the impact of learning environments on students’approaches to learning: comparing conventional and action learning designs”, Assessment& Evaluation in Higher Education, 30(1), pp 87–101Winn W and Snyder D (1996) “Cognitive perspectives in psychology”, in Jonassen (1996),pp 112–142Winograd T (1996) Bringing Design to Software, Reading (MA): Addison-WesleyWinograd T and Flores F (1986) Understanding Computers and Cognition, Reading (MA):Addison-WesleyWittgenstein L (1953) Philosophical Investigations, New York: MacmillanWohlin C, Runes P, Höst M, Ohlsson M C, Regnell B and Wesslen A (2000) Experimentationin Software Engineering: an introduction, London: Kluwer Academic PublishersWood D (1999) “Editorial: representing, learning and understanding”, Computers & Education,33, pp 83–90Wood P (1988) “Action research: a field perspective”, Journal of Education for Teaching,14(2), pp 135–150452

Wood-Harper T (1985) “Research methods in Information Systems: using action research”,in Mumford et al (1985), pp 169–191Woods D R (1996a) Problem-based Learning: helping your students gainthe most from PBL, Waterdown (Ontario): McMaster University, 3rd ed,http://www.chemeng.mcmaster.ca/pbl/chap1.pdfWoods D R (1996b) Problem-based Learning: helping your students gain the mostfrom PBL (for teachers), Waterdown (Ontario): McMaster University, 3rd ed,http://www.chemeng.mcmaster.ca/pbl/chap1.pdfWortley S (2000) “Business as usual or action researh in practice”, Action Researcher E-Report, 6, http://www.cchs.usyd.edu.au/arow/arer/006.htmWright J v (1992) “Reflections on reflection”, Learning and Instruction, 2, pp 59–68Xia F (1998) “What’s wrong with software engineering research methodology”, ACM SIG-SOFT Software Engineering Notes, 23(1), pp 62–65Zeldin T (1998) Conversation: how talk can change our lives, Mahwah (NJ): HiddenSpringZimitat C and Alexander H (1999) “The 4 Step Assessment Task (4SAT)”, Integrity, Innovationand Integration, 4, pp 692–697Zmud R W (1998) “Conducting and publishing practice-driven research”, in Larsen T J,Levine L and DeGross J I (eds) “Proceedings of IFIP WG8.2 and WG8.6 joint workingconference on Information Systems: Current Issues and Future Changes”, Helsinki,http://www.bi.no/dep2/infomgt/wg82-86/proceedings/zmud.pdfZowghi D (2004) “Bottom-up RE methods”,/re-archive/http://research.it.uts.edu.au/reZowghi D, Damian D and Offen R (2001) “Field studies of Requirements Engineering in amulti-site software development organisation: research in progress”, in “6th AustralianWorkshop on Requirements Engineering (AWRE’01)”, SydneyZuber-Skerritt O (1982) Action Research in Higher Education, London: Kogan PageZuber-Skerritt O (1995) “Models for action research”, in Pinchen S and Passfield R (eds)“Moving On: creative applications of action learning and action research”, Mt Gravatt(Qld): Action Learning, Action Research and Process Management Assn, Inc, pp 3–29Zywno M and Waalen J (2001) “The effect of hypermedia instruction on achievement andattitudes of students with different learning styles”, in “Proceedings of the 2001 AmericanSociety for Engineering Education Annual conference and Exposition Session 1330”,http://www.asee.org/conferences/search/00287 2001.PDF453

Appendix AThe Murdoch contextAs noted in Chapter 5, the context of the study was a specific educational environment,within the School of Engineering at Murdoch University. The aim of this appendix is toprovide a more detailed description of the environment in which the study was conducted.Murdoch University’s School of Engineering Science has, since 1995, provided a suite ofprogrammes in Software Engineering (SE) from a 4-year undergraduate degree in Engineering,through post-graduate and masters programmes to a PhD in Software Engineering.Murdoch’s Bachelor of Engineering (Software Engineering) (BE(SE)) was the second programmein Australia to receive full accreditation from the Institution of Engineers, Australia(EA). This means that our graduands are fully accredited professional engineers, with allthat implies.From a curriculum point of view, this also means that the program must conform to EA’srequirements as well as those of the Australian Computer Society (which has also accreditedthe programme).The teaching objectives are focused on producing engineers with a special skill in software.We expect graduates from our BE(SE) to find career opportunities in both professional engineeringindustries that have a strong interest in software as well as the full range of ITdisciplines where the design and implementation of quality software is considered a priority.Pursuing these objectives has meant a gradual shift from more traditional engineering learning,in particular how the learning is undertaken, as we address characteristics specific toSE.454

A.1 Curriculum componentsAs Figure A.1 shows, the curriculum for the BE(SE) may be viewed as three intersectingcomponents, all within an envelope that integrates the knowledge gained. While changescontinue to be made, the underlying conceptual model remained the same throughout thestudy.Figure A.1: BE(SE) Murdoch University: curriculum components pre2002 (Armarego et al,2001)At the commencement of this research, the programme of study was structured as follows,with a degree of overlap between the components to provide learning leverage. The primarycomponents:Computer Science - these units cover fundamental aspects (eg programming, algorithmanalysis, database and operating system concepts) and form the basis of technicalknowledge and skills in software and hardwareSoftware Engineering - these units focus on SE theory and practice (eg, requirements, userinterface, management, metrics and maintenance, organisational issues) and form thebasis of core knowledge and skill in software development and evolution. Assessmentin some of these units focuses on project-based teamworkEngineering - these units offer knowledge and skills in engineering practice and principlesand include those elements of EA’s curriculum requirements not covered in the previous455

components.These are common to all Engineering students within our School (egnatural sciences, mathematics, management, ethics)provided the basis for:Design Project/Engineering Thesis - these are also common to all Engineering students,though the domain of application targets the appropriate discipline of study. While theProject may be industry-based, it is run under controlled conditions, and carefullymonitored by academic staff. The Thesis, on the other hand focuses on industry andmay be linked to work-place experiences: the student spends 25% of the penultimatesemester, rising to 50% in the last. Supervision is joint academic/industry, with thestudent required to complete and present a thesis based on the project.Underlying these is a common set of support material and resources, including web resources,process tools and documentation templates. Students are encouraged to apply this materialas much as possible, and in some instances are formally required to do so.The decision has been made to offer support for units, so far as is practical, online: theWeb was seen as a medium to support student control of the learning process through itscapacity to help learners develop unique knowledge representations (Miller and Miller, 1999)and is said to be well suited to domains of conceptual complexity and case-to-case irregularitywhere teaching in a hands-on medium has application (Brandt, 1997). Many areasof Engineering (and in particular Software Engineering) fit this category of material. ThisICT-based integrated environment is described more fully in Armarego et al (2001).Within this curriculum framework there are eight core SE units: Requirements Engineering(ENG260) is the first to which students are exposed, offered in semester 1 of the second yearof study. Prior to this, all engineering students have undertaken a common first year: theyhave generally been immersed in a scientific/engineering paradigm where problem-solvingthrough laboratory procedure, repeatability of experimentation and rigour in mathematicsare key learning objectives. The result of this is strong preconceptions in first year studentsabout both teaching and learning.As can be seen from the description above, this curriculum closely models a normative professionaleducation curriculum (Waks, 2001) previously discussed (see Chapter 3), in whichstudents first study basic science, then the relevant applied science , so that learning maybe viewed as a progression to expertise through task analysis, strategy selection, try-out andrepetition (Winn and Snyder, 1996).456

A.2 Characteristics of teaching and learningThere has been much debate in the literature regarding influences on learning. While it isaccepted that the practice of education relies on the communicative behaviour, emotions,and beliefs of both teachers and students, some authors (Felder, 1996; Kolb, 1984) lookto personality, life experiences, and the purpose of the learning as prime influences, others(Grundy, 1987; Trigwell and Prosser, 1997) add to a particular set of life experiences, studentexpectations of teachers as the dominant influence on learning.The potential to succeed academically is jeopardised by complex lack of matching betweenthe student and the discipline. Attrition in Engineering also involves a complex set of factorsincluding the quality of student interactions with instructors and peers. Motivation andsuccess are adversely affected if their learning styles, and the learning styles of the staffteaching them, are not taken into account. This issue is discussed in greater detail in Chapter9.It is therefore appropriate to look at both teacher and learner characteristics as importantcomponents of the learning environment in which this research was undertaken.A.2.1Learning stylesOther research undertaken within the School looks at learning styles (Fowler et al, 2000),and focusses on instruments developed by Kolb (1984) and by Soloman and Felder (1999).These instruments are applied to all first year students at Murdoch’s Rockingham campus(where Engineering is situated), early in their first semester and a cumulative database isbeing compiled.The term learning styles refers to an individual’s preferences for receiving, integrating andpresenting ideas and information. Examples, taken from Mills et al (2005) include: findingit easier to understand a new concept by reading a textbook, whilst others prefer a pictorialexplanation; variety in how students most effectively demonstrate their understanding:graphically, verbally, or in writing.Table A.1 provides a comparison between Engineering students (both first and final (fourth)year) and first year students in Arts/Commerce and Computer Science/IT. Engineering Staffare also included, in order to identify any mismatches between teacher/learner, while FigureA.2 indicates that some gender biases are also present.In brief, Kolb’s Learning Style Inventory is a simple test based on experiential learning theory.It looks at four stages of the learning process: concrete experience (CE), reflective observation457

Table A.1: Kolb Learning Style Inventory 1999 - 2003 cumulative results (percentages) (extractedfrom figures presented in Armarego (2004b))Eng1styearA/C1styearCS/IT1styearEng4thyearEngStaffAccomodator 8 13 5 3 0Diverger 18 13 12 7 17Assimilator 33 47 56 38 41.5Converger 41 27 27 52 41.5(RO), abstract conceptualisation (AC), and active experimentation (AE).Twelve questions are presented and the user has to rank four possible answers for eachquestion. The users learning style can be identified as either:Accomodator : (concrete, active) What if? peopleDiverger : (concrete, reflective) Why or why not? peopleAssimilator : (abstract, reflective) What? peopleConverger : (abstract, active) How? people.Figure A.2: Kolb Learning Style Inventory 1999 - 2003 cumulative results 1st year Engineeringstudents plus the researcher (Armarego, 2004b)Engineers are considered to have predominantly Converger characteristics: they are pragmatists(Convergers) who revel in active experimentation (labs, fieldwork) with a tendency tonarrow technical interests, or theorists (Assimilators) with a forté in the basic sciences (Kolb,458

1984). As can be seen both from Table A.1 and Figure A.2, there is a weighting towards thesecharacteristics in our first year Engineering students as well as our staff. Both reasons forand implications of these biases are explored in other research within the School (eg Fowleret al (2001, 2002)).The results in Table A.2 confirm these findings and show that:• 56% of the students learn best actively, yet our teachers are predominantly reflective,and most engineering courses other than laboratories rely almost exclusively on lecturesand readings as the principal vehicles for transmitting information (Felder andSilverman, 1988)• 63% of the students are sensors, yet our teachers tend to be intuitive with traditionalengineering instruction also tending to be heavily oriented toward intuitors, emphasisingtheory and mathematical modelling over experimentation and practical applications(Felder and Silverman, 1988)Table A.2: Soloman & Felder Index of Learning Styles 1999 - 2003 cumulative results (percentages)(extracted from figures presented in Armarego (2004b)Eng1styearA/C1styearCS/IT1styearEng4thyearEngStaffProcessing Perception Input UnderstandingACT REF SEN INT VIS VER SEQ GLO56 44 63 37 77 23 56 4465 35 68 32 76 24 54 4649 51 70 30 84 16 68 3276 24 55 45 86 14 59 4127 73 36 64 73 27 45 55• 77% of the students are visual, yet traditionally material is presented to them verballyor in written form, emphasising written explanations and mathematical formulations ofphysical phenomena over demonstrations and visual illustrations (Felder and Silverman,1988)• 44 % of the students are global learners. However,this dimension does not involve thesame type of mismatch observed for the others: teaching is often narrowly focused. Despitethis apparent match, Felder and Brent (2005) indicates global students constitute459

a strong and important minority: they are the multidisciplinary thinkers, whose broadvision may enable them to become, for example, skilled researchers or chief executiveofficers of corporations.Whilst some allowance needs to be made for the different sample sizes between students andstaff, these figures demonstrate a significant variation between these two groups. In particulara clear preference for active, sensory learning is demonstrated amongst students while staffexhibit a strong preferences for reflective learning.Table A.3: Reported learning style preferences (percentages) (Felder and Brent, 2005)EngStudentAverageEngFacultyAverageProcessing Perception Input UnderstandingACT REF SEN INT VIS VER SEQ GLO N64 36 63 37 82 18 60 40 250645 55 41 59 94 6 44 56 101The strong skew towards visual, as opposed to verbal, learning styles is also noted in otherlearning styles research examining the characteristics of engineering students, as is a strongertendency towards sensory rather than intuitive learning.Felder and Brent (2005), in acollation of figures from nearly 20 studies applying ILS to engineering students and staffsupports these findings. Their figures are summarised in Table A.3.A.3 The learning environmentAs has been noted previously, Requirements Engineering (ENG260) is the first of the core SEunits, offered in the first semester of the second year of study, after a common first year. Theunit had, since its inception in 1999, been taught in workshop mode (two sessions of 2-hourstwice per week). Other than during the first few weeks of classes in 1999, the unit has beenco-ordinated and taught by the same lecturer, with guest experts invited in as required. Allmaterial is available online, so lectures and tutorials are replaced by discussion, exercises andgroup evaluation of alternatives presented.The learning environment, and the appropriateness of its web-enabled (or web-enhanced(Boettcher and Conrad, 1999) nature have been described elsewhere (Armarego et al, 2001):the environment goes a long way in addressing the dichotomy of learning/teaching styles.460

Figure A.3: The SENavigator for the Software FactoryIt should be noted that, although the infrastructure for this system was in existence priorto 1999, material in the Software Factory named RESD (Requirements Engineering andSoftware Design) was produced by (or for) the lecturer, and the pedagogical componentcompletely her work. Briefly, the Software Factory presents a coherent system and learningcontext. Rules are established so that the cognitive overhead required by the medium in minimisedthrough: consistency (limiting the appearance of fragmentation); effortless/automaticnavigation; increased orientation so that the content (not just the user interface) allows thestudent to identify current position, history, options, etc. and learning support in terms ofaccess to discussion fora, email, bugreporting.Within the two clusters (each comprising four units with an emphasis on theory (RESD) orapplication (SE1234)) which make up the Software Factory, topics are categorised mnemonicallyas sections. This allows for ‘chunking big’ and focusses on connections between topicsin the same category for content- and context- dependent knowledge construction (Jonassen,1992). The SENavigator provides the tools for navigating and accessing the unit material. InFigure A.3, the section Requirements Modelling (top row) is highlighted. Since this is an REunit all topics are active. If in the ‘select course’ another unit had been chosen, these topicswould not be active (large X through the button, and no hot link to the material). The labelsare naturally rather short and cryptic, but the full title can be displayed by simply hoveringover the required button.Each unit has a comprehensive study programme comprising a set of topics across relevant461

sections. Once authenticated as a member of the class, a student can access the unit materialboth through the SENavigator or the Production Line. Within each unit topics are sequencedand displayed on a map that provides alternative routes from commencement to completion.To a certain extent these provide choice in the order of topics studied and allow students tovary the sequencing of content. This degree of freedom to control access to information isnot unlimited. While, in theory, the exam date is the only relevant marker for completionof the unit, in practice milestones (in the form of assignments/projects) and support in theform of workshop schedules dictate the dates by which topics must be completed. Externalstudents (when the unit is available in this mode) have some greater degree of freedom bynot being involved in workshops. In addition, the Production Line enables students to easily‘explore the world’ of each unit - each node (see Figure A.4) is directly linked to the relevanttopic for browsability, although backwards/forwards links exist between topics as well. ThisFigure A.4: The production line for ENG260 in the Software Factory462

is one mechanism for addressing the preference for global learning exhibited by some of ourstudents. Colours indicate mandatory (cyan), optional and extension material, and individualprogress may be recorded: the nodes can be decorated to indicate ‘in progress’ or ‘completed’.Notes may also be attached to a node by a student, (eg perhaps as a reminder to check somepoint with the lecturer).Figure A.5: The MCQ environment in the Software FactoryWhile the learning environment has as its focus activities/real-world problem solving, online/interactiveactivities cease to be meaningful if the student hits a snag and is unable toprogress from there. Scaffolds are cognitive tools to help students overcome this problem.Examples include exercises; explanations and background information; monitoring tools (tohelp students keep track of their progress); modelling scratchpads (software tools to createand manipulate models); problem hints; process coordinators(which guide the studentsthrough the complete task cycle); and planning tools (Jong, 2006).The purpose of scaffolding is to provide activity-sensitive help mechanisms. The SoftwareFactory provides examples of all these scaffolds, as appropriate, both as purpose-built activity463

help and underlying manuals. The former through a ‘help’ icon on an activity screen, whilethe latter is best demonstrated through the underlying help in the FM (Formal Methods)topics, where help is activated through ‘hot’ spots in the notation itself. Both of thesemechanisms are not imposed on the student, but are readily available. Links to the helpmechanisms are seamless, which enables the student to maintain focus on the learning activity,rather than on the task of retrieving aid.Students are also able to monitor their own conceptual understanding through the MCQenvironment (see Figure A.5). Described in greater detain in Roy and Armarego (2003) thisenvironment allows the teacher to set several parameters: whether the student can browse;whether a set of questions can be attempted more than once; time/date of test availability.Questions/answers/explanations are input and optionally assigned a degree of difficulty, witha ‘set’ composed by including/excluding specific questions. After an attempt, the studentchooses to have the test marked. Once marked short explanations can often be found underthe ‘?’ buttons. A database records visits, attempts and score achieved, with this informationavailable to the unit co-ordinator.A.4 Student cohort for ENG260Before 2002 the student cohort for ENG260 was predominantly SE majors (see Figure A.6),with a majority of students having completed their first year at Murdoch. The implication ofthis was an enculturation to the environment in the School of Engineering, as well as a clearindividual motivation to do well in this unit – it would determine student ability to completethe degree in their chosen discipline. However, in 2000 University policy changed to enablestudents with qualifications from a near-by technical college to articulate into programmesin the School of Engineering. These students came to Engineering at the commencementof second year, and consequently enrolled in ENG260 with very different expectations ofthe learning environment. A strong bi-polar distribution of the final marks for the ENG260during 2001 was a primary impetus for reflection and examination of the learning environmentand teaching style.464

Figure A.6: Student cohort for Eng260 1999-2002465

Appendix BInstrumentsAlthough most instruments used within this study could be classed as opportunistic use ofthose already developed and tested as reliable, some instruments were either adapted fromthose already existing, or developed from scratch.This appendix provides an overview of the development of these, the treatment of the datacollected, and provides a copy, for information.B.1 Education Relevance SurveyDuring 2001 a survey was constructed to examine local (ie in Perth Western Australia)practitioner perspectives on the relevance of formal education for Requirements Engineering.The surveyWith his permission, this was based on Lethbridge (1999). The instrument was built as aweb-based survey: respondents were requested to complete the survey online and submit.This sent an email message to the researcher, with each response coded. The survey wasmodified based on feedback from the ‘pilot’ participant during 2003 and finally presented atthe end of 2003. Samples of the introduction, response coding and submitting in html areprovided below:...Requirements EngineeringEducation Relevance SurveyVersion2003_d0.1Aug 2003This questionnaire is designed to discover what aspects of y466

our educational background have been useful to youin your career, and is targetted specifically to those who spendsome proportion of their time on RequirementsEngineering tasks....Systems Theory and Practice...Open Ended Question If there are any comments you have about this survey,please write them here: Once again, thank-you for participating!The survey itself is provided as Attachment 1 of this Appendix.As in Lethbridge’s survey, four questions were asked of a number of topics. Responses werebased on a 6-point Likert scale, worked meaningfully for each question1. How much did you learn about this in your formal education (e.g. University or College)?Responses: 0=Learned nothing at all; 1=Became vaguely familiar; 2=Learnedthe basics; 3=Became functional (moderate working knowledge); 4=Learned a lot;5=Learned in depth – became expert (Learned almost everything)2. What is your current knowledge about this, considering what you have learned on thejob as well as forgotten? 0=Know nothing; 1=Am vaguely familiar; 2=Know the basics;3=Am functional (moderate working knowledge); 4=Know a lot; 5=Know in depth /am expert (Know almost everything)467

3. How useful have the details of this specific material been to you in your career as asoftware developer or software manager? Please leave blank if you know little aboutthe material. 0=Completely Useless; 1=Almost never useful; 2=Occasionally useful;3=Moderately useful, but perhaps only in certain activities; 4=Very useful; 5=Essential4. How much influence has learning the material had on your thinking (i.e. your approachto problems and your general intellectual maturity), whether or not you have directlyused the details of the material? Please consider influence on both your career andother aspects of your life. Please leave blank if you know little about the material. 0=Noinfluence at all; 1=Almost no influence; 2=Occasional influence; 3=Moderate influencein some activities 4=Significant influence in many activities; 5=Profound influence onalmost everything I doThe survey comprises five sections:Knowledge Section to help determine what knowledge participants consider influences theway they work. This addresses Requirements Engineering Topics (these are based on anexamination of the discipline and the identification of components that would compriseREBoK described in Chapter 2. Topics are linked to a brief description, (based on theREBoK), in an attempt to have a common understanding for terms, since participantsare drawn from a number of backgrounds. These are also provided in Attachment 1);Other Software Topics; Mathematics Topics; Hardware/Computer Engineering Topics;Non-IT Material Typically StudiedSkills Section to help determine which skills participants consider influence the way theywork. This addresses Generic Skills drawn from the literature of practitioner studies,also discussed in Chapter 2Summary of Generic Skills this addresses the topics Management of Self; Management ofInformation; Management of Others; Management of Task. These are drawn from thework of Bennett et al (2000), who present a model of generic skills, based on a lengthystudy of academic staff, student and employer perspectives of the skills associated withuniversity education and employment, and how those skills are acquired at universityand in employment settings. These topics are also linked to a brief description withinthe survey, again for common understandingDemographic Questions to help determine if people with different types of backgroundhave different needs. Again this section is based on the Lethbridge instrumentOpen Ended Question to capture any additional comments.468

Treatment of the dataParticipation in the survey was purposeful: to assist the study by Minor, organisationsaffiliated with the School of Engineering were approached. These were chosen as exemplars oforganisations whose main business was the development of software from a Computer Science,Information Systems or Software Engineering perspective. In addition, size of organisationspanned small, medium and large. The description of the choice of organisation, and therationale behind the choices made is provided in detail in Minor (2004). The organisationwas asked to nominate participants for the study, based on a set of criteria of which the mostrelevant was practitioner experience in RE activities. The participants were then approachedwith organisational buy-in and backing. They were informed, that in addition to participationin the Minor study, they would be participating in a larger study looking at confirmingor otherwise conclusions drawn from practitioner-based research outside Western Australia.Completion of the survey described above would be the mechanism for achieving this aim.Participant responses were coded and an analysis undertaken, again modelled on Lethbridge.An example of the results of one component of the survey is provided in Figure B.1Figure B.1: Participant responses to questions 1-4 applied to topic: RE processAs the data indicate, a gap exists between participant response to Question 1, and Questions3 and 4. A summary of the results of the survey is provided in Chapter 2.469

B.2 School of Engineering Year SurveyThe Engineering discipline within the School informally surveys all students within each yeargroup to identify general problems that are both unit-specific, and that relate to the mix ofunits undertaken.This survey is administered twice per semester, usually early, near the first point of feedback(preferably Week 4) and then again towards the end of formal classes (preferably Week 11).This allows for timely feedback to students, intervention, if required, to be undertaken andfeedback from students before the completion of semester. A carry over to the next offeringis also indicated as required.The survey is very simple: students are asked to indicate good/bad points of the unit as theyperceive them and to make any general comments.The format therefore is thus:• for each unit available in the particular year (this example is the complete feedback forthe Year 2 2002 week 4 survey for the unit under study)ENG260 REQUIREMENTS ENGINEERINGGood:Can see industry advantages, interactive,good lecturer, theory, easy to understand, suitable info forSoftware engineers, interesting, shows how to analysestuff.Bad:Lots of work, not enough info of what to dowith portfolio and what we are expected to achieve, mind mappingeach topic, some concepts hard to understand where they areuseful, too much of the reading for the week, makes the brainsquirm, too much workload leading to no marks, what is rationalrose for?• GENERAL COMMENTS (this example is the complete feedback for the Year 22002 week 4 survey)What you like:470

Computing III and data Communications, challenging andrewarding, all well organised, nice teaching from people whoknow what they are talking about, the site visits, very industryrelevant and job ready, ICE is pretty interesting and I’m toldthat there are plenty of jobs in this field, units slightlyoverlap so greater understanding is achieved, help is alwaysavailable, South St component, good job prospects, smallnumbers, applicable stuff, staff availability, beats the hellout of statistics, getting more specified in the ICE stream,good timetable layout, lots of labs and small assignments,definitely worthwhile sticking to, the lecturer is enthusiasticand helpful and approachable, the class work is enjoyable andconveyed with humour, great atmosphere at the campus, coursegoing at a good challenging pace, the energy units areinteresting, Data Comms. feels like it rules our total workload,all at Rockingham campus, EC is an interesting subject, I likethe software side of things, particularly programming, wellstructured.What you don’t like:8 contact hours on Thursdays, trip from South St. on Friday forjust one lab on Friday, some units like Data communications isirrelevant for people in SE, Data Comms. is hard, we need morecomputers in the labs or a new general lab that doesn’t havescheduled labs in it, the workload is very heavy, have to dostuff at Rockingham, no pub at Rockingham campus, wait two hoursbetween lecture and lab, travelling to Rockingham, can’t getaccess to computer labs, no females!!!, difficulty in gettingtextbooks, often no computers are available when needed, manycomputers aren’t working, I live 2 hrs from campus and the breakbetween lecture and lab is waste of my time, the unit is spreadover the whole day so I can’t get other units on that day, notsure about Internship I thought uni finds you the internship andI don’t want to do a thesis, too much reading.471

B.3 Student Surveys of UnitsUniversity-supported instruments are provided through the Institutional Research and EvaluationService of the Teaching and Learning Centre. The Student Survey of Unit (SSU) isthe primary formal questionnaire used in this research. However, it should be noted thatsince most classes are considered too small for statistical procedures to be appropriate, thevalue of this feedback is in the qualitative comments. The following description is taken fromthe website (Murdoch, 2004b).The unit questionnaire consists of eight core questions which have been developed with referenceto the university’s Unit Quality Standards. The core questions are:1. It was clear what I was expected to learn in this unit2. The assessment tasks were appropriate to the learning objectives3. The feedback on my marked work was useful for my learning in this unit4. The assessment tasks tested my understanding of the subject area, rather than justmemory5. Activities in this unit helped me achieve the learning objectives6. The unit resources were useful for my learning in this unit7. I was satisfied with the level of support from staff in this unit8. Overall, I was satisfied with the quality of this unit.Questions are asked on a 6-point scale from Strongly Agree to Strongly Disagree with theadditional option of Unable to Judge.In addition to the core questions, unit co-ordinators may select up to ten optional questionsfrom the item bank. There are also three standard open-ended questions for student comments:1. Please comment on what you thought were the best aspects of this unit.2. What improvements (if any) would you suggest?3. Any other comments?All unit surveys are undertaken using MOSS (Murdoch Online Survey System). Surveys aregenerally open in the last two teaching weeks (12 and 13) of the semester/trimester and thestudy break week (where applicable).472

The results are made available once the grades for the unit have been finalised.It should be noted that the School of Engineering decided that all units would be surveyedafter each offering (rather than on the 2-year cycle described above), and that a cash-prizeincentive would be offered to maximise responses.473

B.4 Attachment 1: Educational Relevance Survey instrument474

Questions about the relevance of educational materialThis is the core of the survey...Knowledge SectionThis section will help us determine what knowledge you consider influences the wayyou workRequirementsEngineeringTopicsIf you wish, you mayuse fractionalnumbers (e.g. 2.5,4.5). The scales aredescribed in detailwith this question andabbreviatedelsewhere.Each topic in thissection is linkedto a briefdescriptionHow much didyou learn aboutthis in your formaleducation (e.g.University orCollege)?0=Learned nothing atall1=Became vaguelyfamiliar2=Leaned the basics3=Became functional(moderate workingknowledge)4=Learned a lot5=Learned in depth;became expert(Learned almosteverything)What is yourcurrentknowledge aboutthis, consideringwhat you havelearned on the jobas well asforgotten?0=Know nothing1=Am vaguelyfamiliar2=Know the basics3=Am functional(moderate workingknowledge)4=Know a lot5=Know in depth / amexpert (Know almosteverything)How usefulhave thedetails of thisspecificmaterialbeen to youin yourcareer as asoftwaredeveloper orsoftwaremanager?Please leaveblank if youknow littleabout thematerial.0=CompletelyUseless1=Almost neveruseful2=Occasionallyuseful3=Moderatelyuseful, butperhaps only incertainactivities4=Very useful5=EssentialHow muchinfluencehas learningthe materialhad onyourthinking(i.e. yourapproach toproblemsand yourgeneralintellectualmaturity),whether ornot youhavedirectlyused thedetails ofthematerial?Pleaseconsiderinfluenceon bothyourcareer andotheraspects ofyour life.Pleaseleave blankif you knowlittle aboutthematerial.

Systems Theoryand PracticeRequirementsEngineeringprocessRequirementsElicitationRequirementsAnalysisSoftwareRequirementsSpecificationRequirementsValidationRequirementsManagementKnowledge ofSoftwarePackagesGroup Dynamics0=Noinfluence atall1=Almost noinfluence2=Occasionalinfluence3=Moderateinfluence insomeactivities4=Significantinfluence inmanyactivities5=Profoundinfluence onalmosteverything IdoFormal Methodsand SpecificationtechniquesOther RE Topicsnot specifiedabove:Topic:

Topic:Learned in formaleducation?0=nothing3=moderate/functional5=in depth/expertCurrent knowledge?0=nothing3=moderate/functional5=in depth/expertHow usefulhave detailsbeen?0=useless3=moderatelyuseful5=essentialInfluence ongeneralthinking?0=noinfluence3=moderateinfluence5=profoundinfluenceOther SoftwareTopicsUse the same 0 - 5scale as beforeIf you wish, you may usefractional numbers (e.g.2.5, 4.5). The scales aredescribed in detail in theRE section. andabbreviated elsewhere.Process standards(CMM / ISO 9000etc)Object OrientedConcepts andTechnologyPerformanceMeasurement andAnalysisProject Management(includingscheduling,estimation andmeasurementstechniques)Data Structures andAlgorithmsHow much didyou learn aboutthis in yourformaleducation?0=nothing3=moderate/functional5=in depth/expertWhat is yourcurrentknowledge aboutthis?0=nothing3=moderate/functional5=in depth/expertHowuseful havedetails ofthespecificmaterialbeen inyourcareer?0=useless3=moderatelyuseful5=essentialHow haslearningthematerialinfluencedyourthinkingwhether ornot youhave usedthedetails?0=noinfluence3=moderateinfluence5=profoundinfluence

Fundamentals ofDesignHuman ComputerInteraction / UserInterfacesSoftwareArchitectureDesign PatternsSpecificProgrammingLanguagesProgrammingLanguage TheoryVerification andValidationfundamentalsTesting Tools andMethodsQuality AssuranceLearned in formaleducation?0=nothing3=moderate/functional5=in depth/expertCurrent knowledge?0=nothing3=moderate/functional5=in depth/expertHow usefulhave detailsbeen?0=useless3=moderatelyuseful5=essentialInfluence ongeneralthinking?0=noinfluence3=moderateinfluence5=profoundinfluenceConfiguration andRelease ManagementMaintenance,Reengineering andReverse EngineeringSoftware MetricsDatabasesInformation RetrievalSecurity andCryptographyOperating SystemsSystemsProgrammingData Transmissionand Networks

Parallel andDistributedProcessingReal-Time SystemDesignSimulationLearned in formaleducation?0=nothing3=moderate/functional5=in depth/expertCurrent knowledge?0=nothing3=moderate/functional5=in depth/expertHow usefulhave detailsbeen?0=useless3=moderatelyuseful5=essentialInfluence ongeneralthinking?0=noinfluence3=moderateinfluence5=profoundinfluenceArtificial IntelligencePattern Recognitionand Image ProcessingComputer GraphicsComputationalComplexity andAlgorithm AnalysisOther SoftwareTopics not specifiedabove:Topic:Toipc:Topic:Learned in formaleducation?0=nothing3=moderate/functional5=in depth/expertCurrent knowledge?0=nothing3=moderate/functional5=in depth/expertHow usefulhave detailsbeen?0=useless3=moderatelyuseful5=essentialInfluence ongeneralthinking?0=noinfluence3=moderateinfluence5=profoundinfluenceMathematicsTopicsIf you wish, youmay use fractionalHow much did youlearn about this inyour formaleducation?What is yourcurrentknowledge aboutthis?How usefulhave detailsof thespecificmaterialHow haslearning thematerialinfluencedyour

numbers (e.g. 2.5,4.5). The scales aredescribed in detail inthe RE section. andabbreviatedelsewhere.Differential andIntegral CalculusDifferentialEquationsLinear Algebraand MatricesProbability andStatisticsPredicate Logic0=nothing3=moderate/functional5=in depth/expert0=nothing3=moderate/functional5=in depth/expertbeen in yourcareer?0=useless3=moderatelyuseful5=essentialthinkingwhether ornot youhave usedthe details?0=no influence3=moderateinfluence5=profoundinfluenceSet TheoryGraph TheoryInformationTheoryAutomata theoryQueuing theoryCombinatoricsControl TheoryFormalLanguagesLaplace andFourierTransformsOther MathsTopics notspecified above:Topic:Topic:Learned in formal Current knowledge? How useful Influence on

education?0=nothing3=moderate/functional5=in depth/expert0=nothing3=moderate/functional5=in depth/experthave detailsbeen?0=useless3=moderatelyuseful5=essentialgeneralthinking?0=no influence3=moderateinfluence5=profoundinfluenceHardware /ComputerEngineeringTopicsUse the same 0 - 5scale as beforeIf you wish, you may usefractional numbers (e.g.2.5, 4.5). The scales aredescribed in detail in theRE section. andabbreviated elsewhere.Digital Electronicsand Digital LogicMicroprocessorArchitectureComputer SystemArchitectureNetworkArchitecture andData TransmissionTelephony andTelecommunicationsAnalog ElectronicsRoboticsDigital SignalProcessingVLSIData AcquisitionOther HardwareHow much didyou learn aboutthis in yourformaleducation?0=nothing3=moderate/functional5=in depth/expertWhat is yourcurrentknowledge aboutthis?0=nothing3=moderate/functional5=in depth/expertHow usefulhavedetails ofthespecificmaterialbeen inyourcareer?0=useless3=moderatelyuseful5=essentialHow haslearningthematerialinfluencedyourthinkingwhether ornot youhave usedthe details?0=noinfluence3=moderateinfluence5=profoundinfluence

Topics not specifiedabove:Topic:Topic:Learned in formaleducation?0=nothing3=moderate/functional5=in depth/expertCurrent knowledge?0=nothing3=moderate/functional5=in depth/expertHow usefulhave detailsbeen?0=useless3=moderatelyuseful5=essentialInfluence ongeneralthinking?0=noinfluence3=moderateinfluence5=profoundinfluenceNon-IT MaterialTypically StudiedUse the same 0 - 5scale as beforeIf you wish, you may usefractional numbers (e.g. 2.5,4.5). The scales aredescribed in detail in theRE section. and abbreviatedelsewhere.Science(Physics/Chemistry orother)Business(Accounting/Marketingor other)EconomicsLawManagementEntrepreneurshipOther Engineering (notHow much didyou learn aboutthis in yourformaleducation?0=nothing3=moderate/functional5=in depth/expertWhat is yourcurrentknowledge aboutthis?0=nothing3=moderate/functional5=in depth/expertHowuseful havedetails ofthespecificmaterialbeen inyourcareer?0=useless3=moderatelyuseful5=essentialHow haslearningthematerialinfluencedyourthinkingwhether ornot youhave usedthedetails?0=noinfluence3=moderateinfluence5=profoundinfluence

Computer or Software)PsychologyPhilosophySociologyEthics andProfessionalismSecond Language(Other than English asa Second Language)Other Non-IT Topicsnot specified above:Topic:Topic:Learned in formaleducation?0=nothing3=moderate/functional5=in depth/expertCurrent knowledge?0=nothing3=moderate/functional5=in depth/expertHow usefulhave detailsbeen?0=useless3=moderatelyuseful5=essentialInfluence ongeneralthinking?0=noinfluence3=moderateinfluence5=profoundinfluenceSkills SectionThis section will help us determine which skills you consider influence the way youworkGeneric SkillsUse the same 0 - 5scale as beforeIf you wish, you mayuse fractional numbers(e.g. 2.5, 4.5). Thescales are described indetail in the RE section.and abbreviatedelsewhere.How muchdid youlearn aboutthis in yourformaleducation?0=nothing3=moderate/functional5=indepth/expertWhat is yourcurrentknowledgeabout this?0=nothing3=moderate/functional5=indepth/expertWhich (if any) ofthe abovesubjects/coursesprovided thegreatest input tothis skill?Please choose onespecific topic only, ifpossible.How has learningthe materialinfluenced yourthinking whetheror not you haveused the details?0=no influence3=moderate influence5=profound influencePlease note that column 3requires a text answerIf you wish to be moregeneral, choose thecategory heading

Creativityinstead (eg RE Topicsinstead of a specifictopic such asRequirementsElicitation)Other (pleasespecify)Problem/opportunityidentificationOther (pleasespecify)Verbalcommunications(interviewing,questioning,listening)Verbalcommunications(GivingPresentations to anAudience)Writing skills(Technical/nontechnical)Other (pleasespecify)Other (pleasespecify)Other (pleasespecify)Group/team skillsOther (pleasespecify)LeadershipOther (pleasespecify)Negotiation,conflict resolutionand consensusbuilding techniquesOther (please

(eg Delphi, IBIS)specify)General problemsolvingstrategies (eg divide/conquer,means/end,recursive subgoal,rote)AdaptabilityOther (pleasespecify)Other (pleasespecify)Cultural insight(understanding ofproject context,either cultural,organisational,national , etc)Metacognitivestrategies (how tolearn, reflect on thatlearning andincorporate theseinto tasks)ReasoningOther (pleasespecify)Other (pleasespecify)Other (pleasespecify)Critical Thinking(critical thinker hasan inquiring mind,knows how to askgood questions anduses their answersto makemeaning)Physical skills(expertise of theprocedural tasks,includingappropriatetool/notation use)Cognitive skills(processes ofOther (pleasespecify)Other (pleasespecify)

analysis,interpretation anddecision-makingrequired for the'carrying out' ofprocedural tasks)Other GenericSkills not specifiedabove:Other (pleasespecify)Skill:Other (pleasespecify)Skill:Other (pleasespecify)Learned informaleducation?0=nothing3=moderate/functional5=indepth/expertCurrentknowledge?0=nothing3=moderate/functional5=indepth/expertWhich (if any) of theabove subjects/coursesprovided the greatestinput to this skill?Influence on generalthinking?0=no influence3=moderate influence5=profound influenceSummary ofGeneric SkillsUse the same 0 - 5scale as beforeIf you wish, you may usefractional numbers (e.g.2.5, 4.5). The scales aredescribed in detail in theRE section. andabbreviated elsewhere.How much did youlearn about this inyour formaleducation?0=nothing3=moderate/functional5=in depth/expertWhat is yourcurrent knowledgeabout this?0=nothing3=moderate/functional5=in depth/expertHow has learningthe materialinfluenced yourthinking whether ornot you have usedthe details?0=no influence3=moderate influence5=profound influenceEach topic in thissection is linked to abrief descriptionManagement of Self

Management ofInformationManagement ofOthersManagement of TaskLearned in formaleducation?0=nothing3=moderate/functional5=in depth/expertCurrent knowledge?0=nothing3=moderate/functional5=in depth/expertInfluence on generalthinking?0=no influence3=moderate influence5=profound influenceDemographic QuestionsThese will help us find out if people with different types of background have differentneeds.1. Country where you have workedmost:2. State (in Australia) where youworked most in the last 5 years :3. Highest degree earned:4. Year you obtained your highestdegree:5. Topics of your degrees. If youhave degrees in more than onetopic, or have a double major, thenplease check more than one box:6. Country where you obtained yourhighest degree:7. Cumulative number of yearsworking on computer software:8. Please indicate the approximatepercentage of your total workingtime that you have spent on theSoftware EngineeringComputer ScienceInformation SystemsComputer/Systems EngineeringElectric/Electronic EngineeringOther EngineeringOther ScienceMathematicsBusinessManagement/AdministrationOther

following activities during the lastyear. Please include both timeworking in groups as well asalone:a) Management or projectmanagement:b) Requirements analysis orspecification:c) Software architecture anddesign:d) Working with source code(writing code, understanding codeetc.)e) Testing software written byothersf) Installation, customer supportetc.9. Please indicate the percentage ofyour total working time you havespent understanding or modifyingsoftware or documents written byothers (i.e. performingmaintenance as opposed to newdevelopment):10. Which of the following categoriesmost accurately describes the teamin which you most often work:11. On which of the following types ofsoftware have you performedsignificant work over the last fiveyears? You may select more thanone category:Small team: 1-7 people working on asmall system (typically less than 10 000 linesof code)Medium team: 5-20 people working on amedium sized system (typically 8 000 to 150000 lines of code)Large team: 15 to 60 people working on alarge and complex system (typically between100 000 and 2 million lines of code)Very large team: Over 45 people workingon an extremely large and complex system(typically more than 1.5 million lines of code)(Question not answered)Real-time, embedded, systems ortelecommunications software (in general,software that is developed as part of a largersystem).Management information software orother software for running the business (e.g.accounting, inventory etc.) that is being

12. Which of the following mostclosely describes the industry inwhich you would consider yourcompany to be primarily involved.If you select one of the 'other'options on the right, please specifythe industry here:developed or tailored largely for in-house use.Consumer or mass-market software(typically sold on the open-market in shrinkwrappedpackages).Application software produced forspecialised markets that does not fit into theabove categories.Development of consumer softwareTelecommunications, networkingsoftware developmentEngineering or scientific softwaredevelopmentFinancial software developmentDevelopment of other applicationsoftware for specialised marketsSoftware consultingOther software, please specify at leftMilitaryOther government, including governmentagencies, law enforcement, localBanking, finance, insurance, financialservices or consulting etcTelecommunications or networkingequipment manufacturingManufacturing other computerhardware/peripherals, computer engineeringTelecommunications or networkingserviceAeronautics, space, defense contractingOther engineering or manufacturing,please specify at leftEducation (schools, universities etc)Law, legal servicesHealth careRetailing, marketing, distributing etcBroadcasting, publishing etcOther service industry, please specify atleft

Non-profit associationOther, please specify at leftIdentification Questions (Optional)Your answers to these 3 questions will allow us to match this survey with theinterview you have participated in.In accordance with Murdoch University's Human Ethics Reseach codes of practice,the details below will be recoded so as not to identify you or the organisation forwhich you work.13. Name:14. Email Address:15. Organisation:Open Ended QuestionIf there are any comments you haveabout this survey, please write themhere:Once again, thank-you for participating!Submit Survey

Requirements EngineeringEducation Relevance SurveyThis document provides a brief definition of some of the Topics in the survey, foryour assistance.Requirements Engineering Topicsthis covers system theory and concepts,decision theory and its implementation by ITSystems Theory andPracticefeasibility assessmentrisk assessment and management principlesinfrastructure planningstrategies to foster opportunity finding and detect/identifyproblems to solveintroduces the process, places it in a systems context anddescribes project organisation and contractual issues through thesubtopics:an overview of input activities to the REprocessRequirementsEngineering processprocessmodelsprocessactorsprocesssupport andan introduction of generic process modelsprovides an understanding of the systemdevelopment and modification processenables students to evaluate and select asystem development methodology(procedural, O-O, evolutionary etc)the roles of the people who participate in theRE processthe nature of the 'stake' and compromisesrequiredproject management resources required andconsumed by the RE process

managementprocessquality andimprovementdevelopment and adherence to life cyclestandardsorient the RE process with quality standardsand process improvement modelsconcerned with where requirements come from and how theymay be collected. This is seen as a fundamentally humanactivity, and includes the subtopics:requirementssourcesdesigned to promote an awareness of differentrequirement sources and frameworks formanaging them.the stakeholdersoperational and organisational environmentsare identifiedRequirementsElicitationelicitationtechniquesgroupdecisionprocesssystem goals formulateddomain knowledge identified and acquiredconcentrates on techniques on facilitating thearticulation of requirements. [eg workshops,interviewing techniques, brainstorming, focusgroups, JAD, etc]explain common forms of behaviour that canlead to lack of communicationsIt emphasises the factors for effective communication andintegration with users and user systems.This is seen to be a non-passive activitythe process of analysing requirements to resolve conflicts,determine the bounds of the system and elaborate softwarerequirements from system requirements, through the subtopicsRequirementsAnalysisrequirementsclassificationconceptualmodellingrequirements may be classified on a numberof dimensions to help inform tradeoffs[functional/nonfunctional/other]guidance on the purpose and intent ofmodelling as an aid to understanding of theproblem and its domain.The factors that influence the models

architecturaldesign andrequirementsallocationrequirementsnegotiationscoping andscopecontrolevaluationmetricsdeveloped, and their appropriatemethods/notations are included [O-O models,data models, quality function deploymentmatrices, Use of appropriate modeling tools]derivation of the architectures andcomponents from the requirementsconflict detection and resolutionoptimising, satisficingconsensus buildingdetermine definition of the problemdetermine bounds of the systeminvestigate alternative solutions anddetermine their feasibilitydevelop and apply metrics for systemobjectives and customer satisfactionthe structure, quality and verification of the requirementsdocument(s) as a precondition to successful requirementshandling, described through the subtopics:SoftwareRequirementsSpecificationrequirementsdefinitiondocumentsoftwarerequirementsspecificationdocumentstructure andstandardsdocumentqualitya record of system requirementsand its roles [including basic concepts offormal specification techniques]including factors influencing organisationalinterpretation of theseassessing the quality of the documentsthrough the use of metricsthrough the use of [peer reviews, walkthroughs, scenarios]RequirementsValidationconduct ofrequirementsreviewformally, of documentationprototyping as validation of requirements interpretationmodelvalidationacceptanceeg static analysis or formal reasoningplanning for verification of each requirement

teststhroughRequirementsManagementchangemanagementrequirementsattributesrequirementstracingdescribes its role, procedures and the analysisof proposed changesdevelopment of attributes for requirementsmanagementin order to recover the source of requirementsand predicting their effectKnowledge of Toolsand SoftwarePackagesthroughknowledgeapplicationevaluationunderstanding of the application of differenttools and packagesapply CASE or other tools as a mechanism toensure a particular methodology or standard isused rigorouslyof alternativesunderstand group, team and personal behaviour in a systemscontextGroup Dynamicsteampersonalinterpersonalapplicationofappropriatestrategiesand toolsparticipate as team members, controlledthrough project management. Team conceptsand mechanisms are applied and monitoredleadership, empowerment, use of influence,politicsdevelop skills for effective communicationswith clients, users, team members, and othersassociated with development, operation andmaintenance of the systemto facilitate group work and developconsensus [eg groupware]Formal Methods andSpecificationTechniquesTeam concepts including personal and interpersonal skills arediscussed and monitored. Empowerment concepts will be usedand measured.understand the place of Formal Methods in RequirementsEngineering through an introduction toconceptsspecificationalternatives and applications

languagesformal verificationbasic principlesSummary of Generic SkillsManagement of SelfManagement ofInformationManagement ofOthersManage time effectivelySet objectives, priorities and standardsTake responsibility for own learningListen actively with purposeUse a range of academic skillsDevelop and adapt learning strategiesShow intellectual flexibilityUse learning in new or different situationsPlan/work towards long term goalsPurposely reflect on own learningClarify with criticism constructivelyCope with stressUse appropriate sources of informationUse appropriate technologiesUse appropriate mediaHandle large amounts of informationUse appropriate language and formInterpret a variety of information formsPresent information competentlyRespond to different purposes/contexts and audiencesUse information criticallyUse information in innovative and creative waysCarry out agreed tasksRespect the views and values of othersWork productively in a co operative contextAdapt to the needs of the groupDefend/justify views and actionsTake initiative and lead othersDelegate and stand backNegotiateOffer constructive criticismTake the role of chairLearn in a collaborative contextAssist/support others in learningManagement of TaskIdentify key featuresConceptualise ideasSet and maintain prioritiesIdentify strategic optionsPlan/implement a course of actionOrganise subtasksUse and develop appropriate strategiesAssess outcomes

Appendix CStudent Reflective JournalsReflective journals were a component of the assessment for ENG302 - the follow-on unit forthe Requirements Engineering studio (ENG301). Journal entries were based on considerationof the following questions:what did I achieve this week?what issues did I have this week?what can I do next week to address these issuesin the areas of learning, personally and in the group. Students were also able to make generalcomments.This appendix contains verbatim entries from these journals, and reflect the dynamic natureof the learning experience. A discussion of the value of these enties is provided in Chapter 8.VaughnWeek 2What did I achieve this week:In learning Starting to get a feel for how powerful Z modelling langauageisPersonally Learnt some new Excel spread sheet tricksIn my group Got caught out on back foot when JA wanted us to developthe scope - thought it was going to be a informal talk aboutprojectWhat issues did I have this week:In learning Spent to much time on the activity spread sheetPersonally Time management to many things to little hours :(In my group None the team has got started on the taskWhat can I do next week to address these issues:In learning Need to concentrate on the imporatnt stuff i.e. getting theexercises donePersonally Continue planning my timeIn my group None496

.VaughnWeek 6What did I achieve this week:In learning Became aware of the ideas of patterms in software engineeringPersonally External issues have reduced the amount of time spent onunitsIn my group Team is not working togeather well. The work is becomingfragmented and not of a high standardWhat issues did I have this week:In learning Still need to catch on the Z stuffPersonally Time managementIn my group We need to make up the time lost in the requirement sectionWhat can I do next week to address these issues:In learning Spend more time on Uni unitsPersonally Would like to spend time on Z but now need to spend timeon AchetictureIn my group Keep the pressure on by reminding everyone of what needsto be done to meet the deadlinesVaughnWeek 17What did I achieve this week:In learning The need to keep people focused on time e.g. not to have tobig a presentationPersonally Felt I contributed well to the presentationIn my group We achieved an excellent resultWhat issues did I have this week:In learning Not muchPersonally Not much. Felt quilty that I was not able to help [Markus]moreIn my group None we worked togeather wellWhat can I do next week to address these issues:In learning Reflect a bit more on what has happenedPersonally NothingIn my group Start thinking about next year497

Alaina Week 3What did I achieve this week:In learning Had a thought about the project deeper to create Use CasescenariosPersonally Managed to worked on the project and course material concurrentlyIn my group Created use cases based on the discussion with group membersWhat issues did I have this week:In learning Getting behind in studying the course materials. Also, Inceis getting more difficultPersonally Did not spend much time on the other unitIn my group Did not prepare before meetingWhat can I do next week to address these issues:In learning Need to go back a few chapters of Ince and understand thefundamentalsPersonally Need to plan my studyIn my group Do prepare so that we can focus on discussing issues effectivelyat meetingsAlainaWeek 7What did I achieve this week:In learning I did not feel well this week and did not achieve lotsPersonally Even though I did some reading, it has not really understoodyetIn my group The project cought up with the schedule!What issues did I have this week:In learning I could not work out how those architecture styles in textbooks are used in actual systemsPersonally It took some time to play around with the sequence diagramsand make sense hot the system worksIn my group Although I did not attend the class on Tuesday, the othermembers did excellent job choosing architecture styles andallocating task effectivelyWhat can I do next week to address these issues:In learning It will be good idea to ask about other members’ understandingon those stylesPersonally After having Sequence, Activity and ST diagrams, the structureof our system should be clearIn my group Do the best to complete the Architecture Model by its due!AlainaWeek 18What did I achieve this week:In learning Working System!!Personally Good to review my management and reflectIn my group Great All of us did really good jobWhat issues did I have this week:In learning NervousPersonally Nervous nervousIn my group None!What can I do next week to address these issues:In learningPersonallyIn my group Do a good job next semester!!498

.What did I achieve this week:In learningDermotWeek 3Know how to do Project scope, context diagram, and a newthing the DictionaryPersonallyIn my groupWhat issues did I have this week:In learningPersonallyIn my groupWhat can I do next week to address these issues:In learningPersonallyIn my groupGetting things in on time, remembering to do stuffget organised use diary moreDermotWeek 9What did I achieve this week:In learning learned about design patterns and their wide use in industryPersonally I learned that I very much enjoy being a project manager, andwork better and more efficiently, and effectivly as a manageror with set tasksIn my group Keeping people motivated all the time is hard however wemanaged to make both meetings productive for majority ofthe timeWhat issues did I have this week:In learning design patterns are easy to understand however applyingthem is a whole different storyPersonally getting organised was a taskIn my group people are going to be absent next week this will be a problem,we will have to make do with threeWhat can I do next week to address these issues:In learning keep at itPersonally keep at itIn my group Go TEAM !! No complaints as long as everyone gets theirparts done499

DermotWeek 10This week has been good Management seems to be easier than sitting back andwaiting to be told what we need to do and by when. Enjoying project manageras you get a better overall picture of the assignments and not just your task, ifyou can see how everything fits together it makes the individual tasks that mucheasier.What did I achieve this week:In learning Learned a great deal on patterns, and their use in softwaredevelopment, implementation and designPersonally I think I am doing wekk in this area in terms of understandingthe material, applying it is not as easy but I think im gettingthereIn my group I think the group is doing ok tasks are being completed atleast to an extent by the time that it is dueWhat issues did I have this week:In learning No problems, small issues in implementing and understandingthe original system and the way we want to do itPersonally Grout work is shitty would be so much easier to completetask on my ownIn my group I think that it is ok [Markus] is not here so me and [Alaina]are trying to cover his loadWhat can I do next week to address these issues:In learning BlomPersonally keep at itIn my group Go TEAM !! Complaint as to the level of effort put in , andthe amount of work completed by today concern as to thedue date, and getting it in on time and being a good pieceof workMarkusWeek 2What did I achieve this week:In learning Started recalling previous concepts learnt from the requirementsunit. Started to gain some concept of the Z language,basically the same as logical Table with different symbolsPersonally Some concerns with time with operation pending in September,withdraw from one other unit to hopefully prevent fallingbehind. Don’t want to let the group downIn my group No issues with any group membersWhat issues did I have this week:In learning Some confusion on the specification of the Project and whois doing what workPersonally Time management concerns, withdrew 1 unit loadIn my group Some overlapping of work loadWhat can I do next week to address these issues:In learning Keep reading various text on the Z language and review previouslearning of requirementsPersonally Keep focused on Time ManagementIn my group Clearly knowing of each members task and keeping all groupmembers updated with progess of individual components ofthe project500

MarkusWeek 4What did I achieve this week:In learning Did a lot more with working with Zed, it is difficult gettingused to the syntext of the materialPersonally Feeling more confident on working with requirements modelingfor the project, easier to recall than I thoughtIn my group The group is working more and more as a teamWhat issues did I have this week:In learning Some confusion with if using the correct syntax when writingZed, spoke with Jocelyn and she informed me there are manyways you can write Zed and she was more interested withproper use of the syntax that makes the correct sensePersonally My wife is still not well [words deleted], at home my time isvery hard to manage with the kidsIn my group Having no issues with the group working well, the only issueis I think we are a little behind with our projectWhat can I do next week to address these issues:In learning Practice Practice Practice with the Zed LanguagePersonally Must keep focused and not to distracted and keep TimeMangement under controlIn my group Keep up with the work allocated to me and keep a positivea productive outlook with other group membersMarkusWeek17What did I achieve this week:In learning More confident with working within the java environment,acting as manager of project, back on schedule, completedimplementation reportPersonally Enjoying the Java coding in real world applicationIn my group Been helping group members with their Java codingWhat issues did I have this week:In learning Errors in Java coding especially passig valuesPersonally Having no major issues at this time just many hours neededto complete projectIn my group No issue with group everyone is working their best, somestressing over other unit commitmentsWhat can I do next week to address these issues:In learning Keep revising java to ensure completion of projectPersonally Keep track that group members are keeping up with work,structure out the remaining context of work, help otherswhen neededIn my group Support helping Java coding and understanding501