Evaluation of MSc programmes: MSc in Healthcare Management ...

Evaluation of MSc programmes:
MSc in Healthcare Management
MSc Quality and Safety in Health Care Management
MSc Leadership & Management Development
1

An Evaluation of MSc in Healthcare Management, MSc Quality and Safety in Health
Care Management, MSc Leadership & Management Development Programmes at
the Institute of Leadership, Royal College of Surgeons in Ireland.
Prepared by
Dr. Jonathan Drennan & Dr. Pauline Joyce
Institute of Leadership
Royal College of Surgeons in Ireland
April 2012
2

1.1 Introduction
This report outlines the findings from an educational evaluation that
comprehensively measured the impact that masters’ degrees in healthcare
management, leadership & management development and quality and safety in
healthcare management had on graduates who completed the programmes at
the Institute of Leadership, Royal College of Surgeons (RCSI) in Ireland. The
results presented here include graduates’ evaluation of the quality of their
course, the extent to which graduates reported the development of leadership,
management and generic capabilities and, the impact the master’s programme
had on graduates’ subsequent careers. The participants in this study were
graduates who completed a MSc in Healthcare Management, MSc in Leadership
and Management Development or MSc in Quality and Safety in Healthcare
Management. These programmes have been delivered at RCSI since 2005 with
the first graduates in 2007. The first section of the evaluation reports on the
methods used to evaluate the masters’ programmes at the Institute of
Leadership. This is followed by the presentation of the results which includes:
the demographic profile of the sample, the results from the survey that measured
graduates’ experience of their programme of study, the extent to which
graduates made gains in the development of leadership capabilities and, finally,
the employment destination of graduates following the completion of the
programme.
1.2 Methods
The evaluation is principally concerned with measuring the quality of the
programme and the educational and professional outcomes achieved by
graduates as a consequence of the programme. In this study outcomes are
classified as short to medium term outcomes and longer-term impacts. Short to
medium term outcomes included students satisfaction with their programme of
study, academic achievement, the development of generic capabilities
(graduates’ ability to tackle unfamiliar problems, work as a member of a team,
3

develop problem-solving, analytical and written communication skills), the
development of graduate qualities (motivation toward lifelong learning, the
ability to value perspectives other than their own, the ability to investigate new
ideas), the impact of the educational programme on professional practice and
the development of leadership capabilities. The measurement of longer-term
outcomes included the impact of the programme on the graduates’ employment.
This section outlines the methods that were used to measure these outcomes.
1.2.1 Aims of the Evaluation
The aims of this evaluation were: 1) to evaluate the quality of the masters’
programmes delivered by the Institute of Leadership at RCSI and; 2) to measure
the outcomes that occurred in graduates following the completion of their
programme of study.
1.2.2 Sample
The participants in this study were graduates who completed a MSc in
Healthcare Management, MSc in Leadership and Management Development or
MSc in Quality and Safety in Healthcare Management from the Institute of
Leadership, Royal College of Surgeons in Ireland (RCSI). All graduates were
contacted by e-mail informing of them of the study and inviting them to
participate. Students who had deferred or did not complete their degree were
excluded from the survey. At the time of the study there were 303 graduates
from Dublin. 195 of these were surveyed for this study and the response was 102
(52.3%).
1.2.3 Measures
The following instruments and variables were to evaluate the outcomes
associated with the masters degrees: the Course Experience Questionnaire and
the Extended Course Experience Questionnaire, the Master’s Outcomes Evaluation
Questionnaire and sociodemographic variables.
1.2.3.1 Course Experience Questionnaire (CEQ) and the Extended Course
Experience Questionnaire (ECEQ)
4

Graduates’ experiences of their master’s programme were measured using both
the Course Experience Questionnaire (CEQ) and a number of scales that comprise
the Extended Course Experience Questionnaire (ECEQ). The purpose of the CEQ is
to evaluate graduates’ perceptions of the quality of the courses they completed at
college and the extent to which they perceive they have developed generic skills
(Ainley and Johnson 2000, McInnis et al. 2001). The value of the CEQ is that it
gives a broad perspective on outcomes by focusing on graduates’ perceptions of
their courses rather than on students’ evaluations of particular lecturers. The
CEQ also examines indicators of relative performance in higher education at both
system and institutional levels (McInnis et al. 2001). Theories of teaching and
learning informed the development of the CEQ (Ramsden 1991, Ainley and
Johnson 2000). Its main strength is that the instrument enables the researcher to
link students’ perceptions of the quality of a programme to learning outcomes. In
testing it has been shown to be a reliable and valid instrument. It has also been
claimed to be valuable in ‘improving the quality of teaching in universities and
also for informing student choice, managing institutional performance, and
promoting accountability of the higher education sector’ (McInnis et al. 2001, pg.
3). The CEQ instrument has undergone progressive refinement, development
and testing over the last twenty years culminating in the recent development of
an Extended Course Experience Questionnaire (Griffin et al. 2003).
In its original form the CEQ was found to be a reliable and valid instrument that
identified the quality of teaching in different academic departments and
institutions (Ramsden 1991). However, the original CEQ instrument was deemed
to be lacking in its ability to capture the wider aspects of the student experience
in higher education including the impact of a higher education programme on
student outcomes. Therefore to comprehensively capture the experience of
students at university, in addition to the current scales on the CEQ (Good
Teaching Scale; Clear Goals and Standards Scale; Appropriate Assessment Scale;
Appropriate workload Scale and Generic skills Scale), five further scales were
recommended to extend the instrument. These new scales included the Student
Support Scale, Learning Resources Scale, Learning Community Scale, Intellectual
5

Motivation Scale and Graduate Qualities Scale. These additional instruments
expanded the instrument from a 23-item questionnaire to a 50-item
questionnaire with the addition of five scales (see table 1).
Table 1 Defining Items of the CEQ and Extended CEQ Scales (Adapted from Ramsden 1991,
McInnis et al. 2001, Griffin et al. 2003)
Scale
Defining Item
† Good Teaching* Teaching staff here normally give helpful feedback on how you are going
† Clear Goals and
Standards*
† Appropriate
Workload*
† Appropriate
Assessment*
I usually had a clear idea of where I was going and what was expected of
me in this course.
The sheer volume of work to be got through in this course meant it
couldn't all be thoroughly comprehended.
The staff seemed more interested in testing what I had memorised than
what I had understood.
† Generic Skills* The course developed my problem solving skills
‡ Student Support
(5 items)
‡ Learning Resources
(5 items)
‡ Learning
Community*
(5 items)
‡ Intellectual
Motivation*
(4 items)
Relevant learning resources were accessible when I needed them
It was made clear what resources were available to help me learn
I felt I belonged to the university community
I found my studies intellectually stimulating
‡ Graduate Qualities* The course developed my confidence to investigate new ideas
(6 items)
† Original CEQ scales. ‡ Extended CEQ Scales *Scales used in this study
The purpose of using the CEQ in this study is to identify the quality of the core
aspects of teaching and learning that students experienced during their master’s
programme (Ainley & Johnson 2000). The CEQ is identified as being particularly
valid for the evaluation of coursework master’s programmes in healthcare
management and quality as students are exposed to a wide variety of subjects,
come into contact with a variety of lecturers and experience a variety of
educational experiences. The validity and utility of the CEQ has resulted in its
widespread use as an indicator of quality and student outcomes.
6

Students were asked to rate their level of agreement or disagreement on a fivepoint
scale on 39-items that comprised eight scales of the CEQ and the ECEQ. The
eight scales used in this study included: the Good Teaching Scale, Clear Goals and
Standards Scale, Appropriate Workload Scale, Generic Skills Scale, Learning
Community Scale, Intellectual Motivation Scale and the Graduate Quality Scale.
1.2.3.2 Master’s Outcomes Evaluation Questionnaire
Outcomes related to leadership and management were measured using an
instrument developed specifically for this evaluation: the Masters Outcomes
Evaluation Questionnaire (MOEQ). The items on the subscale were identified and
developed from an extensive review of the literature and course documentation
pertaining to leadership outcomes in healthcare management. The questionnaire
consisted of 31-items measuring four domains seen as relevant to leadership and
management practice: 1) ability to change professional practice (9 items); 2)
leadership capabilities (6 items); 3) communication and teamwork (6 items) and;
3) problem solving (10 items).
The MOEQ was designed to incorporate a retrospective pre-test design. Students
were asked firstly to rate their ability following their master’s programme (posttest)
and then to think back and rate their ability before the commencement of
their master’s programme (retrospective pre-test/then-test) on a seven-point
scale that was anchored by 1 = low understanding/ability to 7 = high
understanding/ability. The rationale for using this approach was to account for
the confounding factor of response-shift bias and to identify whether the
outcomes were being achieved as a consequence of the programme or were due
to other influence such as in-service education, further education, maturation or
employment (Drennan & Hyde 2008).
Table 2 Defining Items of the Masters in Nursing Outcomes Evaluation Questionnaire Scales
Scale
Defining Items
Change Professional
Ability to change and influence practice
Practice
Leadership
Communication and
Teamwork
Ability to use leadership theories to inform professional practice
Ability to communicate and work as a member of a team
7

Problem Solving
Ability to develop solutions to practice problems through inquiry
analysis and interpretation
1.2.3.3 Measurement of Sociodemographic, Professional and Educational Variables
To test for relationships with the dependent variables (outcome measures) a
number of Sociodemographic, professional and educational variables will also be
measured (see table 3).
Table 3 Sociodemographic, Professional and Educational Variables Measured
Sociodemographic Professional Variables Educational Variables
Variables
Age Current area of work Strand (Healthcare management/Quality)
Gender Grade (promotion) Academic qualifications
Site at which degree
completed
Employment during masters
(full-time/part-time/none)
Final award
Hours spent on study/writing (workload)
1.2.4 Procedure
The evaluation survey was delivered online using the web-based online survey
system, SurveyMonkey (www.surveymonkey.com). As in best practice with
surveys, participants were sent up to four reminders to complete the
questionnaire.
1.2.5 Ethical Issues
Access and ethical approval was granted from the ethics committees at RCSI.
1.2.6 Data Analysis
Data obtained was analysed by computer using the Statistical Package for the
Social Sciences (SPSS version 18.0). Both descriptive and inferential statistics
were used in the analysis and description of the data set. To ascertain the specific
differences between students ratings of their ability at the end of the course
(post-test) compared with their ability at the beginning of the course (then-test)
Wilcox on signed rank test was used.
8

1.3 Results
The demographic profile of the sample that completed the evaluation of the
programmes at RCSI in Dublin is outlined in table 4.
Table 4 Demographic, Academic and Professional Profile of Master’s Graduates
Characteristic
Age in Years M, (SD) 39.1 (8.0)
Gender (%)
Female 88.1
Male 11.9
Year Degree Completed (%)
2007 6.7
2008 6.7
2009 37.8
2010 22.2
2011 26.7
Strand Degree Completed (%)
MSc in Healthcare Management
MSc in Leadership & Management Development
MSc in Quality and Safety in Healthcare Management 6
63.8
29.8
1.3.1 Graduates’ Evaluation of the Quality of their Educational Programme
This section reports on the results of the Course Experience Questionnaire (CEQ)
and the Extended Course Experience Questionnaire, which were used to evaluate
graduates’ perceptions of the quality of the courses and the extent to which they
developed a number of generic and graduate capabilities. The results in this
section are reported in relation to students’ perceptions of the quality of
teaching, the extent to which they were provided with clear goals and standards,
the appropriateness of workload, the appropriateness of assessment, graduates’
overall satisfaction with the course and the development of generic skills. The
Extended Course Experience Questionnaire, used in conjunction with the CEQ,
measured graduates’ perceptions of their integration into the learning
community of the Institute, the extent to which they were intellectually
motivated by the programme of study and the development of graduate qualities.
The results are presented in relation to the individual items that comprise each
of the scales of the CEQ and Extended CEQ and then in relation to each of the
summated scale scores. To aid interpretation and standardise scores across the
CEQ and the Extended CEQ the mean item scores in the tables and figures are
based on a linear transformation where the 1 to 5 categories (strongly disagree
9

to strongly agree) have been recoded from -100 to +100. Positive values indicate
degrees of agreement and negative values indicate degrees of disagreement with
each of the items. For further ease of interpretation, overall agreement on each
item is reported by combining agree and strongly agree categories.
1.3.1.1 Graduates’ Overall Evaluation of their Programme of Study– Item Scores on
the CEQ
Graduates rated the development of generic capabilities as the greatest impact
and outcome of their master’s degree. Graduates reported that they had attained
a number of generic capabilities including the ability to communicate in writing,
the development of analytical skills, problem-solving, the ability to tackle
unfamiliar problems and their ability to work as a member of a team (table 5).
Students also reported that they attained relatively high levels of ability in
relation to their ability to plan their own work.
Graduates also highly rated the majority of their experiences of teaching, with
the vast majority in agreement that lecturers made their subjects interesting,
were good at explaining subjects; graduates also perceived that lecturers
motivated students to do their best work. Graduates were positive regarding the
feedback received from teaching staff on how they were doing on the
programme with the majority agreeing that staff made a real effort to
understand the difficulties they might be having with their work. Students also
reported that they were appropriately assessed and that the programme did not
emphasise lower order assessment skills such as factual recall and a reliance on
memory. The item ‘the sheer volume of work to be got through in this course
meant it couldn’t be thoroughly comprehended’ was the only negatively rated
item on the scale. This indicated there was variability in the extent to which
graduates perceived they had the time to assimilate the content of the course. In
addition the item ‘the workload was too heavy’, although positively rated, was
rated relatively lower than other aspects of the programme. However, in relation
to workload, graduates did perceive that they were given time to understand the
content of the course. Graduates also highly rated the organisation of the course
10

and all items related to the measurement of clear gals and expectations were
positively rated.
Percentage agreement was also used to summarise item responses by combining
agree and strongly agree categories into one overall agreement category (table
5). The highest level of agreement (86.7%) was related to the item: ‘The course
sharpened my analytical skills’. High levels of agreement were also associated
with other generic skill items including development of problem-solving skills
(82.2%), the development of capabilities in written communication (80.0%) and
the ability to tackle unfamiliar problems (68.8%). The quality of teaching was
also highly rated. Over eighty per cent of graduates reported that lecturers were
‘extremely good at explaining things’ and that they were motivated to do their
best work. Graduates were also in agreement that teaching staff made a real
effort to understand the difficulties they may having with their work (68.9%)
and that lecturers worked hard to make their subjects interesting (69.0%).
Graduates agreed or strongly agreed (63.8%) that teaching staff normally gave
helpful feedback on the work submitted.
Graduates strongly agreed that they were provided with clear goals and
expectations. In particular 82.3% of graduates were of the opinion that ‘the staff
made it clear what they expected from students’. Two thirds were also in
agreement that they had a clear idea where they were going and what was
expected from them on the course with approximately half of the respondents
reporting that they knew the standard of work expected.
Responses to the level of workload expected were variable. Generally, the
majority of students were happy with elements that measured workload
throughout the programme with over sixty per cent reporting that they were
provided with enough time to understand the things they had to learn. However,
seventy-five per cent reported that the volume of work meant it could not be
thoroughly comprehended. However, the vast majority of graduates did not
perceive that the ‘the workload was too heavy’. One item was used to measure
overall satisfaction with the quality of the programme of study; the vast majority
11

of graduates (66.0 %) were in agreement that they were satisfied with the
overall quality of their course.
12

Table 5 Item Scores* and Percentage Agreement † on the CEQ
CEQ Scale/Item Item Scores Percentage
Agreement †
Mean* SD
Good Teaching
The teaching staff normally gave me helpful feedback on how 55.5 28.6 63.8
I was doing
The staff made a real effort to understand the difficulties I 35.5 36.3 68.9
might be having with my work
My lecturers were extremely good at explaining things 44.4 34.0 84.4
The staff put a lot of time into commenting on my work 11.1 51.0 37.7
The teaching staff motivated me to do my best work 52.2 42.5 80.1
The teaching staff worked hard to make their subjects
38.8 43.8 69.0
interesting
Appropriate Workload
The workload was too heavy 7.8 38.3 22.2
The sheer volume of work to be got through in this course -44.3 36.2 75.0
meant it couldn’t be thoroughly comprehended
I was generally given enough time to understand the things I 32.2 47.8 62.2
had to learn
There was a lot of pressure on me to do well in this course 32.2 46.6 15.6
Clear Goals and Expectations
It was always easy to know the standard of work expected 11.1 49.8 48.8
I usually had clear idea of where I was going and what was 27.7 56.6 66.7
expected of me on this course
It was often hard to discover what was expected of me on 33.3 44.3 27.2
this course
The staff made it clear from the start what they expected 55.5 42.9 82.3
from students
Generic Skills
As a result of my course I feel confident about tackling
44.4 51.5 68.8
unfamiliar problems
The course helped me develop my ability to work as a team 42.2 46.4 64.5
member
The course developed my problem solving skills 47.7 49.8 82.2
The course sharpened my analytical skills 51.1 31.0 86.7
The course improved my skills in written communication 55.5 38.7 80.0
My course helped my ability to plan my own work 22.2 48.3 46.7
Appropriate Assessment
Too many staff asked me questions just about facts 6.66 22.8 26.7
To do well in this course all you really needed was a good 57.7 38.3 4.4
memory
The staff seemed more interested in testing what I had
51.1 48.4 8.9
memorised than what I had understood
Overall Satisfaction
Overall I was satisfied with the quality of this course 35.5 48.8 66.0
*Scores range from – 100 to + 100. Positive scores indicate levels of agreement; negative scores
indicate levels of disagreement.
† Overall percentage agreeing or strongly agreeing on a 5-point scale.
13

1.3.1.2 Student Experience of the Master’s Programmes – Item Scores on the
Extended CEQ
Analysis of the items that comprise the three extended CEQ scales was also
undertaken to identify the level of intellectual motivation reported by students,
the extent to which students reported that they were integrated into the learning
community of the Institute of Leadership and the extent to which they perceived
they had developed a number of graduate qualities (table 6). The highest level of
satisfaction related to the items that comprised the intellectual motivation and
the graduate qualities scales. In the intellectual motivation scale the highest item
scores related to the items: ‘I found the course motivating’, ‘I found my studies
intellectually stimulating’, ‘overall my university experience was worthwhile’. In
the graduate qualities scale the items: ‘I consider what I learned valuable for my
future’, ‘the course provided me with a broad overview of my field of knowledge’,
‘the course developed my confidence to investigate new ideas’ and, in particular,
‘I learned to apply the principles from this course to new situations’ were also
highly rated. Graduates also highly rated items related to the learning
community scale. In particular, graduates reported that they felt ‘part of a group
of students and staff committed to learning’ and that they felt they ‘belonged to
the university community’.
In relation to percentage agreement for each of the items (a combination of agree
and strongly agree responses), the highest percentage agreement related to the
items on the graduate qualities scales and intellectual motivation scales. The
item ‘Overall my university experience was worthwhile’ received the highest
level of agreement at approximately ninety-four per cent. Students were also in
agreement that the course was motivating, stimulating and worthwhile. Similar
levels of agreement were identified in the items that comprised the graduate
qualities scale. This scale measured a number of outcomes achieved as a
consequence of the programme of study. Over ninety-five per cent of graduates
reported that they had learned to apply the principles of the course to new
situations with ninety-two per cent reporting that they had developed the
confidence to investigate new ideas. These findings are further supported by
Joyce (2012) which suggest that education is influenced by the needs of
14

postgraduate students to acquire knowledge and skills for application back to
practice. The remainder of items that comprised the graduate qualities scale
achieved levels of agreement of seventy-eight per cent and greater.
Graduates also reported that they felt part of a learning community. Over
seventy per cent of respondents were in agreement that they felt they belonged
to the university community, that they learned to explore ideas confidently with
others and that they felt part of a group of students and staff committed to
learning. The majority of students also reported that their ideas and suggestions
were used throughout the programme. The only item that received a relatively
lower rating at 50% of respondents agreeing was ‘I learned to explore ideas
confidently with other people’
Table 6 Item Scores* and Percentage Agreement † on the Extended CEQ
CEQ Scale/Item Item Scores Percentage
Agreement †
Mean* SD
Intellectual Motivation
I found my studies intellectually stimulating 46.5 39.4 70.5
I found the course motivating 42.2 41.2 80.0
My course has stimulated my interest in my field of study 18.1 51.8 43.2
Overall my university experience was worthwhile 71.1 31.0 93.3
Learning Community
I was able to explore academic interests with staff and
students
41.8 46.2 74.4
I felt part of a group of students and staff committed to 40.9 30.7 75.0
learning
I learned to explore ideas confidently with other people 17.0 54.9 50.0
I felt I belonged to the university community 50.0 28.2 84.5
Students’ ideas and suggestions were used during the 50.1 36.9 82.2
course
Graduate Qualities
My university experience encouraged me to value
perspectives other than my own
47.7 41.2 77.7
I consider what I learned valuable for my future 54.4 39.6 82.2
The course developed my confidence to investigate new 62.2 40.1 91.1
ideas
I learned to apply the principles from this course to new 67.7 28.5 95.6
situations
The course provided me with a broad overview of my field
83.8
of knowledge
University stimulated my enthusiasm for further learning 52.2 31.9 82.2
*Scores range from – 100 to + 100. Positive scores indicate levels of agreement; negative scores
indicate levels of disagreement.
† Overall percentage agreeing or strongly agreeing on a 5-point scale.
1.3.1.3 Scale Responses to the Course Experience Questionnaire and the Extended
Course Experience Questionnaire
15

The items that comprise the CEQ and the Extended CEQ were summated into six
scales, which measured the overall student experience of teaching, workload,
goals and expectations, the development of generic skills, appropriate
assessment, intellectual motivation, belonging to a learning community and the
development of graduate qualities as part of their course experience. One single
item measured overall satisfaction with the programme of study (table 7).
Table 7 Mean Scores of the CEQ* and Extended CEQ* Scales
Scale Minimum Maximum Mean SD
Good Teaching -50.67 66.67 21.11 25.7
Appropriate Workload -37.50 37.50 7.31 21.45
Clear Goals and Expectations -62.50 100.00 31.94 38.11
Generic Skills -33.33 100.00 40.88 29.21
Appropriate Assessment -50.00 100.00 38.51 33.86
Intellectual Motivation -25.00 100.00 43.60 30.17
Learning Community -10.00 100.00 40.23 29.15
Graduate Qualities -25.00 100.00 56.85 29.37
Overall Satisfaction 0.00 100.00 71.11 31.05
*Scores range from – 100 to + 100. Positive scores indicate levels of agreement; negative scores
indicate levels of disagreement.
The mean scale scores identified that the development of graduate qualities (M =
56.85, SD = 29.37) was the highest outcome identified by students followed by
being intellectually motivated (M = 43.60, SD = 30.17). A high mean score on the
intellectual motivation scale indicated that graduates found their studies
intellectually stimulating. A high mean score on the graduate qualities scale
indicated that graduates developed the confidence to investigate new ideas and
the ability to value perspectives different from their own. The development of
generic skills (M = 40.88, SD = 29.21) and the extent to which graduates felt part
of a learning community (M = 40.23, SD = 29.15) were also positively rated. A
positive rating on generic skills scale indicated that graduates were of the
opinion that they had developed problem-solving and analytical skills as an
outcome of the programme of study; a positive rating on the learning community
scale indicated that students felt part of the academic environment. Although
the mean scores for the good teaching, clear goals and expectations and
16

appropriate assessment scales were rated relatively lower, graduates still
positively evaluated these aspects of their programme. No element of the student
experience received an overall negative rating, however, there was variability on
the scores related to the appropriate workload scale (mean = 7.31, SD = 21.45).
Although this aspect of the programme was positively rated, the overall score
was relatively low compared to graduates’ experience of other elements in the
course. Overall graduates were highly satisfied with the quality of their
programme with a very high overall satisfaction mean score of 71.11 (SD =
31.05).
1.3.3 Graduate Destination
The impact of the master’s degree on subsequent promotion was deemed by
graduates to be relatively influential with over a quarter of those who had
changed grade or been promoted to a higher grade since completing their
programme of study indicating that this had occurred as a result of holding a
master’s degree in healthcare management of quality and safety in healthcare
management (see figure 2).
17

Figure 2 Changed job as a consequence of the Master’s programme
1.4 Conclusion
The evaluation of the masters’ programmes at RCSI Dublin identified that the
experience of studying at master’s level was of a high quality. Graduates also
reported that they had developed a number of generic, management and
leadership and critical thinking capabilities and as a consequence of the
programme. Generic capabilities achieved included problem-solving skills,
analytical ability, the ability to plan work, and the ability to communicate at a
high level. Graduate capabilities included the confidence to investigate new ideas
and the ability to apply the principles of the course to new situations as well as a
commitment to lifelong learning. Graduates were positive about the organisation
of the programme, the standard of teaching, the feedback from the teachers and
the methods of assessment. Graduates also reported that they were involved in
the life of the College and that the programmes were intellectually stimulating.
In summary the programmes at RCSI, measured using reliable and valid
educational outcome instruments, were highly evaluated. Overall students
reported high levels of satisfaction with their programme of study and reported
that the theoretical and practical components of the programme have a positive
impact on their professional working lives.
18

References
Ainley J., Johnson T. (2000). Course Experience Questionnaire 2000: An Interim
Report Prepared for the Graduate Careers Council of Australia. Australian
Council for Educational Research, Canberra.
Drennan J., Hyde A. (2008) Controlling response shift bias: The use of the
retrospective pretest design in the evaluation of a master's programme.
Assessment and Evaluation in Higher Education, 33 (6):699-709.
Griffin P., Coates H., McInnis C., James R. (2003). The development of an
extended Course Experience Questionnaire. Quality in Higher Education, 9,
(3), 259-266.
Joyce P (2012) Learning as doing -common goals and interests across
management and education. Journal of Nursing Management 20(1): 113-119.
McInnis C., Griffin P., James R., Coates H. (2001). Development of the Course
Experience Questionnaire. Department of Education Training and Youth
Affairs. Canberra.
Ramsden P. (1991). A performance indicator of teaching quality in higher
education: The Course Experience Questionnaire. Studies in Higher
Education. 16, 129-150.
OoO
19