PSG FAIMER Regional Institute - PSG-FAIMER-2007 - Wikispaces

1
PSG FAIMER Regional Institute
Scholarly Report of ML Web discussion – November 2008
Topic : STUDENT ASSESSMENT
Group Fellows:
Shital Bhandary and Nirmala Thangavelu (2008 Fellows)
Elaleh Malakan Rad and V. Sreenivasulu Reddy (2007 Fellows)
Faculty Mentors: Vivek Saoji and B. Appla Raju
Submitted by : DR.V. Sreenivasulu Reddy, 2007 Fellow of PSG FAIMER

2
Acknowledgment
It is an acceptable conclusion that in our present setup most of the teaching Faculty in
Medical education are not trained properly in teaching and assessment technologies. This big
deficiency remained and increasing year after year with opening many new Medical Colleges.
In this scenario, in this part of our big country, PSG FAIMER programme was started and it is an
extremely useful gift by FAIMR to our Faculty in Medical education. It is like finding water pond
in a Desert which gives happiness/satisfaction that has to be experienced and cannot be
explained. Thanks to FAIMER for this gift.
PSG FAIMER regional institute Fellowship programme is an excellent programme for
building the knowledge, skills, attitude of the teachers in Medical Education.
I congratulate Shital and Nirmala for contributing and conducting the discussion on
this topic at an elaborate manner. I thank all 2007 fellows and 2008 fellows, for their active
participation in the discussions. I thank all Mentors, FAIMER/PSG FAIMER Faculty, Dr Thomas
chacko, and Dr. Ramalingam for their valuable guidance and knowledgeable inputs into the
discussion. I congratulate Dr.Elaheh (my batch mate for ML web discussion) for submitting an
exhaustive review of literature and some important issues in the form of 60 questions and
answers in her report.
I am submitting this report on the assessment of students using all the discussions,
suggestions, remarks, references, from the month long (November 2008) discussion on this
topic and resources available in Internet. I sincerely thank all the concerned in this regard.
In this Fellow ship training I learned very much and I am greatly indebted to all Fellows,
faculties at PSG‐FAIMER and FAIMER.
My stay, study, and attachment with PSG during this Fellow ship is an unforgettable
experience.

3
Index
S. No Contents Page Number
1 Introduction 4
2 Methodology 6
3 Results 7
4 Discussion 8
5 References 15

4
Introduction:
In the past training of a Physician used to be a small group attached to
teacher/practitioner for long periods during which trainees were trained and observed
(evaluated) by the teacher and allowed to practice. Even then as it was small number of
physicians, all those practicing after their formal training were under continuous evaluation of
their teachers. It was Guru – shishya parampara.
In modern age due to increased requirement of Doctors many modern Medical Schools
were started with large number of intake, taught and evaluated by different teachers and
allowed to practice independently sometimes at distant places/communities and hence no
chance for teachers to watch the performance of their students once they pass out of their
Medical institutions. So in the modern times the assessment of students must be sound and
comprehensive before allowing them to practice medicine. Further with increased usage of
different kinds of instruments, Electronic Machinery in Medical practice, the need came to
change the methods of teaching and training in Medical Schools and so the need to look for
new and effective assessment methods. With more research in psychology and learning skills
teaching methods also changed from simple Lectures and practical’s to more and more of
Problem solving, knowledge application skills and other innovative teaching‐learning methods.
In this scenario student assessment (evaluation) has become very important and every teacher
must know and practice scientific, standard, relevant valid student assessment protocols to
produce competent Doctors and to protect the community from the danger of ill trained/half
trained Doctors.
Proper assessment will also reveal the effectiveness/lacuna in the teaching methods
and give us a chance to rectify defects and to improve teaching. Sound method of assessment
also reveal any weak/slow learners and we can take remedial measures. Hence proper
assessment of the students is very important in the Medical Education.
In the modern world ,Educational assessment is defined as the process of documenting, usually
in measurable terms, knowledge, skills, attitudes and beliefs. Assessment can focus on the individual
learner, the learning community (class, workshop, or other organized group of learners), the institution,
or the educational system as a whole.

5
Assessment is often divided into summative and formative categories for the purpose of
considering different objectives for assessment practices.
• Summative assessment ‐ Summative assessment is generally carried out at the end of a
course or project. In an educational setting, summative assessments are typically used
to assign students a course grade. Summative assessments are evaluative.
• Formative assessment ‐ Formative assessment is generally carried out throughout a
course or project. Formative assessment, also referred to as "educative assessment," is
used to aid learning. In an educational setting, formative assessment might be a teacher
(or peer) or the learner, providing feedback on a student's work, and would not
necessarily be used for grading purposes. Formative assessments are diagnostic.
Educational researcher Robert Stake explains the difference between formative and
summative assessment with the following analogy in simple language:
‘’When the cook tastes the soup, that's formative. When the guests taste the soup,
that's summative’’.
Summative and formative assessment are often referred to in a learning context as
‘’assessment of learning’’ and ‘’assessment for learning’’ respectively.
‘’Assessment of learning’’ is generally summative in nature and intended to measure learning
outcomes and report those outcomes to students, parents, and administrators. Assessment of
learning generally occurs at the conclusion of a class, course, semester, or academic year.
‘’Assessment for learning’’ is generally formative in nature and is used by teachers to consider
approaches to teaching and next steps for individual learners and the class.
In view of importance of student assessment in Medical education PSG FAIMER program
included student assessment in Web discussion in each year and for 2008 batch it was allotted
in November 2008 for this topic.

6
Methodology:
In order to conduct the web discussion in an orderly and organized format, it was decided
to conduct the web discussion on this topic of Student assessment, in four main Sub areas
which are mentioned below in four weeks during November 2008:
1‐ Principles and methods of assessment in Medical education
2‐ Assessment of knowledge
3‐ Assessment of skills
4‐ Assessment of attitude
At the outset I apologize for not active on web ML discussions. However I was reading all
the discussions that were going on. I appreciate and congratulate 2008 fellows of this group
Shital and Nirmala for their active contributions.
The discussion was started by Shital Bhandary and very well coordinated and contributed
by Shital Bhandary and Nirmala Thangavelu. Many knowledgeable fellows of 20007 and 2008
participated, contributed in the discussion. Learned faculty mentors Vivek Saoji and B Appla
Raju and Faculty in PSG FAIMER also contributed with many inputs. Dr.Saira being well
experienced/exposed to this topic in 2007 discussion, posted some worthy inputs and
cautioned to concentrate on core issue of the topic.
Dr.Thomas Chako our beloved super Mentor intervened strategically in key areas to take
the discussion to more knowledgeable experience. The Dean of PSG Dr. Ramalingam nicely
introduced Pygmalion effect. Thus this topic discussion was very interesting and knowledge
gaining experience to all Fellows.

7
Results:
At the end of the month long discussion, the results/observations are:
• assessment of Students knowledge and skills will be comprehensive by using modified
MCQs, MEQs, modified examination system of long case and short case, OSCE/OSPE and
Assessment of attitude 360 degree evaluation, portfolios, P‐CEX (Professionalism mini
evaluation exercise).
• Valid and reliable assessment of knowledge, skills, attitude of the student, will be
achieved by using/selecting appropriate protocols from the available tools of
assessment.
Nirmala and Shital conducted one questionnaire survey asking the Fellows to answer on
the need and on the attributes attitude assessment.
The survey was answered by twenty two respondents. Among these respondents two did
not gave their opinions. The results were analyzed is based on twenty responses. On the basis
of summary of the results most of the respondents were in favor of having attitudinal
assessments. Among the respondents 36% strongly agreed and 50% agreed on the need of
having attitudinal assessment.
Another point of consensus that emerged in the discussions was that teachers must have
sound knowledge and competency in different assessment tools and techniques. For this
teachers must be trained and exposed to all scientific methods/tools of assessment, so that
teachers will decide/select appropriate tools for assessment from among the various available
tools. Teachers must plan/prepare blue print before conducting the assessment.

8
Discussion:
I am presenting this on the basis of discussions on ‘’Assessment of Students’’ that were
conducted in ML‐WEB discussion during November 2008 at PSG FAIMER.
1 st week discussion: Principles and methods of assessment:
As Shital started discussion, Nirmala pointed out the difference between evaluation and
assessment. She explained that Assessment is the measurement of a particular skill/
characteristic of an individual. Whereas evaluation is a comprehensive form of assessment that
is an individual is assessed for many/different characteristics. Hence evaluation is an overall
assessment of putting together all assessments. So evaluation is more agreeable term where
student is tested/assessed as a whole.
Nirmala also described
emphasized that.
assessment Tips/assessment manifesto in which it was
* assessment must point where the student is lagging and help to improve.
* Student must be encouraged to question about their result of assessment.
* to make assessment comprehensive by using variety of assessment tools/techniques.
* Faculty must have sound knowledge and competency in different assessment tools/
techniques.
* wrong usage of assessment to0ls/techniques must be prevented.
Saira added that it is good to make students know that they will be assessed on such
and such knowledge/skill etc., before planning for assessment. Sundar gave some principles
and rules in assessment to be followed where in he focused on important point that
assessment must be part of effective teaching and learning plan.
Barani gave a beautiful link in which Professor C.Y. Kwan from Department of Medicine,
Faculty of Health Sciences McMaster University, Hamilton, Ontario, Canada had explained the
purposes of Student assessment as:
1. To differentiate high and low performers,
2. To provide feedback on students’ learning,
3. help teachers improve their teaching,
4. To decide which students can advance further (be rewarded), and which need to undergo
remedial training/be detained (be punished).
Shital posted many links and references in this area and many fellows participated, posted
more comments, references and discussions in this topic. Then Shital posted a summary from
the Institute for International Medical Education (IIME) on Assessment Methods and
Measurement Instruments.

9
Good, worthy discussion went on in the week focusing,
1.Why should we have assessment in medical education?
2. What are the principles of student assessment in medical education?
3. What are the rationales behind student assessment in medical education?
4. What are the tools and methods available for student assessment in medical education?
It was pointed that in the students' assessment, one most important task is to select
suitable method of assessment. It is because different assessment methods may fail when we
look at reliability and validity, So it is better to use different methods/approaches for
assessment.
Sheetal posted a link and pointed that after looking at different methods/ tools of
assessment, the teacher must work on Blue printing (based on Bloom's Taxonomy of
Objectives. Taxonomy of Bloom defines the three levels of Domains‐ Cognitive, Affective and
Psychomotor) and Standard settings to make assessment more scientific, valid and reliable. It
was emphasised in the discussion that it is a good practice to using a mixture of different
assesment tools/ methods. In the discussion it was noted that Saira report on assessment in
2007 is worth to refer again regarding the standard setting in the Modified Angoff method.
Chitra raised the issue of attitude assessment and stressed that this must be included in the
assessment protocols.
During these discussions PSG Dean Dr. Ramalingam brought a different type of
performance enhancer effect known as “Pygmalion effect” in to the discussion. Shital, Supten,
Feroze actively discussed on “Pygmalion effect”
Salient points at the end of 1 st week after discussion on Principles and methods of assessment:
1. Valid and reliable assessment of Students will helps students to know their lapses and
concentrate to improve themselves and also will help Faculty to know on any lacuna if
any in their teaching and improve in their teaching methodologies.
2. It is to be remembered that as for as possible student assessment must be continuous
(not one year end Exam) process and the assessment tools must relevant, feasible, cost
effective.
3. Assessment must aim for formative and summative assessment.
4. we must choose appropriate assessment tools from the available assessment tools and
methods.
2 nd week discussion: Assessment of knowledge
In the Second week Shital started the discussion on assessment of knowledge . He put up
a query, that expected/accepted minimal level of knowledge that a student must have is
always under discussion specially among psychometricians. Shital described that during

10
assessment of knowledge, it should not be only for memorization (mugging part) but it should
look for comprehension and applied aspect.
Feroze said some sort of mugging and memorizing cannot be avoided in Medical
education. Shital commented that it may be fine to some extent and stressed that students
must be assessed more for their critical thinking and integrating capacity rather than the
memorization of the isolated facts.
It was felt that Faculty may be ill equipped to prepare suitable assements protocols/
questions for testing higher level cognitive knowledge and this may be addressed by providing
training to Faculty. Shital further stressed that teaching methods must be reoriented towards
integrated teaching to make students as self driven learners.
It was rightly pointed out by M.C.Das that at present in our Medical system at MBBS
level, we are assessing mainly memory/recalling (mugging) aspect. He further said that because
of this students are not equipped with knowledge of applied aspect and students are not
recollecting /correlating previous semester subject in next semester. This is because all through
we are assessing using only memorizing/recalling type questions. He suggested that questions
must constructed to test application of knowledge. Many Fellows shared their views and
participated in the discussion.
This week discussion on assessing knowledge ended with a consensus that students
assessment must stress on “Knows how’’ instead of ‘’know’’.
Salient points at the end of 2 nd week after discussion on assessment of knowledge:
1. Assessment of knowledge is important but the way it is done (assessed) traditionally at
present might not be as objective and objective orientated as it should be.
2. Knowledge as objective and objective orientated can be assessed using MCQs, MEQs.
3. MCQs can be constructed in such a way that it could assess the comprehension,
application, analysis, synthesis and evaluation levels of cognitive domains instead of just
the isolated facts. However, it requires trained faculty, resources and time.
4. Modified Essay Questions/MEQs (with Model Answers) are objective and assesses what
is usually not feasible through MCQs.
5. Depending on individual College/university policy the above described assessing tools
may be used for assessing knowledge in more objective , objective oriented way.

11
3 rd week discussion: Assessment of Skills
Shital started the discussion and requested all to contribute on assessment of skills.
He noted that as per Miller's philosophy there are four levels of skills are to be considered while
planning student assessment on skills:
a) Know (Knowledge)
b) Knows How (Competence ‐ Knowledge)
c) Shows (Competence Assessment ‐ Skills)
d) Does (Performance Assessment ‐ Skills).
Shital reminded that various methods have been developed to assess each of these levels
of knowledge and skills. we have already discussed the first two levels in 2 nd week and now in
this week we must discuss on next two levels in this week.
Shital attached a reference that he uses most in his presentations to illustrate these
points and posted document from BMJ on OSCE and skill based assessment in Medical
Education. Padma explained that she was involved and trained in OSCE and they were
conducting as per just referred BMJ document.
Shital added that he was involved in modified OSCE as per Prof. Sam Leinster of
University of East Anglia, UK . Barani raised a point that assessment of communication and
interpersonal skills is also to be kept in mind in assessment of Skills of a student.
Shital further explained that many tools like Mini clinical Examination, Long cases,
Short cases, log books, portfolios, 360 degree evaluation etc., are available( he gave all the Web
links) and institution may choose any few from these tools for assessment of skills (Shows ‐
Competence Skills and Does ‐ Performance Skills).
Feroze, Nirmala, Animesh, Amol, Saira, Chitra, Ashwini and other Fellows participated in
the discussion. It was felt that by using a mixture of assessment tools, it will allow the assessor
to have comprehensive assessment (formative, summative, continuous assessment) and this
will prepare Doctors with sound competency and social responsibility.
Shital, Nirmala, Supten suggested one very good book ‘’Practical Guide to Medical
student assessment’’ authored by Zubair Amin published by World scientific, Singapore which
will help Faculty in preparing various assessment protocols.
Shital further pointed that OSPE/OSCEs are excellent in summative (objective)
assements. He also told that from National board of Examination New Delhi Dr. Batra B and
Agrawal A., presented paper at the 5th APMEC at NUS, on "Comparative Evaluation of OSCE as
a tool for summative assessment with conventional (oral) examination" and concluded that
OSCE is very useful in summative assessment of comprehensive skills and also much useful in
giving uniformity of assessment in examinations held in various settings at different locations.

12
Salient points at the end of 3 rd week after discussion on assessment of Skills :
1. Assessment of Skills demands students to “show” what have been learnt and do what
have been learnt.
2. Mini Clinical Examination are very useful to assess the procedural skills.
3. Even though assessment of skills using long cases and short cases is done traditionally
and may be use full, it is now considered that OSCE is the better and preferred method
for assessing skills in objective way. OSCE are rated high in reliability and validity.
4 th week discussion: Assessment of attitude:
Shital initiated the discussion on assessing attitudes as final sub topic of November 2008
topic of student assessment. He mentioned that discussion must concentrated on
• Principles of assessment of attitude,
• Tools and methods of assessment of attitude,
• Need for assessment of attitude in Medical colleges.
Shital and Nirmala has already given a questionnaire on assessment of student attitude
and it was answered by some Fellows. Shital further pointed that in Medical Education student
assessment, it was recommended to asses in three domains (KSA) such as K (Knowledge),
S (Psychomotor Skills) and A (Attitudes).
For the assessment blue print preparation, the objectives described in the curriculum are
the guidelines. They will help in preparing tools/methods for student assessment.
However in many Medical colleges curriculum, objectives pertaining to attitude are not
described and that is causing difficulties in assessment planning and thus leading to failure to
assess the attitudes while assessing the students.
Hence in traditional Medical Education assessment protocols, assessment of Knowledge
and skills are taken care but assessment of attitudes is neglected.
Shital gave some important references on assessment of attitudes and many Fellows
discussed. Dr.Thomas pointed that assessment of attitude as part of assessment of Medical
student is very much essential. He further pointed that most of the Medical teaching Faculty
are keeping away from the attitude assessment because of some reasons like
a. lack of information ,definition
b. lack of objectives in the Curriculum attitude assessment.

13
He also pointed that most of the Faculty may not have required expertise in designing and
implementing methods of assessment of attitudes. Shital informed that after looking at
objectives in the curriculum of some Medical colleges in different countries, he found that one
from RAK Medical and Health Sciences University of UAE is the best. In this, they have
listed the objectives for the Knowledge, Skills and the Attitude in the basic as well as clinical
sciences. It also contains the proposed assessment schemes for the mentioned objectives.
Dr. Thomas Chacko stressed that in our Medical curriculum also it is essential to add
objectives on attitude and assessment of students on attitude must be taken up on regular
basis as an integral part of student assessment. Dr. Thomas Chacko posted some reference on
this from ACGME.
Komal pointed that after looking at this reference, she felt their assessment protocols
were excellent, which contain assessment methods on attitude in addition to Knowledge and
skills. In their setup, their students are assessed for attitudes by Faculty, by peers, by patients
thus assuring that the Doctors coming out of their assessment are humane and compassionate
Doctors.
Ravi posted one reference from Dr. Huw’s assessment of attitudes and it was well
appreciated by many Fellows.
After the survey on assessment of attitudes Shital and Nirmala made the final list of
some (must) criteria of attitudes in the assessment of attitude among the Medical student
under sub headings of
• Professionalism
• Interpersonal relationship
• Personal attributes
Shital pointed that the focus on assessment of Professionalism in Medical education is
gaining importance. Shital shared one report which was an outcome from the conference of
AAMC and NBME titled "Embedding Professionalism in Medical Education’’.
Salient points at the end of 4 th week after discussion on assessment of attitude :
1. Assessment of attitudes is one neglected area but this is much‐needed area to discuss
and act. So that Assessment of attitudes is also considered as must along with
assessment of Knowledge and skills.
2. There needs to be a planned learning experience on right and suitable attitude in a
Medical student (would be Doctor) incorporated in the curriculum, so that the required
attitude is also inculcated among the learners. Then they need to be assessed.

14
3. Medical students in addition to cognitive skills also should be assessed for
interpersonal communication skills, professionalism, humanism etc., not only by
concerned faculty but also by peers, patients and team members etc, to create
doctors with right attitude.
4. Assessment of professionalism is also very important and it can be done via 360 degree
evaluation, portfolios or new technique like P‐CEX (Professionalism mini
evaluation exercise).
The discussion on student assessment was concluded with shital quoting one advice from wise
men.
‘’keep away teaching pure knowledge and give stress on integrated knowledge with skills’’.
Conclusions:
As, I confessed and self assessed in the beginning that I remained as passive participant in
the discussion, I submit that after looking at and after reading at this huge lots of pages in the
web, contributed /referred in this exhaustive discussion on student assessment, I really became
a trained assessor. Thanks to FAIMER/PSG FAIMER programme.

15
References:
1. http://psg‐faimer‐2007.wikispaces.com/file/view/MLWeb Dialogue
2. http://www.qca.org.uk 500 Tips on Assessment. Brown, S.; Race, P.; Smith, B. (1996).
3. http://www.qca.org.uk/qca_4334.aspx and
4. http://www.qca.org.uk/qca_5067.aspx
5. The Internet Journal of Medical Education 1 (1). Vergis A, Hardy K (2010).
6. Moskal, Barbara M., & Leydens, Jon A ,2000
7. Joint Committee on Standards for Educational Evaluation. (1988). "The Personnel
Evaluation Standards: How to Assess Systems for Evaluating Educators." Newbury Park,
CA: Sage Publications.
8. Committee on Standards for Educational Evaluation. (2003). The Student Evaluation
Standards: How to Improve Evaluations of Students. Newbury Park, CA: Corwin Press.
9. https://learning.dcollege.net/webct/urw/lc5116011.tp0/cobaltMainFrame.dowebct
10. Practical Guide to Medical Student Assessment by Dr Zubair Amin
11. AMEE Medical Education Guide No. 18, 24,25
12. http://www.worldscibooks.com/medsci/6109.html
13. http://uk.centraltest.com/index/slrh.php?infoPAGE=SLRH!trial
14. http://www.worldscibooks.com/medsci/etextbook/6109/6109_chap01.pdf
15. BMJ 1975.Continuous assessment BMJ Cruickshank et al 1975.
16. http://www.cdtl.nus.edu.sg/link/mar2001/assess1.htm Professor C.Y. Kwan
17. http://www.ncbi.nlm.nih.gov Pub Med shows about 6 articles.
18. 20. http://ag.arizona.edu/fcs/cyfernet/cyfar/Portfo~3.htm
19. http://www.teachervision.fen.com/assessment/teaching‐methods/20153.html
20. 3. http://www.teachervision.fen.com/assessment/resource/5942.html?detoured=1
21. http://www.indiana.edu/~reading/ieo/bibs/portfoli.html
22. http://library2.usask.ca/theses/available/etd‐04252008‐
23. http://www.iime.org/documents/elo.htm
24. http://www.biomedcentral.com/content/pdf/1472‐6920‐5‐39.pdf) to find the
25. http://www.skagitwatershed.org/~donclark/hrd/bloom.html
26. http://en.wikipedia.org/wiki/Pygmalion_effect
27. http://www.aftau.org/site/News2?page=NewsArticle&id=6927
28. http://sciencethatmatters.com/archives/category/education
29. http://www.psy.gla.ac.uk/~steve/hawth.html
30. http://ezinearticles.com/?The‐Pygmalion‐Effect&id=86460
31. http://www.envisionsoftware.com/articles/Pygmalion_Effect.html
32. http://www.accomplishlife.com/blogs/8/The‐Pygmalion‐Effect.html
33. http://www.encyclopedia.com/doc/1O87‐Pygmalioneffect.html
34. http://www.yale.edu/ynhti/curriculum/units/1985/2/85.02.06.x.html
35. http://www.nbme.org/publications/item‐writing‐manual‐preface.html
36. http://student.bmj.com/issues/00/10/education
37. http://jama.ama‐assn.org