2005 SAGES Abstracts

POSTER ABSTRACTS
subjects had used a computer generated laparoscopic simulator
in the past. Subjects were given one practice session on
the LapSim® tutorial and dissection module and were supervised
throughout the testing. Instrument motion, time, and
errors were recorded by the LapSim®. A Performance Score
(PS) was calculated using the sum of total errors and time to
task completion. A Relative Efficiency Score (RES) was calculated
using the sum of the path lengths and angular path
lengths for each hand expressed as a ratio of the subject?s
score to the worst score achieved among the subjects. Thus, a
lower PS and RES indicated better performance. All groups
were compared using the Kruskal-Wallis and Mann-Whitney U-
test.
Novices achieved better PS and/or RES in Instrument
Navigation, Suturing, and Dissection (p250
procedures) consultant surgeons were asked to complete the
task breakdown pre-operatively, regarding the tools used and
method employed, for tasks in an ?ideal? procedure. The selfassessment
was then compared with the surgeons? actual
operations, which had been recorded onto DVD after the task
analysis was done, and analyzed for tool use and method
used, by 2 independent observers.
Results
All patients were between ASA 1-2, BMI < 30 and had clinicopathological
grade 1-2 gallbladders. Inter-rater reliability task
analysis was kappa = 0.77. The concordance between surgeons?
perceived and actual styles varied from 73-91%. The
majority of discordance occurred in non-technical tasks, as
opposed to technical actions. Surgeons were unable to
describe the manner in which they used some specific tools.
Conclusions
Surgeon self-evaluation is accurate for technical skills aspects
of the procedure, but is not sufficiently detailed to allow selfevaluation
of technical styles. Surgeon self-evaluation is poor
in non-technical aspects of the procedure. This study demonstrates
that self-appraisal using hierarchical task analysis is
feasible, accurate and practical. We aim to increase the numbers
in our study and recruit also resident surgeons.
P168–Education/Outcomes
CONSTRUCT VALIDITY OF ASSESSORS OF STRUCTURED
SURGICAL TECHNICAL SKILLS ASSESSMENT IN LAPARO-
SCOPIC SURGERY, Sudip K Sarker MD, Avril Chang
MD,Charles Vincent PhD,Ara W Darzi MD, Department of
Surgical Oncology & Technology, Imperial College London, UK
Objectives
Objective structured assessment of technical skills in live operations
removes subjectivity and bias. To date these assessments
have been done blindly and independently, but require
experienced surgeons and are time consuming. We aim in this
present to evaluate novice assessors, with and without surgical
experience, can assess technical skills in live laparoscopic
surgery.
Methods
Two full length versions of laparoscopic cholecystectomies
(LC) performed by two attending/consultant surgeons were
digitally recorded and converted to DVD. A Likert scale for
generic and procedure specific technical skill aspects of LC
were devised. LC were assessed by two experienced surgeons
and then assessed by two groups of 15 clinical medical students
and 15 junior surgical residents respectively. The two
groups assessed the LC without specific instruction but only
using the Likert scales devised.
Results
Inter-rater reliability between the 2 experienced surgeons was
kappa = 0.93, p< 0.05. Kappa coefficient between the 2 experienced
surgical assessors and the medical students collectively
was k = 0.37, p> 0.05 and for the surgical residents collectively
was k = 0.62, p> 0.05. Mann-Whitney test for construct validity
was significant p < 0.05 for the groups (experience surgeons,
junior residents, medical students).
Conclusions
Our study shows that novice technical skills assessors with or
without any surgical experience can not assess live laparoscopic
operations. We aim in the future to assess if surgeons,
with varying surgical experience and specified training in technical
skills assessment can assess technical skills in live open
and laparoscopic surgery.
P169–Education/Outcomes
DEVELOPMENT OF A STRUCTURED GLOBAL TECHNICAL
SKILLS ASSESSMENT TOOL IN OPEN & LAPAROSCOPIC
SURGERY, Sudip K Sarker MD, Avril Chang MD,Charles
Vincent PhD,Ara W Darzi MD, Department of Surgical
Oncology & Technology, Imperial College London, UK
Objectives
Assessing live surgery using objective and structured methodology
is still in its infancy. Assessing live operations in such a
way removes bias and subjectivity and is a fairer assessment
of technical performance of surgeons. We assess a global
assessment tool for technical skills in open and laparoscopic
surgery performed by attending/consultant surgeons and
trainees.
Methods
A global assessment for primary inguinal hernia repair (IH)
and laparoscopic cholecystectomy (LC) using generic and procedure
specific scales for each operation were devised. All
operations were recorded in their entirety and converted to
DVD. Two experienced surgeons assessed the full length operations
on DVD blindly and independently.
Results
All patients were between ASA 1-2, BMI < 30, < 75 years old.
They were all discharged the same or next day. There were no
post-operative complications.
All IH were primary repairs and LC were grade 1-2. 60 live procedures
were assessed (30 IH and 30 LC). 19/30 IH were performed
by consultants, 11/30 performed by trainees. 22/30 LC
were performed by consultants, 8/30 performed by trainees.
Using Mann-Whitney comparing the generic and procedure
specific scores for IH between consultants and trainees was
significant, p = < 0.05, and for LC generic and procedure specific
scores was also significant, p = < 0.05.
Conclusions
SAGES 2005
http://www.sages.org/
169