Experience with Virtual Reality-Based Technology in Teaching ...
Experience with Virtual Reality-Based Technology in Teaching ...
Experience with Virtual Reality-Based Technology in Teaching ...
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
<strong>Experience</strong> <strong>with</strong> <strong>Virtual</strong> <strong>Reality</strong>-<strong>Based</strong><br />
<strong>Technology</strong> <strong>in</strong> Teach<strong>in</strong>g Restorative<br />
Dental Procedures<br />
Judith A. Buchanan, Ph.D., D.M.D.<br />
Abstract: This article reports on extensive experience <strong>with</strong> advanced simulation at the University of Pennsylvania, School of<br />
Dental Medic<strong>in</strong>e (UPSDM). <strong>Virtual</strong> reality-based technology (VRBT) or advanced simulation is currently available for the<br />
<strong>in</strong>struction of dental students <strong>in</strong> precl<strong>in</strong>ical restorative procedures. UPSDM was one of the first schools <strong>in</strong> the world to have<br />
extensive experience <strong>with</strong> VRBT technology us<strong>in</strong>g an advanced simulation unit (DentSim) from DenX, Ltd. UPSDM’s experience<br />
consists of several years of research us<strong>in</strong>g control and experimental groups, employ<strong>in</strong>g students to participate <strong>in</strong> an<br />
<strong>in</strong>vestigative project, and us<strong>in</strong>g the units for remediation and a supplement to the precl<strong>in</strong>ical laboratory. Currently, all first-year<br />
students (Class of 2007 and Class of 2008) are receiv<strong>in</strong>g most of their preparative operative tra<strong>in</strong><strong>in</strong>g on the VRBT units. UPSDM<br />
started <strong>with</strong> one (beta) version unit <strong>in</strong> 1998, which was later updated and expanded first to four units and recently to fifteen units.<br />
This communication is present<strong>in</strong>g the studies that were of fundamental importance <strong>in</strong> mak<strong>in</strong>g the decision to acquire fifteen units<br />
<strong>in</strong> 2003. The areas of ma<strong>in</strong> <strong>in</strong>terest to the SDM concern<strong>in</strong>g this technology were its use <strong>in</strong> teach<strong>in</strong>g, refresh<strong>in</strong>g, and remediat<strong>in</strong>g<br />
students <strong>in</strong> restorative procedures and its effectiveness as a teach<strong>in</strong>g methodology <strong>in</strong> relation to time, efficiency, and faculty.<br />
Several studies <strong>with</strong> vary<strong>in</strong>g parameters were performed at various time po<strong>in</strong>ts. The limited statistical analysis conducted was not<br />
conclusive for all measures, and <strong>in</strong> some cases the data only suggest areas of possible significance. This is due to the low number<br />
of students who could access the limited number of available units and the change of protocols <strong>in</strong> response to student and faculty<br />
<strong>in</strong>put. Overall, the results do suggest, however, that students learn faster, arrive at the same level of performance, accomplish<br />
more practice procedures per hour, and request more evaluations per procedure or per hour than <strong>in</strong> our traditional laboratories.<br />
Students’ attitudes, as measured by surveys, group <strong>in</strong>terviews, and private <strong>in</strong>terviews, are mixed. Our overall evaluation of our<br />
experience <strong>with</strong> this technology has been positive and led to the purchase of additional units, its full <strong>in</strong>corporation <strong>in</strong>to our<br />
curriculum, and curriculum revision to maximize its potential. Our conclusion is that this technology offers significant potential<br />
<strong>in</strong> the field of dental education and that further use and <strong>in</strong>vestigation are both desired and justified.<br />
Dr. Buchanan is Associate Dean, Academic Affairs, Department of Preventive and Restorative Sciences, University of Pennsylvania,<br />
School of Dental Medic<strong>in</strong>e. Direct correspondence and requests for repr<strong>in</strong>ts to Dr. Buchanan at the Office of Academic<br />
Affairs, University of Pennsylvania School of Dental Medic<strong>in</strong>e, 240 S. 40 th Street, Philadelphia, PA 19104-6003; 215-898-1167<br />
phone; 215-573-4761 fax; Jbuchana@pobox.upenn.edu.<br />
This project was funded by a grant from the University of Pennsylvania Research Foundation.<br />
Key words: virtual reality, dental restorative procedures, dental education, technology, computer-assisted <strong>in</strong>struction<br />
Submitted for publication 8/12/04; accepted 10/19/04<br />
Rapid growth has occurred <strong>in</strong> higher education<br />
<strong>in</strong> the areas of distance learn<strong>in</strong>g, <strong>in</strong>teractive<br />
telecommunications, computerassisted<br />
<strong>in</strong>struction, and computer simulations. 1-4 The<br />
use of simulation has been proposed as the next major<br />
step <strong>in</strong> the evolution of health science education 5-<br />
9<br />
although dentistry has been us<strong>in</strong>g various types of<br />
simulation for quite some time. The amaz<strong>in</strong>g growth<br />
of computer hardware and software has led to the<br />
development of virtual worlds that support the field<br />
of advanced simulation. <strong>Virtual</strong> reality creates virtual<br />
worlds us<strong>in</strong>g mathematical models and computer<br />
programs, allow<strong>in</strong>g users to move <strong>in</strong> the created virtual<br />
world <strong>in</strong> a way similar to real life. Although used<br />
extensively for some time <strong>in</strong> other fields such as aeronautical<br />
tra<strong>in</strong><strong>in</strong>g, the technology of virtual reality has<br />
only been recently proposed as a powerful tool for<br />
the education of health professionals. 7,10-22 Some experts<br />
have referred to this technology as the “third<br />
dimension” 4 or the “next generation” learn<strong>in</strong>g environment<br />
<strong>in</strong> medic<strong>in</strong>e. 16<br />
Dental education, of all the health professional<br />
schools, is possibly the discipl<strong>in</strong>e that could most<br />
benefit from virtual reality s<strong>in</strong>ce a significant proportion<br />
of predoctoral dental education is dedicated<br />
to teach<strong>in</strong>g psychomotor cl<strong>in</strong>ical skills. A unit designed<br />
for the <strong>in</strong>struction of dental procedures us<strong>in</strong>g<br />
virtual reality-based technology (VRBT) was <strong>in</strong>troduced<br />
<strong>in</strong>to the dental education marketplace <strong>in</strong> the<br />
late 1990s. The unit, described <strong>in</strong> detail <strong>in</strong> a recent<br />
publication, 1 has several potential advantages over<br />
the traditional methodology used for the <strong>in</strong>struction<br />
1258 Journal of Dental Education ■ Volume 68, Number 12
of restorative preparative procedures. These potential<br />
advantages <strong>in</strong>clude the ability of the unit to give<br />
consistent, unbiased feedback based on evaluation<br />
of the preparation <strong>in</strong> terms of tenths of millimeters.<br />
The <strong>in</strong>stant availability of this feedback and the unlimited<br />
availability of the unit can also be viewed as<br />
potential advantages. Basically, the unit is composed<br />
of “real” items such as a simulated patient head and<br />
torso, KaVo dentoform, handpieces, air/water syr<strong>in</strong>ge,<br />
dental light, and an imag<strong>in</strong>g system that translates<br />
the operator’s tooth preparation <strong>in</strong>to a virtual<br />
image <strong>with</strong><strong>in</strong> a computer and displays it on a screen.<br />
Us<strong>in</strong>g the VRBT unit, a student can prepare<br />
dentoform teeth <strong>in</strong> much the same manner as <strong>with</strong><br />
the common dental simulators available at many<br />
schools. However, the unique property of this unit is<br />
its ability to construct a real-time virtual image of<br />
the student’s preparation <strong>in</strong> the computer. The computer<br />
can evaluate the tooth preparation both immediately<br />
and at the student’s request. Real-time evaluation<br />
for critical, noncorrectable errors is given as<br />
immediate feedback while more complete, detailed<br />
evaluation of a restorative preparation is given when<br />
requested by the student. The extensive evaluative<br />
feedback given when requested is presented <strong>in</strong> visual<br />
and written forms and <strong>in</strong>cludes a numerical<br />
grade. Hence, the VRBT unit offers objective, consistent<br />
evaluation of preparations easily obta<strong>in</strong>ed at<br />
any time dur<strong>in</strong>g the process of prepar<strong>in</strong>g the tooth.<br />
This evaluation <strong>in</strong>cludes both formative (corrective<br />
feedback) and summative (result<strong>in</strong>g <strong>in</strong> a f<strong>in</strong>al grade)<br />
evaluation. This is <strong>in</strong> contrast to an evaluation given<br />
by faculty that consists, for the most part, of evaluation<br />
of an end product.<br />
Although virtual reality-based or advanced<br />
technology is available <strong>in</strong> dental education, there is<br />
limited research <strong>in</strong>vestigat<strong>in</strong>g the effectiveness of this<br />
technology. The available research has produced<br />
conflict<strong>in</strong>g results and needs further review. 22-27<br />
VRBT has matured considerably s<strong>in</strong>ce its <strong>in</strong>troduction,<br />
and it is important to take <strong>in</strong>to consideration<br />
the differences of experience between schools and<br />
the level of maturity of the unit that was used <strong>in</strong> different<br />
studies when review<strong>in</strong>g the outcomes of VRBT<br />
research studies.<br />
The University of Pennsylvania School of Dental<br />
Medic<strong>in</strong>e (UPSDM) was one of the first dental<br />
schools <strong>in</strong> the world to purchase a VRBT unit and<br />
the first to <strong>in</strong>corporate this technology <strong>in</strong>to its curriculum<br />
<strong>in</strong> 1998. Over the next five years, UPSDM<br />
<strong>in</strong>vestigated the effectiveness of this technology <strong>in</strong> a<br />
variety of ways and, <strong>in</strong> the late fall of 2002, made<br />
the decision to not only purchase a significant number<br />
of units, but to modify our curriculum to take<br />
advantage of the potential of VRBT. This article describes<br />
our experience <strong>with</strong> the technology that led<br />
to this decision and reviews the results of three studies<br />
conducted at UPSDM.<br />
Description of VRBT Studies<br />
at UPSDM<br />
Our first study began soon after we had acquired<br />
a VRBT DentSim unit. Be<strong>in</strong>g one of the very<br />
first units, it could be classified as a beta unit. Although<br />
it was much more prone to technical problems<br />
than later units, the important and unique features<br />
were consistent <strong>with</strong> the state of the technology<br />
at that time. A sample of sixteen students was selected<br />
from the n<strong>in</strong>ety-four students enrolled <strong>in</strong> the<br />
first-year class <strong>in</strong> the follow<strong>in</strong>g manner. Students<br />
were first grouped based on three measures: learn<strong>in</strong>g<br />
style/type comb<strong>in</strong>ation, psychomotor skill level,<br />
and the Dental Admission Test (DAT) perceptual<br />
score. Learn<strong>in</strong>g styles were determ<strong>in</strong>ed on all <strong>in</strong>com<strong>in</strong>g<br />
students us<strong>in</strong>g both the 4Mat and Kolb learn<strong>in</strong>g<br />
style <strong>in</strong>ventories. 28,29 After students were grouped,<br />
student pairs were selected <strong>with</strong><strong>in</strong> each group by<br />
match<strong>in</strong>g as closely as possible for the rema<strong>in</strong><strong>in</strong>g<br />
sample characteristics, which were computer comfort<br />
(determ<strong>in</strong>ed by survey), gender, DAT academic<br />
averages, dental school academic averages, and college<br />
academic averages. For each of the eight pairs,<br />
one student of the pair was randomly chosen for the<br />
experimental group and the other assigned to the<br />
control group. This assignment ensured that the control<br />
group and experimental group were very similar<br />
<strong>in</strong> composition, a factor especially important <strong>with</strong><br />
small groups. The sample size was limited to sixteen<br />
students based on the maximum projected use of one<br />
VRBT unit by eight students <strong>in</strong> the experimental<br />
group dur<strong>in</strong>g the time allotted for the restorative procedure<br />
<strong>with</strong><strong>in</strong> the exist<strong>in</strong>g precl<strong>in</strong>ical operative<br />
course. The study sample consisted of twelve men<br />
and four women.<br />
Experimental students were assigned to the unit<br />
<strong>in</strong> two-hour time blocks over a three-week time period.<br />
Students <strong>in</strong> the control group were <strong>in</strong>structed<br />
<strong>in</strong> the UPSDM traditional laboratory. Each station<br />
<strong>in</strong> the traditional laboratory was equipped <strong>with</strong> a light,<br />
handpiece connections, and a post <strong>with</strong> an attached<br />
Dentoform. Manik<strong>in</strong> heads were not rout<strong>in</strong>ely used,<br />
December 2004 ■ Journal of Dental Education 1259
and it should be noted that the variable of more or<br />
less sophisticated physical simulation may affect attitudes<br />
toward the methodology. The experimental<br />
group was <strong>in</strong>structed to practice Class I amalgam<br />
procedures on the simulation unit and to receive no<br />
<strong>in</strong>struction on this procedure from faculty, other students,<br />
or monitors. The control group prepared teeth<br />
<strong>in</strong> the traditional lab and requested evaluation from<br />
faculty.<br />
Although the control group had access to faculty<br />
only dur<strong>in</strong>g course hours, they were allowed to<br />
practice dur<strong>in</strong>g the hours the laboratory was open<br />
(nights and weekends). In both groups, the criteria<br />
for assessment were determ<strong>in</strong>ed by the restorative<br />
faculty <strong>in</strong>volved <strong>in</strong> the precl<strong>in</strong>ical operative course<br />
and were consistent between the two groups. Both<br />
groups were <strong>in</strong>structed to keep detailed time logs <strong>in</strong><br />
which they recorded 1) time used to practice preparations,<br />
2) the number of teeth used for practice, and<br />
3) number of times they requested expert evaluation.<br />
The number of times students asked for expert evaluation<br />
represents the number of times an experimental<br />
student asked the computer for a complete evaluation<br />
of their preparation or when a control student<br />
asked a faculty member for evaluation. Weekly meet<strong>in</strong>gs<br />
and post-experiment personal <strong>in</strong>terviews <strong>with</strong><br />
each student affirmed that students had made every<br />
effort to keep accurate logs, although I acknowledge<br />
the possibility that some <strong>in</strong>accuracy may be <strong>in</strong>herent<br />
<strong>in</strong> data that is self-reported.<br />
At a scheduled time, a practical exam<strong>in</strong>ation<br />
(Class I preparation on tooth number 19) was given<br />
to both control and experimental groups. The teeth<br />
were collected from both groups, and one faculty<br />
member graded all teeth from both groups <strong>in</strong> a<br />
bl<strong>in</strong>ded manner.<br />
The second study was <strong>in</strong>itiated after UPSDM<br />
obta<strong>in</strong>ed four VRBT units. Along <strong>with</strong> the addition<br />
of more units, the software of the units had been<br />
updated several levels. The format from the first study<br />
was repeated except for the follow<strong>in</strong>g items. Twentyeight<br />
students were chosen and paired only for the<br />
follow<strong>in</strong>g parameters: DAT perceptual score, learn<strong>in</strong>g<br />
style/type comb<strong>in</strong>ation, gender, DAT academic<br />
averages, and dental school academic averages. The<br />
determ<strong>in</strong>ation was made randomly as to which student<br />
of the pair was assigned to the control versus<br />
experimental group, <strong>with</strong> fourteen students <strong>in</strong> each<br />
group. Students learned the procedures of Class I and<br />
Class II amalgams, and limited faculty <strong>in</strong>put was allowed.<br />
Faculty <strong>in</strong>put was available for each student<br />
one hour a week while they were assigned to the<br />
VRBT laboratory. Faculty only responded to questions<br />
such as what burs to use, how to correct errors,<br />
when to use the slow speed, etc. The faculty did not<br />
evaluate those parameters evaluated by the unit that<br />
used the same criteria as the faculty used to evaluate<br />
the control group. Detailed logs were kept <strong>in</strong> an identical<br />
manner as <strong>in</strong> Study 1. Aga<strong>in</strong>, student performance<br />
was measured by a practical exam<strong>in</strong>ation of<br />
a Class I and Class II preparation, which was graded<br />
by one faculty member <strong>in</strong> a bl<strong>in</strong>ded manner.<br />
Although the technical problems, especially<br />
<strong>with</strong> calibration, improved <strong>in</strong> each upgrade of the<br />
software, there was frustration among students who<br />
participated <strong>in</strong> the experimental groups of the previous<br />
studies when technical problems <strong>in</strong>terfered <strong>with</strong><br />
their ability to complete an assignment that was part<br />
of their course grade. To remove the pressure of hav<strong>in</strong>g<br />
the experience as part of their course, it was decided<br />
that further study of the VRBT would be done<br />
outside of course requirements. In Study 3, seven<br />
dental students between their freshman and sophomore<br />
years were employed by the school and worked<br />
<strong>in</strong> the <strong>Virtual</strong> <strong>Reality</strong> Laboratory five hours per day<br />
for seventeen days. A dentist was present <strong>in</strong> the room<br />
at all times to help <strong>with</strong> the technology and answer<br />
questions that were not related to preparation evaluation.<br />
Students reached competency <strong>in</strong> amalgam<br />
Class I, Class II, and Class V and composite Class<br />
III, Class IV, and Class V preparations. The students<br />
also worked on crown preparations (full gold, porcela<strong>in</strong><br />
fused to metal, all porcela<strong>in</strong>). Students kept<br />
logs of the number of practice preparations and times<br />
they asked for evaluation per preparation.<br />
One important aspect of the <strong>in</strong>corporation of<br />
new technologies <strong>in</strong>to the curriculum or <strong>in</strong>to dental<br />
practice is the level of acceptance by users. Hence,<br />
students’ attitude toward and acceptance of this technology<br />
were evaluated by surveys and personal <strong>in</strong>terviews<br />
for all three studies.<br />
Results and Discussion<br />
The first study <strong>in</strong>cluded eight students each <strong>in</strong><br />
the control and experimental groups. To verify that<br />
control and experimental students had similar levels<br />
of psychomotor skills, we compared the two groups’<br />
overall laboratory grade (didactic coursework grade<br />
is separate from the laboratory grade) <strong>in</strong> the operative<br />
course and found no statistical difference (p=<br />
0.50). This grade was a summation of n<strong>in</strong>e practical<br />
exams. Control students averaged 6.71 hours pre-<br />
1260 Journal of Dental Education ■ Volume 68, Number 12
par<strong>in</strong>g Class I preparations compared to the experimental<br />
students who spent an average of 3.69 hours<br />
(Table 1, p=0.07). Experimental students asked for<br />
evaluation more times (6.52 evaluation per hour of<br />
practice) than control students (3.21 evaluations per<br />
hour of practice) (Table 1, p=0.08). The experimental<br />
students prepared significantly more teeth per hour<br />
(average of 3.8 preparations/hr) than the control students<br />
(average of 1.6 preparations/hr) and not surpris<strong>in</strong>gly<br />
used more teeth (average of 11.71 versus<br />
6.57 for control, p=0.02). This emphasizes a higher<br />
efficiency <strong>in</strong> prepar<strong>in</strong>g teeth us<strong>in</strong>g VRBT. The results<br />
of the Class I practical exam<strong>in</strong>ation were slightly<br />
higher for the experimental group (72.6 percent versus<br />
61.9 percent) but were not statistically significant<br />
(p=0.23, Table 2).<br />
In Study 2, which conta<strong>in</strong>ed fourteen students<br />
<strong>in</strong> each group (control and experimental), similar<br />
results were found <strong>with</strong> control students<br />
spend<strong>in</strong>g 14.2 hours practic<strong>in</strong>g<br />
Class I and II preparations before<br />
the practical as compared to<br />
the experimental group spend<strong>in</strong>g<br />
8.2 hours practic<strong>in</strong>g (Table 1).<br />
Control students <strong>in</strong> Study 2 prepared<br />
2.1 preparations per hour<br />
compared to the experimental<br />
group, which prepared 3.4 preparations<br />
per hour. The control group<br />
requested an average of 1.99 evaluations<br />
per hour of practice compared<br />
to the experimental group,<br />
which requested 6.18 evaluations<br />
per hour (Table 1). The most obvious<br />
reason for this difference may<br />
be the ease of request<strong>in</strong>g evaluations<br />
from the computer compared<br />
<strong>with</strong> the difficulty or reluctance of<br />
a student to attract the attention of<br />
a faculty member for evaluation.<br />
These evaluations were both formative<br />
and summative. The fact,<br />
however, that the students requested<br />
evaluations this frequently<br />
suggests that the students <strong>in</strong> this<br />
study viewed frequent feedback as<br />
helpful to learn<strong>in</strong>g. This frequency<br />
of evaluations would not be feasible<br />
<strong>in</strong> a traditional laboratory<br />
s<strong>in</strong>ce it would require a faculty<br />
member supervis<strong>in</strong>g twelve students<br />
to give a complete evaluation<br />
of a preparation about every forty-five seconds. Recent<br />
experience <strong>with</strong> VRBT us<strong>in</strong>g an entire class has<br />
shown that students need to be weaned from the availability<br />
of <strong>in</strong>stant evaluation at some po<strong>in</strong>t <strong>in</strong> order to<br />
develop the ability to self-assess us<strong>in</strong>g <strong>in</strong>struments<br />
always available such as a probe and condenser.<br />
In Study 2, students <strong>in</strong> both control and experimental<br />
groups completed two practicals that were<br />
graded by one faculty member <strong>in</strong> a bl<strong>in</strong>ded manner.<br />
Control students scored 79.3 percent and 80.9 percent<br />
on the Class I and Class II practicals respectively,<br />
while the experimental students scored lower<br />
than control <strong>with</strong> 72.0 percent <strong>in</strong> the Class I and similar<br />
to control <strong>in</strong> the more difficult Class II at 79.7<br />
percent (Table 2). The two groups’ f<strong>in</strong>al overall<br />
grades <strong>in</strong> the course (laboratory portion only) were<br />
very similar, <strong>with</strong> the control students earn<strong>in</strong>g<br />
86.0 and the experimental groups averag<strong>in</strong>g 86.7<br />
Table 1. Comparison of time and evaluations requested<br />
Groups Time Spent Practic<strong>in</strong>g Evaluations Requested<br />
Per Hour of Practice<br />
Control Study 1 6.71 hrs (Class I ) 3.21<br />
Experimental Study 1 3.69 hrs (Class I) 6.52<br />
Control Study 2 14.2 hrs (Class I & II) 1.99<br />
Experimental Study 2 8.2 hrs (Class I & II) 6.18<br />
Comb<strong>in</strong>ed groups (Studies 1 & 2)<br />
Control groups’ averages 20.1 hrs 2.41<br />
Experimental groups’ averages 11.8 hrs 6.29<br />
Students recorded <strong>in</strong> log books the hours they spent practic<strong>in</strong>g before the practical<br />
exams and the number of times they requested expert evaluation. For control<br />
students, this was the number of times they requested feedback from an <strong>in</strong>structor;<br />
for experimental students, it was the number of times they requested evaluation<br />
from the unit. Study 1 had eight students per group; Study 2 had fourteen students<br />
per group.<br />
Table 2. Comparison of practical exam<strong>in</strong>ations and f<strong>in</strong>al course grades<br />
Group Practical Exam<strong>in</strong>ations F<strong>in</strong>al Course Grade<br />
Class I Class II<br />
Control Study 1 61.9 89.3<br />
Experimental Study 1 72.6 87.8<br />
Control Study 2 79.3 80.9 86.0<br />
Experimental Study 2 72.0 79.7 86.7<br />
Comb<strong>in</strong>ed averages (Studies 1 & 2)<br />
Control groups’ averages 70.6 80.9 87.7<br />
Experimental groups’ averages 72.3 79.7 87.3<br />
Results of practical exam<strong>in</strong>ations of procedures evaluated by one faculty member<br />
<strong>in</strong> a bl<strong>in</strong>ded manner. Study 1 <strong>in</strong>volved eight students per group; Study 2 <strong>in</strong>volved<br />
fourteen students per group.<br />
December 2004 ■ Journal of Dental Education 1261
(Table 2). It should be noted that there are separate<br />
grades for the psychomotor portion and the didactic<br />
portion of this course and only the psychomotor portion<br />
was used for comparison.<br />
Data from the third study, <strong>in</strong> which seven dental<br />
students were employed to work on the VRBT<br />
unit, shows that these students requested evaluations<br />
at a frequency seen <strong>in</strong> earlier studies (average of 7.60<br />
evaluations per procedure). Students prepared an<br />
average of 4.4 crown preparations per four-hour session.<br />
The crown preparations were impressive to our<br />
faculty even though these students had no previous<br />
didactic or laboratory experience <strong>with</strong> crown preparations.<br />
S<strong>in</strong>ce these students had considerable experience<br />
<strong>with</strong> both the VRBT (over eighty-four hours<br />
total) and traditional laboratory (approximately 300<br />
hours), they were asked to rate on a 1 to 5 scale the<br />
VRBT tra<strong>in</strong><strong>in</strong>g as compared to their experience <strong>with</strong><br />
the traditional laboratory. Results are shown <strong>in</strong> Table<br />
3. All students fully agreed that the VRBT unit improves<br />
manual skills, improves m<strong>in</strong>or movements<br />
<strong>with</strong> a handpiece, and <strong>in</strong>creases speed. The lowest<br />
agreement was <strong>in</strong> the statement that the VRBT widens<br />
the theoretical basis of cavity preparations (3.71<br />
out of 5) and that it gives accurate evaluations (3.29<br />
out of 5). In Figure 1, responses of students directly<br />
compar<strong>in</strong>g the traditional laboratory (LAB) to the<br />
VRBT <strong>in</strong> learn<strong>in</strong>g tooth preparation are shown. Students<br />
were asked if VRBT was more helpful, less<br />
helpful, or the same as the traditional lab for learn<strong>in</strong>g.<br />
Students were positive about some aspects of<br />
Table 3. Summary of student evaluation of VRBT<br />
Questions Rat<strong>in</strong>g VRBT<br />
(after over 80 hours’ experience)<br />
Average Response<br />
1= fully disagree,<br />
5= fully agree<br />
User friendly 4.57<br />
Improves manual skills 5.0<br />
Improves m<strong>in</strong>or movements 5.0<br />
Increases speed 5.0<br />
Widens theoretical basis 3.71<br />
Assists <strong>in</strong> 3D visualization 4.86<br />
Improves self-assurance 4.86<br />
Major teach<strong>in</strong>g source 4.86<br />
Provides detailed feedback 5.0<br />
Accurate evaluation 3.29<br />
Seven students evaluated their experience <strong>with</strong> VRBT<br />
after 80 hours’ experience. This experience <strong>in</strong>cluded<br />
procedures they had learned previously <strong>in</strong> a traditional<br />
laboratory and procedures they had neither previous<br />
experience <strong>with</strong> nor didactic foundational lectures.<br />
VRBT as compared to LAB, <strong>with</strong> the majority <strong>in</strong>dicat<strong>in</strong>g<br />
that VRBT provided more feedback, allowed<br />
self-evaluation, and was more like a real patient situation.<br />
It must be noted, however, that the more sophisticated<br />
manik<strong>in</strong> <strong>in</strong> the VRBT unit, by itself, could<br />
<strong>in</strong>fluence this response. Students felt the VRBT was<br />
harder to use than the traditional setup <strong>in</strong> the precl<strong>in</strong>ical<br />
laboratory. This could be attributed to technical<br />
difficulties <strong>with</strong> earlier units and software versions.<br />
Additional surveys of students <strong>in</strong> all three studies<br />
(data not presented) and focus group discussions<br />
completed by all students <strong>in</strong>volved <strong>with</strong> the VRBT<br />
over several years’ time span can be summarized <strong>in</strong>to<br />
several key observations: 1) students want faculty to<br />
play some part <strong>in</strong> their psychomotor tra<strong>in</strong><strong>in</strong>g even<br />
though students dislike the considerable variability<br />
between <strong>in</strong>structors and are frustrated by wait<strong>in</strong>g for<br />
faculty; 2) students view VRBT technology as hav<strong>in</strong>g<br />
a positive role <strong>in</strong> precl<strong>in</strong>ical tra<strong>in</strong><strong>in</strong>g; 3) students<br />
feel that they learn faster <strong>with</strong> VRBT; and 4) students<br />
feel more confident <strong>with</strong> a high-speed handpiece<br />
after tra<strong>in</strong><strong>in</strong>g on VRBT.<br />
Although several other schools have similar<br />
positive experiences, 23-25 there are some who have<br />
had experience <strong>with</strong> VRBT and have drawn different<br />
conclusions about its potential. A comparison of<br />
these other experiences is necessary to help put the<br />
different op<strong>in</strong>ions <strong>in</strong> perspective. Qu<strong>in</strong>n et al. 26,27 reported<br />
on the Dubl<strong>in</strong> Dental School and Hospital’s<br />
experience <strong>with</strong> VRBT. This <strong>in</strong>stitution’s pilot<br />
project, which used very early generations of simulators,<br />
measured student performance after only four<br />
hours of exposure to VRBT plus sixteen hours of<br />
traditional teach<strong>in</strong>g and, not surpris<strong>in</strong>gly, showed no<br />
difference <strong>in</strong> student performance between a control<br />
group (twenty-one hours of traditional tra<strong>in</strong><strong>in</strong>g) and<br />
experimental groups. A study published soon after<br />
led the same authors to make the statement that “VRbased<br />
skills acquisition is unsuitable for use as the<br />
sole method of feedback and evaluation for novice<br />
students” 26 —a very strong statement <strong>in</strong>deed. Further<br />
<strong>in</strong>spection of the article raises concern over the support<br />
for this statement.<br />
The data presented <strong>in</strong> the second study by<br />
Qu<strong>in</strong>n et al., 26 which aga<strong>in</strong> used early generation<br />
units, is derived from twenty students randomly divided<br />
<strong>in</strong>to two groups. The control group used the<br />
VRBT unit for five hours but rather than us<strong>in</strong>g the<br />
evaluation capabilities of the unit, relied on faculty<br />
feedback. The experimental group also used the<br />
VRBT unit for five hours but received all evaluation<br />
and feedback from the unit. After the practice ses-<br />
1262 Journal of Dental Education ■ Volume 68, Number 12
80<br />
70<br />
60<br />
more<br />
same<br />
less<br />
%<br />
50<br />
40<br />
30<br />
20<br />
10<br />
0<br />
Provides<br />
feedback<br />
Allows<br />
progress<br />
Allows selfevaluation<br />
Like real<br />
patient<br />
Ease of use<br />
Motivates to<br />
learn<br />
Figure 1. Student summary evaluations compar<strong>in</strong>g VRBT to traditional laboratory<br />
Students from Study 3 evaluated their experience <strong>with</strong> VRBT after several months’ experience <strong>with</strong> the traditional laboratory.<br />
They were asked to compare VRBT <strong>with</strong> the traditional laboratory as to whether the VRBT was more helpful, the same, or less<br />
helpful. Graph is courtesy of Dr. Eldad Irani.<br />
sions, both groups executed two Class I cavity preparations<br />
that were graded <strong>in</strong> a bl<strong>in</strong>ded manner by two<br />
faculty scorers. The authors reported there were statistical<br />
differences between the control and experimental<br />
groups <strong>in</strong> three out of five criteria for assessment,<br />
<strong>with</strong> the experimental group perform<strong>in</strong>g less<br />
well. This data is suspect s<strong>in</strong>ce there was no effort to<br />
match control or experimental groups <strong>in</strong> any parameter<br />
such as <strong>in</strong>nate psychomotor ability, academic<br />
performance, gender, previous computer experience,<br />
etc. In fact, no data concern<strong>in</strong>g the makeup of the<br />
control versus experimental students was provided,<br />
<strong>in</strong>clud<strong>in</strong>g their overall performance <strong>in</strong> precl<strong>in</strong>ical<br />
courses. As many operative faculty will attest to, students<br />
can vary widely <strong>in</strong> their <strong>in</strong>nate ability to grasp<br />
the psychomotor skills of operative dentistry, and at<br />
the very least, the control group versus experimental<br />
group performance <strong>in</strong> other psychomotor courses or<br />
other preparations <strong>in</strong> the same course should have<br />
been compared. It is possible that this situation could<br />
have been repeated between two randomized groups<br />
of ten students’ learn<strong>in</strong>g one operative procedure <strong>in</strong><br />
completely identical ways. If randomized selection<br />
was used, it would have been wise to have two control<br />
groups to strengthen any possible conclusions.<br />
If there was no statistical difference between the two<br />
control groups on the evaluation of a preparation <strong>in</strong><br />
any of the five criteria, differences of two or three of<br />
the five criteria between the two control groups and<br />
the VRBT experimental group may have been more<br />
valid. To use the data as presented <strong>in</strong> that publication<br />
to nullify a technology and state that this method<br />
of teach<strong>in</strong>g is unsuitable seems to be far from justified.<br />
Regardless of the design, it would appear unwise<br />
to most <strong>in</strong> dental education to make such a<br />
strong statement based on five hours of experience<br />
of ten students. Our experience <strong>with</strong> the VRBT spann<strong>in</strong>g<br />
over five years and <strong>in</strong>volv<strong>in</strong>g several generations<br />
of hardware and software, multiple studies,<br />
countless student hours, numerous surveys and student<br />
discussions, and many restorative procedures<br />
gives us the confidence that our results are valid and<br />
the decision to further embrace this technology is<br />
justified.<br />
December 2004 ■ Journal of Dental Education 1263
Conclusion<br />
Although our goal was to scientifically demonstrate<br />
the efficiency of VRBT <strong>in</strong> the education of<br />
psychomotor skills, we soon realized that this was a<br />
very difficult task due to various factors and limitations<br />
dur<strong>in</strong>g late 1990s when these units were first<br />
available. The limitations of low numbers of units<br />
and hence low numbers of students, technical problems,<br />
the desire to modify the educational environment<br />
as soon as we realized a better way to do it, and<br />
the frequent upgrades of software and hardware did<br />
not lend itself to the production of statistically significant<br />
data. Most of our data is suggestive, but is<br />
supportive of VRBT as an educational tool, and some<br />
data demonstrated statistically significant differences<br />
between experimental and control groups. But as Dr.<br />
Kathleen Rosen 20 remarks <strong>in</strong> a recent book devoted<br />
to health care simulation, “no one has ever proved<br />
the validity or value of simulation education and<br />
tra<strong>in</strong><strong>in</strong>g <strong>in</strong> any other <strong>in</strong>dustry” before its accepted<br />
use. Hence, it may be that we cannot wait to adapt<br />
VRBT simulation until unequivocal proof of the benefit<br />
had been demonstrated.<br />
In addition to the described research data we<br />
obta<strong>in</strong>ed, we formed op<strong>in</strong>ions of the value of this<br />
technology <strong>in</strong> many other, nonmeasurable ways.<br />
Considerable time was spent watch<strong>in</strong>g students <strong>in</strong><br />
the VRBT lab, talk<strong>in</strong>g <strong>with</strong> them at length, listen<strong>in</strong>g<br />
to them talk to the many visitors who came to observe,<br />
and watch<strong>in</strong>g students demonstrate the VRBT<br />
to others at our school and at national meet<strong>in</strong>gs. The<br />
sum of all our data, experience, evaluation, observations,<br />
and <strong>in</strong>tuition resulted <strong>in</strong> the belief that VRBT<br />
is a powerful educational methodology <strong>with</strong> the potential<br />
to significantly affect dental education. This<br />
belief was the basis for the purchase of fifteen VRBT<br />
units, allow<strong>in</strong>g for the full <strong>in</strong>corporation of this technology<br />
<strong>in</strong>to our courses and associated curriculum<br />
revision to help this technology realize its full potential<br />
<strong>in</strong> dental education.<br />
Acknowledgments<br />
I would like to thank Dr. Denise Stewart, Dr.<br />
Joan Gluch, Dr. Amer Abu-Hanna, Dr. Penny<br />
Hammrich, Dr. Francis Mante, Dr. Mamle Mante,<br />
and Dr. Eldad Irani for their help <strong>with</strong> these studies<br />
and Dr. Peter Berthold for help <strong>with</strong> the preparation<br />
of this manuscript.<br />
REFERENCES<br />
1. Rose JT, Buchanan JA, Sarrett DC. The DentSim system.<br />
J Dent Educ 1999;63(5):421-3.<br />
2. Plasschaert AM, Wilson NH, Cailleteau JG, Verdonschot<br />
EH. Op<strong>in</strong>ions and experiences of dental students and faculty<br />
concern<strong>in</strong>g computer-assisted learn<strong>in</strong>g. J Dent Educ<br />
1995;59:1034-40.<br />
3. Institute for Higher Education Policy. What’s the difference<br />
A review of contemporary research on the effectiveness<br />
of distance learn<strong>in</strong>g <strong>in</strong> higher education.<br />
Wash<strong>in</strong>gton, DC: Institute for Higher Education Policy,<br />
1998.<br />
4. F<strong>in</strong>kel D. Surgery moves <strong>in</strong>to the third dimension. Lancet<br />
1996;348(9043):1726-8.<br />
5. Smith C, Daniel P. Simulation technology: a strategy for<br />
implementation <strong>in</strong> surgical education and certification.<br />
Presence: Tele-operators and <strong>Virtual</strong> Environments<br />
2000;9(6).<br />
6. Buck GH. Development of simulators <strong>in</strong> medical education.<br />
Gesnerus 1991;48(pt 1):7-28.<br />
7. Robb N. Time to “reth<strong>in</strong>k models of medical education.”<br />
Can Med Assoc J 1997;157(30):304-5.<br />
8. Garden A, Rob<strong>in</strong>son B, Weller J, et al. Education to address<br />
medical error: a role for high fidelity patient simulation.<br />
N Z Med J 2002;115:133-4.<br />
9. Cohen L. Medical simulation is the wave of the future, U<br />
of O doctors say. Can Med Assoc J 1999;160:557.<br />
10. Satava RM. Accomplishments and challenges of surgical<br />
simulation: dawn<strong>in</strong>g of the next-generation surgical education.<br />
Surg Endosc 2001;15:232-41.<br />
11. Schwid HA. A flight simulator for general anesthesia.<br />
Comput Biomed Res 1987;20:64-75.<br />
12. Kennedy H. Simulation reshap<strong>in</strong>g military tra<strong>in</strong><strong>in</strong>g: technology<br />
jump<strong>in</strong>g from teenagers’ computers to pilots’ cockpits.<br />
National Defense Magaz<strong>in</strong>e, November 1999. At:<br />
www.nationaldefensemagaz<strong>in</strong>e.org/<br />
article.cfmpage=About. Accessed: August 10, 2004.<br />
13. Human Interface <strong>Technology</strong> Laboratory. At:<br />
www.hitl.wash<strong>in</strong>ton.edu/. Accessed: August 10, 2004.<br />
14. Ota D, Loft<strong>in</strong> B, Saito T, Lea R, Keller J. <strong>Virtual</strong> reality<br />
<strong>in</strong> surgical education. Comput Biol Med 1995;25(2):127-<br />
37.<br />
15. Merril G, Barker V. <strong>Virtual</strong> reality debuts <strong>in</strong> the teach<strong>in</strong>g<br />
laboratory <strong>in</strong> nurs<strong>in</strong>g. J Intra Nurs 1996;19(4):182-7.<br />
16. Satava R. <strong>Virtual</strong> reality, telesurgery, and the new world<br />
order of medic<strong>in</strong>e. J Image Guided Surg 1995;1(1):12-6.<br />
17. Gallagher AG, McClure N, McGuigan J, Crothers I, Brown<strong>in</strong>g<br />
J. <strong>Virtual</strong> reality tra<strong>in</strong><strong>in</strong>g <strong>in</strong> laparoscopic surgery: a prelim<strong>in</strong>ary<br />
assessment of m<strong>in</strong>imally <strong>in</strong>vasive surgical tra<strong>in</strong>er<br />
virtual reality. Endoscopy 1999;31(4):310-3.<br />
18. Tuggy ML. <strong>Virtual</strong> reality flexible sigmoidoscopy simulator<br />
tra<strong>in</strong><strong>in</strong>g: impact on resident performance. J Am Board<br />
Fam Pract 1998;11(6):426-33.<br />
19. Peugnet F, Dubois P, Rouland JF. <strong>Virtual</strong> reality versus<br />
conventional tra<strong>in</strong><strong>in</strong>g <strong>in</strong> ret<strong>in</strong>al photocoagulation: a first<br />
cl<strong>in</strong>ical assessment. Comput Aided Surg 1998;3(1):20-6.<br />
20. Loyd G, Lake C, Greenberg R, eds. Practical health care<br />
simulations. Philadelphia: Elsevier Mosby, 2004.<br />
1264 Journal of Dental Education ■ Volume 68, Number 12
21. Buchanan J, Williams J. Simulation <strong>in</strong> dentistry and oral<br />
surgery. In: Loyd G, Lake C, Greenberg R, eds. Practical<br />
health care simulations. Philadelphia: Elsevier Mosby,<br />
2004:493-512.<br />
22. Buchanan JA. Use of simulation technology <strong>in</strong> dental<br />
education. J Dent Educ 2001;65:1225-31.<br />
23. LeBlanc VR, Urbankova A, Hadavi F, Lichtenthal RM. A<br />
prelim<strong>in</strong>ary study <strong>in</strong> us<strong>in</strong>g virtual reality to tra<strong>in</strong> dental<br />
students. J Dent Educ 2004;68(3):378-83.<br />
24. Abstract by Case Western Reserve University, Cleveland,<br />
OH.<br />
25. Imber S, Shapira G, Gordon M, Judes H, Metzger Z. A<br />
virtual reality dental simulator predicts performance <strong>in</strong><br />
an operative dentistry manik<strong>in</strong> course. Eur J Dent Educ<br />
2003;7:160-3.<br />
26. Qu<strong>in</strong>n F, Keogh P, McDonald A, Hussey D. A study compar<strong>in</strong>g<br />
the effectiveness of conventional tra<strong>in</strong><strong>in</strong>g and virtual<br />
reality simulation <strong>in</strong> the skills acquisition of junior<br />
dental students. Eur J Dent Educ 2003;7:164-9.<br />
27. Qu<strong>in</strong>n F, Keogh P, McDonald A, Hussey D. A pilot study<br />
compar<strong>in</strong>g the effectiveness of conventional tra<strong>in</strong><strong>in</strong>g and<br />
virtual reality simulation <strong>in</strong> the skills acquisition of junior<br />
dental students. Eur J Dent Educ 2003;7:13-9.<br />
28. Kolb DA. Learn<strong>in</strong>g style <strong>in</strong>ventory. Hay McBer Resources<br />
Group technical manual. Boston: <strong>Experience</strong> <strong>Based</strong><br />
Learn<strong>in</strong>g Systems, Inc., 1999.<br />
29. Learn<strong>in</strong>g Type Measure. The 4Mat System. Oak Brook,<br />
IL: Excel, Inc., 1996.<br />
December 2004 ■ Journal of Dental Education 1265