30.01.2015 Views

Experience with Virtual Reality-Based Technology in Teaching ...

Experience with Virtual Reality-Based Technology in Teaching ...

Experience with Virtual Reality-Based Technology in Teaching ...

SHOW MORE
SHOW LESS

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

<strong>Experience</strong> <strong>with</strong> <strong>Virtual</strong> <strong>Reality</strong>-<strong>Based</strong><br />

<strong>Technology</strong> <strong>in</strong> Teach<strong>in</strong>g Restorative<br />

Dental Procedures<br />

Judith A. Buchanan, Ph.D., D.M.D.<br />

Abstract: This article reports on extensive experience <strong>with</strong> advanced simulation at the University of Pennsylvania, School of<br />

Dental Medic<strong>in</strong>e (UPSDM). <strong>Virtual</strong> reality-based technology (VRBT) or advanced simulation is currently available for the<br />

<strong>in</strong>struction of dental students <strong>in</strong> precl<strong>in</strong>ical restorative procedures. UPSDM was one of the first schools <strong>in</strong> the world to have<br />

extensive experience <strong>with</strong> VRBT technology us<strong>in</strong>g an advanced simulation unit (DentSim) from DenX, Ltd. UPSDM’s experience<br />

consists of several years of research us<strong>in</strong>g control and experimental groups, employ<strong>in</strong>g students to participate <strong>in</strong> an<br />

<strong>in</strong>vestigative project, and us<strong>in</strong>g the units for remediation and a supplement to the precl<strong>in</strong>ical laboratory. Currently, all first-year<br />

students (Class of 2007 and Class of 2008) are receiv<strong>in</strong>g most of their preparative operative tra<strong>in</strong><strong>in</strong>g on the VRBT units. UPSDM<br />

started <strong>with</strong> one (beta) version unit <strong>in</strong> 1998, which was later updated and expanded first to four units and recently to fifteen units.<br />

This communication is present<strong>in</strong>g the studies that were of fundamental importance <strong>in</strong> mak<strong>in</strong>g the decision to acquire fifteen units<br />

<strong>in</strong> 2003. The areas of ma<strong>in</strong> <strong>in</strong>terest to the SDM concern<strong>in</strong>g this technology were its use <strong>in</strong> teach<strong>in</strong>g, refresh<strong>in</strong>g, and remediat<strong>in</strong>g<br />

students <strong>in</strong> restorative procedures and its effectiveness as a teach<strong>in</strong>g methodology <strong>in</strong> relation to time, efficiency, and faculty.<br />

Several studies <strong>with</strong> vary<strong>in</strong>g parameters were performed at various time po<strong>in</strong>ts. The limited statistical analysis conducted was not<br />

conclusive for all measures, and <strong>in</strong> some cases the data only suggest areas of possible significance. This is due to the low number<br />

of students who could access the limited number of available units and the change of protocols <strong>in</strong> response to student and faculty<br />

<strong>in</strong>put. Overall, the results do suggest, however, that students learn faster, arrive at the same level of performance, accomplish<br />

more practice procedures per hour, and request more evaluations per procedure or per hour than <strong>in</strong> our traditional laboratories.<br />

Students’ attitudes, as measured by surveys, group <strong>in</strong>terviews, and private <strong>in</strong>terviews, are mixed. Our overall evaluation of our<br />

experience <strong>with</strong> this technology has been positive and led to the purchase of additional units, its full <strong>in</strong>corporation <strong>in</strong>to our<br />

curriculum, and curriculum revision to maximize its potential. Our conclusion is that this technology offers significant potential<br />

<strong>in</strong> the field of dental education and that further use and <strong>in</strong>vestigation are both desired and justified.<br />

Dr. Buchanan is Associate Dean, Academic Affairs, Department of Preventive and Restorative Sciences, University of Pennsylvania,<br />

School of Dental Medic<strong>in</strong>e. Direct correspondence and requests for repr<strong>in</strong>ts to Dr. Buchanan at the Office of Academic<br />

Affairs, University of Pennsylvania School of Dental Medic<strong>in</strong>e, 240 S. 40 th Street, Philadelphia, PA 19104-6003; 215-898-1167<br />

phone; 215-573-4761 fax; Jbuchana@pobox.upenn.edu.<br />

This project was funded by a grant from the University of Pennsylvania Research Foundation.<br />

Key words: virtual reality, dental restorative procedures, dental education, technology, computer-assisted <strong>in</strong>struction<br />

Submitted for publication 8/12/04; accepted 10/19/04<br />

Rapid growth has occurred <strong>in</strong> higher education<br />

<strong>in</strong> the areas of distance learn<strong>in</strong>g, <strong>in</strong>teractive<br />

telecommunications, computerassisted<br />

<strong>in</strong>struction, and computer simulations. 1-4 The<br />

use of simulation has been proposed as the next major<br />

step <strong>in</strong> the evolution of health science education 5-<br />

9<br />

although dentistry has been us<strong>in</strong>g various types of<br />

simulation for quite some time. The amaz<strong>in</strong>g growth<br />

of computer hardware and software has led to the<br />

development of virtual worlds that support the field<br />

of advanced simulation. <strong>Virtual</strong> reality creates virtual<br />

worlds us<strong>in</strong>g mathematical models and computer<br />

programs, allow<strong>in</strong>g users to move <strong>in</strong> the created virtual<br />

world <strong>in</strong> a way similar to real life. Although used<br />

extensively for some time <strong>in</strong> other fields such as aeronautical<br />

tra<strong>in</strong><strong>in</strong>g, the technology of virtual reality has<br />

only been recently proposed as a powerful tool for<br />

the education of health professionals. 7,10-22 Some experts<br />

have referred to this technology as the “third<br />

dimension” 4 or the “next generation” learn<strong>in</strong>g environment<br />

<strong>in</strong> medic<strong>in</strong>e. 16<br />

Dental education, of all the health professional<br />

schools, is possibly the discipl<strong>in</strong>e that could most<br />

benefit from virtual reality s<strong>in</strong>ce a significant proportion<br />

of predoctoral dental education is dedicated<br />

to teach<strong>in</strong>g psychomotor cl<strong>in</strong>ical skills. A unit designed<br />

for the <strong>in</strong>struction of dental procedures us<strong>in</strong>g<br />

virtual reality-based technology (VRBT) was <strong>in</strong>troduced<br />

<strong>in</strong>to the dental education marketplace <strong>in</strong> the<br />

late 1990s. The unit, described <strong>in</strong> detail <strong>in</strong> a recent<br />

publication, 1 has several potential advantages over<br />

the traditional methodology used for the <strong>in</strong>struction<br />

1258 Journal of Dental Education ■ Volume 68, Number 12


of restorative preparative procedures. These potential<br />

advantages <strong>in</strong>clude the ability of the unit to give<br />

consistent, unbiased feedback based on evaluation<br />

of the preparation <strong>in</strong> terms of tenths of millimeters.<br />

The <strong>in</strong>stant availability of this feedback and the unlimited<br />

availability of the unit can also be viewed as<br />

potential advantages. Basically, the unit is composed<br />

of “real” items such as a simulated patient head and<br />

torso, KaVo dentoform, handpieces, air/water syr<strong>in</strong>ge,<br />

dental light, and an imag<strong>in</strong>g system that translates<br />

the operator’s tooth preparation <strong>in</strong>to a virtual<br />

image <strong>with</strong><strong>in</strong> a computer and displays it on a screen.<br />

Us<strong>in</strong>g the VRBT unit, a student can prepare<br />

dentoform teeth <strong>in</strong> much the same manner as <strong>with</strong><br />

the common dental simulators available at many<br />

schools. However, the unique property of this unit is<br />

its ability to construct a real-time virtual image of<br />

the student’s preparation <strong>in</strong> the computer. The computer<br />

can evaluate the tooth preparation both immediately<br />

and at the student’s request. Real-time evaluation<br />

for critical, noncorrectable errors is given as<br />

immediate feedback while more complete, detailed<br />

evaluation of a restorative preparation is given when<br />

requested by the student. The extensive evaluative<br />

feedback given when requested is presented <strong>in</strong> visual<br />

and written forms and <strong>in</strong>cludes a numerical<br />

grade. Hence, the VRBT unit offers objective, consistent<br />

evaluation of preparations easily obta<strong>in</strong>ed at<br />

any time dur<strong>in</strong>g the process of prepar<strong>in</strong>g the tooth.<br />

This evaluation <strong>in</strong>cludes both formative (corrective<br />

feedback) and summative (result<strong>in</strong>g <strong>in</strong> a f<strong>in</strong>al grade)<br />

evaluation. This is <strong>in</strong> contrast to an evaluation given<br />

by faculty that consists, for the most part, of evaluation<br />

of an end product.<br />

Although virtual reality-based or advanced<br />

technology is available <strong>in</strong> dental education, there is<br />

limited research <strong>in</strong>vestigat<strong>in</strong>g the effectiveness of this<br />

technology. The available research has produced<br />

conflict<strong>in</strong>g results and needs further review. 22-27<br />

VRBT has matured considerably s<strong>in</strong>ce its <strong>in</strong>troduction,<br />

and it is important to take <strong>in</strong>to consideration<br />

the differences of experience between schools and<br />

the level of maturity of the unit that was used <strong>in</strong> different<br />

studies when review<strong>in</strong>g the outcomes of VRBT<br />

research studies.<br />

The University of Pennsylvania School of Dental<br />

Medic<strong>in</strong>e (UPSDM) was one of the first dental<br />

schools <strong>in</strong> the world to purchase a VRBT unit and<br />

the first to <strong>in</strong>corporate this technology <strong>in</strong>to its curriculum<br />

<strong>in</strong> 1998. Over the next five years, UPSDM<br />

<strong>in</strong>vestigated the effectiveness of this technology <strong>in</strong> a<br />

variety of ways and, <strong>in</strong> the late fall of 2002, made<br />

the decision to not only purchase a significant number<br />

of units, but to modify our curriculum to take<br />

advantage of the potential of VRBT. This article describes<br />

our experience <strong>with</strong> the technology that led<br />

to this decision and reviews the results of three studies<br />

conducted at UPSDM.<br />

Description of VRBT Studies<br />

at UPSDM<br />

Our first study began soon after we had acquired<br />

a VRBT DentSim unit. Be<strong>in</strong>g one of the very<br />

first units, it could be classified as a beta unit. Although<br />

it was much more prone to technical problems<br />

than later units, the important and unique features<br />

were consistent <strong>with</strong> the state of the technology<br />

at that time. A sample of sixteen students was selected<br />

from the n<strong>in</strong>ety-four students enrolled <strong>in</strong> the<br />

first-year class <strong>in</strong> the follow<strong>in</strong>g manner. Students<br />

were first grouped based on three measures: learn<strong>in</strong>g<br />

style/type comb<strong>in</strong>ation, psychomotor skill level,<br />

and the Dental Admission Test (DAT) perceptual<br />

score. Learn<strong>in</strong>g styles were determ<strong>in</strong>ed on all <strong>in</strong>com<strong>in</strong>g<br />

students us<strong>in</strong>g both the 4Mat and Kolb learn<strong>in</strong>g<br />

style <strong>in</strong>ventories. 28,29 After students were grouped,<br />

student pairs were selected <strong>with</strong><strong>in</strong> each group by<br />

match<strong>in</strong>g as closely as possible for the rema<strong>in</strong><strong>in</strong>g<br />

sample characteristics, which were computer comfort<br />

(determ<strong>in</strong>ed by survey), gender, DAT academic<br />

averages, dental school academic averages, and college<br />

academic averages. For each of the eight pairs,<br />

one student of the pair was randomly chosen for the<br />

experimental group and the other assigned to the<br />

control group. This assignment ensured that the control<br />

group and experimental group were very similar<br />

<strong>in</strong> composition, a factor especially important <strong>with</strong><br />

small groups. The sample size was limited to sixteen<br />

students based on the maximum projected use of one<br />

VRBT unit by eight students <strong>in</strong> the experimental<br />

group dur<strong>in</strong>g the time allotted for the restorative procedure<br />

<strong>with</strong><strong>in</strong> the exist<strong>in</strong>g precl<strong>in</strong>ical operative<br />

course. The study sample consisted of twelve men<br />

and four women.<br />

Experimental students were assigned to the unit<br />

<strong>in</strong> two-hour time blocks over a three-week time period.<br />

Students <strong>in</strong> the control group were <strong>in</strong>structed<br />

<strong>in</strong> the UPSDM traditional laboratory. Each station<br />

<strong>in</strong> the traditional laboratory was equipped <strong>with</strong> a light,<br />

handpiece connections, and a post <strong>with</strong> an attached<br />

Dentoform. Manik<strong>in</strong> heads were not rout<strong>in</strong>ely used,<br />

December 2004 ■ Journal of Dental Education 1259


and it should be noted that the variable of more or<br />

less sophisticated physical simulation may affect attitudes<br />

toward the methodology. The experimental<br />

group was <strong>in</strong>structed to practice Class I amalgam<br />

procedures on the simulation unit and to receive no<br />

<strong>in</strong>struction on this procedure from faculty, other students,<br />

or monitors. The control group prepared teeth<br />

<strong>in</strong> the traditional lab and requested evaluation from<br />

faculty.<br />

Although the control group had access to faculty<br />

only dur<strong>in</strong>g course hours, they were allowed to<br />

practice dur<strong>in</strong>g the hours the laboratory was open<br />

(nights and weekends). In both groups, the criteria<br />

for assessment were determ<strong>in</strong>ed by the restorative<br />

faculty <strong>in</strong>volved <strong>in</strong> the precl<strong>in</strong>ical operative course<br />

and were consistent between the two groups. Both<br />

groups were <strong>in</strong>structed to keep detailed time logs <strong>in</strong><br />

which they recorded 1) time used to practice preparations,<br />

2) the number of teeth used for practice, and<br />

3) number of times they requested expert evaluation.<br />

The number of times students asked for expert evaluation<br />

represents the number of times an experimental<br />

student asked the computer for a complete evaluation<br />

of their preparation or when a control student<br />

asked a faculty member for evaluation. Weekly meet<strong>in</strong>gs<br />

and post-experiment personal <strong>in</strong>terviews <strong>with</strong><br />

each student affirmed that students had made every<br />

effort to keep accurate logs, although I acknowledge<br />

the possibility that some <strong>in</strong>accuracy may be <strong>in</strong>herent<br />

<strong>in</strong> data that is self-reported.<br />

At a scheduled time, a practical exam<strong>in</strong>ation<br />

(Class I preparation on tooth number 19) was given<br />

to both control and experimental groups. The teeth<br />

were collected from both groups, and one faculty<br />

member graded all teeth from both groups <strong>in</strong> a<br />

bl<strong>in</strong>ded manner.<br />

The second study was <strong>in</strong>itiated after UPSDM<br />

obta<strong>in</strong>ed four VRBT units. Along <strong>with</strong> the addition<br />

of more units, the software of the units had been<br />

updated several levels. The format from the first study<br />

was repeated except for the follow<strong>in</strong>g items. Twentyeight<br />

students were chosen and paired only for the<br />

follow<strong>in</strong>g parameters: DAT perceptual score, learn<strong>in</strong>g<br />

style/type comb<strong>in</strong>ation, gender, DAT academic<br />

averages, and dental school academic averages. The<br />

determ<strong>in</strong>ation was made randomly as to which student<br />

of the pair was assigned to the control versus<br />

experimental group, <strong>with</strong> fourteen students <strong>in</strong> each<br />

group. Students learned the procedures of Class I and<br />

Class II amalgams, and limited faculty <strong>in</strong>put was allowed.<br />

Faculty <strong>in</strong>put was available for each student<br />

one hour a week while they were assigned to the<br />

VRBT laboratory. Faculty only responded to questions<br />

such as what burs to use, how to correct errors,<br />

when to use the slow speed, etc. The faculty did not<br />

evaluate those parameters evaluated by the unit that<br />

used the same criteria as the faculty used to evaluate<br />

the control group. Detailed logs were kept <strong>in</strong> an identical<br />

manner as <strong>in</strong> Study 1. Aga<strong>in</strong>, student performance<br />

was measured by a practical exam<strong>in</strong>ation of<br />

a Class I and Class II preparation, which was graded<br />

by one faculty member <strong>in</strong> a bl<strong>in</strong>ded manner.<br />

Although the technical problems, especially<br />

<strong>with</strong> calibration, improved <strong>in</strong> each upgrade of the<br />

software, there was frustration among students who<br />

participated <strong>in</strong> the experimental groups of the previous<br />

studies when technical problems <strong>in</strong>terfered <strong>with</strong><br />

their ability to complete an assignment that was part<br />

of their course grade. To remove the pressure of hav<strong>in</strong>g<br />

the experience as part of their course, it was decided<br />

that further study of the VRBT would be done<br />

outside of course requirements. In Study 3, seven<br />

dental students between their freshman and sophomore<br />

years were employed by the school and worked<br />

<strong>in</strong> the <strong>Virtual</strong> <strong>Reality</strong> Laboratory five hours per day<br />

for seventeen days. A dentist was present <strong>in</strong> the room<br />

at all times to help <strong>with</strong> the technology and answer<br />

questions that were not related to preparation evaluation.<br />

Students reached competency <strong>in</strong> amalgam<br />

Class I, Class II, and Class V and composite Class<br />

III, Class IV, and Class V preparations. The students<br />

also worked on crown preparations (full gold, porcela<strong>in</strong><br />

fused to metal, all porcela<strong>in</strong>). Students kept<br />

logs of the number of practice preparations and times<br />

they asked for evaluation per preparation.<br />

One important aspect of the <strong>in</strong>corporation of<br />

new technologies <strong>in</strong>to the curriculum or <strong>in</strong>to dental<br />

practice is the level of acceptance by users. Hence,<br />

students’ attitude toward and acceptance of this technology<br />

were evaluated by surveys and personal <strong>in</strong>terviews<br />

for all three studies.<br />

Results and Discussion<br />

The first study <strong>in</strong>cluded eight students each <strong>in</strong><br />

the control and experimental groups. To verify that<br />

control and experimental students had similar levels<br />

of psychomotor skills, we compared the two groups’<br />

overall laboratory grade (didactic coursework grade<br />

is separate from the laboratory grade) <strong>in</strong> the operative<br />

course and found no statistical difference (p=<br />

0.50). This grade was a summation of n<strong>in</strong>e practical<br />

exams. Control students averaged 6.71 hours pre-<br />

1260 Journal of Dental Education ■ Volume 68, Number 12


par<strong>in</strong>g Class I preparations compared to the experimental<br />

students who spent an average of 3.69 hours<br />

(Table 1, p=0.07). Experimental students asked for<br />

evaluation more times (6.52 evaluation per hour of<br />

practice) than control students (3.21 evaluations per<br />

hour of practice) (Table 1, p=0.08). The experimental<br />

students prepared significantly more teeth per hour<br />

(average of 3.8 preparations/hr) than the control students<br />

(average of 1.6 preparations/hr) and not surpris<strong>in</strong>gly<br />

used more teeth (average of 11.71 versus<br />

6.57 for control, p=0.02). This emphasizes a higher<br />

efficiency <strong>in</strong> prepar<strong>in</strong>g teeth us<strong>in</strong>g VRBT. The results<br />

of the Class I practical exam<strong>in</strong>ation were slightly<br />

higher for the experimental group (72.6 percent versus<br />

61.9 percent) but were not statistically significant<br />

(p=0.23, Table 2).<br />

In Study 2, which conta<strong>in</strong>ed fourteen students<br />

<strong>in</strong> each group (control and experimental), similar<br />

results were found <strong>with</strong> control students<br />

spend<strong>in</strong>g 14.2 hours practic<strong>in</strong>g<br />

Class I and II preparations before<br />

the practical as compared to<br />

the experimental group spend<strong>in</strong>g<br />

8.2 hours practic<strong>in</strong>g (Table 1).<br />

Control students <strong>in</strong> Study 2 prepared<br />

2.1 preparations per hour<br />

compared to the experimental<br />

group, which prepared 3.4 preparations<br />

per hour. The control group<br />

requested an average of 1.99 evaluations<br />

per hour of practice compared<br />

to the experimental group,<br />

which requested 6.18 evaluations<br />

per hour (Table 1). The most obvious<br />

reason for this difference may<br />

be the ease of request<strong>in</strong>g evaluations<br />

from the computer compared<br />

<strong>with</strong> the difficulty or reluctance of<br />

a student to attract the attention of<br />

a faculty member for evaluation.<br />

These evaluations were both formative<br />

and summative. The fact,<br />

however, that the students requested<br />

evaluations this frequently<br />

suggests that the students <strong>in</strong> this<br />

study viewed frequent feedback as<br />

helpful to learn<strong>in</strong>g. This frequency<br />

of evaluations would not be feasible<br />

<strong>in</strong> a traditional laboratory<br />

s<strong>in</strong>ce it would require a faculty<br />

member supervis<strong>in</strong>g twelve students<br />

to give a complete evaluation<br />

of a preparation about every forty-five seconds. Recent<br />

experience <strong>with</strong> VRBT us<strong>in</strong>g an entire class has<br />

shown that students need to be weaned from the availability<br />

of <strong>in</strong>stant evaluation at some po<strong>in</strong>t <strong>in</strong> order to<br />

develop the ability to self-assess us<strong>in</strong>g <strong>in</strong>struments<br />

always available such as a probe and condenser.<br />

In Study 2, students <strong>in</strong> both control and experimental<br />

groups completed two practicals that were<br />

graded by one faculty member <strong>in</strong> a bl<strong>in</strong>ded manner.<br />

Control students scored 79.3 percent and 80.9 percent<br />

on the Class I and Class II practicals respectively,<br />

while the experimental students scored lower<br />

than control <strong>with</strong> 72.0 percent <strong>in</strong> the Class I and similar<br />

to control <strong>in</strong> the more difficult Class II at 79.7<br />

percent (Table 2). The two groups’ f<strong>in</strong>al overall<br />

grades <strong>in</strong> the course (laboratory portion only) were<br />

very similar, <strong>with</strong> the control students earn<strong>in</strong>g<br />

86.0 and the experimental groups averag<strong>in</strong>g 86.7<br />

Table 1. Comparison of time and evaluations requested<br />

Groups Time Spent Practic<strong>in</strong>g Evaluations Requested<br />

Per Hour of Practice<br />

Control Study 1 6.71 hrs (Class I ) 3.21<br />

Experimental Study 1 3.69 hrs (Class I) 6.52<br />

Control Study 2 14.2 hrs (Class I & II) 1.99<br />

Experimental Study 2 8.2 hrs (Class I & II) 6.18<br />

Comb<strong>in</strong>ed groups (Studies 1 & 2)<br />

Control groups’ averages 20.1 hrs 2.41<br />

Experimental groups’ averages 11.8 hrs 6.29<br />

Students recorded <strong>in</strong> log books the hours they spent practic<strong>in</strong>g before the practical<br />

exams and the number of times they requested expert evaluation. For control<br />

students, this was the number of times they requested feedback from an <strong>in</strong>structor;<br />

for experimental students, it was the number of times they requested evaluation<br />

from the unit. Study 1 had eight students per group; Study 2 had fourteen students<br />

per group.<br />

Table 2. Comparison of practical exam<strong>in</strong>ations and f<strong>in</strong>al course grades<br />

Group Practical Exam<strong>in</strong>ations F<strong>in</strong>al Course Grade<br />

Class I Class II<br />

Control Study 1 61.9 89.3<br />

Experimental Study 1 72.6 87.8<br />

Control Study 2 79.3 80.9 86.0<br />

Experimental Study 2 72.0 79.7 86.7<br />

Comb<strong>in</strong>ed averages (Studies 1 & 2)<br />

Control groups’ averages 70.6 80.9 87.7<br />

Experimental groups’ averages 72.3 79.7 87.3<br />

Results of practical exam<strong>in</strong>ations of procedures evaluated by one faculty member<br />

<strong>in</strong> a bl<strong>in</strong>ded manner. Study 1 <strong>in</strong>volved eight students per group; Study 2 <strong>in</strong>volved<br />

fourteen students per group.<br />

December 2004 ■ Journal of Dental Education 1261


(Table 2). It should be noted that there are separate<br />

grades for the psychomotor portion and the didactic<br />

portion of this course and only the psychomotor portion<br />

was used for comparison.<br />

Data from the third study, <strong>in</strong> which seven dental<br />

students were employed to work on the VRBT<br />

unit, shows that these students requested evaluations<br />

at a frequency seen <strong>in</strong> earlier studies (average of 7.60<br />

evaluations per procedure). Students prepared an<br />

average of 4.4 crown preparations per four-hour session.<br />

The crown preparations were impressive to our<br />

faculty even though these students had no previous<br />

didactic or laboratory experience <strong>with</strong> crown preparations.<br />

S<strong>in</strong>ce these students had considerable experience<br />

<strong>with</strong> both the VRBT (over eighty-four hours<br />

total) and traditional laboratory (approximately 300<br />

hours), they were asked to rate on a 1 to 5 scale the<br />

VRBT tra<strong>in</strong><strong>in</strong>g as compared to their experience <strong>with</strong><br />

the traditional laboratory. Results are shown <strong>in</strong> Table<br />

3. All students fully agreed that the VRBT unit improves<br />

manual skills, improves m<strong>in</strong>or movements<br />

<strong>with</strong> a handpiece, and <strong>in</strong>creases speed. The lowest<br />

agreement was <strong>in</strong> the statement that the VRBT widens<br />

the theoretical basis of cavity preparations (3.71<br />

out of 5) and that it gives accurate evaluations (3.29<br />

out of 5). In Figure 1, responses of students directly<br />

compar<strong>in</strong>g the traditional laboratory (LAB) to the<br />

VRBT <strong>in</strong> learn<strong>in</strong>g tooth preparation are shown. Students<br />

were asked if VRBT was more helpful, less<br />

helpful, or the same as the traditional lab for learn<strong>in</strong>g.<br />

Students were positive about some aspects of<br />

Table 3. Summary of student evaluation of VRBT<br />

Questions Rat<strong>in</strong>g VRBT<br />

(after over 80 hours’ experience)<br />

Average Response<br />

1= fully disagree,<br />

5= fully agree<br />

User friendly 4.57<br />

Improves manual skills 5.0<br />

Improves m<strong>in</strong>or movements 5.0<br />

Increases speed 5.0<br />

Widens theoretical basis 3.71<br />

Assists <strong>in</strong> 3D visualization 4.86<br />

Improves self-assurance 4.86<br />

Major teach<strong>in</strong>g source 4.86<br />

Provides detailed feedback 5.0<br />

Accurate evaluation 3.29<br />

Seven students evaluated their experience <strong>with</strong> VRBT<br />

after 80 hours’ experience. This experience <strong>in</strong>cluded<br />

procedures they had learned previously <strong>in</strong> a traditional<br />

laboratory and procedures they had neither previous<br />

experience <strong>with</strong> nor didactic foundational lectures.<br />

VRBT as compared to LAB, <strong>with</strong> the majority <strong>in</strong>dicat<strong>in</strong>g<br />

that VRBT provided more feedback, allowed<br />

self-evaluation, and was more like a real patient situation.<br />

It must be noted, however, that the more sophisticated<br />

manik<strong>in</strong> <strong>in</strong> the VRBT unit, by itself, could<br />

<strong>in</strong>fluence this response. Students felt the VRBT was<br />

harder to use than the traditional setup <strong>in</strong> the precl<strong>in</strong>ical<br />

laboratory. This could be attributed to technical<br />

difficulties <strong>with</strong> earlier units and software versions.<br />

Additional surveys of students <strong>in</strong> all three studies<br />

(data not presented) and focus group discussions<br />

completed by all students <strong>in</strong>volved <strong>with</strong> the VRBT<br />

over several years’ time span can be summarized <strong>in</strong>to<br />

several key observations: 1) students want faculty to<br />

play some part <strong>in</strong> their psychomotor tra<strong>in</strong><strong>in</strong>g even<br />

though students dislike the considerable variability<br />

between <strong>in</strong>structors and are frustrated by wait<strong>in</strong>g for<br />

faculty; 2) students view VRBT technology as hav<strong>in</strong>g<br />

a positive role <strong>in</strong> precl<strong>in</strong>ical tra<strong>in</strong><strong>in</strong>g; 3) students<br />

feel that they learn faster <strong>with</strong> VRBT; and 4) students<br />

feel more confident <strong>with</strong> a high-speed handpiece<br />

after tra<strong>in</strong><strong>in</strong>g on VRBT.<br />

Although several other schools have similar<br />

positive experiences, 23-25 there are some who have<br />

had experience <strong>with</strong> VRBT and have drawn different<br />

conclusions about its potential. A comparison of<br />

these other experiences is necessary to help put the<br />

different op<strong>in</strong>ions <strong>in</strong> perspective. Qu<strong>in</strong>n et al. 26,27 reported<br />

on the Dubl<strong>in</strong> Dental School and Hospital’s<br />

experience <strong>with</strong> VRBT. This <strong>in</strong>stitution’s pilot<br />

project, which used very early generations of simulators,<br />

measured student performance after only four<br />

hours of exposure to VRBT plus sixteen hours of<br />

traditional teach<strong>in</strong>g and, not surpris<strong>in</strong>gly, showed no<br />

difference <strong>in</strong> student performance between a control<br />

group (twenty-one hours of traditional tra<strong>in</strong><strong>in</strong>g) and<br />

experimental groups. A study published soon after<br />

led the same authors to make the statement that “VRbased<br />

skills acquisition is unsuitable for use as the<br />

sole method of feedback and evaluation for novice<br />

students” 26 —a very strong statement <strong>in</strong>deed. Further<br />

<strong>in</strong>spection of the article raises concern over the support<br />

for this statement.<br />

The data presented <strong>in</strong> the second study by<br />

Qu<strong>in</strong>n et al., 26 which aga<strong>in</strong> used early generation<br />

units, is derived from twenty students randomly divided<br />

<strong>in</strong>to two groups. The control group used the<br />

VRBT unit for five hours but rather than us<strong>in</strong>g the<br />

evaluation capabilities of the unit, relied on faculty<br />

feedback. The experimental group also used the<br />

VRBT unit for five hours but received all evaluation<br />

and feedback from the unit. After the practice ses-<br />

1262 Journal of Dental Education ■ Volume 68, Number 12


80<br />

70<br />

60<br />

more<br />

same<br />

less<br />

%<br />

50<br />

40<br />

30<br />

20<br />

10<br />

0<br />

Provides<br />

feedback<br />

Allows<br />

progress<br />

Allows selfevaluation<br />

Like real<br />

patient<br />

Ease of use<br />

Motivates to<br />

learn<br />

Figure 1. Student summary evaluations compar<strong>in</strong>g VRBT to traditional laboratory<br />

Students from Study 3 evaluated their experience <strong>with</strong> VRBT after several months’ experience <strong>with</strong> the traditional laboratory.<br />

They were asked to compare VRBT <strong>with</strong> the traditional laboratory as to whether the VRBT was more helpful, the same, or less<br />

helpful. Graph is courtesy of Dr. Eldad Irani.<br />

sions, both groups executed two Class I cavity preparations<br />

that were graded <strong>in</strong> a bl<strong>in</strong>ded manner by two<br />

faculty scorers. The authors reported there were statistical<br />

differences between the control and experimental<br />

groups <strong>in</strong> three out of five criteria for assessment,<br />

<strong>with</strong> the experimental group perform<strong>in</strong>g less<br />

well. This data is suspect s<strong>in</strong>ce there was no effort to<br />

match control or experimental groups <strong>in</strong> any parameter<br />

such as <strong>in</strong>nate psychomotor ability, academic<br />

performance, gender, previous computer experience,<br />

etc. In fact, no data concern<strong>in</strong>g the makeup of the<br />

control versus experimental students was provided,<br />

<strong>in</strong>clud<strong>in</strong>g their overall performance <strong>in</strong> precl<strong>in</strong>ical<br />

courses. As many operative faculty will attest to, students<br />

can vary widely <strong>in</strong> their <strong>in</strong>nate ability to grasp<br />

the psychomotor skills of operative dentistry, and at<br />

the very least, the control group versus experimental<br />

group performance <strong>in</strong> other psychomotor courses or<br />

other preparations <strong>in</strong> the same course should have<br />

been compared. It is possible that this situation could<br />

have been repeated between two randomized groups<br />

of ten students’ learn<strong>in</strong>g one operative procedure <strong>in</strong><br />

completely identical ways. If randomized selection<br />

was used, it would have been wise to have two control<br />

groups to strengthen any possible conclusions.<br />

If there was no statistical difference between the two<br />

control groups on the evaluation of a preparation <strong>in</strong><br />

any of the five criteria, differences of two or three of<br />

the five criteria between the two control groups and<br />

the VRBT experimental group may have been more<br />

valid. To use the data as presented <strong>in</strong> that publication<br />

to nullify a technology and state that this method<br />

of teach<strong>in</strong>g is unsuitable seems to be far from justified.<br />

Regardless of the design, it would appear unwise<br />

to most <strong>in</strong> dental education to make such a<br />

strong statement based on five hours of experience<br />

of ten students. Our experience <strong>with</strong> the VRBT spann<strong>in</strong>g<br />

over five years and <strong>in</strong>volv<strong>in</strong>g several generations<br />

of hardware and software, multiple studies,<br />

countless student hours, numerous surveys and student<br />

discussions, and many restorative procedures<br />

gives us the confidence that our results are valid and<br />

the decision to further embrace this technology is<br />

justified.<br />

December 2004 ■ Journal of Dental Education 1263


Conclusion<br />

Although our goal was to scientifically demonstrate<br />

the efficiency of VRBT <strong>in</strong> the education of<br />

psychomotor skills, we soon realized that this was a<br />

very difficult task due to various factors and limitations<br />

dur<strong>in</strong>g late 1990s when these units were first<br />

available. The limitations of low numbers of units<br />

and hence low numbers of students, technical problems,<br />

the desire to modify the educational environment<br />

as soon as we realized a better way to do it, and<br />

the frequent upgrades of software and hardware did<br />

not lend itself to the production of statistically significant<br />

data. Most of our data is suggestive, but is<br />

supportive of VRBT as an educational tool, and some<br />

data demonstrated statistically significant differences<br />

between experimental and control groups. But as Dr.<br />

Kathleen Rosen 20 remarks <strong>in</strong> a recent book devoted<br />

to health care simulation, “no one has ever proved<br />

the validity or value of simulation education and<br />

tra<strong>in</strong><strong>in</strong>g <strong>in</strong> any other <strong>in</strong>dustry” before its accepted<br />

use. Hence, it may be that we cannot wait to adapt<br />

VRBT simulation until unequivocal proof of the benefit<br />

had been demonstrated.<br />

In addition to the described research data we<br />

obta<strong>in</strong>ed, we formed op<strong>in</strong>ions of the value of this<br />

technology <strong>in</strong> many other, nonmeasurable ways.<br />

Considerable time was spent watch<strong>in</strong>g students <strong>in</strong><br />

the VRBT lab, talk<strong>in</strong>g <strong>with</strong> them at length, listen<strong>in</strong>g<br />

to them talk to the many visitors who came to observe,<br />

and watch<strong>in</strong>g students demonstrate the VRBT<br />

to others at our school and at national meet<strong>in</strong>gs. The<br />

sum of all our data, experience, evaluation, observations,<br />

and <strong>in</strong>tuition resulted <strong>in</strong> the belief that VRBT<br />

is a powerful educational methodology <strong>with</strong> the potential<br />

to significantly affect dental education. This<br />

belief was the basis for the purchase of fifteen VRBT<br />

units, allow<strong>in</strong>g for the full <strong>in</strong>corporation of this technology<br />

<strong>in</strong>to our courses and associated curriculum<br />

revision to help this technology realize its full potential<br />

<strong>in</strong> dental education.<br />

Acknowledgments<br />

I would like to thank Dr. Denise Stewart, Dr.<br />

Joan Gluch, Dr. Amer Abu-Hanna, Dr. Penny<br />

Hammrich, Dr. Francis Mante, Dr. Mamle Mante,<br />

and Dr. Eldad Irani for their help <strong>with</strong> these studies<br />

and Dr. Peter Berthold for help <strong>with</strong> the preparation<br />

of this manuscript.<br />

REFERENCES<br />

1. Rose JT, Buchanan JA, Sarrett DC. The DentSim system.<br />

J Dent Educ 1999;63(5):421-3.<br />

2. Plasschaert AM, Wilson NH, Cailleteau JG, Verdonschot<br />

EH. Op<strong>in</strong>ions and experiences of dental students and faculty<br />

concern<strong>in</strong>g computer-assisted learn<strong>in</strong>g. J Dent Educ<br />

1995;59:1034-40.<br />

3. Institute for Higher Education Policy. What’s the difference<br />

A review of contemporary research on the effectiveness<br />

of distance learn<strong>in</strong>g <strong>in</strong> higher education.<br />

Wash<strong>in</strong>gton, DC: Institute for Higher Education Policy,<br />

1998.<br />

4. F<strong>in</strong>kel D. Surgery moves <strong>in</strong>to the third dimension. Lancet<br />

1996;348(9043):1726-8.<br />

5. Smith C, Daniel P. Simulation technology: a strategy for<br />

implementation <strong>in</strong> surgical education and certification.<br />

Presence: Tele-operators and <strong>Virtual</strong> Environments<br />

2000;9(6).<br />

6. Buck GH. Development of simulators <strong>in</strong> medical education.<br />

Gesnerus 1991;48(pt 1):7-28.<br />

7. Robb N. Time to “reth<strong>in</strong>k models of medical education.”<br />

Can Med Assoc J 1997;157(30):304-5.<br />

8. Garden A, Rob<strong>in</strong>son B, Weller J, et al. Education to address<br />

medical error: a role for high fidelity patient simulation.<br />

N Z Med J 2002;115:133-4.<br />

9. Cohen L. Medical simulation is the wave of the future, U<br />

of O doctors say. Can Med Assoc J 1999;160:557.<br />

10. Satava RM. Accomplishments and challenges of surgical<br />

simulation: dawn<strong>in</strong>g of the next-generation surgical education.<br />

Surg Endosc 2001;15:232-41.<br />

11. Schwid HA. A flight simulator for general anesthesia.<br />

Comput Biomed Res 1987;20:64-75.<br />

12. Kennedy H. Simulation reshap<strong>in</strong>g military tra<strong>in</strong><strong>in</strong>g: technology<br />

jump<strong>in</strong>g from teenagers’ computers to pilots’ cockpits.<br />

National Defense Magaz<strong>in</strong>e, November 1999. At:<br />

www.nationaldefensemagaz<strong>in</strong>e.org/<br />

article.cfmpage=About. Accessed: August 10, 2004.<br />

13. Human Interface <strong>Technology</strong> Laboratory. At:<br />

www.hitl.wash<strong>in</strong>ton.edu/. Accessed: August 10, 2004.<br />

14. Ota D, Loft<strong>in</strong> B, Saito T, Lea R, Keller J. <strong>Virtual</strong> reality<br />

<strong>in</strong> surgical education. Comput Biol Med 1995;25(2):127-<br />

37.<br />

15. Merril G, Barker V. <strong>Virtual</strong> reality debuts <strong>in</strong> the teach<strong>in</strong>g<br />

laboratory <strong>in</strong> nurs<strong>in</strong>g. J Intra Nurs 1996;19(4):182-7.<br />

16. Satava R. <strong>Virtual</strong> reality, telesurgery, and the new world<br />

order of medic<strong>in</strong>e. J Image Guided Surg 1995;1(1):12-6.<br />

17. Gallagher AG, McClure N, McGuigan J, Crothers I, Brown<strong>in</strong>g<br />

J. <strong>Virtual</strong> reality tra<strong>in</strong><strong>in</strong>g <strong>in</strong> laparoscopic surgery: a prelim<strong>in</strong>ary<br />

assessment of m<strong>in</strong>imally <strong>in</strong>vasive surgical tra<strong>in</strong>er<br />

virtual reality. Endoscopy 1999;31(4):310-3.<br />

18. Tuggy ML. <strong>Virtual</strong> reality flexible sigmoidoscopy simulator<br />

tra<strong>in</strong><strong>in</strong>g: impact on resident performance. J Am Board<br />

Fam Pract 1998;11(6):426-33.<br />

19. Peugnet F, Dubois P, Rouland JF. <strong>Virtual</strong> reality versus<br />

conventional tra<strong>in</strong><strong>in</strong>g <strong>in</strong> ret<strong>in</strong>al photocoagulation: a first<br />

cl<strong>in</strong>ical assessment. Comput Aided Surg 1998;3(1):20-6.<br />

20. Loyd G, Lake C, Greenberg R, eds. Practical health care<br />

simulations. Philadelphia: Elsevier Mosby, 2004.<br />

1264 Journal of Dental Education ■ Volume 68, Number 12


21. Buchanan J, Williams J. Simulation <strong>in</strong> dentistry and oral<br />

surgery. In: Loyd G, Lake C, Greenberg R, eds. Practical<br />

health care simulations. Philadelphia: Elsevier Mosby,<br />

2004:493-512.<br />

22. Buchanan JA. Use of simulation technology <strong>in</strong> dental<br />

education. J Dent Educ 2001;65:1225-31.<br />

23. LeBlanc VR, Urbankova A, Hadavi F, Lichtenthal RM. A<br />

prelim<strong>in</strong>ary study <strong>in</strong> us<strong>in</strong>g virtual reality to tra<strong>in</strong> dental<br />

students. J Dent Educ 2004;68(3):378-83.<br />

24. Abstract by Case Western Reserve University, Cleveland,<br />

OH.<br />

25. Imber S, Shapira G, Gordon M, Judes H, Metzger Z. A<br />

virtual reality dental simulator predicts performance <strong>in</strong><br />

an operative dentistry manik<strong>in</strong> course. Eur J Dent Educ<br />

2003;7:160-3.<br />

26. Qu<strong>in</strong>n F, Keogh P, McDonald A, Hussey D. A study compar<strong>in</strong>g<br />

the effectiveness of conventional tra<strong>in</strong><strong>in</strong>g and virtual<br />

reality simulation <strong>in</strong> the skills acquisition of junior<br />

dental students. Eur J Dent Educ 2003;7:164-9.<br />

27. Qu<strong>in</strong>n F, Keogh P, McDonald A, Hussey D. A pilot study<br />

compar<strong>in</strong>g the effectiveness of conventional tra<strong>in</strong><strong>in</strong>g and<br />

virtual reality simulation <strong>in</strong> the skills acquisition of junior<br />

dental students. Eur J Dent Educ 2003;7:13-9.<br />

28. Kolb DA. Learn<strong>in</strong>g style <strong>in</strong>ventory. Hay McBer Resources<br />

Group technical manual. Boston: <strong>Experience</strong> <strong>Based</strong><br />

Learn<strong>in</strong>g Systems, Inc., 1999.<br />

29. Learn<strong>in</strong>g Type Measure. The 4Mat System. Oak Brook,<br />

IL: Excel, Inc., 1996.<br />

December 2004 ■ Journal of Dental Education 1265

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!