06.03.2013 Views

Artificial Intelligence and Soft Computing: Behavioral ... - Arteimi.info

Artificial Intelligence and Soft Computing: Behavioral ... - Arteimi.info

Artificial Intelligence and Soft Computing: Behavioral ... - Arteimi.info

SHOW MORE
SHOW LESS

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

21.2.1 Qualitative Methods for Performance Evaluation<br />

Validation of an expert system is carried out by qualitative means. Qualitative<br />

Validation employs subjective comparison of performances [4]. This section<br />

reviews a few methods for qualitative validation for expert systems. Among<br />

the qualitative validation schemes, the most popular is Turing test [7] <strong>and</strong><br />

sensitivity analysis.<br />

Turing Test: This test is named after Alan Turing, a pioneering AI<br />

researcher. Turing’s view on intelligence can be stated as follows. A problem<br />

solving machine is intelligent, if a person trying to assess the intelligence of<br />

the machine cannot determine its identity, while working with a human<br />

problem solver <strong>and</strong> a computing machine. This method of testing intelligence<br />

of a machine is called Turing test. Turing’s test has successfully been applied<br />

to evaluate performance of expert systems like MYCIN <strong>and</strong> ONCOCIN [4].<br />

For assessing the performance of an expert system by Turing’s test, the<br />

expert based on whose expertise the system is built <strong>and</strong> the expert system are<br />

assigned common problems. The assessment of the system performance is<br />

then compared with that of a human expert by another expert, without<br />

disclosing the performer’s identity. If assessments can be measured<br />

objectively, then statistical techniques may be employed to measure the<br />

variations or consistency between the performance of the expert <strong>and</strong> the<br />

system. Examples of objective measurement of performance include number<br />

of correct inferences or the level of performance in grade points like excellent,<br />

good, fair, poor, etc. Turing’s test, however, is not applicable to systems,<br />

where finding objective measurement of performance itself is a complex<br />

problem <strong>and</strong> thus is not amenable.<br />

Input variables<br />

v1<br />

Prob.i v3<br />

p1 p2 p3<br />

Expert System<br />

Expert<br />

inferences<br />

inferences<br />

Fig. 21.5: Validating an expert system performance through<br />

sensitivity analysis.<br />

Performance<br />

Evaluation

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!