Artificial Intelligence and Soft Computing: Behavioral ... - Arteimi.info

Paired t-test: Suppose xi ∈ X and yi ∈ Y are two members of the sets of
random numbers X and Y. Let X and Y be the inferences derived by the
expert system and the expert respectively. Also assume that for each xi there
exists a unique corresponding yi. The paired t-test computes the deviation di =
xi – yi, ∀i and evaluates the standard deviation Sd and mean d for di, 1≤ i≤ n,
where there exist n samples of di. A confidence interval for derived d is now
constructed as follows:
d – t n – 1, α ≤ d ≤ d + t n – 1, α
where t n – 1, α is the value of the t-distribution with n-degrees of freedom and
a level of confidence α. The hypothesis H0 is accepted, if d = 0 lies in the
above interval.
One main difficulty of realizing the t-test is that the output variables of
the system have to be quantified properly. The method of quantification of the
output variables for many systems, however, is difficult. Secondly, paired ttest
should be employed to expert systems having a single output variable xi
(and yi for the expert), which may be obtained for n cases. It may be added
that for multivariate (i.e., systems having more than one variable) responses, ttest
should not be used, as the variables xi may be co-related and thus the
judgement about performance evaluation may be erroneous. Hotelling’s one
sample T 2 -test may be useful for performance evaluation for multivariate
expert systems.
Hotelling’s one sample T 2 -test: Suppose that the expert system has m
output variables. Thus for k set of input variables, there must be k output
vectors, each having m scaler components. The expert, based on whose
reference the performance of the system will be evaluated, should also
generate k output vectors, each having m components. Let the output vectors
of the expert system be [Xi]m x 1,1≤ i ≤k and the same generated by the expert
be [Yi ]m x 1,1≤i ≤m. We now compute error vector Ei = Xi – Yi , 1≤∀i ≤k and
the mean ( ) of error vectors Ei , 1