18th annual conference on manual control.pdf - Acgsc.org

Significance Testing
Two techniques were employed to determine the significance
of practice-related parameter differences: paired-difference t-
tests performed on the model parameters, and the qualitative
cross-comparison scheme proposed by Levison [6,7]. Application
of the t-test was the same as would be applied to the basic
tracking data (say, mean-squared error scores), except that the
identified pilot parameters served as the "data". As a practical
matter, this procedure was limited to analysis of population
means, with subject-paired early/late
,
differences used in the
computation of the "t" statistic.
The cross-comparison technique was applied to both
individual subjects and to subject populations. To perform this
test, three sets of model parameters were identified for a given
tracking task: (i) the set that best matched the early data, (2)
the set that best matched the late data, and (3) the set that
best matched the average of early and late performance. Four
scalar matching errors were computed:
J(E,E) = matching error obtained from the early data, using
parameters identified from the early data (i.e., best
match to the early data).
J(E,A) = matching error obtained from the early data, using the
average parameter set.
J(L,L) = best match to late data.
J(L,A) = matching error obtained from late data, using the
average parameter set.
Finally, an average "matching error ratio" was computed as
[J(E,A)/J(E,E) + J(L,A)/J(L,L)]/2.
The matching error ratio relates to the degradation in
model-matching capability when an average set of parameters is
used instead of the best-fitting parameters. If the average
parameter set matches the data as well as the best-fitting set,
then the matching error ratio is unity, and we accept the null
hypothesis that there is no significant effect of practice on
Significance testing for individual subjects would require
parameter identification for a number of individual trials per
subject, which would generally entail significant computational
requirements.
243