Hindko and Gujari. c - SIL International

A.2 Recorded Text Testing 207
consistent scores are interpreted to be reflections of the inherent intelligibility
between the related varieties. Increasing the number of subjects should not
significantly increase the range of variation of the scores.
However, when some subjects have had significant previous contact with
the speech form recorded on the test, while others have not, the scores should
vary considerably, reflecting the degree of learning that has gone on through
contact. For this reason it is important to include a measure of dispersion which
reflects the extent to which the range of scores varies from the mean — the
standard deviation. If the standard deviation is relatively low, say 10 or below
on a test with 100 possible points (that is, 100 percent), and the mean score for
subjects from the selected test point is high, the implication is that the
community as a whole probably understands the test variety rather well simply
because the variety represented in the recording is inherently intelligible. If the
standard deviation is relatively low and the mean comprehension score is also
low, the implication is that the community as a whole understands the test
variety rather poorly and that regular contact has not facilitated learning of the
test variety to any significant extent. If the standard deviation is high, regardless
of the mean score, one implication is that some subjects have learned to
comprehend the test variety better than others. 5 In this last case, any inherent
intelligibility between the related varieties is mixed with acquired
comprehension which results from learning through contact. 6
Much care was taken in the recorded text testing in these sociolinguistic
surveys, thus the results are discussed with the assumption that the effects from
intervening factors were either negligible or were interpreted appropriately.
However, in contrast to experimentally controlled testing in a laboratory
situation, the results of field administered methods such as the RTT cannot be
completely isolated from all potential biases. It is therefore recommended that
results from recorded text tests not be interpreted in terms of fixed numerical
thresholds, but rather be evaluated in light of other indicators of intelligibility,
such as word lists and dialect opinions, and according to patterns of contact and
communication.
RTTs in Second Language Testing
The procedures of recorded text testing as used for evaluating
comprehension of a second language are similar to those used for dialect
comprehension testing. A personal experience text is prepared by a mother
tongue speaker of the target language. It is validated to be a clear and
5 High standard deviations can result from other causes, such as
inconsistencies in the circumstances of test administration and scoring or
differences in attentiveness or intelligence of test subjects. The researchers
involved in recorded text testing need to be aware of the potential for skewed
results due to such factors, and control for them as much as possible through
careful test development and administration.
6 Questionnaires administered at the time of testing, then, can help
discover which factors are significant in promoting such contact. Travel to trade
centers, radio broadcasts, and intermarriage are examples of the type of channel
through which contact with another dialect can occur. Sample questionnaires
are given in the appendices of the different surveys.