a primer on verbal protocol analysis - Ammon Wiemers' Home Page

C35165/Schmorrow Page 335
A Primer on Verbal Protocol Analysis 335
DATA COLLECTION
A number of issues related to data collection should be addressed before the
first participant even arrives! Several confidentiality issues surround the act of
collecting data, which stem from the potentially sensitive nature of video and/or
audio recording participants as they perform tasks. First, in obtaining informed
consent, it is crucial that participants fully understand the recording process and
have a specific option to agree or disagree to have their words, actions, and bodies
recorded. Thus, consent forms need to contain two lines, as follows:
____ I agree to have my voice, actions, and upper body videotaped.
____ I do not agree to have my voice, actions, and upper body videotaped.
Of course, the actual wording will depend on the scope of recording planned—it
might additionally include gestures or walking patterns, for example.
Second, agreement to have one’s words and actions recorded should not be
taken as agreement to have one’s words and actions shared with others. Although
participants might be quite willing to participate in a study and to be recorded
while doing so, they might be uncomfortable at the thought of their data being
shared, for example, at conferences and other research presentations. If the
researcher is planning to use video clips as illustrations during research talks, it
is important to have an additional section of the consent form that specifically
asks participants whether or not they agree to have their data shared. They have
the right to refuse, and this right must be honored. As mentioned earlier, this does
not necessarily disqualify participants from participating in the study; it just
means that a participant’s video cannot be shared with others. The third issue
relating to confidentiality is that of storing the data. It is especially important to
reassure participants that the data will be stored confidentially—that data will
be identified by a number rather than a participant’s name, that data will be
locked away or otherwise secured, and that access to the data will be strictly limited
to members of the research team.
One of the advantages of verbal protocol data is its richness; the downside of
this richness is that the data quickly become voluminous. Even a relatively short
session can result in pages of transcribed protocol that must be coded and analyzed.
In addition, participants, such as experts, may be scarce and difficult to
recruit. For these reasons, verbal protocol studies generally have fewer subjects
than other kinds of psychological studies. How many participants are needed
The answer depends on the nature of the study. We (and others) have had as
few as one person participate in a particular study. Such single-subject case studies
are extremely useful for generating hypotheses that can then be tested experimentally
on a larger sample. In general, however, we have found that the more
expert the participants, the less variance there is in relevant aspects of performance.
With less expert participants, we generally find slightly higher numbers
of participants work well. Ideally, we try to include 5 to 10 novices, although this
is not always possible. Studies with larger numbers of participants generally
involve less exploratory research, for which a coding scheme is already well
established and for which only a subset of the data needs to be coded and