THE PLURAL OF ANECDOTE IS NOT DATA:1 ... - UDC Law Review

\\jciprod01\productn\D\DCR\16-1\DCR104.txt unknown Seq: 16 29-JAN-13 10:2632 UNIVERSITY OF THE DISTRICT OF COLUMBIA LAW REVIEWAs a result of the pretesting phase, students discovered that some of the questionsin the draft survey were ambiguous. While the questions seemed clear whenthe students wrote them, people who took the survey during the pretesting phaseexpressed uncertainty about what some of the questions meant. Also, one respondenthad difficulty with the way some of the questions were asked. The surveywas designed on a four-point Likert scale that asked respondents to saywhether they strongly agreed, agreed, disagreed, or strongly disagreed with aquestion. The draft survey also had a fifth column for “n/a” or “not applicable,”but during the pretest one person checked “n/a” in answer to every question. Weremoved that choice from the survey and revised the survey so that every questionapplied to both employers and employees.Students also learned the effect that the length and complexity of the surveyquestionnaire had on the rate of participation. 85 Ideally, a survey should ask allthe essential questions, but not be so long that it burdens respondents. 86 Afterseveral people agreed to take the survey but left before finishing it, the studentsconcluded that they needed to limit questions to one page and to make the surveymore visually appealing and easier to complete. They also decided to change theorder of the questions and group similar questions together.One of the most important findings from the pretesting centered on the importanceof finding a suitable place to administer the survey. 87 The original plancalled for inviting participants into an OAH conference room, off one of the hallways,near where hearings were conducted. The hope was that this room wouldoffer participants privacy in which to complete the survey. Several problems immediatelypresented themselves: The conference room was near where the administrativeassistants worked and was often in use. As one student noted,“[c]onducting the survey in the conference room might be misleading. I am afraidthat if we do it in there, people might view us as an agent of the court system.” 88Also, the conference room was situated either down the hall or across from thehearing rooms, and using it would have diverted at least some participants awayfrom the exit, the direction they were heading at the conclusion of their hearings.Finally, there was no place for students to wait unobtrusively for a hearing toconclude.These problems were obvious during the pretesting, so students attempted toadminister the survey in the waiting room, a small area, with parties sitting inthree rows of chairs. This arrangement was not conducive to students introducingthemselves to parties and explaining their purpose or to intercepting parties asthey left the hearings, and it afforded little privacy for completing the survey.85 See FOWLER, supra note 64. R86 Id.87 Id.88 Memorandum from Laura Kakuk, Law Student, Columbus Sch. of Law, to authors (Oct. 26,2009) (on file with authors).