How to Design and Evaluate Research in Education

6 PART 1 Introduction to Research www.mhhe.com/fraenkel7ereach an intelligent decision about a problem that isbothering us. These procedures can be boiled down tofive distinct steps.1. First, there is a problem of some sort—some disturbancein our lives that disrupts the normal or desirablestate of affairs. Something is bothering us. For most ofus who are not scientists, it may be a tension of somesort, a disruption in our normal routine. Exampleswould be if our students are not as attentive as we wishor if we have difficulty making friends. To the professionalscientist, it may be an unexplained discrepancyin one’s field of knowledge, a gap to be closed. Or itcould be that we want to understand the practice ofhuman sacrifice in terms of its historical significance.2. Second, steps are taken to define more precisely theproblem or the questions to be answered, to becomeclearer about exactly what the purpose of the study is.For example, we must think through what we mean bystudent attentiveness and why we consider it insufficient;the scientist must clarify what is meant by humansacrifice (e.g., how does it differ from murder?).3. Third, we attempt to determine what kinds of informationwould solve the problem. Generally speaking,there are two possibilities: study what is alreadyknown or carry out a piece of research. As you willsee, the first is a prerequisite for the second; the secondis a major focus of this text. In preparation, wemust be familiar with a wide range of possibilitiesfor obtaining information, so as to get firsthand informationon the problem. For example, the teachermight consider giving a questionnaire to students orhaving someone observe during class. The scientistmight decide to examine historical accounts orspend time in societies where the practice of humansacrifice exists (or has until recently). Spelling outthe details of information gathering is a major aspectof planning a research study.4. Fourth, we must decide, as far as it is possible, howwe will organize the information that we obtain. Itis not uncommon, in both daily life and research, todiscover that we cannot make sense of all the informationwe possess (sometimes referred to as informationoverload). Anyone attempting to understandanother society while living in it has probably experiencedthis phenomenon. Our scientist will surely encounterthis problem, but so will our teacher unlessshe has figured out how to handle the questionnaireand/or observational information that is obtained.5. Fifth, after the information has been collected andanalyzed, it must be interpreted. While this step mayseem straightforward at first, this is seldom the case.As you will see, one of the most important parts ofresearch is to avoid kidding ourselves. The teachermay conclude that her students are inattentive becausethey dislike lectures, but she may be misinterpretingthe information. The scientist may concludethat human sacrifice is or was a means of trying tocontrol nature, but this also may be incorrect.In many studies, there are several possible explanationsfor a problem or phenomenon. These are called hypothesesand may occur at any stage of an investigation. Some researchersstate a hypothesis (e.g., “Students are less attentiveduring lectures than during discussions”) right at thebeginning of a study. In other cases, hypotheses emerge asa study progresses, sometimes even when the informationthat has been collected is being analyzed and interpreted.The scientist might find that instances of sacrifice seemedto be more common after such societies made contact withother cultures, suggesting a hypothesis such as: “Sacrificeis more likely when traditional practices are threatened.”We want to stress two crucial features of scientificresearch: freedom of thought and public procedures. Atevery step, it is crucial that the researcher be as open ashumanly possible to alternative ways of focusing andclarifying the problem, collecting and analyzing information,and interpreting results. Further, the processmust be as public as possible. It is not a private game tobe played by a group of insiders. The value of scientificresearch is that it can be replicated (i.e., repeated) byanyone interested in doing so.*The general order of the scientific method, then, is asfollows:Identifying a problem or questionClarifying the problemDetermining the information needed and how toobtain itOrganizing the informationInterpreting the resultsIn short, the essence of all research originates incuriosity—a desire to find out how and why things happen,including why people do the things they do, as wellas whether or not certain ways of doing things work betterthan others.*This is not to imply that replicating a study is a simple matter. Itmay require resources and training—and it may be impossible torepeat any study in exactly the same way it was done originally. Theimportant principle, however, is that public evidence (as opposed toprivate experience) is the criterion for belief.