How to Design and Evaluate Research in Education

16 PART 1 Introduction to Research www.mhhe.com/fraenkel7ethe particular situation, but may leave it to the reader toassess applicability. When they do generalize, their generalizationsare usually very limited in scope.Many of the distinctions just described, of course,are not absolute. Sometimes researchers will use bothqualitative and quantitative approaches in the samestudy. This kind of research is referred to as mixedmethodsresearch. Its advantage is that by using multiplemethods, researchers are better able to gather and analyzeconsiderably more and different kinds of data thanthey would be able to using just one approach. Mixedmethodsstudies can emphasize one approach over theother or give each approach roughly equal weight.Consider an example. It is often common in surveysto use closed-ended questions that lend themselves toquantitative analysis (such as through the calculation ofpercentages of different types of responses), but alsoopen-ended questions that permit qualitative analysis(such as following up a response that interviewees giveto a particular question with further questions by theresearcher in order to encourage them to elaborate andexplain their thinking).Studies in which researchers use both quantitativeand qualitative methods are becoming more common,as we will see in Chapter 23.META-ANALYSISMeta-analysis is an attempt to reduce the limitations ofindividual studies by trying to locate all of the studieson a particular topic and then using statistical means tosynthesize the results of these studies. In Chapter 5, wediscuss meta-analysis in more detail. In subsequentchapters, we examine in detail the limitations that arelikely to be found in various types of research. Someapply to all types, while others are more likely to applyto particular types.Critical Analysis of ResearchThere are some who feel that researchers who engage inthe kinds of research we have just described take a bittoo much for granted—indeed, that they make a numberof unwarranted (and usually unstated) assumptionsabout the nature of the world in which we live. Thesecritics (usually referred to as critical researchers) raisea number of philosophical, linguistic, ethical, and politicalquestions not only about educational research as itis usually conducted but also about all fields of inquiry,ranging from the physical sciences to literature.In an introductory text, we cannot hope to do justiceto the many arguments and concerns these critics haveraised over the years. What we can do is provide anintroduction to some of the major questions they haverepeatedly asked.The first issue is the question of reality: As any beginningstudent of philosophy is well aware, there is noway to demonstrate whether anything “really exists.”There is, for example, no way to prove conclusively toothers that I am looking at what I call a pencil (e.g., othersmay not be able to see it; they may not be able to tellwhere I am looking; I may be dreaming). Further, it iseasily demonstrated that different individuals maydescribe the same individual, action, or event quitedifferently—leading some critics to the conclusionthat there is no such thing as reality, only individual(and different) perceptions of it. One implication of thisview is that any search for knowledge about the “real”world is doomed to failure.We would acknowledge that what the critics say iscorrect: We cannot, once and for all, “prove” anything,and there is no denying that perceptions differ. Wewould argue, however, that our commonsense notion ofreality (that what most knowledgeable persons agreeexists is what is real) has enabled humankind to solvemany problems—even the question of how to put a manon the moon.The second issue is the question of communication.Let us assume that we can agree that some things are“real.” Even so, the critics argue that it is virtually impossibleto show that we use the same terms to identifythese things. For example, it is well known thatthe Inuit have many different words (and meanings)for the English word snow. To put it differently, nomatter how carefully we define even a simple termsuch as shoe, the possibility always remains that oneperson’s shoe is not another’s. (Is a slipper a shoe? Isa shower clog a shoe?) If so much of language isimprecise, how then can relationships or laws—whichtry to indicate how various terms, things, or ideas areconnected—be precise?Again, we would agree. People often do not agree onthe meaning of a word or phrase. We would argue, however(as we think would most researchers), that we candefine terms clearly enough to enable different peopleto agree sufficiently about what words mean that theycan communicate and thus get on with the acquisition ofuseful knowledge.The third issue is the question of values. Historically,scientists have often claimed to be value-free, that is