Contents

348 PART V: Analyzing and Reporting ResearchBOX 11.1THREE STAGES OF DATA ANALYSISThe three major stages of data analysis can bedescribed as follows:I Getting to Know the Data In the first stage wewant to become familiar with the data. This is anex ploratory or investigative stage (Tukey, 1977).We inspect the data carefully, get a feel for it, andeven, as some experts have said, “make friends”with it (Hoaglin, Mosteller, & Tukey, 1991, p. 42).Questions we ask include, What is going on in thisnumber set? Are there errors in the data? Do thedata make sense or are there reasons for “suspectingfishiness” (Abelson, 1995, p. 78)? Visualdisplays of distributions of num bers are importantat this stage. What do the data look like? Only whenwe have become familiar with the general featuresof the data, have checked for errors, and have assuredourselves that the data make sense, shouldwe proceed to the second stage.II Summarizing the Data In the second stage weseek to summarize the data in a meaningful way.The use of descriptive statistics and creation ofgraphical dis plays are important at this stage.How should the data be organized? Which waysof describing and sum marizing the data are mostinformative? What hap pened in this study as afunction of the factors of interest? What trends andpatterns do we see? Which graphical display bestreveals these trends and patterns? When the dataare appropriately sum marized, we are ready tomove to the confirmation stage.III Confirming What the Data Reveal In the thirdstage we decide what the data tell us about behavior.Do the data confirm our tentative claim (researchhypothesis) made at the beginning of the study?What can we claim based on the evidence? Sometimeswe look for a categorical, yes-no judgment,and act as judge and jury to render a verdict. Do wehave evidence to convict? Yes or no: Is the effectreal? At this stage we may use various statisticaltechniques to counter ar gu ments that our resultsare simply “due to chance.” Null hypothesis testing,when appropriate, is per formed at this stage ofanalysis. Our evaluation of the data, however, neednot always lead us to a cate gorical judgment aboutthe data (e.g., Schmidt, 1996). We don’t, in otherwords, have to attempt a definitive statement aboutthe “truth” of the results. Our claim about behaviormay be based on an evaluation of the probable rangeof effect sizes for the variable of interest. What, inother words, is likely to happen when this variable ispresent? Confidence intervals are particularly recommendedfor this kind of evaluation (e.g., Cohen, 1995;Hunter, 1997; Loftus, 1996).The confirmation process actually begins at thefirst or exploratory stage of data analysis, whenwe first get a feel for what our data are like. Aswe examine the general features of the data, westart to appreciate what we found. In the summarystage we learn more about trends and patternsamong the observations. This provides feedbackthat helps to confirm our hypotheses. The finalstep in data analysis is called the confirmationstage to empha size that it is typically at this pointwhen we come to a decision about what the datamean. Information obtained at each stage of dataanalysis, however, contributes to this confirmatoryprocess (e.g., Tukey, 1977).THE ANALYSIS STORY• When data analysis is completed, we must construct a coherent narrativethat explains our findings, counters opposing interpretations, and justifiesour conclusions.Making a convincing argument for a claim about behavior requires morethan simply analyzing the data. A good argument requires a good story. A trialattorney, in order to win a case, not only must call a jury’s attention to the factsof a case, but also must be able to weave those facts into a coherent and logicalstory. If the evidence points to the butler, then we want to know “why” the