Contents

220 PART III: Experimental Methodsdifferent rooms or different experimenters, must not be allowed to confoundthe independent variable of interest.A more serious threat to the internal validity of the random groups designis involved when subjects fail to complete the experiment successfully. Selectivesubject loss occurs when subjects are lost differentially across the conditionsand some characteristic of the subject that is related to the outcome ofthe experiment is responsible for the loss. We can help prevent selective lossby restricting subjects to those likely to complete the experiment successfully,or we can compensate for it by selectively dropping comparable subjectsfrom the group that did not experience the loss. Demand characteristics andexperimenter effects can be minimized through the use of proper experimentalprocedures, but they can best be controlled by using placebo control anddouble-blind procedures.Data analysis and statistics provide an alternative to replication for determiningwhether the results of a single experiment can be used as evidence toclaim that an independent variable has had an effect on behavior. Data analysisinvolves the use of both descriptive statistics and inferential statistics. Describingthe results of an experiment typically involves the use of means, standarddeviations, and measures of effect size. Meta-analysis makes use of measures ofeffect size to provide a quantitative summary of the results of a large number ofexperiments on an important research question.Inferential statistics are important in data analysis because researchers needa way to decide whether the obtained differences in an experiment are dueto chance or are due to the effect of the independent variable. Confidence intervalsand null hypothesis testing are two effective statistical techniques researcherscan use to analyze experiments. Statistical analysis cannot guarantee,however, that experimental findings will be meaningful or be of practical significance.Replication remains the ultimate test of the reliability of a researchfinding.Researchers also strive to establish the external validity of their experimentalfindings. When testing psychological theories, researchers tend toemphasize internal validity over external validity. One effective approachfor establishing the external validity of findings is to select representativesamples of all dimensions on which you wish to generalize. By conductingfield experiments, researchers can increase the external validity of their researchfindings to real-world settings. Partial replications and conceptualreplications are two common ways that researchers use to establish externalvalidity.The matched groups design is an alternative to the random groups designwhen only a small number of subjects is available, when a good matchingtask is available, and when the experiment requires separate groups foreach treatment. The biggest problem with the matched groups design is thatthe groups are equated only on the characteristic measured by the matchingtask. In the natural groups design, researchers select the levels of independentvariables (usually individual differences, or subject, variables) andlook for systematic relationships between these independent variables andother aspects of behavior. Essentially, the natural groups design involves