Chapter 6

It0The Inprovenent GuideIn most tests of change, conditions during the test are a more important considerationthan sample size (the number of items tested). A change may have theoredicted effect in the short term, but most tests also have the aim of determiningwhether the effect will persist in the future. Since conditions will naturallychange over dme, more information is usually obtained from a sample selectedover a long period than from a larger sample collected over a short period. (Onecycle or several cycles might be used to provide an appropriate period ) The timerequired to increase degree of belief that the change will result in improvementand that the improvement will persist is a matter of iudgement.The importance of time in determining if a change is an improvement sets a contextfor determining sample size for the test. The sample size needs to be adequateto detect pattems that indicate improvement. Some ruIes of thumb for determiningthe number ofplotted poins necessary to detect pattems on a Iun chart are:Number oJ PointsFewer than 10l) Io fu50 to 100More than I00SituationExpensive tests, expensive prototypes, or long periodsbetween available data poins; large effects anticipated.Usually sufficient to discem patterns indicating improvementsthat are moderate or large.The effect of the change is expected to be small relativeto the variation in the system.The chanee is intended to affect a rare event.Usually, fifteen to twenty-frve plotted points will be sufficient to recognize patternsindicating improvement. Sometimes as many as fifty poins might be necessarylf no historical data are available to establish a baseline before the changeis made. From fifty to one hundred points would be necessary only when theeffect of the change is anticipated to be small relative to the variation in ihe system.Examples are situations when the variation in the measuremenG themselvesare large or when the variations among people-students, Patients, or customers-canmask the effect of the change. More than one hundred poins mightbe needed if the change was intended lo affect a rare event, such as the side effectof a new drug or a serious but rarely occurring defect. In cases such as these, itmight be wise to consult a statistician to help in the design of the test.In certain situations it is not practical to collect fifteen or more points duringthe testing of a change. Protot)?es might be very expensive, such as prototypesof a new automobile engine, or the cost of conducting the test might be high.Possibly the data is financial in nature and only available on a monthly basis. Incases such as these, it may be that some data is better than none However, ifonly small amounts of data are available, their wonh can be enhanced by inciudinga wide variety of conditions in the test.len l)systcharditicSmnthe (belieof tepeobilit:ent (diffeISomcon!ci.enditirreas1inclwhilectrd"glncrovetheave2tesltobtestSlS Iwhi3intdorterpatisixljudinc