SO - Health Care Compliance Association

By Wendy Rotz and Sara Dorn-HavlikJuly 200110Editor’s Note: Ms. Rotz and Ms. Dorn-Havlik are from the QuantitativeEconomics and Statistics practice at Ernst &Young LLP in Washington, D.C. They maybe reached on 202/327-7822.Sampling of medical records inhealth care has many regulatoryand non-regulatory applications.These include CorporateIntegrity Agreements (CIAs), providerself-disclosure studies, quality improvementefforts, and administrative initiatives.The cost of these studies is directlytied to the sample size.Understanding key factorsTen key factors affect sample size andultimately the cost of these studies.Understanding the key factors and howthey relate to sample size may helpestablish clear sample goals, plan moreefficient studies, and negotiate less costlyCIAs.Typically, a random sample is selectedfrom a data file (or sampling frame) consistingof medical records. These recordscomprise the population or universe of allrecords of interest for the study, forexample, all billed CPT codes from aparticular department.A sample of records is used to estimatea value for the population, such as thetotal amount overpaid by an insurer.Below are common factors that affectsample sizes in this typical setting.The ten factors1. Variance. A major driver of samplesizes is variance, which measures theinherent spread or variability in data.In general, more consistent data requiresmaller sample sizes and highly variabledata require larger samples. For example,when the overpaid amounts perrecords are consistently between $30and $50 a smaller sample is neededthan when the overpaid amounts arebetween $30 and $600 with erraticamounts as low as $5 or as high as$4,500. Variance is a characteristic ofthe data itself. It cannot be altered;however, it can be controlled throughefficient sample designs.2. Sample design. The choice of sampledesign has a significant impact onthe sample size. A simple random sample,one where each item in the populationhas an equal chance of selection, usuallyre q u i res an inefficient large sample, especiallywhen the data are highly va r i a b l e .By contrast, the size of a stratified randomsample can be 30 to 70% smaller.In stratified designs, the population isdivided into homogenous groups, orstrata, and each stratum is sampled separately.This controls variation andreduces the sample size. Other complexdesigns may provide smaller samplesizes in the right settings. A statisticianmay help determine the right design foryour data.3. Confidence and precision requirements.The desired level of reliability forthe estimates has a major impact on thesample size. Accuracy re q u i rements forestimates are typically specified usingc o n f i d e n c eand p re c i s i o n. Confidencedescribes the chances of drawing a samplethat will produce an estimate in theb a l l p a rk of the true value, and the pre c i-sion describes how large the ballpark is.Higher confidence levels and better precisionre q u i re larger samples.Precision requirements can be specifiedin terms of the desired margin of error,or absolute precision. This is the plus orminus amount reported with an estimate.For example, an overpayment isestimated to be $100,000 ± $20,000.Precision can also be specified with relativeprecision, which is the margin oferror divided by the estimate. In thisexample, it is 20,000/$100,000=20%.How the precision is specified caninfluence the sample size. Paradoxically,smaller estimates require larger samplesizes to meet relative precision goals.When a small error is anticipated, anabsolute precision goal is more easilyattained.Specifying the right level of confidence