Quality of the estimate. December, p. 47 - Health Care Compliance ...

Confidence and precision in claims audits: Quality of the estimate ...continued from page 47
December 2011
48
of certainty that the sample correctly
depicts the universe) and
precision (i.e., range of accuracy)
are the third piece needed for
a quality estimate of the total
overpayment in the universe.
Precision and confidence
trade–off
The quality of the estimate depends
on the precision and confidence
levels reached in the estimation process.
Both high confidence and high
precision are desirable qualities in
an overpayment estimate. They are
expressed in ranges or percentages.
There are one- and two-sided confidence
intervals that are reported
at various levels of precision.
One-sided confidence intervals deal
with whether the true value for the
universe (e.g., total overpayment) is
greater or smaller than the estimate.
Two-sided confidence intervals deal
with whether the true value for
the universe is between two given
numbers (lower and upper bounds),
i.e., a bounded range.
In overpayment extrapolations,
the quality of the estimate of
total overpayment in the universe
can be described as a range or a
percentage for the so-called “point
estimate.” The point estimate is
simply the average overpayment in
the sample inflated by the universe
size. Using the point estimate views
the universe as just a larger version
of the sample. The observed
confidence and precision levels are
statistical measures calculated from
the data of a particular sample
and qualify that view by reporting
the uncertainty that is associated
with the particular point estimate.
Namely, confidence and precision
allow us to give a range of uncertainty
around the point estimate.
This range implies that the random
sample that was drawn is not an
exact miniature version of the
universe. Each sample may render
a somewhat different picture,
hence a range.
For example, if an auditor reports
the overpayment estimate of
$10,000 with a two-sided 90%
confidence interval and 5%
precision level, this means that the
auditor is 90% certain that the
true overpayment value for the
universe is $10,000 +/- $500, (i.e.,
is between $9,500 and $10,500).
The $500 is the precision amount
and the precision percentage is
5%. Clearly, if we could raise the
confidence level to 95% or even
99% for that same precision level
of 5%, that would render a higher
quality estimate. Similarly, if we
could tighten the precision range
and thereby make the estimate
more precise at 90% confidence
level (e.g., make it +/- 3% or
equivalent to a $600 range), that
would also be preferable.
For any given sample, the auditor
can always raise the certainty
(i.e., raise the confidence level) by
making the statement about the
estimate in relation to the true
Health Care Compliance Association • 888-580-8373 • www.hcca-info.org
value in the universe less precise.
The extreme position may be
illustrative. For example, one can
always say for the point estimate
that one is 100% confident that the
true value is between minus infinity
and plus infinity (i.e., pairing complete
imprecision with complete
certainty/confidence). Any other
more meaningful combinations
of confidence/precision levels for
the same point estimate that was
generated from one-and-the-same
sample are just re-statements of
an inherent tradeoff. This tradeoff
renders the same statistical information.
To conclude, whenever
the sample size is fixed and the
point estimate is calculated, there
is a tradeoff between confidence
and precision levels. One cannot
improve both any more.
Another valid method of stating
confidence and precision levels
that is sometimes used for point
estimates in overpayment cases is
using the one-sided confidence
interval. In this approach, the auditor
typically states the confidence
level of the point estimate at a
lower limit. Rather than reporting
a bounded range around the
estimate, the range becomes openended.
For example, the auditor
may report a point estimate of
$10,000 for a one-sided 90%
confidence interval with a $9,650
lower limit. This means the auditor
reports a point estimate of $10,000
for which he/she is 90% confident
that the true overpayment in the