5. Public Reporting as a Quality Improvement Strategy

The public reporting intervention in eight studies about choice of health plans was public
reports or plan rating sheets produced by employers for use by their employees. 96-101,113,181 In the
other studies about choice of plans the public report was HEDIS, 95 CAHPS, 102-104,183 or
both. 23,105,112,114 The studies of health plan behavior also examined the effect of HEDIS, 53-55,70,71
CAHPS, 57 or both. 184 HEDIS is a set of clinically oriented measures developed by the National
Council on Quality Assurance (NCQA) that included measures related to screening, prevention,
care coordination, and treatment of specific conditions. HEDIS has been in use for over 20 years
and is currently used by over 90 percent of managed care plans and an increasing number of
preferred provider organizations (PPOs). It is required in 34 States for reporting on private and
public health plans. CAHPS is a measure of member experience initially developed by the
Federal government for health plans in the United States. Several versions have been created and
used in different health care settings and other countries. 57
The comparator in the majority of health plan studies was either (a) time period during
which the report was not available or (b) groups that did not have the publicly reported
information. However some studies did not have a true comparator as they measured only
change over time after public reporting began.
HEDIS, CAHPS, and custom reports on the quality of health plans have been distributed and
published in phases over time. This has allowed for use of a variety of study designs (see
Appendix D for definitions of the study design terminology used in this report). The comparator
and study design influence but do not alone determine the quality assessments of articles
included in this review. Potential confounding as well as the strength of the comparison
(similarity across compared groups or compared time periods) was given more weight than other
criteria. (See Appendix G for the quality assessment for these studies and Appendix F for a
description of the quality assessment criteria). Seven studies were rated as good, 23,55,71,96,98,102,103
13 as fair, 53,54,70,95,96,99-101,104,105,112-114 and 4 were poor according to these criteria. 56,57,97,183
Three studies of health plans used random assignment 102-104 which is rare in studies of public
reporting. These studies all examined the impact of CAHPS on plan selection by randomly
assigning Medicaid beneficiaries to receive or not receive CAHPS information in their
enrollment materials and then compared plan selection across the groups. In one case the random
assignments were not recorded by the company responsible for the mailing and the investigators
had to ask survey respondents to self-report whether they had received the CAHPS report in their
materials. 104 Studies that included data only post public reporting were most common, with two
“post only” time series, 53,56 five studies that reported data for one group post public
reporting, 23,71,95,96,105 and six studies that included a comparison group. 54,57,97,98,114,183 Studies that
included data collected prior to public reporting included one study that interviewed Medicare
beneficiaries before various versions of plan information were mailed and compared their plan
selections to beneficiaries who received the different mailings; 112 three studies that compared
different groups before and after public reporting; 55,70,99 and four studies that analyzed data from
one group of subjects pre and post reporting. 100,101,113,181
The most common outcomes in these studies (17 out of 24) were the selection of health plans
by employees, people eligible for public insurance programs, or employers (Key Question
4). 23,95-105,112-114,181,183 This selection outcome was operationalized in several ways including the
likelihood of selecting a plan conditional on its quality rating, the probability of switching plans,
or the retention of members by plans. The outcomes of studies that analyzed the changes in
health plan performance in response to public reporting were changes in the quality of care
provided by health plans (Key Question 1). 53,54,56,57 Two studies tested potential harms (Key
84