Quality and Outcome Indicators for Acute Healthcare Services

Quality and Outcome Indicators for Acute Healthcare Services

Quality and Outcome Indicators for Acute Healthcare Services


You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

© Commonwealth of AustraliaISBN 0 644 37481 XFirst published March 1997This work is copyright. Apart from any use as permitted under the Copyright Act 1968, no partymay be reproduced by any process without prior written permission from the Australian GovernmentPublishing Service. Requests <strong>and</strong> enquiries concerning reproduction <strong>and</strong> rights should be addressedto the Manager, Commonwealth In<strong>for</strong>maiton <strong>Services</strong>, Australian Government Publishing Service,GPO Box 84, Canberra ACT 2601Responsibiltiy <strong>for</strong> the content of this report remains with the authors. This report does not necessarilyreflect the views of the Commonwealth Department of Health <strong>and</strong> Family <strong>Services</strong>.Cover design by the Publications <strong>and</strong> Design (Public Affairs, Parliamentary <strong>and</strong> Access Branch)Commonwealth Department of Health <strong>and</strong> Family <strong>Services</strong>Produced by the Australian Government Publishing Service

2ContinuityDischarge PlanningCoordinationTechnical Proficiency -Appropriateness -These dimensions of care are reflected variably in currently available indicators. They include concepts ofquality that overlap (that is, there is some redundancy in this classification) <strong>and</strong> they are notcomprehensive (that is, there are important quality dimensions - such as equity <strong>and</strong> benevolence - whichare inadequately represented. These dimensions of care are conceptually complex <strong>and</strong> poorly translatedinto quantitative indicators. Consequently no feasible indicators of these dimensions were identified).We have proposed a number of attributes of quality of care indicators which provide a basis <strong>for</strong> assessing<strong>and</strong> deciding the potential utility of any putative indicator <strong>for</strong> a nationally consistent quality <strong>and</strong> outcomeindicator set. These attributes are:ATTRIBUTEReliabilityValidityENCOMPASSINGInternal consistencyTest/re-test stabilityInterrater reliabilityFace validityContent validityConstruct validityPredictive validitySensitivitySpecificityPredictive valueResponsiveness -Interpretability -SignificanceBurdenVulnerability toundesired effectsRelevanceCollection impostGamingDistortion of healthservice deliveryUtility -Availability ofalternate <strong>for</strong>msAmenity toindependent corroborationLanguage adaptedLiteracy adaptedCulturally adaptedAuditData SourceIt is frequently stated that the science of healthcare quality indicators is in its infancy. Certainly we foundthat good data on the attributes of proposed indicators <strong>and</strong> specific in<strong>for</strong>mation on their developmental

3rigour were frequently missing. Some indicators have no basis in science. Nonetheless, we foundsufficient evidence supporting quality of care indicators to suggest that an appropriate growth <strong>and</strong>development analogy would place quality indicators in early childhood, rather than infancy, with severalquality <strong>and</strong> outcome indicator programs having passed significant developmental milestones. Most existingquality <strong>and</strong> outcome indicators are imperfect. In the majority of indicators we examined there wasinsufficient in<strong>for</strong>mation available on indicator attributes to allow us to draw firm conclusions aboutindicator per<strong>for</strong>mance against all of our chosen attributes. We see the current generation of indicators asstepping stones to future better indicators. It will only be by their application in the health sector thatindicators will improve. Provided all those using current indicators are aware of their limitations <strong>and</strong>appropriately cautious in their interpretation, no harm will come from use of less than perfect per<strong>for</strong>mancemeasures. Existing - albeit imperfect - indicators do provide useful pointers to quality of care. If resultsof indicator data occasionally differ significantly from user expectations it may be necessary <strong>for</strong> thoseusing these tools to consider revising their underst<strong>and</strong>ing of the role of indicators in monitoring qualityrather than discarding these indicators <strong>and</strong> persisting with attempts to find “perfect” indicators.This report contains insights derived from an examination of how we <strong>and</strong> other western healthcare systemshave applied quality of care indicators. These experiences contain valuable lessons <strong>for</strong> those embarking onlarge scale quality monitoring in healthcare - particularly on how best to manage the introduction ofquality <strong>and</strong> outcome indicators to maximise their value as accountability tools, instruments guidingconsumer choice <strong>and</strong> facilitators of provider ef<strong>for</strong>ts to improve quality. One recurring imperativeidentified by our review is the requirement <strong>for</strong> a collaborative approach to indicator development <strong>and</strong> thesubsequent interpretation <strong>and</strong> dissemination of quality <strong>and</strong> outcome indicator data. Such collaborationmust embrace purchasers, providers <strong>and</strong> consumers of healthcare to see quality <strong>and</strong> outcome indicatorssuccessfully achieve their multiple goals.<strong>Quality</strong> of care indicators are constructed from data obtained from administrative databases, abstractedfrom medical records, reported by patients in response to structured survey or by some combination ofthese methods. Administrative databases are a relatively complete data source, although there aresignificant risks of inaccuracy if data are used <strong>for</strong> purposes other than those originally intended. Patientclassification <strong>and</strong> categorisation within existing coding systems mask patient differences of potentialimportance <strong>for</strong> quality <strong>and</strong> outcome indicator construction. In contrast to the low cost of indicator datafrom administrative databases, medical record abstraction is expensive. The in<strong>for</strong>mation present in recordsis inconsistent <strong>and</strong> frequently incomplete. Occasionally, the added expense of collection is justified by theclinical detail obtained by record review, although such abstraction cannot be a basis <strong>for</strong> core nationalindicators. Patient surveys are uniquely capable of providing valuable insights into the processes <strong>and</strong>outcomes of care <strong>and</strong> care acceptability. Although expensive, as indicator data sources go, patient surveyswill be essential <strong>for</strong> a worthwhile national quality monitoring program embracing consumer perspectiveson care delivery.The development of explicit guidelines <strong>for</strong> the care of particular clinical conditions has been a majorachievement of several international <strong>and</strong> national expert groups over recent years. The National Health<strong>and</strong> Medical Research Council (NHMRC) has undertaken initiatives to develop guidelines <strong>for</strong> national use,including areas targeted as national health priority areas. Clinical practice guidelines can readily beadapted to generate quality of care indicators <strong>for</strong> specific conditions or interventions.Following a review of existing indicators available <strong>for</strong> the identified key dimensions of quality of care inthe acute healthcare sector, <strong>and</strong> assessment of their relative strengths <strong>and</strong> weaknesses, we have made aseries of recommendations regarding the potential development of a quality <strong>and</strong> outcome indicator programnationally.

41.1 STRUCTURE OF A NATIONAL INDICATOR PROGRAM1.1.1 National quality <strong>and</strong> outcome indicators be developed as two complementary sets - a coreindicator set intended <strong>for</strong> long-term continuous collection <strong>and</strong> a series of indicator modulesfocused on periodic collection <strong>for</strong> a finite duration at a defined frequency.1.1.2 Core indicators focus on aspects of care that are common across conditions.1.1.3 Indicator modules be targeted to specific conditions, diseases, diagnoses or interventions<strong>and</strong> include a balanced portfolio of clinical indicators, health status, acceptability <strong>and</strong> costindicators. It is likely that most benefit <strong>for</strong> facility-level quality improvement in acutehealthcare will flow from energy devoted to such targeted quality of care indicator modules.1.1.4 Where possible, the development of indicator modules should commence in identifiednational health priority areas.1.1.5 Core indicator sets be based upon data from administrative databases or patient surveys.1.1.6 Indicator modules include hybrid data collection - including medical record sampling,review <strong>and</strong> data abstraction - to establish the cost benefit of more sophisticated indicatorconstruction methodologies in a national quality monitoring context.1.1.7 Indicator development programs be cooperative - involving all relevant agencies - <strong>and</strong>multidisciplinary.1.1.8 Appropriate sampling technique be used in indicator data acquisition wherever possible toincrease the complexity of data available <strong>for</strong> a given cost.1.1.9 A resource be developed to assist providers with indicator data collection, analysis,feedback <strong>and</strong> interpretation <strong>and</strong> with the potential to provide independent confirmation ofdata integrity.1.1.10 <strong>Indicators</strong> developed <strong>for</strong> national use include comprehensive operational definitions,preferably encoded within suitable software packages, to enhance data reliability.1.2 SUGGESTED INDICATORS FOR ADOPTION IN AUSTRALIAThere are no indicators with unequivocal <strong>and</strong> universal support as “gold st<strong>and</strong>ard” means ofmonitoring healthcare quality. Comparative studies of indicator per<strong>for</strong>mance in acute healthcare arerare <strong>and</strong> do not provide a basis <strong>for</strong> the unquestioned support of any individual quality of careindicator or indicator set in an Australian context. From the in<strong>for</strong>mation available on currentquality indicators we have selected a reasonable, basic set of indicators which we believe havesufficient evidence to support their probable utility. This evidence includes knowledge that thepremise underpinning the indicator is sound, that data collection is feasible (given current orreasonably anticipated future health services in<strong>for</strong>mation systems) <strong>and</strong> that the indicator would havepractical utility. The absence of compelling evidence to guide our choice of suggested indicatorsmeans that ultimately, those indicators recommended receive that recommendation based on anassessment by us of their overall worth - following our in-depth review of available in<strong>for</strong>mation onquality indicators. A balance has been sought between the size of the initial indicator set <strong>for</strong>adoption <strong>and</strong> the desire <strong>for</strong> comprehensive coverage of the quality of acute healthcare services.Over time a larger indicator library can be developed based upon local research, development <strong>and</strong>experience <strong>and</strong> a close linkage of our national indicator programs to international initiatives inquality indicators <strong>for</strong> acute healthcare.

51.2.1 Access- Elective surgery waiting times- Emergency Department waiting times- Waiting times in Emergency Department prior to emergency admission- Patient-based reports of elective surgery, Emergency Department <strong>and</strong> OutpatientDepartment waiting times <strong>and</strong> the acceptability of these waiting times to patients1.2.2 Efficiency- Cost per casemix adjusted separation1.2.3 Safety- <strong>Indicators</strong> of adherence to best practice guidelines <strong>for</strong> the processes of care or variationsin observed to predicted outcomes should be incorporated in indicator modules targetingparticular clinical circumstances. In view of the importance of cardiovascular disease,modules addressing interventions in patients with ischaemic heart disease (angioplasty,CABG, angiography <strong>and</strong> AMI) should be included in any initial module sets.1.2.4 Effectiveness- Generic health status (SF36 or SF12) be included in the core indicator set, examiningchange in health status about an acute episode of care.- Health status measures, either generic or condition-specific, be incorporated intoindicator modules targeting specific clinical circumstances.- Mortality rates, stratified with data available within administrative databases, becollected <strong>for</strong> key clinical conditions <strong>and</strong> compared to risk-adjusted mortality indicesdeveloped within indicator modules. Such analyses will establish the cost-benefit ofmore sophisticated risk-adjustment modelling in the context of a national qualitymonitoring program (commencing with angiography, CABG, angioplasty <strong>and</strong> AMI).- Unplanned readmission after inpatient treatment <strong>for</strong> asthma (paediatric <strong>and</strong> adultgroups).- Low <strong>and</strong> very low birthweight rates <strong>for</strong> infants.1.2.5 Continuity- Patient-based assessments using relevant modules from the Picker CommonwealthSurvey instrument.1.2.6 Acceptability- A national survey on the acceptability of care be based upon components of the PickerCommonwealth, Hospital Corporation of America <strong>and</strong> Royal College of Surgeonsinstruments.1.2.7 Technical Proficiency- Avoid putative generic indicators of facility-wide technical proficiency- Promote risk-adjusted technical proficiency indicators developed within indicatormodules addressing targeted clinical circumstances.1.2.8 Appropriateness- Use relative utilisation rates of targeted interventions as proxies <strong>for</strong> appropriateness.

6These recommended indicators are summarised on Page 8.The complexity underlying the process of trialing putative quality indicators should not beunderestimated. Even apparently simple indicators require in-depth analysis of competingoperational definitions <strong>and</strong> generation of an agreed, detailed operational definition (preferablysupported by written materials with computer software expansion, clarification <strong>and</strong> problem-solvingguides). This agreed operational definition is then to be rigorously <strong>and</strong> uni<strong>for</strong>mly applied in thefield. In the case of modular indicator sets, project teams with a range of relevant skills <strong>and</strong>experience should build balanced sets of indicators from existing available indicators or develop denovo indicators to address relevant per<strong>for</strong>mance measurement needs. Both classes of indicatorsshould be trialed in a collaborative, cooperative atmosphere <strong>and</strong> involve provider, purchaserregulator <strong>and</strong> consumer representation in the project teams. Pilot projects across interestedrepresentative facilities should precede any more widespread implementation to ensure that thenational indicator set, when launched, is valuable to all concerned <strong>and</strong> credible. Engaging existingdrivers of quality indicators in the trialing process (e.g. the States <strong>and</strong> Territories <strong>and</strong> TheAustralian Council on <strong>Healthcare</strong> St<strong>and</strong>ards Care Evaluation Program (ACHS CEP)) will beessential to ensure that common operational definitions <strong>for</strong> quality indicators are arrived at. Thiswill avoid the creation of competing, overlapping data collection requirements of healthcarefacilities - which could lead to inefficiencies, the disenfranchisement of providers <strong>and</strong> a drasticreduction in the reliability of indicator data.1.3 FUTURE DIRECTIONS FOR INDICATOR DEVELOPMENT1.3.1 Dimensions of care with particular needs <strong>for</strong> indicator development:- Unmet access needs- Allocative <strong>and</strong> technical efficiency- Acceptability- Continuity- Appropriateness- Indicator modules <strong>for</strong> vulnerable patient populations1.3.2 <strong>Quality</strong> of care <strong>and</strong> outcomes indicators <strong>for</strong> healthcare developed by multidisciplinarygroups should focus on producing balanced portfolios of indicators targeting particularconditions, diseases or interventions <strong>and</strong> reflecting many key dimensions of healthcarequality <strong>and</strong> the perspectives of purchasers, providers <strong>and</strong> patients.1.3.3 Most of the significant health concerns of Australians relate to chronic illnesses, where theper<strong>for</strong>mance of the integrated healthcare system is far more important than the per<strong>for</strong>manceof its isolated components. In the long term, quality <strong>and</strong> outcome indicator programs shouldaddress integrated healthcare system per<strong>for</strong>mance rather than retaining an acute healthcaresector focus. Measurement of the quality <strong>and</strong> outcomes offered by the care continuum willthen in<strong>for</strong>m the success of overall service delivery.1.3.4 Australia should develop centres of excellence pursuing quality of care indicator research<strong>and</strong> seek their inclusion in the WHO <strong>Quality</strong> Assurance Collaboration.Changes in the science of quality of care indicators <strong>and</strong> the science of medicine will require that quality<strong>and</strong> outcome indicator programs are dynamic, iterative programs attuned to advances in academic <strong>and</strong>

7clinical knowledge. Few indicators should there<strong>for</strong>e be built around m<strong>and</strong>ating changes in routineminimum data set collections. It is unlikely that we will ever see single indicators or small indicator setsthat work <strong>for</strong> all parties, at all times <strong>and</strong> <strong>for</strong> all purposes. Above all else, effective use of quality <strong>and</strong>outcome indicators requires knowledge of their limitations <strong>and</strong> intelligence, good-will <strong>and</strong> common sense intheir applications. Programs to “perfect” indicators must be complemented by educational campaigns toimprove the underst<strong>and</strong>ing of their use by all with an interest in monitoring healthcare quality.

8<strong>Indicators</strong> Recommended <strong>for</strong> Trialing in a Core National Set• Access- Elective surgery waiting times- Emergency Department waiting times- Waiting times in Emergency Department prior to Emergency admission- Patient-based reports of elective surgery, Emergency Department <strong>and</strong>Outpatient Department waiting times <strong>and</strong> the acceptability of these waitingtimes to patients• Efficiency- Cost per casemix adjusted separation• Effectiveness- Generic health status (SF36 or SF12)• Continuity- Patient-based assessments using relevant modules from the PickerCommonwealth survey instrument• Acceptability- A national survey based upon components of the Picker Commonwealth,Hospital Corporation of America <strong>and</strong> Royal College of Surgeons instruments• Appropriateness- Relative utilisation rates of targeted procedures<strong>Indicators</strong> Recommended <strong>for</strong> Trialing in Indicator Modules at a Defined Frequency<strong>and</strong> <strong>for</strong> a Finite DurationProjects should be promoted to develop indicator modules targeting particular clinicalcircumstances. These balanced sets of quality of care indicators should include a variety ofperspectives on the quality of care chosen from relevant dimensions of care such as:AccessEfficiencySafetyEffectivenessTechnicalProficiencyAppropriateness- Waiting times- Cost- Adherence to best practice guidelines- Observed to expected outcome ratios- Condition-specific health status or health-related quality of lifemeasures- Stratified mortality rates- Unplanned readmission rates (asthma)- Low <strong>and</strong> very low birthweight infants (obstetric care)- Risk-adjusted technical proficiency indicators- Condition/procedure-specific relative utilisation rates

92. INTRODUCTION2.1 The Aim of the ProjectThe National Hospital <strong>Outcome</strong>s Program (NHOP) commissioned this Project to critically review thestatus of Australian <strong>and</strong> overseas knowledge regarding the development <strong>and</strong> use of quality of care <strong>and</strong>health outcome indicators in acute care services. It specifically sought to identify those per<strong>for</strong>manceindicators that contribute to improvements in the quality <strong>and</strong> outcomes of care to in<strong>for</strong>m the developmentof a set of nationally consistent quality of care <strong>and</strong> health outcome indicators <strong>for</strong> acute healthcare servicesin Australia.The Project wished to:• Identify key issues surrounding the use of quality <strong>and</strong> outcome indicators in a national context• Identify key dimensions of quality of care <strong>for</strong> indicator development <strong>and</strong> implementation• Assess the usefulness of existing indicators <strong>for</strong> application in the Australian context• Advise on future directions <strong>for</strong> quality <strong>and</strong> outcome indicator developmentsThe research was to include commentary on hospital access, technical <strong>and</strong> allocative efficiency,effectiveness, appropriateness, nursing, change in health status, discharge planning <strong>and</strong> patient safetyindicators at a minimum. Health concerns of vulnerable patient groups, such as Aboriginals <strong>and</strong> TorresStrait Isl<strong>and</strong>er peoples, were to be addressed in the context of indicator utilisation. The Projectspecifically excluded replication of work undertaken within complementary projects.The Commonwealth, States <strong>and</strong> Territories in recent years have been putting substantial ef<strong>for</strong>ts intoimproving the cost efficiency of public hospitals. Health consumers are greatly concerned about thequality <strong>and</strong> outcomes of care, particularly in an environment of programs pursuing efficiencyimprovements. The Commonwealth, States <strong>and</strong> Territories are concerned that quality of care <strong>and</strong> healthoutcomes are at least maintained - <strong>and</strong> preferably improve - whilst efficiency gains are progressing. InSchedule I of the 1993-1998 Medicare Agreements, the Commonwealth, States <strong>and</strong> Territories agreed tothe development of national quality of care <strong>and</strong> health outcome indicators <strong>for</strong> hospitals. It is believedessential that mechanisms be developed to monitor quality of hospital care so that consumers <strong>and</strong>governments can be sure of the value of delivered care (i.e. the ratio of quality to cost) <strong>and</strong> providers ofcare are stimulated to pursue quality improvement within acute healthcare programs.The NHOP seeks to develop <strong>and</strong> implement per<strong>for</strong>mance indicators <strong>and</strong> determine st<strong>and</strong>ards of quality ofcare <strong>for</strong> Australian hospitals. It is a three year program to develop <strong>and</strong> trial workable indicators that canbe used to promote improvements in the quality of care <strong>and</strong> health outcomes in Australian hospitals. TheProject is to identify a manageable set of indicators that would primarily serve the accountabilityrequirements of a large purchaser of acute care services. This Project does not seek identification ofmicroindicators that might be relevant in the internal management of quality by acute care providers. TheProject is the first stage in the programmed development of a nationally consistent quality <strong>and</strong> outcomeindicator set <strong>and</strong> will provide a basis <strong>for</strong> future work to develop, refine <strong>and</strong> test indicator sets suitable <strong>for</strong>national application.Data <strong>for</strong> generation of quality <strong>and</strong> outcome indicators has either to be specifically created or obtained fromdata sources developed <strong>for</strong> other purposes. There has been a strong trend to manipulate existing datasources to produce quality indicators at low cost. As is discussed in detail later in this report (Section 5.4)there are considerable difficulties in the use of data <strong>for</strong> purposes other than those originally intended bydata gatherers. <strong>Indicators</strong> based solely around data from administrative databases will always be ofsomewhat limited utility. The longterm success of quality <strong>and</strong> outcome indicator programs will requirethat necessary data are identified <strong>and</strong> subsequent steps taken to obtain such data (be it routine collectionfrom all or sampling strategies targeting patient population subsets) rather than persevering with strategies

10attempting to build credible indicators entirely from existing computer database contents. At presentAustralian healthcare providers collect various data items - with no uni<strong>for</strong>m definitions applicable to manyapparently similar data (such as elective surgery waiting times). As requisite data points <strong>for</strong> qualitymonitoring are identified, national uni<strong>for</strong>m data definitions must be incorporated into the national datadictionary <strong>and</strong> mechanisms enacted to audit compliance with definition application in the field. Such aprospective approach to quality indicator data collection will undoubtedly increase the cost of indicatorcollection - but the consequent increase in the value of indicator data flowing from the prospectivedesignation <strong>and</strong> collection of reliable, relevant data will be more than commensurate.2.2 The Aim of this ReportIn this final report we convey, in a summary <strong>for</strong>mat, our opinions <strong>and</strong> recommendations regarding qualityof care <strong>and</strong> health outcome indicators <strong>for</strong> acute healthcare services. We seek to do this in an accessibleway, hence the report on these complex 1-1136 <strong>and</strong> at times controversial matters is relatively succinct.Those interested in more detailed analyses <strong>and</strong> commentaries will find these in the attached appendices tothis report or within the bibliography listing of in<strong>for</strong>mation sources.<strong>Quality</strong> <strong>and</strong> outcome indicators are of interest to many customer groups. These include:• Patients <strong>and</strong> their families• <strong>Healthcare</strong> professionals• Purchasers of healthcare• Regulators of healthcare• Health system employees• CommunitiesOccasionally, the needs <strong>and</strong> expectations of these groups regarding quality <strong>and</strong> outcome indicators do notalign, <strong>and</strong> sometimes these differences are substantial. Our report repeatedly places commentary <strong>and</strong>recommendations in a context of the intended use of derived indicator data. This report encompassesdiscussion of the history of per<strong>for</strong>mance indicators in acute healthcare, recommendations on keydimensions of quality of care requiring monitoring <strong>and</strong> the identification of pertinent issues aroundintroduction of a nationally consistent quality <strong>and</strong> outcome indicator set. It seeks to point out gapsbetween existing indicators <strong>and</strong> those desired by potential users. Knowledge of these gaps permitsprioritisation of subsequent research <strong>and</strong> development ef<strong>for</strong>ts.3. BACKGROUND3.1 History of Indicator DevelopmentHistorically, the responsibility <strong>for</strong> healthcare quality was thought to reside only with the medical <strong>and</strong>clinical care professions (the so-called “professional model” of accountability) 297 . Consumers <strong>and</strong>purchasers assumed that relatively uni<strong>for</strong>m quality of care existed across any given healthcare deliverysystem <strong>and</strong> that investigation <strong>and</strong> treatment approaches were relatively st<strong>and</strong>ard within that system of care.Several decades ago there was a move to make provider facilities accountable <strong>for</strong> care delivered withinthose facilities, which saw an increased emphasis on facility-level peer-review of per<strong>for</strong>mance regardingquality of care. Facility peer-review is essentially a strengthening of the professional model ofaccountability which typically fails to provide objective evidence on per<strong>for</strong>mance <strong>for</strong> external review.Concerns that existed regarding significant variations in quality of care <strong>and</strong> the nature of care deliveredwithin the acute healthcare sector coincided with interest in improving acute care efficiency <strong>and</strong> restrainingtotal expenditure on acute healthcare in several western communities 16-28 . Whilst the relative balancebetween consumerism <strong>and</strong> cost-control varied between communities, all have demonstrated a progressiveinterest in objective quality evaluation in healthcare. Consumers, purchasers, regulatory agencies,

11governments <strong>and</strong> providers increasingly seek to compare provider per<strong>for</strong>mance to address accountabilityrequirements, promote quality improvement <strong>and</strong> in<strong>for</strong>m consumer decisions.<strong>Quality</strong> of care <strong>and</strong> outcome indicators first appeared almost a century ago with proposals to monitor theimpact of hospital care by pioneers such as Florence Nightingale in the UK <strong>and</strong> Ernest Codman in theUSA. These proposals were not widely acceptable to healthcare professionals <strong>and</strong> the mechanics of largescale data collection were impractical, hence they were not embraced. <strong>Indicators</strong> resurfaced in the early1970s when the potential <strong>for</strong> improving quality by using measurement was rediscovered by socialscientists. In North America, groups that sought to systematically monitor acute healthcare quality(Professional St<strong>and</strong>ards Review Organisations) debuted in 1973 <strong>and</strong> began applying crude qualityindicators to the hospital sector. In the 1980s the availability of improved data management systems, theperceived impact of efficiency strategies such as prospective payment systems <strong>and</strong> Deming’s <strong>Quality</strong>Improvement principles rejuvenated interest in quality monitoring <strong>and</strong> saw a move from indicator usage <strong>for</strong>quality maintenance (quality assurance) to use <strong>for</strong> quality upgrading (quality improvement).In the mid 1980s programs were launched to evolve complete sets of quality <strong>and</strong> outcome indicators,rather than use of individual indicators on an ad-hoc basis. These indicator sets were designed to be partof comprehensive quality monitoring programs in hospitals (e.g. The Maryl<strong>and</strong> Hospital Association<strong>Quality</strong> Indicator Project; 1985 55,304,335 . The Joint Commission on Accreditation of <strong>Healthcare</strong>Organisations Indicator Measurement System; 1986 14 . ACHS CEP Clinical Indicator Program;1989 367,507,508 ). More recently the focus in healthcare monitoring has moved from acute healthcare qualityassessments, specific <strong>for</strong> an episode of care, to assessment of the per<strong>for</strong>mance of integrated healthcaresystem per<strong>for</strong>mance (e.g. The Consortium Research on <strong>Indicators</strong> of System Per<strong>for</strong>mance; 1990 300-303 .The Health Employers Data <strong>and</strong> In<strong>for</strong>mation Set; 1989 307 . The Clinical Accountability <strong>and</strong> SystemsPer<strong>for</strong>mance Evaluation <strong>Quality</strong> <strong>and</strong> <strong>Outcome</strong> Indicator Set; 1996) embracing review of the continuum ofcare. This move to a broader focus on healthcare provision is driven by the recognition that:• Chronic illnesses, in particular, do not have clearly defined start/stop points <strong>for</strong> per<strong>for</strong>manceassessment based on a single episode of care.• The quality of integration <strong>and</strong> coordination of care is as important as the quality of componentservices.• Per<strong>for</strong>mance indicators should reflect whether required care was delivered to communities requiringcare, not simply how well care was delivered to hospitalised individuals who have successfullyaccessed care.Initial quality indicators tended to address healthcare structures (such as the level of equipment <strong>and</strong> staff).Later, indicator ef<strong>for</strong>ts were directed at process measures (such as if care was delivered as planned) whenprocess management was recognised as the key to successful quality improvement 255 . In the late 1980s astrong emphasis on outcome indicators emerged - driven by Paul Elwood 649 <strong>and</strong> others - requiringsystematic evidence of the end result of healthcare interventions (such as the duration or quality of lifeachieved).Original indicator sets tended to focus on narrow aspects of healthcare delivery (such as technicalproficiency or outcomes as perceived by providers of care) 320,321 . More recently, indicator sets haveemphasised the importance of a balanced portfolio of indicators, encompassing clinical outcomes, theacceptability of care, functional health outcomes, critical process measures <strong>and</strong> cost of care delivery (suchas Hospitals Corporation of America’s “Value Compass” Indicator Sets) 492,930 .Typically, indicator development programs commence with an emphasis on hospital episode of carereview, because of the availability of data systems in hospital practice <strong>and</strong> an assumption that hospitalbasedepisodes of care represent critical moments in individual patients’ overall healthcare. Such hospitalbasedindicator measures should only represent the beginning of an integrated approach to comprehensivequality of care <strong>and</strong> outcome monitoring within a healthcare system.

12SUMMARY OF TRENDS IN INDICATOR USE• Autonomy → Peer Review → Facility Review → External Review• Structure → Process → <strong>Outcome</strong> Focus• Individual <strong>Indicators</strong> → Indicator Sets• R<strong>and</strong>om Indicator Development → Balanced Sets of <strong>Indicators</strong>• <strong>Quality</strong> Control → <strong>Quality</strong> Assurance → <strong>Quality</strong> Improvement Context3.2 TerminologyThis field of health services research is replete with examples of individuals using the identical term torepresent quite different underlying concepts <strong>and</strong> different terms to reflect a common theme. No universallexicon exists. There has been a relatively recent growth in interest in quality <strong>and</strong> outcome indicators <strong>and</strong>the appropriation of words from common usage to serve as descriptors <strong>for</strong> very specific quality <strong>and</strong>outcome concepts. We provide brief working definitions we have used <strong>for</strong> important terms appearing inthis report. (A more detailed glossary of some examples of what others believe to be the various preferreddefinitions of terms is contained in Appendix 1).Structure:comprises the characteristics of care or resources compiled todeliver care to the patient. It includes the physical facilities, the staff<strong>and</strong> the licensing <strong>and</strong> credentialling of healthcare providers <strong>and</strong>selected patient characteristics 297 .Process:refers to the actual delivery of care. The series of linked, often (butnot necessarily) sequential steps that convert an input into anoutput, cause some set of outcomes to occur, generate usefulknowledge or add value 315 .<strong>Outcome</strong>:the significant result or end product of care delivery, such asimproved survival, functional health status or quality of life 304 .Health <strong>Outcome</strong>:a change in the health of an individual or a group of people orpopulation which is attributable to an intervention or series ofinterventions 352,504 .Health Intervention:any action which is intended to improve someone’s health (orreduce the rate at which it deteriorates), whether the action is aimedat health promotion, disease prevention, early diagnosis, a clinicalintervention, counselling or social service support, educational orpreventive measures, a change in administrative or budgetaryresponsibilities, regulations relating to safety, the relief of poverty,better housing or whatever 352,504 .Measures: seek to directly quantify quality of care or health outcomes 352,504 .<strong>Indicators</strong>:statistics or other units of in<strong>for</strong>mation which reflect, directly orindirectly, the per<strong>for</strong>mance of the healthcare system in maintainingor increasing the well-being of its target population 352,504 .Generic Measures orare measures or indicators that can be applied to individuals in any<strong>Indicators</strong>:health condition 352,504 .Clinically Specific<strong>Indicators</strong> or Measures:Attribution:Association:are indicators or measures that relate to specific clinical conditionsor measures of function that may have particular significance <strong>for</strong>particular conditions 352,504 .a health outcome is attributable to an intervention if the interventionhas been shown in a rigorous scientific way to cause the change inhealth status 352,504 .a health outcome is associated with an intervention if the change inhealth status generally occurs following the intervention but has notbeen demonstrated through rigorous scientific study to cause thechange 352,504 .

13Health Status:an integrated indicator of health (i.e. well-being), typicallyincorporating biological function, physical <strong>and</strong> mental health, social<strong>and</strong> role functioning 505 .Health-related <strong>Quality</strong> of that component of quality of life related to the sense of health (i.e.Life:well-being) of the individual concerned 764 .Customer:the recipient of a service within healthcare or anyone who hasexpectations regarding healthcare delivery 339 .Consumer: the population of potential customers 339 .The differential application of “indicator” versus “measure” <strong>and</strong> “quality/outcome” versus “per<strong>for</strong>mance”was particularly noted in our review of the literature. Terms such as “quality indicator” <strong>and</strong> “qualitymeasure” or “outcome indicator” <strong>and</strong> “outcome per<strong>for</strong>mance indicator” are effectively usedinterchangeably by some - yet are regarded as substantially different by others.If indicators are identifiable markers which describe or relate to quality of care <strong>and</strong> measures arequantifiable dimensions of objects or functions then clearly there is a continuum between an identifiabledescriptor related to quality (i.e. an indicator) <strong>and</strong> a quantified marker of quality (i.e. a measure). It is amatter of opinion as to when an “accurate description” merges into “quantification” 333 .We have elected to use the term “indicator” <strong>for</strong> all instruments that estimate the extent to which qualityhealthcare services are delivered, believing the overall status of quality evaluation in healthcare warrantsuse of a term implying less precision in the inferential relationship between evaluation <strong>and</strong> quality of care333,341 . We acknowledge that all “quality indicators” are strictly-speaking “quality per<strong>for</strong>mance indicators”337 . They provide reflections on the care setting (structure) <strong>and</strong> on things providers do (processes) orachieve (outcomes) rather than judgements based upon these reflections. For brevity’s sake, <strong>and</strong> indeference to common usage, we have frequently applied the term “quality indicator” without reference tothe “per<strong>for</strong>mance” qualifier.4. METHODOLOGY FOR THE PROJECT4.1 Literature ReviewAll consultant members contributed to an initial bibliographic database search <strong>and</strong> h<strong>and</strong>search of relevantdata sources. One month into the six month consultancy the Centre <strong>for</strong> Health Program Evaluationmembers assumed primary responsibility <strong>for</strong> further literature review of indicators of access <strong>and</strong> efficiencywhile The Alfred <strong>Healthcare</strong> Group <strong>and</strong> Department of Epidemiology <strong>and</strong> Preventive Medicine, MonashUniversity, members continued the broad literature search <strong>for</strong> other identified key dimensions of quality ofcare (see 5.1 below). Further details of the literature search methodologies are contained in Appendix 2.In common with other researchers in this field, we found the literature search to be a relatively inefficient<strong>and</strong> ineffective tool <strong>for</strong> accessing the body of knowledge of healthcare quality indicator applications320,321,340 . This reflects poorly developed key-word identifiers <strong>for</strong> research in indicators (limiting thesuccess of computer-based data searches) <strong>and</strong> the spread of relevant literature across a broad range ofjournals (rendering h<strong>and</strong> searches of journals relatively inefficient). Additionally, much of the detailedknowledge surrounding quality <strong>and</strong> outcome indicators is not published in conventional <strong>for</strong>ms - but iscontained in commissioned consultancy reports, purchaser or regulatory bodies internal workingdocuments or within healthcare providers’ in<strong>for</strong>mation systems. At Project’s end, the computer-basedbibliographic search <strong>and</strong> the h<strong>and</strong>-search strategies produced relatively equivalent numbers of relevantcitations.

144.2 Expert ConsultationThe Project team (Appendix 3) allocated responsibilities <strong>for</strong> canvassing expert opinion to complement theknowledge of quality of care <strong>and</strong> health outcome indicators obtained from the literature review in thefollowing way:• The Centre <strong>for</strong> Health Program Evaluation consulted with a number of States <strong>and</strong> Territories relevantto access <strong>and</strong> efficiency indicators.• The Alfred <strong>Healthcare</strong> Group members consulted with the Commonwealth on related researchprograms <strong>and</strong> overseas consultation with acknowledged leaders in indicator development <strong>and</strong>implementation. This included site visits to centres of excellence in the United States <strong>and</strong> the UnitedKingdom (Appendix 4).• The Department of Epidemiology <strong>and</strong> Preventive Medicine consulted with academic research groupspursuing relevant research within Australia.• The Australasian Association <strong>for</strong> <strong>Quality</strong> in <strong>Healthcare</strong> consulted with State <strong>and</strong> Territorygovernments to ascertain the status of relevant indicator developments <strong>and</strong> experience withinAustralia.Taken together, these contacts with experts working in the field were extraordinarily productive, both inproviding textural details <strong>and</strong> balanced opinion frequently absent from published reports <strong>and</strong> in helpingidentify additional sources of relevant in<strong>for</strong>mation <strong>and</strong> mechanisms to access these sources. These expertconsultations were crucial in obtaining the operational details of many indicators <strong>and</strong> indicator sets - detailswhich frequently in<strong>for</strong>med our decisions regarding the ultimate utility of proposed indicators. Additionaldetails concerning these site visits are included in Appendix 5. We acknowledge the generosity of those whoshared their valuable time <strong>and</strong> accumulated wisdom with us over the six months of the consultancy <strong>and</strong>contributed to our decision-making processes.5. RESULTS FROM THE PROJECT5.1 Key Dimensions of <strong>Quality</strong>The definition of quality in healthcare remains a challenge 12 . As <strong>for</strong> all abstract concepts it is possibleto define healthcare quality equally correctly from a variety of perspectives. We believe no singleperspective or resultant definition to be inherently superior to others. Use of several definitions isboth possible <strong>and</strong> legitimate. We consider the following definitions contribute to a broadunderst<strong>and</strong>ing of what is meant by quality in healthcare.• The degree of excellence. OXFORD DICTIONARY• How closely the result of a medical service approaches the fundamental objectives ofprolonging life, relieving distress, restoring function <strong>and</strong> preventing disability. PAULLEMBCKE• Achieving <strong>and</strong> producing health <strong>and</strong> satisfaction. AVEDIS DONABEDIAN• Consistently meeting or exceeding in<strong>for</strong>med customers’ expectations. W.EDWARDS DEMING• The degree to which health services <strong>for</strong> individuals <strong>and</strong> populations increase the likelihood ofdesired health outcomes <strong>and</strong> are consistent with current professional knowledge. INSTITUTEOF MEDICINE

15Whilst there are multiple dimensions to healthcare quality, only some of these are currently amenable toquantification. Following a preliminary review of the available literature we agreed with our SteeringCommittee to address indicators in the following care dimensions:AccessDIMENSIONTimelinessENCOMPASSINGEfficiencyTechnical EfficiencyAllocative EfficiencySafety -EffectivenessEfficacyAcceptabilityConsumer PerceptionCustomer PerceptionSatisfactionRelevanceCultural AppropriatenessConsumer Involvement in Health <strong>Services</strong>ContinuityDischarge PlanningCoordinationTechnical Proficiency -Appropriateness -The working definitions <strong>for</strong> these designated dimensions of care are:Access:Efficiency:The capacity of individuals to obtain the same quality of service.Maximising benefits (or outcomes) <strong>for</strong> a given cost:♦♦Technical efficiency: the degree to which the least cost combination ofresource inputs occur in production of a particular service.Allocative efficiency: the degree to which maximum benefit (oroutcomes) are obtained from available resources.Safety:Effectiveness:Acceptability:Continuity:TechnicalProficiency:The extent to which potential risks were avoided <strong>and</strong> inadvertent harmminimised in care delivery processes.The degree to which an intervention produces measurable increases in survivalor improved quality of life (or improved outcomes) when applied in routinepractice.The degree to which the service meets or exceeds the expectations of in<strong>for</strong>medcustomers <strong>and</strong> consumers.The extent to which an individual episode of care is coordinated <strong>and</strong> integratedinto overall care provision.The extent to which the per<strong>for</strong>mance of interventions by healthcareprofessionals is consistent with contemporary st<strong>and</strong>ards <strong>and</strong> knowledge ofskills relevant to that intervention.

16Appropriateness:The extent to which potential benefits of an intervention exceed the risksinvolved.Some identified dimensions of quality (such as equity) are only partially represented in the above schema(access reflects some aspects of equity). Others (such as benevolence) see aspects represented withinseveral of our proposed dimensions of quality of care, but are still inadequately represented overall. Wechose not to include care dimensions <strong>for</strong> the purpose of this Project when initial review failed to identifyany feasible indicators likely to be applicable in any national indicator program. These oversights do notindicate that dimensions not incorporated into this review are less crucial than those featuring - rather theseidentified dimensions of quality of care are a reasonable starting point from which to begin building acomprehensive quality <strong>and</strong> outcome indicator set <strong>for</strong> acute healthcare services. This final choice ofdimensions of quality of care encompasses aspects of care relevant to patients, providers <strong>and</strong> purchasers.There is minimal overlap between individual dimensions <strong>and</strong> the coverage of aspects of quality iscomprehensive. Because of the importance of patient safety in acute healthcare, we have chosen toemphasise safety by its designation as a discrete dimension of care, despite considerable content overlapwith matters analysed in the dimensions of effectiveness, technical proficiency <strong>and</strong> appropriateness.5.2 Criteria <strong>for</strong> Assessing <strong>Indicators</strong><strong>Healthcare</strong> quality indicators need to be judged against criteria which indicate whether they are likely tofulfil their intended purposes. We chose the following assessment criteria:♦Reliability:The degree to which an indicator is free from r<strong>and</strong>om error, is reproducible (or stable) over time <strong>and</strong>shows interrater agreement at one point in time.This includes the concepts of:• Internal consistency• Test/retest stability• Interrater reliabilityReliability will be largely dependent on the adequacy of the operational definition <strong>for</strong> the indicator<strong>and</strong> the rigour of data collection, data analysis <strong>and</strong> data audit.♦Validity:Given the quality monitoring purpose <strong>for</strong> which it is intended, do inferences regarding quality ofcare based upon the indicator accurately reflect the quality of care delivery?Validity is a matter of degree <strong>and</strong> must be judged with an underst<strong>and</strong>ing of the intended applicationof an indicator (that is, is it valid <strong>for</strong> its intended purpose?).Judgements of validity are based upon review of:• Face validity: Does the indicator appear to relate to quality of care?• Content validity: How closely does the indicator relate to quality of care <strong>and</strong> how well arerelevant aspects of care quality covered by the indicator?• Construct validity: What relation does the indicator have to other measures of quality?• Predictive validity: How well does an indicator of good/poor care predict that good/poorcare was delivered?

17Un<strong>for</strong>tunately criterion validity (the degree to which an indicator is related to widely accepted validmeasures of quality) could not be included in our assessments as criterion measures <strong>for</strong> quality areunavailable in acute healthcare.Within validity determinations lies knowledge of quality indicator sensitivity, its “true positive” rate<strong>and</strong> specificity, its “true negative” rate.♦♦♦♦♦♦♦♦Responsiveness:How does the indicator change as quality of care changes? Is the indicator capable of detecting thesorts of differences in quality of care typically experienced in acute healthcare services?Interpretability:Does the indicator make sense? Does it communicate a consistent message to those who use it?Significance:Does the indicator reflect aspects of care that matter to users of the indicator <strong>and</strong> are relevant incurrent healthcare contexts?Burden:How difficult or costly is indicator data collection <strong>and</strong> indicator construction?Utility:Has the indicator been proven to be of value when used in acute healthcare (either <strong>for</strong>accountability, directing consumer decisions or quality improvement)?Vulnerability to Undesired Effects:What is the likelihood that use of the indicator would create perverse incentives <strong>for</strong> healthcareproviders (such as to corrupt indicator data or alter healthcare provision in undesirable ways)?Availability of Alternate Forms:Can the indicator be altered to allow its use in different target populations (e.g. those requiringlanguage or cultural adaptations)?Amenity to Independent Corroboration: Can indicator data be confirmed by others?These attributes of indicators were culled from several proposed assessment criteria panels 320,321,324 . Theyprovide a realistic basis upon which to judge indicator suitability <strong>for</strong> application in an Australian context.In the majority of cases examined we found insufficient in<strong>for</strong>mation upon indicator per<strong>for</strong>mance to enableus to make appropriate firm judgements regarding indicator per<strong>for</strong>mance against these desired attributes.Final judgements on the utility of indicators involved assessments of all available in<strong>for</strong>mation regardingindicators <strong>and</strong> the potential <strong>for</strong> local collection of requisite in<strong>for</strong>mation.5.3 Current Status of Indicator Development5.3.1 AustraliaThe major program <strong>for</strong> acute healthcare quality indicators underway in Australia is the ACHSCEP <strong>and</strong> specialist medical colleges conjoint development of clinical indicators 367,507,508 .Although this work was specifically excluded from our terms of reference, it is such a dominant<strong>for</strong>ce in indicator development that we believe some brief comments on its achievements thus farare appropriate. In 1996 there are seven indicator sets in use - with 16 clinical indicator setsplanned to be available <strong>for</strong> use within ACHS accreditation programs in 1997 (Appendix 6).These indicators have been cooperative developments between the ACHS <strong>and</strong> relevant medicalspecialty colleges <strong>and</strong> strongly reflect the perspectives of providers, with a heavy emphasis ontechnical proficiency. Although drawing upon overseas indicator developments, operational

18definitions <strong>for</strong> ACHS CEP indicators are sufficiently different from international definitions torender direct comparisons impossible. Their focus upon clinician involvement in the derivationof indicator data means that they frequently cannot be simply collected from routineadministrative databases <strong>and</strong> hence may not be easily applied as core indicators at a nationallevel. At present indicators are not risk-adjusted using patient level data, but are stratified on thebasis of provider-facility characteristics. Throughout their development there has been relativelylittle data made available on the methodologic rigour of indicators - with little current dataavailable on their reliability, validity <strong>and</strong> responsiveness. These are not necessarily criticisms ofthe AHCS CEP nor the contributions of specialist colleges to this clinical indicator program.Rather, they reflect the context within which the ACHS program has progressed <strong>and</strong>, inparticular, its stated aims to promote facility quality improvement <strong>and</strong> engage clinicians inmeasurement endeavours. It does mean that ACHS clinical indicators are not immediatelysuitable <strong>for</strong> transfer to a national quality of care <strong>and</strong> health outcome indicator program withoutclose scrutiny of their suitability <strong>for</strong> that purpose <strong>and</strong> subsequent <strong>for</strong>mal trials of theirapplication in such a context.Numerous other indicator initiatives have occurred within facilities, benchmarking projects, bestpractice programs <strong>and</strong> the Health Departments of States <strong>and</strong> Territories. These initiatives havelargely not attempted to generate the sort of high level indicators which would be of value in anational monitoring program. Rather, they looked at indicators suitable <strong>for</strong> the specific purposeat h<strong>and</strong>. We found quality of care issues dealt with by a broad range of interested parties(including a variety of departments <strong>and</strong> sections in governmental <strong>and</strong> regulatory bodies) <strong>and</strong> itproved difficult to access comprehensive in<strong>for</strong>mation on work in progress or recently completedindicator development work in the timeframe of the Project. Additionally, it appeared that someagencies were ambivalent about contributing to a developmental program <strong>for</strong> a nationallyconsistent set of quality <strong>and</strong> outcome indicators <strong>and</strong> may have tempered their contributions toour research in proportion to this ambivalence.Some examples of identified quality indicator applications include:Commonwealth The Commonwealth Department of Health <strong>and</strong> Family <strong>Services</strong> supports theACHS CEP clinical indicator program (see above). Within the Health <strong>Services</strong> <strong>Outcome</strong>sBranch it has groups looking at primarily patient safety, health models, consumer perspectiveson health, health outcomes policy, workplace change promotion <strong>and</strong> healthcare quality <strong>and</strong>outcomes. Recent key initiatives include the funding of a study addressing the reliability, validity<strong>and</strong> risk-adjustment issues around four so-called “hospital-wide quality of care indicators” (i.e.wound infection, nosocomial bacteraemia, unplanned return to theatre <strong>and</strong> emergencyreadmission).The Health Models Development Section supports an Ambulatory Care Data Working Groupwhich has recently proposed per<strong>for</strong>mance indicators <strong>for</strong> Ambulatory Care with a potentialrelevance <strong>for</strong> the acute healthcare sector. These are:♦♦♦♦♦♦Waiting times in Emergency Departments - based upon The Australian College ofEmergency Medicines five triage categories.Waiting times <strong>for</strong> admission to hospital from the Emergency Department.Cost per outpatient separation.Patient satisfaction.Waiting times <strong>for</strong> outpatients appointments.Waiting times in outpatients per attendance.At present these indicators have conceptual definitions, with operational definitions <strong>and</strong>subsequent field trials pending.

19The National Demonstration Hospital Program has identified per<strong>for</strong>mance indicators whichrequire development at three levels (program, project <strong>and</strong> hospital). To date, some 20 programlevel per<strong>for</strong>mance indicators have been advanced. They have crude operational definitionsprovided, which would be inadequate <strong>for</strong> routine applications. These indicators do not covermany of the significant dimensions of care quality (e.g. effectiveness, technical proficiency,safety) being restricted to access <strong>and</strong> efficiency indicators (as is the purpose of the program).They are:♦ Percentage of Category 1 patients waiting more than specified days.♦ Percentage of Category 2 patients waiting more than appropriate time.♦ What is the average excess wait (days) <strong>for</strong> patients classed as Category 1?♦ What is the average excess wait (days) <strong>for</strong> patients classed as Category 2?♦ By how many does the total waiting list number exceed appropriate levels in each category?♦ Trend in elective admissions from the waiting list this month.♦ Does the percentage change in all admissions exceed the percentage change in electiveadmissions?♦ Does the percentage change in admissions exceed the percentage change in annual funding?♦ Change in unplanned readmissions within 28 days.♦ Percentage of elective patients surveyed <strong>for</strong> satisfaction.♦ Level of overall elective patient satisfaction with total admission.♦ Is the waiting list at least equal to a month’s elective admissions?♦ Surgical patients treated per sessional hour per month.♦ Is operating room utilisation improving?♦ Percentage elective admissions on planned day of surgery.♦ Percentage of same-day surgical admissions.♦ Percentage of electives cancelled on planned day of surgery.♦ Patients treated per available bed per month.♦ Months since last audit based on direct patient survey.The Health Service <strong>Outcome</strong>s Branch, through the <strong>Quality</strong> <strong>and</strong> <strong>Outcome</strong>s Section, are funding atrial of an Integrated <strong>Quality</strong> Management Model which will involve, amongst a range ofactivities, the application of indicator sets within provider facilities. These will be developedaround the principle of the “Value Compass” (of Hospitals Corporation of America) 492 <strong>and</strong> willinvolve balanced indicator sets <strong>for</strong> specific clinical conditions incorporating indicators of clinicaloutcomes, functional health status, patient perception <strong>and</strong> cost. The actual indicator sets to beapplied are currently under development.The Commonwealth is engaged in developing numerous additional indicators, many of which arein areas outside acute health services. A complete list is included in Appendix 6.The Australian Institute of Health <strong>and</strong> Welfare play a pivotal role in the collection <strong>and</strong> analysisof current per<strong>for</strong>mance indicator data. They have been instrumental in establishing the interimpopulation norms <strong>for</strong> the SF-36, a functional health status questionnaire designed to measuregeneric aspects of health. They also have produced the First National Report on Health SectorPer<strong>for</strong>mance <strong>Indicators</strong> <strong>for</strong> the National Health Ministers’ Benchmarking Working Group. Thisinaugural report contains indicators relevant to quality as follows:♦♦♦♦♦♦Cost per casemix adjusted separation.Cost of treatment per outpatient.Inpatient average length of stay <strong>for</strong> top 20 AN-DRGs.Rate of emergency patient readmissions within 28 days.Rate of unplanned return to operating room.Patient satisfaction.

20♦♦♦♦♦♦Proportion of facilities accredited with ACHS.Waiting times <strong>for</strong> elective surgery.Accident <strong>and</strong> emergency waiting times.Outpatient waiting times.Variations in intervention rates.Separations per 1000 population.The authors of this report emphasise that consistent national data is currently unavailable <strong>for</strong>many of these indicators <strong>and</strong> caution - appropriately - that they be interpreted in light of thesemethodological deficiencies. These caveats notwithst<strong>and</strong>ing, it is likely that this initiative of TheAustralian Health Ministers’ Conference marks the first step on a journey to improvedper<strong>for</strong>mance indicators of increased relevance <strong>and</strong> utility.Victoria The Department of Health <strong>and</strong> Human <strong>Services</strong> (then Health <strong>and</strong> Community <strong>Services</strong>)completed a comprehensive review of quality activities in acute healthcare in 1995 - the resultsof which are contained in “A New Framework <strong>for</strong> <strong>Quality</strong> in Victoria’s Public Hospitals: FinalReport” 357,358 . This identifies several major quality <strong>and</strong> outcome indicator initiatives acrossVictoria sponsored by The Department of Health <strong>and</strong> Human <strong>Services</strong>:♦♦♦♦♦Monitoring of unplanned hospital readmissions.St<strong>and</strong>ard survey of patient satisfaction (based upon the Picker Commonwealth Instrument).A requirement <strong>for</strong> hospital quality plans.Subsidisation <strong>for</strong> participation in the ACHS hospital accreditation program.Monitoring of Emergency Department per<strong>for</strong>mance.Additional identified indicators included:• Percentage of private separations/facility.• Percentage of private bed days/facility.• Average available beds.• Average length of stay (by ANDRG).• Non-admitted patient occasions of service.• Separations/facility.• Same-day medical separations as percentage of all separations.• Non-same day emergency separations as percentage of all separations.• Non-same day elective separations as a percentage of all separations.• Nursing home type bed days <strong>and</strong> separations.• Regional separations as percentage of all separations.• Outside region non-tertiary separations as a percentage of all non-tertiary separations.• Non-tertiary separations from tertiary hospitals as percentage of all network non-tertiaryhospitals.• Tertiary separations from tertiary hospitals as percentage of all network tertiary hospitals.• Transfers between network campuses as a percentage of total separations.• Transfers to/from other networks as a percentage of total separations.South Australia Activities initiated through the Casemix <strong>and</strong> Clinical Costing Unit of the SouthAustralian Health Commission include:♦♦♦♦♦Emergency patient readmissions.Returns to theatre.Nosocomial infections.Health status assessment (SF36).Patient Satisfaction.

21♦♦♦♦♦♦Waiting times <strong>for</strong> elective surgery.Available bed days: Daily average.Cost per casemix adjusted separation.Cost per casemix treatment per patient.Inpatient average length of stay by DRG.Proportion of facilities accredited by ACHS.The South Australian Health Commission has also undertaken considerable research <strong>and</strong>development on utilisation review. This management tool particularly addresses issues aroundsite of care (such as appropriateness of admission) or intensity of care (such as numbers ofinvestigations per case). We regard utilisation review as an important facility managementinstrument with little application in any national quality <strong>and</strong> outcome indicator programaddressing acute hospital care <strong>and</strong> have not included review of this, or other utilisation reviewprotocols, within our review.Australian Capital Territory The Department of Health <strong>and</strong> Community Care identifiedevaluation of the SF36 health status survey as a relevant activity <strong>for</strong> quality <strong>and</strong> outcomeindicator development, occurring under the auspices of the Care Continuum <strong>and</strong> Health<strong>Outcome</strong>s Project. Additional developmental work is currently underway in:• Ambulatory care.• Palliative care.• Breast cancer.• Hospital-based cancer care.• Discharge planning.• Asthma.• Injury.• Orthopaedic rehabilitation domiciliary service.• Cardiac disease.Western Australia The Health Department of Western Australia indicated that a number ofper<strong>for</strong>mance measures had been applied in the past, although existing purchaser/providercontracts have not had any quality indicators reported. A sample of past per<strong>for</strong>mance measuresincluded:♦ Average cost per bed day.♦ Average cost per admission.♦ Average cost per non-inpatient occasion of service.♦ Average length of stay per admission.♦ Median wait time from hospital waiting list.♦ Percentage of clients satisfied with their hospital stay.♦ Overall satisfaction index.♦ Percentage of hospitals operating quality assurance programs.♦ Percentage of hospitals accredited.♦ Hospital acquired wound infection.♦ Apgar score of four or less at five minutes <strong>for</strong> newborns whose birthweight was at least 500grams.♦ Geographic variation in admission rates per 1000 population.Tasmania Under a “Health Goals <strong>and</strong> Targets <strong>for</strong> Tasmania” program, a process to identify keyhealth issues was commenced in May 1992. Currently there are no state-wide indicatorinitiatives, although Regional Business Plans have an agreed set of per<strong>for</strong>mance indicators -including some quality indicators - against which they report. These indicators include:

22♦ Emergency Department waiting times by triage code.♦ Outpatient waiting times by specialist area.♦ ACHS CEP hospital-wide medical indicators (nosocomial infection, unplanned return totheatre, post-operative pulmonary embolism <strong>and</strong> unplanned readmission).Northern Territory No state-wide indicator collection is currently occurring. Major providerfacilities (e.g. Royal Darwin Hospital) are collecting ACHS CEP hospital wide medicalindicators, including:♦ Unplanned return to the operating room.♦ Nosocomial infection.♦ Unplanned readmissions.Queensl<strong>and</strong> Queensl<strong>and</strong> Health have data available on:♦ Client satisfaction with Accident <strong>and</strong> Emergency Department care delivery.♦ Separation rates.♦ Cost of care delivery.They support pursuit of ACHS accreditation (<strong>and</strong> hence indirectly promote use of ACHS CEPclinical indicators). They noted a current inability to track <strong>and</strong> report elective surgery accesstimes because of limitations to existing in<strong>for</strong>mation systems. They identify ongoing projectsaround waiting lists, commercial key per<strong>for</strong>mance indicators, client services st<strong>and</strong>ards <strong>and</strong> bestpractice implementation.New South Wales The New South Wales Health Department has been reviewing the utility of anumber of ACHS hospital-wide medical indicators in quality assessment (including unplannedreadmission, unplanned return to theatre) <strong>and</strong> have an extensive developmental programaddressing issues surrounding the identification of nosocomial infection. In addition, routinemonitoring of elective surgery queuing times <strong>and</strong> emergency department waiting times isoccurring. Pilot patient satisfaction surveys have been per<strong>for</strong>med across NSW hospitals,including an emergency department survey.Specific indicators currently applied by New South Wales Health include:♦ Unplanned readmission to hospital or inpatient health service within 28 days of dischargefrom an inpatient health service where the original admission had been <strong>for</strong> an electiveprocedure.♦ Patients assessed in Outpatient Department within 1 month of referral.♦ Patient satisfaction with their access to in<strong>for</strong>mation.♦ GP satisfaction with their access to in<strong>for</strong>mation.♦ “Urgency 1 <strong>and</strong> 2” patients waiting longer than 1 month.♦ “Urgency 3” patients waiting longer than 6 months.♦ “Urgency 3” patients waiting longer than 12 months.♦ “Urgency 3” elective surgical patients given definite dates of admission with 12 weeks ofplanned procedure.♦ Elective surgical patients attending preadmission process.♦ Elective patient satisfaction with their waiting time <strong>for</strong> inpatient care.♦ Elective patients having their planned admission date brought <strong>for</strong>ward.♦ Elective patients admitted on day of procedure (selected).♦ Patients having an unplanned return to operating room.♦ Patients having a procedure as a “same day” patient (selected).♦ Patients discharged on anticipated date.

23♦ Patients in<strong>for</strong>med of their discharge date on admission.♦ Relative stay index <strong>for</strong> 5 elective surgical procedures.♦ Elective patients delayed (not self deferred).♦ Waiting time from Emergency Department arrival to be seen by medical officer (by triagecategory).♦ Emergency Department treatment time: from time seen by medical officer to admission toinpatient bed.It should be noted that existing State <strong>and</strong> Territory programs have paid little attention toascertaining the reliability <strong>and</strong> validity of indicator data provided to central indicator monitoringprograms. Typically, indicator data are used to aid policy implementation <strong>and</strong>/or to satisfyrudimentary accountability requirements. The only research <strong>and</strong> development activitiesidentified by the Project team primarily directed at acute care quality <strong>and</strong> outcome indicators isthe nosocomial infection indicator work underway within NSW 360 . Overall States <strong>and</strong>Territories are tending to promote ACHS-CEP hospital-wide medical indicators, rather th<strong>and</strong>eveloping de novo indicator programs, although the operational definitions <strong>for</strong> these indicatorsare frequently modified by individual States <strong>and</strong> Territories to suit the unique requirements oftheir data h<strong>and</strong>ling systems. As a result, even the in<strong>for</strong>mation available on apparently similarACHS CEP-based indicators may not be comparable between States <strong>and</strong> Territories.The State <strong>and</strong> Territory indicators have largely been selected from those in common usagenationally <strong>and</strong> internationally. Typically, they are readily available from routine data sources.Those indicators requiring additional data collection have frequently not yet been introduced intoroutine monitoring programs. Many comments on the lack of knowledge of the reliability <strong>and</strong>validity were made. Concerns were expressed frequently about lack of indicator risk adjustment.Although some limited evidence of the application of indicators in local quality improvementef<strong>for</strong>ts were reported, typically little was known of the utility of indicator data. Often thoseindicators in use <strong>for</strong> longer periods had more caveats <strong>and</strong> concerns regarding their usefulness -perhaps reflecting accumulated knowledge of their limitations with continued application (<strong>for</strong>example, many reported uncertainty regarding the value of the unplanned readmission indicator).5.3.2 North America The past two decades have seen an explosion of activity in quality <strong>and</strong>outcome indicator development <strong>and</strong> implementation in North America. The range of indicatorsapplied <strong>and</strong> the multitude of agencies developing <strong>and</strong> implementing indicator programs makes acomprehensive review of these activities a daunting task. We have elected to present briefsummaries of programs that illustrate seminal principles in indicator development, novelindicator strategies or competing views on indicator applications. Additional detail on NorthAmerican quality indicators is contained in Appendix 6.NATIONAL PROGRAMS/MULTISTATE PROGRAMS OF NOTE IN USA1. Joint Commission on Accreditation of Health Care Organisations (JCAHO)Indicator Measurement System (IM System)2. National Committee <strong>for</strong> <strong>Quality</strong> Assurance (NCQA)Health Employer Data <strong>and</strong> In<strong>for</strong>mation Set (HEDIS)3. Maryl<strong>and</strong> Hospitals Association (MHA) <strong>Quality</strong> Indicator Project (QIP)4. Consortium Research on <strong>Indicators</strong> of System Per<strong>for</strong>mance (CRISP)

24JCAHO IM System: This quality <strong>and</strong> outcome indicator program has been under developmentsince 1985 14,138,144 . As at 1996 it consists of 33 indicators covering obstetrics, perioperativecare, cardiovascular, trauma, oncology, infection control, medication use <strong>and</strong> health carenetwork per<strong>for</strong>mance. In 1997, additional indicators <strong>for</strong> depressive disorders, long term care <strong>and</strong>home infusion therapy are planned <strong>for</strong> introduction.The JCAHO launched the IM System with a view to making it the pre-eminent indicator programin the USA - the use of which would eventually be m<strong>and</strong>atory <strong>for</strong> providers wishing to seekJCAHO accreditation 142 . There was considerable industry opposition to a clinical per<strong>for</strong>mancemonitoring system developed by a regulatory body. This opposition <strong>and</strong> the desire by JCAHO tom<strong>and</strong>ate IM System use broadly across the USA resulting in a rigorous developmental programwhich has actively sought in<strong>for</strong>mation regarding the reliability, validity <strong>and</strong> relevance ofindicators.The IM System has detailed operational definitions available <strong>for</strong> all indicators 142 (see example<strong>for</strong> nosocomial infection: Appendix 13) which help ensure data reliability. An extensiveresearch program, based around an initial (Alpha) testing <strong>for</strong> face validity <strong>and</strong> indicatorfeasibility is followed by a more exhaustive second phase (Beta testing) examination addressingreliability, validity <strong>and</strong> relevance issues. Whilst some tens of facilities participate in Alphatesting, surveys indicator data collection <strong>and</strong> on-site visits <strong>for</strong> several hundred facilities occur inBeta testing - providing firm data upon which to base judgements on indicator attributes.The IM System is based around a common software program which guides accurate datacollection <strong>and</strong> includes audit-checks <strong>for</strong> data reliability encoded in the software. A substantialresearch <strong>and</strong> development group <strong>and</strong> service infrastructure support indicator application byfacilities. Despite these features, it has enjoyed only modest penetration in the US market -reflecting competing indicator systems, concerns that the IM System is still developmental <strong>and</strong>resistance because of its origins in a regulatory body. In 1996 JCAHO elected to discontinue theplanned m<strong>and</strong>ating of IM Systems <strong>for</strong> the purposes of JCAHO accreditation, instead opting toreview <strong>and</strong> approve any indicator that fulfilled their criteria <strong>for</strong> methodological rigour. A call <strong>for</strong>indicators was made, inviting submission of any existing indicators or indicator sets to JCAHO<strong>for</strong> review by their expert panel. At completion of this review process, the JCAHO will publisha compendium of quality <strong>and</strong> outcome indicators which are deemed sufficiently wellcharacterised to meet their methodological rigour requirements <strong>and</strong> hence be approved <strong>for</strong> use infacilities seeking JCAHO accreditation. This compendium will offer a valuable entree to thestate-of-the-art in indicator science in North America (it is scheduled <strong>for</strong> release in the NorthernHemisphere Autumn 1996).The JCAHO IM System provides facilities with risk-adjusted 144 indicator data based uponpatient, not hospital, characteristics. Individual risk adjustment methodologies are designed <strong>for</strong>each indicator - typically using their database to generate “predicted” indicator rates with whichfacilities can compare their actual rates per indicator.NCQA: HEDIS HEDIS was designed to permit employers to judge the value of the healthcarethey are buying <strong>for</strong> their employees <strong>and</strong> to make health plans accountable <strong>for</strong> their per<strong>for</strong>mance307,313 . The NCQA - an accreditation body that focuses on health plans rather than hospitals <strong>and</strong>is the major player in this market - has developed the HEDIS measure set in three iterations (thusfar), i.e. HEDIS 1.0, HEDIS 2.0 <strong>and</strong> HEDIS 2.5. The HEDIS measures were a response to theproliferation of competing indicator systems amongst health plans which were typically notproducing comparable data. HEDIS was seen as providing a common language <strong>for</strong> guidingchoice of health plans. The measures were chosen by a Per<strong>for</strong>mance Assessment Committee(PAC), consisting of employers, NCQA <strong>and</strong> technical expertise representatives. To date, no dataare available on indicator/measure attributes, such as reliability or validity, although it is planned

25to collect such data <strong>for</strong> the HEDIS 3.0 measures currently under development. HEDIS 2.5includes more than 60 measures across the broad areas of:♦♦♦♦<strong>Quality</strong> of care.Member access <strong>and</strong> satisfaction.Membership <strong>and</strong> utilisation.Finance.The particular measures within each of these major areas were chosen on the basis of theirperceived relevance <strong>and</strong> value to the employer community, the reasonable ability of health plansto provide the requested data <strong>and</strong> their potential impact on improving health care delivery. Themeasures are a mix of population-based per<strong>for</strong>mance <strong>and</strong> episode of care per<strong>for</strong>mance indicators(see Appendix 6). Those specific <strong>for</strong> acute episodes of care could feasibly be adapted <strong>for</strong> use inacute healthcare in Australia. At present HEDIS data, although frequently stratified, are notrisk-adjusted. Relevant indicators in HEDIS 3.0 are proposed to be risk-adjusted using indicatorspecificmodels developed around patient level data. NCQA has also had a “Call <strong>for</strong> <strong>Indicators</strong>”in 1996, seeking indicators suitable <strong>for</strong> inclusion in HEDIS 3.0.MHA: QIP The oldest of the multifacility indicator monitoring programs, the MHA QIPprovides comparable indicator data on 10 inpatient quality <strong>and</strong> outcome indicators <strong>and</strong> 5ambulatory care indicators to over 1000 participating facilities in North America, Japan, Europe<strong>and</strong> the United Kingdom (Appendix 6). It is the largest single indicator monitoring program inexistence anywhere in acute healthcare. The Project seeks to educate participants in the use ofquality indicator data <strong>for</strong> quality improvement. The MHA QIP guarantees confidentiality <strong>for</strong> allparticipant facilities. It does not risk-adjust data, primarily because of a belief that it isunnecessary as indicators are intended only <strong>for</strong> facility quality endeavours <strong>and</strong> not comparativejudgement by others. <strong>Indicators</strong> have available data on reliability, validity <strong>and</strong> relevance -collected over their 10 years of experience with indicator use. The software provided to facilitatedata collection contains data reliability checks. There are several documented case-studiesavailable which demonstrate favourable strategies developed in response to initial indicator datareview (such as a progressive reduction in absolute rates <strong>and</strong> the extent of variation betweencentres <strong>for</strong> Caesarean section in New Hampshire).CRISP This is a voluntary collaborative program administered from The Henry Ford HealthSystem in Detroit. Eighteen integrated healthcare systems participate in a research programwhich aims to identify preferred indicators of healthcare delivery per<strong>for</strong>mance. <strong>Indicators</strong> aredivided by category:♦♦♦♦♦♦♦♦♦♦♦Population health.Community benefit.<strong>Quality</strong> of care.Episode prevention.Episode characteristics.Satisfaction.Efficiency.Capacity.Financial per<strong>for</strong>mance.Research.Education.At present, indicators lie in two tiers, the first representing indicators of established reliability,validity <strong>and</strong> utility <strong>and</strong> the second tier a group of developmental indicators 300,301,302,303 (seeAppendix 6 <strong>for</strong> further detail). CRISP uses risk adjustment to increase the comparability ofper<strong>for</strong>mance indicators across the systems. The risk adjustment models are developed <strong>for</strong> each

26indicator <strong>and</strong> based on patient level data, typically demographic descriptors (e.g. age, sex,race/ethnicity, educational attainment). CRISP maintains all indicator data as confidentialcommunications - arguing that, at present, the data lie within a research program, are notsufficiently well characterised <strong>for</strong> further release <strong>and</strong> failure to protect participant confidentialitywould compromise the ultimate development of credible indicators suitable <strong>for</strong> public release.There are several other noteworthy examples of quality <strong>and</strong> outcome indicator applications inNorth America. They include:CONQUEST Computer needs-oriented quality measurement evaluation system (CONQUEST)is a system of interlocking databases summarising in<strong>for</strong>mation on approximately 1200 clinicalper<strong>for</strong>mance measures used by public <strong>and</strong> private sector organisations to examine technicalquality of clinical care. Developed as a collaboration between the Centre <strong>for</strong> <strong>Quality</strong> of CareResearch <strong>and</strong> Education at the Harvard School of Public Health (in Boston) <strong>and</strong> the Centre <strong>for</strong>Health Policy Studies (in Colombia) it is essentially a typology of clinical per<strong>for</strong>mance measureswhich allows identification of measures on the basis of characteristics or properties of thesemeasures. This software is based upon a research project commissioned by the Agency <strong>for</strong>Health Care Policy <strong>and</strong> Research (AHCPR) in 1994. It can be used to search <strong>for</strong> measuresunder:• Name.• Rigour of development.• Organisation type developed <strong>for</strong>.• Intended use.• Extent of use.• Practicality.• Clinical event.• Age of patient.• Care needs.• Care setting.• Process/outcome.• Data source.• Sampling frame.• Stratification method.• Time window.• Allowance <strong>for</strong> patient factors in scoring.• Comparative st<strong>and</strong>ards.• Display of results.Its widespread availability (anticipated <strong>for</strong> mid-1996) to health service researchers should speedaccess to in<strong>for</strong>mation on existing indicators <strong>and</strong> indicator sets.FACCT: The newly established Foundation <strong>for</strong> Accountability plans to endorse qualityindicators, advocate their widespread use by providers <strong>and</strong> promote consumer use of endorsedindicators. They have developed criteria <strong>for</strong> assessing per<strong>for</strong>mance indicators <strong>for</strong> use byconsumers (Guidebook <strong>for</strong> Per<strong>for</strong>mance Measurement) <strong>and</strong> are likely to contribute significantlyto the adoption of indicators by health plans <strong>and</strong> acute care users across America.Veterans’ Affairs: The US Department of Veterans Affairs has an active research programencompassing a range of studies on the use of process <strong>and</strong> outcome indicators in healthcare, therelationships between selected process measures <strong>and</strong> observed outcomes, the reliability <strong>and</strong>validity of particular indicators <strong>and</strong> the application of patient-based measures of health status,process <strong>and</strong> outcome in per<strong>for</strong>mance assessment. They have refined risk-adjustment

27methodologies <strong>for</strong> use in veterans populations <strong>and</strong> developed instruments to facilitate technicalproficiency assessments. The VA per<strong>for</strong>m much of the detailed methodological researchnecessary to ensure that indicators are appropriately used <strong>for</strong> quality monitoring in their acute<strong>and</strong> integrated healthcare systems.Clevel<strong>and</strong> Health <strong>Quality</strong> Choice: This exemplary program was the first healthcare marketre<strong>for</strong>m plan of its kind in the USA which brought business, providers, physicians <strong>and</strong> expertisein quality measurement together in a voluntary collaborative ef<strong>for</strong>t to measure <strong>and</strong> improve thequality <strong>and</strong> af<strong>for</strong>dability of healthcare services. It commenced in 1989. It has developed its ownindicators to measure <strong>and</strong> compare the quality of selected services at participating hospitals inthe Greater Clevel<strong>and</strong> area. It evaluates four service areas:• Surgery: e.g. Total hip replacement• General Medicine: e.g. Pneumonia treatment• Intensive Care: e.g. Respiratory failure• Obstetrics <strong>and</strong> Gynaecology e.g. ChildbirthIt reports utilisation, in-hospital mortality, length of stay <strong>and</strong> patient satisfaction across a rangeof high volume or high cost interventions (see detail: Appendix 6). All data is risk-adjusted,using the <strong>Acute</strong> Physiology <strong>and</strong> Chronic Health Evaluation (APACHE III) System of adjustment<strong>for</strong> intensive care patients <strong>and</strong> a customised, patient-level risk adjustment system <strong>for</strong> noncriticallyill populations. The quality indicator data is first validated by a stringent reviewprocess involving independent experts, the CHQC <strong>and</strong> providers. Following the confirmation ofdata reliability <strong>and</strong> validity it is released publicly. The initial development <strong>and</strong> confirmation ofdata credibility prior to public indicator data release required a three year cooperative program.Semi-annual consumer reports are now published which provide indicator data <strong>and</strong> guidancewith interpretation. A summary <strong>for</strong>mat is available to the public. Because of the complexity ofthe complete set of data <strong>and</strong> the sensitivity associated with the interpretation of the 300 page pluscomprehensive report, it is m<strong>and</strong>atory to attend a half day training workshop in appropriate useof the data be<strong>for</strong>e being provided access to the comprehensive report. This program,components of which has been replicated in various guises across North America, contains anumber of cardinal features which have determined its success, including rigorous developmentof credible measures <strong>and</strong> the cooperative development program involving providers <strong>and</strong>purchasers.New York <strong>and</strong> Pennsylvania States: These States collect, calculate <strong>and</strong> publish in<strong>for</strong>mation onquality of care obtained from m<strong>and</strong>ated hospital discharge data sets. They report risk-adjustedmortality <strong>for</strong> Coronary Artery Bypass Grafting (CABG), major morbidity, length of stay,charges <strong>and</strong> physician-specific CABG mortality. These have been controversial programs, attimes appearing confrontational. There is no evidence that consumers’ or providers’ behaviourshave changed favourably as a consequence of public release of this data. Initial reports ofimproved patient outcomes with CABG proved to be due to conscious or unconsciousmanipulation of contributed data to favourably realign risk-adjusted mortality prediction. Thecosts of wide-scale data collection <strong>for</strong> the risk-adjustment methodologies used - <strong>and</strong> <strong>for</strong> systemsto audit data reliability - are considerable.HCFA’s Mortality Analyses: In 1986 the Health Care Financing Administration (HCFA)released via the media risk-adjusted overall mortality data <strong>for</strong> US hospitals treating Medicarepatients. For several subsequent years, HCFA continued to release these outcomes data asputative quality indicators, despite concerns about the accuracy of administrative databasesunderpinning indicator construction <strong>and</strong> the detail of the risk-adjustment models used to predictwhether quality of care fell within likely acceptable bounds. Considerable energy <strong>and</strong> expensewere devoted to identifying the deficiencies of this attempt at public accountability. EventuallyHCFA itself ceased publication of such indicator data when internal review failed to demonstrate

28that mortality categorisation of quality of care aligned with independent determinations of carequality (i.e. the approach was invalid). In many respects this experiment is a model <strong>for</strong> how notto introduce putative quality of care <strong>and</strong> health outcome indicators.HCFA’s Health Care <strong>Quality</strong> Improvement Initiative: As part of a new focus on qualityimprovement, HCFA’s HCQII is seeking to improve the mainstream of care, rather than relyingon identifying isolated apparent error. The initiative seeks to get doctors <strong>and</strong> hospitals involvedin the analysis of treatment patterns <strong>and</strong> promote improvement through adherence to bestpractice processes. The initiatives see HCFA, doctors, hospitals <strong>and</strong> specialty societies worktogether to develop indicators of quality of care. HCFA then facilitate data collection, analysis<strong>and</strong> confidential feedback to providers <strong>and</strong> promote use of the data <strong>for</strong> internal qualityimprovement <strong>and</strong> benchmarking activities.An example of this program, focused on care of patients with myocardial infarction, saw 26proposed quality of care indicators refined to 12 final quality indicators (see Appendix 6 <strong>for</strong>detail). These indicators reflect the provider focus of these activities. They are largely measuresof technical proficiency based upon acknowledged guidelines <strong>for</strong> the management of acutemyocardial infarction. The approach taken by HCQII has much to recommend it <strong>and</strong> reflects therealisation by HCFA that its previous unilateral approach to quality monitoring, in<strong>for</strong>mingconsumers <strong>and</strong> accountability (see HCFA Mortality Analysis - above) was unsuccessful.Health Care Report Cards: We have identified three dozen or so Health Care Report Cards,released to the public in the USA in recent years (predominantly post 1993). They contain avariety of putative quality <strong>and</strong> outcome indicators (see Appendix 6). Typically, there is no detailas to the operational definition <strong>for</strong> presented indicators nor <strong>for</strong> data reliability or validity. Giventhe “<strong>for</strong> profit” status of many groups generating these self-reports of per<strong>for</strong>mance, it issignificant that none of the reports contain data independently audited <strong>for</strong> accuracy. It is open todebate how valuable the contained in<strong>for</strong>mation is to healthcare consumers - as the targetaudience <strong>for</strong> these initial attempts at consumer reports was frequently healthcare purchasers,rather than the direct recipients of service delivery.5.3.3 United KingdomThe predominant focus of activities in the UK has been the rein<strong>for</strong>cing <strong>and</strong> st<strong>and</strong>ardisation ofClinical Audit <strong>and</strong> an aligning of indicators of quality <strong>and</strong> outcomes of care with the NHSPatient Charter. The UK has been at the <strong>for</strong>efront of the evolution of so-called “Evidence-BasedMedicine” <strong>and</strong> are currently active in designing quality <strong>and</strong> outcome indicators within thiscontext. There has been relatively little in the way of comprehensive national quality <strong>and</strong>outcome indicator programs to date - although a report from the Clinical Accountability <strong>and</strong>System Per<strong>for</strong>mance Evaluation (CASPE) Research Group to the Department of Health(anticipated <strong>for</strong> the European summer, 1996) will identify quality <strong>and</strong> outcome indicators <strong>for</strong> tencommon health conditions (including asthma, diabetes, heart failure, bladder outlet obstruction,childbirth, back pain, hypertension, ischaemic heart disease <strong>and</strong> menorrhagia) that span theinterests <strong>and</strong> perspectives of patients, providers <strong>and</strong> purchasers of health. Many of theseindicators will have data on reliability <strong>and</strong> validity based upon field-testing over the past threeyears. It is unclear at present what use the Department of Health will make of these indicators.They are targeted at assessment of an integrated healthcare system, rather than hospitals, but arelikely to include indicators of relevance to acute healthcare.The National Health System (NHS) so-called “League Tables” have been published <strong>for</strong> severalyears - encompassing a range of indicator data including waiting times <strong>and</strong> overall mortality.The data is not risk-adjusted <strong>and</strong> is not audited. There is no evidence that it has materiallyaltered provider behaviour with regard to care delivery. Several academic departments <strong>and</strong>research consortia are active in the development, characterisation <strong>and</strong> promotion of health status<strong>and</strong> quality of life assessment instruments (e.g. The London H<strong>and</strong>icap Scale). Whilstcontributing much to the characterisation of such instruments, there is little evidence of attempts

29to systematically introduce these outcome measures into current practice on a wide scale. Recentchanges in funding of acute healthcare, with enhancement of the purchaser/provider split (socalled),are seeing increasing sophistication in contractual arrangements between those fundingcare <strong>and</strong> those delivering care. Based upon the examples made available to us, there was littleevidence of sophistication in quality per<strong>for</strong>mance indicators within these contracts with mostemphasis on volume <strong>and</strong> cost. Those indicators used were “conventional” (e.g. patientsatisfaction; wound infection). Accreditation of acute hospitals is a relatively recent concept inthe UK. The King’s Fund Organisational Audit Unit is the predominant <strong>for</strong>ce in accreditation.They currently adopt a st<strong>and</strong>ards-based approach, not dissimilar to the ACHS prior toimplementation of the Charter <strong>for</strong> Change. There are no requirements <strong>for</strong> the routine applicationof quality of care indicators <strong>for</strong> the purposes of accreditation.In the absence of a systematic national approach to quality <strong>and</strong> outcome indicators, the NHS hasfunded a regional trial of the feasibility of UK hospitals’ participation in the MHA QIP. Thistrial of participation, coordinated from Newcastle University, has been judged a success by thoseinvolved. It is currently being exp<strong>and</strong>ed to involve more facilities. It provides an interestingmodel <strong>for</strong> “instant” access to an established quality indicator program, providing comparativedata <strong>for</strong> quality improvement at a relatively modest cost. Such an approach may prove attractiveto individual hospitals or hospital groups in Australia, although the confidentiality requirementsof MHA QIP would preclude its use as a substitute <strong>for</strong> a national monitoring program.5.3.4 EuropeMost European countries are involved in active, large-scale quality monitoring programs. Themajority are based on cooperative, quality improvement focused activities (such as the WorldHealth Organisational confidential indicator feedback programs in diabetes care, hypertension<strong>and</strong> dental care). In large part, these programs are similar to initiatives described <strong>for</strong> NorthAmerica <strong>and</strong> are using indicators that are conceptually similar, although operational definitionstend to be unique as they are tailored <strong>for</strong> local practice patterns <strong>and</strong> in<strong>for</strong>mation systems.Exemplary programs identified include:♦♦♦WHO collaborative demonstration project: This has established quality <strong>and</strong> outcomeindicators <strong>for</strong> common conditions (e.g. diabetes mellitus <strong>and</strong> dental caries) <strong>and</strong> fed-backconfidential reports to individual clinicians on their own per<strong>for</strong>mance vis-a-vis theseindicators (e.g. retinopathy screening <strong>for</strong> diabetes or caries rates <strong>for</strong> dental care). Linked toeducational programs based around benchmark processes of care (identified through theseprograms), these programs have seen a progressive <strong>and</strong> substantive improvement in auditedoutcomes.Establishing <strong>Quality</strong> Improvement in <strong>Healthcare</strong> in Spain: Over the past 10 years theMinistry of <strong>Healthcare</strong> <strong>and</strong> Consumer Affairs in Spain has been promoting a program tointroduce routine per<strong>for</strong>mance measurement <strong>and</strong> the utilisation of measured per<strong>for</strong>mance <strong>for</strong>improvement. This program has focused on ensuring that data collected to construct qualityindicators is reliable <strong>and</strong> continues to work to develop appropriate risk-adjustment modelsbased on patient-level data. Initial confidential data collection <strong>and</strong> validation was followedby collaborative public release of conventional indicator data (e.g. mortality; queuingtimes).Norway’s Contract <strong>for</strong> <strong>Quality</strong>: Norway has established explicit access targets <strong>for</strong>healthcare providers based upon designated medical or surgical conditions. These targetsstate access st<strong>and</strong>ards (e.g. waiting times <strong>for</strong> elective surgery or ambulatory specialtyreview by condition) but do not as yet stipulate outcome targets.

305.4 Data Sources <strong>for</strong> Indicator ConstructionThere are essentially four means of obtaining data <strong>for</strong> indicator construction:a) Administrative Databasesb) Medical Recordsc) Patient Surveysd) Hybrid Methods (i.e. more than one of the above).5.4.1 Administrative DatabasesLarge databases established <strong>for</strong> other purposes (e.g. billing; risk management; utilisation review;costing) are increasingly able to be linked together <strong>and</strong>/or applied to quite different purposes,such as quality indicator construction. There are inherent dangers in using data obtained <strong>for</strong> onepurpose in a different context, largely centred on data accuracy. Typically, data items perceivedto be of little direct relevance to the primary intent of data collection (e.g. comorbidity recordedin some billing databases) are poorly collected <strong>and</strong> hence inaccurate. These inaccuracies tend toresolve over time when it is appreciated that these data have an important secondary purpose -albeit of little relevance to the primary purpose of data collection.More serious limitations of administrative database are their lack of clinical detail relevant toquality indicator construction. This absence of significant clinical detail (e.g. history <strong>and</strong>examination findings, investigation results <strong>and</strong> a full intervention profile) severely limit theirusefulness <strong>for</strong> construction of clinically relevant indicators. Common coding systems (such asAN-DRG <strong>and</strong> ICD9-CM) frequently group conditions <strong>and</strong> interventions which do not logicallybelong together <strong>for</strong> process or outcome analyses - <strong>and</strong> it is not possible to “unbundle” thesedisparate groups after their numerical codes have been allocated. Administrative databases tendto offer more complete data at considerably less expense than alternative data sources. Untilcomprehensive, compatible clinical databases are developed around computerised medicalrecords, indicators derived from administrative databases will remain attractive, low-cost options<strong>for</strong> those implementing large-scale quality <strong>and</strong> outcome monitoring programs.5.4.2 Medical RecordsExisting paper medical records are a rich source of narrative detail on an individual episode ofcare. They are, however, frequently incomplete data sources <strong>and</strong> inherently of variable content<strong>and</strong> quality. The need <strong>for</strong> trained data abstracters makes data derived from medical recordsexpensive. Differences between the content of different medical record systems may hamperbroad-based monitoring programs - although knowledge of the need to identify particularin<strong>for</strong>mation within the medical record <strong>for</strong> subsequent quality monitoring ef<strong>for</strong>ts would be likelyto drive st<strong>and</strong>ardisation of entry of these data points nationally.5.4.3 Patient SurveysPatient surveys provide vital in<strong>for</strong>mation about the acceptability of care delivery, reports ofexperiences with the processes of care <strong>and</strong> health status <strong>and</strong> quality of life, patients’ healthbehaviours <strong>and</strong> intervention-specific outcomes. In recent years it has been established thatpatients provide accurate <strong>and</strong> reliable estimates of all these aspects of care - with the onlydemonstrated limitation being in reports of resource utilisation (where they consistentlyunderestimate utilisation rates in most reported survey comparisons). Although relativelyexpensive to administer, patient surveys provide invaluable in<strong>for</strong>mation that cannot be obtainedfrom any other source. Traditional written surveys are increasingly complemented or replacedby verbal (usually telephone) surveys or computer-aided surveys. There is also a trend to theamalgamation of components of specific-purpose surveys to create omnibus measures,generating data on several different domains of quality of care. These might include:

31• Patient details (e.g. age/sex/self reported diagnosis)• Description of processes of care (e.g. waiting times, communication, dischargeplanning)• Functional health status (e.g. SF36) or health related quality of life• Acceptability of care (e.g. satisfaction)• <strong>Outcome</strong>s of care (e.g. visual acuity after cataract extraction).Hybrid methods combine the advantages, <strong>and</strong> the disadvantages, of the above methods. Inparticular, they sum the cost burdens, <strong>and</strong> typically should be avoided unless the clinicalcircumstance dem<strong>and</strong>s the complexity <strong>for</strong> credible analysis, thereby justifying the added cost.Many indicators would more readily be derived if requisite data were m<strong>and</strong>ated in universalminimum data set requirements. However, many quality <strong>and</strong> outcome indicators may only berelevant <strong>for</strong> a relatively short period of time, as advances in medical care result in newper<strong>for</strong>mance indicators, supplanting previous indicators (such as in heart attack, where time tothrombolytic therapy has replaced use of anticoagulant drugs). Most indicators should be builtaround samples of care. Only when indicators are deemed robust, useful <strong>and</strong> likely to provedurable in the longterm, should there be consideration of revising minimum data sets solely <strong>for</strong>indicator construction. Although much useful in<strong>for</strong>mation exists within Australian minimumdatasets at present, quality <strong>and</strong> outcome indicators should not be restricted to all conceivablepermutations <strong>and</strong> combinations of existing data within administrative databases. Usefulindicators will require extending data collection into medical record abstraction <strong>and</strong> patientsurveys, with very occasional need to modify the uni<strong>for</strong>m data set, most probably to obtainrelevant risk-adjustment in<strong>for</strong>mation. Existing administrative coding systems (AN-DRG <strong>and</strong>ICD classifications) may require alterations to enable easy quality <strong>and</strong> outcome indicatorconstruction.Future revisions of these categorisation systems should seek input from those engaged in qualitymonitoring within hospitals to maximise any potential <strong>for</strong> routine indicator construction withminimisation of duplication of ef<strong>for</strong>t.5.5 Examples of <strong>Indicators</strong> by Dimensions of <strong>Quality</strong>This section of the results discusses a range of the specific indicators identified by us over the course ofthis Project. As already indicated, there are literally several thous<strong>and</strong> indicators in use (or proposed <strong>for</strong>use) worldwide. It is not practical - <strong>and</strong> we believe it undesirable - to discuss each of these severalthous<strong>and</strong> indicators serially, with comments on their indicator attributes. Rather, we will discuss measuresthat have some promise as future indicators in a nationally consistent quality <strong>and</strong> outcome indicator set<strong>and</strong> will illustrate, by example, indicators which failed to impress our reviewers - highlighting the specificcriteria with which substantive deficiencies were identified. More detailed discussion of each dimension isprovided within Appendices 11 <strong>and</strong> Access♦Waiting TimesMany quality monitoring programs use the concept of waiting time as the principal index ofaccess to care. Hence, waiting times are recorded <strong>for</strong>:: elective surgery...* by urgency category* by procedure* by specialty

32: outpatient appointments* by urgency category* by specialty: emergency departments* by urgency category: emergency admission: outpatient waiting time at episode of careThe use of any of these queuing indicators in initial national indicator sets would requirethat adequate data sources exist or could reasonably be anticipated being brought intoexistence over a short timeframe. Waiting times are typically recorded by providers <strong>and</strong>varying operational definitions (e.g. of waiting time, urgency categorisation) render manyindicators of apparently similar concepts non-comparable.There has been a move to st<strong>and</strong>ardise these operational definitions <strong>and</strong> to monitor queuingtime from the decision to deliver care to actual care delivery (sometimes called “decision toincision” time) to avoid gaming of queuing indicators <strong>and</strong> allow independent audit of actualper<strong>for</strong>mance.There have also been moves to stratify conditions in some relationship to need <strong>for</strong>healthcare. This is either on the basis of perceived global urgency (as is common practice,in various <strong>for</strong>ms, in Australia currently) or specific diagnostic groupings (as is the case inseveral Sc<strong>and</strong>inavian countries currently). Global urgency measures leave the decisionregarding the need <strong>for</strong> care with clinicians providing care. Narrow diagnosticcategorisation, although superficially attractive, would still require clinical judgement abouturgency <strong>and</strong> could provide incentives to shift diagnostic classification to suit queuingtargets. The necessity to retain urgency categorisation <strong>and</strong> add detailed categorisation wouldmake queuing data management difficult. Consensus on target waiting times within eachcategory <strong>and</strong> between categories may be difficult to obtain in provider communities, assingle procedures such as coronary artery disease requiring CABG may be eitherurgent/semi-urgent or non-urgent depending on the clinical circumstance on the patient.The use of patient-reported waiting times <strong>and</strong> judgements on the acceptability of reportedwaiting times are commonplace in North America <strong>and</strong> Europe. They accurately reflectindependently observed waiting times <strong>and</strong> are not subject to gaming by providers.Satisfaction with waiting times from the consumer perspective is relevant - <strong>and</strong> of particularvalue when interpreted in the light of the actual wait time. Satisfaction is largely dependentupon consumer expectations - which are moulded by numerous patient, provider <strong>and</strong>societal influences. Patients are more likely to report the total waiting time (i.e. time fromdecision to treat to receipt of treatment) than provider-based indicators such as those fromdecision time to the time of allocation of a booking <strong>for</strong> the procedure.Refinements of waiting time indicators include:• Clearance time: the time that would be required to clear the current waiting listusing current hospital elective surgery throughput estimates.• Proportion of patients admitted after waiting inappropriately.(See also discussion on Commonwealth access <strong>and</strong> efficiency indicators - within 5.3.1 -<strong>for</strong> additional refinements proposed by the National Demonstration Hospitals Program)

33An indirect indicator of access - patient emergency department walkouts after triage - hasbeen proposed as a useful indicator of emergency care access. Although influenced bywaiting time, several patient-related factors strongly impact upon absolute walkout rates(e.g. presence of psychiatric illness <strong>and</strong>/or substance abuse). Although a useful facilitylevelindicator <strong>for</strong> monitoring trends over time, we cannot recommend it <strong>for</strong> nationalapplication in the absence of suitable risk-adjustment <strong>for</strong> casemix. The experience of theMaryl<strong>and</strong> Hospitals Association with this indicator supports our view - with widelydiscrepant “walkout” rates between inner urban <strong>and</strong> other community facilities which havebeen relatively fixed <strong>for</strong> given facilities with particular geographic <strong>and</strong> casemix profilesover lengthy time intervals.♦Preventable Admissions/Avoidable DeathsThese are excellent indicators of the adequacy of primary care although at best indirectindicators of access to appropriate acute care services. They have a real role in assessingquality in an integrated healthcare system. However, the inability to dissect failures inprimary care delivery from any index of acute care access render them of limited value injudging quality of acute care services. Comparisons between regions or facilities wouldalso require knowledge of the relative prevalence of the index conditions in the servicedpopulation. As future indicators of overall health sector per<strong>for</strong>mance they are stronglysupported, but they are not of value as acute healthcare quality indicators.♦Condition Specific UtilisationThere have been attempts at inferring access to care based upon the relative utilisation ofapparently clinically related interventions/conditions. Although offering useful insights onclinical care patterns, these are of limited value as access indicators.e.g.Access to acute hospital care <strong>for</strong> ischaemic heart disease measured by the ratio ofIHD admission to IHD death.COMMENT: This would be confounded by severity of illness <strong>and</strong> quality of care <strong>and</strong>would tell little of access but much about the appropriateness of accessed care,although desired indicator rates are unknown.e.g.Access to CABG measured by rates of CABG compared with rates of angiography.COMMENT: This utilisation statistic would infer more about clinical practicepatterns regarding the appropriateness of these interventions than access.♦Geographic Access MeasuresAlthough some complex models exist to define geographic access to care, these are usuallyderived to justify policy decisions regarding relocation of services or guide future planningof service profiles. Most studies of consumers identify time to service delivery as crucial -rather than distance - with a hierarchy of acceptable times based upon the level ofsophistication <strong>and</strong> complexity of care required. Assumptions about transportation modelsinherent in most geographic access models (e.g. public transport versus private vehicle;motor vehicle versus air ambulance) further limit their practical value as access measures inacute healthcare.

34♦Stratified Utilisation DataKnowledge of the age, sex, ethnic background <strong>and</strong> social class of those receiving acute careservices, coupled with knowledge of disease prevalence in these population strata, enableinferences about relative access of care. Such analyses are particularly valuable as pointersto groups who have needs <strong>for</strong> care but fail to access care. Linking of such utilisation rateindicator data to population health needs data (such as from the Australian Bureau ofStatistics Health Status <strong>and</strong> Health-Related Behaviour Survey) would be a mechanism bywhich indicator data could assist decision-making on access.Utilisation (e.g. discharge rate per 1000 population; average length of stay) can becompared within populations without knowledge of prevailing disease patterns as indices ofattained/achieved access. Such measures in an Australian context probably tell more aboutappropriateness of care than access, as rationing of particular interventions is rarelybelieved to be preventing receipt of appropriate care.♦Unmet NeedThese survey methodologies seek out patient reports of conditions warranting acute care(e.g. chest pain) <strong>and</strong> establish what care was sought <strong>and</strong> provided. The survey methods areexpensive <strong>and</strong> only applicable on larger population samples. They do, however, produceaccurate statistics on how successfully potential needs <strong>for</strong> care are matched by patterns ofexisting care delivery.♦Ambulance Bypass5.5.2 EfficiencyThis indicator has been applied in some jurisdictions as an index of access to emergencyservices. Our majority opinion is that this indicator has too many limitations to warrantinclusion in a national indicator program. It reflects access to services <strong>for</strong> a very smallpatient population judged to be not critically ill (i.e. patients using ambulance transport tohospital deemed non-time-critical by ambulance officers) <strong>and</strong> is amenable to gaming (as thedecision to declare an emergency department on bypass is arbitrary <strong>and</strong> based solely on thesubjective judgement of an emergency physician).Technical Efficiency♦Cost/Activity RatiosThis most common technique <strong>for</strong> measuring the technical efficiency of hospitals essentiallyanalyses the ratio of cost to some index of activity (i.e. quantity of delivered care). Theseindicators include:• Cost per Casemix adjusted separation• Cost of treatment per outpatient episode• Comparative average length of stay (ALOS).Whilst conceptually simple, costing data that is accurate, reliable <strong>and</strong> comparable is notreadily available in most healthcare systems - <strong>and</strong> is certainly currently not available withinAustralian hospitals. It would appear preferable to commence with simple measures ofefficiency <strong>and</strong> ensure that data used to construct these are accurate, reliable <strong>and</strong> comparablebe<strong>for</strong>e progressing to more complex measures. Data on ALOS are more readily available<strong>and</strong> potentially reliable. Their use as efficiency measures assumes that DRG outputs areequal in health outcome terms <strong>and</strong> that close relationships exist between cost <strong>and</strong> ALOS -

35assumptions that may not be reasonable given condition <strong>and</strong> severity differences withinadministrative database patient populations based on AN-DRG <strong>and</strong> ICD9-CMclassifications <strong>and</strong> differences in inpatient bed utilisation patterns such as those betweenurban <strong>and</strong> rural providers. Data on cost-per-casemix adjusted separation are currentlyprobably less accurate <strong>and</strong> considerably less comparable than ALOS data. Use of thisindicator will undoubtedly motivate improvements to data quality. At present we believecost data on outpatient episodes of care is likely to be so inaccurate (by virtue of the verylimited application of in<strong>for</strong>mation technology in outpatient costing) as to warrant avoidingrecommending its widespread use as a quality indicator in the short-term.♦Cost Frontier Estimation (or Production Frontier Estimation)These complex techniques, such as Data Envelopment Analysis, take account of themultiple inputs (such as labour, use of capital) <strong>and</strong> outputs (such as inpatient <strong>and</strong> outpatientservices, teaching, research) that are identified in acute healthcare. They involve modellingof inputs <strong>and</strong> outputs to generate weighted sums of inputs <strong>and</strong> outputs. The results of thismodelling can only be used to compare similar types of healthcare providers 368-402 . Thearbitrary allocation of weights to given inputs or outputs allow skewing of any modeltowards/against a significantly different service provision pattern (<strong>for</strong> example, if hospitalsdo not provide some services, weighting of these outputs appropriately in the model couldresult in apparently efficient or inefficient per<strong>for</strong>mance by comparison with a facilityproviding that service). Like all sophisticated mathematical models they depend heavily onthe precision of the manipulated original data points 403-404 . As indicated above, theaccuracy <strong>and</strong> comparability of currently available input <strong>and</strong> output data in Australianhealthcare would indicate that cost frontier estimates (such as Data Envelopment Analysis)are not realistic options <strong>for</strong> efficiency monitoring nationally at the present time.Allocative Efficiency♦The Oregon ExperimentThis well-publicised experiment in priority setting in healthcare produced a <strong>Quality</strong>Adjusted Life Year (QALY) league table based upon comparative cost-utility analyses.Although the approach is generally considered to have merit, it is compromised by the grosssimplifications in the costing of health interventions, the limited scope of services coveredby the model <strong>and</strong> the way QALYs were measured 407-411 .The requirement <strong>for</strong> marginal analysis as distinct from average analysis was ignored in theexperiment. Rankings of health interventions were based on average cost-utility ofinterventions, without due recognition that at the margin, cost-effectiveness will vary withpatient population, size of program <strong>and</strong> specific program characteristics.♦Program Budgeting with Marginal Analysis (PBMA)The PBMA approach has been developed by the Health Economics Research Unit inAberdeen <strong>and</strong> is being trialed by the NSW Health Department, through the Centre <strong>for</strong>Health Economics Research <strong>and</strong> Education 350,406 .The model focuses on the marginal levels of health gains <strong>and</strong> health losses associated withparticular healthcare interventions. In these terms priority is given to exp<strong>and</strong>ing programswhere the resultant health gains would be greatest, with a reduction in health services orprograms where the losses are minimal. With some modifications this approach has thepotential to be applied to the acute health sector.

36PBMA requires health service providers or funders to classify their services into programstreams be<strong>for</strong>e evaluating where the gains <strong>and</strong> losses might lie.The use of this technique <strong>for</strong> management <strong>and</strong> decision-making purposes must be supportedby consistent <strong>and</strong> reliable evidence on both the likely costs <strong>and</strong> outcomes of varioustreatments or interventions to enable priorities to be set with confidence. This model can bemore readily applied to a single sector of the healthcare system (such as the hospital sector)than the other allocative efficiency models currently being tested in Australia.♦Purchaser-Provider SplitThe “Purchaser-Provider” <strong>and</strong> “Managed Competition” models of health service funding<strong>and</strong> delivery have been designed to promote both allocative <strong>and</strong> technical efficiency. In bothof these models there is a separation of the purchasers <strong>and</strong> the providers of healthcare, with,in the pure model, the purchaser being responsible <strong>for</strong> the total health care of a communityor constituency, receiving commensurate funds <strong>for</strong> the purchase of services on behalf oftheir community 348,405 . A small number of countries have moved towards such a healthfunding system, including Britain <strong>and</strong> New Zeal<strong>and</strong>. The possible relevance to Australiahas been extensively debated in Australia 412,413 . Importantly, any “purchaser-provider”system requires advice to the purchaser, on how to select between competing healthinterventions. Logically, purchasers would need quality indicators to guide their purchasingchoice. Evidence available from existing programs suggests that initial decisions werealmost exclusively based upon cost <strong>and</strong> an assurance of quality. Such programs arecurrently seeking realistic indicators of quality of care which they could subsequently use tosupplant “cost-only” decision-making. Our enquiries found little evidence of sophisticationin their quality indicator utilisation to date. These models of health service funding <strong>and</strong>delivery require, <strong>and</strong> are well placed, to make use of a rigorous approach to priority setting.A number of health economists in Australia have recommended the purchaser/providerframework as a means <strong>for</strong> achieving optimal resource allocation <strong>for</strong> health promotion, incombination with the adoption of a rigorous approach to priority setting (e.g. using modelssuch as the disease-based framework <strong>for</strong> priority setting by Segal <strong>and</strong> Richardson 348 ).♦Guidelines <strong>for</strong> Listing of Drugs on the Pharmaceutical Benefits Schedule (PBS)In Australia in 1993 the Commonwealth Government developed in its health planningframework the systematic use of economic analysis to in<strong>for</strong>m pharmaceutical resourceallocation decisions, be<strong>for</strong>e the listing of drugs on the Pharmaceutical Benefits Schedule(PBS). Economic appraisal (cost-minimisation analysis, cost-effectiveness or cost-utilityanalysis), must be undertaken by pharmaceutical companies in support of the listings ofnew drugs on the PBS. Guidelines <strong>for</strong> the industry on preparation of submissions to thePBS have been developed, the most recent being updated in November 1995 414 . Theeconomic analyses are submitted to the Pharmaceutical Benefits Advisory Committee(PBAC) <strong>for</strong> evaluation prior to approval <strong>for</strong> listing on the PBS.Comparative cost-effectiveness (or cost-utility analysis) provides the framework withinwhich the inclusion or exclusion of new drugs to the PBS is made. However, this approach,like many of the models <strong>for</strong> priority setting does have limitations. In this example, practicalproblems associated with the definition of evidence <strong>and</strong> the limited choice of comparatorslimit its usefulness as a model <strong>for</strong> priority setting.The PBS Guidelines provide <strong>for</strong> the comparison of drug <strong>and</strong> non-drug approaches tomanagement, <strong>and</strong> thus potentially extend priority setting beyond pharmaceuticals. If thePBAC were to seek economic analysis of classes of drugs, <strong>and</strong> a comparison with non-drug

37approaches to management, this would extend substantially the scope of interventionscovered.♦Macro Economic Evaluation Model (MEEM)The MEEM approach is being developed by the Health Economics Unit of the Centre <strong>for</strong>Health Program Evaluation at Monash University 415 . This approach is designed <strong>for</strong> diseaseimpact costing, <strong>and</strong> resource allocation in health promotion <strong>and</strong> illness prevention. TheMEEM is essentially the construction of a ranking index from a systematic analysis ofavailable databases allowing the evolution of a broad-based framework <strong>for</strong> priority setting.Its component parts (disease impact assessments, life expectancy analysis, <strong>and</strong> projectappraisal) can also be used <strong>for</strong> descriptive analysis, including issues of equity <strong>and</strong> needsassessment. The underlying rationale of MEEM is that judgements about priorities <strong>for</strong>illness prevention <strong>and</strong> health promotion should be guided by in<strong>for</strong>mation on: the publichealth significance of health problems (measured by a range of indicators, including cost ofillness); the theoretical preventability (efficacy) <strong>and</strong> practical preventability (effectiveness)of the health problems; <strong>and</strong> the relative cost-effectiveness (efficiency) of individualpreventive measures aimed at achieving the potential <strong>for</strong> prevention.The MEEM model has limited applicability to the acute sector, as its primary focus is onillness prevention <strong>and</strong> health promotion. However, the disease costing component of themodel does provide a very comprehensive estimation of the annual hospital cost of treatingparticular diseases (based on ICD-9 classifications) in Australia. This may be of someinterest to the Commonwealth in helping to gain an underst<strong>and</strong>ing of the public hospitalexpenditure on particular disease categories.♦Disease-based Framework <strong>for</strong> Priority SettingThe Centre <strong>for</strong> Health Program Evaluation has developed a further approach to resourcepriority setting, designed to cover the entire health sector, based on the principles ofallocative efficiency, with disease groupings as the context <strong>for</strong> analysis 348,405 . The model iscurrently being implemented in relation to non-insulin dependent diabetes mellitus, withsupport from the Public Health Section of the Department of Health <strong>and</strong> Community<strong>Services</strong>, Victoria 402 .While the theoretical prerequisites <strong>for</strong> allocative efficiency can be defined, implementationof the above models is problematical. The measurement of marginal costs <strong>and</strong> benefits isdifficult <strong>and</strong> the magnitude of the research task <strong>for</strong> health sector-wide allocative efficiencyis daunting. The aim of the models of allocative efficiency is to link healthcare inputs(dollars) to healthcare outcomes (such as quality adjusted life years - QALYS) in such away that different services <strong>and</strong> treatments can be compared on a common basis - using <strong>for</strong>example a measure of outcome, the QALY, as a generic outcome indicator. Priority settingthen allows decision makers to choose between healthcare services which produce thegreatest outcome (QALY) per unit of input (dollars). A recent review of outputmeasurement <strong>for</strong> resource allocation decisions in healthcare discusses the uses <strong>and</strong>limitation of such an approach 416 .There are a number of practical problems with these approaches <strong>and</strong> their applicability tothe hospital sector. Firstly, most of the models require a large investment of resources <strong>for</strong>measuring <strong>and</strong> analysing the costs <strong>and</strong> outcomes associated with the full range of allpossible healthcare interventions. Secondly, most of the models take a societal perspective(that is, they are concerned with the costs <strong>and</strong> outcomes of all members of the community -patients, funders etc) <strong>and</strong> do not confine their measurement to one sector of healthcare (e.g.

385.5.3 Safetya hospital). In addition, restricting the outcome measure to a single health delivery systemmay generate distortions <strong>and</strong> cost shifting.A further issue is that little attention has been paid to developing a single outcome measure(or measures) to capture the extent to which hospitals or healthcare systems achieveallocative efficiency.♦Incident Reporting SystemsThe reporting, categorisation <strong>and</strong> subsequent utilisation of in<strong>for</strong>mation on incidents (that is,events which had the potential to produce injury or did in fact produce injury) in safetypromotion within hospitals has enormous potential <strong>for</strong> improving safety margins in acutehealthcare 42-47,50,73-75,114,129,134,135,150-153 . Anonymous incident reporting offers a relatively lowcost, sensitive index of safety per<strong>for</strong>mance with qualitative detail which can direct safetyimprovement endeavours 9,10 . The expansion of the successful anaesthesia/intensive careincident monitoring program across acute care services (in an initiative of The AustralianPatient Safety Foundation funded by the Commonwealth) should provide importantconfirmation of the utility of such incident monitoring approaches. The use of externallyreported adverse event or incident reports which rely upon the self-identification of error -which include identifying in<strong>for</strong>mation - cannot be supported as a quality of care indicator,as such identified incident reporting systems would create perverse incentives <strong>for</strong> providersto either not detect or report incidents. The bulk of evidence suggests that the majority ofincidents are currently unreported in conventional hospital practice 42,47,50,73,74,111 . It wouldbe undesirable to create any indicator of per<strong>for</strong>mance which magnified this incidentawareness blindspot, by creating an incentive to under-report incidents to meet externallyapplied st<strong>and</strong>ards or thresholds of incident reporting.♦Adverse Event MonitoringVarious Adverse Event (that is, reports of injury as a consequence of a medical interventionor failure to intervene appropriately) monitoring systems have been proposed, based eitherupon administrative databases or medical record review. These events must be clearly dueto the intervention rather than primary illness or be preventable if good care was given <strong>and</strong>patient preferences regarding risk taking must be known be<strong>for</strong>e tradeoffs between risks <strong>for</strong>different outcomes may occur.Administrative database software systems use algorithms to flag adverse outcomes whichmay reflect quality of care problems 54,89,93,99,137,150,151 . Because existing common codingsystems (e.g. AN-DRG <strong>and</strong> ICD9-CM) do not reliably indicate when an event occurredduring an episode of care, such computer-based screens identify a mixture of events presentat the time of hospitalisation <strong>and</strong> events occurring during hospitalisation. They are thuscurrently relatively crude indicators of adverse events due to hospitalisation. Futureimprovements in coding <strong>for</strong> administrative databases (such as stipulation of pre-existent <strong>and</strong>hospital-related events by placing events in time sequence <strong>and</strong> improved codes <strong>for</strong>intervention complications) <strong>and</strong> improvements in software algorithms to detect <strong>and</strong> weight arange of potential intervention-related adverse events may see future algorithms <strong>for</strong>administrative database safety screens fulfil their potential as patient safety indicators.Medical record review - seeking evidence of adverse events - may be undertaken at r<strong>and</strong>omor on records identified following an initial “screen” <strong>for</strong> features suggestive of an adverseevent (such as the presence of an unplanned return to the operating room) 1,47,50,63,69,114,135,145 .Such preliminary screens may involve use of manual record review <strong>for</strong> criteria flagging

39records <strong>for</strong> expert review or use of computer-based screens of administrative database codesto detect records warranting focused review. Reviewed records are assessed using eitherexplicit criteria <strong>for</strong> quality of care or implicit (expert opinion) determine to review if anadverse event has occurred <strong>and</strong> if patient safety has been compromised unacceptably. Suchadverse event reviews have been associated over time with a decline in detected adverseevents, rein<strong>for</strong>cing the application of medical record-based adverse event screening <strong>for</strong>safety improvement 185 .Record review <strong>for</strong> adverse events <strong>and</strong> quality of care analysis had been the major tool usedby the Peer Review Organisations (PROs) in the USA until recently. PROs are independentcontractors to US government purchasers whose charter is quality of care oversight. Formany years, implicit review of medical records (identified <strong>for</strong> review by specificoccurrences), <strong>for</strong>med the basis of PROs quality assessments. However, analysis of theinterrater reliability of implicit review in these circumstances suggests that there was poorconcordance between independent reviews. In addition, provider defensiveness againstexternal review of the processes of care has limited the improvement utility of PRO adverseevent review. Both these factors have led to a decision to broaden quality PRO initiativesbeyond these medical record reviews (e.g. HCFA’s HQII: see above) <strong>and</strong>, many StatePRO’s have discontinued such external record reviews because of poor cost-effectiveness127,294 .5.5.4 EffectivenessConceptually, any external quality <strong>and</strong> outcome indicator program based on adverse eventdetection in the medical record has a potential to modify the nature of that record. Routineexternal adverse event detection programs could create incentives <strong>for</strong> non-recording ofmatters relevant to adverse events. Thus, whilst internal promotions of adverse eventscrutiny based on medical record review as one component of facility-based patient safetyinitiatives are applauded - they cannot be advocated as useful in any national quality <strong>and</strong>outcome monitoring scheme. Promoting patient safety will require a layering of strategies,including the development <strong>and</strong> local implementation of practice guidelines, facilityaccreditation, staff credentialling <strong>and</strong> hospital-based improvement programs to detect,interpret <strong>and</strong> respond to safety concerns. Safety is the dimension of quality of care wheremost responsibility must rest with providers. External review indicators risk masking safetyproblems, rather than promoting remedies. Accountability <strong>for</strong> facility-wide safety must beprovided by accreditation programs which incorporate a focus on safety issues rather thanexternal generic safety indicator programs.Targeted safety indicators can be constructed <strong>for</strong> specific conditions or interventions muchmore successfully than generic indicators of safety across facilities. Such indicators shouldbe included in modular indicator sets relevant to diagnoses/conditions/interventions ofinterest (see 7.1 below) <strong>and</strong> should be reported to reviewing bodies which include relevantprofessional colleges <strong>and</strong> specialist societies to help guarantee full <strong>and</strong> frank eventreporting. These indicators typically relate adherence to best practice processes of care(such as time to delivery of thrombolytic therapy in patients with acute myocardialinfarction) or report adverse outcomes (such as procedure specific complications or death).As the risks of intervention are often proportional to the severity of patient illness - <strong>and</strong> notinfrequently to the anticipated benefits of the intervention - such indicators requireadjustment <strong>for</strong> patient factors (such as severity of illness <strong>and</strong> treatment preferences) be<strong>for</strong>ethey can reasonably be used to monitor safety per<strong>for</strong>mance.<strong>Indicators</strong> of effectiveness are essentially either of outcomes of care or outcome-proxies (whichare output measures or process measures known to be related to subsequent health outcomes).<strong>Outcome</strong>s can be defined either by providers or patients.

40♦Provider-Assessed <strong>Outcome</strong>sMortality: <strong>Acute</strong> care systems typically look at intrahospital mortality as one indicator ofhealth outcome 7,35,38,85,86,92,131,153,208 . When possible, linkage of data systems to enablereporting of mortality within longer timeframes (30 days/60 days) perhaps provides a moreaccurate indication of the impact of an episode of care on overall health <strong>and</strong> avoids biascreated by premature discharge or interfacility transfer 92,116,119,131 . The interpretation ofmortality requires knowledge of the anticipated outcome in the absence of the intervention<strong>and</strong> of the best achievable outcome with that intervention. For use as a quality of careindicator, it is important that variations in care quality translate into mortality variations. Inmany situations there are deficits in our knowledge of process-outcome links - <strong>and</strong>currently, mortality rates are not widely accepted as credible quality indicators 131,133,138,156 .They are, however, easy to measure <strong>and</strong> intuitively attractive, if the issues of adjusting <strong>for</strong>illness severity can be resolved (see 6.2 below). Global mortality indices provide riskadjustmenthurdles that may well be insuperable given the inadequacies of current modelling<strong>and</strong> the costs of collecting the necessary clinical detail <strong>for</strong> analysis 156 . Mortality withinstratified population groups (e.g. diagnosis or intervention) could be readily collected <strong>for</strong>some conditions. There is the potential <strong>for</strong> relative mortality per<strong>for</strong>mance to be a crediblequality of care indicator if it proves that population strata contain a relatively uni<strong>for</strong>m mixof case-severity. This would minimise the need <strong>for</strong> collection of additional risk-adjustmentdata 13,138,141 .Morbidity: There have been a number of models which aggregate intrahospital morbidity tocontribute to an index of outcome. These have been described under the Patient Safetydimension (see 5.5.3 above).Clinical <strong>Outcome</strong>s: Various clinician/provider-assessed indices of physical <strong>and</strong>psychological function fall into a range of measures frequently labelled as “clinicaloutcomes” 31,779,1005 . These are often laboratory, medical imaging or physiological measureswhich are used as proxy measures <strong>for</strong> increased longevity or improvement in quality of lifeby clinicians.For example:• Range of joint movement at a defined time after total joint replacement.• Coronary artery patency after coronary angioplasty.• Reduction in tumour size in cancer therapy.• Fall in white blood cell count in treatment of leukaemia.• Exercise tolerance after CABG.The relationships between these measures <strong>and</strong> genuine outcomes (i.e. survival <strong>and</strong> quality oflife) are variable <strong>and</strong> sometimes tenuous 55,146,148,149 . These proxy measures are, however,easily quantified <strong>and</strong> provide useful short-term feedback to clinicians regarding the apparentsuccess of interventions. They are more or less specific <strong>for</strong>interventions/diagnoses/conditions <strong>and</strong> should <strong>for</strong>m part of balanced indicator setsdeveloped <strong>for</strong> key targeted clinical circumstances (see 7.1 below). At present, mostindicators are built around immediate <strong>and</strong> short-term measures which capture changes invariables related to health status within an episode of care. Mechanisms need to bedeveloped to routinely record clinical outcomes at relevant more remote time points if thetargeted condition warrants such longer-term indicators of clinical outcomes (<strong>for</strong> example,the 12 month infection rate <strong>for</strong> total joint replacement or 12 month coronary artery patencyrates following coronary artery angioplasty) 148,149 .

41♦Patient Assessed <strong>Outcome</strong>sPatient-based reports of outcomes of healthcare: These have a number of strengths <strong>and</strong>limitations. Obtaining comparable in<strong>for</strong>mation directly from patients (typically bystructured survey) enables the analysis of short-term <strong>and</strong> long-term outcomes of care fromthe patient’s perspective 207,291,306,310,320,321,1061,1071 . These outcomes may include reactions tothe intervention itself (pain, swelling, incapacity <strong>and</strong> so on) as well as longer term changesin symptoms <strong>and</strong> function. Detailed in<strong>for</strong>mation can be obtained on the extent to whichspecific functional impairments or symptoms changed <strong>for</strong> the better or worse aftersurgery/interventions. Patient-reported data are relevant, but not all-inclusive 291 . Theycomplement clinical observations <strong>and</strong> tests while providing comprehensive <strong>and</strong> reliable dataon perceptual dimensions of health outcomes. Patient-reported outcomes are particularlyappropriate when the indication <strong>for</strong> an intervention is primarily improving function orrelieving symptoms. Patient-reported outcomes tend to integrate all processes of care <strong>and</strong>their consequences. They may suffer from incorrect attribution of outcomes tointerventions, but over a range of conditions they have proven largely reliable <strong>and</strong> valid (bycomparison with other parallel outcome assessments) 291,306,1061,1071 . Survey instrumentsexist <strong>for</strong> a variety of interventions (such as cataract surgery) <strong>and</strong> chronic conditions (suchas hypertension, arthritis, asthma <strong>and</strong> bronchitis, low back pain <strong>and</strong> diabetes) <strong>and</strong> more areunder development by the AHCPR-funded Patient <strong>Outcome</strong> Research Teams (PORTS)317,491,494<strong>for</strong>:• Hip fracture repair <strong>and</strong> osteoarthritis.• Prevention of low birth-weight infants.• Benign prostatic hypertrophy <strong>and</strong> localised prostate cancer.• Pneumonia.• Back pain.• Biliary tract disease.• Ischaemic heart disease.• Schizophrenia.• Secondary <strong>and</strong> tertiary stroke prevention.• <strong>Acute</strong> myocardial infarction.• Cataract.• Childbirth.• Diabetes.Health status measures: are st<strong>and</strong>ardised instruments to quantify functional health statusfrom patient reports 662,649,652,780,806,899,1071,1104,1105,1106,1110,1130 . They typically divide healthfunction into domains (such as physical, emotional, psychological, social <strong>and</strong> role functions)<strong>and</strong> are either generic (that is, applicable to all) or specific <strong>for</strong> a targeted population(condition/diagnosis/intervention/age group). The science underpinning validation of theseindicators of health status has advanced significantly over the past decade 1104-1106 . Manyhealth status instruments are now well characterised, with detail of their per<strong>for</strong>mance in arange of populations <strong>and</strong> particular health states known. They are responsive tointerventions aimed at improving healthcare <strong>and</strong> the significance of measured changes infunctional status is increasingly understood 1011,1048,1063,1110,1130,1134 . Interventions <strong>for</strong> acutedepression, heart valve replacement, treatment <strong>for</strong> chronic arthritis, total knee replacement,total hip replacement <strong>and</strong> cardiac revascularisation in ischaemic heart disease are amongstthe growing list of conditions in which the delivery of medical services have been shown toimpact positively on health status as measured by the Short-Form 36 (SF36) instrument572,576,665,669,798,809,863,931,951,968,11041106 . This health status measure has the most data available onits use in acute healthcare, both within Australia <strong>and</strong> internationally. The Short Form 36(SF36) is a generic measure of health status that can be used to assess health changefollowing an acute episode of care 1106 . Recent demonstrations that health status prior to an

425.5.5 Acceptabilityepisode of care is reliably reported retrospectively by patients would enable single surveyspost episode of care to detect the impact of care 192 . The availability of shorter surveyinstruments such as the SF36 related Short Form 12 (SF12), will facilitate the incorporationof health status quantification into broader, omnibus surveys of patients’ experiences ofhealth <strong>and</strong> healthcare 490 . The Medical <strong>Outcome</strong>s Trust in Boston has established aScientific Advisory Committee to review <strong>and</strong> endorse health status instruments. It is likelythat instruments endorsed by this body will increasingly become the health status instrumentst<strong>and</strong>ards within healthcare quality <strong>and</strong> outcomes indicator programs.Health-related <strong>Quality</strong> of Life (HRQOL): These are instruments which encompass aspectsof functional health status <strong>and</strong> patients’ reports of the impact of their health on theirenjoyment of life 659,663,670,671,1027,1085 . Rather than representing a discrete measurementmethodology distinct from health status, the two approaches offer a continuum, withfunctional health status more focused on self-reports of what one can or cannot do <strong>and</strong>HRQOL more reflecting assessments of how health impacts the enjoyment of life. A largeliterature base on HRQOL instruments exists206 . There are both generic <strong>and</strong>disease/condition specific HRQOL instruments. There is consensus that they are of value inhealth services research, although it would be our view that current HRQOL instrumentsare insufficiently characterised to recommend their widespread application in large scaleacute healthcare outcome indicator programs. Further developments in HRQOLinstruments may well allow such applications in the near future.This report was not to address many of the complex issues surrounding acceptability of caregiven that a major report on the role of patient satisfaction surveys in a national approach tohospital quality management has recently been completed <strong>for</strong> the Commonwealth which canvasesthe issues around the reporting of consumers’ experiences <strong>and</strong> assessments of quality of care 188 .Patient satisfaction is known to be associated with better results from healthcare interventions(such as better compliance with preventative care/medications) <strong>and</strong> improved functional healthstatus 198 . Although there is still much to learn about how best to obtain feedback from patientsabout their needs <strong>and</strong> wants <strong>and</strong> how best to use this knowledge to drive improvement processes,the current state of knowledge allows application of proven instruments whilst furtherdevelopmental programs proceed. Provider generated instruments such as those of the HospitalCorporation of America (HCA) or the Royal College of Surgeons (RCS), <strong>and</strong> patient-focusedinstruments such as the Picker-Commonwealth survey, could easily be adapted to provide initialreference data on the level of acceptability of a range of processes <strong>and</strong> outcomes of acutehealthcare. The Picker-Commonwealth instrument has been applied in the USA, Canada, theUK, Australia <strong>and</strong> Europe - <strong>and</strong> thus offers an additional benefit of potential internationalcomparisons of healthcare acceptability.5.5.6 Continuity♦<strong>Indicators</strong> of ProcessAt a facility level, it is common to seek evidence that appropriate discharge planning <strong>and</strong>care integration steps have been instituted as part of quality monitoring. External reportingof compliance with such process measures would be unreliable as a quality of care indicatoras such self-reports of compliance with discharge planning processes would be open togaming <strong>and</strong> not amenable to independent audit <strong>for</strong> confirmation of data integrity.

43♦<strong>Indicators</strong> of <strong>Outcome</strong>The success of endeavours to ensure a care continuum may be judged by surveying thosemost intimately involved in care processes. Patients <strong>and</strong> their families are the principalexperts on care continuity, with primary care physicians <strong>and</strong> other community-basedhealthcare providers secondary resources 261,337 . Surveys of these participant groups willreadily in<strong>for</strong>m how well integrated an acute episode of care appeared to those involved <strong>and</strong>provide descriptive commentary on the success of processes in place to optimise continuityof care. Modules on discharge planning <strong>and</strong> communication regarding ongoing care exist inseveral existing survey instruments 188 . Continuity indicators are prime examples of theusefulness of patient reports on care processes <strong>and</strong> outcomes in quality of care indicatorsets.5.5.7 Technical Proficiency♦Observed versus Predicted <strong>Outcome</strong>s AnalysisMany indicators of technical proficiency are based upon some permutation of observed topredicted outcome ratios. Predicted outcomes are based upon historical norms, bestpractice targets, arbitrary st<strong>and</strong>ards or models of anticipated median outcomes built onrelevant patient level data on existing clinical practice profiles. Observed outcomes arecompared to predicted, <strong>and</strong> where significant variation is identified it is inferred that thesevariations are related to variations in technical proficiency 6,7,13,16,83,85,93,131,132,133,156 .For example:• Myocardial infarction rates following coronary angioplasty• Stroke rates follow carotid endarterectomy• Anastomotic dehiscence rates following bowel anastomosesAll comparisons of observed-to-predicted outcomes are critically dependent on knowledgeof a link between technical proficiency <strong>and</strong> outcome <strong>and</strong> the validity of the estimatedfrequency of the predicted outcome of interest. In both these domains the current science ofquality monitoring has significant limitations (see especially 6.2 below). Unequivocalinferences about quality/technical proficiency can rarely be made on the basis ofobserved/predicted outcomes data, although they provide a practical stepping-stone toimproved proficiency measurement 140,143 .♦Compliance with Guidelines <strong>for</strong> CareThere is a growing body of guidelines <strong>for</strong> care delivery in particular conditions. Potentiallythese stipulate what should be done, how it should be done, when it should be done <strong>and</strong>indicate the likely outcomes if good care (as defined by the guideline) is delivered.<strong>Indicators</strong> of technical proficiency targeted to particular conditions/interventions can bedeveloped based around these guidelines. There are now comprehensive methodologiesavailable to help translate clinical practice guidelines into instruments to evaluate quality ofcare 189 . The HCFA Cooperative Cardiovascular Project, within its HQII program,provides an example of how existing care guidelines can be converted into quality process<strong>and</strong> outcome indicators (see Appendix 6 <strong>for</strong> additional detail) 127 . As many guidelines arebased upon evidence - or at worse expert consensus - they <strong>for</strong>m an important foundation <strong>for</strong>quality indicator development, hastening indicator development <strong>and</strong> reducing the risk ofgenerating indicators of limited relevance.

445.5.8 Appropriateness♦Case-by-Case Comparison with Expert CliniciansTraditional approaches to appropriateness evaluation see comparison of each individualepisode of care to expert opinion on appropriate care - typically by reference to consensuscriteria on what constitutes appropriate care (e.g. the clinical circumstances in whichinterventions are warranted given knowledge of the relative risks <strong>and</strong> benefits) 20,21,25,39-41,52,67 . This clinician perspective is subject to criticism if doubt exists about the evidencesupporting applied criteria <strong>for</strong> appropriateness - although within given providercommunities there is a reasonable concordance of opinion on the appropriateness of targetedinterventions21,39-41,67 . These case-by-case reviews are sufficiently imperfect <strong>and</strong>impractical <strong>for</strong> use in any national indicator program as to be discounted <strong>for</strong> this purpose 53 .♦Inferences Based on Variation in UtilisationA more practical approach to large scale monitoring of appropriateness of care is theanalysis of comparative utilisation statistics 22-24,33,37,92,110,112 . Relative rates of application ofa particular intervention provide in<strong>for</strong>mation suggesting potential overuse (if rates areinexplicably high) or problems with access to care (if rates are inexplicably low). The useof utilisation rates as proxies <strong>for</strong> appropriateness is dependent upon the assumption thatpotential access is relatively uni<strong>for</strong>m - as would be the case <strong>for</strong> many medical interventionswithin urban Australia. Data stratification (such as by age, sex or ethnicity) allowsidentification of areas of potential inappropriate intervention or access difficulty inparticular vulnerable subgroups. Utilisation best reflects appropriateness when theprevalence of the relevant condition in the serviced population is known <strong>and</strong> can <strong>for</strong>m thedenominator in the utilisation rate equation (that is, a rate of intervention/1000 in populationat risk of needing intervention) 116,1003 . Although imperfect appropriateness indicators, suchutilisation rates are available at low cost <strong>and</strong> would provide interesting comparativein<strong>for</strong>mation <strong>for</strong> Australian populations.N.B.It should be noted that “appropriateness” is used in several ways in health servicesresearch. Its use as a dimension of quality in this report is restricted to the assessment ofnet benefit versus risk (that is, was an intervention worth doing). It does not encompassthe site of the intervention (which we would see primarily as an efficiency issue), nor theappropriateness from the patient’s perspective (which we have encompassed within thedimensions of acceptability <strong>and</strong> effectiveness).5.5.9 Other DimensionsThe original Project brief asked <strong>for</strong> specific comment on health status, discharge planning <strong>and</strong>nursing indicators. Health status measures are tools used to monitor effectiveness - <strong>and</strong> arediscussed within this context (see 5.5.4 above). Discharge planning constitutes one aspect ofcoordination of the continuum of care <strong>and</strong> is addressed within this context (see 5.5.6 above).After reviewing an extensive body of in<strong>for</strong>mation on indicators of nursing 171-178,209-261,285-290,323 ,we elected to treat all dimensions of care as indicators of integrated care delivery by all relevantclinical professionals. We thus include nursing contributions to care <strong>and</strong> those of other involvedhealthcare professionals within indicators <strong>for</strong> all identified care dimensions (see Appendix 7 <strong>for</strong>additional detail).

456. Discussion6.1 Rationale <strong>for</strong> a Nationally Consistent <strong>Quality</strong> <strong>and</strong> <strong>Outcome</strong> Indicator ProgramNational indicator programs would seek to provide:• Accountability by providers.• In<strong>for</strong>mation to guide consumer decisions.• Incentives <strong>for</strong> quality improvement.They would achieve these aims without imposing an unacceptable collection burden on providers, atreasonable cost to the community <strong>and</strong> without creating incentives <strong>for</strong> undesirable behaviour by providers(such as gaming of data or inappropriate skewing of service delivery profiles). An ability to comparequality <strong>and</strong> outcomes data, confident that the same indicator instrument was being applied in a consistentfashion across facilities, should stimulate national benchmarking activities. As indicators are expensive todevelop, a move to a nationally consistent set could conceivably reduce the net cost of indicatordevelopment - by allowing the routine application of well-characterised instruments broadly acrossAustralia. Inevitably, national indicators will lead to a prioritisation of quality improvement ef<strong>for</strong>ts - ashas occurred elsewhere following widespread application of specific quality of care indicators. Thisinfluence on provider behaviours could be used to direct their attention to national health goals <strong>and</strong> targets.National indicator sets should encompass aspects of quality of healthcare services of relevance to patients,providers <strong>and</strong> purchasers of care. Such a balanced, comprehensive approach to monitoring dimensions ofhealthcare quality is necessary to avoid unintended <strong>and</strong> undesired consequences which might arise fromskewing of a system’s per<strong>for</strong>mance to produced favourable results in narrow segments of monitoredservice delivery. We believe indicators spanning the dimensions of care identified as pivotal by this report(i.e. access, efficiency, safety, effectiveness, acceptability, continuity, technical proficiency <strong>and</strong>appropriateness) would provide a firm basis <strong>for</strong> a successful, coordinated national indicator program witha balance between comprehensive coverage <strong>and</strong> feasibility.The cost of indicator collection would be substantial - but would constitute a small proportion of overallhealth expenditure. Judgements on the value of indicator monitoring are varied - but input frompurchasers, providers <strong>and</strong> consumers must be integrated be<strong>for</strong>e final value decisions are reached. Anyintroduction of national quality of care indicators must be accompanied by parallel projects to demonstratethe per<strong>for</strong>mance of the indicators themselves against reasonable expectations regarding indicatorattributes. This current review has highlighted that many indicators have been developed <strong>and</strong> applied inthe workplace with little or no evidence of attributes such as reliability or validity <strong>and</strong> no systematicprocesses to monitor responsiveness, burden, interpretability or utility. Throughout the western worldthere is a progressive move to large-scale monitoring of healthcare - in part because of a need to monitorthe healthcare industry because of its importance economically <strong>and</strong> in part a desire to in<strong>for</strong>m consumers<strong>and</strong> drive improvement ef<strong>for</strong>ts. Within Australia, many States are moving to collect quality <strong>and</strong> outcomeindicator data. There would be little sense in collecting disparate data at a State <strong>and</strong> Territory level, ifagreement on indicator sets would allow national comparison at a minimal marginal cost. Whatever <strong>for</strong>mfinal national indicator programs adopt, systematic tracking of the attributes of individual indicators <strong>and</strong>indicator sets will better characterise quality indicators <strong>and</strong> support decision-making regarding qualityindicator applications. Future reviews similar to this would then find evidence available upon which tobase firm judgements on the value of indicator programs <strong>and</strong> analyse attributes of their component parts.One of the enduring debates around quality <strong>and</strong> outcome indicator collection concerns the compatibility ofindicators <strong>for</strong> internal quality improvement, external accountability functions <strong>and</strong> in<strong>for</strong>ming consumerchoice. There is no consensus on the ability of a single indicator to serve these functions. Some argue thatthese functions are inherently incompatible, with accountability programs inherently “unsafe”environments <strong>for</strong> healthcare providers, in contrast to the “safe” environment required <strong>for</strong> successful qualityimprovement programs. Whilst agreeing with the need <strong>for</strong> both activities, these experts opine that externalindicator programs will never enhance local quality enhancements - <strong>and</strong> at worse may detract from them.

46Others argue that it is the behaviour of those participating in indicator programs which determines theircongruence <strong>for</strong> accountability <strong>and</strong> quality improvement purposes. It will be essential that education (ofproviders <strong>and</strong> those receiving indicator data) <strong>and</strong> a collaborative development of credible indicatorsunderpins any national indicator implementation in Australia. Although there are stark examples offailures in indicator applications (such as HCFA - <strong>and</strong> others - release of inadequately risk-adjustedfacility mortality data), there are equally examples of successful dual-usage indicator programs (such asthe Clevel<strong>and</strong> Health <strong>Quality</strong> Choice), suggesting that the optimism of those supporting compatibility ofthese uses of some selected quality of care indicators is justified.The introduction of a nationally consistent quality <strong>and</strong> outcome indicator program would undoubtedly beassociated with changes in practice, as have occurred elsewhere 12,304,339 . The rationale <strong>for</strong> these changes inpractice <strong>and</strong> their relationship to indicator usage per se have never been systematically analysed. Whilst itwould be hoped that the availability of indicator data <strong>and</strong> the ability to benchmark per<strong>for</strong>mance motivatedquality improvement effects which resulted in improved processes <strong>and</strong> outcomes (such as a reduction inCaesarean section rates or readmission rates) these changes may have resulted from a halo effect of qualitymonitoring or indeed have occurred because of unrelated changes in health service delivery. In the leastdesirable circumstance, quality indicators may drive data manipulation exercises such that apparentimprovements merely represent the quality monitoring equivalent of “creative accounting”. As isemphasised throughout this report, the likely health service delivery consequences of a national qualityindicator program will be inextricably linked to the context of indicator development <strong>and</strong> use.6.2 Risk AdjustmentAt present we believe that none of the available severity adjustment systems reviewed are clearly superiorto others. There are a number of recent enlightening reviews of this subject 34,35,71,131,133,141,144,154,156,778 .They agree to the value of adjusting indicator data <strong>for</strong> patient characteristics so that the process oroutcome indicators better reflect the quality of delivered care <strong>and</strong> not factors unrelated to care quality (socalled“confounding factors”). Ideal risk adjustment systems incorporate a range of modifiers related tothe patient’s suitability <strong>for</strong> application of care processes (such as allergy to recommend treatment options)<strong>and</strong> their intrinsic vulnerability <strong>for</strong> a given outcome (such as the specific condition afflicting the patient,the severity of that disease, the presence of other medical conditions <strong>and</strong> patient’s overall physical,psychological, emotional <strong>and</strong> social resources). Following ideal risk adjustment, any remaining differencesin indicator rates would directly reflect differences only in the quality of care provided.Ideal risk adjustment methodologies do not currently exist. There is evidence that the more in<strong>for</strong>mationavailable <strong>for</strong> risk adjustment, the better the resulting model - although the gain in predictive abilities ofrisk-adjustment models (their ultimate validation) is not directly proportional to the quantity of data pointsprovided <strong>for</strong> modelling 293,329 . The principal question posed is there<strong>for</strong>e whether the gain in comparativeprecision warrants the costs of data collection <strong>and</strong> risk adjustment. Providers typically are not responsiveto unadjusted indicator data 141 . Credibility with providers is an important motivation <strong>for</strong> risk adjustment.It is crucial that the limitations of adjusted or unadjusted indicator data are known to all who use the data<strong>and</strong> that they act on the data accordingly. Serious problems arise if crude or incompletely risk-adjustedindicator data are acted upon as if they mean precisely what they may seem to mean (<strong>for</strong> example, ifoverall hospital mortality rates are reported as quality indicators <strong>and</strong> specialist cancer hospitals providingpalliative care services with high mortality rates are sanctioned).More sophisticated risk-adjustment models not only use more factors <strong>for</strong> adjustment, they also attempt tocompensate <strong>for</strong> known problems with data (<strong>for</strong> example, they seek in<strong>for</strong>mation from medical recordsrather than administrative databases to ensure accurate patient profiling) 156 . Better models avoid maskingdifferences in process or outcome that could potentially reflect differences in quality of care (<strong>for</strong> example,if ethnic minorities systematically received poorer quality of care - adjusting <strong>for</strong> ethnicity would concealthat aspect of relative per<strong>for</strong>mance) 320,324 .

47An alternative to patient-level risk adjustment is facility-level adjustment 293 . We believe this approachrisks the masking of systematic differences in care delivery between facilities <strong>and</strong> is not desirable. Nomajor respected risk-adjustment system studied included such facility-level adjustment in its modelling.Stratification of patient populations provides a relatively simple means of attempting to correct <strong>for</strong> patientleveldifferences impacting on process or outcome indicator data. The premise underpinning stratificationis that the identified groups (strata) are either homogeneous or that they may be treated as such <strong>for</strong>comparative purposes because any patient-level differences average out across the groups compared.These strata/groups may reflect primary diagnosis, disease severity, age or sex, or some combination ofsuch factors. When used successfully stratification allows indicator comparisons at low cost - providedthe data <strong>for</strong> stratification are routinely available in administrative databases.Stratification may be the only practical approach to addressing the monitoring of quality of care <strong>and</strong> healthoutcomes in vulnerable patient groups, such as Aboriginal <strong>and</strong> Torres Strait Isl<strong>and</strong>er peoples. Providedin<strong>for</strong>mation permitting stratification is readily available within data sets (such as indicators ofAboriginality or postcodes as a proxy <strong>for</strong> socio-economic class) it would be possible to reviewcomparative indicator data in those deemed potentially vulnerable. Indicator rate comparisons wouldsuggest areas of inequity in service delivery warranting further scrutiny.<strong>Quality</strong> indicators have been subjected to various levels of risk adjustment. Any single indicator reviewed<strong>for</strong> this report has been presented in <strong>for</strong>mats ranging from unadjusted facility or practitioner-level data todata reflecting risk-adjustment <strong>for</strong> 30 or more potential confounding variables. Expert opinion differs onthe utility of risk-adjustment <strong>and</strong> in particular its cost-effectiveness. There is no evidence to categoricallydecide the value of risk adjusting clinical indicators - although there is consensus that risk adjustmentimproves provider acceptance of indicator data <strong>and</strong> significantly alters apparent comparative per<strong>for</strong>manceof up to 30% of individual facilities. Each indicator requires the development of a unique risk adjustmentmodel based upon local experience of indicator <strong>and</strong> confounder data. Risk adjustment models do notsuccessfully “transport” from one healthcare environment to another - presumably because many apparentconfounders are in fact proxy measures of patient characteristics which act as valid proxies only in adefined patient population. The need to develop risk adjustment models <strong>for</strong> those indicators in the nationalindicator set will depend on the intended use of indicator data. It is probable that raw <strong>and</strong> adjusted orstratified indicator data will be required <strong>for</strong> most of the proposed indicators. The development of locallyrelevant risk adjustment models should <strong>for</strong>m an integral part of the construction of detailed operationaldefinitions <strong>for</strong> each indicator to be trialled as part of a putative national indicator set. Guidance on datacollection requirements necessary to adjust <strong>for</strong> confounding influences will initially be based around expertlocal opinion <strong>and</strong> the experience of others in risk-adjusting quality indicators in healthcare103,106,118,144,156,302,328,342 .6.3 Characteristics of a Successful National <strong>Quality</strong> <strong>and</strong> <strong>Outcome</strong> Indicator ProgramWe believe a successful national indicator program will require several key characteristics. It will be:• Comprehensive: A sufficient breadth of indicators will be used to guarantee that a range ofdimensions of care <strong>for</strong> common, relevant conditions are encompassed within the program.• Collaborative: The program will evolve with the active participation of all interested parties pursuinga common goal of attaining a credible indicator set which can satisfy accountability requirements,guide consumer-choice <strong>and</strong> promote quality improvement.• Consumer-focused: Indicator programs must embrace consumer involvement in the identification ofareas of care delivery warranting monitoring <strong>and</strong> in the design, implementation, interpretation <strong>and</strong>feedback on indicators of quality of care <strong>and</strong> outcomes.• Current: National indicator sets will need to be updated regularly to keep abreast of developments inthe science of quality of care monitoring <strong>and</strong> the science of medical care delivery.

48• Cost-efficient: Wherever practical, indicators should utilise existing data <strong>for</strong> indicator construction,making incremental changes to routine databases as the needs <strong>for</strong> additional data points are identifiedby appropriate research <strong>and</strong> field trials.Such a national indicator program would then be credible to all concerned <strong>and</strong> be of value in guidingpurchasing decisions, integral to facility quality improvement programs <strong>and</strong> in<strong>for</strong>m consumers’ healthcaredecisions. Achievement of these desired goals within the framework of a national indicator program willrequire more than agreement to a nationally consistent indicator set. Assistance with data collection, theinterpretation of statistical analyses, benchmarking, the design <strong>and</strong> implementation of improvementprojects <strong>and</strong> coordination of collaborative projects with other facilities will require the availability ofresources beyond those currently seen in most Australian healthcare facilities. To ensure that in<strong>for</strong>mationsharing on acute healthcare per<strong>for</strong>mance enhances local quality improvement ef<strong>for</strong>ts, provideraccountability <strong>and</strong> consumer choice, it will be necessary to exp<strong>and</strong> the community of professionalscommitted to quality <strong>and</strong> working collaboratively with hospitals to translate the potential benefits ofquality <strong>and</strong> outcome monitoring into achieved gains.The provision of this human resource <strong>and</strong> logistical support is an absolute requirement <strong>for</strong> the success ofthe national indicator program. A number of options exist <strong>for</strong> how this resource might be provided.Existing government, professional or regulatory agencies could be delegated responsibility <strong>for</strong> some or allof these functions. Alternatively, such functions could be concentrated within a group focusing on qualityissues in healthcare (such as those centres of excellence proposed in the final report of the Task<strong>for</strong>ce on<strong>Quality</strong> in Australian <strong>Healthcare</strong>). The use of some existing agencies to provide the program support mayraise concerns about potential <strong>for</strong> apparent conflicts of interest. St<strong>and</strong>-alone, quality-focused agenciesmight be deemed an unwanted additional complexity <strong>and</strong> expense. A USA model worthy of review in thissetting is the HCFA <strong>Quality</strong> Improvement Initiative. HCFA contracts out these support functions <strong>for</strong> thisquality monitoring project, which are provided by dedicated groups (typically set up by existing agenciesas independent entities with the purpose of enhancing improvement ef<strong>for</strong>ts).7. RECOMMENDATIONS7.1 Structure of a National Indicator ProgramThe development of a nationally consistent set of quality of care <strong>and</strong> health outcome indicators will be aniterative process, with more in common with a research <strong>and</strong> development program than with the workplaceimplementation of an established technology. If it is agreed that a variety of interests are best served bythe availability nationally of credible comparative in<strong>for</strong>mation on the per<strong>for</strong>mance of acute healthcareservices, there is the need to begin with the best available feasible indicators <strong>and</strong> progressively improve<strong>and</strong> supplement these, rather than deferring all action because of the absence of perfect indicators.We recommend:7.1.1 National quality <strong>and</strong> outcome indicators be developed as two complementary indicator sets:A core indicator set, intended <strong>for</strong> continuing collection <strong>and</strong> a series of indicator modulescontaining indicator sets intended <strong>for</strong> collection at a defined frequency <strong>for</strong> a finite duration.7.1.2 The core indicator set should focus on generic aspects of care - including access, efficiency<strong>and</strong> acceptability of care.7.1.3 The indicator modules be targeted to specific conditions, diseases, diagnoses or interventions<strong>and</strong> include a balanced array of indicators encompassing clinical indicators, health statusindicators, acceptability indicators <strong>and</strong> efficiency/cost indicators.

49Examples of the type of indicators contained within such condition-specific modules areprovided at the end of this section. It is emphasised that these are listed to explain theconcept of indicator modules <strong>and</strong> that <strong>for</strong>mal development of indicator modules <strong>for</strong> trialingrequires expert local input (see 7.1.5).7.1.4 The core indicator set be based around in<strong>for</strong>mation derived from administrative databases(because of cost efficiency) or patient surveys (because of the high value of patient-basedin<strong>for</strong>mation).7.1.5 The indicator modules targeting a particular clinical circumstance be based on hybrid datacollection <strong>and</strong> include in their function a <strong>for</strong>mal assessment of the contribution of additionaldata points obtained from medical record data abstraction to the utility of indicator data.7.1.6 Indicator modules targeting particular clinical circumstances be developed as cooperativeactivities between government, regulatory bodies, providers, specialist <strong>and</strong> professionalcolleges <strong>and</strong> societies, consumers <strong>and</strong> those with expertise in indicator application. Theseprograms could be based at centres with expertise in data h<strong>and</strong>ling (such as AIHW) butshould include active participation of professional groups who will enhance the peercredibility of quality monitoring <strong>and</strong> help ensure that accurate data are transmitted <strong>for</strong>analysis.7.1.7 Wherever possible sample techniques be used to obtain detailed representative data ratherthan attempting routine data collection from all episodes of care. This will increase thecomplexity of available data whilst controlling costs.7.1.8 As part of a national indicator development program a resource be developed to assistfacilities with indicator data collection <strong>and</strong> interpretation <strong>and</strong> provide independentconfirmation of data integrity.7.1.9 Indicator programs should develop comprehensive operational definitions <strong>for</strong> indicators toenhance data reliability, preferably including software programs with inbuilt data reliabilitychecks.Examples of Condition-Specific<strong>Quality</strong> <strong>and</strong> <strong>Outcome</strong> Indicator ModulesCoronary Artery Bypass Grafting4-8 indicators chosen from:• Unplanned return to operating room• Inhospital mortality• 5 year mortality• Change in health status (at 6 or 12 months)• Secondary prevention strategies• Health related quality of life at 3 months• Underst<strong>and</strong>ing by patient of indication <strong>for</strong> CABG• Average length of stay• Satisfaction with inpatient episode of care• Other process or outcome indicators

50Myocardial Infarction4-8 indicators chosen from:• Compliance with Best Practice Guideline <strong>for</strong> Care- Time to revascularisation attempt (either thrombolysis or angioplasty)- Inhospital survival- Health status at 3 months- Administration of appropriate medicationsü Beta Blockersü Aspirinü ACE Inhibitors• Myocardial function at 12 months• Survival at 5 years• Risk factor education success• Risk factor intervention rates• Return to previous activity index• Satisfaction with inpatient episode of care• Length of stay• Cost of care• Other process or outcome indicators7.2 Suggested <strong>Indicators</strong> <strong>for</strong> Trial in AustraliaIt is a difficult judgement to support particular quality <strong>and</strong> outcome indicators without detailed knowledgeof the intended use of these indicators. As we have repeatedly stated, the determination of the adequacy ofan indicator is not categorical (that is, indicators are not either adequate or inadequate, good or bad),rather, indicator validity <strong>and</strong> utility lie upon a continuum with final adjudication on their relative adequacyrequiring knowledge of the context <strong>for</strong> per<strong>for</strong>mance measurement. Our recommendations assume acollaborative, iterative, developmental program <strong>for</strong> progressing towards a nationally consistent set ofquality of care <strong>and</strong> health outcome indicators.We have found little in<strong>for</strong>mation available on indicator attributes which would guide dogmatic choices ofindicators <strong>for</strong> our recommendations. The majority of putative quality <strong>and</strong> outcome indicators have no dataon their per<strong>for</strong>mance. The available data would suggest that many indicators are of some value - with fewst<strong>and</strong>ing out as examples dem<strong>and</strong>ing inclusion in quality monitoring programs. The recommendationsbelow are based upon our judgements regarding indicators of potential value, cognisant of existing <strong>and</strong>feasible short-term data availability. Our assessments of these indicators against the indicator attributeprofile is included in Appendix 6 - which also encompasses notation of identified indicators which havesufficient in<strong>for</strong>mation available to suggest that they also could be adapted <strong>for</strong> use in an Australian context<strong>and</strong> examples of indicators with significant shortcomings when judged against these criteria.♦Access7.2.1 Existing elective surgery waiting times be refined to a common operational definition whichencompasses a uni<strong>for</strong>m categorisation of urgency. Waiting time should be the time from thedecision to intervene to intervention, to reduce incentives <strong>for</strong> provider manipulation of data.Derivative measures, such as clearance times, help with interpretation of waiting/queuingtimes <strong>and</strong> should be continued.7.2.2 Following successful implementation of an indicator of uni<strong>for</strong>m waiting time <strong>for</strong> electivesurgery, consideration be given to stratification of queuing time by specialty in addition to

51urgency <strong>for</strong> core use in indicator sets or by intervention/procedure within modular setstargeting particular clinical circumstances.7.2.3 Emergency Department waiting times - stratified by triage category.7.2.4 Waiting times in Emergency Department prior to inpatient admission.7.2.5 Patient surveys be undertaken to derive reported elective surgery waiting times, EmergencyDepartment waiting times, time in Emergency Department awaiting admission, outpatientwaiting times (<strong>for</strong> appointment) <strong>and</strong> outpatient waiting time <strong>for</strong> particular attendances.This in<strong>for</strong>mation could be stratified by patient-reported urgency of care <strong>and</strong> the specialtyproviding service <strong>and</strong> the acceptability of reported queuing times recorded.♦Efficiency7.2.6 The continued development of cost/casemix adjusted separation be supported to resolveoperational definition <strong>and</strong> reliability issues be<strong>for</strong>e proceeding to consider more complex,high level technical efficiency indicators.7.2.7 No indicators of allocative efficiency are currently suitable <strong>for</strong> trialing.♦Safety7.2.8 No indicators of patient safety are currently suitable <strong>for</strong> trialing in the core indicator set.Progression of implementation of comprehensive, anonymous incident reporting systemsare seen as the optimal national strategy <strong>for</strong> monitoring safety in hospitals. All reviewedgeneric safety indicators were considered either conceptually flawed or liable to createundesirable incentives to bias the identification <strong>and</strong> reporting of safety concerns.7.2.9 Targeted safety indicators should be included in relevant modules of indicators focusedupon defined clinical circumstance. These will typically reflect adherence to best practiceguidelines or some variation on observed to predicted adverse outcome monitoring <strong>and</strong>could best be initially developed <strong>for</strong> interventions <strong>for</strong> ischaemic heart disease(angiography, CABG, angioplasty <strong>and</strong> AMI).♦Effectiveness7.2.10 Generic health status measures (the SF36 or SF12) be part of omnibus surveys (see 5.4.3)administered to recent recipients of care <strong>and</strong> these health status reports be compared toself-reported health on admission as indicators of care effectiveness <strong>for</strong> use in the coreindicator set.7.2.11 Health status measures (SF36 or SF12 as generic measures or other validated conditionspecific measures relevant to the targeted clinical circumstances) be included in thebalanced indicator sets developed <strong>for</strong> indicator modules addressing particular conditions,diagnoses, diseases or interventions.7.2.12 Mortality rates <strong>for</strong> selected clinical conditions, diseases, procedures or interventions becollected - stratified by readily available administrative data - <strong>and</strong> apparent quality of careper<strong>for</strong>mance be compared to more sophisticated health outcome indicators based uponrisk-adjusted mortality developed within targeted modules of indicators addressing theseprocedures. This would provide in<strong>for</strong>mation as to how significantly complex riskadjustmentalters apparent comparative per<strong>for</strong>mance <strong>and</strong> will guide judgements on the

52long term utility, in an Australian acute healthcare context, of these approaches tooutcomes monitoring.7.2.13 Unplanned readmission following index admission <strong>for</strong> asthma - stratified by age (0-19 <strong>and</strong>20-49) provides a valid indicator of the effectiveness of the overall asthma care plan <strong>and</strong>its revision or rein<strong>for</strong>cement during the initial hospital admission. Unplanned readmissionis, we believe, likely to be restricted in use in future to such condition specific applicationsrather than continuing as a generic hospital-wide medical indicator - although finaldecisions on its generic utility must await the results of studies underway under theauspices of the National Hospital <strong>Outcome</strong>s Program.7.2.14 Low (less than 2500 grams) <strong>and</strong> very low (less than 1500 grams) birthweight rates bemonitored. Although difficulties with attribution of outcome to facility-level quality ofcare are encountered with this indicator, at a population level it provides oversight of theadequacy of maternity care <strong>and</strong> examination of stratified results may yield evidence ofcomposite healthcare system quality problems (including aspects of access <strong>and</strong>effectiveness). Hospitals providing childbirth care have a crucial role to play in thepromotion or direct provision of adequate maternal antenatal care <strong>and</strong> in advocacy <strong>for</strong>social support <strong>for</strong> pregnant women. Thus, whilst this indicator reflects much more thanfacility-level per<strong>for</strong>mance during an inpatient episode of care, it is not an unreasonableindex of the broader success of healthcare providers in the management of antenatal care.♦Continuity7.2.15 Patient-based assessment: Surveys of patients’ perceptions of care should includeassessments of the success of discharge planning <strong>and</strong> integration of care based upon themodules currently within the Picker Commonwealth survey. In circumstances wherepatients themselves are unable to provide such feedback, the in<strong>for</strong>mation should beobtained from their carers.♦Acceptability7.2.16 Acceptability: We believe that a national survey program based upon r<strong>and</strong>om sampling ofrecent acute care patients using instruments built upon well validated surveys - such as thePicker Commonwealth survey, the Hospitals Corporation of America survey <strong>and</strong> theRoyal College of Surgeons surveys - is the appropriate action in the short term.7.2.17 Needs: A national program to better identify the needs of Australian consumers, inparticular vulnerable subgroups such as Aboriginals <strong>and</strong> Torres Strait Isl<strong>and</strong>ers, shouldbe implemented. This in<strong>for</strong>mation should be the basis <strong>for</strong> refinements of the nationalsatisfaction survey instrument.7.2.18 Contemporaneous reporting of process <strong>and</strong> outcome in<strong>for</strong>mation: Whenever practicable,surveys of acceptability should be linked to simultaneous patient reporting of theirperceptions of processes <strong>and</strong> outcomes of care.♦Technical Proficiency7.2.19 Targeted technical proficiency indicators: The major thrust <strong>for</strong> indicators of technicalproficiency should be within the modular indicator sets focusing upon specific clinicalconditions, diagnoses, diseases or interventions. These should be appropriately riskadjustedprocess indicators built upon accepted care guidelines or outcome indicators builtupon observed to predicted morbidity or mortality ratios (where the frequency of themonitored outcome is sufficiently high to avoid masking of quality of care issues by

53anticipated r<strong>and</strong>om variation). Initial modules should be built around cardiovasculardisease interventions. Later modules could examine conditions such as the care ofdepression, the management of newly diagnosed cancer or cataract extraction - asexamples of common conditions with a serious health impact, which have effectivetreatments <strong>and</strong> per<strong>for</strong>mance indicators available.♦Appropriateness7.2.20 Case by case analysis of appropriateness of care: The limitations of this analytictechnique, its cost <strong>and</strong> its influence on a collaborative development of external indicatorprograms render it unsuitable <strong>for</strong> a national indicator program. Such individualappropriateness reviews should be encouraged within provider facilities.7.2.21 Utilisation as a proxy appropriateness indicator: It would be valuable to analysepopulation- based differences in interventions believed to have significant unexplainedvariation in utilisation as indirect indicators of appropriateness of care. Where reliabledata on the population-based incidence of primary disease processes is available, suchutilisation data would prove more valuable in judgements of apparent comparativeappropriateness <strong>and</strong> might direct more focused review of a representative sample ofindividual cases <strong>for</strong> analysis of individual case appropriateness. Conditions to consider<strong>for</strong> such utilisation review include:• Cardiac catheterisation.• Coronary artery bypass grafting.• Angioplasty (PTCA).• Cholecystectomy.• Hysterectomy.• Laminectomy.• Caesarean Section (primary).• Vaginal birth after primary Caesarean Section.• Prostatectomy.7.3 Future Directions in Indicator Development7.3.1 Dimensions of care quality needing indicator developmentWhilst an argument can be made that all areas of quality <strong>and</strong> outcome indicators requiresubstantial research <strong>and</strong> development, we believe there to be a particular need <strong>for</strong> indicators thatreflect:♦♦Unmet access: Most existing hospital access indicators deal with characteristics of thosewho have succeeded in achieving access to our acute healthcare services. Betterin<strong>for</strong>mation is required on consumer health needs. This requires in<strong>for</strong>mation about thosewho fail to access healthcare to better in<strong>for</strong>m judgements on equity of healthcare services.Such indicators will require population-based surveys seeking evidence of likely need <strong>and</strong>the health system’s response to that need.Allocative efficiency: Much needs to be done to refine instruments <strong>for</strong> allocativeefficiency that more accurately reflect the balance between consumers’ needs <strong>and</strong> evidencethat services can meet those needs <strong>and</strong> offered care influence health outcomes - rather thanmodels mirroring directions of health policy. This will require improved in<strong>for</strong>mation onconsumer needs <strong>and</strong> an explicit linkage of this in<strong>for</strong>mation to evidence of existing capacityto address these needs (including evidence of effectiveness such as that provided by theCochrane Centre Collaborations).

54♦♦♦Acceptability: There is a particular need <strong>for</strong> better instruments to monitor the culturalappropriateness of care <strong>and</strong> the perceptions of vulnerable subgroups receiving care (suchas Aboriginals <strong>and</strong> Torres Strait Isl<strong>and</strong>ers).Continuity: Feasibility studies addressing the methodologies <strong>for</strong> obtaining relevantfeedback from representative, community-based healthcare professionals on the success ofintegration of acute healthcare services into the care continuum should be supported.Appropriateness: Research should be pursued into methods <strong>for</strong> linking patient-basedperceptions of the outcomes of care <strong>and</strong> preferences regarding the weighting of indications<strong>for</strong> care delivery (based upon patient perception of need <strong>for</strong> care) to provider assessmentsof the risks <strong>and</strong> benefits of care delivery. Initial studies should focus on commonconditions with evidence that patient election regarding care is a significant issue (e.g.treatment <strong>for</strong> benign prostatic hypertrophy <strong>and</strong> primary breast cancer).There are important dimensions of quality of care which are inadequately represented in ourproposed framework <strong>for</strong> per<strong>for</strong>mance measurement. Equity is only partly encompassed by accessmeasures, benevolence is incompletely represented in indicators of effectiveness <strong>and</strong> acceptability<strong>and</strong> the ethics of care delivery essentially unrepresented. These, <strong>and</strong> other, care dimensions areimportant. If it were feasible to develop quality indicators that better reflect these more complex<strong>and</strong> value-laden aspects of healthcare it would clearly be desirable. It is our opinion that thelikelihood of developing credible, quantitative indices of these more complex dimensions of care islow <strong>and</strong> that <strong>for</strong> the <strong>for</strong>eseeable future most research <strong>and</strong> development ef<strong>for</strong>ts in quality indicatordevelopment in acute healthcare would usefully be focused within the framework offered by ourdimensions of care quality.7.3.2 Indicator Development Strategies:♦We strongly recommend that future national indicator development be addressed bymultidisciplinary groups <strong>and</strong> be directed towards specific clinical circumstances with aview to the evolution of balanced sets of indicators (i.e. reflecting clinical indicators,patient functional health outcomes <strong>and</strong> HRQOL, the acceptability of care <strong>and</strong> cost of care)rather than exhaustive sets of indicators addressing a relatively narrow range ofdimensions of the quality of care delivery processes or achieved health outcomes.Australian quality <strong>and</strong> outcome indicator development should be more closely linked tointernational developments in health services research. Wherever possible, we should seekto build upon the knowledge established by others in much larger population bases <strong>and</strong>adapt putative indicators <strong>for</strong> local utilisation trials rather than building all quality of careindicators “from the ground up”. Linking our acute healthcare indicators more closely tointernational indicators would allow <strong>for</strong> more international comparisons - which webelieve will be increasingly relevant in health services as in benchmarking per<strong>for</strong>mance inother major national industries. The trialing of such derivative indicators would alsoallow our intellectual <strong>and</strong> fiscal resources to concentrate upon indicators reflectingnuances of healthcare delivery that are uniquely Australian <strong>and</strong> may speed thedevelopment of indicators of relevance to health issues or consumer groups currentlypoorly served by existing quality of care <strong>and</strong> health outcome indicators.♦We strongly recommend that work be supported to develop an indicator module whichspecifically addresses the health concerns of vulnerable patient groups, in particularAboriginal <strong>and</strong> Torres Strait Isl<strong>and</strong>er peoples. Together with stratification of indicatordata, this approach would strengthen knowledge of current gaps in service delivery <strong>and</strong>assist in monitoring of the impact of strategies to enhance care of vulnerable subgroups.

55♦Australia should seek to identify centres of excellence to link into the WHO <strong>Quality</strong>Assurance Collaboration Centre, to promote interfaces between clinical practice <strong>and</strong>academic developments in quality of care indicators.7.3.3 Integrated Health System Per<strong>for</strong>mance Assessment:♦♦National quality <strong>and</strong> outcome indicators should increasingly focus on integrated healthservice delivery rather than the per<strong>for</strong>mance of individual sectors such as acute care.Most of the major health concerns of Australians relate to chronic illnesses, where theper<strong>for</strong>mance of the integrated system of care is far more important than the per<strong>for</strong>manceof its isolated components. Patients turn to the healthcare services seeking adequate care<strong>and</strong> hoping <strong>for</strong> good overall outcomes. With chronic illnesses (such as hypertension,ischaemic heart disease, diabetes, arthritis <strong>and</strong> cancer) the overall success of interventionsmust sum the inputs of health promotion, preventative care, early detection programs <strong>and</strong>episodes of care in community <strong>and</strong> hospital sectors. Analyses of per<strong>for</strong>mance must judgewhether appropriate care has been delivered (e.g. education about risk reduction,screening <strong>for</strong> early disease, risk factor identification <strong>and</strong> therapeutic intervention) notwhere that care was delivered. Many of the best indicators of quality of care are suchglobal measures of per<strong>for</strong>mance (such as have patients been advised about relevantlifestyle changes, has blood pressure or cholesterol-level been checked, has adequacy ofdiabetes control <strong>and</strong> diabetes self-education been implemented, have necessary vaccinesbeen administered in a timely fashion).Any move to an integrated per<strong>for</strong>mance monitoring would require both the adoption of aunique identifier which respected the privacy <strong>and</strong> confidentiality needs of patients <strong>and</strong> thecooperation of the myriad of agencies - both at government <strong>and</strong> regulatory levels - whichoversight our compartmentalised health system. Such a move towards a more holisticper<strong>for</strong>mance appraisal may seem impractical given current realities, but the common goalof improving the value of healthcare, by raising the quality of care <strong>and</strong> lowering ormaintaining costs, will not be realised until such a global perspective on health servicesper<strong>for</strong>mance is adopted.

56BIBLIOGRAPHYNote: *Represents works regarded by us as key in in<strong>for</strong>ming our decision-making <strong>for</strong> the purposes of thisreport.1. Brennan T, Leape L, Laird N et al. The Harvard Medical Practice Study. New Engl<strong>and</strong> Journal of Medicine 324: 370-384,1991 *2. Metropolitan Hospitals Planning Board - Phase 2: Discussion Paper: Criteria to Assess the Per<strong>for</strong>mance of Networks.Metropolitan Hospitals Planning Board Report. Victorian Government Printing Office. 19953. Scally G. Dealing with duffers. The Lancet 346: 720, 19954. Stiell I, Wells G, Laupacis A et al. Multicentre trial to introduce the Ottawa Ankle Rules <strong>for</strong> use of radiography in acuteankle injuries. British Medical Journal 311: 594-597, 19955. Castella X, Artigas A, Bion J et al. The European/North American Severity Study Group. Critical Care Medicine 23:1327-1335, 19956. Rapoport J, Teres D, Barnett R et al. A comparison of Intensive Care Unit utilisation in Alberta <strong>and</strong> WesternMassachusetts. Critical Care Medicine 23: 1336-1346, 1995 *7. McGarvey RN, Harper JJ. Pneumonia mortality reduction <strong>and</strong> quality improvement in a community hospital. <strong>Quality</strong>Review Bulletin 19: 124-130, 19938. Berwick D. Eleven worthy aims <strong>for</strong> clinical leadership of health system re<strong>for</strong>m. Journal of the American MedicalAssociation 272: 797-802, 1994 *9. Symposium on the Australian Incident Monitoring Study. Anaesthesia <strong>and</strong> Intensive Care 21: 501-595, 1993 *10. Leape LL, Bates DW, Cullen DJ. Systems analysis of adverse drug events. JAMA 274: 35-43, 199511. Leape LL. Error in medicine. JAMA 272: 1851-1857, 1994 *12. Headrick L <strong>and</strong> Neuhauser D. <strong>Quality</strong> health care. JAMA 273: 1718-1720, 1995 (R), JAMA 271: 1711-1712, 199313. Goldman RL, Thomas TL. Using mortality rates as a screening tool: The experience of the Department of VeteransAffairs. Joint Commission Journal on <strong>Quality</strong> Improvement 20 (9): 511-521, 1994 *14. Braff JB, Way BB, Steadman HJ. Incident Reporting: Evaluation of New York’s pilot incident logging system. <strong>Quality</strong>Review Bulletin 12 (3): 90-98, 198615. Department of Health. Review of Guidance on Doctors' Per<strong>for</strong>mance: Maintaining Medical Excellence. London. DOH.1995. AND Editorial Lancet 346: 431, 199516. Anderson GM, Newhouse JP, Roos LL. Hospital care <strong>for</strong> elderly patients with diseases of the circulatory system: Acomparison of hospital use in the USA <strong>and</strong> Canada. NEJM, 321: 1443-1448, 198917. Simbourg DW. DRG Creep: A new hospital-acquired disease. NEJM 304: 1602-1604, 198118. Stern RS, Epstein AM. Institutional responses to prospective payment based on diagnosis-related groups: Implications <strong>for</strong>cost, quality <strong>and</strong> access. NEJM 312: 621-627, 198519. Ginsburg PB, Carter GM. Medicare case-mix index increase. Health Care Finance Review 7: 51-65, 198620. Winslow CM, Kosecoff JB, Chassin M, Kanousee D, Brook RH. The appropriateness of per<strong>for</strong>ming coronary artery bypasssurgery. JAMA 260: 505-509, 198821. Brook RH, Kosecoff JB, Park RE, Chassin MR, Winslow CM, Hampton JR. Diagnosis <strong>and</strong> treatment of coronary disease:comparison of doctors' attitudes in the USA <strong>and</strong> UK. Lancet 750-753, 1988 *22. Wennberg JE, Gittelsohn A. Small-area variations in health care delivery. Science 182: 1102-1108, 1972

5723. Wennberg JE, Gittelsohn A. Variations in medical care among small areas. Scientific American 46: 120-134, 198224. Chassin MR, Brook RH, Park PE et al. Variations in the use of medical <strong>and</strong> surgical services by the Medicare population.NEJM 314: 285-290, 198625. Leape LL, Park RE, Solomon DH, Chassin MR, Kosecoff JJ, Brook RH. Does Inappropriate Use Explain Small-areaVariations in the Use of Health Care <strong>Services</strong>? JAMA 263: 669-672, 199026. Burns H. Interventions vary (thrombolytics in AMI, haemorrhoidectomy, prostatectomy, hysterectomy, D&C, tonsillectomy,CABG) as do outcomes (cancer). Healthcover 5: 36-39, 199527. Classen DC, Evans S, Pestotnik SL, Horn SD, Menlove RL, Burke JP. The timing of prophylactic administration ofantibiotics <strong>and</strong> the risk of surgical wound infection. NEJM 326: 281-286, 199228. Rapoport J, Teres D, Lemeshow S, Gehlbach S. A method <strong>for</strong> assessing the clinical per<strong>for</strong>mance <strong>and</strong> cost-effectiveness ofintensive care units: A Multi-Centre Inception Cohort Study. Critical Care Medicine 22: 1385-1391, 199429. Jollis JG, Ancukiewicz M, DeLong E, et al. Discordance of databases designed <strong>for</strong> claims payment versus clinicalin<strong>for</strong>mation systems. Implications <strong>for</strong> outcomes research. Annals of Internal Medicine 119: 844-850, 199330. Miller M, Miller L, Fireman B, Black S. Variation in practice <strong>for</strong> discretionary admissions. Impact on estimates of qualityof hospital care. JAMA 271: 1493-1498, 199431. Orchard C. Comparing healthcare outcomes. BMJ 308: 1493-1496, 199432. Comorbidities, complications <strong>and</strong> coding bias. Does the number of diagnosis codes matter in predicting in-hospitalmortality? JAMA 267: 2197-2203, 1992 *33. Personal Communication. Dr Marie Miller. Department of <strong>Quality</strong> <strong>and</strong> Utilisation. Kaiser Permanente Medical CareProgram. 1814 Franklin St., Oakl<strong>and</strong> CA 94612, 199634. Concato J, Horwitz R, Feinstein AR et al. Problems of comorbidity in mortality after prostatectomy. JAMA 267: 1077-1082, 1992 *35. Budois RW, Rogers WH, Moxley JH, Draper D, Brook RH. Hospital inpatient mortality - is it a predictor of quality?NEJM 317: 1674-1680, 1987 *36. Marc R, de Leval M et al. Analysis of a cluster of surgical failures. Application to a series of neonatal arterial switchoperations. Journal of Thoracic <strong>and</strong> Cardiovascular Surgery 107: 914-924, 199437. Brook RH <strong>and</strong> Lohr K. Monitoring quality of care in the Medicare program. Two proposed systems. JAMA 258: 3138-3141, 198738. Greenfield S, Aronow H, Elashoff R, Watanabe D. Flaws in mortality data. The hazards of ignoring comorbid disease.JAMA 260: 2253-2255, 1987 *39. Leape L, Hillborne L, Park R et al. The appropriateness of use of coronary artery bypass graft surgery in New York State.JAMA 269: 753-760, 1993 *40. Hillborne L, Leape L, Bernstein S et al. The appropriateness of use of PCTCA in New York State. JAMA 269: 761-765,1993 *41. Bernstein S, Hillborn L, Leape L, et al. The appropriateness of use of coronary angiography in New York State. JAMA269: 766-769, 1993 *42. Bates D, Cullen D, Laird N et al. Incidence of adverse drug events <strong>and</strong> potential adverse drug events. Implications <strong>for</strong>prevention. JAMA 274: 29-34, 1995 *43. Cooper JB, Newbower RS, Kitz RJ. An analysis of major errors <strong>and</strong> equipment failures in anaesthesia management:Considerations <strong>for</strong> prevention <strong>and</strong> detection. Anaesthesiology 60: 34-42, 198444. Gaba DM, Maxwell M, DeAnda A. Anaesthetic mishaps: Breaking the chain of accident evolution. Anaesthesiology 66:670-676, 198745. Bogner MS Ed. Human error in medicine. Hillsdale NJ. Erlbaum 1994

5846. Davies JM, Webb RK. Adverse events in anaesthesia: The wrong drug. Canadian Journal of Anaesthesia 41: 83-86,199447. O'Neil AC, Peterson LA, Cook EF et al. Physician reporting compared with medical record review to identify adversemedical events. Annals of Internal Medicine 119: 370-376, 1993 *48. Rubenstein L, Kahn K, Reinisch E et al. Changes in quality of care <strong>for</strong> 5 diseases measured by implicit review, 1981 to1986. JAMA 264: 1974-1979, 199049. Williams S, Bryan P, Schlup M. <strong>Quality</strong> control: An application of the cusum. BMJ 304: 1359-1361, 199250. Brennan T, Localio R, Leape L et al. Identification of adverse events occurring during hospitalisation. A cross-sectionalstudy of litigation, quality assurance <strong>and</strong> medical records at two teaching hospitals. Annals of Internal Medicine 112: 221-226, 1990 *51. Knaus W, Draper E, Wagner D, Zimmerman J. An evaluation of outcome from intensive care in major medical centres.Annals of Internal Medicine 104: 410-418, 198652. Caplan R, Posner K, Cheney F. Effects of outcome on physician judgements of appropriateness of care. JAMA 265: 1957-1960. Anaesthesia B2-AN, 1991 *53. Watching the doctor-watchers. How well do PRO methods detect hospital care quality problems. JAMA 267: 2349-2354,199254. Panniers TL, Newl<strong>and</strong>er J. The adverse patient occurrences inventory: Validity, reliability <strong>and</strong> implications. <strong>Quality</strong>Review Bulletin, September 198655. Kaz<strong>and</strong>jian VA. Relating outcomes to processes of care: The Maryl<strong>and</strong> Hospital Association's <strong>Quality</strong> Indicator Project.Joint Commission Journal on <strong>Quality</strong> Improvement 530-538, November 199356. Kaz<strong>and</strong>jian VA, Lawthers J, Cernak C <strong>and</strong> Pipesh F. <strong>Quality</strong> <strong>and</strong> accountability. Joint Commission Journal on <strong>Quality</strong>Improvement 20: 7, 199457. Burstin HR, Lipsitz SR, Brennan TA. Socioeconomic status <strong>and</strong> risk <strong>for</strong> subst<strong>and</strong>ard medical care. JAMA 268: 2383-2387,199258. Chassain MR. Improving the quality of healthcare. NEJM 335: 1060-1062,199659. Weissman JS, Gatsonis C, Epstein AM. Rates of avoidable hospitalization by insurance status in Massachusetts <strong>and</strong>Maryl<strong>and</strong>. JAMA 268: 2388-2394, 199260. Blumenthal D, Epstein A. The role of physicians in the future of quality management. NEJM 335: 1328-1331, 199661. Brennan TA, Hebert LE, Laird NM et al. Hospital characteristics associated with adverse events <strong>and</strong> subst<strong>and</strong>ard care.JAMA 265: 3265-3269, 199162. Couch NP, Tilney NL, Rayner AA, Moore FD. The high cost of low-frequency events. The anatomy <strong>and</strong> economics ofsurgical mishap. NEJM 304: 634-637, 198163. Steel K, Gertman PM, Crescenzi C, Anderson J. Iatrogenic illness on a general medical service at a University TeachingHospital. NEJM 304: 638-642, 198164. Roessner J. The healthiest place in America. <strong>Quality</strong> Connection 2: 10-11, 199365. Hiatt HH, Barnes BA, Brennan TA et al. A study of medical injury <strong>and</strong> medical malpractice. NEJM 321: 480-484, 198966. McDonald C. Protocol-based computer reminders, the quality of care <strong>and</strong> the non-perfectability of man. NEJM 295: 1351-1355, 197667. Phelps C E. The methodologic foundations of studies of the appropriateness of medical care. NEJM329: 1241-1245, 199368. Tannenbaum S. What physicians know. NEJM 329: 1268-1271, 199369. Vincent CA. Research into medical accidents: A case of negligence? BMJ 299: 1150-1153, 1989 *

5970. Nadzam DM. Infection control indicators in critical care settings. Heart & Lung 21 (5): 477-481, 199271. Schafer J et al. <strong>Outcome</strong> prediction models on admission in a medical intensive care unit: Do they predict individualoutcome? Critical Care Medicine 18 (10): 1111-1117, 199072. Nadzam DM et al. Data-driven per<strong>for</strong>mance improvement in health care: The Joint Commission's Indicator MeasurementSystem (IM System). Joint Commission Journal on <strong>Quality</strong> Improvement 19 (11): 492-500, November 199373. Hart GK, Baldwin I, Gutteridge G, Ford J. Adverse incident reporting in intensive care. Anaesthesia <strong>and</strong> Intensive Care 22:556-561, 199474. Giraud T et al. Iatrogenic complications in adult intensive care units: A prospective two-centre study. Critical CareMedicine 21: 40-51, 199375. Allnut MF. Human factors in accidents. British Journal of Anaesthesia 856-864, 1987 *76. Luft HS, Bunker JP, Enthoven AC. Should operations be regionalised? The empirical relationship between surgicalvolume <strong>and</strong> mortality. NEJM 301: 1364-1369, 197977. Kimmel SE, Berlin JE <strong>and</strong> Laskey WK. The relationship between coronary angioplasty procedure volume <strong>and</strong> majorcomplications. JAMA 274: 1137-1142, 1995 **78. Lee A, Bishop G, Hillman KM, Daffun K. The medical emergency team. Anaesthesia <strong>and</strong> Intensive Care 23: 183-186,199579. Quigley P, Mathis A <strong>and</strong> Nodhturft V. Improving clinical documentation quality. Journal of Nursing Care <strong>Quality</strong> 8 (4):66-73, 199480. Hartz AJ et al . Assessing providers of coronary revascularization: A method <strong>for</strong> peer review organisations. AmericanJournal of Public Health 82 (12): 1631-1640, 199281. Winograd CH et al. Screening <strong>for</strong> frailty: Criteria <strong>and</strong> predictors of outcomes. Journal of the American Geriatrics Society39: 778-784, 199182. Ferguson RP et al. Serum albumin <strong>and</strong> prealbumin as predictors of clinical outcomes of hospitalized elderly nursing homeresidents. Journal of the American Geriatrics Society 41: 545-549, 199383. Iezzoni L, Ash A, Coffman G, Moskowitz M. Predicting in-hospital mortality. A comparison of severity measurementapproaches. Medical Care 30: 347-359, 1992 *84. Holloway JJ, V<strong>and</strong>erburg Medendrop S, Bromberg J. Risk factors <strong>for</strong> early readmissions among veterans. Health <strong>Services</strong>Research 25: 213-237, 199085. DesHarnis S, McMahon L, Woblewski R. Measuring outcomes of hospital care using multiple risk-adjusted indices. Health<strong>Services</strong> Research 26: 425-445, 1991 *86. Bradbury RC, Stearns FE, Steen P. Interhospital variations in admission-severity-adjusted hospital morbidity <strong>and</strong> mortality.Health <strong>Services</strong> Research 26: 408-424, 199187. Nettleman MD <strong>and</strong> Nelson AP. Adverse occurrences during hospitalisation on a general medical service. ClinicalPer<strong>for</strong>mance <strong>and</strong> <strong>Quality</strong> Health Care 2: 67-72, 199488. Mark BA, Burleson DL. Measurement of patient outcomes: Data availability <strong>and</strong> consistency across hospitals. Journal ofNursing Administration 25: 52-59, 199589. Bates DW, O'Neil AC et al. Potential identifiability <strong>and</strong> preventability of adverse events using in<strong>for</strong>mation systems.Journal of the American Medical In<strong>for</strong>matics Association 1: 404-411, 199490. Hammond P, Harris AL, Das SK, Wyatt JC. Safety <strong>and</strong> decision support in oncology. Methods of In<strong>for</strong>mation in Medicine33: 371-381, 199491. Happ BA. The effect of point of care technology on the quality of patient care. Proceedings - The Annual Symposium onComputer Applications in Medical Care 183-187, 199392. Roos NP. Using administrative data from Manitoba, Canada to study treatment outcomes: developing control groups <strong>and</strong>

60adjusting <strong>for</strong> case severity. Social Science <strong>and</strong> Medicine 28: 109-113, 1989 *93. Silber JH, Williams SV, Krakauer H, Schwartz JS. Hospital <strong>and</strong> patient characteristics associated with death after surgery.A study of adverse occurrences <strong>and</strong> failure to rescue. Medical Care 30: 615-629, 199294. Conducting medical effectiveness research: A report from the Inter-PORT Work Groups. (Various Authors). Medical Care32 (Suppl): JS1-JS110, 1994 *95. Anderson GF, Alonso J, Kohn LT, Black C. Analyzing health outcomes through international comparisons. Medical Care32: 526-534, 199496. Young WW, Kohler S, Kowalski J. PMC patient severity scale: Derivation <strong>and</strong> validation. Health <strong>Services</strong> Research 29:367-390, 199497. Iezzoni L, Heeren T, Foley S et al. Chronic conditions <strong>and</strong> risk of in-hospital deaths. Health <strong>Services</strong> Research 29: 435-460, 199498. Aiken L, Smith HL, Lake ET. Lower Medicare mortality among a set of hospitals known <strong>for</strong> good nursing care. MedicalCare 32: 771-787, 199499. Iezzoni LI, Daley J, Heeren T et al. Identifying complications of care using administrative data. Medical Care 32: 700-715, 1994 *100. Romano P, Mark D. Bias in the coding of hospital discharge data <strong>and</strong> its implications <strong>for</strong> quality assessment. MedicalCare 32: 81-90, 1994101. Roos LL, Mustard CA, Nicol JP et al. Registries <strong>and</strong> administrative data: Organisation <strong>and</strong> accuracy. Medical Care 31:201-212, 1993102. Best WR <strong>and</strong> Cowper DC. The ratio of observed-to-expected mortality as a quality of care indicator in non-surgical VApatients. Medical Care 32: 390-400, 1994103. Des Harnais SI, McMahan LF, Wroblewsk RT <strong>and</strong> Hogan AT. Measuring hospital per<strong>for</strong>mance. The development <strong>and</strong>validation of risk-adjusted indices of mortality, readmissions <strong>and</strong> complications. Medical Care 28: 1127-1141, 1990 *104. Green LA <strong>and</strong> Becker MP. Physician decision making <strong>and</strong> variation in hospital admission rates <strong>for</strong> suspected acute cardiacischaemia. A tale of two towns. Medical Care 32: 1086-1097, 1994105. Selker HP, Griffith JL <strong>and</strong> D'Agostino RB. A time-insensitive predictive instrument <strong>for</strong> acute hospital mortality due tocongestive heart failure: Development, testing <strong>and</strong> use <strong>for</strong> comparing hospitals: A multicentre study. Medical Care 32:1040-1052, 1994106. Hartz AJ, Guse C, Sigmann P et al. Severity of illness measures derived from the Uni<strong>for</strong>m Clinical Data Set (UCDS).Medical Care 32: 881-901, 1994107. Friede A, Taylor W, Wadelman L. On-line access to a cost-benefit/cost-effectiveness analysis bibliography via CDCWONDER Medical Care 31: JS12-JS18,1993108. Brennan TA, Localio RT, Laird NL. Reliability <strong>and</strong> validity of judgements concerning adverse events suffered byhospitalised patients. Medical Care 27: 1148-1158, 1989 *109. Peterson LA, Brennan TA, O'Neill AC et al. Does housestaff discontinuity of care increase the risk <strong>for</strong> preventable adverseevents? Annals of Internal Medicine 121: 866-872, 1994110. Hartz AJ, Kuhn EM, Kayser KL, et al. Assessing providers of coronary revascularisation - a method <strong>for</strong> peer revieworganisations. American Journal of Public Health 82: 1631-1640, 1992111. Sutton J, St<strong>and</strong>en P, Wallace A. Unreported accidents to patients in hospital. Nursing Times 90: 46-49, 1994112. Krakauer H, Bailey BC, Cooper H, Yu WK, Skellan KJ, Kattakkuzhy G. The systematic assessment of variations inmedical practices <strong>and</strong> their outcomes. Public Health Reports 110: 2-12, 1995113. Lindley CM, Tully MP, Paramsothy V, Tallis RC. Inappropriate medication is a major cause of adverse drug reactions inelderly patients. Age <strong>and</strong> Aging 21: 294-300, 1992

61114. Wolff AM. Limited adverse occurrence screening. A medical quality control system <strong>for</strong> medium sized hospitals. MJA 156:449-452, 1992 *115. Clayton DG, Barker L, Runciman WB. Evaluation of safety procedures in anaesthesia <strong>and</strong> intensive care. Anaesthesia <strong>and</strong>Intensive Care 21: 670-672, 1993 *116. Roos NP, Black CD, Roos LL, et al. A population-based approach to monitoring adverse outcomes of medical care.Medical Care 33: 127-138, 1995 *117. Hardwick RH, Saltrese-Taylor A, Collins D. Need to measure outcome after discharge in surgical audit. <strong>Quality</strong> in HealthCare 1: 165-167, 1992118. Naessens JM, Leibson CL, Krishan I, Ballard DJ. Contribution of a measure of disease complexity (COMPLEX) to aprediction of outcome <strong>and</strong> charges among hospitalised patients. Mayo Clinic Proceedings 67: 1140-1149, 1992119. Riley G, Lubitz J, Gornick M, Mentech R, Eggers P, McBean M. Medicare beneficiaries: Adverse outcomes afterhospitalisation <strong>for</strong> eight procedures. Medical Care 31: 921-949, 1993120. H<strong>and</strong> R, Pointek F, Klemka-Walden L, Inczauskis D. Use of statistical control charts to assess outcomes of medical care:Pneumonia in Medicare patients. American Journal of the Medical Sciences 307: 329-334, 1994121. Ashton CM, Kuykendall DH, Johnson ML, et al. A method of developing <strong>and</strong> weighting explicit process of care criteria <strong>for</strong>quality assessment. Medical Care 32: 755-770, 1994122. Knaus W, et al. APACHE-II: A severity of disease classification system. Critical Care Medicine 13 (10): 818-829, 1985123. Lemeshow S, Teres D, et al. A comparison of methods to predict mortality of intensive care unit patients. Critical CareMedicine 15 (8): 715-722, 1987124. Brannen A L, Godfrey L V, Goetter W E. Prediction of outcome from critical illness: A comparison of clinical judgementwith a prediction rule. Archives of Internal Medicine 149, 1989125. Lemeshow S, et al. Refining intensive care unit outcome prediction by using changing probabilities of mortality. CriticalCare Medicine 16 (5): 470-477, 1988126. Segal HE, Rummel L, Wu B. The utility of PRO data on surgical volume: The example of carotid endarterectomy. <strong>Quality</strong>Review Bulletin 19: 152-157, 1993127. Jencks S F. Measuring quality of care under Medicare <strong>and</strong> Medicaid. Health Care Financing Review 16: 39-54, 1995 *128. R<strong>and</strong>all D, Cebul. <strong>Quality</strong> improvement, casemix <strong>and</strong> preventive procedures: When should the playing field be leveled <strong>for</strong>carotid endarterectomy? <strong>Quality</strong> Review Bulletin 19: 150-151, 1993129. Leape LL, Lawthers AG, Brennan TA, Johnson WG. Preventing medical injury. <strong>Quality</strong> Review Bulletin 19: 144-149, 1993130. Thomas JW <strong>and</strong> Holloway JJ. Investigating early readmission as an indicator <strong>for</strong> quality of care studies. Medical Care 29(4): 377-394, April 1991131. Dubois RW, et al. SPECIAL REPORT: Hospital inpatient mortality: Is it a predictor of quality? NEJM 317 (26): 1674-1679, 1987 *132. Chang RWS, et al. Use of APACHE II severity of disease classification to identify intensive care unit patients who wouldnot benefit from total parenteral nutrition. Lancet : 1483-1486, 28 June 1986133. Iezzoni LI, et al. Comorbidities, complications <strong>and</strong> coding bias - Does the number of diagnosis codes matter in predictingin-hospital mortality? JAMA 267 (16): 2197-2203, 22/29 April 1992 *134. Oulton R. Use of incident report data in a system-wide quality assurance/risk management program. <strong>Quality</strong> ReviewBulletin 10: 583-587, December 1994135. Hannan EL, et al. A methodology <strong>for</strong> targeting hospital cases <strong>for</strong> quality of care record reviews. American Journal ofPublic Health 79 (4): 430-436, April 1989136. Bion JF, et al. Sickness scoring <strong>and</strong> response to treatment as predictors of outcome from critical illness. Intensive CareMedicine 14: 167-172, 1988

62137. Westbrook JJ, Rushworth RL, Rob MI. Evaluating health care: What can hospital seperation data tell us about thecomplications of hospital care? Journal of <strong>Quality</strong> in Clinical Practice 14: 157-166, 1994138. Communication: Deborah M Nadzam PhD RN, Director, Indicator Measurement, JCAHO139. DesHarnais SI, Simpson KN. Indices <strong>for</strong> monitoring hospital outcomes in developed countries. Health Policy 21: 1-15,1992140. Health care re<strong>for</strong>m "report cards" are useful but significant issues need to be addressed. GAO. Report to the Chairman,Committee of Labour <strong>and</strong> Human Resources, US Senate, Sept 1994. US GAO Health, Education <strong>and</strong> Human <strong>Services</strong>Division **141. Kaz<strong>and</strong>jian VA. <strong>Outcome</strong>s research <strong>and</strong> accountability: the race to adjust per<strong>for</strong>mance data <strong>for</strong> risk of public disclosure.Vice President <strong>for</strong> Research, Maryl<strong>and</strong> Hospital Association, Feb 1995 (submitted <strong>for</strong> publication) *142. Joint Commission on Accreditation of <strong>Healthcare</strong> Organisations. IMSystem General In<strong>for</strong>mation. 24 February 1995143. Kaz<strong>and</strong>jian V, et al. Balancing science <strong>and</strong> practice in Indicator development: The Maryl<strong>and</strong> Hospital Association <strong>Quality</strong>Indicator (QI) Project. International Journal <strong>for</strong> <strong>Quality</strong> in Health Care 7: 39-46, 1995144. Joint Commission Risk Adjustment Methodology. Joint Commission on Accreditation of <strong>Healthcare</strong> Organisations. 24February 1995145. Bates D, et al. Evaluation of screening criteria <strong>for</strong> adverse events in medical patients. Medical Care 33: 452-462, 1995146. Hammermeister K et al. A literature review of evidence <strong>for</strong> process <strong>and</strong> outcome interrelationships. Medical Care 33: 055-056, 1995 *147. Hofer TP <strong>and</strong> Hayward RA. Can early readmission rates accurately detect poor quality hospitals? Medical Care 33: 234-245, 1995 *148. Wray P et al. Selecting disease-outcome pairs <strong>for</strong> monitoring the quality of hospital care. Medical Care 33: 75-89, 1995 *149. Hammermeister et al. Why it is important to demonstrate linkages between outcomes of care <strong>and</strong> processes <strong>and</strong> structuresof care. Medical Care 33: 055-056, 1995 *150. Classen D et al. Computerised surveillance of adverse drug events in hospital patients. JAMA 266: 2847-2851, 1991151. Evans RS et al. Prevention of adverse drug events through computer surveillance. Proceedings - The Annual Symposium onComputer Applications in Medical Care 437-441, 1992152. Westbrook JJ, Rushworth RL, Rob MI. Evaluating health care: What can hospital seperation data tell us about thecomplications of hospital care? Journal of <strong>Quality</strong> in Clinical Practice 14: 157-166, 1994153. Wolff AM. A review of methods used <strong>for</strong> medical quality assurance in hospitals: Advantages <strong>and</strong> disadvantages. Journalof <strong>Quality</strong> in Clinical Practice 14: 85-97, 1994154. Localio AR <strong>and</strong> Hamory BH. A report card <strong>for</strong> report cards. Annals of Internal Medicine 123: 802-803, 1995 *155. Sibritt DW. Readmissions of day-only patients in NSW acute hospitals. Journal of <strong>Quality</strong> in Clinical Practice 14: 31-38,1994156. Iezzoni LI et al. Predicting who dies depends on how severity is measured: Implications <strong>for</strong> evaluating patient outcomes.Annals of Internal Medicine 123: 763-770, 1995 **157. Clement D, et al. Access <strong>and</strong> outcomes of elderly patients enrolled in managed care. JAMA 271: 1487-1492, 1994158. Access of Medicaid recipients to outpatient care. NEJM 330: 1426-1430, 1994159. Majeed FA et al. Monitoring <strong>and</strong> promoting equity in primary <strong>and</strong> secondary care. BMJ 308: 1426-1429, 1994160. Friedman E. Bibliography of "Money isn't everything - Non-financial barriers to access". JAMA 271: 1535-1538, 1994161. Jost T. Health System Re<strong>for</strong>m. Forward or backward with quality insight. JAMA 271: 1508-1511, 1994

63162. Freeman H <strong>and</strong> Corey CR. Insurance status <strong>and</strong> access to health services among poor persons. Health <strong>Services</strong> Research 28:531-541, 1994163. Economic appraisal <strong>and</strong> health technology policy. Social Science <strong>and</strong> Medicine 38: 1591-1678, 1994164. Lairsen DR et al. Equity of healthcare in Australia. Social Science <strong>and</strong> Medicine 41: 475-482, 1995 *165. Carr-Hill R A. Efficiency <strong>and</strong> equity implications of the Health Care Re<strong>for</strong>ms. Social Science <strong>and</strong> Medicine 39: 1189-1201, 1994166. Vagero D. Equity <strong>and</strong> efficiency in health re<strong>for</strong>m. A European view. Social Science <strong>and</strong> Medicine 39: 1203-1210, 1994167. Bashshur RL et al. Beyond the uninsured: Problems in access to care. Medical Care 32: 409-419, 1994168. Britton S. A hospital-wide outcome study. Australian Clinical Review 11, 132-135, 1991169. Stipek CE. Grouping systems enhance outcomes analysis. Topics in Health In<strong>for</strong>mation Management 15 (4): 14-25, 1995170. Bruster S, Jarman B et al. National survey of hospital patients. BMJ 309: 1542-1546, 1994171. North American Nursing Diagnosis Association (NANDA). Nursing Diagnosis: Definition <strong>and</strong> Classification 1995-1996.Philadelphia PA NANDA 1994172. Martin K. Omaha systems. Classification systems <strong>for</strong> describing nursing practice. Kansas City, MO, American NursesAssociation, 1990173. Moorhead S et al. Nursing interventions classification (NIC). A comparison with the Omaha System <strong>and</strong> the Home CareClassification. Journal of Nursing Administration 23: 23-29, 1993174. Delaney C, Moorhead S. The nursing minimum data set, st<strong>and</strong>ardised language <strong>and</strong> health care quality. Journal ofNursing Care <strong>Quality</strong> 10: 16-30, 1995175. Marek KD. <strong>Outcome</strong> measurement in nursing. Journal of Nursing <strong>Quality</strong> Assurance 4 (1): 1-9, 1989176. Rantz MJ. <strong>Quality</strong> measurement in nursing: Where are we now? Journal of Nursing Care <strong>Quality</strong> 9 (2): 1-7, 1995177. McDaniel C <strong>and</strong> Nash J. Compendium of instruments measuring patient satisfaction with nursing care. <strong>Quality</strong> ReviewBulletin 16: 182-188, 1990178. Coenen A <strong>and</strong> Schoneman D. The nursing minimum data set: Use in the quality process. Journal of Nursing Care <strong>Quality</strong>10: 9-15, 1995179. Khuri SF, Daley J et al. The National Veterans Administration Surgical Risk Study: risk adjustment <strong>for</strong> the comparativeassessment of the quality of care. Journal of the American College of Surgeons 180: 519-531, 1995180. O'Leary A. Patient satisfaction as a measure of quality in the care of the elderly. BMJ 1 (9): 470-472, 1992181. Ewing H et al. Developing clinical indicators <strong>for</strong> cholecystectomy. Australian <strong>and</strong> New Zeal<strong>and</strong> Journal of Surgery 63:181-185, 1993182. Ruffin M. Developing <strong>and</strong> using a data repository <strong>for</strong> quality improvement: The genesis of IRIS. Joint CommissionJournal on <strong>Quality</strong> Improvement 21 (10): 512-520, October 1995183. Cullen D et al. The incident reporting system does not detect adverse drug events: A problem <strong>for</strong> quality improvement.Joint Commission Journal on <strong>Quality</strong> Improvement 21 (10): 541-548, October 1995184. Schiff G. Red man syndrome <strong>and</strong> the red button system: Where next <strong>for</strong> inpatient adverse drug reaction reporting? JointCommission Journal on <strong>Quality</strong> Improvement 21 (10): 549-552, October 1995185. AM Wolff. Limited adverse occurrence screening: using medical record review to reduce hospital adverse patient events.MJA 164: 458-461, 1996186. Morris A, Clemmer T <strong>and</strong> Pestotnik S. Personal Communications.

64187. James B. Bibliographic material on computer assisted decision-making. Vice President of Medical Research <strong>and</strong>Continuing Medical Education. IHC Institute <strong>for</strong> Health Care Delivery Research. Utah, USA.188. Draper M <strong>and</strong> Hill S. The role of patient satisfaction surveys in a national approach to hospital quality management.Australian Government Publishing Service, Oct 1995189. Agency <strong>for</strong> Health Care Policy <strong>and</strong> Research. Using clinical practice guidelines to evaluate quality of care. AHCPR Pub95-0045, March 1995190. Blumenthal D. <strong>Quality</strong> of care - what is it? NEJM 335: 891-4, 1996191. American Pain Society <strong>Quality</strong> of Care Committee. <strong>Quality</strong> improvement guidelines <strong>for</strong> the treatment of acute pain <strong>and</strong>cancer pain. JAMA 274: 1874-1880, 1995192. Guadagnoli E, Cleary PD. How consistent is patient-reported preadmission health status when collected duringhospitalisation <strong>and</strong> posthospitalisation. Medical Care 33: 106-112, 1995 *193. Ayonin JZ, Guadagnoli E, Cleary PD. Physical <strong>and</strong> psychosocial functioning of women <strong>and</strong> men after coronary artery bypassgrafting. JAMA 274: 1767-1770, 1995 *194. LaCombe MA. What is it patients want? American Journal of Medicine 99: 558-589, 1995195. Donabedian A. Explorations in quality assessment <strong>and</strong> monitoring. Vol 3. The methods <strong>and</strong> findings of qualityassessment <strong>and</strong> monitoring: an illustrated analysis. Ann Arbor, Mich: Health Administration Press, 1995196. Iezzoni LI, Restuccia JD, Shwartz M et al. The utility of severity of illness in<strong>for</strong>mation in assessing the quality of hospitalcare: the role of the clinical trajectory. Medical Care 30: 438-444, 1992197. Weingarten SR et al. A study of patient satisfaction <strong>and</strong> adherence to preventive care practice guidelines. AmericanJournal of Medicine 99: 590-596, 1995198. Cleary PD, McNeil BJ. Patient satisfaction as an indicator of quality of care. Inquiry 25: 25-36, 1988 *199. McFadden ER et al. Protocol therapy <strong>for</strong> acute asthma: Therapeutic benefits <strong>and</strong> cost savings. American Journal ofMedicine 99: 651-661, 1995200. Cleary PD, McNeil BJ. Patient satisfaction as an indicator of quality of care. Inquiry 25: 25-36, 1988201. Jennifer Daley, Lisa Iezzoni <strong>and</strong> Shukri Khuri. Complication rates as a measure of <strong>Quality</strong> of Care (letter) JAMA 274:1674-1675, 1995. Comment on Silber JH et al. Evaluation of the complication rate as a measure of quality of care. JAMA274: 317-323, 1995202. Gottlieb TW. <strong>Quality</strong> assurance in a long term care facility. <strong>Quality</strong> Review Bulletin 10: 51-54, 1984203. Emanuel EJ <strong>and</strong> Emanuel LL. What is accountability in healthcare? Annals of Internal Medicine 124: 229-239 1996 <strong>and</strong>Emanuel LL. A professional response to dem<strong>and</strong>s <strong>for</strong> accountability. Practical recommendations regarding ethical aspectsof patient care. Annals of Internal Medicine 124: 240-249, 1996204. Bion JF et al. Validation of a prognostic score in critically ill patients undergoing transport. BMJ 291: 432-434, 1985205. Iezzoni LI. Severity of illness measures. Comments <strong>and</strong> Caveats. Medical Care 28: 757-761, 1990 *206. <strong>Quality</strong> of life bibliography <strong>and</strong> indexes. Medical Care (Suppl) 28: 12: DS1-DS77 *207. Patient judgements of hospital quality: Report of a pilot study. Meicald Care (Suppl) 28: 9: S1-S56, 1990208. Thomas JW, Holloway JJ <strong>and</strong> Guire KE. Validating risk-adjusted mortality as an indicator <strong>for</strong> quality of care Inquiry 30(1): 6-22, Spring 1993209. Rantz M. <strong>Quality</strong> measurement in nursing: Where are we now? Journal of Nursing Care <strong>Quality</strong> 9 (2): 1-7, 1995 *210. Ar<strong>for</strong>d P <strong>and</strong> Allred C. Value = <strong>Quality</strong> + Cost. Journal of Nursing Administration. 25 (9): 64-69, 1995211. Griffiths P. Progress in measuring nursing outcomes. Journal of Advanced Nursing 21: 1092-1100, 1995

65212. Ryan M et al. Measuring patient satisfaction: A case study. Journal of Nursing Care <strong>Quality</strong> 9 (2): 44-54, 1985213. Redfern S, et al The reliability <strong>and</strong> validity of quality assessment measures in nursing. Journal of Clinical Nursing 1: 47-51, 1992214. Brooten D <strong>and</strong> Naylor M. Nurses' effects on changing patient outcomes. Journal of Nursing Scholarship 27 (2): 95-99,1995215. Slater P. The past <strong>and</strong> future of nursing research in Australia. The Australian Journal of Advanced Nursing 1 (2): 43-54,1984216. Nadzam P. The Agenda <strong>for</strong> Change: Update on indicator development <strong>and</strong> possible implications <strong>for</strong> the nursing profession.Journal of Nursing Care <strong>Quality</strong> 5 (2): 18-22, 1991217. Rudolph B <strong>and</strong> Hill C. The components of hospital quality: A nursing perspective. Journal of Nursing Care <strong>Quality</strong> 9 (1):57-65, 1994218. Higgins M, et al. Assessing the outcomes of nursing care. Journal of Advanced Nursing 17: 561-568, 1992219. Valentine K. Comprehensive assessment of caring <strong>and</strong> its relationship to outcome measures. Journal of Nursing <strong>Quality</strong>Assurance 5 (2): 59-68, 1991220. Lang D <strong>and</strong> Dorman Marek K. The policy <strong>and</strong> politics of patient outcomes. Journal of Nursing <strong>Quality</strong> Assurance 5 (2): 7-12, 1991221. Stewart B <strong>and</strong> Archbold P. Focus on psychometrics - nursing intervention studies require outcome measures that aresensitive to change: Part 2. Research in Nursing <strong>and</strong> Health 16: 77-81, 1983222. Stewart B <strong>and</strong> Archbold P. Focus on psychometrics - nursing intervention studies require outcome measures that areSensitive to Change: Part 1. Research in Nursing <strong>and</strong> Health 15: 477-481, 1992223. Hegyvary S. Issues in outcomes research. Journal of Nursing <strong>Quality</strong> Assurance 5 (2): 1-6, 1991224. Schroeder P. From the Editor. Journal of Nursing <strong>Quality</strong> Assurance 5 (1): vi, 1991225. Schroeder P. From the Editor. Journal of Nursing <strong>Quality</strong> Assurance 5 (2): viii, 1991226. Schroeder P. From the Editor. Journal of Nursing <strong>Quality</strong> Assurance 6 (3): viii, 1992227. Schroeder P. From the Editor. Journal of Nursing <strong>Quality</strong> Assurance 8 (2): vii, 1994228. Schroeder P. From the Editor. Journal of Nursing <strong>Quality</strong> Assurance 9 (3): viii, 1995229. Schroeder P. From the Editor. Journal of Nursing <strong>Quality</strong> Assurance10 (1): viii, 1995230. Bond S <strong>and</strong> Thomas L. Measuring Patients' Satisfaction with Nursing Care. Journal of Advanced Nursing 17: 52-63, 1992231. Libreri F. An <strong>Acute</strong> Pain Service: A <strong>Quality</strong> Assurance Survey of Nurses <strong>and</strong> Doctors. Australian Journal of AdvancedNursing 12 (4): 33-38, 1995232. Pratt R, et al. The Effects of all RN <strong>and</strong> RN-EN Staffing on the <strong>Quality</strong> <strong>and</strong> Cost of Patient Care. Australian Journal ofAdvanced Nursing 10 (3): 27-38, 1993233. Hanson R et al. Patient Dissatisfaction in a Paediatric Accident <strong>and</strong> Emergency Department. Journal of <strong>Quality</strong> ClinicalPractice 14: 137-143, 1994234. Directory of Nursing Research in Australia 1991-1993, Royal College of Nursing Australia, Seventh Edition, Print MintSouth Melb, Vic, 1994235. Roberts K. A snapshot of australian nursing scholarship 1993-1994. Collegian 13 (1): 4-10, 1996236. Senior S, et al. Continuous quality improvement in a day ward. Journal of <strong>Quality</strong> in Clinical Practice 15: 177-181, 1995237. Webster J. <strong>Quality</strong> assurance: Measures of maternal knowledge <strong>and</strong> satisfaction. Australian Clinical Review 13: 119-126,1993

66238. Donnells M, et al . Improving care through a medication administration process action team. Journal of Nursing Care<strong>Quality</strong> 9 (3): 38-44, 1995239. Crowley B, et al . Clinical <strong>Indicators</strong>: A tool <strong>for</strong> improving pain management documentation. Journal of Nursing Care<strong>Quality</strong> 6 (1): 40-46, 1991240. Bechtel G, et al. A continuous quality improvement approach to medication administration. Journal of Nursing Care<strong>Quality</strong> 7 (3): 28-34, 1993241. Ruckstuhl M, et al. Patient falls: An outcome indicator. Journal of Nursing Care <strong>Quality</strong> 6 (1): 25-29, 1991242. Davis B <strong>and</strong> Bush H. Developing effective measurement tools: A case study of the Consumer Emergency Care SatisfactionScale. Journal of Nursing Care <strong>Quality</strong> 9 (2): 26-35, 1995243. Cooper M. Can a zero defects philosophy be applied to drug errors? Journal of Advanced Nursing 21: 487-491, 1995244. Andersen Y. European nurses take lead in quality assurance. International Nursing Review 41 (1): 13-16, 1994245. Pelkonen M. Nursing quality assurance in Finl<strong>and</strong>. International Nursing Review 41 (1): 23-26, 1994246. Giebing H. Nursing quality management in the Netherl<strong>and</strong>s. International Nursing Review 41 (1): 17-21, 1994247. Guva M Collaboration: The pivotal point <strong>for</strong> quality patient outcome. Journal of Nursing Care <strong>Quality</strong> 9 (3): 53-58, 1995248. Windle P. Comit - improving patient outcomes. Nursing Management 26 (9): 64 AA, 64 DD, 64 FF, 64 HH, 1995249. Mark B <strong>and</strong> Burleson D. Measurement of patient outcomes - Data availability <strong>and</strong> consistency across hospitals. Journal ofNursing Administration 25 (4): 52-59, 1995250. Davis S <strong>and</strong> Adams-Greenly M. Integrating patient satisfaction with a quality improvement program. Journal of NursingAdministration 24 (12): 28-31, 1994251. Pearson A. Determining quality in a unit where nursing is the primary intervention. Journal of Advanced Nursing 14: 269-273, 1989252. Abramowitz S, et al. Analysing patient satisfaction: A multi-analytic approach. <strong>Quality</strong> Review Bulletin 122-130, 1987253. Stratmann W, et al. Patient satisfaction surveys <strong>and</strong> multicollinearity. <strong>Quality</strong> Management in Health Care 1-12, 1994254. Rosenthal M, et al. Developing methods <strong>and</strong> research instruments <strong>for</strong> assessing physical <strong>and</strong> psychosocial outcomes ofpatient care as a function of nursing staff mix. The Australian Journal of Advanced Nursing 8 (3): 34-41, 1991255. Donabedian A. <strong>Quality</strong> assessment <strong>and</strong> assurance: Unity of purpose, diversity of means. Inquiry 25: 173-192, 1988256. Marek K. <strong>Outcome</strong> measurement in nursing. Journal of Nursing <strong>Quality</strong> Assurance 4 (1): 1-9, 1989257. Lower M <strong>and</strong> Burton S. Measuring the impact of nursing interventions - The challenges of the 1990s. Journal of Nursing<strong>Quality</strong> Assurance 4 (1): 27-34, 1989258. Bond S <strong>and</strong> Thomas. Issues in measuring outcomes of nursing. Journal of Advanced Nursing 16: 1492-1502, 1991259. Bieniek J. Medication administration program - Caulfield General Medical Centre, Unpublished Paper, 1993260. Bieniek J. Establishing the relationship between possible falls risk factors <strong>and</strong> the incidents of patient falls Unpublishedpaper, 1993261. Rosenthal M, et al. Developing research methods <strong>and</strong> instruments <strong>for</strong> assessing patient outcomes, 1991262. Goldman R. The reliability of peer assessments of quality of care. JAMA 276 (7): 958-960, 1992 *263. Morgan CJ, Branthwaite MA. Severity scoring in intensive care. BMJ 292: 1546, 14 June 1986264. Edwards RT <strong>and</strong> Barlow J. Rationing health care by waiting list: An extra-welfarist perspective. Discussion paper 114.Centre <strong>for</strong> Health Economics, University of York, UK, 1994

67265. Money RC, Fine DJ <strong>and</strong> Loree SW. Comparing the allocative efficiencies of hospitals. International Journal ofManagement Science 18 (1): 71-83, 1990266. Richardson J, Segal L, Carter R, Cat<strong>for</strong>d J, Galbally R <strong>and</strong> Johnson S. Prioritising <strong>and</strong> financing health promotion inAustralia. Research Report 4. Centre <strong>for</strong> Health Program Evaluation, Melbourne, 1995267. Sherman HD. Hospital efficiency measurement <strong>and</strong> evaluation: Empirical test of a new technique. Medical Care 22 (10):922-38, 1984268. Grosskopf S <strong>and</strong> Valdmanis V. Measuring hospital per<strong>for</strong>mance: A non-parametric approach. Journal of HealthEconomics 6: 89-107, 1987269. Ehreth J. The development <strong>and</strong> evaluation of hospital per<strong>for</strong>mance measures <strong>for</strong> policy analysis. Medical Care 32 (6): 568-87, 1994270. Hao L <strong>and</strong> Pegals CC. Evaluating relative efficiencies of Veterans Affairs Medical Centres using data envelopment, ratio<strong>and</strong> multiple regression analysis. Journal of Medical Systems 18 (2): 55-67, 1994271. Register CA <strong>and</strong> Bruning ER. Profit <strong>and</strong> technical efficiency in the production of hospital care. Southern EconomicJournal 53: 899-913, 1986272. Salinas-Jimenez J <strong>and</strong> Smith P. Data Envelopment Analysis applied to quality in primary health care. Discussion Paper124. Centre <strong>for</strong> Health Economics. York University, UK, 1994273. Mooney G, Gerard K, Donaldson C <strong>and</strong> Farrar L. Priority setting in purchasing: Some practical guidelines. ResearchPaper No.6. The Health Economics Research Unit, University of Aberdeen, 1992274. Ozcan YA, Luke RD <strong>and</strong> Haksever C. Ownership <strong>and</strong> technical efficiency across hospital types. Medical Care, 30 (9):781-794, 1992275. Ozcan YA <strong>and</strong> Luke D. A national study of efficiency of hospitals in urban markets. Health <strong>Services</strong> Research, 27 (6):719-739, 1993276. Ozcan YA <strong>and</strong> Bannick RR. Trends in Department of Defence hospital efficiency. Journal of Medical Systems, 18 (2): 69-83, 1994277. Grosskopf S <strong>and</strong> Valdmanis V. Evaluating hospital per<strong>for</strong>mance with case-mix adjusted outputs. Medical Care 31 (6):525-532, 1993278. Finkler MD <strong>and</strong> Wirtschafter DD. Cost effectiveness <strong>and</strong> data envelopment analysis. Health Care Management Review 18(3): 81-88, 1993 *279. Rosko MD. Measuring technical efficiency in healthcare organisations. Journal of Medical Systems 14 (5): 307-322, 1990280. Ran S, Lowe CN <strong>and</strong> Harder G. A quantitative approach to quality improvement <strong>and</strong> resource allocation. Journal of<strong>Quality</strong> in Clinical Practice, 15: 11-16, 1995281. Daly PM. Surgical waiting lists in Victorian hospitals. MJA 155, 19 August 1991282. Eddy DM. Clinical decision making: From theory to practice, "Rationing resources while improving quality". JAMA 272(10), 4 September 1994 *283. Managed Health Care Professionals present strategies <strong>for</strong> managing cost <strong>and</strong> quality. (Meeting Update). <strong>Quality</strong> ReviewBulletin, November 1991284. Pella J, Cook A. Neonates <strong>and</strong> DRGs: Resolving one weakness. Australian Casemix Bulletin, 1 (2), June 1989285. Kelly, K, et al. The Medical <strong>Outcome</strong>s Study: A nursing perspective. Journal of Professional Nursing 10(4): 209-216,1994286. Gillette, B., Jenko, M. Major clinical functions: A unifying framework <strong>for</strong> measuring outcomes. Journal Nursing Care<strong>Quality</strong> 6 (1): 20-24, 1991287. Eriksen, L. Patient satisfaction: An indicator of nursing care quality. Nursing Management, pp 31-35, 1987

68288. Megivern, K., Halm, M. Jones, G. Measuring patient satisfaction as an outcome of nursing care. Journal of Nursing Care<strong>Quality</strong> 6 (4): 9-24,1992289. Ambler Peters D. <strong>Outcome</strong>s: The mainstay of a framework <strong>for</strong> quality care. Journal of Nursing Care <strong>Quality</strong> 10 (1): 61-69,1995290. Podgorny K. Developing nursing focused quality indicators: A professional challenge. Journal of Nursing Care <strong>Quality</strong> 6(1) 47-52, 1991291. CATARACT SURGERY Patient-Reported Data on appropriateness <strong>and</strong> outcomesUnited States General Accounting Office (Report to Congressional Requesters), April 1993 *292. BONE MARROW TRANSPLANTATION International comparisons of availability <strong>and</strong> appropriateness of use. United States GeneralAccounting Office (Report to the Chairman, Committee on Labor <strong>and</strong> Human Resources, U.S. Senate), March 1994293. HEALTH CARE REFORM "Report Cards" are useful but significant issues need to be addressed. United States General AccountingOffice (Report to the Chairman, Committee on Labor <strong>and</strong> Human Resources, U.S. Senate), September 1994 **294. MEDICARE Increased HMO oversight could improve quality <strong>and</strong> access to care. United State General Accounting Office (Report tothe Special Committee on Aging, U.S. Senate), August 1995295. HEALTH CARE Employers <strong>and</strong> individual consumers want additional in<strong>for</strong>mation on quality. United States General AccountingOffice (Report to the Ranking Minority Member, Committee on Labor <strong>and</strong> Human Resources, U. S. Senate), September 1995296. Goldberg HI <strong>and</strong> Cummings MA (Co-Editors). Conducting medical effectiveness research: A report from the Inter-PORT work groups.Medical Care (Suppl) 32 (7) , July 1994 *297. Neuhauser D (Editor). <strong>Quality</strong> <strong>and</strong> accountability in practice: measuring, managing, <strong>and</strong> making it all work in a re<strong>for</strong>med health caresystem. Medical Care (Suppl) 33 (1), 1995298. Gelb D, Safran D, Rogers W, Lieberman W, Kosinski M. The primary care assessment survey: A tool <strong>for</strong> improvement, overview of apractice-based per<strong>for</strong>mance assessment. The Health Institute New Engl<strong>and</strong> Medical Center, Boston MA, November 1995299. Kazis LE et al. A modified version of a short <strong>for</strong>m health survey <strong>for</strong> assessing health record quality of life in veterans: The VeteransHealth Study. Bed<strong>for</strong>d Health <strong>Services</strong> Research <strong>and</strong> Development Field Program, Department of Veterans Affairs, Bed<strong>for</strong>d MA,January 1996300. Nerenz DR, Zajac BM, Rosman HS <strong>and</strong> Zuckerman H. Consortium Research on <strong>Indicators</strong> of System Per<strong>for</strong>mance (CRISP) Report ofPhase I. Center <strong>for</strong> Health System Studies, Henry Ford Health System, Detroit, Michigan, October 23, 1992301. Nerenz DR, Zajac BM, Rosman HS <strong>and</strong> Zuckerman H. Consortium Research on <strong>Indicators</strong> of System Per<strong>for</strong>mance (CRISP) Phase IIProposal. Centre <strong>for</strong> Health System Studies, Henry Ford Health System, Detroit, Michigan, November 20, 1992302. Nerenz DR et al. Consortium Research on <strong>Indicators</strong> of System Per<strong>for</strong>mance (CRISP) Phase III Proposal. Centre <strong>for</strong> Health SystemStudies, Henry Ford Health System, Detroit, Michigan, December 1994303. Neslund-Dudas C, (Project Coordinator). Consortium Research on <strong>Indicators</strong> of System Per<strong>for</strong>mance (CRISP), Report Sampler.Centre <strong>for</strong> Health System Studies, Henry Ford Health System, Detroit, Michigan, May 1995304. Kaz<strong>and</strong>jian VH, Lawthers J, Cernak CM <strong>and</strong> Pipesh FC. Relating outcomes to processes of care: The Maryl<strong>and</strong> Hospital Association's<strong>Quality</strong> Indicator Project (QI Project). The Journal of the Joint Commission on Accreditation of <strong>Healthcare</strong> Organisations pp 530 -538, November 1993305. Medical affairs issues report, The Group Health Association of America (GHAA) <strong>and</strong> the American Managed Care <strong>and</strong> ReviewAssociation (AMCRA). GHAA/AMCRA Washington DC, November/December 1995306. Primary care assessment survey. The Health Institute, New Engl<strong>and</strong> Medical Center, Boston MA, January 1996307. Measuring quality: Guide <strong>for</strong> presenting HEDIS 2.0 to employers, Group Health Association of America, Inc., 1995 *308. Health care report Cards: Profiles of all major report cards, per<strong>for</strong>mance reports, shopping guides, <strong>and</strong> consumer satisfaction surveys(Special Report), Accountability News <strong>for</strong> Health Care Managers, Atlantic In<strong>for</strong>mation <strong>Services</strong>, Inc. Washington DC, 1995309. Health care costs in Massachusetts, Executive Summary, Massachusetts Rate Setting Commission, Boston MA, December 1992

69310. Griswold PR, Chairman <strong>and</strong> Freedman LI, Commissioner. Health care quality <strong>and</strong> the importance of outcomes measurement . AReport of the Massachusetts Rate Setting Commission to the Joint Committee on Health Care, Boston MA, December 1993 *311. Griswold PR, Chairman, Freedman LI, Commissioner <strong>and</strong> R<strong>and</strong>le ML, Commissioner. Preventable hospitalization in Massachusetts.Massachusetts Rate Setting Commission, Boston MA, January 1994312. Weinstein BE, Chairman, Freedman LI, Commissioner <strong>and</strong> R<strong>and</strong>le ML, Commissioner. Improving primary care: Using preventablehospitalization as an approach. Massachusetts Rate Setting Commission, Boston MA, April 1995313. Weinstein BE, Chairman, Freedman LI, Commissioner <strong>and</strong> R<strong>and</strong>le ML, Commissioner. Monitoring the acute care hospital industry.Massachusetts Rate Setting Commission, Boston MA, May 1995314. Weinstein BE, Chairman, Freedman LI, Commissioner <strong>and</strong> R<strong>and</strong>le MI. Commissioner. Comparing hospital costs: Continuing thedialogue <strong>for</strong> improvement. Massachusetts Rate Setting Commission, Boston MA., July 1995315. Abbott L <strong>and</strong> James B. Involving physicians in health outcomes assessment. Medical <strong>Outcome</strong>s Trust Bulletin 4 (2): 1, March 1996316. Damiano AM. The Sickness Impact Profile: Part 1. Medical <strong>Outcome</strong>s Trust Bulletin 4 (2): 2, March 1996317. Greene RJ. Status of the Patient <strong>Outcome</strong>s Research Teams (PORTs). Medical <strong>Outcome</strong>s Trust Bulletin 4 (2): 3, March 1996318. Reverby S. Stealing the golden eggs: Ernest Amory Codman <strong>and</strong> the science <strong>and</strong> management of medicine. Bulletin of the History ofMedicine 55: 156-171, 1981319. DesHarnais SI <strong>and</strong> Simpson KN. Indices <strong>for</strong> monitoring hospital outcomes in developed countries. Health Policy 21: 1-15, 1992320. Underst<strong>and</strong>ing <strong>and</strong> choosing clinical per<strong>for</strong>mance measures <strong>for</strong> quality improvement: Development of a typology, Report <strong>for</strong> theDepartment of Health <strong>and</strong> Human <strong>Services</strong>, Public Health Service, AHCPR, Rockville, MD. Centre <strong>for</strong> Health Policy Studies,Columbia, MD, 31 January, 1995 **321. Development of a typology of clinical per<strong>for</strong>mance measures <strong>for</strong> quality improvement. Attachments. Center <strong>for</strong> <strong>Quality</strong> of CareResearch <strong>and</strong> Education, Harvard School of Public Health, Boston MA <strong>and</strong> Center <strong>for</strong> Health Policy Studies, Columbia, MD, 12August 1994 **322. Patients Charter Annual Report 1994 - 1995, Bed<strong>for</strong>dshire Health Consumer Affairs Department, Bed<strong>for</strong>dshire UK323. Marek KD. <strong>Outcome</strong> measurement in nursing. Journal of Nursing <strong>Quality</strong> Assurance 4 (1): 1- 9, 1989324. The <strong>Quality</strong> Indicator Study Group. An approach to the evaluation of quality indicators of the outcome of care in hospitalized patients,with a focus on nosocomial infection indicators. Infection Control <strong>and</strong> Hospital Epidemiology. 16 (5): 308 -316, May 1995 **325. Lavis JN <strong>and</strong> Anderson GM. Appropriateness in health care delivery: Definitions, measurement <strong>and</strong> policy implications. CanadianMedical Association Journal 154 (3): 321- 328, February 1996326. Fahey PP <strong>and</strong> Gibberd RW. Monitoring post-operative pulmonary embolisms in Australia. Variation between hospitals. InternationalJournal <strong>for</strong> <strong>Quality</strong> in <strong>Healthcare</strong> [In Press]327. Rontal R et al. Applications <strong>for</strong> risk-adjusted outcome measures. <strong>Quality</strong> Assurance in Health Care 13 (4): 283 - 292, 1991328. Fahey P. Clinical indicators, casemix & quality: Issues via example. NSW Health <strong>Services</strong> Research Group. Department of Statistics,University of Newcastle, Australia, May 1994329. <strong>Quality</strong> of Measurement or <strong>Quality</strong> of Medicine? (Editorial), JAMA 273 (19): 1537 - 1538, 17 May 1995 *330. Variation in DRG-based pulmonary embolism rates between NSW acute care hospitals, Paper presented at The Sixth NationalCasemix Conference, August 1994 Hobart, Tasmania. Paul Fahey331. Defining per<strong>for</strong>mance of organisations. Joint Commission Journal on <strong>Quality</strong> Improvement 19 (7): 215 - 221, July 1993332. Hopkins A et al, What do we mean by appropriate health care? <strong>Quality</strong> in Health Care 2: 117 - 123, 1993333. Health Policy Newsletter. Jefferson Medical College, Thomas Jefferson University, January 1996334. Kaz<strong>and</strong>jian VA <strong>and</strong> Sternberg EL. <strong>Indicators</strong> of per<strong>for</strong>mance or the search <strong>for</strong> the best pointer dog. The Epidemiology of <strong>Quality</strong>, 1995

70335. <strong>Quality</strong> Indicator Project Users Guide. Maryl<strong>and</strong> Hospital Association, Lutherville, MD, 1995336. Goldman RL <strong>and</strong> Thomas TL. Using mortality rates as a screening tool: The experience of the Department of Veterans Affairs. JointCommission Journal on <strong>Quality</strong> Improvement 20 (9): 511 - 521, 1994337. Nerenz DR <strong>and</strong> Zajac BM. <strong>Indicators</strong> of per<strong>for</strong>mance <strong>for</strong> vertically integrated health systems (Ray Woodham visiting fellowshipprogram project summary report). Center <strong>for</strong> Health System Studies, Henry Ford Health System, 1994 *338. Cullen DJ et al. The incident reporting system does not detect adverse drug events: A problem <strong>for</strong> quality improvement. JointCommission Journal on <strong>Quality</strong> Improvement 21 (10): 541 -552, October 1995339. Kaz<strong>and</strong>jian VA. The 1996 Medical <strong>Outcome</strong>s & Guidelines Sourcebook, A Progress Report <strong>and</strong> Resource Guide on Medical<strong>Outcome</strong>s Research <strong>and</strong> Practice Guidelines: Developments, Data, <strong>and</strong> Documentation. Maryl<strong>and</strong> Hospital Association, LuthervilleMD340. Kaz<strong>and</strong>jian V, Wood P <strong>and</strong> Lawthers J. Balancing science <strong>and</strong> practice in indicator development: The Maryl<strong>and</strong> Hospital Association<strong>Quality</strong> Indicator (QI) Project. International Journal <strong>for</strong> <strong>Quality</strong> in Health Care 7 (1): 39 - 46, 1995341. Measuring the outcomes of medical care. Papers based on a conference held in September 1989 organised by the Royal College ofPhysicians <strong>and</strong> the King's Fund Centre <strong>for</strong> Health <strong>Services</strong> Development. Edited by Anthony Hopkins <strong>and</strong> David Costain, AntonyRowe Ltd., Wiltshire.342. Romano PS et al. The Cali<strong>for</strong>nia Hospital <strong>Outcome</strong>s Project: Using administrative data to compare hospital per<strong>for</strong>mance. JointCommission Journal on <strong>Quality</strong> Improvement 21 (12): 668 - 682, December 1995343. Health <strong>Outcome</strong>s Bulletin, Australian Health <strong>Outcome</strong>s Clearing House (AHOCH), Australian Institute of Health <strong>and</strong> Welfare(AIHW). Number 4 Summer 1995344. Victorian Department of Health <strong>and</strong> Community <strong>Services</strong>. A New Framework <strong>for</strong> <strong>Quality</strong> in Victoria's Public Hospitals, Final Report,Volume 1, November 1995345. Drummond MF. Principles of Economic Appraisal in Health Care, Ox<strong>for</strong>d Medical <strong>Services</strong>, 1988346. Drummond MF, Stoddart GL <strong>and</strong> Torrance GW. Methods <strong>for</strong> the Economic Evaluation of Health Care Programs, Ox<strong>for</strong>d MedicalPublications, 1988347. Donaldson C <strong>and</strong> Magnussen J. Economic Framework <strong>for</strong> Allocative Efficiency in the Health Sector. The Australian EconomicReview: 89-98, 1994348. Segal L <strong>and</strong> Richardson J. Efficiency in resource allocation. National Centre <strong>for</strong> Health Program Evaluation, Working Paper No. 34,March 1994349. Scotton RB <strong>and</strong> Owens HJ. Case payments in Australian Hospitals: issues <strong>and</strong> options. Public Sector Management Institute, MonashUniversity, August 1990350. Mooney G, Gerard K, Donaldson C <strong>and</strong> Farrar L. Priority setting in purchasing: some practical guidelines, Research Paper 6, HealthEconomics Research Unit, University of Aberdeen, UK, 1992351. National Health Ministers’ Benchmarking Working Group. First national report on health sector per<strong>for</strong>mance indicators: publichospitals - the state of play, Australian Institute of Health <strong>and</strong> Welfare, Canberra, 1996 *352. Australian Institute of Health <strong>and</strong> Welfare (AIHW). National Health Data Dictionary, Version 4.0, Canberra, 1995353. Ehreth J. The development <strong>and</strong> evaluation of hospital per<strong>for</strong>mance measures <strong>for</strong> policy analysis. Medical Care 32 (6 Suppl): s68-s87,1994354. Rosko MD. Measuring technical efficiency in healthcare organisations. Journal of Medical Systems 14 (5): 307-322, October 1990355. Metropolitan Hospitals Planning Board Victoria. Developing Melbourne’s hospital network. Phase 1 Report, 1995356. Metropolitan Hospitals Planning Board Victoria 1995. Criteria to assess the per<strong>for</strong>mance of networks. Phase 2 Report, 1995357. Victorian Department of Health <strong>and</strong> Community <strong>Services</strong>. A new framework <strong>for</strong> quality in Victoria’s public hospitals. Final report.Volume 1, 1995

71358. Victorian Department of Health <strong>and</strong> Community <strong>Services</strong>. A new framework <strong>for</strong> quality in Victoria’s public hospitals. Final report.Volume 2, 1995359. Victorian Department of Health <strong>and</strong> Community <strong>Services</strong>. Hospital service report, December 1995360. NSW Health. Annual report 1994-95, 1995361. NSW Health. Caring <strong>for</strong> health - equity, efficiency <strong>and</strong> quality. NSW Government’s Economic Statement <strong>for</strong> Health, 1995362. Woolh<strong>and</strong>ler S <strong>and</strong> Himmelstein DU. The deteriorating administrative efficiency of the US health care system. NEJM 324: 1253-8,1991363. Michael R <strong>and</strong> Piper K. Flinders Medical Centre non-inpatient costing study. Australian Health Review. 14 (2): 127-36, 1991364. Lagaida R <strong>and</strong> Hindle D. A casemix classification system <strong>for</strong> hospital-based ambulatory services. A report to the National AmbulatoryCasemix Project. State Health Publication No.IC 92-89. New South Wales Department of Health, 1992.365. Keith AR, Pirkus JE, et al. Delivery of primary care in hospital <strong>and</strong> community settings in Australia. <strong>Quality</strong> Assurance in HealthCare 5 (2): 131-41, 1993366. Jackson T, Sevil P, Tate R <strong>and</strong> Collard K. The development of relative resource weights <strong>for</strong> non-admitted patients. National Centre<strong>for</strong> Health Program Evaluation, Monash University, May 1991367. The Australian Council on <strong>Healthcare</strong> St<strong>and</strong>ards (ACHS) Care Evaluation Program. Hospital-wide medical indicators: a users’manual. ACHS Sydney, July 1991 *368. Zuckerman S, Hadley J <strong>and</strong> Iezzoni L. Measuring hospital efficiency with frontier cost functions. Journal of Health Economics 13 (3):255-280, 1994 *369. Vitaliano DF <strong>and</strong> Toren M. Cost <strong>and</strong> efficiency in nursing homes: a stochastic frontier approach. Journal of Health Economics 13 (3):281-300, 1994370. Kooreman P. Nursing home care in The Netherl<strong>and</strong>s: a nonparametric efficiency analysis. Journal of Health Economics 13 (3): 301-316, 1994371. Newhouse JP. Frontier estimation: How useful a tool <strong>for</strong> health economics. Journal of Health Economics 13 (3): 323-328,1994 *372. A Dor. Non-minimum cost functions <strong>and</strong> the stochastic frontier: on applications to health care providers. Journal of HealthEconomics 13 (3): 329-334, 1994373. Hadley J <strong>and</strong> Zuckerman S. The role of efficiency measurement in hospital rate setting. Journal of Health Economics 13 (3):335-340, 1994374. Vitaliano DF <strong>and</strong> Toren M. Frontier analysis: a reply to Skinner, Dor <strong>and</strong> Newhouse. Journal of Health Economics 13 (3):341-344, 1994375. Kooreman P. Data envelopment analysis <strong>and</strong> parametric frontier estimation: complementary tools. Journal of HealthEconomics 13 (3): 345-346, 1994376. Eckerman S. Data envelopment analysis. Australian Institute of Health <strong>and</strong> Welfare, unpublished report obtained by request377. Charnes A, Cooper W <strong>and</strong> Rhodes E. Measuring the efficiency of decision making units. European Journal of OperationalResearch 2 (6): 429-444, 1978378. Salinas-Jimanez J <strong>and</strong> Smith P. Data envelopment analysis applied to quality in primary health care. Discussion Paper 124,Centre <strong>for</strong> Health Economics, York Univeristy, UK 1994379. Bessant A, Bessant W, Kennington J <strong>and</strong> Reagan B. An application of mathematical programming to assess productivity inthe Houston independent school district. Management Science 28: 1355-67, 1982380. Charnes A, Cooper WW <strong>and</strong> Rhodes E. Evaluating program per<strong>for</strong>mance <strong>and</strong> management efficiency: an application of dataenvelopment analysis to program follow through. Management Science 27: 668-97, 1981

72381. Lewin AY, Morey RC <strong>and</strong> Cook TJ. Evaluating administrative efficiency of courts. Omega 10 (4): 401-11, 1982382. Fare R <strong>and</strong> Primont D. Efficiency measures <strong>for</strong> multi-plant firms. Operation Research Letters 3: 257-60, 1984383. Banker RD. Estimating the most productive scale size using data envelopment analysis. European Journal of OperationsResearch 17: 35-44, 1984384. Fare R, Grosskopf S <strong>and</strong> Pasurka C. The effect of environmental regulations on the efficiency of electric utilities: 1969 versus1975. Applied Economics 21: 225-32, 1989385. Charnes A, Clarke CT, Cooper WW <strong>and</strong> Golany B. A developmental study of data envelopment analysis in measuring theefficiency of maintenance units in the US air <strong>for</strong>ce. Annals of Operations Research 2: 95-112, 1985386. Byrnes P, Fare R <strong>and</strong> Grosskopf S. Measuring productive efficiency: an application to Illinois Strip Mines. ManagementScience 30: 671-81, 1984387. Byres P <strong>and</strong> Fare R. Surface mining of coal: efficiency of US interior mines. Applied Economics 19: 1665-73, 1987388. Whiteman J <strong>and</strong> Pearson K. Benchmarking telecommunications using data envelopment analysis. Economic papers 12 (3):97-105, 1993389. Sherman HD. Hospital efficiency measurement <strong>and</strong> evaluation: empirical test of a new technique. Medical Care 22 (10): a22-a38, 1984390. Burgess JF <strong>and</strong> Wislon PW. Technical efficiency in Veterans Administration Hospitals in The Measurement of ProductiveEfficiency, eds HO Fried, CAK Lovell <strong>and</strong> SS Schmidt, Ox<strong>for</strong>d University Press, Ox<strong>for</strong>d 1993391. Grosskopf S <strong>and</strong> Valdmanis V. Evaluating hospital per<strong>for</strong>mance with casemix adjusted outputs. Medical Care 31 (6): 525-532, 1993392. Grosskopf S <strong>and</strong> Valdmanis V. Measuring hospital per<strong>for</strong>mance: a non-parametric approach. Journal of Health Economics6: 89-107, 1987393. Hao L <strong>and</strong> Pegals CC. Evaluating relative efficiencies of Veterans’ Affairs medical centres using data envelopment, ratio <strong>and</strong>multiple regression analysis. Journal of Medical Systems 18 (2): 55-67, 1994394. Morey RC, Fire DJ <strong>and</strong> Loree SW. Comparing the allocative efficiencies of hospitals. International Journal of ManagementScience 18 (1): 71-83, 1990395. Ozcan YA <strong>and</strong> Luke D. A national study of efficiency of hospitals in urban markets. Health <strong>Services</strong> Research 27 (6): 719-739, 1993396. Ozcan YA, Luke RD <strong>and</strong> Haksever C. Ownership <strong>and</strong> technical efficiency across hospital types. Medical Care 30 (9): 781-794, 1992397. Ozcan YA <strong>and</strong> Bannick RR. Trends in Department of Defence hospital efficiency. Journal of Medical Systems 18 (2): 69-83,1994398. Register CA <strong>and</strong> Bruning ER. Profit <strong>and</strong> technical efficiency in the production of hospital care. Southern Economic Journal535: 899-913, 1986399. Sexton TR, Leiken AM, Nolan AH, Liss L, Hogan A <strong>and</strong> Silkman RH. Evaluating managerial efficiency of VeteransAdministration Medical Centres using data envelopment analysis. Medical Care 27 (12): 1175-1188, 1989400. Valdmanis V. Sensitivity analysis <strong>for</strong> DEA models - an empirical analysis using public versus NFP hospitals. Journal ofPublic Economics 48: 185-205, 1992401. Finkler MD <strong>and</strong> Wirtschafter DD. Cost effectiveness <strong>and</strong> data envelopment analysis. Health Care Management Review 18(3): 81-88, 1993402. Granley JA <strong>and</strong> Cubbin JS. Public Sector Efficiency Measurement: Applications of Data Envelopment Analysis, Amsterdam,North Holl<strong>and</strong>, 1992403. Thrall RM. Classifications transitions under expansion of inputs <strong>and</strong> outputs in DEA. Managerial <strong>and</strong> Decision Economics10 (2): 159-162, 1990

73404. Rosenstein AH. Cost-effective health care: tools <strong>for</strong> analysis. Health Care Management Review 19 (2): 53-61, 1994405. Richardson J, Segal, Carter R, Cat<strong>for</strong>d J, Galbally R <strong>and</strong> Johnson S. Prioritising <strong>and</strong> financing health promotion in Australia,Research Report 4. Centre <strong>for</strong> Health Program Evaluation, Melbourne, 1995 *406. Mooney G. Program budgeting: an aid to planning <strong>and</strong> priority setting in healthcare. Effective Health Care 2 (2): 65-68,1994407. Eddy DM. What’s going on in Oregon. Journal of the American Medical Association 266 (3): 417-420, 1991 *408. Kaplan RM. Value judgement in the Oregan medicaid experiment. Medical Care 32 (10): 975-988, 1994 *409. Nord E. An alternative to QALYs: the saved young life equivalent (SAVE). BMJ 15 (2):124-134, 1992410. Hall J <strong>and</strong> Haas M. The rationing of health care: should Oregon be transported to Australia? Australian Journal of PublicHealth 16 (4): 340-349, 1993411. Street A <strong>and</strong> Richardson J. The value of health care: what can we learn from Oregon? Australian Health Review 15 (2): 124-134, 1992 *412. Scotton RB 1993. Restructuring the financing of health care. Paper presented <strong>for</strong> the Office of the Economic PlanningAdvisory Council Seminar Investing in Health Care - A Challenge <strong>for</strong> the Future, Canberra, October 1993413. Street A. Purchaser/provider <strong>and</strong> managed competition: importing chaos. National Centre <strong>for</strong> Health Program Evaluation.Working Paper No. 36. April 1994414. Commonwealth Department of Human <strong>Services</strong> <strong>and</strong> Health. Guidelines <strong>for</strong> the pharmaceutical industry on preparation ofsubmissions to the pharmaceutical benefits advisory committee: including major submissions involving economic analysis.Australian Government Publishing Service, Canberra. November 1995415. Carter R. A macro approach to economic appraisal in the health sector. Australian Economic Review 2nd quarter: 105-112,1994 *416. Drummond MF. Output measurement <strong>for</strong> resource allocation decisions in health care. Ox<strong>for</strong>d Review of Economic Policy 5(1): 59-74, 1993417. Morey RC, Fine DJ, Loree SW et al. The trade-off between hospital cost <strong>and</strong> quality of care: an exploratory empiricalanalysis. Medical Care 30 (8): 677-698, 1992418. Evans R. Canada: the real issues. Journal of Health Politics, Policy <strong>and</strong> Law 17 (4): 738-762, 1992419. Kenney G. How access to long-term care affects homehealth transfers. Journal of Health Politics, Policy <strong>and</strong> Law 18 (4):937-965, 1993420. Ford E, Cooper R. Racial/ethnic differences in health care utilisation of cardiovascular procedures. Health Service Research30 (1): 237-250, 1995421. Madhok R, Green S. Orthopaedic outpatient referral guidelines: experiences from an English health district. InternationalJournal <strong>for</strong> <strong>Quality</strong> in Health Care 6 (1): 73-76, 1994422. Wells K, Rogers W, Davis L et al. <strong>Quality</strong> of care <strong>for</strong> depressed elderly pre-post prospective payment system: differences inresponse across settings. Medical Care 32 (3): 257-276, 1994423. Arrington B, Haddock C. Who really profits from not-<strong>for</strong>-profit hospitals? Health <strong>Services</strong> Research 25 (2): 291-304, 1990424. Green J, Dale J. Primary care in accident <strong>and</strong> emergency <strong>and</strong> general practice: a comparison. Social Science <strong>and</strong> Medicine35 (8): 987-995, 1992425. Dranove D, White W <strong>and</strong> Wu L. Segmentation in local hospital markets. Medical Care 31 (1): 52-64, 1993426. Fitzgerald J, Freund D, Hughett B, et al. Influence of organisational components on the delivery of asthma care. MedicalCare 31 (3): MS61-MS73, 1993

74427. Bel<strong>and</strong> F, et al. Socio-spatial perspectives on the utilisation of emergency hospital services in two urban territories in Quebec.Social Science <strong>and</strong> Medicine 30 (1): 53-66, 1990428. de Marco P, Dain C, Lockwood T, et al. How valuable is feedback of in<strong>for</strong>mation on hospital referral patterns? BMJ 307:1465-1466, 1993429. Azeem Majeed F, et al. Monitoring <strong>and</strong> promoting equity in primary <strong>and</strong> secondary care. BMJ 308: 1426-1429, 1994430. Bailey J, Black M, Wilkin D. Specialist outreach clinics in general practice. BMJ 308: 1083-1086, 1994431. Smith T. Waiting times: monitoring the total post-referral wait. BMJ 309: 593-596, 1994432. Pollock A. Where should health services go: local authorities versus the NHS? BMJ 310: 1530-1539, 1995433. Every N, Larson E, Litwin P, et al. The association between on-site cardiac catheterisation facilities <strong>and</strong> the use of coronaryangiography after acute myocardial infarction. NEJM 329: 546-551, 1993434. Chirboga D et al. A community wide perspective of gender differences <strong>and</strong> temporal trends in the use of diagnostic <strong>and</strong>revascularisation procedures <strong>for</strong> acute myocardial infarction. American Journal Cardiology 71: 268-273, 1993435. Muus K, Luktke R, Gibbens B. Community perceptions of rural hospital closure. Journal of Community Health 20 (1): 65-73, 1995436. Mirvis D, et al. Variation in utilisation of cardiac procedures in the Department of Veterans Affairs health care system: effectof race. Journal Americal College Cardiology 24: 1297-304, 1994437. Shea S et al. Predisposing factors <strong>for</strong> severe uncontrolled hypertension in an inner city minority population. NEJM 327: 776-781, 1992438. Lloyd P, McCarthy W, Nolan J. The beds to the west scheme: the politics <strong>and</strong> outcomes of the 1982 NSW hospitalrationalisation program. MJA 153: 486-491, 1990439. Naylor CD, Levington C, Baigrie R. Adapting to waiting lists <strong>for</strong> coronary revascularisation: do Canadian specialists agree onwhich patients come first? Chest 101 (3): 715-722, 1992440. Hanson R, Clifton-Smith B, Fasher B. Patient dissatisfaction in a paediatric emergency department. Journal of <strong>Quality</strong> inClinical Practice 14: 137-143, 1994441. Gittelsohn A, Powe N. Small area variations in health care delivery in Maryl<strong>and</strong>. Health <strong>Services</strong> Research 30 (2): 295-317,1995442. McCell<strong>and</strong> A. In Fair Health? Equity <strong>and</strong> the Health System. National Health Strategy Background Paper No. 3.Commonwealth of Australia, 1991443. Harvey R. Making it Better: strategies <strong>for</strong> improving the effectiveness <strong>and</strong> quality of health services in Australia.Background paper No. 8. National Health Strategy. Commonwealth of Australia, 1991,444. Hospital services in Australia: access <strong>and</strong> financing. National Health Strategy Issues Paper No. 2. Commonwealth ofAustralia, 1991445. Health Department, Victoria. Equitable Regional Funding <strong>for</strong> <strong>Acute</strong> Hospitals: a discussion paper. Victorian Department ofHealth, 1986446. Mays L. Prioritising Elective Patients, AIHW Working paper 1994447. Mays L. <strong>and</strong> Gillett S. Uses of waiting list in<strong>for</strong>mation, AIHW 1994448. Gillett S, Katauskas E. Waiting lists - a look at the literature, AIHW 1993449. Gillett S. <strong>and</strong> Mays L. Interpreting waiting list in<strong>for</strong>mation, AIHW 1994450. Richardson B. Doctor supply utilisation <strong>and</strong> health outcomes: an economic analysis ASHSA University NSW School HealthAdministration, 1994

75451. Newton J <strong>and</strong> Goldacre M. How many patients are admitted in districts other than their own <strong>and</strong> why? Journal of PublicHealth Medicine 16 (2): 159-164, 1994452. Papadimitriou D, Mathur M <strong>and</strong> Hill D. A survey of rural road fatalities. Australian <strong>and</strong> New Zeal<strong>and</strong> Journal of Surgery64: 479-483, 1994453. Keife C et al. Is cost a barrier to screening mammography <strong>for</strong> low income women receiving medicare benefits? Archives ofInternal Medicine 154: 1217-1224, 1994454. Skolnick A. Are there too many US transplantation centres? JAMA 271: 1062-3, 1994455. Blustein J. High technology cardiac procedures - the impact of service availability on service use in New York State. JAMA270: 344-349, 1993456. Wright R. Community oriented primary care. JAMA 269: 2544-2547, 1993457. Boyle F, Bell D. Breast cancer treatment outside teaching hospitals. Australian <strong>and</strong> New Zeal<strong>and</strong> Journal of Surgery 22:309-310, 1994458. Yedidia M . The impact of social factors on the content of care. Archives of Internal Medicine 152: 595-600, 1992459. Del Mar C et al. Shared antenatal care in Brisbane. Australian <strong>and</strong> New Zeal<strong>and</strong> Journal of Obstetrics <strong>and</strong> Gynaecology31: 305, 1991460. Weissman J et al. Delayed access to health care: risk factors, reasons <strong>and</strong> consequences. Annals of Internal Medicine 114:325-31, 1991461. Kellerman A. Too sick to wait. JAMA 266: 1123-24, 1991462. Nesbitt T, et al. Access to obstetric care in rural areas : effect on birth outcomes. American Journal of Public Health 80: 814,1990463. Stewart J et al. Major complications of coronary arteriography: the place of cardiac surgery. British Heart Journal 63: 74-77, 1993464. Thakker Y et al. Peadiatric inpatient utilisation in a district general hospital. Archives of Disease in Childhood 70: 488-92,1994465. Brownell M, Roos N. Variation in length of stay as a measure of efficiency in Manitoba hospitals. Canadian MedicalAssociation Journal 152: 675, 1995 *466. French JA et al. Effect of in<strong>for</strong>mation about waiting lists on referral patients of general practitioners. British Journal ofGeneral Practice 40: 186-189, 1990467. Naylor CD et al. Waiting <strong>for</strong> coronary revascularisation in Toronto: 2 years’ experience with a regional referral office.Canadian Medical Association Journal 149: 955, 1993468. Mahon A et al. Factors that influence general practitioners’ choice of hospital when referring patients <strong>for</strong> elective hospital.British Journal of Practice 43: 272-6, 1993469. Nunez D. Inappropriate outpatient referrals increasing? British Journal of Clinical Practice 47: 73, 1993470. Ho E et al. Ontario patients’ acceptance of waiting times <strong>for</strong> knee replacements. Journal of Rheumatology 21: 2101, 1994*471. Baume P, Wolk J. Difficulties in admitting patients to metropolitan public hospitals: the view of medical practitioners. MJA163: 401, 1995472. Baume P. Waiting time <strong>for</strong> non-urgent specialist appointments. MJA 162: 648, 1995473. Duckett S. Casemix funding in Victoria; the first year. MJA 162: 650, 1995474. American Association of Family Physicians - Position statement on access to health care. American Family Physician 45:2797, 1992475. Bengston A et al. The epidemiology of a coronary waiting list. Journal of Internal Medicine 235: 263, 1994 *

76476. Roberts M et al. Time delays to lytic therapy <strong>and</strong> outcomes in 100 consecutive patients. European Heart Journal 15: 594-601, 1994477. Dos Santos L et al. Paediatric emergency department walkouts. Paediatric Emergency Care 10: 76, 1994478. Brown S et al. Management of minor head injuries in the accident <strong>and</strong> emergency department: the effect of an observationward. Journal of Accident <strong>and</strong> Emeregency Medicine 11: 144-148, 1994479. Tennyson Williams P. Generalist physicians in nonmetropolitan counties in Ohio. Archives of Family Medicine 3: 425-428,1994480. Allen D, Kamradt J. Relationship of infant mortality to the availability of obstetrical care in Indiana. Journal on FamilyPractice 33: 609-612, 1991481. Charlton J et al. Geographical variation in mortality from conditions amenable to medical intervention in Engl<strong>and</strong> <strong>and</strong> Wales.Lancet 691-694, 1983482. Parchman M. Primary care physicians <strong>and</strong> avoidable hospitalisations. Journal on Family Practice 39: 123-128, 1994483. Metropolitan Hospitals Planning Board. Criteria to assess the per<strong>for</strong>mance of networks 1995484. National Health Ministers’ Benchmarking Working Group. First National Report on Health Sector Per<strong>for</strong>mance <strong>Indicators</strong>AIHW, Commonwealth of Australia, February 1996485. Victorian Department of Health <strong>and</strong> Community <strong>Services</strong>. Hospital <strong>Services</strong> Report, October 1995486. Montalto M, Dunt D. Delivery of traditional hospital services to patients at home. MJA 159 (4): 263-265, 1993487. Monalto M. Using referral letters to measure quality <strong>and</strong> per<strong>for</strong>mance in general practice. Journal of <strong>Quality</strong> in ClinicalPractice 15: 45-50, 1995488. Montalto M . Hospital in the home units: early issues <strong>for</strong> GP involvement. Australian Family Physician 24 (5): 797-801,May 1995489. Pirkis J, Montalto M. General practitioners in hospitals. Australian Family Physician 24 (6): 1026-29, 1995490. Short Form 12 (SF-12). A users’ manual. Medical <strong>Outcome</strong>s Trust, Boston, 1996 *491. <strong>Outcome</strong>s <strong>and</strong> PORTs [Editorial]. Lancet 340: 1439, 1992 *492. Developing a patient measurement system <strong>for</strong> the future: An interview with Eugene C. Nelson, DSc, MPH. Joint Commission Journalon <strong>Quality</strong> Improvement 19: 368-373, 1993.493. David Nash on the future of health care. <strong>Outcome</strong>s Measurement & Management 2-3, Jan/Feb 1993494. PORT-IIs told - if the usual study models won't work, create new ones. <strong>Outcome</strong>s Measurement & Management 5-6,November/December 1993.495. Clinical trials <strong>and</strong> clinical practice. Lancet 342: 877-878, 1993496. [Special issue on health care outcomes]. Critical Public Health 4 (4): 1993497. Limitations of cost-utility research. <strong>Outcome</strong>s Measurement & Management 6-7, July/August 1993 *498. Patient satisfaction, quality of care, <strong>and</strong> outcomes. <strong>Outcome</strong>s Measurement & Management 4-5, July/August 1993499. Strengthening a health outcomes approach <strong>for</strong> Victoria, 1993500. The management of hypertension: a consensus statement. MJA (Suppl): 1-16, 1994501. Disinterring the doctor's mistakes. Canberra Times 23 Mar, 1994502. Abstracts of papers, Canberra: International Network on Health Expectancy, 1994

77503. Improving survey measurement: health status transitions in a longitudinal list sample design. Paper presented to the 7th Meeting of theInternational Network on Health Expectancy, Canberra, 23-25 February, 1994504. Australian Health Ministers’ Advisory Council. Sunshine Statement, 1993 (UnPub)505. Draft proposal to establish nursing home medical advisory panels towards securing high quality outcomes <strong>for</strong> elderly Australiansrequiring hostel or nursing home accommodation, 1991. (UnPub)506. Aaronson, NK, Meyerowtiz BE, Bard M <strong>and</strong> et al. <strong>Quality</strong> of Life research in oncology: past achievements <strong>and</strong> future priorities.Cancer 67: 839-843, 1991507. ACHS, Charter <strong>for</strong> Change, Zetl<strong>and</strong>, NSW: Australian Council on <strong>Healthcare</strong> St<strong>and</strong>ards, 1994508. ACHS Care Evaluation Program, Developing clinical indicators - the ACHS Care Evaluation Program, (UnPub)509. Aharony L. <strong>and</strong> Strasser S. Patient satisfaction: What we know about <strong>and</strong> what we still need to explore. Medical Care Review 50: 49-79, 1993 *510. AHCPR, Medical Treatment Effectiveness Program (MEDTEP) active research projects as of September 30, 1990. AHCPR ProgramNote, Rockville, MD:Agency <strong>for</strong> Health Care Policy <strong>and</strong> Research, 1990511. Alonso JA, Anto JP, Gonzalez M <strong>and</strong> et al. Measurement of general health status of non-oxygen-dependent chronic obstructivepulmonary disease patients. Medical Care 30 (Suppl): MS125-MS135, 1992512. Amadio PC. <strong>Outcome</strong>s measurements [Editorial]. The Journal of Bone <strong>and</strong> Joint Surgery 75-A: 1583-1584, 1993513. Anderson GF, Alonso J, Kohn LT <strong>and</strong> Black C. Analysing health outcomes through international comparisons. Medical Care 32: 526-534, 1994514. Anderson JJ <strong>and</strong> Chernoff MC. Sensitivity to change of rheumatoid arthritis clinical trial outcome measures. Journal of Rheumotology20 (3): 535-537, 1993515. Davies AR, Ware JE Jr. Involving consumers in quality of care assessment. Health Affairs (Millwood) 7 (1): 33-48, 1988516. Anderson JP, Kaplan RM, Berry CC et al. Interday reliability of function assessment <strong>for</strong> a health status measure: the quality of wellbeingscale. Medical Care 27 (11): 1076-1083, 1989517. Anderson J StC, Sullivan F <strong>and</strong> Usherwood TP. The medical outcomes study instrument (MOSI) - use of a new health status measurein Britain. Journal on Family Practice 7 (20): 5-218, 1990 *518. Andrews G, Hall. Goldstein G <strong>and</strong> et al. The economic costs of schizophrenia. Archives of General Psychiatry 42: 537-543, June 1985519. Anon, In<strong>for</strong>mation needs <strong>for</strong> health care quality assurance <strong>and</strong> outcomes, (UnPub). Personal communication: J. Ware Jnr520. Anon, <strong>Quality</strong> of life questionnaire, 1991. (UnPub). Personal communication: J. Ware Jnr521. Anon, Cross design synthesis: a new strategy <strong>for</strong> studying medical outcomes? Lancet 340: 944-946, 17 October 1992522. Avison ER, Verrier M <strong>and</strong> Wood-Dauphinee S. Rehabilitation outcome measures: Where do we st<strong>and</strong>? Canadian Journal of PublicHealth 83 (Suppl): S4-S6, 1992523. Ayanian JZ. Race, class, <strong>and</strong> the quality of medical care. JAMA 271: 1207-8, 1994 *524. Bailit HL. Use of effectiveness research in managed care plans. In: Effectiveness <strong>and</strong> outcomes in health care: Proceedings of aninvitational conference by the Institute of Medicine, edited by Heithoff, K. A. <strong>and</strong> Lohr, K. N. Washington, DC: National AcademyPress, 1990525. Baker R. Avedis Donabedian: an interview. <strong>Quality</strong> in Health Care 2: 40-46, 1993526. Balaban DJ, Sagi PC, Goldfarb NI <strong>and</strong> et al, Weights <strong>for</strong> scoring the quality of well-being instrumental among rheumatoid arthritics; acomparison to general population weights. Medical Care 24 (11): 973-980, November 1986527. Banta D. Developing outcome st<strong>and</strong>ards <strong>for</strong> quality assurance activities. <strong>Quality</strong> Assurance in Health Care 4 (1): 25-32, 1992

78528. Banta HD, Engel GL <strong>and</strong> Schersten T. Volume <strong>and</strong> outcome of organ transplantation. International Journal of Technology Assessmentin Health Care 8 (3): 490-505, 1992529. Bardsley M. <strong>and</strong> Cole J. Measured steps to outcomes. Health <strong>Services</strong> Journal 18-20, 17 October 1991530. Bardsley MJ <strong>and</strong> Coles JM. Practical experiences in auditing patient outcomes. <strong>Quality</strong> in Health Care 1: 124-130, 1992531. Barendregt JJ <strong>and</strong> Bonneux L. Changes in incidence <strong>and</strong> survival of cardiovascular disease <strong>and</strong> their impact on disease prevalence <strong>and</strong>health expectancy. Paper presented to the 7th Meeting of the International Network on Health Expectancy, Canberra, 23-25 February1994532. Barnum H. Evaluating healthy days of life gained from health projects. Social Science <strong>and</strong> Medicine 24 (10): 833-841, 1987533. Bartlett J <strong>and</strong> Cohen J. Building an accountable, improvable delivery system. Administration <strong>and</strong> Policy in Mental Health 21: 51-58,1993534. Battista RN. Discussion: Morris <strong>and</strong> Sherwood's "<strong>Quality</strong> of Life of Cancer Patients at Different Stages in the Disease Trajectory".Journal of Chronic Diseases 40 (6): 555-556, 1987535. Baue AE. Breast-conservation operations <strong>for</strong> treatment of cancer of the breast. JAMA 271: 1204-1205, 1994536. Baum FE <strong>and</strong> Cooke RD. Community health needs assessment: use of the Nottingham Health Profile in an Australian study. MJA 150:581-590, 1989537. Baume P. Of ends <strong>and</strong> means [letter]. MJA 155: 142-4, 1991538. Baume P. Goals, financing <strong>and</strong> outcomes: The pursuit of better health in Australia, 1991. (UnPub)539. Bellamy T, Newton JS, LeBaron NM <strong>and</strong> Horner RH. <strong>Quality</strong> of life <strong>and</strong> lifestyle outcomes: a challenge <strong>for</strong> residential programs. In:<strong>Quality</strong> of life: perspectives <strong>and</strong> issues, edited by Schalock RL <strong>and</strong> Begab MJ, Washington, DC: American Ass in Mental Retardation,1990540. Bendsten P <strong>and</strong> Hornquist JO. Severity of rheumatoid arthritis, function <strong>and</strong> quality of life: sub-group comparisons. Clinical <strong>and</strong>Experimental Rheumatology 11: 495-502, 1993541. Berg AO. Variations among family physician's management strategies <strong>for</strong> lower urinary tract infection in women: A report from theWashington Family Physicians Collaborative Research Network. Journal of the American Board of Family Practice 4: 327-330, 1991542. Berg KO, Wood-Dauphinee SL, Williams JI <strong>and</strong> Maki B. Measuring balance in the elderly: Validation of an instrument. CanadianJournal of Public Health 83: S7-S11, 1992543. Bergner M. Measurement of health status. Medical Care 23: 696-704, 1985544. Bergner M. <strong>Quality</strong> of life, health status, <strong>and</strong> clinical research. Medical Care 27 (Suppl): S148-S156, 1989545. Bergner M, Barry MJ, Bowman MA, Doyle MA, Guess HA <strong>and</strong> Nutting PA. Where do we go from here? Opportunities <strong>for</strong> applyinghealth status assessment measures in clinical settings. Medical Care 30 (Suppl): MS219-230, 1992546. Bergner M, Bobbitt RA, Carter WB <strong>and</strong> et al. The sickness impact profile: development <strong>and</strong> final revision of a health status measure.Medical Care 19: 787-805, 1981 *547. Bergner M, Bobbitt RA, Carter WB <strong>and</strong> Gilson BS. The Sickness Impact Profile: Development <strong>and</strong> final revision of a health statusmeasure. Medical Care 19: 787-805, 1981548. Bergner M, Bowman M, Doyle MA <strong>and</strong> et al. Where do we go from here? General audience discussion. Medical Care 30 (Suppl):MS231-9, 1992549. Berwick D M. Health services research <strong>and</strong> quality of care. Assignments <strong>for</strong> the 1990's. Medical Care 27 (8): 763-771, August 1989550. Berwick DM, Murphy JM, Goldman PA <strong>and</strong> et al. Per<strong>for</strong>mance fo a five-item mental health screening test. Medical Care 29: 169-176,1991551. Berwick DM <strong>and</strong> Weinstein MC. What do patients value? Willingness to pay <strong>for</strong> ultrasound in normal pregnancy. Medical Care 23: 7July: 881-893, 1985

79552. Best J. The matter of outcome. MJA 148: 650-652, 1988553. Bevan G. <strong>Quality</strong>-adjusted life years (qalys); an exposition <strong>and</strong> critique, 1988. (UnPub)554. Bion J. <strong>Outcome</strong>s in intensive care. BMJ 307: 953-954, 1993555. Blake RL <strong>and</strong> V<strong>and</strong>iver TA. The reliability <strong>and</strong> validity of a ten-item measure of functional status. Journal of Family Practice 23 (5):455-459, 1986556. Blim RD. Strategies <strong>for</strong> improving <strong>and</strong> exp<strong>and</strong>ing the application of health status measures in clinical settings [Discussion]. MedicalCare 30 (Suppl.): MS196-7, 1992557. Bloom BS. Does it work? The outcomes of medical interventions. International Journal of Technology Assessment in Health Care 6:326-332, 1990558. Boers M <strong>and</strong> Tugwell P. The validity of pooled outcome measures (indices) in rheumatoid arthritis clinical trials. Journal ofRheumatology 20 (3): 568-574, 1993559. Bolduc M. For a conceptual model that better reflects the environment, 1992 pp1-20 (UnPub)560. Bombardier C, Ware J, Russell IJ <strong>and</strong> et al. Auranofin therapy <strong>and</strong> quality of life in patients with rheumatoid arthritis. Results of amulti-center trial. American Journal of Medicine 81: 4 October: 565-578, 1986561. Bond S <strong>and</strong> Thomas L. Input on outcomes. Nursing Times 88 (38): 50-52, 16-22 September 1992562. Bone M, Bebbington AC <strong>and</strong> Nicolaas G. Policy relevance <strong>and</strong> comparability problems of health expectancy indicators. Paperpresented to the 7th Meeting of the International Network on Health Expectancy, Canberra, 23-25 February, 1994563. Bone MR. International ef<strong>for</strong>ts to measure health expectancy. Journal of Epidemiology <strong>and</strong> Community Health 46: 555-558, 1992564. Borowitz M <strong>and</strong> Sheldon T. Controlling health care: from economic incentives to micro-clinical regulation. Health Economics 2: 201-204, 1994565. Borthwick-Duffy SA. <strong>Quality</strong> of life persons with severe or profound mental retardation. In: <strong>Quality</strong> of life: perspectives <strong>and</strong> issues,edited by Schalock RL <strong>and</strong> Begab MJ, Washington, DC: American Ass on Mental Retardation, 1990566. Boshuizen HC <strong>and</strong> van der Water HPA. Socio-economic differences in health expectancy in the Netherl<strong>and</strong>s. Paper presented to the 7thMeeting of the International Network on Health Expectancy, Canberra, 23-25 February, 1994567. Bowling A. The conceptualization of functioning, health <strong>and</strong> quality of life. In: Measuring Health: A Review of <strong>Quality</strong> of LifeMeasurement Scales, Philadelphia: Open University Press: pp 1-11, 1991568. Bowling A. Measuring health: a review of quality of life measurement scales, Buckingham, Engl<strong>and</strong> <strong>and</strong> Bristol Pa: Open UniversityPress: pp 1-11, 1991 *569. Boyd NF, Selby PJ, Sutherl<strong>and</strong> HJ <strong>and</strong> et al. Measurement of the clinical status of patients with breast cancer: evidence <strong>for</strong> the validityof self assessment with linear analogue scales. Journal of Clinical Epidemiology 41 (3): 243-250, 1987570. Brazier J. The SF-36 health survey questionnaire: A tool <strong>for</strong> economists. Health Economics 2: 213-215, 1993 *571. Brazier J, Jones N <strong>and</strong> Kind P. A comparison of two health-status measures: Euroqol meets SG-36, 1993. (UnPub)572. Brazier JE, Harper R, Jones NMB <strong>and</strong> et al. Validating the SF-36 health survey questionnaire: new outcome measures in primarycare. BMJ 305: 160-164, 1992 *573. Brazier JE, Harper R, Joens NMB, O'Cathain A, Thomas KJ, Usherwood T <strong>and</strong> Westlake L. Validating the SF-36 health surveyquestionnaire: new outcome measure <strong>for</strong> primary care. BMJ 305: July 18: 160-164, 1992574. Brekke JS. An examination of the relationships among three outcome scales in schizophrenia. The Journal of Nervous <strong>and</strong> MentalDisease 180 (3): 162-167, 1993575. Cairns J <strong>and</strong> Schackley P. Sometimes sensitive, seldom specific: A review of the economics of screening. Health Economics 2: 43-53,1993

80576. Callahan CM, Drake BG, Heck DA <strong>and</strong> Dittus RS. Patient outcomes following tricompartmental total knee replacement. JAMA 271:1349-57, 1994577. Callahan D. Ethics <strong>and</strong> priority setting in Oregon. Health Affairs 78-87, Summer 1991578. Calman KC. <strong>Quality</strong> of life in cancer patients - an hypothesis. Journal of Medical Ethics 10: 124-127, 1984579. Cangialose CB. <strong>Outcome</strong>s research [letter]. NEJM 330: 435, 1994580. Carey RG, Rondinelli RD <strong>and</strong> Murphy JR. Functional <strong>Outcome</strong> Predictors. Archives of Physical Medicine <strong>and</strong> Rehabilitation 73:April: 400-401, 1992581. Carnes BA <strong>and</strong> Olshanky SJ. Evolutionary perspectives on human senescence. Population Development Review 1994582. Carr-Hill RA. The measurement of patient satisfaction. Journal of Public Health Medicine 14: 236-249, 1992583. Carr-Hill RA <strong>and</strong> Morris J. Current practice in obtaining the "Q" in QALYs: a cautionary note. BMJ 303 (6804): 699-701, 21September 1991 *584. Carter R. An overview of health-related quality of life measures - conceptual origins, applications <strong>and</strong> usefulness. (UnPub)585. Carter WB, Bobbitt RA, Bergner M. <strong>and</strong> et al. Validation of an interval scaling: the sickness impact profile. Health <strong>Services</strong> Research11: 516-528, 1976586. CASPE-HKS Ltd, Using occurrence screening in hospital-wide clinical audit, 1991. (UnPub)587. Cella DF. Overcoming difficulties in demonstrating health outcome benefits. Journal of Parenteral <strong>and</strong> Enteral Nutrition 16 (6Suppl): 1065-1115, Nov-Dec 1992588. Chambers LW, Haight M, Norman G <strong>and</strong> et al. Sensitivity to change <strong>and</strong> the effect of mode of administration on health statusmeasurement. Medical Care 25: 470-480, 1987589. Chambers LW, MacDonald LA, Tugwell P <strong>and</strong> et al. The McMaster Health Index Questionnaire as a measure of quality of life <strong>for</strong>patients with rheumatoid disease. Journal of Rheumatology 9: 780-784, 1982590. Chambers LW, Sackett DL, Goldsmith CH <strong>and</strong> et al. Development <strong>and</strong> application of an index of social function. Health <strong>Services</strong>Research 11: 430-441, 1976591. Chapman S. QALYs: a personal experience [letter]. Australian Journal of Public Health 17: 397, 1993592. Conn VS, Taylor SG <strong>and</strong> Casey B. Cardiac rehabilitation program participation <strong>and</strong> outcomes after myocardial infarction.Rehabilitation Nursing 17 (2): 58-62, Mar-Apr 1992593. Connelly JE, Philbrick JT, Smith GR, Kaiser DL <strong>and</strong> Wymer A. Health perceptions of primary care patients <strong>and</strong> the influence onhealth care utilization. Medical Care 27 (Suppl): S99-S109, 1989594. Conroy J <strong>and</strong> Feinstein CS. Measuring quality of life: Where have we been? Where are we going? In: <strong>Quality</strong> of life: perspectives <strong>and</strong>issues, edited by Schalock RL <strong>and</strong> Begab MJ, Washington, DC: American Ass on Mental Retardation, 1990595. Cook DJ, Guyatt GH, Juniper E <strong>and</strong> et al. Interviewer versus self-administered questionnaires in developing a disease-specific, healthrelatedquality of life instrument <strong>for</strong> asthma. Journal of Clinical Epidemiology 46: 529-34, 1993596. Coombs HC. The quality of life <strong>and</strong> its assessment, 1977. (UnPub)597. Cotton P. Doctors are asked: Does it work? JAMA 270: 153-8, 1993598. Cox DR, Fitzpatrick R, Fletcher AE, Gore SM, Spiegelhalter J <strong>and</strong> Jones DR. <strong>Quality</strong> of life assessment: can we keep it simple? J RStatist Soc 155: Part 3: 353-393, 1992599. Cribb A. <strong>Quality</strong> of life - a response to KC Calman. Journal of Medical Ethics 11: 142-145, 1985600. Crockett A, Moss J <strong>and</strong> Alpers J. The continuing impact of home oxygen therapy <strong>for</strong> respiratory patients on a hospital budget.Australian Health Review 15: 259-68, 1992

81601. Crockett AJ, Alpers JH <strong>and</strong> Moss JR. Home oxygen therapy: An audit of survival. Australian <strong>and</strong> New Zeal<strong>and</strong> Journal of Medicine21: 217-21, 1991602. Culyer AJ. The morality of efficiency in health care: Some uncom<strong>for</strong>table implications. Health Economics 1: 7-18, 1992603. Culyer AJ. Need, values <strong>and</strong> health status measurement. In: Economic Aspects of Health <strong>Services</strong> edited by Culyer AJ <strong>and</strong> Wright KG,pp 9-31, London: M. Roberston, 1978604. Cummins RA. The Comprehensive <strong>Quality</strong> of Life Scale - Intellectual Disability: An instrument under development. Australia <strong>and</strong>New Zeal<strong>and</strong> Journal of Developmental Disabilities 17: 259-64, 1991605. Cummins RA. Comprehensive <strong>Quality</strong> of Life Scale <strong>for</strong> Adults, 4th edition (ComQol-4), 1993 (UnPub)606. Cummins RA. Health promotion <strong>and</strong> the Comprehensive <strong>Quality</strong> of Life Scale. Health Promotion Journal of Australia 3: 46-47, 1994607. Cummins RA <strong>and</strong> Baxter C. A case <strong>for</strong> the inclusion of subjective quality of life data in service-delivery evaluations. In: Evaluation:Does it make a difference? Conference proceedings, Brisbane Hilton Hotel, 7-9 July, 1993, Brisbane: Australasian Evaluation Society,1993608. Cummins RA <strong>and</strong> Baxter C. Choice of outcome measures in service evaluations <strong>for</strong> people with an intellectual disability. EvaluationJournal of Australasia 1994 (In Press)609. Cummins RA, McCabe MP, Romeo Y <strong>and</strong> Gullone E. The Comprehensive <strong>Quality</strong> of Life Scale (ComQol): Instrument development<strong>and</strong> psychometric evaluation on college staff <strong>and</strong> students. Educational <strong>and</strong> Psychological Measurement (In Press)610. D'Arcy C, Holman J, Donovan RJ <strong>and</strong> Corti B. Evaluating projects funded by the Western Australian Health Promotion Foundation: asystematic approach. Health Promotion International 8: 199-208, 1993611. Dagher M <strong>and</strong> Lloyd RJ. Managing negative outcome by reducing variances in the emergency department. <strong>Quality</strong> Review Bulletin 15-21, January 1991612. Daniels N. Is the Oregon Rationing Plan fair? JAMA 265: 2232-2235, 1991613. Darke S. The measurement of outcome in opiate treatment evaluation studies. Drug <strong>and</strong> Alcohol Review 11: 169-174, 1992614. Das V. Cultural, social <strong>and</strong> behavioural determinants of health; what is the evidence? 1989 (UnPub)615. Dearing G. Measuring <strong>and</strong> managing outcomes improve quality <strong>and</strong> reduce costs. <strong>Outcome</strong>s Measurement & Management 3-5,May/June 1994616. DeFriese G. Measuring the effectiveness of medical interventions: new expectations of health services research [Editorial] HSR: Health<strong>Services</strong> Research: 691-695, 1990617. Dempsey DT. <strong>Outcome</strong>s research in laparoscopic surgery: The National Laparoscopic Surgery Register. Journal of LaparoendoscopicSurgery 3: 375-378, 1993618. DesHarnais S, McMahon LF, Jr. <strong>and</strong> Wroblewski R. Measuring outcomes of hospital care using multiple risk-adjusted indexes.Health <strong>Services</strong> Research 26 (4): 425-445, Oct 1991619. DesHarnais SI, Chesney JD, Wroblewski RT <strong>and</strong> et al. The risk-adjusted mortality index. A new measure of hospital per<strong>for</strong>mance.Medical Care 26 (12): 1129-1148, 1988620. DesHarnais SI <strong>and</strong> Simpson KN. Indices <strong>for</strong> monitoring hospital outcomes in developed countries. Health Policy 21: 1-15, 1992621. DeVivo MJ, Rutt RD, Black KJ <strong>and</strong> et al. Trends in spinal cord injury demographics <strong>and</strong> treatment outcomes between 1973 <strong>and</strong> 1986.Archives of Physical Medicine <strong>and</strong> Rehabilitation 73 (5): 424-430, May 1992622. Deyo RA <strong>and</strong> Carter WB. Strategies <strong>for</strong> improving <strong>and</strong> exp<strong>and</strong>ing the application of health status measures in clinical settings.Medical Care 30 (Suppl): MS176-MS186, 1992623. Deyo RA, Cherkin D <strong>and</strong> Conrad D. The back pain outcome assessment team. Health <strong>Services</strong> Research 25: 5 December: 1990624. Deyo RA <strong>and</strong> Patrick DL. Barriers to the use of health status measures in clinical investigation, patient care, <strong>and</strong> policy research.Medical Care 27 (Suppl): S254-S268, 1989

82625. Dickey B <strong>and</strong> Azeni H. Impact of Managed Care on Mental Health <strong>Services</strong>. Health Affairs 197-204, Fall 1992626. Dixon J <strong>and</strong> Welch HG. Priority setting: lessons from oregon. Lancet 337 (8746): 891-894, 13 April 1991627. Donabedian A. Basic approaches to assessment: Structure, process <strong>and</strong> outcome. In: The definition of quality <strong>and</strong> approaches to itsassessment. Volume 1, Explorations in quality assessment <strong>and</strong> monitoring, Ann Arbor: Health Administration Press 9: 79-128, 1980628. Donabedian A. <strong>Quality</strong> assessment <strong>and</strong> assurance: unity of purpose, diversity of means. Inquiry 25: 173-192, Spring 1988629. Donabedian A. The role of outcomes in quality assessment <strong>and</strong> assurance. <strong>Quality</strong> Review Bulletin 356-60, November 1992630. Donabedian A. The quality of care: How can it be assessed? JAMA 260: 1743-8, 1988 *631. Donaldson C, Atkinson A, Bond J <strong>and</strong> et al. QALYs <strong>and</strong> long term care <strong>for</strong> elderly people in the UK: scales <strong>for</strong> assessment of qualityof life. Age Ageing 17: 379-387, 1988632. Donaldson C, Atkinson A, Bond J <strong>and</strong> et al. QALYs <strong>and</strong> long-term care <strong>for</strong> elderly people in the UK:Scales <strong>for</strong> assessment of quality oflife. International Journal of Social Psychiatry 34 (4): 243-247, Winter 1988633. Donovan JL, Frankel SJ <strong>and</strong> Eyles JD. Assessing the need <strong>for</strong> health status measures. Journal of Epidemiology <strong>and</strong> Community Health47: 158-162, 1993634. Donovan K, Sanson-Fisher RW <strong>and</strong> Redman S. Measuring quality of life in cancer patients. Journal of Clinical Oncology 7: 7 July:959-968, 1989635. Dougherty CJ. Setting health care priorities. Hastings Center Report 21 (3): 1-10, 1991636. Dowie J. A short <strong>and</strong> slightly impolite paper about health status <strong>and</strong> health service outcome measurement, 1991 (UnPub)637. Dowie J. Equity, efficiency <strong>and</strong> ethics. In: The ethics of allocating health resources. Report of a conference at Westmead Hospital,February 15-16, 1991, Sydney: NSW Health, 1991638. Dowie J. The PIQALI: the appropriate way to deal with 'process utility' in health service evaluation, 1993. (UnPub)639. Dowie J. 'Process utility' can seriously damage your health service evaluation but the generic measure of benefit should include 'serviceoutcomes', 1993. (UnPub)640. Dowie JA. Valuing the benefits of health improvement. Australian Economic Papers 21-41, June 1970641. Drummond M, Br<strong>and</strong>t A, Luce B <strong>and</strong> et al. St<strong>and</strong>ardizing economic evaluation methodologies in health care: practice, problems <strong>and</strong>potential. International Journal of Technology Assessment in Health Care 1991642. Drummond M <strong>and</strong> O'Brien B. Clinical importance, statistical significance <strong>and</strong> the assessment of economics <strong>and</strong> quality-of-lifeoutcomes. Health Economics 2: 205-212, 1993643. Drummond MF. Discussion: Torrance's "Utility approach to measuring health-related quality of life." Journal of Chronic Diseases 40(6): 601-603, 1987644. Dugdale P. The NSW Health <strong>Outcome</strong>s Initiative in Context, 1992. (UnPub)645. Duncan PG, Cohen MM, Tweed WA <strong>and</strong> et al. The Canadian four-centre study of anaesthetic outcomes: III. Are anaestheticcomplications predictable in day surgical practice? Canadian Journal of Anaesthesia 39 (5): 440-448, 1992646. Early Breast Cancer Trialists' Collaborative Group, Systematic treatment of early breast cancer by hormonal, cytotoxic, or immunetherapy. Lancet 339: 1-14-71-85, 1992647. Edgerton RB. <strong>Quality</strong> of life from a longitudinal research perspective. In: <strong>Quality</strong> of Life: Perspectives <strong>and</strong> Issues, edited by SchalockRL <strong>and</strong> Begab MJ. Washington, DC: American Ass on Mental Retardation, 1990648. Ellis BH, Jr, McCan I, Price G <strong>and</strong> et al. The New Mexico treatment outcome study: evaluating the utility of existing in<strong>for</strong>mationsystems. Journal of Health Care <strong>for</strong> the Poor <strong>and</strong> Underserved 3 (1): 138-150, Summer 1992649. Ellwood PM. Shattuck Lecture - outcomes management. A technology of patient experience. NEJM 318 (23): 1549-1556, June 9, 1988

83650. Endicott J, Nee J, Harrison W <strong>and</strong> Blumenthal R. <strong>Quality</strong> of Life Enjoyment <strong>and</strong> Satisfaction Questionnaire: A new measure.Psychopharmacology Bulletin 29: 321-6, 1993651. Engel W, Freund DA, Stein JS <strong>and</strong> Fletcher RH. The treatment of patients with asthma by specialists <strong>and</strong> generalists. Medical Care27: 306-12, 1989.652. Epstein AM. The outcomes movement: will it get us where we want to go? NEJM 323: 266-269, 1990653. Epstein AM, Hall JA, Tognetti J, Son LH <strong>and</strong> Conant L. Using proxies to evaluate quality of life. Medical Care 27 (3 Suppl): S91-S98,1989654. Erickson P, Kendall EA, Anderson JP <strong>and</strong> et al. Using composite health status measures to assess the nation's health. Medical Care 27(3 Suppl): S66-S77, 1989655. Essink-Bot M, Stouthard MEA <strong>and</strong> Bonsel GJ. Generalizability of valuations on health states collected with the EuroQolc-Questionnaire. Health Economics 2: 237-246, 1993656. Essink-Bot ML, Bonsel GJ <strong>and</strong> Van der Maas PJ. Valuation of health states by the general public: feasibility of a st<strong>and</strong>ardizedmeasurement procedure. Social Science <strong>and</strong> Medicine 31 (11): 1201-1206, 1990657. Evans I. The challenge of breast cancer [Lancet Conference]. Lancet 343: 1085-6, 1994658. Evans RG. Tension, compression <strong>and</strong> shear: directions, stresses <strong>and</strong> outcomes of health care cost control. Journal of Health Politics,Politics, Policy <strong>and</strong> Law 15 (1): 101-128, Spring 1990659. Faden R <strong>and</strong> Leplege A. Assessing quality of life. Medical Care 30 (Suppl): MS166-MS175, 1992660. Falloon IRH. Early intervention <strong>for</strong> first episodes of schizophrenia: A preliminary exploration. Psychiatry 55: 4-15, 1992661. Fallowfield LJ, Hall A, Maguire GP <strong>and</strong> Baum M. Psychological outcomes of different treatment policies in women with early breastcancer outside of clinical trial. BMJ 301: 575-80, 1990662. Fanshel S <strong>and</strong> Bush JW. A health status index <strong>and</strong> its application to health services outcomes. Operations Research 18: 1021-1066,1970663. Feeny DH <strong>and</strong> Torrance GW. Incorporating utility-based quality-of-life assessment measures in clinical trials: Two examples. MedicalCare 27 (Suppl): S190-S204, 1989664. Feinstein AR. Benefits <strong>and</strong> obstacles <strong>for</strong> development of health status assessment measures in clinical settings. Medical Care 30:MS50-MS56, 1992665. Ferrans CE <strong>and</strong> Powers MJ. <strong>Quality</strong> of life of hemodialysis patients. ANNA Journal 20: 575-81, 1993666. Ferraris VA <strong>and</strong> Propp ME. <strong>Outcome</strong> in critical care patients: a multivariate study. Critical Care Medicine 20 (7): 967-976, July 1992667. Fisher B, Costantino J, Redmond C <strong>and</strong> et al. Lumpectomy compared with lumpectomy <strong>and</strong> radiation therapy <strong>for</strong> the treatment ofintraductal breast cancer. NEJM 328: 1581-1586, 1993668. Fisher ES <strong>and</strong> Wennberg JE. Administrative data in effectiveness studies: the prostatectomy assessment. In: Effectiveness <strong>and</strong><strong>Outcome</strong>s in Health Care, edited by Heithoff KA <strong>and</strong> Lohr KN. Washington, DC: National Academy Press, 1990669. Fitzpatrick R. Patient satisfaction <strong>and</strong> quality of life measures. In: <strong>Outcome</strong> measures in orthopaedics, edited by Pynsent PB, FairbankJCT <strong>and</strong> Carr A. Ox<strong>for</strong>d: Butterworth, 1993670. Fitzpatrick R, Fletcher A, Gore S. <strong>and</strong> et al. <strong>Quality</strong> of life measures in health care. I: applications <strong>and</strong> issues in assessment. BMJ 305(6861): 1074-1077, 31 October 1992671. Fletcher A, Jones D, Cox D <strong>and</strong> et al. <strong>Quality</strong> of life measures in health care II: design, analysis <strong>and</strong> interpretation. BMJ 305 (6862):1145-1148, 7 November 1992672. Fletcher AE <strong>and</strong> Bulpitt CJ. Measurement of quality of life in clinical trials of therapy. Cardiology 75 (Suppl) 1: 41-52, 1988673. Flood AB, Lorence DP, Ding J, McPherson K <strong>and</strong> Black NA. The role of expectations in patient's reports of post-operative outcomes<strong>and</strong> improvement following therapy. Medical Care 31: 1043-56, 1993

84674. Fowler FJ, Wennberg JE, Timothy RP <strong>and</strong> et al. Symptom status <strong>and</strong> quality of life following prostatectomy. Journal of the AmericanMedical Record Association 259: 3018-3022, 1988675. Fox DM <strong>and</strong> Leichter HM. Rationing care in Oregon: The new accountability. Health Affairs 7-27, Summer 1991676. Fox DM <strong>and</strong> Leichter HM. The ups <strong>and</strong> downs or Oregon's rationing plan. Health Affairs 66-70, Summer 1993677. Frater A. Meeting report: Health <strong>Outcome</strong>s. Health <strong>Services</strong> Journal conference, London, November 1992. Health <strong>Services</strong> Journal 69678. Frater A <strong>and</strong> Costain D. Any better? <strong>Outcome</strong> measures in medical audit. BMJ 304 (6826): 519-520, 29 February 1992679. Freund D <strong>and</strong> Harvey R. Making it better. MJA 155: 583-4, 1991680. Freund DA, Dittus RS, Fitzgerald J <strong>and</strong> et al. Assessing <strong>and</strong> improving outcomes: total knee replacement. Health <strong>Services</strong> Research 25(5): 723-726, December 1990681. Freund DA, Stein J, Hurley R, Engel W, Woomert A <strong>and</strong> Lee B. The Kansas City Asthma Care Project: specialty differences in thecost of treating asthma. Annals of Allergy 60: 3-7, 1988682. Freund O. National survey of total knee replacement, 1991 (UnPub)683. Fried BJ, Boers M <strong>and</strong> Baker PRA. A method <strong>for</strong> achieving consensus on rheumatoid arthritis outcome measures: the OMERACTConference Process. Journal of Rheumatology 20 (3): 548-551, 1993684. Fries JF. The hierarchy of outcome assessment. Journal of Rheumatology 20 (3): 546-547, 1993685. Fries JF <strong>and</strong> Ramey DR. Platonic outcomes. Journal of Rheumatology 20 (3): 415-417, 1993686. Froberg DG <strong>and</strong> Kane RL. Methodology <strong>for</strong> measuring health-state preferences - II: scaling methods. Journal of ClinicalEpidemiology 42 (5): 459-471, 1989687. Froberg DG <strong>and</strong> Kane RL. Methodology <strong>for</strong> measuring health-state preferences - III: population <strong>and</strong> context effects. Journal ofClinical Epidemiology 42 (6): 585-592, 1989688. Froberg DG <strong>and</strong> Kane RL. Methodology <strong>for</strong> measuring health-state preferences - I: measurement strategies. Journal of ClinicalEpidemiology 42 (4): 345-354, 1989689. Froberg DG <strong>and</strong> Kane RL. Methodology <strong>for</strong> measuring health-state preferences - IV: progress <strong>and</strong> a research agenda. Journal ofClinical Epidemiology 42 (7): 675-685, 1989690. Frommer M, Rubin G <strong>and</strong> Lyle D. The NSW Health <strong>Outcome</strong>s Program. NSW Public Health Bulletin 3: 136-137691. Frostick SP <strong>and</strong> Hunter JB. Complications. In: <strong>Outcome</strong> measures in orthopaedics, edited by Pynsent PB, Fairbank JCT <strong>and</strong> Carr A.Ox<strong>for</strong>d: Butterworths, 1993692. Furlong W, Feeny D, Torrance GW <strong>and</strong> et al. Guide to design <strong>and</strong> development of health-state utility instrumentation, 1990. (UnPub)693. Gafni A. The quality of QALYs (quality-adjusted-life-years):do QALYs measure what they at least intend to measure? Health Policy13: 81-83, 1989694. Gafni A <strong>and</strong> Birch S. Equity considerations in utility-based measures of health outcomes in economic appraisals: an adjustmentalgorithm, 1990 (UnPub)695. Ganiats TG, Palinksa LA <strong>and</strong> Kaplan RM. Comparison of <strong>Quality</strong> of Well-Being Scale <strong>and</strong> Functional Status Index in patients withatrial fibrillation. Medical Care 30: 958-964, 1992696. Garratt A, Ruta DA, Abdalla MI, Buckingham JK <strong>and</strong> Russell IT. The SF-36 health survey questionnaire: an outcome measure suitable<strong>for</strong> use within the NHS? BMJ 306: 1440-1444, 1993697. Garratt AM, Macdonald LM, Ruta DA <strong>and</strong> et al. Towards measurement of outcome <strong>for</strong> patients with varicose veins. <strong>Quality</strong> in HealthCare 2: 5-10, 1993698. Gaut B. Ethics, distribution <strong>and</strong> Oregon. MJA 159 (4): 280-281, 16 August 1993699. Geigle R <strong>and</strong> Jones SB. <strong>Outcome</strong>s measurement: A report from the front. Inquiry 27: 7-13, 1990

85700. Gelber RD <strong>and</strong> Gelman RS. A quality of life oriented endpoint <strong>for</strong> comparing therapies. Biometrics 45: September: 781-795, 1989701. Gerard K, Dobson M <strong>and</strong> Hall J. Framing <strong>and</strong> labelling effects in health descriptions: <strong>Quality</strong> adjusted life years <strong>for</strong> treatment of breastcancer. Journal of Clinical Epidemiology 46: 77-84, 1993702. Gerard K <strong>and</strong> Mooney G. QALY league tables: H<strong>and</strong>le with care. Health Economics 2: 59-64, 1993703. Gerrity MS, Gaylord S <strong>and</strong> Williams ME. Short versions of the Timed manual Per<strong>for</strong>mance Test. Medical Care 31: 617-628, 1993704. Gillam L. Resource allocation: the unavoidable debate (Part 1). Bioethics News 11: 49-57, 1992705. Goldberg HI, Pantell RH <strong>and</strong> Weber JR. Finale Panel: Reactions, reflections <strong>and</strong> predictions. Medical Care 30 (Suppl): MS283-MS293, 1992706. Goldberg J, Eisen SA, True WR <strong>and</strong> Henderson WG. Health effects of military service: Lessons learned from the Vietnam experience.AEP 2: 841-853, 1992707. Golden WE. Health status measurement: implementation strategies. Medical Care 30 (Suppl): MS187-MS195, 1992708. Goldsmith CH, Boers M, Bombardier C <strong>and</strong> et al. Criteria <strong>for</strong> clinically important changes in outcomes: development, scoring <strong>and</strong>evaluation of rheumatoid arthritis patient <strong>and</strong> trial profiles. Journal of Rheumotology 20 (3): 561-565, 1993709. Golenski JD <strong>and</strong> Thompson SM. A history of Oregon's basic health services act: an insider's account. <strong>Quality</strong> Review Bulletin 144-149,May 1991710. Gomez CR. <strong>Outcome</strong>s of carotid endarterectomy. Annals of Internal Medicine 114 (8): 703-704, 15 April 1991711. Goodwin A. The NSW health outcomes initiative. NSW Public Health Bulletin 3: 25-26, 1992712. Gornick M, Lubitz J <strong>and</strong> Riley GUS. Initiatives <strong>and</strong> approaches <strong>for</strong> outcomes <strong>and</strong> effectiveness research. Health Policy 17: 209-225,1991713. Gough I <strong>and</strong> Thomas T. Need satisfaction <strong>and</strong> welfare outcomes: Theory <strong>and</strong> explanations. Social Policy <strong>and</strong> Administration 28: 33-56, 1994714. Gouveia W A. Measuring <strong>and</strong> managing patient outcomes. American Journal of Hospital Pharmacy 49 (9): 2157-2158, September1992715. Graetz B. Health consequences of employment <strong>and</strong> unemployment: longitudinal evidence <strong>for</strong> young men <strong>and</strong> women. Social Science<strong>and</strong> Medicine 36 (6): 715-724, 1993716. Greco P <strong>and</strong> Eisenberg JM. Changing physicians’ practices. NEJM 329: 1271-1274, 1993717. Greenfield S. The state of outcome research: are we on target? NEJM 320: 1142-1143, 1989 *718. Greenfield S <strong>and</strong> Nelson EC. Recent developments <strong>and</strong> future issues in the use of health status assessment measures in clinicalsettings. Medical Care 30 (Suppl): MS23-MS41, 1992719. Greenfield S, Nelson EC, Zubkoff M <strong>and</strong> et al. Variations in resource utilization among medical specialties <strong>and</strong> systems of care.Results from the Medical <strong>Outcome</strong>s Study. Journal of the American Research Association 267 (12): 1624-1630, 25 March 1992720. Greenough G <strong>and</strong> Fraser RD. Assessment of outcome in patients with low-back pain. Spine 17 (1): 36-41, 1992721. Grembowski D, Patrick D, Diehr P, Durham M, Beres<strong>for</strong>d S, Kay E <strong>and</strong> Hecht J. Self-efficacy <strong>and</strong> health behaviour among olderadults. Journal of Health <strong>and</strong> Social Behaviour 34: 89-104, 1993722. Grill, R, Mainini F, Penna A <strong>and</strong> et al. Inappropriate Halsted mastectomy <strong>and</strong> patient volume in Italian hospitals. American Journal ofPublic Health 83: 1762-4, 1993723. Grisso J. Making comparisons. Lancet 342: 157-60, 1993724. Gross PF. Health indicator conceptualization, measurement <strong>and</strong> data collection: a state-of-the-art review, 1977. (UnPub)

86725. Grotvedt L <strong>and</strong> Viks<strong>and</strong> G. Life expectancy without disease <strong>and</strong> disability in Norway. Paper presented to the 7th Meeting of theInternational Network on Health Expectancy, Canberra, 23-25 February, 1994726. Gudex C. QALYs <strong>and</strong> their use by the Health Service, 1986. (UnPub)727. Gudex C <strong>and</strong> Kind P. The QALY toolkit, 1987. (UnPub)728. Guillemin F, Bonbardier C <strong>and</strong> Beaton D. Cross-cultural adaption of health-related quality of life measures: Literature review <strong>and</strong>proposed guidelines. Journal of Clinical Epidemiology 46: 1417-1432, 1993729. Gulli<strong>for</strong>d MC. Evaluating prognostic factors: Implications <strong>for</strong> measurement of health care outcome. Journal of Epidemiology <strong>and</strong>Community Health 46: 323-326, 1992730. Gulli<strong>for</strong>d MC, Barton JR <strong>and</strong> Bourne HM. Selection <strong>for</strong> oesophagectomy <strong>and</strong> postoperative outcome in a defined population. <strong>Quality</strong>in Health Care 2:17-20, 1993731. Guyatt GH, Eagle DJ, Sachett B <strong>and</strong> et al. Measuring quality of life in the frail elderly. Journal of Clinical Epidemiology 46: 1433-1444, 1993732. Guyatt GH, Feeny DH <strong>and</strong> Patrick DL. Measuring health-related quality of life. Annals of Internal Medicine 118: 622-629, 1993733. Guyatt GH, S<strong>and</strong>er JO, Veldhuyzen van Zanten SJO <strong>and</strong> et al. Measuring quality of life in clinical trials: a taxonomy <strong>and</strong> review.Canadian Medical Association Journal 140 (12): 1441-1448, 15 June 1989734. Haas M <strong>and</strong> Hall J. The Oregon Plan. NSW Public Health Bulletin 3: 5-1, 1992735. Hadorn DC. The Oregon priority-setting exercise. Hastings Center Report 21 (3): 11-16, 1991736. Haines A <strong>and</strong> Feder G. Guidance on guidelines. Writing them is easier than making them work. BMJ 305 (6857): 785-786, 3 October1992737. Hall J. <strong>Quality</strong> vs. quantity. NSW Public Health Bulletin 2: 115-116, 1994738. Hall J, Hall N, Fisher E <strong>and</strong> et al. Measuring outcomes of general practice, (UnPub)739. Hall J <strong>and</strong> Masters G. Measuring outcomes of health services: a review of some available measures. Community Health Studies 10 (2):147-155, 1986740. Hall J, Shiell A <strong>and</strong> CHERE, Health <strong>Outcome</strong>s: A Health Economics Perspective, Sydney: Centre <strong>for</strong> Health Economics Research <strong>and</strong>Evaluation, 1993 *741. Hall JA, Epstein AM <strong>and</strong> McNeil BJ. Multidimensionality of health status in an elderly population: Construct validity of ameasurement battery. Medical Care 27 (Suppl): S168-S177, 1989742. Hall JA, Milburn MA <strong>and</strong> Epstein AM. A causal model of health status <strong>and</strong> satisfaction with medical care. Medical Care 31: 84-94,1993743. Hall W, Andrews G <strong>and</strong> Goldstein G. The costs of Schizophrenia. Australian <strong>and</strong> New Zeal<strong>and</strong> Journal of Psychiatry 19: 3-5, 1985.744. Hall W, Goldstein G, Andrews G <strong>and</strong> et al. Estimating the economic costs of Schizophrenia. Schizophrenia Bulletin 11 (4): 598-610,1985745. Hankey GJ, Dennis MS, Slattery JM <strong>and</strong> Warlow CP. Why is the outcome of transient ischaemic attacks different in different groups ofpatients? BMJ 306: 1107-1111, 1993746. Harada N, Sofaer S <strong>and</strong> Kominski G. Functional status outcomes in rehabilitation. Medical Care 31: 345-57, 1993747. Harris J. QALYfing the value of life. Journal of Medical Ethics 13: 117-123, 1987748. Harris L <strong>and</strong> Richardson J. Pressure <strong>for</strong> health care re<strong>for</strong>m. MJA 160: 463-5, 1994749. Harris MD. <strong>Outcome</strong>s of care from patient's perspectives. Home Health Care 10 (3): 64-68, 1992750. Harrison RI, Glenn DC, Niesche FW <strong>and</strong> et al. Surgical management of breast cancer: Experience of the Central Sydney Area HealthService Breast X-ray Programme, 1988 - 1991. MJA 160: 617-20, 1994

87751. Hart PM <strong>and</strong> Wearing AJ. Problems of stability <strong>and</strong> change in quality of life research: Implications <strong>for</strong> a longitudinal model ofpersonality, health <strong>and</strong> well-being. Paper presented at the First Conference of the Australian Centre <strong>for</strong> Social Research, Launceston,January 25-27, 1994752. Harwood RH, Gompertz PH <strong>and</strong> Ebrahim S. <strong>Outcome</strong> measures should be relevant. BMJ 304 (6831): 917-918, 4 April 1992753. Hasnain R <strong>and</strong> Garl<strong>and</strong> M. Health care in common, 1990. (UnPub)754. Hays LB. The Health Care Financing Administration <strong>and</strong> the Effectiveness Initiative. In: Effectiveness <strong>and</strong> <strong>Outcome</strong>s in Health Care,edited by Heithoff KA <strong>and</strong> Lohr KN. Washington, DC: National Academy Press, 1990755. Hays RD, Sherbourne CD <strong>and</strong> Mazel RM. The R<strong>and</strong> 36-Item Health Survey 1.0. Health Economics 2: 217-227, 1993756. Hayward MD, Crimmins EM <strong>and</strong> Friedman S. Cognitive functioning changes among the chronically impaired elderly in the UnitedStates. Paper presented to the 7th Meeting of the International Network on Health Expectancy, Canberra, 23-25 February, 1994757. Heal LW <strong>and</strong> Sigelman CK. Methodological issues in measuring the quality of life of individuals with mental retardation. In: <strong>Quality</strong>of life: perspectives <strong>and</strong> issues, edited by Schalock RL <strong>and</strong> Begab MJ. Washington, DC: American Ass on Mental Retardation, 1990758. Health <strong>Services</strong> Research Group, <strong>Outcome</strong>s <strong>and</strong> the management of health care. Canadian Medical Association Journal 147: 1775-80,1992759. Heap MJ, Munglani R, Klinck JR <strong>and</strong> Males AG. Elderly patients' preferences concerning life-support treatment. Anaesthesia 48:1027-1033, 1993760. Heathcote CR <strong>and</strong> McDermid IM. Projections of cohort life expectancy based on weighted least squares method. Paper presented tothe 7th Meeting of the International Network on Health Expectancy, Canberra, 23-25 February, 1994761. Heithoff KA, Lohr KN <strong>and</strong> Rettig RA. Genesis of the Effectiveness Initiative <strong>and</strong> IOM's role. In: Effectiveness <strong>and</strong> outcomes in healthcare: Proceedings of an invitational conference by the Institute of Medicine, edited by Heithoff KA. <strong>and</strong> Lohr KN. Washington, DC:National Academy Press, 1990762. Higgins M, McCaughan D, Griffiths M <strong>and</strong> et al. Assessing the outcomes of nursing care. Journal of Advanced Nursing 17: 561-568,1992763. Hirst C, Furnival CM, Fielding GD <strong>and</strong> Porter AJ. The detection of early breast cancer: three-year results from a diagnostic breastclinic. MJA147: 328-30, 1987764. Holl<strong>and</strong>sworth JG. Evaluating the impact of medical treatment on the quality of life: a 5 year update. Social Science <strong>and</strong> Medicine 26(4): 425-434, 1988765. Holzemer WL, Bakken Henry S, Stewart A <strong>and</strong> Janson-Bjerklie S. The HIV quality audit marker (HIV-QAM): an outcome measure<strong>for</strong> hospitalized AIDS patients. <strong>Quality</strong> of Life Research 2: 99-107, 1993766. Hopkins A. Measuring the quality of medical care, London: Royal College of Physicians, 1990 *767. Hopkins A, Fitzpatrick R, Foster A <strong>and</strong> et al. What do we mean by appropriate health care? <strong>Quality</strong> in Health Care 2: 117-123, 1993768. Horton R. Errors admitted over falsified US cancer data. Lancet 343: 1029, 1994769. Horvath DG. The ethics of medical resource allocation, Canberra: Australian Health Ethics Committee, 1993770. Houston MC. New insights <strong>and</strong> new approaches <strong>for</strong> the treatment of essential hypertension: selection of therapy based on coronaryheart disease risk factor analysis, hemodynamic profiles, quality of life <strong>and</strong> subsets of hypertension. American Heart Journal 117: 4April: 911-950, 1989771. Hugh TB, Chen FC, Hugh TJ <strong>and</strong> et al. Laparoscopic cholecystectomy. A prospective study of outcome in 100 unselected patients.MJA 156 (5): 318-320, 2 March 1992772. Human rights <strong>and</strong> Equal Opportunity Commission, Human rights <strong>and</strong> mental illness: Report of the National Inquiry into the HumanRights of People with Mental Illness, Canberra: AGPS, 1993773. Hume AL. Applying quality of life data in practice. Considerations <strong>for</strong> antihypertensive therapy. Journal of Family Practice 28 (4):403-411, 1989

88774. Hunt SM, McEwen J <strong>and</strong> McKenna SP. Measuring Health Status, London:Croom Helm, 1986775. Hunt SM, McKenna SP, McEwen J <strong>and</strong> et al. The Nottingham Health Profile: subjective health status <strong>and</strong> medical consultations.Social Science <strong>and</strong> Medicine 15A: 221-229, 1981776. Hurley SF. Screening: the need <strong>for</strong> a population register. MJA 153: 310-311, 1990777. Hyl<strong>and</strong> ME. The validity of health assessments: Revolving some recent differences. Journal of Clinical Epidemiology 46: 1019-1023,1993778. Iezzoni LI. Monitoring quality of care: What do we need to know? Inquiry 30: 112-114, 1993779. IOM Core Committee, Promise <strong>and</strong> limitations of effectiveness <strong>and</strong> outcomes research. In: Effectiveness <strong>and</strong> outcomes in health care:Proceedings of an invitational conference by the Institute of Medicine, edited by Heithoff KA <strong>and</strong> Lohr KN. Washington, DC: NationalAcademy Press, 1990780. Irwig L. An approach to evaluating health outcomes. Public Health Bulletin 4 (12): 135-136, 1993781. Irwig L, Tosteson ANA, Gastonis C <strong>and</strong> et al. Guidelines <strong>for</strong> meta-analyses evaluating diagnostic tests. Annals of Internal Medicine120: 667-676, 1994782. Jadad AR <strong>and</strong> McQuay HJ. The measurement of pain. In: <strong>Outcome</strong> measures in orthopaedics, edited by Pynsent PB, Fairbank JCT <strong>and</strong>Carr A. Ox<strong>for</strong>d: Butterworth, 1993783. Jagger C <strong>and</strong> Clarke M. Annual screening of the elderly in the UK: a data source <strong>for</strong> health life expectancy calculations? Paperpresented to the 7th Meeting of the International Network on Health Expectancy, Canberra, 23-25 February, 1994784. Jencks SF. Issues in the use of large data bases <strong>for</strong> effectiveness research, In: Effectiveness <strong>and</strong> outcomes in health care, edited byHeithoff KA <strong>and</strong> Lohr KN. Washington, DC: National Academy Press, 1990 *785. Jenkins CD, Jono RT, Stanton BA <strong>and</strong> et al. The measurement of health-related quality of life: major dimensions identified by factoranalysis. Social Science <strong>and</strong> Medicine 31 (8): 925-931, 1990786. Jenkinson C. Why are we weighting? A critical examination of the use of item weights in a health status measure. Social Science <strong>and</strong>Medicine 32 (12): 1413-1416, 1991787. Jenkinson C, Coulter A, <strong>and</strong> Wright L. Short <strong>for</strong>m 36(SF-36) health survey questionnaire: normative data <strong>for</strong> adults of working age.BMJ 306: 1437-1440, 1993788. Jenkinson C, Fitzpatrick R <strong>and</strong> Argyle M. The Nottingham Health Profile: an analysis of its sensitivity in differentiating illness groups.Social Science <strong>and</strong> Medicine 27: 1411-1414, 1988789. Johnson EW. <strong>Outcome</strong> measures.....a Chimera? American Journal of Physical Medicine <strong>and</strong> Rehabilitation 71 (4): 201, 1992790. Johnson J, Weissman MM <strong>and</strong> Klerman GL. Service utilization <strong>and</strong> social morbidity associated with depressive symptoms in thecommunity. JAMA 267: 1478-83, 1992791. Johnson SH. Nursing outcomes: focusing on results. Dimensions of Critical Care Nursing 11 (1): 4-5, January-February 1992792. Johnston ME, Langton KB, Haynes RB <strong>and</strong> Mathieu A. Effects of computer-based clinical decision support systems on clinicianper<strong>for</strong>mance <strong>and</strong> patient outcome. Annals of Internal Medicine 120 (2): 135-142, 1994793. Johnston RC, Fitzgerald RH, Harris WH, Muller ME <strong>and</strong> Sledge CB. Clinical <strong>and</strong> radiographic evaluation of total hip replacement.The Journal of Bone <strong>and</strong> Joint Surgery 72-A: 161-168, 1990794. Juniper EF, Guyatt GH, Willan A <strong>and</strong> Griffith LE. Determining a minimal important change in a disease-specific quality of lifequestionnaire. Journal of Clinical Epidemiology 47: 81-87, 1994795. Kahn CN. III Policy implications of outcomes research. International Journal of Technology Assessment in Health Care 6: 295-296,1990796. Kane RL <strong>and</strong> Lurie N. Appropriate effectiveness: a tale of carts <strong>and</strong> horses. <strong>Quality</strong> Review Bulletin 322-326, October 1992

89797. Kanouse DE <strong>and</strong> Jacoby I. When does in<strong>for</strong>mation change practitioners' behaviour? International Journal of Technology Assessmentin Health Care 4: 27-33, 1988798. Kantz ME, Harris W, Levitsky K, Ware JE <strong>and</strong> Ross Davies A. Methods <strong>for</strong> assessing condition-specific <strong>and</strong> generic functional statusoutcomes after total knee replacement. Medical Care 30 (Suppl): MS240-MS252, 1992799. Kaplan RM. Health outcome models <strong>for</strong> policy analysis. Health Psychology 8 (6): 723-735, 1989800. Kaplan RM. Behaviour as the central outcome in health care. American Psychologist 45 (11): 1211-1220, November 1990801. Kaplan RM. <strong>Quality</strong> of life assessment <strong>for</strong> health resource allocation. Paper presented to the Harkness Health Conference, Canberra, 8-9 December, 1993802. Kaplan RM <strong>and</strong> Anderson JP. A general health policy model: update <strong>and</strong> applications. Health <strong>Services</strong> Research 23: 2 June: 203-235,1988803. Kaplan RM. <strong>and</strong> Anderson JP. The general health policy model: an integrated approach. In: <strong>Quality</strong> of Life Assessments in ClinicalTrials, edited by Spilker B. New York: Raven Press Ltd, pp 131-149, 1990804. Kaplan RM, <strong>and</strong> Anderson JP. A general health policy model: applications of new public health indicators in studies of aging. In:Reducing Frailty <strong>and</strong> Falls in Older Persons, edited by Weindruch R, Hadley E, Ory M <strong>and</strong> et al, pp 385-411, 1991805. Kaplan RM, Anderson JP, Wu AW. <strong>and</strong> et al. The quality of well-being scale. Applications in AIDS, Cystic Fibrosis <strong>and</strong> Arthritis.Medical Care 27 (3) (Suppl): S27-S43, March 1989806. Kaplan RM, Bush JW <strong>and</strong> Berry CC. Health status: types of validity <strong>and</strong> the index of well-being. Health <strong>Services</strong> Research 11: 478-507, Winter 1976807. Kaplan RM <strong>and</strong> Ernst JA. Do category rating scales produce biased preference weights <strong>for</strong> a health index? Medical Care 21 (2): 193-207, February 1983808. Kaplan RM <strong>and</strong> Hartwell SL. Differential effects of social support <strong>and</strong> social network on physiological <strong>and</strong> social outcomes in men <strong>and</strong>women with type II diabetes mellitus. Health Psychology 6 (5): 387-398, 1987809. Kaplan SH, Greenfield S <strong>and</strong> Ware JE. Assessing the effects of physician-patient interactions on the outcomes of chronic disease.Medical Care 27 (Suppl): S110-S127, 1989810. Kassirer JP. The quality of care <strong>and</strong> the quality of measuring it. NEJM 329 (17): 1263-1265, 1993 *811. Kaufmann MA, Buchmann B, Scheidegger D <strong>and</strong> et al. Severe head injury: should expected outcome influence resuscitation <strong>and</strong> firstdaydecisions? Resuscitation 23: 199-206, 1992812. Kazis LE, Anderson JJ <strong>and</strong> Meenan RF. Effect sizes <strong>for</strong> interpreting changes in health status. Medical Care 27 (Suppl): S178-S189,1989813. Keeler EB, Manning WG <strong>and</strong> Wells KB. The dem<strong>and</strong> <strong>for</strong> episodes of mental health services. Journal of Health Economics 7: 369-392, 1988814. Kerridge R, Glasziou PP <strong>and</strong> Hillman KM. The use of '<strong>Quality</strong>-adjusted life years' (QALYs) to evaluate outcome <strong>and</strong> economicjustification of treatment in intensive care, 1992. (UnPub)815. Kettlewell MGW <strong>and</strong> et al. Surgical review in the 80's: measuring surgical outcomes, (UnPub)816. Kind P. The designs <strong>and</strong> construction of quality of life measures, (UnPub)817. Kind O <strong>and</strong> Gudex CM. Measuring health status in the community: a comparison of methods. Journal of Epidemiology <strong>and</strong>Community Health 48: 86-91, 1994818. Kind P, Roser R <strong>and</strong> Williams A. Valuation of <strong>Quality</strong> of Life: Some psychometric evidence. In: The Value of Life <strong>and</strong> Safety, edited byJones-Lee MW. North-Holl<strong>and</strong> Publishing Company, pp 159-170, 1982819. Kirwan JR. <strong>Outcome</strong> measures in rheumatoid arthritis clinical trials: assessing improvement. Journal of Rheumatology 20 (3): 543-545, 1993820. Kitzhaber JA. The Oregon Basic Health <strong>Services</strong> Act, 1990. (UnPub)

90821. Kitzhaber JA. Summary: the health services prioritization process, 1990 (UnPub)822. Kitzbaher JA. A healthier approach to health care. Issues in Science <strong>and</strong> Technology 7 (2): 59-65, 1991823. Kitzhaber JA. Constructive debate in a real world. Health Management Quarterly First Quarter: 1991824. Kitzhaber JA. Prioritising health services in an era of limits: the Oregon experience. BMJ 307: 373-377, 1993 *825. Mant J. <strong>and</strong> Hicks N. Health status measurement <strong>and</strong> assessment of medical care. International Journal <strong>for</strong> <strong>Quality</strong> in Health Care 8:107-110, 1996826. Klapow JC, Slater MA, Patterson TL, Doctor JN, Atkinson JH <strong>and</strong> Garfin SR. An empirical evaluation of multidimensional clinicaloutcome in chronic low back pain patients. Pain 55: 107-118, 1993827. Klein R. Warning signals from Oregon. BMJ 304: 1457-1458, 1992828. Klerman G, Olfson M, Leon A <strong>and</strong> Weissman MM. Measuring the need <strong>for</strong> mental health care. Health Affairs 24-33, Fall 1992829. Klingman D, Pine PL <strong>and</strong> Simon J. <strong>Outcome</strong>s of surgery under Medicaid. Health Care Financing Review 11: 1-16, 1990830. Knippenberg FCEV <strong>and</strong> de Haes JCJM. Measuring the quality of life of cancer patients: psychometric properties of instruments.Journal of Clinical Epidemiology 41 (11): 1043-1053, 1988831. Koska MT. <strong>Outcome</strong>s research: hospitals face confidentiality concerns. Hospitals 66 (1): 32-34, 5 January 1992832. Krefting L, Warren S <strong>and</strong> Grace M. Measuring long-term outcome after traumatic brain injury. Canadian Journal of Public Health 83(Suppl): S64-S68, 1992833. Kuhse H <strong>and</strong> Singer P. The quality/quantity-of-life distinction <strong>and</strong> its moral importance <strong>for</strong> nurses. International Journal of NursingStudies 26 (3): 203-212, 1989834. Kurtin PS, Ross Davies A, Meyer KB, DeGiacomo JM <strong>and</strong> Kantz ME. Patient-based health status measures in outpatient dialysis.Medical Care 30 (Suppl): MS136-MS149, 1992835. LaPuma J <strong>and</strong> Lawlor EF. <strong>Quality</strong>-adjusted life-years: Ethical implications <strong>for</strong> physicians <strong>and</strong> policy makers. Journal of the AmericanMedical Record Association 263: 21 June 6: 2917-2921, 1990836. Laffel G <strong>and</strong> Blumenthal D. The case <strong>for</strong> using industrial quality management science in health care organisations. JAMA 262: 2869-2873, 1989 *837. Laffel GL, Barnett AI, Finkelstein S <strong>and</strong> Kaye MP. The relation between experience <strong>and</strong> outcome in heart transplantation. NEJM 327(17): 1220-1225, October 1992 *838. L<strong>and</strong>ry FJ, Parker JM <strong>and</strong> Phillips YY. <strong>Outcome</strong> of cardiopulmonary resuscitation in the intensive care setting. Archives of InternalMedicine 152 (11): 2305-2308, November 1992839. Lansky D, Butler JBV <strong>and</strong> Waller FT. Using health status measures in the hospital setting: from acute care to "outcomes management".Medical Care 30: MS57-73, 1992 *840. Ledesert B, Ritchie K <strong>and</strong> Touchon J. Disability due to dementia. Paper presented to the 7th Meeting of the International Network onHealth Expectancy, Canberra, 23-25 February, 1994841. Lee PP <strong>and</strong> Javitt JC. Measuring the benefit <strong>and</strong> value of services. Archives of Ophthalmology 112: 32, 1994842. Leeder S. Valuable health:what do we want, <strong>and</strong> how do we get it? Australian Journal of Public Health 16: 6-14, 1992 *843. Leeder S. The recent Medicare agreement, goals <strong>and</strong> targets, <strong>and</strong> health outcomes. Australian Journal of Public Health 17: 87-88,1993844. Leeder S <strong>and</strong> Lazarus R. Clinical applications of epidemiology. NSW Public Health Bulletin 3: 1-2, 1992845. Lehman AF, Postrado LT, Roth D <strong>and</strong> et al. Continuity of care <strong>and</strong> client outcomes in the Robert Wood Johnson Foundation Programon chronic mental illness. Milbank Quarterly 72: 105-22, 1994

91846. Leigh JP. Gender, firm size, industry <strong>and</strong> estimates of the value-of-life. Journal of Health Economics 6: 255-273, 1987847. Leighton Read J, Quinn RJ <strong>and</strong> Hoefer MA. Measuring overall health: an evaluation of the three important approaches. Journal ofChronic Diseases 40: 7S-21S, 1987848. Lemay P. <strong>Quality</strong> of life - measuring outcomes of pharmaceutical management. Summary of workshop proceedings. Canadian Journalof Public Health 83 (3): s5-s16, May-June 1992849. Lena HF <strong>and</strong> London B. The political <strong>and</strong> economic determinants of health outcomes: a cross-national analysis. International Journalof Health <strong>Services</strong> 23: 585-602, 1993850. Leu RE. Economic evaluation of new drug therapies in terms of improved life quality. Social Science <strong>and</strong> Medicine 21 (10): 1153-1161, 1985851. Levine MN, Gafni A, Markham B <strong>and</strong> MacFarlane D. A bedside decision instrument to elicit a patient's preference concerningadjuvant chemotherapy <strong>for</strong> breast cancer. Annals of Internal Medicine 117: 53-8, 1992852. Lewis CC, Pantell RH <strong>and</strong> Kieckhefer GM. Assessment of children's health status: Field test of new approaches. Medical Care 27(Suppl): S54-S65, 1989853. Lewis G <strong>and</strong> Wilkinson G. Another British disease? A recent increase in the prevalance of psychiatric morbidity. Journal ofEpidemiology <strong>and</strong> Community Health 47: 358-361, 1993854. Li SC, Ioannides-Demos LL, Spicer WJ <strong>and</strong> et al. Prospective audit of aminoglycoside usage in a general hospital with assessments ofclinical processes <strong>and</strong> adverse clinical outcomes. MJA 151 (4): 224-232, 21 August 1989855. Lichter PR. <strong>Outcome</strong>s research <strong>and</strong> its future impact on physician payment re<strong>for</strong>m. Ophthalmology 99 (5): 649-650, May 1992856. Lichter PR. Different providers <strong>and</strong> differing error rates in health care outcomes: cataract co-management at what price?Ophthalmology 100 (4): 445-446, 1993857. Lightfoot RW. Relevance of outcomes research to the practice of rheumatology. Bulletin on the Rheumatic Diseases 41 (8): 1-3, 1992858. Lim LL, Valenti LA, Knapp JC, Dobson AJ, Plotnikoff R <strong>and</strong> et al. A self-administered quality of life questionnaire after acutemyocardial infarction. Journal of Clinical Epidemiology 46: 1249-56, 1993859. Lipscomb J. Time preference <strong>for</strong> health in cost-effectiveness analysis. Medical Care 27: S232-S253, 1989860. Lipton HL <strong>and</strong> Bird JA. Drug utilization review in ambulatory settings: State of the science <strong>and</strong> directions <strong>for</strong> outcomes research.Medical Care 31: 1069-82, 1993861. Lohr KN. <strong>Outcome</strong> measurement: concepts <strong>and</strong> questions. Inquiry 25: 37-50, Spring 1988 *862. Lohr KN. Advances in health status assessment: overview of the conference. Medical Care 27 (3) (Suppl): S1-S11, March 1989 *863. Lohr KN. Applications of health status assessment measures in clinical practice. Medical Care 30 (Suppl): MS1-MS14, 1992 *864. Lomas J, Anderson GM, Domnick-Pierre K <strong>and</strong> et al. Do practice guidelines guide practice? The effect of a consensus statement on thepractice of physicians. NEJM 321 (19): 1306-1311, 9 November 1989 *865. Long AF, Dixon P, Hall R <strong>and</strong> et al. The outcomes agenda: contribution of the UK clearing house on health outcomes. <strong>Quality</strong> inHealth Care 2: 49-52, 1993866. Loome F <strong>and</strong> McKenzie L. The use of QALYs in health care decision making. Social Science <strong>and</strong> Medicine 28 (4): 299-308, 1989867. Lord SR <strong>and</strong> Sinnett PF. Femoral neck fractures: admissions, bed use, outcome <strong>and</strong> projections. MJA 145: 493-496, 1986868. Lovell DJ. Newer functional outcome measurements in juvenile rheumatoid arthritis: a progress report. Journal of Rheumotology 33:(Suppl): 28-31, April 1992869. Ludke RL, Booth B <strong>and</strong> Lewis-Beck JA. Relationship between early readmission <strong>and</strong> hospital quality of care indicators. Inquiry 30: 95-103, 1993870. MacKeigan LD <strong>and</strong> Pathak DS. Overview of health-related quality-of-life measures. American Journal of Hospital Pharmacy 49 (9):2236-2245, September 1992

92871. MacLean DS. <strong>Outcome</strong> <strong>and</strong> cost of family physician's care: Pilot study of three diagnosis-related groups in elderly inpatients. Journal ofthe American Board of Family Practice 6: 588-93, 1993872. Mael<strong>and</strong> JG <strong>and</strong> Havik OE. After the myocardial infarction. A medical <strong>and</strong> psychological study with special emphasis on perceivedillness. Sc<strong>and</strong>inavian Journal of Rehabilitation Medicine (Suppl) 22: 1-87, 1989873. Maklan C. MEDical Treatment Effectiveness Program Poem (MEDTEPP). Milbank Quarterly 68: 170, 1990874. Malycha P. Treatment options <strong>for</strong> breast cancer. Australian Family Physician 22: 35-39, 1993875. Manchester D. Neuroleptics, learning disability, <strong>and</strong> the community: some history <strong>and</strong> mystery. BMJ 307: 184-7, 1993876. Manton KG. Epidemiological , demographic <strong>and</strong> social correlates of disability among the elderly. Milbank Quarterly 67 (Suppl) 2 Pt 1:13-58, 1989877. Martin CM <strong>and</strong> Douglas RM. Getting "value <strong>for</strong> money": measuring the quality <strong>and</strong> outcome of general practice care. MJA 159 (4):253-256, 16 August 1993878. Martin KS, Scheet NJ <strong>and</strong> Stegman MR. Home Health clients: Characteristics, outcomes of care, <strong>and</strong> nursing interventions. AmericanJournal of Public Health 83: 1730-1734, 1993879. Marwick C. Federal Agency focuses on outcomes research. JAMA 270: 164-165, 1993880. Mata GV, Fern<strong>and</strong>ez, RR, Carmona AG <strong>and</strong> et al. Factors related to quality of life 12 months after discharge from an intensive careunit. Critical Care Medicine 20 (9): 1257-1262, 1992881. Mathers C. Health expectancies in Australia 1993: Preliminary results. Paper presented to the 7th Meeting of the International Networkon Health Expectancy, Canberra, 23-25 February, 1994882. Mathers C <strong>and</strong> Robine J. Health expectancy indicators: recommendations <strong>for</strong> terminology. Paper presented to the 7th Meeting of theInternational Network on Health Expectancy, Canberra, 23-25 February, 1994883. McCallum J, Shadbolt B <strong>and</strong> Wang D. Self-rated health <strong>and</strong> survival: a 7-year follow-up study of Australian elderly, 1992. (UnPub)884. Connelly JE, Philbrick JT, Smith GR. Health perceptions of primary care patients <strong>and</strong> their influence on health care utilisation.Medical Care 27 (3 Suppl): S99-S109, 1989885. McCarthy M. Data fraud in US breast cancer trial [letter]. Lancet 343: 908, 1994886. McCarthy PR, Gelber S <strong>and</strong> Dugger DE. <strong>Outcome</strong> measurement to outcome management: the critical step. Administration <strong>and</strong> Policyin Mental Health 21: 59-68, 1993887. McClellan M <strong>and</strong> Brook RH. Appropriateness of care. A comparison of global <strong>and</strong> outcome methods to set st<strong>and</strong>ards. Medical Care 30(7): 565-586, July 1992 *888. McColl M <strong>and</strong> Skinner HA. Measuring psychological outcomes following rehabilitation. Canadian Journal of Public Health 83(Suppl): S12-S18, 1992889. McCormick K. Research: areas of outcome research <strong>for</strong> nursing. Journal of Professional Nursings 8 (2): 77, March- April 1992890. McCorvey E, Wright JT, McKenney JM <strong>and</strong> et al. Does antihypertensive therapy influence quality of life? Clinical Pharmacology <strong>and</strong>Therapeutics 8 (5): 359-364, May 1989891. McCulloch D. Can we measure 'output'? <strong>Quality</strong>-adjusted life years, health indices <strong>and</strong> occupational therapy. British Journal ofOccupational Therapy 54 (6): 219-221, June 1991892. McGorry PD. Paradigm failure in functional psychosis: Review <strong>and</strong> implications. Australian <strong>and</strong> New Zeal<strong>and</strong> Journal of Psychiatry25: 43-55, 1991893. McHorney CA, Ware JE <strong>and</strong> Raczek AE. The MOS 36-item Short-Form health survey (SF-36): II. Psychometric <strong>and</strong> clinical tests ofvalidity in measuring physical <strong>and</strong> mental health constructs. Medical Care 31: 247-263, 1993894. McHorney CA, Ware JE, Rodgers W <strong>and</strong> et al. The validity <strong>and</strong> relative precision of MOS short-<strong>and</strong> long-<strong>for</strong>m health status scales <strong>and</strong>Dartmouth COOP charts: results from the Medical <strong>Outcome</strong>s Study. Medical Care 30 (Suppl): MS253-MS265, 1992

93895. McLean AJ, Harrison PM, Byrne AJ <strong>and</strong> et al. The choice of ulcer healing agent influences duodenal ulcer relapse rate <strong>and</strong> long-termclinical outcome. Australian <strong>and</strong> New Zeal<strong>and</strong> Journal of Medicine 15: 367-374, 1985896. McLean AJ, Ioannides-Demos LL, Horne MK <strong>and</strong> et al. Impact of a pharmacokinetics consultation service on clinical outcomes in anambulatory care epilepsy clinic. American Journal of Hospital Pharmacy 45: 1549-1551, July 1988897. McMahon RP, Proschan M, Geller NL, Stone PH <strong>and</strong> Sopko G. Sample size calculation <strong>for</strong> clinical trials in which entry criteria <strong>and</strong>outcomes are counts of events. Statistics in Medicine 13: 859-70, 1994898. McNeil BJ, Weichselbaum R <strong>and</strong> Pauker SG. Speech <strong>and</strong> survival: Tradeoffs between quality <strong>and</strong> quantity of life in laryngeal cancer.NEJM 305: 982-987, 1981899. Meenan RF <strong>and</strong> Pincus T. Editorial. The status of patient status measures. Journal of Rheumatology 14 (3): 411-414, 1987900. Megivern K, Halm MA <strong>and</strong> Jones G. Measuring patient satisfaction as an outcome of nursing care. Journal of Nursing Care <strong>Quality</strong> 6(4): 9-24, July 1992901. Mendeloff JM <strong>and</strong> Kaplan RM. Are large differences in "Lifesaving" costs justified? A psychometric study of the relative value placedon preventing deaths. Risk Analysis 9 (3): 349-363, 1989902. Merigan TC <strong>and</strong> Byar DP. Sounding Board. NEJM 323 (19): 1341-1348, 1990903. Michel TH. <strong>Outcome</strong> assessment in cardiac rehabilitation. International Journal of Technology Assessment in Health Care 8 (1): 76-84, 1992904. Micossi P, Carbone M, Stancanelli G <strong>and</strong> Fortino A. Measuring products of healthcare systems [letter]. Lancet 341: 1566-1567, 1993905. Mills I. <strong>Outcome</strong> measures - getting there. Health <strong>Services</strong> Journal 16: 822, July 1987906. Minaire P. Disease, illness <strong>and</strong> health: theoretical models of the disablement process. Bulletin of the World Health Organisation 70(3): 373-379, 1992907. Mitchell JB. The role of large data bases in effectiveness research. In: Effectiveness <strong>and</strong> <strong>Outcome</strong>s in Health Care, edited by HeithoffKA <strong>and</strong> Lohr LN. Washington, DC: National Academy Press, 1990908. Moinpour C.McM Feigl P, Metch B <strong>and</strong> et al. <strong>Quality</strong> of life end points in cancer clinical trials: review <strong>and</strong> recommendations. Journalof the National Cancer Institute 81 (7): 484-495, 1989909. Molzahn AE. Should quality of life measures be used to assess quality of care? ANNA Journal 19 (1): 88, February 1992910. Montegue J. <strong>Outcome</strong>s on the upswing. Hospital <strong>and</strong> Health Networks Jan 20: 56, 1994911. Moon M. The economic situation of older Americans: emerging wealth <strong>and</strong> continuing hardship. Pediatriatric Annals 17 (12): Dec:762-764, 1988912. Mor V. Cancer patients' quality of life over the disease course: lessons from the real world. Journal of Chronic Diseases 40 (6): 535-544, 1987913. Morris JN <strong>and</strong> Sherwood S. <strong>Quality</strong> of life of cancer patients at different stages of the disease trajectory. Journal of Chronic Diseases40 (6): 545-553, 1987914. Morris PLP <strong>and</strong> Jones B. Life satisfaction across treatment methods <strong>for</strong> patients with end-stage renal failure. MJA 150 (8): 428-432, 17April 1989915. Morrison GC <strong>and</strong> Gyldmark M. Appraising the use of contingent valuation. Health Economics 1: 233-243, 1992916. Mosteller F, Ware JE <strong>and</strong> Levine S. Final Panel: Comments on the Conference on Advances in Health Status Assessment. MedicalCare 27 (Suppl): S282-S294, 1989917. Moulds RFW. Measurement of cost-effectiveness of drug therapy: a review of the treatment of hypertension. MJA 153 (Suppl): 54-56,6 August 1990918. Muir Gray JA. Two classes of creativity - improving systematic reviews. Journal of Epidemiology <strong>and</strong> Community Health 48: 4-5,1994

94919. Mulley AG. Assessing patients' utilities. Can the ends justify the means? Medical Care 27 (3) (Suppl): S269-S281, March 1989920. Mulley AG. Applying effectiveness <strong>and</strong> outcomes research to clinical practice. In: Effectiveness <strong>and</strong> outcomes in health care:Proceedings of an invitational conference by the Institute of Medicine, edited by Heithoff KA <strong>and</strong> Lohr KN. Washington, DC: NationalAcademy Press, 1990921. Mulrow CD, Aguilar C <strong>and</strong> Endicott JE. <strong>Quality</strong> of life changes <strong>and</strong> hearing impairment. A r<strong>and</strong>omized trial. Annals of InternalMedicine 113: 188-194, 1990922. Mulrow CD, Gerety MB, Kanten D, Cornell JE, DeNino LA <strong>and</strong> et al. A r<strong>and</strong>omised trial of physical rehabilitation <strong>for</strong> very frailnursing home residents. JAMA 271: 519-524, 1994923. Murphy DJ <strong>and</strong> Cluff LE. Support: study to underst<strong>and</strong> prognoses <strong>and</strong> preferences <strong>for</strong> outcomes <strong>and</strong> risks of treatments. Journal ofClinical Epidemiology 43:1990924. Murray C. Quantifying the burden of disease. The technical basis <strong>for</strong> disability adjusted life years. 1993 (Unpub)925. Murray LS, Teasdale GM, Murray GD <strong>and</strong> et al. Does prediction of outcome alter patient management? Lancet 341: 1487-91, 1993926. Mutafova M, Maleshkov C <strong>and</strong> Tonkova S. Disability-free life expectancy: a pilot investigation. Paper presented to the 7th Meeting ofthe International Network on Health Expectancy, Canberra, 23-25 February, 1994927. Myers A. The clinical Swiss Army Knife: empirical evidence on the validity of IADL functional status measures. Medical Care 30(Suppl): MS96-MS111, 1992928. Najman J <strong>and</strong> Levine S. Evaluating the impact of medical care <strong>and</strong> technologies on the quality of life: A review <strong>and</strong> critique. SocialScience <strong>and</strong> Medicine 15F: 107-15, 1981929. Nelson E <strong>and</strong> Berwick DM. The measurement of health status in clinical practice. Medical Care 27 (Suppl): S77-S90, 1989 *930. Nelson EC. Using outcome measures to improve care delivered by physicians <strong>and</strong> hospitals. In: Effectiveness <strong>and</strong> outcomes in healthcare: Proceedings of an invitational conference by the Institute of Medicine, edited by Heithoff KA <strong>and</strong> Lohr KN. Washington, DC:National Academy Press, 1990931. Nerenz DR, Repasky DP, Whitehouse FW <strong>and</strong> Kahkonen DM. Ongoing assessment of health status in patients with diabetes mellitus.Medical Care 30 (Suppl): MS112-MS124, 1992932. Nord E. EuroQol: health related quality of life measurement. Valuations of health states by the general public in Norway. HealthPolicy 18: 25-36, 1991933. Nord E. An alternative to QALYs: the saved young life equivalent (SAVE). BMJ 305: 875-877, 1992934. Nord E. Methods <strong>for</strong> quality adjustment of life years. Social Science <strong>and</strong> Medicine 34 (5): 559-569, 1992935. Nord E. Social evaluation of health care versus personal evaluation of health states. International Journal of Assessment in HealthCare 9: 463-478, 1993936. Nord E. Unjustified use of the <strong>Quality</strong> of Well-Being scale in priority setting in Oregon. Health Policy 24: 45-53, 1993937. Nord E. The QALY: A measure of social value rather than individual utility? Health Economics 3: 89-93, 1994938. NSW Health Department, Health care at any cost? 1991 (UnPub)939. NSW Health Department, Getting it right: Focusing on the outcomes of health services <strong>and</strong> programs, Sydney: NSW Health, 1994940. Nusselder WJ, van der Velden J, van Sonsbeek JLA <strong>and</strong> et al. The effect of elimination of selected chronic diseases on the disabilityfreelife expectancy: compression or expansion of morbidity? Preliminary results. Paper presented to the 7th Meeting of theInternational Network on Health Expectancy, Canberra, 23-25 February, 1994941. Olsen J. Time preferences <strong>for</strong> health gains: An empirical investigation. Health Economics 2: 257-265, 1993942. Olshanky SJ <strong>and</strong> Carnes BA. Demographic perspectives on human senescence. Paper presented to the 7th Meeting of the InternationalNetwork on Health Expectancy, Canberra, 23-25 February, 1994

95943. Olweny CLM. <strong>Quality</strong> of life in cancer care. MJA 158: March 15: 429-432, 1993944. Orenstein DM, Pattishall EN, Nixon PA <strong>and</strong> et al. <strong>Quality</strong> of well-being be<strong>for</strong>e <strong>and</strong> after antibiotic treatment of pulmonaryexacerbation in patients with cystic fibrosis. Chest 98: 1081-1084, 1990945. Orentlicher D. Rationing <strong>and</strong> the Americans with Disabilities Act. JAMA 271: 308-14, 1994946. Overy S. Government encouraging doctors to change focus. Australian Doctor 22 April: 25, 1994947. Oxner RBG, Simmonds NJ, Gertner DJ, Nightingdale JMD <strong>and</strong> Burnham WR. Controlled trial of endoscopic injection treatment <strong>for</strong>bleeding from peptic ulcers with visible vessels. Lancet 339: 966-968, 1992948. Pailthorpe CA. <strong>Outcome</strong>s in trauma. In: <strong>Outcome</strong> measures in orthopaedics, edited by Pynsent PB, Fairbank JCT <strong>and</strong> Carr A. Ox<strong>for</strong>d:Butterworth, 1993949. Palmer RM, Saywell RM, Zollinger T <strong>and</strong> et al. The impact of prospective payment system on the treatment of hip fractures in theelderly. Archives of Internal Medicine 149: 2237-22341, 1989950. Panisset M, Roudier M, Saxton J <strong>and</strong> Boller F. Severe Impairment Battery: A neuropsychological test <strong>for</strong> severely demented patients.Archives of Neurology 51: 41-45, 1994951. Parkerson GR, Connis RT, Broadhead WE, Patrick DL, Taylor TR <strong>and</strong> Tse C. Disease-specific versus generic measurement of healthrelatedquality of life in insulin-dependent diabetic patients. Medical Care 31: 629-639, 1993952. Parkerson GR, Deyo RA, Golden WE <strong>and</strong> et al, Strategies <strong>for</strong> improving <strong>and</strong> exp<strong>and</strong>ing the application of health status measures inclinical settings. Medical Care 30: MS210-MS218, 1992953. Parmenter TR. A study of the quality of life of people with severe physical disabilities, 1988 (UnPub)954. Parsonage M <strong>and</strong> Neuburger H. Draft. Discounting <strong>and</strong> QALYs, 1991 (UnPub)955. Parsonage M <strong>and</strong> Neuburger H. Discounting <strong>and</strong> health benefits. Health Economics 1: 71-79, 1992956. Pashos CL <strong>and</strong> McNeil BJ. Consequences of variation in treatment <strong>for</strong> acute myocardial infarction. Health <strong>Services</strong> Research 25 (5):717-722, December 1990957. Patrick DL. Assessing health-related quality of life outcomes. In: Effectiveness <strong>and</strong> <strong>Outcome</strong>s in Health Care, edited by Heithoff KA<strong>and</strong> Lohr KN. Washington, DC: National Academy Press, 1990958. Patrick DL. Strategies <strong>for</strong> improving <strong>and</strong> exp<strong>and</strong>ing the application of health status measures in clinical settings [Discussion]. MedicalCare 30 (Suppl): MS198-MS199, 1992959. Patrick DL <strong>and</strong> Bergner M. Measurement of health status in the 1990s. Annual Review of Public Health 11: 165-83, 1990960. Patrick DL <strong>and</strong> Deyo RA. Generic <strong>and</strong> disease-specific measures in assessing health status <strong>and</strong> quality of life. Medical Care 27(Suppl): S217-S232, 1989961. Petrou S <strong>and</strong> Renton A. The QALY: A guide <strong>for</strong> the public health physician. Public Health 107: 327-336, 1993962. Phelps CE. Death <strong>and</strong> taxes. An opportunity <strong>for</strong> substitution. Journal of Health Economics 7: 1-24, 1988963. Pill R <strong>and</strong> Stott NCH. Development of a measure of potential health behaviour: a salience of lifestyle index. Social Science <strong>and</strong>Medicine 24 (2): 125-134, 1987964. Piper MC. Rehabilitation outcome measures: Where do we go from here? Canadian Journal of Public Health 83: S69-S70, 1992965. Pollack VE, Pesce A <strong>and</strong> Kant KS. Original investigations. Continuous quality improvement in chronic disease: a computerisedmedical record enables description of a severity index to evaluate outcomes in end-stage renal disease. American Journal of KidneyDiseases XIX: 6 June: 514-522, 1992966. Prescott M. <strong>Outcome</strong>s solutions may impede access to health care. <strong>Outcome</strong>s Measurement & Management 4-5, May/June1994967. Pryce-Jones M. Assessing the quality of discharge procedures <strong>for</strong> elderly people. Health <strong>Services</strong> Management 23-26, June 1992

96968. Rad<strong>for</strong>d PJ. General outcome measures. In: <strong>Outcome</strong> Measures in Orthopaedics, edited by Pynsent PB, Fairbank JCT <strong>and</strong> Carr A.Ox<strong>for</strong>d: Butterworth, 1993969. Rantucci MJ <strong>and</strong> Segal HJ. Over-the-counter medication. <strong>Outcome</strong> <strong>and</strong> effectiveness of patient counselling. Journal of Social <strong>and</strong>Administrative Pharmacy 3 (3): 81-91, 1986970. Raphael B. Psychiatry 'at the coal-face'. Australian <strong>and</strong> New Zeal<strong>and</strong> Journal of Psychiatry 20: 316-322, 1986971. Raskin IE <strong>and</strong> Maklan CW. Medical treatment effectiveness research: A view from inside the Agency <strong>for</strong> Health Care Policy <strong>and</strong>Research. Evaluation <strong>and</strong> the Health Professional 161-186, June 1991972. Rawles J, Light J <strong>and</strong> Watt M. Loss of quality adjusted days as a trial endpoint: effect of early thrombolytic treatment in suspectedmyocardial infarction. Journal of Epidemiology <strong>and</strong> Community Health 47: 377-381, 1993973. Regier DA, Hirschfield RMA <strong>and</strong> Goodwin FK. The NIMH Depression Awareness, Recognition, <strong>and</strong> Treatment Program: Structure,aims <strong>and</strong> scientific basis. American Journal of Psychiatry 145: 1351-1357, 1988974. Reinertsen JL. <strong>Outcome</strong>s management <strong>and</strong> continuous quality improvement: the compass <strong>and</strong> the rudder. <strong>Quality</strong> Review Bulletin 19(1): 5-7, January 1993975. Reisine ST. The impact of dental conditions on social functioning <strong>and</strong> the quality of life. Annual Review of Public Health 9: 1-19, 1988976. Relman AS. Assessment <strong>and</strong> accountability: The third revolution in medical care. NEJM 319: 1220-2, 1988 *977. Reverby S. Stealing the golden eggs: Ernest Amory Codman <strong>and</strong> the science <strong>and</strong> management of medicine. Bulletin of the History ofMedicine 55: 156-171, 1981978. Reves Network on Health Expectancy <strong>and</strong> the Disability Process, Introduction to the Network, 1991. (UnPub)979. Revicki DA. Relationship between health utility <strong>and</strong> psychometric health status measures. Medical Care 30 (Suppl): MS274-282, 1992980. Richardson J. Cost utility analyses in health care: present status <strong>and</strong> future issues - Working Paper No. 8 In: Research Into HealthCare: Designs, Dilemmas <strong>and</strong> Disciplines, edited by Daly J, McDonald I <strong>and</strong> Willis E. Sage Publications, 1989981. Richardson J. Cost utility analyses: what should be measured - utility, value or healthy year equivalents? 1989 (UnPub)982. Richardson J. Economic analyses in health care: theory <strong>and</strong> practice, 1990 (UnPub)983. Richardson J. Economic assessment of health care: theory <strong>and</strong> practice. Australian Economic Review 1991984. Richardson J, Schwartz S <strong>and</strong> Glasziou PP. QALYs <strong>for</strong> resource allocation: a reply to Burrows <strong>and</strong> Brown [letter]. Australian Journalof Public Health 17: 394-6, 1993985. Rickard MT, Lee W, Read JW <strong>and</strong> et al. Breast cancer diagnosis by screening mammography: early results of the Central Sydney AreaHealth Service Breast X-ray Programme. MJA 154: 126-131, 1991986. Ries AL, Kaplan RM <strong>and</strong> Blumberg E. Use of factor analysis to consolidate multiple outcome measures in chronic obstructivepulmonary disease. Journal of Clinical Epidemiology 44 (6): 497-503, 1991987. Ringdal GI <strong>and</strong> Ringdal K. Testing the EORTC <strong>Quality</strong> of Life Questionnaire on cancer patients with heterogeneous diagnoses.<strong>Quality</strong> of Life Research 2: 129-140, 1993988. Ritchie K. International comparisons of dementia-free life expectancy: a critical review of the results obtained. Paper presented to the7th Meeting of the International Network on Health Expectancy, Canberra, 23-25 February, 1994989. Rob M. Current health indicators. NSW Public Health Bulletin 2: 7990. Roberts MM, Alex<strong>and</strong>er FE, Anderson TJ <strong>and</strong> et al. Edinburgh trial of screening <strong>for</strong> breast cancer: mortality at seven years. Lancet335: 241-246, 1990991. Roberts RS. Pooled outcome measures in arthritis: the pros <strong>and</strong> cons. Journal of Rheumatology 20 (3): 566-567, 1993992. Robine J. Disability-free life expectancy trends in France: 1981-1991, international comparison. Paper presented to the 7th Meeting ofthe International Network on Health Expectancy, Canberra, 23-25 February, 1994

97993. Robine J, Brouard N, Jagger C, Ritchie K <strong>and</strong> van de Water H. Harmonization of health expectancy in Europe. Paper presented to the7th Meeting of the International Network on Health Expectancy, Canberra, 23-25 February, 1994994. Robine J <strong>and</strong> Mormiche P. L'esperance de vie sans incapacite augmente. Paper presented to the 7th Meeting of the InternationalNetwork on Health Expectancy, Canberra, 23-25 February, 1994995. Robinson R. Cost-effectiveness analysis. BMJ 307: 793-795, 1993996. Rochon PA, Gurwitz JH, Simms RW <strong>and</strong> et al. A study of manufacturer-supported trials of nonsteroidal anti-inflammatory drugs in thetreatment of arthritis. Archives of Internal Medicine 154: 157-163, 1994997. Rockl<strong>and</strong> LH. A review of supportive psychotherapy, 1986-1992. Hospital <strong>and</strong> Community Psychiatry 44: 1053-1060, 1993998. Roel<strong>and</strong>s M <strong>and</strong> van Oyen H. Mental <strong>and</strong> social variable as operationalisation of health in the calculation of health expectancy. Paperpresented to the 7th Meeting of the International Network on Health Expectancy, Canberra, 23-25 February, 1994999. Roel<strong>and</strong>s M, van Oyen H <strong>and</strong> Baro F. Dementia free life expectancy in Belgium. Paper presented to the 7th Meeting of theInternational Network on Health Expectancy, Canberra, 23-25 February, 19941000. Roizen MF, Coalson D, Hayward RSA <strong>and</strong> et al. Can patients use an automated questionnaire to define their current health status?Medical Care 30 (Suppl): MS74-MS84, 19921001. Romieu I <strong>and</strong> Robine J. World atlas on health expectancy calculations. Paper presented to the 7th Meeting of the International Networkon Health Expectancy, Canberra, 23-25 February, 19941002. Rontal R, Kiess MJ, DesHarnais SI <strong>and</strong> et al. Applications <strong>for</strong> risk-adjusted outcome measures. <strong>Quality</strong> Assurance in Health Care 3(4): 283-292, 19911003. Roos LL, Fisher ES, Brazauskas R <strong>and</strong> et al. Health <strong>and</strong> surgical outcomes in Canada <strong>and</strong> the United States. Health Affairs 56-72,Summer 1992 *1004. Roper WL, Winkenwerder W, Hackbarth GM <strong>and</strong> et al. Effectiveness in health care: an initiative to evaluate <strong>and</strong> improve medicalpractice. NEJM 319 (18) November 3: 1197-1202, 19881005. Ross Davies A, Doyle MAT, Lansky D, Stevic MO <strong>and</strong> Doyle JB. <strong>Outcome</strong>s assessment in clinical settings: A consensus statement onprinciples <strong>and</strong> best practices in project management. Joint Commission Journal on <strong>Quality</strong> Improvement 6-16, January 19941006. Rosser R. Issues of measurement in the design of health indicators: a review, 1979. (UnPub)1007. Rosser R, Allison R, Butler C et al. The index of health-related quality of life (IHQL): a new tool <strong>for</strong> audit <strong>and</strong> cost-per-QALY analysis,(UnPub)1008. Rosser R, Allison R, Butler C <strong>and</strong> et al. The index of health-related quality of life (IHQL): a new tool <strong>for</strong> audit <strong>and</strong> cost-per QALYanalysis. In: <strong>Quality</strong> of Life Assessment: Key Issues in the 1990s, edited by Walker SR <strong>and</strong> Rosser RM. Dordrecht, Netherl<strong>and</strong>s:Kluwer Academic Publishers, 19931009. Rosser R <strong>and</strong> Kind P. A scale of valuations of states of illness: is there a social concensus? International Journal of Epidemiology 7(4): 347-357, 19781010. Rost K, Smith G R, Burnam MA <strong>and</strong> Burns BJ. Measuring the outcomes of care <strong>for</strong> mental health problems: The case of depressivedisorders. Medical Care 30 (Suppl): MS266-MS273, 19921011. Rothman ML, Hedrick S <strong>and</strong> Inui T. The Sickness Impact Profile as a measure of the health status of noncognitively impaired nursinghome residents. Medical Care 28 (Suppl): S157-S167, 19891012. Rothman ML <strong>and</strong> Revicki DA. Issues in the measurement of health status in asthma research. Medical Care 31: MS82-MS96, 19931013. Rubenstein LV, Calkins, DR, Greenfield S <strong>and</strong> et al. Health status assessment <strong>for</strong> elderly patients. Report of the Society of GeneralInternal Medicine Task Force on Health Assessment. Journal of the American Geriatrics Society 37: 562-569, 19891014. Rubin G, Frommer M, Morey S <strong>and</strong> Leeder S. On the right track. NSW Public Health Bulletin 2: 1-2, 19911015. Ruta DA, Garratt AM <strong>and</strong> Russell IT. A new approach to the measurement of quality of life: the patient generated index (PGT), 1992(UnPub)

981016. Rutstein DD, Berenberg W, Chalmers TC <strong>and</strong> et al. Measuring the quality of medical care (second revision of tables May 1980): aclinical method. NEJM 294: March 11: 582-588, 19761017. Sackett DL, Chambers LW, Macpherson AS <strong>and</strong> et al. The development <strong>and</strong> application of indices of health: general methods <strong>and</strong> asummary of results. American Journal of Public Health 67: 423-428, 19771018. Sackett DL <strong>and</strong> Torrance GW. The utility of different health states as perceived by the general public. Journal of Chronic Diseases 31:697-704, 19781019. Sacks NPM <strong>and</strong> Baum M. Primary management of carcinoma of the breast. Lancet 342: 1402-1408, 19931020. Sage WM. <strong>Outcome</strong>s research [letter]. NEJM 330: 434-435, 19941021. Salive ME, Mayfield JA <strong>and</strong> Weissman NW. Patient outcomes research teams <strong>and</strong> the agency <strong>for</strong> health care policy <strong>and</strong> research.Health <strong>Services</strong> Research 25 (5): 697-708, December 19901022. Sansoni J. Evaluation <strong>and</strong> management of entrepreneurial education activities [draft], (UnPub)1023. Saravay SM, Steinberg MD, Weinschel B, Pollack S <strong>and</strong> Alovis N. Psychological comorbidity <strong>and</strong> length of stay in the generalhospital. American Journal of Psychiatry 148: 324-329, 19911024. Sarna L. Women with lung cancer: impact on quality of life. <strong>Quality</strong> of Life Research 2: 13-22, 19931025. Saunders CM <strong>and</strong> Baum M. <strong>Quality</strong> of life during treatment <strong>for</strong> cancer. British Journal of Hospital Medicine 48 (2): 119-123, 19921026. Sawyer M G, Sarris A, Baghurst PA, Cornish CA <strong>and</strong> Kalucy RS. The prevalence of emotional <strong>and</strong> behaviour disorders <strong>and</strong> patterns ofservice utilisation in children <strong>and</strong> adolescents. Australian <strong>and</strong> New Zeal<strong>and</strong> Journal of Psychiatry 24: 323-330, 19901027. Schalock RL. Where do we go from here? In: <strong>Quality</strong> of Life: Perspectives <strong>and</strong> Issues, edited by Schalock RL <strong>and</strong> Begab MJ.Washington, DC: American Association on Mental Retardation, 19901028. Schalock RL. Attempts to conceptualize <strong>and</strong> measure quality of life. In: <strong>Quality</strong> of Life: Perspectives <strong>and</strong> Issues, edited by Schalock RL<strong>and</strong> Begab MJ. Washington, DC: American Association on Mental Retardation, 19901029. Schieber GJ, Poullier J <strong>and</strong> Greenwald LM. Health spending, delivery, <strong>and</strong> outcomes in OECD countries. Health Affairs 120-129,Winter 19921030. Schilling RF, El-Bassel N, Gilbert L <strong>and</strong> Glassman M. Predictors of changes in sexual behaviour among on methadone. AmericanJournal of Drug <strong>and</strong> Alcohol Abuse 19: 409-422, 19931031. Schoenbaum SC. An attempt to manage variation in obstetrical practice. In: Effectiveness <strong>and</strong> <strong>Outcome</strong>s in Health Care: Proceedingsof an Invitational Conference by the Institute of Medicine, edited by Heithoff KA <strong>and</strong> Lohr KN. Washington, DC: National AcademyPress, 19901032. Schoenbaum SC. Editorial: Toward fewer procedures <strong>and</strong> better outcomes. JAMA 269 (6): 794-896, 10 February 19931033. Schwartz JS <strong>and</strong> Lurie N. Assessment of medical outcomes. New opportunities <strong>for</strong> achieving a long sought-after objective.International Journal of Assessment in Health Care 6: 333-339, 19901034. Schwartz S. Preferences <strong>for</strong> health outcomes: Factors affecting health state utilities, 1994. (UnPub)1035. Schwartz S, Richardson J <strong>and</strong> Glasziou P. <strong>Quality</strong>-adjusted life years: origins, measurements, applications, objections. AustralianJournal of Public Health 17: 272-278, 19931036. Scottish Cancer Trials Breast Group, <strong>and</strong> ICRF Breast Unit, Adjuvant ovarian ablation versus CMF chemotherapy in premenopausalwomen with pathological stage II breast carcinoma: the Scottish trial. Lancet 341: 1293-1298, 19931037. Secretary of State <strong>for</strong> health, The health of the nation: a strategy <strong>for</strong> health in Engl<strong>and</strong>, 1992. (UnPub)1038. Seedhouse D. Core health services: a fiction? New Zeal<strong>and</strong> Medical Journal 961 (Suppl): 8, 19931039. Shah S, Vanclay F <strong>and</strong> Cooper B. Stroke rehabilitation: Australian patient profile <strong>and</strong> functional outcome. Journal of ClinicalEpidemiology 44: 21-28, 1991

991040. Shah SK, Vanclay F <strong>and</strong> Cooper B. Improving the sensitivity of the Barthel index <strong>for</strong> stroke rehabilitation. Journal of ClinicalEpidemiology 42 (8): 703-709, 19891041. Shapiro E <strong>and</strong> Roos NP. Elderly non-users of health care services: their characteristics <strong>and</strong> their health outcomes, (UnPub), 19951042. Sheldon TA <strong>and</strong> Borowitz M. Changing the measure of quality in the NHS: from purchasing activity to purchasing protocols. <strong>Quality</strong> inHealth Care 2: 149-150, 19931043. Shepherd SL, Hovell MF, Harwood IR <strong>and</strong> et al. A comparative study of the psychosocial assets of adults with cystic fibrosis <strong>and</strong> theirhealthy peers. Chest 97: 1310-1316, 19901044. Shern DL, Wilson NZ, Coen AS <strong>and</strong> et al. Client outcomes II. Longitudinal client data from the Colorado Treatment <strong>Outcome</strong> Study.Milbank Q 72: 123-148, 19941045. Showstack J, Katz P, Amend W <strong>and</strong> et al. The effect of cyclosporine on the use of hospital resources <strong>for</strong> kidney transplantation. NEJM321 (16): 1086-1092, 19 October 19891046. Silver GA. <strong>Outcome</strong>s measures. Lancet 337: 1088-1089, 19911047. Simes RJ <strong>and</strong> Glasziou PP. Meta-analysis <strong>and</strong> quality of evidence in the economic evaluation of drug trials. Pharmaco Economics 1:282-292, 19921048. Singleton N <strong>and</strong> Turner A. SF 36 is suitable <strong>for</strong> elderly patients [letter]. BMJ 307: 126-127, 19931049. Siu AL, McGlynn EA, Morgenstern H <strong>and</strong> et al. Choosing quality of care measures based on the expected impact of improved care onhealth. Health <strong>Services</strong> Research 27 (5): 619-650, December 19921050. Skolnick AA. Joint Commission will collect, publicise outcomes. JAMA 270: 165-171, 19931051. Smith D. Does therapy work? Sydney Morning Herald 1 June 19931052. Smith L. Reflections on the next stage of the epidemiological transition. Paper presented to the 7th Meeting of the InternationalNetwork on Health Expectancy, Canberra, 23-25 February, 19941053. Smith P. Per<strong>for</strong>mance indicators: are they worth it? Audit 74-78, 19871054. Smith R <strong>and</strong> Dobson M. Measuring utility values <strong>for</strong> QALYs: Two methodological issues. Health Economics 2: 349-355, 19931055. Snow CA & Associates Report. Page 7:"It's not blank." Opinions about <strong>and</strong> suggested changes <strong>for</strong> a "quality of life" questionnaire, 1991(UnPub)1056. Sox HC. Effectiveness research <strong>and</strong> changing physician practice patterns. In: Effectiveness <strong>and</strong> <strong>Outcome</strong>s in Health Care: Proceedingsof an Invitational Conference by the Institute of Medicine, edited by Heithoff KA <strong>and</strong> Lohr KN. Washington, DC: National AcademyPress, 19901057. Spiegelhalter DJ, Gore S <strong>and</strong> Fitzpatrick R. <strong>Quality</strong> of life measures in health care.III: resource allocation. BMJ 305 (6863): 1205-1209, 14 November 19921058. Stalfelt A <strong>and</strong> Wadman B. Assessing quality of life in leukaemia: Presentation of an instrument <strong>for</strong> assessing quality of life in patientswith blood malignancies. <strong>Quality</strong> Assurance in Health Care 5: 201-211, 19931059. Stedman T, Haire M <strong>and</strong> Welham J. <strong>Quality</strong> of life <strong>and</strong> tertiary prevention <strong>for</strong> the chronically mentally ill, 1991 (UnPub)1060. Stein R, Gortmaker SL, Perrin EC <strong>and</strong> et al. Severity of illness: concepts <strong>and</strong> measurements. Lancet December 26: 1506-1509, 19871061. Steinberg EP, Bergner M, Sommer A <strong>and</strong> et al. Variations in cataract management: patient <strong>and</strong> economic outcomes. Health <strong>Services</strong>Research 25 (5): 727-731, December 19901062. Stewart AL, Hays RD <strong>and</strong> Ware JE. The MOS Short-Form General Health Survey: reliability <strong>and</strong> validity in a patient population.Medical Care 26: 724-732, 19881063. Stewart AL, Sherbourne CD, Hays RD, Wells KB, Nelson EC, Kamberg CJ, Rogers WH, Berry SH <strong>and</strong> Ware JE. Summary <strong>and</strong>discussion of MOD measures. In: Measuring Functioning <strong>and</strong> Well-Being. The Medical <strong>Outcome</strong>s Study Approach, edited by StewartAL <strong>and</strong> Ware JE,Jr. Durham: Duke University Press, pp 345-403, 1992

1001064. Stockbridge H, Hardy RI <strong>and</strong> Glueck CJ. Public cholesterol screening: motivation <strong>for</strong> participation, follow-up outcome, self-knowledge<strong>and</strong> coronary heart disease risk factor intervention. Journal of Laboratory <strong>and</strong> Clinical Medicine 114 (2): 142-151, 19891065. Studenski S <strong>and</strong> Woods Duncan P. Measuring rehabilitation outcomes. Geriatric Rehabilitation 9: 823-830, 19931066. Swain SM. In situ or localised breast cancer: How much treatment is needed? NEJM 328: 1633-1634, 19931067. Szabadi E. Physical exercise <strong>and</strong> mental health. BMJ 296: 5 March: 659-660, 19881068. Tancredi LR <strong>and</strong> Bovbjerg RR. Advancing the epidemiology of injury <strong>and</strong> methods of quality control: ACEs as an outcomes-basedsystem <strong>for</strong> quality improvement. <strong>Quality</strong> Review Bulletin 201-209, June 19921069. Tanenbaum SJ. What physicians know. NEJM 329: 1268-1271, 1993 *1070. Tanenbaum SJ. <strong>Outcome</strong>s research [letter]. NEJM 330: 435, 19941071. Tarlov AR, Ware JE, Greenfield S <strong>and</strong> et al. The Medical <strong>Outcome</strong>s Study: an application of methods <strong>for</strong> monitoring the results ofmedical care. Journal of the American Medical Record Association 262: 925-930, 1989 *1072. Tatchell M. Measuring hospital output: a review of the service mix <strong>and</strong> case mix approaches. Social Science <strong>and</strong> Medicine 17 (13):871-883, 19831073. Temkin NR, Dikmen S, Machamer J <strong>and</strong> McLean A. General versus disease-specific measures. Medical Care 27 (Suppl): S44-S53,19891074. Testa MA <strong>and</strong> Simonson DC. Management of hypertension <strong>and</strong> nephropathy in diabetes. Measuring quality of life in hypertensivepatients with diabetes. Postgraduate Medical Journal 64 (Suppl) 3: 50-58, 19881075. The Essendon Breast X-ray Program Collaborative Group, A mammographic screening pilot project in Victoria 1988-1990. MJA 157:670-673, 19921076. The Lancet, Breast cancer: clearing trails in the <strong>for</strong>est without losing our way [Editorial]. Lancet 343: 1049-1050, 19941077. Thier SO. Forces motivating the use of health status assessment measures in clinical settings <strong>and</strong> related clinical research. MedicalCare 30 (Suppl): MS15-MS22, 19921078. Thomas S, Steven I, Browning C <strong>and</strong> et al. Focus groups in health research: a methodological review. Annual Review of Health <strong>and</strong>Social Sciences 2: 7-20, 19921079. Thompson JF, McGhan WF, Ruffalo RL <strong>and</strong> et al. Long-term care. Clinical pharmacists prescribing drug therapy in a geriatric setting:outcome of a trial. Journal of the American Geriatrics Society 32: 2 February: 154-159, 19841080. Toevs CD, Kaplan RM <strong>and</strong> Atkins CJ. The costs <strong>and</strong> effects of behavioural programs in chronic obstructive pulmonary disease.Medical Care 22:12 December: 1088-1101, 19841081. Tollefson GD <strong>and</strong> Holman SL. Analysis of the Hamilton Depression rating Scale factors from a double-blind, placebo-controlled trial offluoxetine in geriatric major depression. International Clinical Psychopharmacology 8: 253-259, 19931082. Tollefson GD, Souetre E, Thom<strong>and</strong>er L <strong>and</strong> Potvin JH. Comorbid anxious signs <strong>and</strong> symptoms in major depression: impact onfunctional work capacity <strong>and</strong> comparative treatment outcomes. International Clinical Psychopharmacology 8: 281-293, 19931083. Torrance GW. Social preferences <strong>for</strong> health states: an empirical evaluation of three measurement techniques. Socio-economicPlanning Sciences 10: 129-136, 19761084. Torrance GW. Measurement of health stat utilities <strong>for</strong> economic appraisal. A review. Journal of Health Economics 5: 1-30, 19861085. Torrance GW. Utility approach to measuring health-related quality of life. Journal of Chronic Diseases 40 (6): 593-600, 19871086. Torrance GW, Boyle MH <strong>and</strong> Horwood SP. Application of multi-attribute utility theory to measure social preferences <strong>for</strong> health states,1982. (UnPub)1087. Tu EJ <strong>and</strong> Chen K. Recent changes in active life expectancy in Taiwan. Paper presented to the 7th Meeting of the InternationalNetwork on Health Expectancy, Canberra, 23-25 February, 1994

1011088. Tugwell P <strong>and</strong> Boers M. OMERACT Conference on outcome measures in rheumatoid arthritis clinical trials: introduction. Journal ofRheumatology 20 (3): 528-530, 19931089. Turner JA, Ersek M, Herron L <strong>and</strong> et al. Patient outcomes after lumbar spinal fusions. JAMA 268 (7): 907-911, 19 August 19921090. Tuteur PG. Strategies <strong>for</strong> improving <strong>and</strong> exp<strong>and</strong>ing the application of health status measures in clinical settings. Medical Care 30(Suppl): MS202-MS204, 19921091. UK Clearing House, <strong>Outcome</strong>s briefing, 19931092. US Department of Health <strong>and</strong> Human <strong>Services</strong>, Capping Medicaid prescription drugs may backfire. Research Activities 148:December:1-7, 19911093. US Department of Health <strong>and</strong> Human <strong>Services</strong>, Progress of research on outcomes of health care services <strong>and</strong> procedures, 19911094. Valkonen T, Sihvonen A <strong>and</strong> Lahelma E. Disability-free life expectancy by level od education in Finl<strong>and</strong>. Paper presented to the 7thMeeting of the International Network on Health Expectancy, Canberra, 23-25 February, 19941095. van den Berg Jeths A, Ruwaard D <strong>and</strong> Kramers PGN. Healthy life expectancy in the Dutch document "Public health status <strong>and</strong><strong>for</strong>ecasts". Paper presented to the 7th Meeting of the International Network on Health Expectancy, Canberra, 23-25 February, 19941096. van den Brink W, Leenstra A, Ormel J <strong>and</strong> van de Willige G. Mental health intervention programs in primary care: their scientificbasis. Journal of Affective Disorders 21: 273-284, 19911097. van Oyen H, Taf<strong>for</strong>eau J <strong>and</strong> Roel<strong>and</strong>s M. Regional inequities in health expectancy in Belgium (draft). Paper presented to the 7thMeeting of the International Network on Health Expectancy, Canberra, 23-25 February, 19941098. Vanclay F. Functional outcome measures in stroke rehabilitation. Stroke 22: 105-108, 19911099. Verbrugge LM <strong>and</strong> Balaban DJ. Patterns of change in disability <strong>and</strong> well-being. Medical Care 27 (Suppl ): S128-S147, 19891100. Verbrugge LM, Lepkowski JM <strong>and</strong> Imanaka Y. Comorbidity <strong>and</strong> its impact on disability. Milbank Quarterly 67: 450-484, 19891101. Veronesi U, Luini A, Del Vecchio M <strong>and</strong> et al, Radiotherapy after breast-preserving surgery in women with localized cancer of thebreast. NEJM 328: 1587-1591, 19931102. Von Kroff M, Ustum TB, Ormel J, Kaplan I <strong>and</strong> Sartorius N. Self-report of disability: Reliability <strong>and</strong> validity in an internationalprimary care study, 1993. (UnPub)1103. Ward M, King L, Rubin G <strong>and</strong> Stewart G. Setting a new agenda. NSW Public Health Bulletin 2: 5-6, 19911104. Ware J. Measuring patient function <strong>and</strong> well-being: Some lessons from the Medical <strong>Outcome</strong>s Study. In: Effectiveness <strong>and</strong> <strong>Outcome</strong>sin Health Care, edited by Heithoff KA <strong>and</strong> Lohr KN. Washington, DC: National Academy Press, 1990 *1105. Ware JE. St<strong>and</strong>ards <strong>for</strong> validating health measures: definition <strong>and</strong> content. Journal of Chronic Diseases 40: 473-480, 1987 *1106. Ware JE. Comments on the use of health status assessment in clinical settings. Medical Care 30 (Suppl): MS205-MS209, 1992 **1107. Ware JE <strong>and</strong> Hays RD. Methods <strong>for</strong> measuring patient satisfaction with specific medical encounters. Medical Care 26: 393-402, 1988*1108. Ware JE <strong>and</strong> Sherbourne CD. The MOS 36-Item Short<strong>for</strong>m Health Survey (SF-36): conceptual framework <strong>and</strong> item selection. MedicalCare 30: 473-483, 1992 **1109. Ware JE,Jr, Sherbourne CD <strong>and</strong> Davies AR. 16. Developing <strong>and</strong> testing the MOS 20-item short-<strong>for</strong>m health survey: a generalpopulation application. In: Measuring Functioning <strong>and</strong> Well-Being. The Medical <strong>Outcome</strong>s Study Approach, edited by Stewart AL <strong>and</strong>Ware JE, Jr. Durham: Duke University Press, pp 277-303, 19921110. Wasson J, Keller A, Rubenstein L <strong>and</strong> et al. Benefits <strong>and</strong> obstacles of health status assessment in ambulatory settings. Medical Care 30(Suppl): MS42-MS49, 19921111. Watchel T, Piette J, Mor V <strong>and</strong> et al. <strong>Quality</strong> of life in persons with human immunodeficiency virus infection: measurement by theMedical <strong>Outcome</strong>s Study Instrument. Annals of Internal Medicine 116: 129-137, 19921112. Wennberg JA. AHCPR <strong>and</strong> the strategy <strong>for</strong> health care re<strong>for</strong>m. Health Affairs 67-71, Winter 1992

1021113. Wennberg JE. <strong>Outcome</strong>s research, cost containment <strong>and</strong> the fear of health care rationing. NEJM 323 (17): 1202-1204, 25 October19901114. Wennberg JE. <strong>Outcome</strong>s research, patient preference, <strong>and</strong> the primary care physician. Journal of the American Board of FamilyPractice 4: 365-367, 1991 *1115. Wennberg JE, Barnes BA <strong>and</strong> Zubkoff M. Professional uncertainty <strong>and</strong> the problem of supplier-induced dem<strong>and</strong>. Social Science <strong>and</strong>Medicine 16: 811-824, 19821116. Wennberg JE, Bunker JP <strong>and</strong> Barnes B. The need <strong>for</strong> assessing the outcome of common medical practices. Annual Review of PublicHealth 1: 277-295, 1980 *1117. Westbrook J, Frommer M. <strong>and</strong> Rubin G. What is the difference between quality assurance <strong>and</strong> health outcomes? Public HealthBulletin 4 (12): 133-134, 19931118. Westlie L, Fauchald P, Talseth T, Jakobsen A <strong>and</strong> Flatmark A. <strong>Quality</strong> of life in Norwegian kidney donors. Nephrology, Dialysis,Transplantation 8: 1146-1150, 19931119. White<strong>for</strong>d H. A national mental health policy <strong>for</strong> Australia. MJA 157: 510-511, 19921120. White<strong>for</strong>d H. Future directions <strong>for</strong> mental health services in Australia. Australian Journal of Public Health 16: 350-353, 19921121. Whiteneck GG, Charlifue SW, Gerhart KA <strong>and</strong> et al. Quantifying h<strong>and</strong>icap: a new measure of long-term rehabilitation outcomes.Archives of Physical Medicine <strong>and</strong> Rehabilitation 73 (6): 519-526, June 19921122. WHOQOL Group, Study protocol <strong>for</strong> the World Health Organisation project to develop a <strong>Quality</strong> of Life assessment instrument(WHOQOL). <strong>Quality</strong> of Life Research 2: 153-159, 19931123. Whynes DK <strong>and</strong> Neilson AR. Convergent validity to two measures of quality of life. Health Economics 2: 229-235, 19931124. Wilkins R, Chen J <strong>and</strong> Ng E. Changes in health expectancy in Canada from 1986 to 1991. Paper presented to the 7th Meeting of theInternational Network on Health Expectancy, Canberra, 23-25 February, 19941125. Williams A. The cost-effectiveness approach to the treatment of angina. In: The Management of Angina Pectoris, edited by PattersonD. Castle House Publications pp 131-141, 19871126. Williams A. 200 years is not enough (UnPub), 19891127. Williams GH. Beyond blood pressure control. Effect of antihypertensive therapy on quality of life. American Journal of Hypertension1 (4): Part 2: 363S-365S, October 19881128. Wilson AA, Hartnett M <strong>and</strong> Ferrari P. <strong>Outcome</strong> measurement from the functional status perspective. Home Health Care 10:3: 32-46,19921129. Wilson D, Wakefield M <strong>and</strong> Taylor A. The South Australian Health Omnibus Survey. In<strong>for</strong>matics in Health Care 3: 65-68, 19941130. Wilson RW <strong>and</strong> Drury TF. Interpreting trends in illness <strong>and</strong> disability: health statistics <strong>and</strong> health status. Annual Review of PublicHealth 5: 83-196, 19841131. Wirtschafter DD, Jones KR <strong>and</strong> Thomas JT. Using health care outcomes to improve patient care in the NICU. Joint CommissionJournal on <strong>Quality</strong> Improvement 20: 57-65, 19941132. Woolf SH. Practice guidelines, a new reality in medicine. Archives of Internal Medicine 152: 946-952, 19921133. Worcester MC, Hare DL, Oliver RG, Reid MA <strong>and</strong> Goble AJ. Early programmes of high <strong>and</strong> low intensity exercise <strong>and</strong> quality of lifeafter acute myocardial infarction. BMJ 307: 1244-1247, 19931134. Wu AW, Rubin HR, Mathews WC <strong>and</strong> et al. A health status questionnaire using 30 items from the Medical <strong>Outcome</strong>s Study:preliminary validation in persons with early HIV infection. Medical Care 29: 786-798, 19911135. Yates DW, Wood<strong>for</strong>d M <strong>and</strong> Hollis S. Preliminary analysis of the care of injured patients in 33 British hospitals: first report of theUnited Kingdom major trauma outcome study. BMJ 305 (6856): 737-740, 26 September 19921136. Z<strong>and</strong>er K. Focusing on patient outcome: case management in the 90's. Dimensions of Critical Care Nursing 11 (3): 127-129, May-June 1992

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!