(EPA): Quality management in primary care (Executive Summary 35)

10 years
NATIONAL
PRIMARY CARE
RESEARCH AND
DEVELOPMENT
CENTRE
executive summary 35
Background
es
European Practice Assessment (EPA):
Quality management in primary care
While there is substantial research describing both the
development and validation of clinical quality indicators, this is not
the case for indicators describing the organisational aspects and
management of primary care practices. There is general agreement
within Western Europe about the importance of general practice.
However, the structures, financing and role of primary care within
the wider health care system differs considerably between
countries. Whilst the health systems of the member states operated
independently of each other, this was not an issue. However, the
ongoing process of European unification, introducing free
movement between countries for health care services, has resulted
in an increased interest in the need for quality indicators that allow
comparisons of health care facilities. The European Practice
Assessment (EPA) project aimed to develop and validate a set of
indicators and tools describing the organisational aspects of
primary care practices, to enable such comparisons to be made
(Grol et al 2004). This executive summary provides an overview of
this project.
Objectives
1. To develop a framework and quality indicators describing
general practice organization and management.
2. To develop and validate a European Practice Assessment
instrument using this framework and these indicators.
xecutive ummary35
1

2
Countries taking part
esxecutive ummary
Nine European countries took part: Austria; Belgium; France; Germany; Israel;
Netherlands; Slovenia; Switzerland and the United Kingdom (represented by
NPCRDC for England and the Department of Primary Care, University of Wales
Swansea Clinical School for Wales). The research was co-ordinated by the Centre for
Quality of Care Research, WOK at the University of Nijmegen in the Netherlands,
funded by the participating countries and supported by the Bertelsmann Foundation
in Guttersloh, Germany (http://en.bertelsmann-stiftung.de/foundation_kp.html).
How we developed the indicators
In 2002 and 2003 EPA partners in the participating countries utilised expert panels to
develop a set of quality indicators representing primary care practice management.
We identified 171 potential indicators by undertaking a comprehensive review of the
published and grey literature. Grey literature is material which might not be formally
published, such as organizational reports. The findings were reviewed and refined at
a workshop for all participants in 2002. We then undertook a two-round postal Delphi
survey between June 2002 and January 2003 (Engels et al 2005). A Delphi survey is
designed to measure consensus among individual responses and normally consists
of a series of rounds where information is fed back to panel members using
questionnaires. It is the most commonly used method of developing quality indicators.
An expert panel was established in each country, consisting of ten people for each
panel, predominately general practitioners, but also practice managers in the UK and
NL. They were asked to rate all of the indicators for usefulness on a scale of 1 to 9,
with 1 being not useful at all and 9 being very useful. Analyses were based on the
Rand Appropriateness Method (Fitch et al 2001), in which indicators with an average
rating above 7 were considered valid for that panel/country, whereas those with a
national average of 1-3 were considered invalid. All other indicators were rated
equivocal. Only indicators that were rated valid by all six panels were included in the
European set of indicators. We looked at how many indicators were rated valid and
invalid by all countries and per country respectively.
35

executive summary 35
The response rate for the Delphi process was 95%. Sixty-two indicators (37%) were
rated valid by all six panels. None of the indicators were rated invalid by any of the
countries.
Creating a framework for the indictors
High quality practice management was defined in terms of the systems (structures
and processes), which help to deliver good quality patient care. This definition sets
practice management apart from the actual clinical care provided by the practice and
served as the basis for an organizing framework for the indicators. There were five
key areas to this framework: infrastructure, people, information, finance and quality
and safety. Each area was divided into a number of dimensions; for example,
infrastructure included premises and medical equipment. All of the quality indicators
were then allocated to the most appropriate dimension on the basis of discussions
between the research partners.
Developing and validating the instrument
EPA partners in all nine countries took part in this stage of the project (Grol et al 2004).
The 62 indicators were operationalised into 202 questions. These questions were
then allocated to one of 6 questionnaires which constitute the EPA instrument. Four
questionnaires are designed to be self-completed (GP, staff, practice manager, and
patient) whilst an external visitor had to complete the additional two during a practice
visit, an observer checklist and a structured interview with the practice manager.
In addition to these questionnaires, the EPA instrument contained two other quality
assessment tools, EUROPEP and the Maturity Matrix. EUROPEP is an
internationally validated instrument that enables patients to evaluate the quality of
primary care (Wensing et al 2002). The Maturity Matrix Family Practice (TM) is an
optional tool within the EPA instrument. It is a formative educational group designed
to give practice teams new insights into into their organisation and performance. It is
used by the whole practice team with the assistance of a trained facilitator (Elwyn et
al 2004).
3

4
EPA: Quality management in primary care
The EPA instrument was developed in English and translated into relevant languages
by at least two independent translators per country (an EPA partner in cooperation
with an independent translator).
The whole instrument was piloted in 3-5 practices per country in Autumn 2003. Minor
modifications were made as a result of this pilot and then between January and June
2004, data were collected from over 270 practices across the nine European countries
(about thirty practices per country). As we used data collected from practices who
volunteered to take part, rather than from a randomly selected sample, it would be
inappropriate to present comparative data in this report.
What did we find?
The data collected from this larger scale study was then examined to test the
properties of each question. Sixty questions were excluded from our analysis
because:
• They were open to misinterpretation;
• They were duplicates of other questions;
• There was more than 5% missing data for self-completed questionnaires;
• More than 95% of practices in all nine countries achieved the question, which
meant it was not a useful discriminator between practices
The remaining questions were analysed using factor analyses (this reduces the
number of questions to a smaller set of indicators by allowing us to group related
questions together) and reliability analysis (checking to ensure that each dimension
contained the appropriate indicators). This process resulted in a set of 57 indicators,
spread across the five areas of the practice management framework.
Implications for policy and practice
The EPA study has successfully developed and tested a novel and comprehensive
instrument which can be used to compare the organisational elements of general
practice care in different European countries. We think that the instrument is useful for
both formative (as an educational tool to help practices identify areas for
improvement) and summative (to judge how well a practice is doing, usually
compared to other practices) purposes.

The challenges of undertaking a large scale international project such as this should
not be underestimated. The participants from the different countries had a long track
record of working successfully together and good will and commitment was required
to overcome the significant challenges associated with different languages,
understandings, cultures and health systems. Each country made an important
contribution to the research process and to the final product and the participants
gained new insights into their own health systems from discussions with others in the
group and from examining the study findings. As far as the UK is concerned, its long
established system of general practice, mature primary health care teams and
professionalized managerial processes contributed much to the project. In addition,
the NPCRDC’s expertise in developing and using quality indicators was highly valued
by the other partners.
The future of EPA
executive summary 35
The EPA instrument is not a finished product and requires further development and
testing. The focus will be on three areas:
1. Updating the indicators as new evidence or experience comes on line;
2. Using the instrument in a representative sample of practices so that meaningful
comparisons can be made between organizations and countries;
3. Examining ways of supporting practices which make use of the instrument in
order to maximize its potential to improve the performance of primary care
practices.
5

6
EPA: Quality management in primary care
The extent to which the instrument will be used in different countries will depend upon
the local context. In the UK, for example, there is considerable overlap between the
EPA indicators and the structural/organisational indicators within the Quality and
Outcomes Framework of the new General Medical Services contract (BMA, 2004).
Given that this is a high priority to British general practices, it is less likely that the
instrument will be used widely in the UK in the short term. It is, however, more
comprehensive than the contract indicators and will be a useful source of information
when the contract is revised. In contrast, most other European countries are not
currently using organizational indicators and so EPA is likely to be of significant use to
them.
Partners in the nine countries that took part in the EPA, including NPCRDC and the
Bertelsmann foundation, founded the TOPAS-Europe Collaboration in January 2005,
coordinated by Professor Richard Grol from the Centre for Quality of Care Research
in Nijmegen. This collaboration aims to develop, validate and continuously revise and
improve a wide range of indicators, instruments and tools for assessing health care
practices and professionals, representing a range of dimensions of quality of care.
Those who want to use the tools, including EPA, can apply to the TOPAS-Europe
Collaboration to do so. For further information, visit (http://www.topas-europe.org).
References
British Medical Association. Quality and Outcomes Framework, BMA, London, 2004
(Available at: http://www.bma.org.uk)
Elwyn G, Rhydderch M, Edwards A, Hutchings H, Marshall M, and Grol R, Assessing
organisational development in primary medical care using a group based
assessment: The Maturity Matrix. Qual & Saf Healthcare, 2004. 13: p. 287-94.
Engels Y, Campbell S, Dautzenberg M, van den Hombergh P, Brinkmann H,
Szécsényi J et al. Developing a framework of, and quality indicators for, general
practice management in Europe. Family Practice 2005; 22: 215-222.
Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR, Lazaro P et al. The
RAND/UCLA Appropriateness Method User's Manual, RAND, Santa Monica, 2001.
Grol R, Dautzenberg M, Brinkmann H (eds) Quality Management in Primary Care,
Verlag Bertelsmann Stiflung, Gutersloh, 2004.
Wensing M, Vedsted P, Kersnik J, Peersman W, Klingenberg A, Hearnshaw H, et al.
Patient satisfaction with availability of general practice: an international comparison.
International Journal of Quality in Health Care 2002; 14: 111-8.

executive summary 35
Authors (on behalf of the EPA project partners):
• Centre for Quality of Care Research (WOK), Universities of Nijmegen and
Maastricht, the Netherlands: Richard Grol, Yvonne Engels, Maaike
Dautzenberg, Pieter van den Hombergh
• National Primary Care Research and Development Centre, University of
Manchester, UK: Stephen Campbell, Martin Marshall
• Centre for Health Sciences Research, Cardiff University, UK (formerly Swansea
University during the EPA study): Glyn Elwyn
• Swansea University, UK: Melody Rhydderch
• Bertelsmann Foundation, Gütersloh, Germany: Henrik Brinkmann, Andreas
Esche, Jan Böcken
• AQUA Institute, Göttingen, Germany: Joachim Szecsenyi, Ferdinand Gerlach,
Björn Broge, Petra Wippenbeck
• Société de Formation Thérapeutique du Généraliste, Paris, France: Marianne
Samuelson, Hector Falcoff
• Swisspep Institute for Quality and Research in Healthcare, Gümligen,
Switzerland: Beat Künzi, Walter Oswald
• Scientific Society of Flemish General Practitioners, Berchem (Antwerp),
Belgium: Luc Seuntjens, Nicole Boffin
7

EPA: Quality management in primary care
Further information
A book has also been produced - Grol R, Dautzenberg M, Brinkmann H (eds) Quality
Management in Primary Care, Verlag Bertelsmann Stiflung, Gutersloh, 2004. ISBN 3-
89204-790-1. More details are available from the Bertelsmann Foundation at:
http://en.bertelsmann-stiftung.de/media/English_Publications_2005_Flyer.pdf
A report produced by Wensing et al for the EUROPEP group which summarises the
process is available at: http://www.swisspep.ch/pdf/europep.pdf
Study team contact details
For more information about the study please contact the UK project lead Dr Stephen
Campbell at (email: stephen.campbell@manchester.ac.uk, or telephone: 0161 275
7655).
About NPCRDC
NPCRDC is a multi-disciplinary centre, established in 1995 to carry out policy related
research in primary care. Our centre is a collaboration between the Universities of
Manchester and York, with our main base at the University of Manchester. We are
committed to excellence in primary care research and dissemination.
If you would like further copies of this summary, please contact NPCRDC.
National Primary Care Research and Development Centre,
Williamson Building,
The University of Manchester,
Oxford Road, Manchester. M13 9PL.
Telephone: +44-(0)161 275-7634
Fax: +44-(0)161 275-0611.
Email: communit@fs1.man.ac.uk
Full details of the indicators are available to download from: www.npcrdc.ac.uk
esxecutive ummary35
8