Evidence-based Design of Communication Development ... - Phonak

Evidence, Ethics & Program
Quality
Martyn Hyde, PhD
Mount Sinai Hospital,
University of Toronto, Ontario, Canada
mhyde@mtsinai.on.ca
1

Evidence-based practice:
a totem of the third millenium
• ‘…has acquired the kind of sanctity often
accorded to motherhood, home and the flag…’
Feinstein A R, Horwitz I
Problems in the ‘evidence’ of ‘evidence-based
medicine’
Am J Med 1997;103:529-35
2

growth of 'evidence based'
publications in Medline
number of eb articles
2500
2000
1500
1000
500
0
1993 1995 1997 1999 2001 2003
year
3

Hilarious Evidence
• In a 2004 talk on human rights and abortion,
Hilary Clinton delivered the ultimate insult to
the Bush administration:
‘ WASHINGTON IS AN EVIDENCE-FREE ZONE’
4

Benefits of EBP
• Devaluation of ‘expert’ opinion
Expert opinion is the lowest level of evidence!
old way:
This is how I do it... We’ve always done it this way…
Dr X says this is how to do it…
Our pet expert is better than your pet expert…
new way:
What’s the real question? Is there evidence?
How good is it? Let’s do an evidence review…
5

More benefits of EBP
• Raised awareness of methodology issues
More training in scientific methodology
Critical appraisal as an essential skill
Evidence review as a routine tool
Higher publication standards, editorially and in
primary reviews
6

EBP limitations: evidence
structures with sand foundations?
• MOST primary reports have serious flaws
USPSTF and UNHS
A handful of key evidence reports
CWGCH and communication development
No substantive evidence of effectiveness
Gravel 2 days ago: 6,534 > 318 > 27 papers
7

‘The gem cannot be polished
without friction, nor man perfected
without trials’
Chinese proverb
• Assuming RCT is relevant, ethical, practicable
• Generalizability issues:
Selected subjects, expert process delivery.
Focus: few actions, outcomes, confounders.
Group validity only, findings may not apply to
specific individuals
8

EBP limitations: systematic
reviews and meta-analyses
• Systematic reviews:
Clear question? Right question?
Search quality
Bias in key evidence filters
Subjectivity in quality ratings
From clinical opinion to methodologic opinion?
• Meta-analyses:
Apples, oranges and unspecified fruits
Studies differ in quality and relevance
9

Cannibalism in the EB community
• Primary literature
• Systematic reviews and meta-analyses
• Guidelines for SRs & metas
• SRs of SRs
• Metas of metas
• CPGs
• Guidelines for CPGs
• SRs of CPGs……..
10

‘The pure and simple truth is
rarely pure and never simple’
Oscar Wilde
• Q: What is the prevalence of permanent
childhood hearing impairment?
• A: Well, it depends on impairment severity,
frequency, type, laterality, age at
determination, method and skill of
measurement, racial mix, healthcare
practices, socioeconomic variables…...
11

PCHI prevalence overview
1. Fortnum H, Audiol Med 2003
2. Canadian WG on Childhood Hearing 2004
3
2.5
cases per thousand
2
1.5
1
0.5
0
0 20 40 60 80 100 120
minimum criterion average dB HL
12

‘Everything should be made as
simple as possible, but not simpler’
Albert Einstein
• There is a hierarchy of evidentiability
• Dimensions:
Outcomes:
Provider actions:
Recipient preferences:
Confounding variables:
#, level (metric, ICF), stability
Provider-recipient interactions:
Process duration:
#, complexity, automation
#, diversity
#, strength
#, strength
event < series < continuum
13

Hierarchy examples, for EHDI
• Screening
outcomes simple, few events, no preferences, little interaction,
automated, modest skills…
• Definitive audiometry
outcomes complex, few events, no preferences, little
interaction, some automation, high skills
• Amplification
outcomes complex, event series, more preferences, more
interaction, some automation, high skills
• Communication development
outcomes very complex, action continuum, many preferences,
strong interactions, little automation, high skills
14

EBP for difficult problems
• Clinicians vary in skills for complex problems
• ‘Deconstruct’ intuitive/gestalt behaviour of
successful clinicians
• Piecewise construct validation and modelling
of the care process
• Sophisticated, situation-specific info & clinical
decision support algorithms
• Train others, improve minimum standards
15

Evidence, values and ethics
• Ethical principles are based on societal values
• Values govern outcome choices and weight
• eg: ‘all children deserve equal access to care’
places massive weight on ANY screening
outcome, so the incremental benefit of UNHS
vs targeted screening becomes massive and
outweighs standard benefit/harm arguments
• The entire EBP process is subordinate to ethics
16

EBP, ethics and EHDI programs
• Large programs must conform to EBP zeitgeist
• Ethical principles govern macrostructure
• EBP assists microprocesses
• Programs MUST be evaluable
• Needless practice variation compromises
evaluability and equity of care
• EBP defines standard of care, where applicable
• Individualized care, where EBP not applicable
17

The curious case of EHDI and
the USPSTF…(et al).
• UNHS ===> language status at >3 years
• Systematic review of the wrong question?
• The vision analogy...
• Hearing impairment is the target disorder
• Early improved hearing is a 1 ry outcome, NOT
a proxy or surrogate for language status
• Access to early, effective communication,
…outcome or basic family right…?
• The equity of care argument…
18

Useful evaluation tools
• Primary literature
Omni.ac.uk, Trisha Greenhalgh
• Systematic reviews
upenn ebmr
DARE, ACP Journal Club, Cochrane, Ovid EBM
• CPGs/PPGs
AGREE instrument cluzeau
19

Having two ears and one tongue,
we should listen twice as much as we speak…
Turkish proverb
20