Simulation of Clinical Trials with Mathematical Models ... - CBI
Simulation of Clinical Trials with Mathematical Models ... - CBI
Simulation of Clinical Trials with Mathematical Models ... - CBI
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
<strong>Simulation</strong> <strong>of</strong> <strong>Clinical</strong> <strong>Trials</strong> <strong>with</strong><br />
<strong>Mathematical</strong> <strong>Models</strong>:<br />
Applications to<br />
Comparative Effectiveness Research<br />
November 18, 2010<br />
Badri Rengarajan, M.D.<br />
www.archimedesmodel.com<br />
... a KAISER PERMANENTE Innovation
Goals<br />
• Describe role <strong>of</strong> simulating clinical trials in<br />
comparative effectiveness research (CER)<br />
– Use Archimedes Model as example<br />
• Show examples<br />
– Case studies <strong>of</strong> diabetes treatments<br />
– Describe a tool for comparing >20 interventions for cardiometabolic<br />
risk<br />
2
Topics<br />
• Rationale, mechanisms, and challenges <strong>of</strong> simulating<br />
clinical trials<br />
• Archimedes Model<br />
– Overview<br />
– Validation<br />
– Capabilities and applications<br />
• Case Studies<br />
– DPP expansion and extension<br />
– A-L-L<br />
– Results <strong>of</strong> 20 simulated trials <strong>of</strong> interventions for<br />
cardiometabolic risk<br />
3
Ideally use clinical trials to compare<br />
different interventions<br />
• Not feasible<br />
– Large populations<br />
– Long follow-up times<br />
– High cost, budget constraints<br />
– Ethical issues<br />
– Provider and patient participation<br />
– Large number <strong>of</strong> interventions and options to be studied<br />
• Best available alternative is to simulate the desired<br />
trials<br />
– Design trials<br />
– Replace trials<br />
4
There are several ways to try to predict results<br />
<strong>of</strong> trials, each has limitations<br />
5<br />
Mechanism<br />
Comparison <strong>of</strong> trials for different<br />
treatments<br />
(also tests, devices, procedures)<br />
Registry data<br />
Questionnaires for physicians,<br />
patients<br />
Potential Limitations<br />
• “Apples vs. Oranges” comparisons<br />
̶ Different patient populations<br />
̶ Different interventions<br />
̶ Different endpoints/outcomes<br />
• Confounding factors<br />
• Missing/incomplete data<br />
• Calibration <strong>of</strong> responses<br />
• Inability to verify<br />
PK/PD modeling<br />
Pilot trial<br />
• May not be predictive <strong>of</strong> clinical<br />
outcomes – too far upstream<br />
• Insufficient powering<br />
• Requires time for protocol development,<br />
enrollment, and analysis
<strong>Simulation</strong> methods have their own challenges,<br />
and differ in their capabilities<br />
Potential Challenges<br />
• Limited to current knowledge <strong>of</strong> drug action, efficacy/effectiveness,<br />
safety, dosing<br />
• May not capture <strong>of</strong>f-target toxicities<br />
• Not a substitute for a real trial<br />
Differences Among Methods<br />
• Level <strong>of</strong> accuracy in capturing physiological and pathophysiological<br />
relationships<br />
• Level <strong>of</strong> integration across different organs and disease processes<br />
• Ability to capture clinical care processes<br />
• Ability to capture healthcare system costs<br />
• Ability to capture downstream/ripple effects on outcomes and costs<br />
6
Topics<br />
• Rationale, mechanisms, and challenges <strong>of</strong> simulating<br />
clinical trials<br />
• Archimedes Model<br />
– Overview<br />
– Validation<br />
– Capabilities and applications<br />
• Case Studies<br />
– DPP expansion and extension<br />
– A-L-L<br />
– Results <strong>of</strong> 20 simulated trials <strong>of</strong> interventions for<br />
cardiometabolic risk<br />
7
Archimedes uses mathematics to<br />
enhance decision making<br />
• The Archimedes Model is a mathematical<br />
model <strong>of</strong> human physiology, diseases,<br />
interventions, and healthcare systems<br />
– Physiology-based<br />
– Realistic<br />
– Comprehensive<br />
– <strong>Clinical</strong>ly and administratively detailed<br />
– Rigorously validated<br />
• Enables decision makers to understand likely<br />
outcomes <strong>of</strong> various interventions<br />
8
9<br />
Physiological Pathways are Represented as<br />
Equations in Virtual People<br />
Family<br />
history<br />
Sex<br />
Race/<br />
ethnicity<br />
Age<br />
BMI<br />
Height<br />
Type 1<br />
Diabetes<br />
feature<br />
Type 2<br />
Diabetes<br />
feature<br />
Age, sex,<br />
race/<br />
ethnicity<br />
Weight<br />
Diabetes<br />
blood<br />
pressure<br />
factor<br />
Peripheral<br />
resistance<br />
Cardiac<br />
output<br />
Gliburide<br />
Arterial<br />
compliance<br />
Insulin Insulin<br />
treatment<br />
Systolic<br />
Cardiac<br />
output<br />
blood<br />
pressure<br />
Pulse \<br />
pressure<br />
Insulin<br />
Insulin<br />
Insulin<br />
level level level level level<br />
Unexplained<br />
variance in<br />
OGT<br />
FPG<br />
OGT<br />
Ketoacidosis<br />
Random<br />
plasma<br />
glucose<br />
Insulin<br />
Amount<br />
Insulin<br />
Joint Insulin<br />
Glucose<br />
Random<br />
Amount<br />
Diabetes Amount Insulin<br />
uptake by<br />
error and<br />
feature Amount<br />
muscle uptake by<br />
variation<br />
Amount uptake by<br />
muscle uptake by<br />
Efficiency <strong>of</strong><br />
muscle FPG<br />
insulin use<br />
Glucose muscle uptake by FPG<br />
HbA1c<br />
production muscle<br />
FPG<br />
by by liver Glucose<br />
FPG<br />
production Glucose<br />
Untreated<br />
by production liver<br />
FPG<br />
Glucose insulin level<br />
Care<br />
liver<br />
insulin level<br />
processes<br />
Metformin<br />
Normal liver by production<br />
liver Unexplained<br />
Family<br />
Insulin<br />
OGT<br />
OGTT test<br />
Gliburide<br />
Insulin level<br />
variance<br />
GlucoseUntreated<br />
in<br />
history<br />
treatment<br />
glucose<br />
OGT<br />
production productionby production<br />
liver insulin Untreated level<br />
Random<br />
Random<br />
Diabetes<br />
plasma<br />
plasma<br />
Sex<br />
diagnosis<br />
Normal liver<br />
Fractional<br />
Type 1<br />
glucose<br />
glucose test<br />
Insulin<br />
Diabetes<br />
production<br />
insulin level<br />
feature Joint (Pancreas) Glucose<br />
glucose Normal liver by liver Untreated change in<br />
Race/<br />
Random<br />
FPG test<br />
Diabetes<br />
uptake by<br />
ethnicity<br />
error and<br />
Type 2 feature<br />
muscle<br />
Insulin<br />
variation<br />
Insulin<br />
Diabetes<br />
efficiency production FPG<br />
glucose<br />
insulin level<br />
feature<br />
(Muscle)<br />
Glucose<br />
HbA1c test<br />
Age<br />
Normal production liver<br />
Untreated<br />
Insulin<br />
by liver<br />
efficiency<br />
Untreated<br />
Care<br />
Urine<br />
Diet and<br />
Age, sex,<br />
(Liver) glucose insulin level<br />
processes insulin ketone test level<br />
BMI<br />
race/<br />
Normal liver<br />
To<br />
exercise<br />
FPG<br />
ethnicity<br />
glucose<br />
treatment<br />
production<br />
Normal liver Fractional<br />
Ketoacidosis<br />
models<br />
Metformin<br />
change in<br />
UKPDS<br />
Insulin<br />
data<br />
glucose<br />
Height Weight<br />
Diet and<br />
Hypoglycemia<br />
FPG<br />
exercise<br />
LDL<br />
production Diabetes Propensity<br />
HDL<br />
Triglyceride<br />
Patient<br />
Diabetes<br />
takes action<br />
LDL<br />
cholesterol cholesterol s<br />
Smoking Diabetes Propensity<br />
blood<br />
HDL<br />
Triglyceride<br />
cardiac risk<br />
Blurred<br />
to blurred<br />
Smoking cardiac risk to blurred<br />
pressure cholesterol cholesterol s<br />
vision<br />
factor<br />
vision<br />
factor<br />
factor vision<br />
Mean<br />
arterial<br />
pressure<br />
Peripheral<br />
resistance<br />
Mean<br />
arterial<br />
pressure<br />
Arterial<br />
compliance<br />
Systolic<br />
blood<br />
pressure<br />
Pulse \<br />
pressure<br />
To the<br />
Retinopathy<br />
model<br />
Coronary<br />
artery<br />
stenosis<br />
Coronary<br />
To the To the<br />
Nephropath artery Neuropathy<br />
y model model<br />
stenosis<br />
FPG<br />
To the<br />
Coronary<br />
artery<br />
disease<br />
model<br />
Propensity<br />
to polyuria<br />
Propensity<br />
to fatigue<br />
Propensity<br />
to thirst<br />
Polyuria<br />
Fatigue<br />
Thirst<br />
Memory<br />
Perception<br />
Propensity<br />
to polyuria<br />
Propensity<br />
to fatigue<br />
OGTT test<br />
Random<br />
plasma<br />
glucose test<br />
FPG test<br />
HbA1c test<br />
Urine<br />
ketone test<br />
Hypoglycemia<br />
Blurred<br />
vision<br />
Polyuria<br />
Fatigue<br />
Diabetes<br />
diagnosis<br />
To<br />
treatment<br />
models<br />
Patient<br />
takes action<br />
Memory<br />
Perception<br />
To the<br />
Retinopathy<br />
model<br />
To the<br />
Nephropath<br />
y model<br />
To the<br />
Neuropathy<br />
model<br />
To the CAD<br />
model<br />
Propensity<br />
to thirst<br />
Thirst
Type 1<br />
10<br />
The Model is comprehensive<br />
Family<br />
history<br />
Sex<br />
Race/<br />
ethnicity<br />
Age<br />
BMI<br />
Height<br />
Diabetes<br />
feature<br />
Type 2<br />
Diabetes<br />
feature<br />
Age, sex,<br />
race/<br />
ethnicity<br />
Weight<br />
Diabetes<br />
blood<br />
pressure<br />
factor<br />
Insulin<br />
Gliburide<br />
Insulin level<br />
treatment<br />
Insulin<br />
production<br />
Joint (Pancreas) Glucose<br />
Diabetes<br />
uptake by<br />
feature<br />
muscle<br />
Insulin<br />
efficiency<br />
(Muscle)<br />
Glucose<br />
production<br />
Insulin<br />
by liver<br />
efficiency<br />
(Liver)<br />
Normal liver<br />
glucose<br />
production<br />
Metformin<br />
UKPDS<br />
data<br />
Diet and<br />
exercise<br />
HDL<br />
LDL Triglyceride<br />
cholesterol cholesterol s<br />
Unexplained<br />
variance in<br />
OGT<br />
FPG<br />
Untreated<br />
insulin level<br />
Diabetes<br />
Smoking cardiac risk<br />
factor<br />
Mean Systolic Coronary<br />
Peripheral<br />
arterial blood artery<br />
resistance<br />
pressure pressure stenosis<br />
FPG<br />
OGT<br />
Random<br />
plasma<br />
glucose<br />
Random<br />
error and<br />
variation<br />
Care<br />
processes<br />
Fractional<br />
change in<br />
Insulin<br />
FPG<br />
Propensity<br />
to blurred<br />
vision<br />
Propensity<br />
to polyuria<br />
OGTT test<br />
Random<br />
Diabetes<br />
plasma<br />
diagnosis<br />
glucose test<br />
FPG test<br />
HbA1c test<br />
Urine<br />
ketone test<br />
To<br />
treatment<br />
Ketoacidosis<br />
models<br />
Hypoglycemia<br />
Patient<br />
takes action<br />
Blurred<br />
vision<br />
Memory<br />
Polyuria<br />
Perception<br />
Behaviors<br />
Symptoms<br />
Signs<br />
Diseases<br />
Outcomes<br />
Cardiac<br />
output<br />
Arterial<br />
compliance<br />
Pulse \<br />
pressure<br />
Propensity<br />
to fatigue<br />
Fatigue<br />
To the<br />
To the To the To the Coronary<br />
Retinopathy Nephropath Neuropathy artery<br />
model y model model disease<br />
model<br />
Propensity<br />
to thirst<br />
Thirst
11<br />
The Model is clinically and administratively detailed,<br />
enabling detailed costing and economic analysis<br />
Patient:<br />
• Has chest pain and presents to the ER<br />
• Receives EKG, chest x-ray, and blood tests<br />
• Is diagnosed <strong>with</strong> MI, admitted to the hospital, and given an angioplasty <strong>with</strong> drug-eluting stent<br />
• Remains in the hospital for 2.1 days and is discharged<br />
Who is reimbursed Service performed Codes<br />
Cardiologist<br />
Reimbursement<br />
MS-DRG 247: Percutaneous<br />
Cardiovascular Procedure <strong>with</strong><br />
Consultation<br />
Read EKG<br />
Perform cardiac catheterization<br />
Drug-Eluting Stent<br />
Perform angioplasty <strong>with</strong> drug-eluting stent<br />
•2008 Medicare base payment:<br />
$10,365<br />
CPT<br />
CPT (pr<strong>of</strong>essional component only)<br />
CPT<br />
CPT<br />
Radiologist Read x-ray CPT (pr<strong>of</strong>essional component only)<br />
Anesthesiologist Provide anesthesia during angioplasty RVG code and time patient is anesthetized<br />
(can be converted to a CPT)<br />
•Medicare average length <strong>of</strong> stay:<br />
2.1 days<br />
Other physicians Inpatient consultations CPT<br />
Hospital All services performed during stay DRG
The Model spans several diseases<br />
• Coronary artery disease<br />
• Stroke<br />
• Diabetes and complications<br />
• Obesity<br />
• Metabolic syndrome<br />
• Hypertension<br />
• Dyslipidemia<br />
• Congestive heart failure<br />
• Atrial fibrillation<br />
• Asthma<br />
• Lung cancer<br />
• Breast cancer<br />
• Colon cancer<br />
Includes a model <strong>of</strong> family pedigrees,<br />
enabling studies <strong>of</strong> risk-prediction,<br />
genetic and genomic testing,<br />
and cost-effectiveness <strong>of</strong> screening<br />
strategies.<br />
12
Over 50 trials have been used to validate<br />
the Model<br />
4S DCCT Lifetime Risk <strong>of</strong> Diabetes SEATTLE<br />
ADVANCE DPP Look-AHEAD SHEP<br />
AGE DREAM MERIT SOLVD<br />
ALLHAT<br />
ANBP2<br />
European Orlistat<br />
Obesity Study<br />
FHS<br />
MicroHOPE<br />
Minnesota FOBT<br />
Screening Trial<br />
ARIC Flechtner-Mors National Polyp Study TNT<br />
STENO-2<br />
SYSTEUR<br />
ASCOTT LLA GLAI PPP TRACE<br />
ATBC HOPE PROactive UKPDS 33<br />
CAPRICORN HPS PROSPER UKPDS 34<br />
CARDS IDEAL RENAAL UKPDS 45<br />
CARE IDNT RIO-Europe UKPDS 74<br />
CPS-II Breast<br />
Lieberman Colonoscopy<br />
Screening<br />
RIO-Lipids UKPDS 80<br />
CPS-II Colon LIFE all RIO-North America VALUE<br />
CPS-II Lung LIFE diabetes SAVE WESDR<br />
13
The Model can reveal health and economic effects <strong>of</strong><br />
multiple interventions under different scenarios<br />
• Can include active comparators <strong>of</strong> interest, e.g.<br />
– Multiple drugs (diagnostics, devices)<br />
– Different regimens, schedules, frequencies<br />
– Different care protocols and guidelines<br />
• Can create and tailor specific relevant populations,<br />
settings and costs<br />
– Populations<br />
• Demographics, behaviors, test results, symptoms, past medical histories,<br />
current conditions, current treatments<br />
• All at individual level<br />
– Protocols and practice patterns<br />
– Costs<br />
• Tests, treatments, visits, hospitalizations, procedures<br />
14
The Model can enhance trial design,<br />
CER, and product strategy<br />
• <strong>Clinical</strong> Trial Design<br />
– Appropriate powering (<strong>with</strong> knowledge <strong>of</strong> baseline/control arm event<br />
rates)<br />
– Observations on primary and secondary endpoint responses<br />
– Patient population segmenting<br />
– Examination <strong>of</strong> different permutations <strong>of</strong> I/E criteria, trial size, timelines,<br />
treatment effects<br />
• Comparative Effectiveness<br />
– Comparison on health outcomes<br />
– Comparison on costs<br />
– Examination <strong>of</strong> different permutations <strong>of</strong> patient subsets, practice<br />
guidelines, treatment compliance, clinical guideline adherence, costs<br />
• Product Strategy (Indication statement, Positioning, Pricing, etc.)<br />
15
Topics<br />
• Rationale, mechanisms, and challenges <strong>of</strong> simulating<br />
clinical trials<br />
• Archimedes Model<br />
– Overview<br />
– Validation<br />
– Capabilities and applications<br />
• Case Studies<br />
– DPP expansion and extension<br />
– A-L-L<br />
– Results <strong>of</strong> 20 simulated trials <strong>of</strong> interventions for<br />
cardiometabolic risk<br />
16
Only simulation modeling could have enabled<br />
expansion and extension <strong>of</strong> the DPP trial<br />
Situation<br />
• American Diabetes Association (ADA) sought to<br />
understand cost-effectiveness <strong>of</strong> screening and<br />
management guidelines to prevent/delay<br />
development <strong>of</strong> T2DM in high-risk individuals<br />
• Three-year DPP (Diabetes Prevention Program) trial<br />
comparing current care, metformin, and lifestyle<br />
modification was nearly complete<br />
• However, ADA and Department <strong>of</strong> Health and Human<br />
Services (HHS) were interested in long-term health<br />
and economic outcomes <strong>of</strong> different strategies, as<br />
well as several questions outside scope <strong>of</strong> DPP trial<br />
• Existing trial had already cost over $300M<br />
17
The approach involved matching populations &<br />
protocols, adding an arm, and extending duration<br />
Approach<br />
• Created a simulated population matching DPP<br />
inclusion/exclusion criteria and patient baseline<br />
characteristics<br />
• Conducted prospective simulation <strong>of</strong> DPP trial (same<br />
duration, interventions) to validate Model’s ability to<br />
reproduce population, interventions, and outcomes<br />
• Added intervention arm (lifestyle intervention<br />
initiated after diagnosis – FPG >125)<br />
• Extended duration <strong>of</strong> simulated trial to 30 years<br />
18
19<br />
This enabled generation <strong>of</strong> longitudinal health<br />
data on a larger population<br />
Results<br />
Actual Trial<br />
Simulated Trial<br />
Size N=3,234 N=10,000<br />
Mean trial duration 3.2 years 30 years<br />
Rates <strong>of</strong> diabetes • Current care: 28.9%<br />
• Metformin: 21.7%<br />
• Lifestyle: 14.4%<br />
30-year data<br />
27.4%<br />
21.9%<br />
13.2%<br />
Risk <strong>of</strong> DM 72%<br />
Benefits <strong>of</strong> Lifestyle Program (over entire period)<br />
Risk reduction<br />
Risk <strong>of</strong> serious complication<br />
(high-risk group)<br />
Risk <strong>of</strong> death from DM<br />
complication<br />
Cost per QALY (vs. no<br />
intervention)<br />
Down 15% (relative)<br />
38% to 30%<br />
13% to 11%<br />
$143K
Cumulative Incidence <strong>of</strong> Diabetes<br />
20<br />
The simulation was prospectively validated<br />
against the original trial<br />
0.5<br />
DPP: Diabetes Progression<br />
0.45<br />
0.4<br />
0.35<br />
0.3<br />
0.25<br />
0.2<br />
0.15<br />
0.1<br />
0.05<br />
0<br />
Current<br />
28.9% 27.4%<br />
0 0.5 1 1.5 2 2.5 3 care 3.5 4<br />
Time (years)<br />
3-Yr<br />
Timepoint<br />
Actual<br />
control<br />
metformin<br />
lifestyle<br />
Simulated<br />
Metformin 21.7% 21.9%<br />
Lifestyle 14.4% 13.2%
In simulation, the DPP trial was simultaneously<br />
expanded (fourth arm) and extended (30 yrs)<br />
Fraction developing diabetes<br />
0.8<br />
0.7<br />
Effect <strong>of</strong> Four Programs on Progression to Diabetes<br />
Postpone one decade<br />
Prevent 11%<br />
0.6<br />
0.5<br />
0.4<br />
21<br />
0.3<br />
0.2<br />
0.1<br />
0<br />
Baseline<br />
LifeStyle<br />
Metformin<br />
Lifestyle when FPG>125<br />
0 5 10 15 20 25 30<br />
Time (years)<br />
21
Estimating longitudinal health outcomes<br />
generated CE insight (Baseline vs. Lifestyle)<br />
Without Lifestyle (baseline)<br />
Difference <strong>with</strong> Lifestyle<br />
Years <strong>of</strong> follow-up 10 20 30 10 20 30<br />
Diabetes 56.9% 68.6% 72.2% -14.3% -11.6% -10.8%<br />
CAD//CHF<br />
Have an MI 4.0% 8.5% 12.0% -0.4% -1.1% -1.7%<br />
Develop CHF (systolic or diastolic) 0.2% 0.7% 1.2% -0.1% -0.1% -0.1%<br />
Stroke (ischemic or hemorrhagic) 2.9% 7.0% 11.6% -0.5% -1.0% -1.4%<br />
Some serious complication 11.2% 26.1% 38.2% -3% -6.6% -8.4%<br />
Deaths<br />
22% 17% decrease<br />
CHD 2.2% 6.6% 11.9% -0.6% -1.1% -2.0%<br />
Stroke 0.4% 0.9% 1.5% -0.1% -0.3% -0.3%<br />
Renal disease 0.00% 0.02% 0.1% 0.00% -0.01% -0.04%<br />
Death from any complication 2.6% 7.6% 13.5% -0.7% -1.3% -2.3%<br />
Life Years 24.032 0.288<br />
22<br />
22
Estimating longitudinal cost outcomes also<br />
generated CE insight (Baseline vs. Lifestyle)<br />
Expected Costs in a Health Plan <strong>with</strong> 100,000 Members<br />
$Millions<br />
23<br />
Without Lifestyle<br />
(Baseline)<br />
Difference <strong>with</strong> Lifestyle<br />
Years <strong>of</strong><br />
follow-up 5 10 20 30 5 10 20 30<br />
Admissions $10 $23 $58 $96 $0.8 $0.7 $1.5 $2.2<br />
Visits $8.3 $16 $33 $48 -$0.4 -$11 -$1.7 -$2<br />
Procedures $7.4 $16 $38 $57 -$0.8 -$2 -$4 -$5.5<br />
Interventions $3.4 $6.6 $16 $26 $14 $26 $48 $64<br />
Total $29 $62 $144 $227 $14 $24 $44 $59<br />
PMPM for<br />
high-risk $57 $50 $46 $41<br />
PMPM for all<br />
members $2.29 $2.00 $1.83 $1.64<br />
23
30-year QALY/person<br />
The cost effectiveness <strong>of</strong> each arm<br />
over 30 years was revealed<br />
11.50<br />
QALY vs Cost for Four Programs<br />
11.48<br />
11.46<br />
$201,800<br />
11.44<br />
11.42<br />
11.40<br />
11.38<br />
11.36<br />
11.34<br />
11.32<br />
$24,523 $35,523<br />
$62,600<br />
No Program<br />
Lifestyle when FPG>125<br />
DPP Lifestyle<br />
Metformin<br />
24<br />
11.30<br />
$37,000 $39,000 $41,000 $43,000 $45,000 $47,000 $49,000<br />
30-year Cost/person<br />
24
<strong>Simulation</strong> enabled trial expansion and<br />
extension at a fraction <strong>of</strong> the cost <strong>of</strong> a real trial<br />
25<br />
Results<br />
Actual Trial<br />
Simulated Trial<br />
Size N=3,234 N=10,000<br />
Trial arms 3 3 + 1<br />
Mean trial duration 3.2 years 30 years<br />
Rates <strong>of</strong> diabetes • Current care: 28.9%<br />
• Metformin: 21.7%<br />
• Lifestyle: 14.4%<br />
27.4%<br />
21.9%<br />
13.2%<br />
Trial cost >$300M
Topics<br />
• Rationale, mechanisms, and challenges <strong>of</strong> simulating<br />
clinical trials<br />
• Archimedes Model<br />
– Overview<br />
– Validation<br />
– Capabilities and applications<br />
• Case Studies<br />
– DPP expansion and extension<br />
– A-L-L<br />
– Results <strong>of</strong> 20 simulated trials <strong>of</strong> interventions for<br />
cardiometabolic risk<br />
26
A-L-L: <strong>Clinical</strong> trial simulation enabled testing<br />
<strong>of</strong> a novel hypothesis<br />
Situation<br />
• A group <strong>of</strong> physicians at Kaiser Permanente<br />
suggested that a combination <strong>of</strong> aspirin, lovastatin,<br />
and lisinopril would reduce complications <strong>of</strong><br />
diabetes<br />
• However, there was no direct evidence and no way<br />
to determine effects on health outcomes and costs<br />
• Yet, data would be needed to support adoption<br />
27
The approach involved matching populations, care<br />
protocols, and costs<br />
Approach<br />
• Created a simulated population representing all<br />
diabetics over 55 years in Kaiser Permanente (CA)<br />
• Constructed trial <strong>with</strong> three groups:<br />
– Current care (based on national guidelines and realistic<br />
performance and compliance)<br />
– A-L-L regimen on top <strong>of</strong> Current Care<br />
– Reference group: 1% HbA1c reduction<br />
• Used KP’s costs for drugs, lab tests, <strong>of</strong>fice visits, and<br />
admissions for MIs and strokes<br />
• Set duration <strong>of</strong> simulated trial at 30 years<br />
********<br />
• KP’s Care Management Institute conducted an<br />
independent real trial (2 years) to evaluate A-L-L<br />
28
29<br />
A-L-L has a bigger effect than 1%<br />
HbA1c reduction<br />
0.045<br />
0.04<br />
0.035<br />
0.03<br />
Average annual risk <strong>of</strong> various events<br />
Nothing<br />
HbA1c control<br />
ALL<br />
0.025<br />
0.02<br />
0.015<br />
0.01<br />
0.005<br />
0<br />
MI Stroke ESRD Blind Dying
30<br />
The effect begins immediately<br />
0.12<br />
Annual risk <strong>of</strong> four complications or death<br />
0.1<br />
0.08<br />
Nothing<br />
ALL<br />
HbA1c<br />
0.06<br />
0.04<br />
0.02<br />
0<br />
0 5 10 15 20 25<br />
Time since start <strong>of</strong> program
31<br />
CMI Independent Evaluation (2007)<br />
• In 2004-2005 28% <strong>of</strong> KP’s eligible study population in northern +<br />
southern CA (n=170,024) had received A-L-L at low exposure, 13%<br />
at high exposure (59% no exposure)<br />
• Modeled results consistent <strong>with</strong> actual findings by CMI:<br />
– By 2006, heart attacks and strokes decreased by 15 per 1000<br />
members for the low exposure group (p
Topics<br />
• Rationale, mechanisms, and challenges <strong>of</strong> simulating<br />
clinical trials<br />
• Archimedes Model<br />
– Overview<br />
– Validation<br />
– Capabilities and applications<br />
• Case Studies<br />
– DPP expansion and extension<br />
– A-L-L<br />
– Results <strong>of</strong> 20 simulated trials <strong>of</strong> interventions for<br />
cardiometabolic risk<br />
32
A population dataset <strong>with</strong> pre-run interventions serves as a reference tool<br />
for development and implementation <strong>of</strong> novel approaches/medications<br />
Alternate <strong>Simulation</strong> Tool for CE: Cardio-Metabolic Risk Dataset<br />
Population<br />
Cardiometabolic<br />
interventions<br />
Treatment<br />
scenarios<br />
Settings<br />
• 100,000 individuals<br />
• Drawn from NHANES data 1999-2006<br />
• Subpopulations by age, gender, risk factors (e.g., CAD, DM)<br />
• Weight management<br />
• BP management<br />
• Cholesterol management<br />
• Medications (e.g., aspirin)<br />
• Glycemia management<br />
• Smoking cessation<br />
• Guidelines followed perfectly<br />
• Medications initiated at study start <strong>with</strong> no future adjustments<br />
• “Magic pills” - Medications that maintain outcome goal (e.g., LDL
The results <strong>of</strong> a variety <strong>of</strong> interventions can be<br />
captured<br />
Interventions<br />
Target Populations<br />
Weight Management<br />
Lower BMI to 30 BMI > 30<br />
Lower BMI to 25 BMI > 25<br />
Lower BMI by 5% not to go below 25 BMI > 25<br />
Take weight loss intervention<br />
As per guidelines<br />
Blood pressure management<br />
Lower BP to 130/80 SBP > 130 or DBP > 80<br />
Take BPC meds (ACE-I/ARB, diuretic, beta blocker, CCB) As per guidelines<br />
Cholesterol management<br />
Lower LDL to 100 LDL > 100<br />
Lower LDL to 130 LDL > 130<br />
Lower LDL to 160 LDL > 160<br />
Take statin<br />
As per guidelines<br />
Take medication<br />
Take polypill<br />
Is diagnosed <strong>with</strong> diabetes or CAD<br />
Take ASA<br />
Has had MI or ischemic stroke<br />
Glycemia management<br />
Lower FPG to 125 mg/dL<br />
FPG > 125 mg/dL<br />
Lower HbA1c to 9% Is diagnosed <strong>with</strong> diabetes and HbA1c > 9%<br />
Lower HbA1c to 8% Is diagnosed <strong>with</strong> diabetes and HbA1c > 8%<br />
Lower HbA1c to 7% Is diagnosed <strong>with</strong> diabetes and HbA1c > 7%<br />
Take diabetes medication (metformin, glitazone,<br />
As per guidelines<br />
sulfonylurea, insulin)<br />
Lower FPG by 10% not to go below 100%<br />
FPG between 100 and 125 mg/dL<br />
Lower FPG <strong>with</strong> Metformin<br />
FPG between 100 and 125 mg/dL<br />
Smoking Cessation<br />
Stop smoking<br />
Smokers<br />
34
Health and cost outcomes can be captured<br />
Outcomes in Dataset<br />
Biomarkers<br />
Health<br />
CE<br />
• Proportion smokers, BMI, weight, SBP, DBP, LDL, HDL, total cholesterol,<br />
triglycerides, FPG, HbA1c, serum creatinine<br />
• MI, stroke, MACE, revascularization, diabetes, diabetic proliferative<br />
retinopathy, bilateral blindness, ESRD, diabetic foot ulcer, diabetic foot<br />
amputation.<br />
• Total deaths, CHD deaths, stroke deaths<br />
• Medication usage: aspirin, antihypertensives, ACE-inhibitor, beta blocker,<br />
CCB, diuretics, dyslipidemia medications, oral DM agents, metformin,<br />
sulfonylurea, TZD, insulin, weight loss interventions, polypills<br />
• Life years<br />
• Medical costs<br />
• QALYs, Discounted QALYs<br />
• Cost/QALY<br />
• Cost per averted event<br />
35
<strong>Clinical</strong> Trial <strong>Simulation</strong> modeling is an essential<br />
component <strong>of</strong> comparative effectiveness analysis<br />
• Overcomes cost, time, and<br />
recruiting/operational challenges <strong>of</strong> real<br />
trials<br />
• Improves trial design<br />
• Generates data quickly<br />
• Provides preview <strong>of</strong> real trial outcomes<br />
• Facilitates testing <strong>of</strong> unlimited scenarios<br />
• With integration <strong>of</strong> physiology/disease<br />
<strong>with</strong> healthcare processes and costs, can<br />
provide CE insights and evidence<br />
Meaningful<br />
component <strong>of</strong><br />
CE Toolkit,<br />
but not a<br />
substitute for<br />
real trials<br />
36
Appendix<br />
www.archimedesmodel.com<br />
... a KAISER PERMANENTE Innovation<br />
37
Archimedes is a health care modeling<br />
company<br />
• Healthcare Modeling Company<br />
• HQ in San Francisco<br />
• Core technology - Archimedes Model<br />
– <strong>Mathematical</strong> model <strong>of</strong> human physiology,<br />
diseases, interventions and healthcare<br />
systems<br />
• Highly detailed<br />
• Carefully validated<br />
– In development since 1993<br />
• David Eddy MD, PhD<br />
• Len Schlessinger PhD<br />
– Owned by Kaiser Permanente<br />
• Spun out as independent organization 2006<br />
38
Publications<br />
• Age at initiation and frequency <strong>of</strong> screening to detect type 2<br />
diabetes: a cost-effectiveness analysis<br />
[»The Lancet, 3/30/2010 ]<br />
• Preventing Myocardial Infarction and Stroke <strong>with</strong> a Simplified<br />
Bundle <strong>of</strong> Cardioprotective Medications<br />
[ »Am. Journal <strong>of</strong> Managed Care, 10/1/2009]<br />
• Effect <strong>of</strong> Smoking Cessation Advice on Cardiovascular Disease<br />
[ »Am. Journal <strong>of</strong> Medical Quality, Vol. 24, No. 3, (2009)]<br />
• The Relationship between Insulin Resistance and Related<br />
Metabolic Variables to Coronary Artery Disease: A<br />
<strong>Mathematical</strong> Analysis<br />
[ »Diabetes Care Publish Ahead <strong>of</strong> Print, 11/18/2008 ]<br />
• The potential effects <strong>of</strong> HEDIS performance measures on the<br />
quality <strong>of</strong> care<br />
[ »Health Affairs, 9/15/2008 ]<br />
• The Impact <strong>of</strong> Prevention on Reducing the Burden <strong>of</strong><br />
Cardiovascular Disease<br />
[ »Circulation, 7/29/2008 ]<br />
• Validation <strong>of</strong> Prediction <strong>of</strong> Diabetes by Archimedes and<br />
Comparison <strong>with</strong> Other Predicting <strong>Models</strong>.<br />
[ »Diabetes Care, 5/28/2008 ]<br />
• The Metabolic Syndrome and Cardiovascular Risk: Implications<br />
for <strong>Clinical</strong> Practice.<br />
[ »International Journal <strong>of</strong> Obesity, 5/1/2008 ]<br />
• Diabetes Risk Calculator: A Simple Tool for Detecting<br />
Undiagnosed Diabetes and Prediabetes.<br />
[ »Diabetes Care, 5/1/2008 ]<br />
39<br />
• Cure, Care, and Commitment: What Can We Look Forward To?<br />
[ »Diabetes Care, 4/15/2008 ]<br />
• Reflections on science, judgment, and value in evidence-based<br />
decision making: a conversation <strong>with</strong> David Eddy<br />
[ »Health Affairs, 6/19/2007 ]<br />
• Medical Decision-making: Why it must, and how it can, be<br />
improved<br />
[ »Expert Voices, 5/15/2007 ]<br />
• Linking Electronic Medical Records To Large-Scale <strong>Simulation</strong><br />
<strong>Models</strong>: Can We Put Rapid Learning On Turbo?<br />
[ »Health Affairs, 1/26/2007 ]<br />
• Accuracy versus transparency in Pharmacoeconomic<br />
modeling: finding the right balance.<br />
[ »Pharmacoeconomics, 6/6/2006 ]<br />
• Bringing health economic modeling to the 21st century.<br />
[ »Value in Health, 5/30/2006 ]<br />
• <strong>Clinical</strong> outcomes and cost-effectiveness <strong>of</strong> strategies for<br />
managing people at high risk for diabetes.<br />
[ »Annals <strong>of</strong> Internal Medicine, 8/16/2005 ]<br />
• Earlier intervention in type 2 diabetes: The case for achieving<br />
early and sustained glycaemic control.<br />
[ »International Journal <strong>of</strong> <strong>Clinical</strong> Practice, 11/28/2005 ]<br />
• Evidence-based medicine: a unified approach.<br />
[ »Health Affairs, 02/15/2005 ]<br />
• Validation <strong>of</strong> the Archimedes diabetes model.<br />
[ »Diabetes Care, 11/15/2003 ]<br />
• Archimedes: a trial-validated model <strong>of</strong> diabetes.<br />
[ »Diabetes Care, 11/15/2003 ]
The Model is clinically and administratively<br />
detailed in following care processes<br />
40<br />
Modeling health care processes and clinical practice guidelines (e.g. ATP III)
41<br />
The validation approach is rigorous<br />
People<br />
Treatments<br />
Randomized<br />
Controlled<br />
<strong>Trials</strong><br />
Real<br />
Outcomes<br />
Same<br />
(Virtual)<br />
People<br />
Same<br />
(Virtual)<br />
Treatments<br />
Same?<br />
Virtual<br />
Outcomes
Fraction <strong>of</strong> patients<br />
42<br />
The Model has undergone external,<br />
independent validation<br />
0.14<br />
0.12<br />
Major Coronary Events in Heart Protection Study<br />
Solid lines are the trial results<br />
0.1<br />
0.08<br />
Placebo<br />
0.06<br />
0.04<br />
0.02<br />
Treated <strong>with</strong> Simvastatin<br />
0<br />
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5<br />
Years
Fraction <strong>of</strong> patients<br />
43<br />
The Model yielded a reasonable<br />
approximation <strong>of</strong> the HPS Study<br />
0.14<br />
0.12<br />
0.1<br />
0.08<br />
Major Coronary Events in Heart Protection Study<br />
Solid lines are the trial results<br />
Dotted lines are the Model’s results<br />
Placebo<br />
0.06<br />
0.04<br />
0.02<br />
Treated <strong>with</strong> Simvastatin<br />
0<br />
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5<br />
Years
Fraction<br />
44<br />
It has also prospectively predicted CARDS<br />
Trial results<br />
CARDS Trial: Major Coronary Events<br />
0.14<br />
0.12<br />
0.1<br />
CARDS Integrated Hazard: control<br />
Archimedes Integrated Hazard: control<br />
CARDS Integrated Hazard: treated<br />
Archimedes Integrated Hazard: treated<br />
0.08<br />
0.06<br />
0.04<br />
0.02<br />
0<br />
0 1 2 3 4<br />
Years
In the DPP simulation, populations and interventions<br />
were matched, and identical outcomes were set up<br />
45<br />
• The people: “high risk <strong>of</strong> diabetes”<br />
– BMI >24<br />
– FPG 5.3 – 6.9 mmol/L = 95 – 125 mg/dL<br />
– OGTT 7.8 – 11 mmol/L = 140 – 199 mg/dL<br />
• The interventions<br />
– Metformin (originally $742/year, now $260/year)<br />
• Reduces FPG<br />
• Reduces LDL and TG<br />
• Retards weight gain<br />
– DPP Lifestyle ($1356 in 1 st year, then $672/year thereafter)<br />
• Reduces weight<br />
• Reduces FPG, BP, LDL, Total Cholesterol<br />
• Raises HDL<br />
– Add Lifestyle after diagnosis <strong>of</strong> diabetes<br />
• Outcomes<br />
– Progression to diabetes<br />
• FPG > 7 mmol/L (125 mg/dl) or OGTT > 11 mmol/L (199 mg/dl)<br />
– Myocardial infarction and stroke<br />
– End stage renal disease, retinopathy<br />
– Quality <strong>of</strong> life<br />
– Costs<br />
45
Net 30-year Cost<br />
(Discounted) to Health<br />
Plan<br />
DPP simulation revealed net<br />
cost <strong>of</strong> intervention<br />
Net Cost to Health Plan (Discounted) <strong>of</strong> two<br />
Programs, as Function <strong>of</strong> Annual Cost <strong>of</strong> Lifestyle<br />
Program<br />
5000<br />
4000<br />
3000<br />
2000<br />
DPP Lifestyle vs No<br />
Program<br />
DPP Lifestyle<br />
is cost neutral<br />
at $100/year<br />
Lifestyle after<br />
diabetes diagnosis<br />
is cost neutral at<br />
Lifestyle after FPG+125<br />
vs. No Program $240/year<br />
1000<br />
0<br />
-1000<br />
0 200 400 600<br />
46<br />
-2000<br />
Annual Cost <strong>of</strong> Lifestyle Program<br />
46
Cost/ QALY<br />
DPP simulation also enables Cost/QALY<br />
calculation over 30 years<br />
Cost/QALY <strong>of</strong> Three Comparisons as Function <strong>of</strong><br />
Cost <strong>of</strong> Lifestyle Program<br />
$250,000<br />
DPP Lifestyle vs. No Program<br />
$200,000<br />
$150,000<br />
Lifestyle after FPG=125 vs. No<br />
Program<br />
DPP Lifestyle vs. Lifestyle after<br />
FPG+125<br />
$100,000<br />
$50,000<br />
47<br />
$0<br />
-$50,000<br />
0 200 400 600<br />
Annual Cost <strong>of</strong> Lifestyle Program<br />
47
48<br />
A-L-L regimen saves more money<br />
$5,000<br />
Average annual cost per person<br />
$4,500<br />
$4,000<br />
$3,500<br />
Nothing<br />
HbA1c control<br />
ALL<br />
$3,000<br />
$2,500<br />
$2,000<br />
$1,500<br />
$1,000<br />
$500<br />
$0<br />
Cost
A-L-L: The savings begin immediately<br />
$6,000<br />
Annual cost per person<br />
$5,000<br />
$4,000<br />
$3,000<br />
$2,000<br />
$1,000<br />
$0<br />
Nothing<br />
ALL<br />
HbA1c<br />
0 5 10 15 20 25<br />
Years after start <strong>of</strong> program<br />
49
Summary <strong>of</strong> Dataset <strong>with</strong> Pre-Run<br />
Interventions<br />
• Community <strong>of</strong> 100,000k<br />
• 8 subpopulations<br />
• 20 intervention arms<br />
• 15 control arms<br />
• 20 time points for following outcomes<br />
– 12 biomarker outcomes<br />
– 51 health outcomes<br />
– 1 lifeyears outcome<br />
– 19 cost effectiveness outcomes<br />
– 83 differences, 83 ratios<br />
50