discrete-event simulation in clinical trials - Institut für Statistik ...

DRUGS CLINICAL TRIAL OPTIMIZATION
USING DISCRETE EVENT SIMULATION, ITS
MODELING AND VALIDATION
25th July
Institut für Statistik
Ludwig-Maximilians-Universität München
Ludwigstr. 33
München
dose
⎛ CL ⎞
C ( t)
= exp ⎜ − t ⎟
V ⎝ V ⎠
Dr.Toni Monleón Getino
Departament of Statistics
y = f ( t , b ) + e
i i i i
= X B +
i i i i
University of Barcelona (Spain) / GORS (Optimization in
Health Research Group)
amonleong@wanadoo.es. Phone: 0034934021560
b
1
n
1
2
ZB
−
− 2
⎡
−1
⎤
L= (2 π) V exp ⎢− ( y − Xθ)' V ( y − Xθ)
⎥
⎣ 2
⎦

ABSTRACT
The possibility of performing complete simulations of
clinical trials, based on pharmacological action models, has
been considered since the advent of the computer era, as
a tool to optimize their practical realisation. Thanks to the
advances in computation technology and in discrete event
simulation tools, today it is possible to perform realistic,
large-scale clinical trial simulations in a regular basis using
suitable tools of simulation.
In this seminar, we illustrate the process of constructing
realistic simulation models based in linear and non-linear
mixed models using SAS and the LeanSim framework.
LeanSim is an object-oriented general purpose simulation
tool, developed in C/C++, with a process-interaction
modelling approach. These characteristics of LeanSim
make it very flexible, facilitating its adaptation to simulate
clinical trials.
Some clinical trials (repeated measures designs) will be
simulated and a methodology to build models of clinical
trials and to simulate, validate and verify statistically them
will be shown. A second part of the talk will be centered in
the statistical and data analysis facets of model validation
and verification, based on the porcentage variation of
likelihood criteria, comparing the conceptual model and the
simulation replications. A last and very incomplete, for the
moment, part of this research is centered in the use of
Bayesian inference to predict the behaviour of drugs in the
organism of a new patient, based in a population model.
2

Schedule
• Clinical trial framework
• Modelling and simulation of clinical trials
• Discrete events integrated simulation
• Open questions
3

What is a clinical trial?
DRUG
Specific effect
Regression to mean
Disease state
Health State
Placebo
Stochastic variations
4

Randomization
5

Clinical trials phases
Phase I: Researchers test a new drug or treatment in a small group of
people for the first time to evaluate its safety, determine a safe dosage range,
and identify side effects.
Phase II: The drug or treatment is given to a larger group of people to see if
it is effective and to further evaluate its safety.
Phase III: The drug or treatment is given to large groups of people to
confirm its effectiveness, monitor side effects, compare it to commonly used
treatments, and collect information that will allow the drug or treatment to be
used safely.
Phase IV: Studies are done after the drug or treatment has been marketed
to gather information on the drug's effect in various populations and any side
effects associated with long-term use.
6

The bio-pharmacological basis of clinical trials
Drug in pharmaceutical form
Compartment models
Particle
Periferic compartment (tissues)
Liver
Central compartment
Biological effects
Disolution
dose ⎛ CL ⎞
C ( t)
= exp ⎜ − t ⎟
V ⎝ V ⎠
( t)
= a exp − α t + b exp − β t + c exp − k
( ) ( ) ( t )
C
01
7

The solution of the DE system model will be
discretized and its parametes estimated by means of
NLMM to contemplate population variations within
and betwen subjetcs.
8

Dk
k
[ exp( −ket)
− exp( −kat
] ei
e a
C
p
= ) +
Cl(
ka
− ke
)
where
k
a
= exp(0,47 + b )
0i
Cl = exp( − 3,22 + b )
b0i∼N(0; 0,03),
b1i∼N(0;0,4)
eij∼N(0; 0,7).
1i
9

Conceptual Models
PK
e a ( −ket) ( −kat)
Cp
= e − e + ei
Cl( ka
−ke)
⎣
⎦
PD
Dk k
⎡
⎤
% of Movility
85,00
84,50
84,00
83,50
83,00
82,50
82,00
81,50
81,00
80,50
80,00
79,50
79,00
78,50
78,00
Y i = X i b + Z i b i + e i
2 4 6 8 10 12 14
Weeks c s n
Repeated measures models
Phase II-IV
EF
( E
e
max
=
Log ( EC )
+ b1
i)
dosis
50
+ dosis
PK/PD
10

Simulation general
framework

Problem
What if?
Solution?
Model
Simulation
The body
12

Modelization/Simulation process
Statistical analysis (p-value,…)
PK models
2 compartent 1st order model, macro-constants, lagtime,
1st order elimination
( − α t ) + b exp ( − t ) + c exp ( k t )
C ( t)
= a exp
β −
01
discrete approximation of the continuous differential equations
n replications
Statistical analysis -
inference (p-value,…)
Monte-Carlo
Simulation
y = f ( t , b ) + e
b
i i i i
= X B +
ZB
i i i i
13

Why simulate a CT?
Extrapolate a clinical trial
Optimize the results
Over parametrized models!
Time extrapolation!
14

Sargent paradigm
15

Validation
‣ The model has been
validated and verified in
function of a fixed end
points (goals).
‣ Setting end points and
establishing the scope
of the model is more
difficult than the
methodology itself.
16

Statistic phases of the process
‣ Validation of the conceptual model:
• Try model
• Parameter estimation or Adjustment
• Validation of the model suppositions:
residual analysis, confidence intervals,
etc
‣ Operational validation: basically
comparation between real data and
simulated data
17

Operational validation
‣ Does the model (conceptual + simulation)
reproduces in a suitable form the real system?
‣ Usual scope: lihelihood – confidence bands:
goodness of fit
• Null hypothesis: “the model is valid”
• Althernative hypothesis: “The model is wrong”
18

Operational validation:
methodological difficulties
‣ Given an
end point, a
perfectly
valid model
will be
rejected if
the patient
size (of
simulation)
is
sufficiently
big
19

Use of mixed models in CTS
‣ Statistical model used
frequently to conceptual
hierarchical (multilevel) of
repeated measures and in
PK/PD (Proc MIXED /
NLMIXED in SAS).
‣ We can model covariance
structure “between” levels and
“within” levels.
‣ We can calculate individual
trajetories for any patient
20

Mixed model: General form
‣ Linear mixed model form:
Y
i
= X β + Z b + ε
‣ Y i vector of observations
‣ X i y Z i design matrix
‣ β fixed parameters vector
‣ b i ~ N(0,D) random vector of
observations
‣ ε i ~ N(0,Σ i ) residual vector
‣ b i y ε i they are independent
i
i
i
i
21

Mixed model
‣ Linear mixed model
Y
i
=X β + Z b + ε
i
i
i
i
‣ Fitted to data:
• Parameter estimation
• Parameter CI
• Tests of goodness of fit
22

Types of simulations in function of “t”:
Continous
Discrete
Discrete events.
The models based on simulation of discreet events have sufficient
flexibility and adequacy to the clinical reality to be used, on the other
way round, the trees of decision and the models of Markov have been
the methods used with major frequency in the evaluations
farmacoeconomics, and very rarely in the simulation of clinical trials
(Caro JJ. Pharmacoeconomics 23(4):323-332 2005 ).
Time (t)
General planning in discrete time events:
INICIALIZATION
t = 0;
Initialize states of the system and the statistical
counters;
Initialize events list;
PRINCIPAL PROGRAM
If simulation time < time rule
Do:
counters;
list;
End;
RESULTS
Determine type of next event;
Advance simulation time
Up date condition system + statistical
Generate future events + put in the event
Compute + print interesting estimations;
23

Introduction to Discrete Event Simulation
www.dmem.strath.ac.uk/~pball/simulation/simulate.html#experimentation
Peter Ball Design Manufacture & Engineering Management University of
Strathclyde p.d.ball@strath.ac.uk
SDL->DEVS->PETRI
A discrete event is something that occurs at an instant of time
There are a number of different ways of representing the logic
within a discrete event simulation model: •Event (patient)
•Activity (Generator, Destroyer, server)
•Process
Detail of the event approach structure (from Kreutzer, 1986) 24

SIMULATION METHODOLOGY:
Statistical pakage SAS (SAS Institute, Cary, NC).
LeanSim environment (Guasch y col., 2003)
LEANSIM
New methodology used
complex
environments.
in
Open and suitable environment.
Discrete events. Library of C+ clases. Class to generate random
multivariate variables bases in the method of Jacobi.
Components for managing simulation resources. System of
synchronization between activities and objects. Kernel of simulation
adapted to support multiple executions. Withdrawal and statistical
analysis of a standard form for all the elements.
Set of classes that allow a representation in a three-dimensional
universe.
LeanGen
LeanEditor
LeanStatistics
25

Integrated models and simulation
Logistics
26

Realistic simulation by means of discrete envents
Hospital 1
Pacient
…
Hospital n
Total poblation considered
Eligible for the study
Non eligible for the study
Registered on the study
Reflected for the study
Ramdomized
Study treatment assignation
No randomized
Alternativ assignation treatment
Withdrawn
Drop out
Queing during visits
Hospital i
T 1
= W(β=2, λ, γ =0)
0 moths
T 2
= fix
2 months+/- 10 days
+/- 10 days
4 months
+/- 10 days
6 months
+/- 10 days
8 meses
Holydays / weekends
N(0, 10)
+/- 10 days
10 months
Time flow
Y ( u n i d a d e s )
Y (unidades)
Riesgo AA
10,3
10,25
10,2
10,15
16
15
14
13
12
11
10
0 2 4 6 8 10
1,2
1
0,8
0,6
0,4
0,2
Pacebo (Conceptual)
10,1
0 2 4 6 8 10
Pacebo (Conceptual)
50 mg (Conceptual)
100 mg (Conceptual)
Placebo (Conceptual)
50 mg (Conceptual)
Modelo de enfermedad
Tiempo (meses)
100 mg (Conceptual)
Atribute variation
Modelo de enfermedad
Tiempo (meses)
0
0 5 10 15 20 25
Tiempo (meses)
Dosis de máximo efecto
Dosis óptima
Complete clinical trial
Models
Simulation:
•Var between-patients,
•Var inter-patient
•Global error
Disease sub-model
Efficacy
submodel
Safety submodel
27

EXAMPLE OF MODELLING AND SIMULATION OF PHASE II-III
CLINICAL TRIAL BASED IN DRUG ACTION AND DISEASE
MODEL
‣N=144 patients (40 per dosing group (+ withdrawn):
placebo, 50 mg or 100 mg). α=0,05 y β=0,80; δ = 3,2 (Y)
‣End point: Searching the most efficient dose to treat
disease (E), but minimizing the risk of adverse events (side
effects).
‣ p-value treatments < 0,05
‣ p-valor adverse events < 0,05
‣Scope of the simulation:
Pharmacological action (PD).
Disease model (E)
Use of different parameters PK/PD of previous clinical
trials (Fase I).
Efficacy.
Safety.
Logistic subjects (number of hospitals recuiting
patients, recuiting time, time between visits, time of last
visit (end of clinical trial) withdrawn, etc).
‣Time schedule: 11 visits, every moth (0,1,2,…, 10 months).
‣First end point: Y (efficacy), RAA (Safety): Risk of adverse
event.
28

STATISTICAL MODELLING
1-Disease progress:
E = 10+ 0,1( b + slope )
0i
b 0i ∼N(0; 1)
slope = oscillations of the disease
in the time = 2
2-Effect of the drug over Y (Efficacy):
EF
( E
e
max
=
Log ( EC )
+ b1
i)
dosis
50
+ dosis
EF = drug effect
b 1i ∼N(0; 1) E max = 0,5
Log(EC50) = 1,5 (100mg); 4,5 (50 mg)
Y = E *(1 + EF * RAF)
+
e ij ∼N(0, 2).
e i
Sub-models
RAF
= 1−
e
−RATE*
tiempo
RAF = effect of the time delay
on the drug action
RATE = 0,5
3-Adverse event risk (Stanski y Jenkins,2004) (Safety):
RAA
e
1+
e
( −k
+ ( c+
b
1i
=
( −k
+ ( c+
b
)* tiempo)
1i
)* tiempo)
+ e
K = moment of rise AA
c = 0,11 (placebo), 0,35 (50 mg) y 0,45 (100 mg).
b 1i ∼N(0; 0,01) e ij ∼N(0; 0,1).
i
4-Logistic characteristics-> Integrated model
•Withdrawns: 20% (5% per visit)
•Stochastic distribution of patients recruitement ∼ Weibull
•Error time during the visit ∼ Normal
•Behaviour of hopitals during recruitement (same number of
patients / time delay)
29

CONCEPTUAL MODEL SIMULATION (Planning)
Generator
N=144
W(β=2, δ, γ=0)
W 1
W
δ =10
2 W 3
δ =15 δ =20
HOSPITAL 1
HOSPITAL 2
HOSPITAL 3
n=6 visits
W i
+ t j
+ N(0, 10) [t = time between visits]
Waiting
Server
p withdrawn
=0,03
.
.
.
10,3
.
.
.
Change in attributes
Pacebo (Conceptual)
Destructor:
End
Destructor:
Withdrawn
Y (unidades)
10,25
10,2
10,15
Modelo de enfermedad
Placebo
10,1
0 2 4 6 8 10
Tiempo (meses)
Treatment
assignation
50 mg
100 mg
Y (unidades)
16
15
14
13
12
11
Pacebo (Conceptual)
50 mg (Conceptual) Dosis de máximo efecto
100 mg (Conceptual)
Dosis óptima
Modelo de enfermedad
10
0 2 4 6 8 10
Tiempo (meses)
Riesgo AA
1,2
1
0,8
0,6
0,4
Placebo (Conceptual)
50 mg (Conceptual)
100 mg (Conceptual)
0,2
0
0 5 10 15 20 25
Tiempo (meses)
30

N=144
Start of CT
W(10)
0 months
Time
Hospital 1
Generator
48 patients
48 patients
48 patients
Hospital 2
+/- 10 days
2 monts 0 months
+/- 10 days 2 months
4 months
+/- 10 days
4 months
6 months
+/- 10 days
8 months
+/- 10 days
10 months
Time
W(15) W(20)
6 months
8 months
Recruiting delay TIME
Hospital 3
0 months
2 months
4 months
Time
6 months
8 months
End CT
10 months
10 months
ECT = TR + TBV (fix) + TEBV + Queing + Calendar + Withdrawns
Queing (restrictions)
Calendars (Hollidays, …)
31

Variation in clinical end points
Efficacy submodel
16
Placebo (Conceptual) 50 mg (Conceptual)
100 mg (Conceptual) Placebo (Simulación)
50 mg (Simulación) 100 mg Simulación
Risk AA Y (unities)
Y (unidades)
Riesgo AA
15
14
13
12
11
10
PVMCBV:
1
0,8
0,6
0,4
0,2
0 2 4 6 8 10
Time (months)
Tiempo (meses)
p= 0,0466 [-0,01136; 0,10472],
VMCBP:
0
λ : 0,78 a un 12%
Safety sub-model
Placebo (Conceptual) 50 mg (Conceptual)
100 mg (Conceptual) Placebo (Simulación)
50 mg (Simulación) 100 mg (Simulación)
0
0 2 4 6 8 10
Tiempo (meses)
Time (months)
0
PVMCBV: p= 0,0230 [-0,0208; 0,06699]
32

Determination of the ending moment of the CT:
The eding moment CT have calculated (Day 0 a Visit 10 month) per
hospital, using 40 simulation replications:
•Hospital 1: 385,525 [381,132; 389,917] days
•Hospital 2: 385,975 [381,169; 390,780] days
•Hospital 3: 393,575 [388,773; 398,376] days
8
7
6
5
4
3
2
1
0
Centro 1
Centro 2
Centro 3
234
264
275
280
287
292
297
302
308
315
321
326
331
336
343
350
363
374
399
Duration od the Clinical trial(days)
Duración del ensayo clínico (días)
Simulation scenarios:
1- Increase 50 % the speed of recruitment respect regard to the initial
model:
•Hospital 1: 381,125 [375,822; 386,427] días
•Hospital 2: 387,475 [381,271; 317,222] días
•Hospital 3: 391,075 [386,617; 395,532] días
2-The recruitment period for the 3 hopitals has been changed for 90
days average:
•Hospital 1: 531,414 [519,611; 543,217] días
•Hospital 2: 532,829 [522,036; 543,622] días
•Hospital 3: 535,780 [525,433; 546,127] días
33

Open questions
‣ Prediction of a new trajectory for a new case
(imcomplete) using the population information in PK
(Bayesian prediction)
Interesting
implications in adjust
the drug dose to the
patient, minimize risk
of toxicity, increase
efficacy
Prior
Posterior
SAS
34

‣Validation/Verification methodologies for
complex simulation models
C oncentración (m g/L)
12
10
8
6
4
2
0
0 2 4 6 8 10 12 14 16 18 20 22 24
Tiempo (h)
Conceptual
Conceptual model
Concentración Real data real
Concentración (mg/L)
12
10
8
6
4
2
Simulation model (IC95%)
0
0 2 4 6 8 10 12 14 16 18 20 22 24
Tiempo (h)
Simulation
Simulación Conceptual
PVLIR
i
LLR − LLRi
100
LLR
=
2
2
35

1
n
1
2
−
− 2
⎡
−1
⎤
L= (2 π) V exp ⎢− ( y −Xθ)' V ( y −Xθ)
⎥
⎣ 2
⎦
%-LL
12
C
C
11 C
C
C
C
C
C
CC
C
C
C
C
10
C
C C
C
C C
C
C C
C
C
C
C C C
C C C
C C C C
C C
C C C
C C
C
CC
C C
C C C
CC
CC
C
C C
C
CC
C C
C
9
C
CC
C C C
C C
C
C C
C
C
C
C
C
CC
8
C C
CC
CC
C C
C C
CC
C
C
C
C C
CC
C C
C
C C
C C
C C C
C
C
C
C
C
7
C C
C
C
C
C
C C
C
C
C
C C
C C C
C C
C C
C C
C
C
CC
C C C
C
C
C
C C
C C C
C
C
C
C C
C
C
C
C
C
C
C
C
C
C
C C
C
C
C
C C
C CC C C
C C
C
C
C
C C
C
C
C
C
C
C 6
C
C
C
C
CC
C C
C
C
C C
C
C
C C
C
C
C C
C
CC
C
C C
C
C C
C
C
C
C
C
CC
C
C
C
C
C C
C
C
C C
C C
C
C
C C
C
C C
C
C
C
C C C C
CC
CC
C
C C
CC C
C
C
C C C
C
C C
C
C
C
CC
C C
C
C C
C
C C
C
C C C
C C
C
C
C
C C
C
C C
C
C C
C
C
C C
CC
C C
C C C
C
C
C
C
CC
C
C
C C
C
C C
C C
C
C
C C
C
C
C C
C
C
C
C
C
C C
C C C
C
5
C
C
C C
C
C C
C C C
C
C C
C
CC
C C
C
C
C
C
C C
C
C C
C
C
C C C
C C
C
C
C
C C C
C C
C
C C C
C
C
C C C C C
C
C
C
C
C
C C
C C
C
C
C
C
C C
C C C
C
C
4 C
C
C
C C
C C
C
C C C
C
C C
C
C
C
C C
C
C C C C C
C C
C
C
C
C
C
C
C
C
C C
C
C
3
C
C
C
C
C
2
C
0 100 200 300 400 500 600 700 800 900 1000
Number of replications
PLOT %-LL %AIC %AIC CCC%AIC
Percentage of PVL between 1000 simulation replicas and conceptual model using the
computacional model (σ= 2,8532) (Range:2.44379-11.8056 for BIC, 2.19090-
11.56709 for AICC, 2.18605- 11.56225 for AIC, 2.09318-11.49733 for -LLR ).
%-LL
35
C
34
C
C
C
C C
33
C
C
C
C
C
C
C
32 C
C C
C CC
C C
C
CC
C C
C C
C
C CC
C C
C
C
C
C C
C
C CC
CC
C
C
C
C
C
C C
C
31
C
CC
CC C
C
C
C
C
C
C
C
C
C
C C
C
C
C
30
C
C C
C
C
C
C
C C C
C C
C C
C
C C C
C
C
C
C
C
C C
CC
C
C
C
C C
C
C
C C C C
C
C
C C
C
C
C C C C
C
C
C
C
C
29
C
C C
C
C C
C C
C
C
C
C
C
C
C
C C
C
C
C C C C
C
C
C
C
CC
C
C
C C C
C
28 C
C
C
C
C
C
C
C C
C C C
C
CC
C
C
C C
C C
C
C
C
C
C
C
C
C C
C
C
C
CC
C
C C
C C
C C
CC
C
C
C
C
C C
C
CC
C
C
C
C C
C
CC C
C
C C
C CC
C
CC
C
C
C
C
CC
C CC
C
C C CC C
C
C
C
C
C
C
C C C
C
C CC
C
C C
C
C C
C
27
C
C
C
C C
C
C
C C
C
CC
C
CC
C
C
CC
C C C C
C
C
C
C
C C
C
C
C
C
C
C
C
C CC
CC
C
C
C C C
C
C
C C C
C C C C
C
C C C
C
C
CC
C
C
C
C
C
C
C
C C C CC C
C
C C
C
C C
C
C
C
C
C
C
C
C C C C C
C
C
C
CC
C C C
C
C C
C C C C
C C
C
C
C
C
C
C
C C C
C
C C C C
C C
C
C
CC
C
C
C
CC
C C C
C C C
26 C C CC
C
C C
C
C C C C
C C C C
C C
C
C CC
C
C
C
C
C
C
C
C
C
25
C C
C C
C
C
C
C
24
σ
0 100 200 300 400 500 600 700 800 900 1000
Number of replications
PLOT %-LL %AIC %AIC CCC%AIC
Percentage of PVL between 1000 simulation replicas and conceptual model increasing
the variability of the global model error (σ=5) in the upper range (Range: 23.9939-
33.9592 for BIC, 24.2873- 34.2680 for AICC, 24.2922- 34.2729 for AIC, 24.4640 -
34.4744 for -LLR )
36

Vielen dank für
Ihre
Aufmerksamkeit
37