Detection of Adverse Drug Reaction Signals in the Thai FDA Database

Waranee Bunchuailua,
BPharm, PhD candidate
Department of Community
Pharmacy, Faculty of
Pharmacy, Silpakorn
University, Nakhon Pathom,
Thailand; Department of
Tropical Hygiene, Faculty of
Tropical Medicine, Mahidol
University, Bangkok,
Thailand
Ilene H. Zuckerman,
PharmD, PhD
Department of
Pharmaceutical Health
Services Research,
University of Maryland
School of Pharmacy,
Baltimore, Maryland
Vithaya Kulsomboon, PhD
Department of Social and
Administrative Pharmacy,
Faculty of Pharmaceutical
Sciences, Chulalongkorn
University, Bangkok,
Thailand
Wimon Suwankesawong,
BSc (Pharm), MA
Health Products Vigilance
Center, Thai Food and Drug
Administration, Nonthaburi,
Thailand
Pratap Singhasivanon,
MBBS, DTM&H (Bangkok),
DrPH (Epidemiology), MPH
Department of Tropical
Hygiene, Faculty of
Tropical Medicine, Mahidol
University, Bangkok,
Thailand
Jaranit Kaewkungwal, PhD
Department of Tropical
Hygiene, Faculty of
Tropical Medicine, Mahidol
University, Bangkok,
Thailand
Key Words
Adverse drug reaction;
Spontaneous reporting;
Signal detection; ROR;
BCPNN
Correspondence Address
Waranee Bunchuailua,
Department of Community
Pharmacy, Faculty of
Pharmacy, Silpakorn
University, Sanamchantra
Palace Campus, Muang,
Nakhon Pathom 73000
(email: waranee@su.ac.th).
Drug Information Journal, Vol. 44, pp. 393–403, 2010 • 0092-8615/2010
Printed in the USA. All rights reserved. Copyright © 2010 Drug Information Association, Inc.
p h a r m a c o v i g i l a n c e 393
Detection of Adverse Drug Reaction Signals in
the Thai FDA Database: Comparison Between
Reporting Odds Ratio and Bayesian Confidence
Propagation Neural Network Methods
The study aimed to compare performance between
the reporting odds ratio (ROR) and the
Bayesian confidence propagation neural network
(BCPNN) methods in identifying serious
adverse drug reactions (ADRs) using the Thai
FDA spontaneous database. The two methods
were retrospectively applied to identify new, serious
ADRs reported with antiretroviral therapy
(ART) drugs using the data set between
1990 and 2006. We plotted the ROR and the
information component against time to compare
the differential timing of signal detection
i n T r o D U c T i o n
Advances in antiretroviral therapy (ART), especially
the use of highly active antiretroviral
therapy (HAART), have resulted in significant
reductions in morbidity and mortality associated
with HIV infection (1–3). As a combination
regimen of three different classes of drugs (nucleoside
analog reverse transcriptase inhibitors,
NRTIs; nonnucleoside reverse transcriptase inhibitors,
NNRTIs; and protease inhibitors),
HAART has the ability to effectively suppress
HIV-1 viral replication (4,5). However, the
treatment is associated with a number of adverse
effects that have short-term and long-term
consequences ranging from mild to severe (6).
Adverse reactions were reported in up to 50%
of patients treated with ART, including gastrointestinal,
metabolic, neurologic, and dermatologic
effects (7). Moreover, some adverse effects
may result in increased morbidity and represent
additional risk factors for future complications
that require treatment interruptions along with
therapeutic drug monitoring for management
(8). Studies have indicated that adverse effects
of ART are a major cause of treatment discon-
and the pattern of signaling over time between
these methods. The ROR and the BCPNN
methods identified the associations between
ART drugs and serious ADRs at the same time.
Both methods were similar in detecting the first
signal of a potential ADR. However, the pattern
of signaling seems relatively different with
each method. Additional analyses of different
drugs, ADRs, and databases will contribute to
increase understanding of methods for postmarketing
surveillance using spontaneous reporting
system.
tinuation and patient nonadherence, which
may lead to drug resistance and treatment failure
(9–11). Although adverse effects have been
assessed in clinical trials, these trials have several
limitations, including small sample size,
short duration of follow-up, and homogeneous
patient populations, which limit opportunities
to identify rare, serious, and long-term adverse
events (12). Spontaneous reporting systems
(SRSs) play a major role in postmarketing safety
surveillance and aim to detect drug safety signals
as early as possible in an environment that
is representative of drug use outside of the clinical
trial setting.
According to the WHO, a signal is defined as
“reported information on a possible causal relationship
between an adverse event and a drug,
of which the relationship is unknown or incompletely
documented previously.” In SRS, a signal
is a statistical association between a drug and
an adverse drug reaction (ADR) that is considered
important to investigate further, and that
may provide new information about an unknown
association or additional information
about a previously known association (13,14).
Several statistical methods have been used to
Submitted for publication: April 9, 2009
Accepted for publication: November 5, 2009

394 p h a r m a c o v i g i l a n c e
Bunchuailua et al.
screen and identify a signal in spontaneous reporting
databases (15–18). These statistical
methods are based on the measures of disproportionality
of reporting, and they detect drug-
ADR pairs that are reported with higher than
expected frequency compared to the background
(all ADRs reported with all other drugs
in the database). The most commonly used
methods are the Bayesian confidence propagation
neural network (BCPNN), which has been
developed and used by the Uppsala Monitoring
Center in routine pharmacovigilance with its
WHO database (15) and the reporting odds ratio
(ROR), which has been applied in the Netherlands
Pharmacovigilance Foundation Lareb
(17) and also in Thailand.
In Thailand, SRS has been employed to monitor
the safety of all pharmaceutical products in
the market since 1983. The Thai FDA utilizes the
ROR method to screen and detect drug safety
signals from its spontaneous reporting database.
Criteria to identify potential signals for
further investigation include the strength of
the statistical association between a drug and
an ADR, consideration of the seriousness of the
event, the quality of reports, and public attention.
However, whether ROR is the most suitable
method for signal detection in the Thai
FDA database is unknown. The objectives of this
study were to compare signal detection performance
of two signal detection methods: ROR
and BCPNN using ADR reports in the Thai FDA
spontaneous reporting database and to describe
how the two methods can identify ADR
signals associated with ART drugs with respect
to timing and pattern of signaling over time.
m aT e r i a l S a n D m e T h o D S
The Thai FDA spontaneous reporting database,
which consists of more than 150,000 ADR reports
and covers the period 1990 through
2006, was included in the analysis. These reports
contain patient demographic information,
ADR data, and medication data. Data concerning
the ADRs and the drugs are coded
using the WHO-Adverse Reaction Terminology
(WHO-ART) (19) and the Anatomical Therapeutic
Chemical (ATC) classification system (20),
respectively. In the ATC classification system,
the drugs are divided into different groups according
to the organ or system on which they
act and their chemical, pharmacological, and
therapeutic properties. ART drugs used for HIV
infection were selected using ATC codes beginning
with J05A. Reported drugs specifically suspected
of causing the reported ADR were indicated
as S (suspected) or I (interacting).
Concomitant drugs were coded as O (other).
We considered the ADR when the reported
ADR was marked as a Critical Term in the WHO-
ART dictionary. WHO Critical Term ADRs are
adverse events that suggest a serious disease
state that may be fatal or life-threatening, or
may result in prolonged hospitalization or persistent
disability (21).
RepoRting odds Ratio
We used the report as the unit of analysis for
calculating ROR. On the basis of case/noncase
analysis, ROR is the product of the exposure
odds among the reported cases of the ADR of
interest with respect to the exposure odds
among the reported noncases (22). The cases
are the reports describing the ADR of interest,
and noncases are all of the other ADR reports
included in the database. The exposure is defined
as a report with the ART drug of interest.
The calculation of ROR is based on the 2 × 2
table (Figure 1). The ROR for each drug-ADR
pair and its 95% confidence interval were calculated
by:
a/c ad
ROR = =
b/d bc
⎡ln(ROR)
± 1.96
⎣⎢
95%CI = e
+ + +
1 1 1 1
( a b c d )
For this analysis, we used signal detection criteria
of the Thai FDA consisting of ROR > 1, lower
limit of 95% CI > 1, and three or more case
reports to identify a possible signal (23).
Bayesian ConfidenCe pRopagation
neuRal netwoRk
The BCPNN methodology uses neural network
architecture to identify dependencies between
variables within the SRS database and measures
⎤
⎦⎥

Detection of Adverse Drug Reaction Signals p h a r m a c o v i g i l a n c e 395
Reports with drug of interest (exposed)
Reports without the drug of interest (unexposed)
how the dependencies change with additional
data (15,24). A measure of disproportionality
called the information component (IC) is used
to measure the strength of the quantitative dependency
between a drug and an ADR. The IC
value is based on the number of reports with a
particular drug, the number of reports with a
particular ADR, the number of reports with the
specific drug-ADR pair, and the total number of
reports in the database. A positive IC value indicates
that a particular drug-ADR pair is reported
in the database more frequently than statistically
expected from the rest of the reports in
the database, while a negative IC value indicates
that the drug-ADR pair is occurring less
frequently than statistically expected in the database.
The higher the value of the IC, the more
the drug-ADR pair stands out from the background
(24). New data may cause the IC to either
increase or decrease. If a positive IC value
increases over time and the confidence interval
narrows, this shows a likelihood of a positive
quantitative association between a drug and an
ADR. The values for the IC and its variance were
calculated according to Bate et al. (15). In
Bayesian statistics we use SDs rather than SEs
since we refer to the posterior distribution of
the parameter and do not refer to parameter estimates.
As the IC is the logarithm of the ratio of
the posterior and prior probabilities, it represents
the change in probability on addition of
new data. It allows the calculation of the IC as a
distribution, rather than just a point estimate,
based on prior and posterior distributions. We
calculated the standard deviation (SD) for the
IC to assess the criterion for possible signal detection.
A possible signal was considered if the
Drug Information Journal
Reports with the
ADR of interest
(cases)
Reports without the
ADR of interest
(noncases)
a b
c d
a = number of reports with mention of both the drug and ADR of interest
b = number of reports with mention of the drug of interest but not the ADR of interest
c = number of reports with mention of the ADR of interest but not the drug of interest
d = number of reports with mention of neither the drug nor ADR of interest
difference between IC and 2*SD (IC-2SD) was
greater than zero (15,24).
CompaRative analysis
As we were interested in serious ADRs associated
with ARV drugs, we evaluated the seriousness
of the reactions of all ADRs reported with
ARV in the database and considered only serious
reactions. For this analysis, we selected two
serious adverse reaction terms, “lactic acidosis”
and “hepatitis,” for consideration as ADRs of interest.
Lactic acidosis was chosen as a case
study ADR that represents a specificity of an
ADR associated with an ART drug. For hepatitis,
the ADR was chosen as a representative of an
ADR that can generally be associated with all
drugs. In addition, the related adverse reaction
terms “lactate blood increase” and “hepatitis
toxic” were selected and combined with lactic
acidosis and hepatitis, respectively. Associations
between each ART drug and each of the
two ADRs were analyzed using the ROR and
BCPNN methods. ROR with its 95% CI, and IC
with its SD, were computed for ART drugs that
were reported with those ADRs of interest at
quarterly (3-month) time intervals. The signal
detection was performed not only on the suspected
(S) or interacting (I) drug-ADR pairs but
also on the concomitant drug-ADR pairs (coded
as O). Since reported drugs were assessed as being
suspected of having caused the reaction, as
interacting drugs, or as concomitant medications
according to the health professionals’
opinion on the relationship between the drug
and the reaction, we decided also to include
ART drugs that were reported as concomitant
drugs in this analysis. Moreover, we separately
F i g U r e 1
The 2 × 2 table for calculation
of ROR.

396 p h a r m a c o v i g i l a n c e
Bunchuailua et al.
T a B l e 1
Total Number of Reports for Lactic Acidosis
in the Database
Lactic Acidosis
Time All Reports of
(Year/Quarter) Reports Stavudine
2001/4 2 1
2002/1 4 3
2002/2 4 3
2002/3 4 3
2002/4 5 4
2003/1 6 5
2003/2 6 5
2003/3 8 7
2003/4 9 8
2004/1 10 9
2004/2 11 10
2004/3 16 15
2004/4 18 17
2005/1 29 27
2005/2 40 36
2005/3 58 52
2005/4 70 61
2006/1 75 64
2006/2 82 69
2006/3 88 75
2006/4 94 77
analyzed drug-ADR pairs reported with combination
drug products (eg, GPO-Vir) for each
single drug (ie, stavudine, lamivudine, and nevirapine)
when a reported drug was a combination
drug product.
We plotted ROR with its 95% CI, and IC with
its IC ± 2SD, against time for each drug-ADR
pair to compare the differential timing of signal
identification and the change in values over
time between the two methods.
r e S U lT S
From January 1990 until December 2006, the
total numbers of reports of lactic acidosis/lac-
tate blood increase and hepatitis/hepatitis toxic
in the database were 94 and 953, respectively.
Of these, 81 of lactic acidosis/lactate blood increase
and 277 of hepatitis/hepatitis toxic were
reported for the ART drugs. The total number of
reports of lactic acidosis/lactate blood increase
for all drugs and stavudine in each quarter is
presented in Table 1. For hepatitis/hepatitis
toxic, the total number of reports for all drugs
and nevirapine is presented in Table 2.
Of the reports of lactic acidosis/lactate blood
increase, 77 (82%) were reported with stavudine.
The first case of lactic acidosis/lactate
blood increase with ART drugs was introduced
into the database in the fourth quarter of 2001,
corresponding to a case of stavudine. The development
of the association between stavudine
and lactic acidosis is presented in Figures 2 and
3. The first occurrence of signal identification
was in the first quarter of 2002, when three cases
of the association were included. At that time
the association was identified by both BCPNN
(IC 1.99; IC ± 2SD 0.03 to 3.96) and ROR (ROR
2571.81; 95% CI 264.15 to 25039.84) methods.
For the BCPNN method, the IC value increased
considerably, and the confidence interval decreased
over time (Figure 2). The signal was
clearly established as indicated by stabilizing IC
values from the third quarter of 2005, when the
IC was 4.59 (IC-2SD = 4.04) until 2006. In
contrast to the IC value, the ROR value increased
from 2001 until the first quarter of
2002 when the association met the signal detection
criteria and decreased until the second
quarter of 2003 (Figure 3). The ROR values
fluctuated between 2003 and 2004 and then
decreased continuously until 2006.
Among reports of hepatitis/hepatitis toxic in
the database, 244 (26%) were reported in association
with nevirapine. The strength of the association
over time between nevirapine and
hepatitis is presented in Figures 4 and 5. The
first case of hepatitis/hepatitis toxic with nevirapine
was introduced into the database in the
fourth quarter of 2002. In the first quarter of
2003, when the IC-2SD values were above zero
and the lower limit of 95% CI of ROR values
were greater than 1, the association was identi-

Detection of Adverse Drug Reaction Signals p h a r m a c o v i g i l a n c e 397
fied as a signal by both methods. At that time,
the IC value was 2.32 (IC ± 2SD 1.33 to 3.32)
and the ROR value was 10.79 (95% CI 5.26 to
22.13) when eight cases of the association were
included. For the BCPNN, the IC value increased
substantially and its confidence interval
became narrow (Figure 4). The association
was clearly established as indicated by stabilizing
IC values from the third quarter of 2004,
when the IC was 3.82 (IC ± 2SD 3.44 to 4.19)
until 2006. Similarly, the ROR value increased
significantly and the 95% CI decreased over
time until the third quarter of 2004 (Figure 5).
Although there were fluctuating values of ROR
between 2004 and 2005, the signal appeared
stable from the first quarter of 2006.
D i S c U S S i o n
This is the first published study of signal detection
reporting using the Thai FDA database that
compares two methods of signal detection. We
chose lactic acidosis as a representative of specific
ADRs and hepatitis as a representative of
common ADRs for the analysis.
Lactic acidosis is a rare but potentially fatal
complication of NRTI therapy for HIV infection
(25–29). Although all NRTIs are associated with
lactic acidosis and hyperlactatemia, people taking
stavudine seem to be at greater risk than
people taking other NRTIs (28–30). The proposed
mechanism for lactic acidosis and other
adverse reactions (eg, myopathy, neuropathy,
pancreatitis, and peripheral lipoatrophy) associated
with NRTIs is suggested to be via inhibition
of mitochondrial DNA polymerase gamma,
resulting in impaired synthesis of mitochondrial
enzymes that generate ATP by oxidative phosphorylation
(6). Lactic acidosis is a recognized
adverse reaction that has been listed on the label
of stavudine. However, the US FDA recently
revised the boxed warning label on stavudine as
a result of the increasing number of fatal lactic
acidosis cases (31).
Hepatotoxicity is a general term for liver damage.
It includes such conditions as hepatitis,
hepatic necrosis, and hepatic steatosis (32). In
this study, we were interested in detecting signal
for hepatitis specifically and not all hepato-
Drug Information Journal
Total Number of Reports for Hepatitis
in the Database
Hepatitis
Time All Reports of
(Year/Quarter) Reports Nevirapine
2002/4 290 2
2003/1 330 8
2003/2 358 10
2003/3 383 15
2003/4 413 25
2004/1 454 39
2004/2 491 53
2004/3 543 72
2004/4 570 85
2005/1 623 113
2005/2 680 145
2005/3 754 191
2005/4 798 207
2006/1 841 214
2006/2 870 221
2006/3 921 236
2006/4 953 244
toxicity reactions. Hepatotoxicity can occur
with many medications and has been reported
with many antiretroviral drugs used in HAART.
Among NNRTIs, nevirapine is the drug most
frequently associated with hepatotoxicity. The
occurrence of hepatotoxicity was 12.5–17% in
nevirapine-treated patients (33,34). Moreover,
safety warning information about the risk of severe,
life-threatening hepatotoxicity, including
hepatitis, has been added to the product labeling
for nevirapine (35). Two mechanisms have
been involved in nevirapine-associated hepatitis.
The first mechanism is immune-mediated;
the second mechanism (with a delayed onset)
seems to reflect metabolic idiosyncrasy (36,37).
A retrospective analysis of spontaneous ADR
reports in the Thai FDA database showed the
statistical associations between stavudine and
T a B l e 2

398 p h a r m a c o v i g i l a n c e
Bunchuailua et al.
F i g U r e 2
The development of the association
between stavudine
and lactic acidosis
using the BCPNN method
(Point 1, first time that
the IC met the signal detection
criteria).
F i g U r e 3
The development of the
association between
stavudine and lactic acidosis
using the ROR method
(Point 1, first time that
the ROR met the signal
detection criteria). Some
outliers of the upper limit
of 95% CI were discarded
from the graph.
Information Component
Reporting Odds Ratio
6
5
4
3
2
1
0
–1
–2
3000
2700
2400
2100
1800
1500
1200
900
600
300
0
1
Stavudine-Lactic Acidosis
1998/2
1998/3
1998/4
1999/4
2000/1
2000/4
2001/1
2001/2
2001/3
2001/4
2002/1
2002/2
2002/3
2002/4
2003/1
2003/2
2003/3
2003/4
2004/1
2004/2
2004/3
2004/4
2005/1
2005/2
2005/3
2005/4
2006/1
2006/2
2006/3
2006/4
lactic acidosis, and nevirapine and hepatitis.
The results support the evidence of risks of lactic
acidosis and hepatitis associated with ART
drugs as indicated previously. A spontaneous
reporting system represents an essential and
cost-efficient way of detecting signals that may
1
Stavudine-Lactic Acidosis
1998/2
1998/3
1998/4
1999/4
2000/1
2000/4
2001/1
2001/2
2001/3
2001/4
2002/1
2002/2
2002/3
2002/4
2003/1
2003/2
2003/3
2003/4
2004/1
2004/2
2004/3
2004/4
2005/1
2005/2
2005/3
2005/4
2006/1
2006/2
2006/3
2006/4
Time (Year/Quarter)
IC IC-2SD IC+2SD
Time (Year/Quarter)
ROR Lower 95% CI Upper 95% CI
refer to unknown association or additional information
about a previously known association.
However, it cannot be used to establish a
causal association between a drug and an adverse
event; associations may be present because
the reaction is more frequent or more of-

Detection of Adverse Drug Reaction Signals p h a r m a c o v i g i l a n c e 399
Information Component
Reporting Odds Ratio
5
4
3
2
1
0
–1
–2
30
25
20
15
10
5
0
2000/3
2001/1
2000/3
2001/1
2001/2
2001/3
2001/2
2001/3
2001/4
2002/1
2001/4
2002/1
Drug Information Journal
2002/2
2002/3
2002/2
2002/3
1
2002/4
2003/1
1
2002/4
2003/1
ten reported. Once a signal has been identified,
it needs careful review with regard to both scientific
credibility and clinical relevance followed
by pharmacoepidemiology studies to further
evaluate a causal relationship.
Nevirapine-Hepatitis
2003/2
2003/3
2003/4
2004/1
Time (Year/Quarter)
2004/2
2004/3
IC IC-2SD IC+2SD
Nevirapine-Hepatitis
2003/2
2003/3
2003/4
2004/1
Time (Year/Quarter)
2004/2
2004/3
2004/4
2005/1
2004/4
2005/1
ROR Lower 95% CI Upper 95% CI
2005/2
2005/3
2005/2
2005/3
2005/4
2006/1
2005/4
2006/1
2006/2
2006/3
2006/2
2006/3
2006/4
2006/4
Although this study showed that both the
BCPNN and the ROR methods could be used to
detect serious adverse reactions, a generalization
cannot be made from this result. Particularly
with HIV infection, causal associations be-
F i g U r e 4
The development of the
association between nevirapine
and hepatitis using
the BCPNN method
(Point 1, first time that
the IC met the signal detection
criteria).
F i g U r e 5
The development of the
association between nevirapine
and hepatitis using
the ROR method (Point 1,
first time that the ROR
met the signal detection
criteria).

400 p h a r m a c o v i g i l a n c e
Bunchuailua et al.
tween drugs and adverse events are so often
complicated by the multiplicity of treatments
that any drug might be the cause of toxic effects,
and the disease is itself a risk factor for
the adverse event of interest.
We found that, using our criteria for signal
detection, BCPNN and ROR detect serious
known ADRs (lactic acidosis and hepatitis) associated
with ART at the same time. There are
limited data on the differential timing of signal
detection when comparing between simple
(ROR) and Bayesian (BCPNN) methods. However,
results from previous comparative studies
showed that simple method-proportional reporting
ratio generates signals in advance of
Bayesian method multi-item gamma Poisson
shrinker by several years (38,39). This might explain
the variation between signal detection
criteria, especially the minimum number of case
reports (cell a in the 2 × 2 table, Figure 1) required
for signal identification. The ROR method
would detect signals for both serious adverse
reactions, at the first time the drug-ADR pair
was introduced, in advance of BCPNN, regardless
of the number of case reports (lower limit
95% CI ROR lactic acidosis = 71, a = 1; lower limit
95% CI ROR hepatitis = 1.13, a = 2). This means
that if we reduce the minimum number of case
reports, the ROR will be more sensitive than the
BCPNN. On the other hand, it may result in an
increase of false-positive signals.
Regarding pattern of signal development,
BCPNN showed an increase of the IC value over
time for stavudine-lactic acidosis and nevirapine-hepatitis
associations as well. As the number
of reports of drug-ADR pairs increases, and
as the number of reports of the drug and the
ADR increases, so the CI estimate of the IC decreases.
Bate et al. have reported that “the confidence
interval estimate of an IC becomes narrow
as time passes is the property of all
drug-ADR time scan since the estimation is
based on more reports and the IC stabilizes.
Therefore, a stabilized positive IC value for
drug-ADR association may imply an ever-increasing
likelihood of a true signal as further
reports are added to the database” (15).
Although both the ROR and BCPNN methods
identified the initial signals at the same quarter,
the pattern changes over time seem relatively
different with each method. The BCPNN values
increased over time, with decreasing 95% CI,
for both stavudine-lactic acidosis and nevirapine-hepatitis.
However, the ROR value increased
over time for the nevirapine-hepatitis association
whereas the ROR value for the stavudinelactic
acidosis association decreased. The decreasing
of the association over time for the
ROR method may be due to the nature of lactic
acidosis, which is rare and more likely specific
to ART. Moreover, when considering the reporting
of stavudine, which also is reported with
many other adverse reactions, the association
may also be decreased if another reaction, specific
to the drug, is more frequently reported,
thus diluting their association by increasing
the presence of the drug in the noncase reports.
The calculations of ROR and IC in this study
were performed without stratification since we
avoided the presence of any very small strata
due to the limited size of our data set. Although
stratification can be used to reduce confounding,
overstratification might lead to decreased
sensitivity (40,41). However, it is possible to
perform the calculation with standard stratification
(patient age, patient sex, and time of reporting)
for future studies.
Since there is a lack of a gold standard for signal
detection in SRS databases, we cannot judge
which disproportionality measure is better (or
worse) than another. Each method has advantages
as well as disadvantages. The ROR is a
transparent measure, easy to interpret, and possible
for different adjustments in logistic regression
analysis (42). ROR is independent of
general underreporting for a specific drug or a
specific ADR (43,44). However, ROR calculation
is impossible when the denominator is zero (eg,
specific ADRs) and results may not be reliable
when small numbers are reported in the 2 × 2
table (Figure 1) (42). The BCPNN method is always
applicable and large numbers of calculations
can be made efficiently (42). This method
has been recognized to reduce the fluctuations
of disproportionate reporting associated with
low reporting frequency and can protect against

Detection of Adverse Drug Reaction Signals p h a r m a c o v i g i l a n c e 401
generating multiple false positive signals due to
multiple independent comparisons (15,43,44).
As our analysis is based on the spontaneous
reports, IC and ROR values are subject to the
biases of such reporting, including confounding
by indication, where the disease is itself a
risk factor of the event of interest. Channeling
bias occurs when drug therapies within a similar
therapeutic class are preferentially prescribed
to patients with different levels of disease
severity, which may lead them to be
associated with the risk of the event of interest.
“Innocent bystander” results from the detection
of an apparent association of a drug with an adverse
event that is in fact caused by a frequently
coprescribed therapy (44).
Additionally, a number of factors may influence
the reporting of ADRs, such as background
knowledge of adverse reactions and the
drug, health professionals’ attitudes and behavior
regarding reporting ADRs, public attention
(eg, media, “dear health care provider” letter),
the magnitude of drug use, and time the drug
has been on the market.
c o n c l U S i o n
In conclusion, we have reported ADR signal detection
in the Thai FDA database using two
methods. Although the methods were similar in
detecting the first signal of a potential ADR, the
subsequent patterns of signaling over time were
different. Additional analyses of different drugs,
adverse reactions, and databases will contribute
to increased understanding of methods for
postmarketing surveillance using spontaneous
reporting systems.
Acknowledgments—We are grateful to Dr. David Olaleye
(SAS Institute, Inc., Cary, NC) for his help in statistical
analysis.
r e F e r e n c e S
1. Palella Jr FJ, Delaney KM, Moorman AC, et al. Declining
morbidity and mortality among patients
with advanced human immunodeficiency virus
infection. N Engl J Med. 1998;338:853–860.
Drug Information Journal
2. Crum NF, Riffenburgh RH, Wegner S, et al. Comparisons
of causes of death and mortality rates
among HIV-infected persons: analysis of the pre-,
early, and late HAART (highly active antiretroviral
therapy) eras. J Acquir Immune Defic Syndr.
2006;41(2):194–200.
3. Sterne JAC, Hernan MA, Ledergerber B, et al.
Long-term effectiveness of potent antiretroviral
therapy in preventing AIDS and death: a prospective
cohort study. Lancet. 2005;366:378–
384.
4. Bartlett JA, Fath MJ, DeMasi R, et al. An updated
systematic overview of triple combination therapy
in antiretroviral-naive HIV-infected adults.
AIDS. 2006;20:2051–2064.
5. Yazdanpanah Y, Sissoko D, Egger M, Mouton Y,
Zwahlen M, Chene G. Clinical efficacy of antiretroviral
combination therapy based on protease
inhibitors or non-nucleoside analogue reverse
transcriptase inhibitors: indirect comparison of
controlled trials. BMJ. 2004;328:249–255.
6. Carr A, Cooper D. Adverse effects of antiretroviral
therapy. Lancet. 2000;356:1423–1430.
7. Fellay J, Ledergerber B, Bernasconi E, et al. Prevalence
of adverse events associated with potent
antiretroviral treatment: Swiss HIV cohort study.
Lancet. 2001;358:1322–1327.
8. Schiller D. Identification, management, and prevention
of adverse effects associated with highly
active antiretroviral therapy. Am J Health-Syst
Pharm. 2004;61:2507–2522.
9. Monforte AA, Lepri AC, Rezza G, et al. Insights
into the reasons for discontinuation of the first
highly active antiretroviral therapy (HAART) regimen
in a cohort of antiretroviral naive patients.
AIDS. 2000;14:499–507.
10. Ammassari A, Murri R, Pezzotti P, et al. Self-reported
symptoms and medication side effects influence
adherence to highly active antiretroviral
therapy in persons with HIV infection. J Acquir
Immune Defic Syndr. 2001;28:445–449.
11. Yuan Y, L’Italien G, Mukherjee J, Iloeje U. Determinants
of discontinuation of initial highly active
antiretroviral therapy regimens in a US HIVinfected
patient cohort. HIV Med. 2006;7(3):
156–162.
12. Bisson G, Gross R, Miller V, Weller I, Walker A.
Monitoring of long-term toxicities of HIV treat-

402 p h a r m a c o v i g i l a n c e
Bunchuailua et al.
ments: an international perspective. AIDS. 2003;
17:2407–2417.
13. Bate A, Lindquist M, Edwards IR, Orre R. A data
mining approach for signal detection and analysis.
Drug Saf. 2002;25:393–397.
14. Shakir SAW. Thoughts on signal detection in
pharmacovigilance. Drug Saf. 2007;30:603–606.
15. Bate A, Lindquist M, Edwards IR, et al. A Bayesian
neural network method for adverse drug reaction
signal generation. Eur J Clin Pharmacol. 1998;
54:315–321.
16. Evans SJW, Waller PC, Davis S. Use of proportional
reporting ratios (PRRs) for signal generation
from spontaneous adverse drug reaction
reports. Pharmacoepidemiol Drug Saf. 2001;10:
483–486.
17. Egberts ACG, Meyboom RHB, van Puijenbroek
EP. Use of measures of disproportionality in
pharmacovigilance: three Dutch examples. Drug
Saf. 2002;25:453–458.
18. Szarfman A, Machado SG, O’Neill RT. Use of
screening algorithms and computer systems to
efficiently signal higher-than-expected combinations
of drugs and events in the US FDA’s spontaneous
reports database. Drug Saf. 2002;25:381–
392.
19. Edwards IR, Aronson JK. Adverse drug reactions:
definitions, diagnosis, and management. Lancet.
2000;356:1255–1259.
20. WHO Collaborating Center for Drug Statistics
Methodology. About the ATC/DDD system.
http://www.whocc.no/atcddd/. Accessed October
20, 2008.
21. WHO Collaborating Center for International
Drug Monitoring: The Uppsala Monitoring Center.
Concepts in critical terms. http://who-umc
.org/DynPage.aspx?id=22686. Accessed October
20, 2008.
22. van Puijenbroek EP, Diemont WL, van Grootheest
K. Application of quantitative signal detection
in the Dutch spontaneous reporting system
for adverse drug reactions. Drug Saf. 2003;26(5):
293–301.
23. Faculty of Pharmaceutical Science, Chulalongkorn
University. Development of signal detection
and assessment tool for the Thai FDA spontaneous
reporting database. Bangkok; 2005.
24. Lindquist M, Stahl M, Bate A, Edwards IR, Mey-
boom RHB. A retrospective evaluation of a data
mining approach to aid finding new adverse drug
reaction signals in the WHO international database.
Drug Saf. 2000;23:533–542.
25. Lonergan J, Behling C, Pfander H, Hassanein T,
Mathews W. Hyperlactatemia and hepatic abnormalities
in 10 human immunodeficiency virusinfected
patients receiving nucleoside analogue
combination regimens. Clin Infect Dis. 2000;
31(1):162–166.
26. John M, Moore CB, James IR, et al. Chronic hyperlactatemia
in HIV-infected patients taking
antiretroviral therapy. AIDS. 2001;15:717–723.
27. Arenas-Pinto A, Grant A, Edwards S, Weller I. Lactic
acidosis in HIV infected patients: a systematic
review of published cases. Sex Transm Infect.
2003;79:340–343.
28. Falco V, Rodriguez D, Ribera E, et al. Severe nucleoside-associated
lactic acidosis in human immunodeficiency
virus-infected patients: report
of 12 cases and review of the literature. Clin Infect
Dis. 2002;34:838–846.
29. Boubaker K, Flepp M, Sudre P, et al. Hyperlactatemia
and antiretroviral therapy: the Swiss HIV
cohort study. Clin Infect Dis. 2001;33:1931–1937.
30. Miller K, Cameron M, Wood L, Dalakas M, Kovacs
J. Lactic acidosis and hepatic steatosis associated
with use of stavudine: report of four cases. Ann
Intern Med. 2000;133:192–196.
31. Medwatch: the FDA safety information and adverse
drug event reporting program. Safety-related
drug labeling change summaries approved by
FDA Center for Drug Evaluation and Research.
http://www.fda.gov/medwatch/SAFETY/2002/
feb02.htm#zerit. Accessed October 22, 2008.
32. AIDSInfo website. http://www.aidsinfo.nih.gov/
HealthTopics/HealthTopicDetails.aspx?expand
able=3&HealthTopicID=75&ClassID=53. Accessed
October 28, 2006.
33. Martinez E, Blanco JL, Arnaiz JA, et al. Hepatotoxicity
in HIV-1-infected patients receiving nevirapine-containing
antiretroviral therapy. AIDS.
2000;15:1261–1268.
34. Sanne I, Mommeja-Marin H, Hinkle J, et al. Severe
hepatotoxicity associated with nevirapine
use in HIV-infected subjects. J Infect Dis.
2005;191:825–829.
35. Medwatch: the FDA safety information and ad-

Detection of Adverse Drug Reaction Signals p h a r m a c o v i g i l a n c e 403
verse drug event reporting program. Safety-relat-
ed drug labeling change summaries approved by
FDA Center for Drug Evaluation and Research.
http://www.fda.gov/medwatch/safety/2000/
nov00.htm#viramune. Accessed October 22,
2008.
36. de Requena DG, Núnez M, Jiménez-Nácher I, Soriano
V. Liver toxicity caused by nevirapine. AIDS.
2002;16(2):290–291.
37. Spengler U, Lichterfeld M, Rockstroh JK. Antiretroviral
drug toxicity—a challenge for the hepatologist?
J Hepatol. 2002;36(2):283–294.
38. Hauben M, Reich L. Safety related drug-labelling
changes: findings from two data mining algorithms.
Drug Saf. 2004;27:735–744.
39. Hauben M, Reich L. Drug-induced pancreatitis:
lessons in data mining. Br J Clin Pharmacol. 2004;
58:560–562.
The authors report no relevant relationships to disclose.
Drug Information Journal
40. Hopstadius J, Noren GN, Bate A, Edwards IR. Impact
of stratification on adverse drug reaction
surveillance. Drug Saf. 2008;31:1035–1048.
41. Evans SJW. Stratification for spontaneous report
databases. Drug Saf. 2008;31:1049–1052.
42. van Puijenbroek EP, Bate A, Leufkens HGM,
Lindquist M, Orre R, Egberts ACG. A comparison
of measures of disproportionality for signal detection
in spontaneous reporting systems for adverse
drug reactions. Pharmacoepidemiol Drug Saf.
2002;11(1): 3–10.
43. Hauben M, Patadia V, Gerrits C, Walsh L, Reich L.
Data mining in pharmacovigilance: the need for
a balanced perspective. Drug Saf. 2005;28:835–
842.
44. Almenoff J, Tonning JM, Gould AL, et al. Perspectives
on the use of data mining in pharmacovigilance.
Drug Saf. 2005;28:981–1007.