Strategies to Assess Validity of Self-Reported Exposures ... - Impact

impact.kenniscentrum.nl

Strategies to Assess Validity of Self-Reported Exposures ... - Impact

Environmental Research Section A 81, 195^205 (1999)

Article ID enrs.1999.3977, available online at http://www/idealibrary.com on

Strategies to Assess Validity of Self-Reported Exposures

during the Persian Gulf War 1

Linda A. McCauley,* ,2 Sandra K. Joos,{ Peter S. Spencer,* Michael Lasarev,*Tomas Shuell,* and

Members of the Portland Environmental Hazards Research Center

*Center for Research on Occupational and Environmental Toxicology, Oregon Health Sciences University; and

{PortlandVeterans A¡airs Medical Center, Portland, Oregon

Received May 19, 1998

Research in the area of Persian Gulf War Unexplained

Illnesses (PGWUI) is heavily dependent on

self-reports of exposures.The Portland Environmental

Hazards Research Center (PEHRC) conducted a population-based

case-control study utilizing techniques to

measure the magnitude of potential error in self-reports

of exposure.While it is impossible to verify most

exposures in the Persian Gulf War (PGW), results of

our study reveal signi¢cant overreporting of exposures

that can be veri¢ed based on the time period

served in the Persian Gulf. Test-retest reliability estimates

indicate inconsistency in frequency and rate of

self-reported exposures during the PGW. Unexplained

illness in PGW veterans has received much political

and scienti¢c attention. Self-reported exposures in

surveys returned preceeding and following media reports

on particular exposure such as nerve gas or pesticides

are presented. These results are useful in the

interpretation of ¢ndings related to the PGWUI and

in the design of future investigations. # 1999 Academic Press

Key Words: Persian Gulf War; exposure assessment;

epidemiolgy; reporting bias; reliability.

INTRODUCTION

Veterans of the Persian Gulf War (PGW) were exposed

to a broad range of environmental factors in

1 Funded by the Department of Veterans A¡airs to the Portland

Environmental Hazards Research Center, a joint project of the

Portland Veterans A¡airs Medical Center and the Center for Research

on Occupational and Environmental Toxicology, Oregon

Health Sciences University, Portland, OR.

2 To whom correspondence should be addressed at Center for Research

on Occupational and Environmental Toxicology, 3181 SW

Sam Jackson Park Road, L606, Portland, OR 97201, Fax: (503) 494-

4278; E-mail: mccauley@ohsu.edu.

All protocols in this study were conducted in accordance with national,

Department of Veterans A¡airs, and OHSU institutional

guidelines for the protection of human subjects.

195

the months preceding con£ict, during the combat period,

and in the months following the conclusion of the

war. Several national reports concur that few exposure

data exist on most risk factors, and that research

into the relationships between exposures and health

e¡ects is exceedingly di¤cult, if not impossible

(IOM, 1996; Presidential Advisory Committee, 1996B;

United States General Accounting O¤ce, 1997). To

date, research on environmental factors and Persian

Gulf War Unexplained Illnesses (PGWUI) has been

entirely dependent on anecdotal reports and surveys

of self-reports of exposures in the Southwest Asia

theater-of-operations. These methods are prone to

multiple sources of error including, among others,

lack of understanding of the task by the subject, failure

in recall of the required data, or the e¡ects of the

perceived threat of a topic of questioning on the subject's

response to it (Armstrong et al.,1994).

Validity problems associated with self-reported exposures

of veterans after military con£ict are not

without precedent. Previous research of veterans of

the V|etnam War demonstrated the poor validity of

self-reported exposure data (Goldberg et al., 1992). In

an attempt to provide more valid exposure data, the

Department of Defense (DoD) has made its congressionally

mandated Persian Gulf Registry of Unit Locations

available to government and private

researchers, but the database lacks the precision and

detail necessary to be of assistance in epidemiological

studies of exposure and health outcomes (Presidential

Advisory Committee,1996). Furthermore, the assumption

on which the database was premisedÐthat individuals

remain with their unitsÐfrequently was not

true in the Southwest Asia theater-of-operations.

Data on the location and dates of oil well ¢res and

smoke plume dispersion could potentially provide

information on the exposure of speci¢c units to these

environmental agents, but individual exposure

0013-9351/99 $30.00

Copyright # 1999 by Academic Press

All rights of reproduction in any form reserved.


196 MCCAULEY ET AL.

estimates are dependent on self-reports of proximity

to oil well ¢res by date and time. Authorized use of

pesticides in the Gulf was reported to have followed

strict guidelines, with use only after arthropod surveys

and estimates of arthropod prevalence (Bolton,

1995). Distribution or use of pesticides for other than

personal purposes was restricted to trained or certi-

¢ed personnel or contractors. While the DoD can

document what pesticides were shipped to the Gulf,

no records are available on the personal use of insecticide

repellent (33% diethyltoluamide (DEET) applied

to the skin) or insecticide (permethrin spray to

uniforms). Some soliders utilized unauthorized strategies

to control insects (e.g., £ea collars requested

from home) (Haley et al., 1997). Vaccine records are

available on subgroups of the U.S. troops who received

the anthrax vaccine, but overall medical record keeping

on vaccines was incomplete (Presidential Advisory

Committee, 1996B). Ingestion of pyridostigmine

bromide (PB), a nerve gas antidote enhancer that

was self-administered during the combat period, is

impossible to verify because no records were kept of

self-administered medications (Presidential Advisory

Committee, 1995).

Two studies have reported associations between

self-reported exposures and health e¡ects (Haley et

al., 1997; Iowa Persian Gulf Study Group, 1997);

neither considered the e¡ects of exposure misclassi¢cation

error on their results. Given the dependence on

self-reported data, and the demand from veterans and

the public for information on links between exposures

in the Gulf War and health e¡ects, approaches are

needed to examine the limits of self-reported exposure

data in a systematic and statistically useful manner.

The Portland Environmental Hazards Research

Center (PEHRC) conducted a population-based casecontrol

study utilizing three di¡erent techniques to

measure the magnitude of potential error in self-reports

of exposure.These techniques included: (a) comparison

of self-reported exposures among veterans

who served in Southwest Asia at di¡erent periods of

time during which some exposures, but not others,

were known to occur; (b) measurement of the e¡ect

of news media on self-reported exposures during the

PGW; and (c) test-retest measures of the reliability of

self-reported exposure information. In this paper, we

describe each of these three approaches and present

our ¢ndings on the reliability and validity of self-reports

based on these techniques.

METHODS AND MATERIALS

This investigation was conducted in two phases.

Phase I was a mail survey of randomly selected veterans

residing in the northwestern United States who

were deployed to Southwest Asia in the 1-year period

following August 1990. Phase II was a clinical casecontrol

study with subjects drawn from responders

to the mailed questionnaire. Cases consisted of veterans

who met the following PEHRC working case de¢nition

for unexplained PGW illness.

One or more of the following signs or symptoms:

. Muscle/joint pain,

. Cognitive changes including memory loss, confusion,

inability to concentrate, mood swings, and/or

somnolence,

. Diarrhea,

. Skin or mucous membrane lesions, and

. Unexplained fatigue, accompanied by four other

associated symptoms.

Onset of symptoms must have been during or after

deployment to the Persian Gulf. Symptoms must have

persisted for 1 month or longer and have occurred during

the 3-month period preceding the clinical examination.

The purpose of our study was to examine

subjects with ``unexplained'' illness; therefore, subjects

were not eligible to participate in the clinical

case-control study if they reported on the questionnaire

certain exclusionary diagnoses or diagnoses

that could explain the symptoms they were experiencing

(e.g., hepatitis, HIV, major psychiatric disorders,

and endocrine and degenerative neurological diseases).

Controls denied all symptoms in our working

case de¢nition, but could have had other health complaints

such as hypertension or allergies. Potential

cases and controls were recruited for clinical testing

in the PEHRC within 3 months of returning the mail

questionnaire.

Sample

The study population was identi¢ed from a

database provided by the Defense Manpower Data

Center and maintained by the U.S. Department of

Defense.The Desert Shield/Desert Storm data tape includes

names and last-known addresses of active

military, reservist, retired and/or discharged personnel,

dates of deployment, and other military information.

The database also includes veterans who were

deployed in the time period following Desert Storm.

The population of interest was all military personnel

deployed to Southwest Asia between August 1, 1990,

and July 31, 1991, who listed either Oregon or

Washington as their home-state-of-record at the time

of their deployment. Potential subjects also had to

have a current Washington or Oregon address

(determined from 1995 United States Internal

Revenue Service ¢les) to be considered eligible for


VALIDITY OF SELF-REPORTED PGW EXPOSURES 197

the study. This resulted in a total of 8603 veterans eligible

for our study.

Approach 1: Comparison of Reported Exposures across

Deployment Periods

The time of deployment to Southwest Asia potentially

represents a useful method for comparing differences

in the health status and risk exposure of

troops serving in Southwest Asia prior to, during,

and following the PGW.Troops serving in di¡erent deployment

periods were exposed to uniquely di¡erent

sets of environmental risks. Using speci¢c deployment

periods, one can identify groups that were or

were not at risk for exposure to pyridostigmine bromide,

certain vaccines, combat, potential chemical/

biological agents, or smoke from burning oil wells.

For example, troops deployed to Southwest Asia who

served only during the period known as Desert Shield

and who returned to the United States before combat

began (``pre-combat'') were exposed to environmental

factors of heat, sand, and insects. They could have

used or been exposed to pesticides, but they were not

exposed to combat, vaccines to prevent e¡ects from

biological warfare, or nerve gas antidote-enhancing

pills (pyridostigmine bromide). Likewise, exposure to

pyridostigmine bromide (PB), a short-acting agent

that was given to enhance the e¡ect of nerve gas antidotes

that would be injected in the event of a nerve gas

attack, could occur only among personnel who were

present during the ground war when the military

forces were preparing for possible chemical warfare

attacks (Fig. 1). Pyridostigmine bromide was authorized

for use in December 1990 (Presidential Advisory

Committee, 1996A) and initiation of use began on

January 16, 1991.

A strati¢ed random sample was selected with over

sampling of veterans who served in distinct deployment

periods associated with speci¢c exposures. The

deployment periods were de¢ned as ``pre combat only''

(August 1990±December 31, 1990),``combat only'' (January

1, 1991±March 31, 1991),``post combat only'' (April

1, 1991±July 31, 1991), and time combinations with

combat plus one or two of the other periods. Table 1

displays the sampling strata and the numbers

sampled from each stratum.The proportions of the population

serving in the pre combat only, combat-only,

and post combat only were very small (each less than

10%); therefore the veterans from unique deployment

periods were proportionally over sampled to yield suf-

¢cient numbers for analysis. Reservists were also over

sampled. All females were sampled in each of the deployment

stratum.The deployment strata allowed the

comparison of self-reported exposures in di¡erent

time periods and, speci¢cally, quanti¢cation of the degree

of over reporting of exposures not expected in

speci¢c time periods. The responders to the survey

served as the unit of analysis for the comparison of

reported exposures across deployment periods.

Approach 2: Time-Dependent Information Bias

PGWUI and its possible association with manifold

environmental factors have been widely published in

the media for several years. Our study design, which

incorporated waves of questionnaires mailed approximately

every 4 months, allowed us to compare the frequency

of self-reported exposures in responses

Period Date Event

Precombat 2 Aug 90 Iraq invades Kuwait

9 Aug 90 First U.S. forces deployed to Saudi Arabia

Dec 90

FDA permits DoD to use investigational products without informed consent

Combat 15 Jan 91 UN deadline for Iraqi withdrawal from Kuwait

16 Jan 91 Allied air o¡ensive begins

16 Jan 91 First Scuds launched against allies

16 Jan 91 Initiation of prophylactic use of pyridostigmine

23 Feb 91 Iraqi forces set ¢re to more than 700 oil wells

24 Feb 91 Allied ground attack begins

28 Feb 91 Cease-¢re takes e¡ect

1 Mar 91 Allied air o¡ensive ends

4 Mar 91 Demolition of Bunker 73 at Khamisiyah ASP

10 Mar 91 Demolition of munitions in ``pit''at Khamisiyah ASP

Postcombat 6 Aug 91 Last Kuwaiti oil well ¢re extinguished

FIG. 1.

A chronology of Persian Gulf War events.


198 MCCAULEY ET AL.

TABLE 1

Oregon and Washington Target Population, Sampling Frame, and Response Rates

Total Total Participated in

Deployment period served sampled Eligible Located Responders case-control study

Precombat period only 617 278 218 196 110 38

Combat period only 1106 378 324 286 182 70

Postcombat period only 474 200 139 126 58 26

Time combination:

Reservists 785 426 399 318 223 113

Time combination:

Active duty 5621 1061 942 834 546 196

Total 8603 2343 2022 1760 1119 443

received in time periods preceding and following extensive

media coverage of particular exposures such

as nerve gas or pesticides.

Mailings and follow-up of nonresponders began in

November 1995 and continued until June 1998. The

dates of the mailing waves are shown in Fig. 2, along

with media reports that might be expected to have in-

£uenced self-reporting of exposures in the PGW. Three

media events that occurred within the time span when

questionnaires were being returned were considered

most likely to have in£uenced the PGW veteran population.

Factors considered in choosing these events were

the signi¢cance of the exposure information being reported

by the media and the degree of circulation of

the information (i.e., internet releases, NewYork Times,

releases in Northwest U.S. newspapers, repeated stories,

and follow-up). Media Event 1, the release of information

on experimental animal studies of interaction

between PB and insecticides in promoting neurotoxic

e¡ects (Abou-Donia et al., 1996), occurred on April 23,

1996, about 1 week before our Wave 2 mailing. Media

Event 2, the publication of information on nerve gas releases

from the region of Khamisiyah in Coalition-occupied

Iraq (Sloyan, 1996), occurred on September 26,

1996, about 1 month before our Wave 4 mailing. Media

Event 3 was the publication of information on health effects

associated with self-reported exposures to insecticides,

low-dose chemical warfare agents, and PB (Haley

et al., 1997), occurring on January 23, 1997, about 3

weeks before ourWave 6 mailing. Examples of the headlines

in newspapers with large circulations and on the

internet are shown in Fig. 3.

We hypothesized that the frequency of self-reported

exposures to pesticides, PB, and chemical warfare

agents would be in£uenced by the release of information

about these exposures in the media. The responders

to the survey served as the unit of analysis for the

assessment of time-dependent information bias. We

compared responses across four time intervals de-

¢ned by the three selected media events: before Event

1, after Event 1, after Event 2, and after Event 3. We

also narrowed the focus of our assessment of any media

e¡ect by comparing responses received within 60

days following each media event to responses received

before the ¢rst media event.We measured any bias associated

with self-reported exposures in early responders

to the mailed questionnaire (i.e., a tendency for

an early responder to answer positively to multiple exposures

due to their inherent interest in the research

topic). An early responder was de¢ned as someone

who returned a questionnaire within 30 days of the

mailing date.

Approach 3: Test-Retest Reliability of Self-Reported

Exposures

All responders to the mailed questionnaire who met

the case de¢nition criteria or were eligible to be a control

(n = 799) were invited to participate in the clinical

phase of the study beginning in January 1996 and ending

in March 1998. We examined 443 veterans in our

case-control study. The ¢rst 305 veterans (cases and

controls) evaluated in our research clinic provided

the sample for the test-retest reliability analyses. The

mail questionnaire asks for personal experiences and

information in the following areas: (a) military service

history, duties, rank, dates, and locations in

Southwest Asia; (b) exposures during the Persian

Gulf War, including smoke, petroleum products, pesticides,

sand, heat, vaccines, vectors, anti-chemical

warfare agents, diet, water, living conditions, stress,

and perceptions of exposure to danger; (c) health history

and symptoms experienced during and after the

Persian Gulf War; and (d) lifestyle factors and psychosocial

adjustment after the war.

On the day of the clinical examination, each subject

(cases and controls) was given a short self-administered

questionnaire to complete in the waiting area

of the clinic. This questionnaire contained a random

subsample of the exposure questions on the mailed


VALIDITY OF SELF-REPORTED PGW EXPOSURES 199

Media Event

Date

Survey

Mailing Wave

Animal Studies of PB and Insecticide Interaction öööööööööööööööö April 23, 1996

;

Kamisiyah Nerve Gases Releases öööööööööööööööööööööö September 26, 1996

;

Human Studies of PB, Insecticide, and Chemical Nerve Agents ööööööööö January 22, 1997

;

November 17, 1995 öööööööööWave 1

;

April 30, 1996 ööööööööööWave 2

;

August 13, 1996 öööööööööWave 3

;

October 22, 1996 öööööööööWave 4

;

December 9, 1996 öööööööööWave 5

;

February 19, 1997 öööööööööWave 6

;

April 21, 1997 ööööööööööWave 7

;

October 28, 1997 öööööööööWave 8

FIG. 2. Portland Environmental Hazards Research Center: Dates of survey mailing waves and Persian Gulf War Unexplained Illness reports

in the media (1995±1998).

questionnaire.Therefore each subject provided retest

information on approximately one-third of the original

questionnaire exposure items.The retest questionnaire

questions were worded and formatted exactly as

the original questionnaire. For categorical responses,

such as a positive or negative history of taking PB, the

kappa statistic for nonindependent responses was

used as a measure of agreement between the two responses

(Fleiss, 1981). The kappa statistic measures

agreement between the ¢rst questionnaire response

and the response on the retest questionnaire by introducing

a correction for ``chance'' agreement. Kappa

statistics greater than 0.75 represent excellent agreement

beyond chance; values between 0.40 and 0.75 may

be taken to represent fair-to-good agreement beyond

chance; values below 0.40 represent poor agreement

(Landis and Koch, 1977).

RESULTS

Deployment Period Approach

There were 1119 subjects that responded to the mail

survey and 1091 provided su¤cient exposure data to


200 MCCAULEY ET AL.

Event 1: Information on Animal Studies of Interaction between PB and Insecticides

April 16, 1996 CNN Gulf War Injections Are Toxic Cocktail When Combined, Researchers Say

April 18, 1997 Associated Press Chicken Experiment May Help Explain Gulf War Syndrome

Event 2: Information on Nerve Gas Releases in the Kamisiyah Area

August 20, 1996 Newsday Gulf War Gas Exposure May Be Wide

September 21, 1996 Reuters Health Information Services Inc Pentagon Acknowledges Possible Nerve Gas Exposure Involving

Gulf War Soldiers

October 2, 1996 LA Times-Washington Post Service Thousands More Might Have Been Exposed to Gas

October 19, 1996 NewYork Times Czechs Say They Warned U.S. of Chemical Weapons In Gulf

Event 3: Information of Health E¡ects Associated with Self-Reported Exposures to Insecticides.

Low-Dose Chemical Warfare Agents, and PB

January 8, 1997 U.S. News Study Says Gulf War Illnesses Caused by Toxins

January 9, 1997 Associated Press Study Links Nerve Gas to Illness

January 22, 1997 NewYork Times U.S. Agency Links Chemicals to One Illness of Gulf War Soldiers

FIG. 3.

Examples of news headlines of chemical exposures and unexplained illnesses in PGW veterans.

be included in the analyses reported here. Dates of deployment

reported on the survey questionnaire were

compared to dates reported in the Department of Defense

Desert Shield/Desert Storm database. The Department

of Defense provided deployment dates for

1072 of the 1091 veterans (98.3%), and 1045 of the 1091

responders provided deployment dates on the survey

questionnaire (95.8%). This resulted in 1027 veterans

with both DoD and self-reported deployment dates.

Of the 873 veterans for whom the DoD database recorded

that they were deployed during the combat

period, the self-reported questionnaire data con-

¢rmed this combat deployment in 95.5% of the veterans

(n ˆ 834). DoD-reported noncombat deployment

(deployed only during precombat only or postcombat

only) was con¢rmed by self-report for 105 out of 154

veterans (68.2%) (Table 2).

For the deployment-period analyses, veterans were

considered to have been in a combat deployment period

if veri¢ed by both DoD records and self-report

(n ˆ 834) and considered not to have been exposed to

combat if veri¢ed by both DoD records and self-report

(n ˆ 105). Table 3 shows the self-reported exposure

rates for the combat and noncombat troops. The rates

of self-reported anthrax and botulinum toxoid in non

combat troops were 9.5 and 5.7%, respectively. Of the

noncombat veterans, 8.6% reported that they were

given pyridostigmine bromide. SCUD alarms were

heard by 15.2% of the noncombat veterans, and SCUD

detonations were reported by 2.9% of noncombat

troops. Twenty-eight percent of the precombat veterans

and 47% of the postcombat veterans reported

they believed they had been exposed to chemical warfare

agents during their service in Southwest Asia.

TABLE 2

Numbers of Veterans Deployed in Speci¢c Combat Periods:

Department of Defense and Self-Reported Data (n =1027)

DoD database

Precombat

Postcombat

Self-report only Combat only

Precombat only 69 4 0

Combat 33 834 16

Postcombat only 0 35 36

Time-Dependent Information Bias

We examined reported exposures in veterans who

responded (1) before and after the release of experimental

animal data reporting insecticide and PB interaction

(Event 1) (Abou-Donia et al.); (2) after Event

2 (nerve gas releases at Khamisiyah); and (3) after

Event 3 (Haley et al.'s report of neurological manifestations

correlated with self-reported exposures to insecticides,

PB, and nerve agents). Figure 4 shows that

veterans who returned questionnaires after Event 3


VALIDITY OF SELF-REPORTED PGW EXPOSURES 201

TABLE 3

Rate of Self-Reported Exposures in Combat and Noncombat Troops (n =939)

Noncombat troops

Combat troops (pre and postcombat) Precombat only Postcombat only

Exposure (n ˆ 834) (n ˆ 105) (n ˆ 69) (n ˆ 36)

Anthrax vaccine 35.7 9.5 10.1 8.3

Botulism vaccine 17.8 5.7 7.3 2.8

Pyridostigmine bromide 65.0 8.6 11.6 2.8

Heard SCUD alarms 75.3 15.2 21.7 2.8

Head SCUD detonate 51.7 2.9 5.3 0.0

Chemical warfare agents 63.4 34.3 27.5 47.2

were more likely to report having used insect repellent

cream (P ˆ 0.028) than the responders returning

questionnaires before the ¢rst media event. Veterans

returning questionnaires after Event 3 were more

likely to report using insecticide sprays on uniforms

than the responders between Event 2 and Event 3

(P ˆ 0.026). Exposures to other agents, including a belief

of being exposed to chemical warfare agents

(CWA), did not appear to di¡er between the time intervals

preceding and following the media report of

low levels of chemical warfare agents at Khamisiyah.

The self-reported exposures in questionnaires returned

within 60 days of a media event were compared

to the questionnaire responses in the time periods before

the media events. There was no apparent di¡erence

in rates of self-reported exposures for veterans

returning questionnaires within 60 days of a media

event compared to rates preceding the event. A comparison

of early responders to the questionnaire (returning

the questionnaire within 30 days of mailing)

and other responders also revealed no statistically

signi¢cant di¡erences in the frequency of self-reported

exposures (data not shown).

Test-Retest Reliability of Self-Reported Exposure

The reliability of self-reported exposures to 21 factors

associated with deployment to Southwest Asia

was assessed using kappa statistics. A total of 305 subjects

participated in the test-retest measurements.

Subjects received di¡erent sections of exposure questions

on the retest, resulting in each subject completing

approximately one-third of the total exposure

questions included on the survey questionnaire.

Table 4 shows that test-retest agreement for the veterans

who responded to the question about whether

they had been given PB pills was high (kappa ˆ 0.86,

SE ˆ 0.05) (Table 5). Lower but fairly good agreement,

with kappas between 0.40 and 0.75, was found for questions

pertaining to ¢lling sandbags, working outside

greater than 4 h per day, seeing smoke from oil well

¢res, being told they were given experimental vaccines,

receiving anthrax and/or botulism vaccines,

hearing SCUD alarms, seeing SCUD detonations, eating

local foods, handling diesel and petrochemical

fuels, using creams or sprays against insects, and belief

in the possibility of exposure to chemical warfare

agents. Poor test-retest agreement was found for exposure

to £ea collars, drinking water from local taps

and wells, skin contact with petrochemicals, exposure

to chemical agent-resistant coating (CARC), and depleted

uranium (kappa less than 0.40). The kappa values

for the reported exposures did not di¡er

signi¢cantly for cases compared to controls.

DISCUSSION

FIG. 4. Self-reported Persian Gulf War exposures in time intervals

following media events: Percentage of positive responses and

95% con¢dence intervals (n ˆ 1091).

The results of this study provide objective evidence

of the challenges inherent in studying risk factors associated

with service during the Persian Gulf War


202 MCCAULEY ET AL.

TABLE 4

Frequency of Self-Reported Exposures in Gulf War Veterans

(n = 1091) and Test^Retest Reliability Coe¤cient Kappa

with Standard Error (SE) for 305 Veterans

Percentage

reporting

exposure

Agent (n ˆ 1091) Kappa (SE)

Given pyridostigmine bromide (PB) 56 0.86 (0.05)

Filled sandbags 65 0.70 (0.06)*

Worked outside 44 h/day 82 0.69 (0.06)*

Smoke from oil well ¢res 65 0.69 (0.08)*

Experimental vaccine 22 0.66 (0.07)*

Heard SCUD alarm 66 0.64 (0.06)*

Saw SCUD detonate 45 0.64 (0.06)

Ate local foods 74 0.61 (0.06)

Handled diesel/petro fuel 59 0.55 (0.10){

Insecticide cream 28 0.54 (0.09){

Took 43 PB pills/day 03 0.53 (0.13){

Anthrax vaccine 34 0.51 (0.08){

Believe exposed to CWA 59 0.44 (0.07){

Botulism vaccine 18 0.43 (0.09){

Insecticide spray 26 0.41 (0.10){

Water from local taps 34 0.38 (0.08)**

Flea collar on ankle 04 0.35 (0.19)**

Skin contact w/petro fuel 51 0.32 (0.09)**

Chemical agent-resistant coating 11 0.30 (0.18)**

Drank water from local wells 06 0.28 (0.11)**

Depleted uranium 16 0.07 (0.09)

{ Not signi¢cantly greater than 0.40 (poor agreement).

* Not signi¢cantly less than 0.70 (Excellent agreement).

** Not signi¢cantly less than 0.40 (moderate agreement).

and unexplained illness.We applied three methodological

approaches to quantify potential misclassi¢cation,

validity, and reliability of self-reported

exposure information.

Deployment Analysis

We used two data sources (self-reported information

and DoD Desert Shield/Desert Storm database

information) to determine the period of deployment

to Southwest Asia. The intermethod validity estimates

of deployment period in the Persian Gulf War

reveal large discrepancies between DoD data and

self-reported information for troops deployed at times

other than the combat period. Anecdotally, veterans

have described situations in which their assigned

units were scheduled for return to the United States,

but individuals were given opportunities to volunteer

for longer tours of duty. It is assumed that these situations

could explain some of the discrepancies. This

source of error is further demonstrated by the fact

that in our Northwest veteran sample, at least 8.5%

of the veterans whom the DoD Desert Storm/Desert

Shield database indicates were deployed to the Gulf

were never actually deployed according to accounts

from the veterans. Their units were scheduled for deployment,

but individuals factors prevented the deployment

of individuals within the units. Due to the

observed error in the DoD database and the lack of another

method to validate self-reported deployment

dates, we only included veterans for the deploymentperiod

analyses if self-report and DoD data were in

agreement (939 out of 1091 survey responders).

Examination of self-reported exposures in groups of

veterans deployed in di¡erent deployment periods can

yield valuable information about reporting validity.

There are exposures that were only possible for troops

who were in the theater-of-operations during the combat

period; these include SCUD alarms and detonations,

pyridostigmine bromide, and antibiological

warfare vaccines. Our results indicate that some veterans

who were not deployed during the combat period

do report exposures to anti-biological warfare

vaccines, SCUD alarms and detonations, PB, and chemical

warfare agents.

The rate of self-reported administration of botulinum

toxoid vaccine in noncombat veterans was 5.7%

compared to 17.8% in the combat veterans. The total

estimated number of troops that received the botulinum

toxoid vaccine is believed to be only 8000 service

members (less than 5% of combat troops), primarily in

the First Marine Division and the Army VII Corps

(IOM, 1996). The veteran population sampled in this

study did not have a greater representation of First

Marine Division or Army VII Corps troops than expected,

which suggests a general overreporting of

the botulinum toxoid vaccination.These rates also demonstrate

an over reporting of receiving the anthrax

vaccine (35.7% of the combat veterans and 9.5% of the

noncombat veterans). Many veterans probably did not

know what vaccines were being administered; however,

instead of responding ``don't know,''our data suggest

some chose ``yes''.

Pyridostigmine bromide was given to protect military

personnel from the e¡ects of organophosphate

nerve agents; pyridostigmine bromide would enhance

the therapeutic e¤cacy of atropine if atropine was administered

at the time of exposure to the nerve agent

soman (Xia et al., 1981). Commanding o¤cers were given

the pills, in distinctive blister packets, to distribute

to the troops. Pyridostigmine bromide was

intended to be taken during combat situations when

exposure to nerve agents was a possibility. Of the veterans

deployed only during Desert Shied and removed

from Southwest Asia by December 1, 1990,

11.6% reported having been given PB, while a much

smaller proportion (2.8%) of the troops deployed only


VALIDITY OF SELF-REPORTED PGW EXPOSURES 203

postcombat report being given PB. Given the distinctive

packaging of PB, the report of PB in noncombat

troops can be viewed as an index of exposure misclassi¢cation

in the subgroup of veterans serving in PGW,

but not during the combat period. If veterans

deployed during time periods in which PB was not administered

report being given pills, one can assume

that a certain but unknown rate of misclassi¢cation

also occurs in the troops deployed during combat.

The DoD estimates that approximately 250,000 personnel

were given PB during the Gulf War (Presidential

Advisory Committee, 1996A), less than the

estimated 400,000 troops present during the combat

period (IOM, 1995). For PB, the packaging was so distinctive

that it seems likely that, if any of the pills

were taken, veterans would at least remember getting

the packet(s), whether or not they could remember

how many pills they actually took. Because of the

packaging, one would expect that forgetting would

be less of a problem than for some of the other self-reported

exposures, and that inadvertently saying they

had received PB when they had not would also be unlikely.

Twenty-eight percent of veterans who served only

during the precombat period believed they were possibly

exposed to chemical warfare agents, along with

47% of those deployed during postcombat only. No

predominant reason for this belief was given by precombat-only

veterans and there are currently no data

to support the conclusions that chemical warfare

agents were present in the theater-of-operations during

Desert Shield. The most frequent reasons given

for the belief of exposure to chemical warfare agents

in postcombat veterans were that cleaning and loading

contaminated equipment could have exposed one

to chemical warfare agents, and smoke from oil well

¢res could carry chemicals. It would appear, therefore,

that questions pertaining to chemical warfare

exposures cannot be usefully employed in searching

for associations between environmental factors and

Persian Gulf War Unexplained Illness.

While the deployment-period analyses allow us to

detect overreporting among people who were highly

unlikely to have been exposed, we have no way to determine

how many veterans were exposed but did not

report that exposure (underreporting).

Time-Dependent Analysis

In the 8 years subsequent to the Gulf War, there

have been several postulated putative exposures

published and circulated in news media. As research

information was released, we hypothesized that this

news release could result in increased numbers of

veterans reporting the putative exposures. In an

attempt to quantify the in£uence of reports in the

media on memory of exposures during the Persian

Gulf War, we analyzed self-reported exposures in

questionnaires returned in di¡erent time periods

between 1995 and 1997. We could discern little or no

e¡ect of media events on self-reported rates of

exposure during this time period. However this

approach has several limitations. The survey was

initiated in 1995; therefore we were unable to assess

changes in self-reported exposures in the years from

1991 until 1995. Media releases describing purported

exposures during the PGW were so pervasive prior to

1995 that the e¡ect on memory already may have occurred.

Until 1996, the federal government uniformly

denied the possibility of chemical warfare agents in

the theater-of-operations, and this assertion was supported

in several scienti¢c reports on PGWUI (IOM,

1995, 1996; Defense Science Board, 1994; IOM).The reversal

of the federal government's position, with the

release of information regarding the detonation of

chemical stores at Khamisiyah, did not result in any

change in the report of believing exposure to chemical

warfare agents had occurred in responses received

after the release of that information. A similar lack

of an e¡ect of media events on self-reported exposures

in veterans enrolling in the Department of Veterans

A¡airs PGW Registry has been reported by others

(Kang, 1997).

This approach does not allow any measurement of

the amount of exposure that veterans had to the news

releases. We purposely chose media events that were

circulated by the Associated Press, publicized in television

news reports, and discussed on internet news

forums. It is possible that a veteran could have heard

of the news event months after it was initially published.

The analyses that we did compared questionnaires

returned before the news event to subsequent

time periods. Therefore if veterans who heard the

news event at a late period were in£uenced to overreport

the associated putative exposure, trends in increasing

rates could have been detected.

Reliability of Self-Reported Exposures

In the absence of a speci¢c exposure criterion measure

or ``true value,''some sense of the quality of a measurement

can be obtained by comparing multiple

imperfect measurements to each other (Kelsey et al.,

1996). We administered a retest of a random subsample

of the exposure questions on the survey questionnaire

to the ¢rst 305 subjects evaluated in our

case-control study. The cases in our study were

not seriously ill or hospitalized at the time of their


204 MCCAULEY ET AL.

evaluation. They did not di¡er from the controls on

their test-retest kappa values.

The only self-reported exposure factor that was

rated highly reliable (consistent over time) was having

been given packets of pyridostigmine bromide

pills. If veterans report having been given pyridostigmine

bromide at one point in time, they tend to report

it again. The pyridostigmine bromide pills were dispensed

in distinctive packaging, and our study questionnaire

and retest questionnaire included a picture

of this distinct packaging. High reliability on being given

packets of pyridostigmine bromide pills cannot be

taken as evidence that those who were given pills subsequently

ingested the medication. Keeler et al. (1991)

reported that approximately one-half of the 34,000

American soldiers who took pyridostigmine bromide

pills reported side e¡ects associated with the medicationÐthese

associated side e¡ects could have increased

the ability to recall whether or not one took

pyridostigmine bromide pills.

Moderate to low test-retest agreement was also

found for reports of exposure to personal-use creams

(containing diethyltoluamine) and sprays (containing

permethrin) intended to control insects, and belief of

exposure to chemical warfare agents. In addition to

the general limitation of self-reported exposures,

these results on the reliability of self-reported use of

agents to control insects and exposure to chemical

warfare agents need to be considered when interpreting

the meaning of published associations between

unexplained symptoms and self-reported exposure to

these and other factors (Haley and Kurt, 1997).

The dependence on self-reported exposures during

the PGW is a serious limitation of any epidemiological

study of unexplained illnesses in this population.

While the present results provide useful information

on the consistency between original and retest report,

we have no way of determining what proportion of

people was exposed to certain factors but do not report

those exposures. If reliability of self-reported exposures

is low, then accuracy must be low as well.

However, if reliability is high, then accuracy may or

may not be acceptably high (Kelsey et al., 1996).

CONCLUSION

The results of this investigation re£ect the di¤culties

inherent in determining the validity of self-reported

exposures which occurred during the Persian

Gulf War. Three approaches to objectively measure

response bias, reliability, and overreporting of exposures

in a randomly selected sample of PGW veterans

were used with each approach yielding information

regarding the utility of self-reported exposure data.

We have shown that a veteran's recollection of possible

exposures during his/her tour of duty may not always

be in£uenced by media events. A veteran may report

exposures that are highly improbable given the dates

of deployment and, in many instances be unreliable

when measured at two di¡erent points in time. Attempts

to identify models relating health symptoms

to risk factors associated with the Persian Gulf War

must incorporate consideration of inherent sources

of variation within the data. Findings which make

sole use of self-reported ¢ndings, for both outcome

and exposure, must be viewed with caution or

suspicion.

ACKNOWLEDGMENTS

The authors and members of the PEHRC would like to thank the

veterans of the Persian Gulf War for participating in this research

project and providing helpful contributions to the research team in

the design of the research instruments and study protocol.

REFERENCES

Abou-Donia, M. B., Wilmarth, K. R., Jensen, K. F., Oehme, F. W.,

Kurt, T. L. (1996). Increased neurotoxicity following concurrent

exposure to pyridostigmine bromide, DEET, and chlorpyrofos.

Fundam. Appl.Toxicol. 34, 201±222.

Armstrong, B. K., White, E., and Saracci, R. (1994). ``Principles of

Exposure Measurement in Epidemiology.'' Oxford Univ. Press,

Oxford.

Bolton, H. T. (1995). Use and safety of pesticides (repellents) in Persian

Gulf. Presentation to the DVA: Update on health consequences

of Persian Gulf service, Baltimore, Md., July 1995.

Defence Science Board (1994). Final report: Defence Science Board

Task force on Persian Gulf War health e¡ects. O¤ce of the Under

Secretary of Defense for Acquisition and Technology, Washington,

DC.

Fleiss, J. L. (1981).``Statistical Methods for Rates and Proportions.''

Wiley, New York.

Goldberg, J., Eisen, S. A., True, W. R., and Henderson, W. G. (1992).

Health e¡ects of military service: Lessons learned from the V|etnam

experience. Ann. Epidemiol. 2(6), 841±853.

Haley, R.W., and Kurt,T. L. (1997). Self-reported exposure to neurotoxic

chemical combinations in the Gulf War: A cross-sectional

epidemiologic study. J. Am. Med. Assoc. 277, 231±237.

IOM (1995).``Institute of Medicine, Committee to Review the Health

Consequences of Service during the Persian Gulf War. Health

Consequences of Service during the Persian Gulf War: Initial

Findings and Recommendations for Immediate Action,'' Natl.

Acad. Press,Washington, DC.

IOM (1996).``Institute of Medicine, Committee to Review the Health

Consequences of Service during the Persian Gulf War. Health

Consequences of Service during the Persian Gulf War: Recommendations

for Research and Information Systems.'' Natl. Acad.

Press,Washington, DC.


VALIDITY OF SELF-REPORTED PGW EXPOSURES 205

Iowa Persian Gulf Study Group (IPGSG) (1997). Self-reported illness

and health status among Gulf War Veterans. J.Am.Med.Assoc.

277, 238±245.

Kang, H. (1997).VA data sources: Experiences from the registry and

mortality study. Presented at the Persian Gulf Research Federal

Investigator's Meeting, Fort Detrick, Frederick, Maryland, January

30, 1997.

Keeler, J. R., Hurst, C. G., and Dunn, M. A. (1991). Pyridostigmine

used as a nerve agent pretreatment under wartime conditions.

J.Am.Med.Assoc.266, 693±695.

Kelsey, J. L.,Whittemore, A. S., Evans, A. S., and Thomposom,W. D.

(1996). ``Methods in Observational Epidemiology.'' Oxford Univ.

Press, NewYork.

Landis, J. R., and Koch, G. G. (1977). The measurements of observer

agreement on categorical data. Biometrics 33, 59±74.

Presidential Advisory Committee (1996A). Presidential Advisory

Committee on Gulf War Veterans' Illnesses: Interim report. U.S.

Government Printing O¤ce,Washington, DC.

Presidential Advisory Committee (1996B). Presidential Advisory

Committee on Gulf War Veterans' Illnesses: Final report. U.S.

Govt. Printing O¤ce,Washington, DC.

Sloyan, P. J. (August 20, 1996). Gulf War gas exposure may be wide.

Newsday.

United States General Accounting O¤ce (1997). Gulf War illnesses.

Improved monitoring of clinical progress and reexamination of

research emphasis are needed.

Xia, D.Y.,Wang, L. X., and Pei, S. Q. (1981).The inhibition and protection

of cholinesterase by physostigmine and pyridostigmine

against Soman poisoning in vivo. Fundam. Appl.Toxicol.1, 217±221.

More magazines by this user
Similar magazines