Performance Evaluation of Contemporary Spirometers - Chest

Performance Evaluation of Contemporary 

Spirometers* 

Steven B. Nelson, M.S.; Reed M. Gardner, Ph.D.; 

Robert 0. Crapo, M.D., F.C.C.P; and Robert L. Jensen, Ph.D. 

A comprehensive evaluation of 62 spirometers from 37 

different sources was performed using a two-part protocol: 

calibrated syringe, and dynamic waveform testing. All 

testingwas done with ambient air. Calibrated syringe testing 

examined the ability of the spirometers to accurately 

measure the output of a 3 L calibrating syringe under 

varying conditions. The accuracy, FVC volume linearity, 

and stability of each spirometer was determined from these 

data. AU but five of 42 spirometers accurately measured a 

3 L calibrating syringe to within ±3 percent. Dynamic 

waveform testing consisted of introducing 24 standard 

waveforms into the spirometer from a computer-controlled 

air pump. The values ofFVC, FEy1, and FEF2.5-75% were 

compared to the actual values for each waveform to 

determine a performance rating. Only 35 (56.5 percent) of 

the spirometers performed acceptably when measuring the 

T he need for accuracy in measuring spirometry 

parameters has been widely recognized and ac- 

cepted. -I The American College ofChest Physicians, 

the Association for the Advancement of Medical 

Instrumentation,5 and American Thoracic Society#{176}’#{176} 

have all published recommendations for spirometer 

performance. Earlier studies of spirometer accuracy 

have found several performance problems. FitzGerald 

and co-workers’ found that all seven “electronic” 

spirometers they tested in 1973 were incapable of 

For editorial comment see page 258 

measuring FVC and FEy1 accurately or reproducibly 

when compared with a water-seal spirometer. Gardner 

et al tested 12 volume-based and seven flow-based 

spirometers. Eight (67 percent) of the volume-based 

devices performed acceptably, while all seven of the 

flow-based devices had performance difficulties. We- 

ver et al,6 in a clinical test, found that ony two of five 

spirometers were acceptable. 

With the proliferation of computerized spirometry 

systems, it is now important to test the volume or flow 

transducer as a system with the computer, rather than 

*Fmm the Departments of Medical Informatics and Medicine, 

Pulmonary Division, School of Medicine, University of Utah, Salt 

Lake City. 

Evaluation results and listing of brand names does not constitute 

endorsement or approval by the University of Utah, the authors, 

or the American College ofChest Physicians. 

Manuscript received May 8; revision accepted June 22. 

Reprint requests: Dr. Gardner, LDS Hospital, Salt Lake City, Utah 

84143 

24 standard waveforms. Nine (14.5 percent) were marginal 

and 18 (29.0 percent) were unacceptable. Fifty-nine (95 

percent) ofthe 62 spirometers were computerized. Software 

errors were found in 25 percent ofthe computerized systems 

evaluated. Although using a 3 L syringe for quality control 

purposes is essential, simple testing of spirometers with a 

3 L calibrating syringe for validation purposes was madequate 

to assess spirometer perftirmance when compared to 

dynamic waveform testing. Dynamic waveform testing is 

essential to accurately measure and validate acceptability 

of spirometer system performance. (Chest 1990; 97:288-97) 

[-ACCPAmerican College of Chest Physicians; AAMI Association 

for the Advancement of Medical Instrumentation; 

ATS American Thonacic Society 

testing only parts of the system as some authors have 

chosen.” A volume or flow transducer may be inher- 

ently accurate and precise, but may be attached to a 

computer with an inadequate sampling rate, made- 

quate flow or volume resolution, or inadequate call- 

bration or computation ri’#{176} 

Our principal objective was to determine whether 

contemporary spirometers were able to meet perform- 

ance criteria when measuring a set of 24 standard 

spirometry waveforms.5’7”0 To accomplish this objec- 

tive, we developed a computer-controlled, stepper 

motor-driven air pump. The pump was then used to 

evaluate 62 contemporary spirometers. 

In addition, we wanted to determine whether 

testing spirometer performance with a calibrated 

syringe alone was adequate. Daily quality control 

checks done with a 3 L syringe can show when a 

change in accuracy of a particular instrument has 

occurred . However, these checks are performed under 

the basic assumption that the spirometer is inherently 

capable of proper operation. Therefore, an additional 

objective was to determine whether the testing with a 

3 L syringe alone was adequate to differentiate spirom- 

eters that performed correctly from those that did 

not. The need for daily quality control still exists; it is 

required for maintaining proper laboratory quality 

control and operations. 

288 Performance Evaluation of Contemporary Spsrometers (Nelson et a!) 

Downloaded From: http://journal.publications.chestnet.org/ on 07/13/2013 

M 

All testing was performed with ambient air at about 21#{176}C, relative 

humidity 30 percent and 647 mm Hg barometric pressure. Using

OHIO 822 

ROLLING SEAL 

SPIROMETER 

FI;uRF: 1 Photograph ofcoluputer-c()ntrolled air I)LIIO1P. The stepper iootor is cOIlII(’cte(l 1)) a 1 thread it’r 

inch latll sci#{149}ev alIt1 feflIl shaft to on Ohio 822 horizontal rolling st-al spironoeter. 

aIfll)ielot air allowed testing the spiroInett’rs SIII(ler l)est-case CoIl- 

ditiosis, oiid elinii,iated pote’Iltial Illicertainties relatcd to the oct,,al 

te’fIl1)eratLlrt- ltI(l hunliditv of gusts (lvliere(l to the flov or olIIIne 

sensors. The sI)irS1ot’t(-rs Ise(l a ariet of 1)atiellt corlIIectu)us ith 

varying (lead-S1)aCe voltiiiot’s. Tl’ oritble voltiiiies aIl(1 COIIIICCtiOIIS 

%VOLII(l ln’e int,odi,ct’d 0I)1)re(lictal)k’ gas-ccliIlg effects. ‘I’he ability 

of svstens to deal with BTPS C()II(litiofls, 1 soslrie of error that is 

greater thati 10 1erct’Ilt, was itot test#{128}’(l. 

(alibrated syringe testing t’xainiin’cl the actiraoc liit’ait; aIlCI 

stal)ilitv over tilOc. For each of these tests, the 3 L calibrating 

syringe was emptied into each spirometer ten times. Accuracy of 

the spironieters was determined by manually injecting the volunoc 

from the 3 L calibrating syringe “fast” and “slow” as reconhoownde(l 

13%’the American Tluracic Societ’ and the lnternlollntain Tliracic 

Societs’’2 The fast flow test was coslclllcte(l l)\ elliptyilig the 3 L 

syringe in less thato one secon(I with o I)eak Hon of at least 6 IJs 

(ten repetitions). The slow flow test emptied the syringe over at 

least a six-second perit1, not exceeding 0.5 Us (tell repetitions). 

Linearity ssas measured in spirosneters with volume scissors (eg, 

hellosvs) l)\ eniptving the syringe at t’sv() (liflerent starting vo1uiies 

in the spirooneter; approximately 1 L, then approximately 4 L. 

Linearity for flow seissrs s’as perfrnle(l l)% comparing the voliinw 

ineasured (luring fast and slow syringe injections. Stability sas 

measured I)\ repeating the accuracy tests after the device had been 

in operation for at least eight hours. 

Dynamic waveform testing was performed using a co)mI)uter- 

controlled air pump to inject 24 standard waveh)rlus into the 

spirometer l)eing tested. ‘‘ The AAMI and the ATS recommend 

perfornance testing for validation ofspirolneter caI)al)ilit\ Earlier 

spirometry testing used 16 waveforms, 12 expo)nelltial, lIl(l four 

patient waeforins. ‘ The 24 standard waveforms allow for snore 

comprehensive testing of spirometers.7 Testing start- and end-of- 

test algorithons is also best (lone with actual patient waveforms. 

A co)Inpo,ter-controlle(I air 1”1P constructed with sn IBNI 

PC-c’cnpatible coml)tlter connected to a ConlpoItslotor 106-178 

stepper motor (Fig 1). The stepper motor prodtlce(l rOtatn)flal 

displacement ill discrete steps iii respnse to electrical pulses. The 

motor svas capable of resolving one revolutn)n into 25,(XX) discrete 

steps. The motor was then connected to a nl()(lifie(l Ohio 822 

spirneter with i lo friction. low hysteresis ball-scrcw assenhi)ly. 

The ‘svavefirm silnulator (Ielivcrc(l voliuno’ in iIlcreul(’IIts of 0.06e 

ml for each polse gelicrated i the co,optiter. Tloe i,uiiiiiio,,n flos 

tilt’ 1)510111) could (ieli\er svas 0.(X)49 1./s. tist’ tiocoictical InaXinulni 


\vats 159 L/s, altin)ugh 16 1Js scas tile I)ractical tper ii,nit. NLLXiIIILII33 

Hos’ VL5 lilllit(’(l i)\ the softsvare routines re(I,ire(i to gra(iually 

dccCleldte the IllOtOl to Pre’s’(’Ilt stalling aI)(l i)\ the liinitt’d solllnle 

(lisl)la(eIIIeIIt of tue (lrv-rolling seal spironlett-r. The c()Inputer- 

controlled air 1)111131) \L5 (lesigne(l to have an accuracy of ± 1 I)crteI,t 

for F’V(, FE\71 a,id FEF2575C/o. I I SIIbse(1ucnt Vali(latioll of the 

air 1)111131) showed less thall 0.5 percellt ariaoinlit Figure 2 is a 

flow-voluult’ 1)lotofstandard waseforln 1 overlai(i 1 1 times illustrat- 

ing tiot’ excelleit repro(loIcii)ilitv of tiu’ air )01IIlp. 

rioC values iieasured liv spirometrrs for FVC, FEV and FEF25- 

75% for cach of the 24 waveforms were assessed. The 1979 ATS 

sPir1etrY recommendations for FVC and FEV, accuracy allow 

errors of ± 5() ml tip to 1 667 ml, then ± 3.0 PerceSlt of tile known 

altIe. For FEF25-75%, accuracy liIllits ere ± 2()0 sol/s up to 4,()()0 

1131/5, tileIl ± 5.0 percesit of the ksoown ‘aisle. ‘l() all() for the 

Vaflai)ility intr()dt1ced b the coniptter-controlled Munp, we cx- 

C’) 

0 

-J 

IL. 

14 0 

1? .0 

1’.. 

40 

0.0 00 

10 2.0 30 40 ‘30 60 70 

VOLUME (L) 

F’I( Uhf: 2. Flov-voiuiiie Plot of 1 1 iteratu)ns of the out1)lIt from (lot 

colnpo,ter-controli(-d air j)tIIl() illustrating (lu itcurac’ and repro- 

(llICil)ilitt of the air 1)111011). hit’ inset waveforn, shows tiot’ fiuoal liter 

of the F\( CX()all(l(’(l 4 thoies. Note e’xtrt’noe’lv repro(hlcii)lt’ results 

(‘vt-Il with tin’ eXpan(lt’(l scale. 

CHEST I 97 I 2 I FEBRUARY, 1990 289

Spirometer NamelModel Type Version 

Computer Size Source 

AEC Spirocomp 

Biotnne MultiSpiro PC 

Brentwood Spiroscan 2000 

CDX 101 

CDX 110 

Chesebrough-Ponds Respiradyne 

Cybermedic CM5 

Cybermedic CM31O 

Cybermedic Moose 

Fukuda Sangyo ST200 

Gould Telemetry MR1 

Gould 21E 

Gould 2180 

Gould Spiroscreen 

Infomed IFU 

Infomed MWS 6000 

Kinetex Windmill 

KL Engineering Pneumoscan 

LDSH Ohio 822 Telemetry 

Med Equip Design Autospiro 

Medical Graphics 1070 

MedScience 3200 AT 

NIOSH 

Omnitek w/Ohio 822 

PK Morgan Pocket Respirometer 

PK Morgan SPIROFLOW 12 

PK Morgan TFC 

Pneumedics Dataloop 

Pneumedics Spiroloop 

Puritan-Bennett VS400 

Riko AS500 

S&M Inst w/Infomed 

S&M Inst w/Jaeger Screenmate 

S&M Inst w/Mijnhardt Vicatest 3 

S&M Inst w/Ohio 822 

S&M Inst w/WE Collins Svy III 

SensorMedics Horizon 2 

Sherwood Medical Respiradyne II 

Spirometrics SM! 3 

Spirotech 5500 

Tenet wfPK Morgan Spiroflow 

VacuMed UCI 500 

Vertek VR5000 

Vitalograph Alpha 

Vitalograph Compact 

Vitalograph PFr II 

WE Collins DS 520 


WE Collins DS-Plus 

WE Collins Eagle!! w/pneumotach 

WE Collins Eaglell w/Survey 

WE Collins MLA 3000 

WE Collins 13.5L Respirometer 

Spirometer 1 

Spirometer 2 

Spirometer 3 

Spirometer 4 

Spirometer 5 

Spirometer 6 

Spirometer 7 

Spirometer 8 

Spirometer 9 

C 

C 

F 

C 

C 

R 

F 

B 

F 

F 

M 

1. 1X 

1.21 

N/A 

industrial 

2.1X 

5-7905 

2.16 

2.16 

2.19 

N/A 

N/A 

MP 

PPC 

MP 

other 

MP 

MP 

IBM 

IBM 

IBM 

MP 

other 

H 5.0 IBM 

H 4.1C APPLE 

V N/A M 

M 1ASX MP 

D 2-25-86-T IBM 

H 2-25-86-T IBM 

V 9-3-85 IBM 

D 9-3-85 IBM 

W 9-3-85 IBM 

D 2.05 TI 

R N/A MP 

V 2.1 MP 

D 3C.7 IBM 

D N/A other 

D 1.OD PPC 

F N/A MP 

0 N/A MP 

F PP5 MP 

B 39.070 MP 

W 3.83 APPLE 

W 2.86 IBM 

W 2.86 IBM 

F 2.86 MP 

W 2.86 MP 

W 6.84 APPLE 

W N/A other 

F 

B 

B 

290 Performance Evaluation of Contemporary Spirorneters (Ne!son at a!) 


Table 1-ldentficaUon ofSpirometers Tested 

V 

M 

S 

D 

D 

T 

T 

D 

S 

H 

B 

D 

D 

T 

D 

D 

0 

V 

V 

V 

V 

H 

Software 

POS-W010205 

POS-F030301 

G3 

R3.40F 

R4.041 

N/A 

N/A 

N/A 

N/A 

1070.3 

3200-2-3-A 

Version 2 

1.3 

N/A 

N/A 

N/A 

other 

other 

MP 

MP 

CTNGEN 

MP 

MP 

other 

MP 

CTNGEN 

PC AT 

other 

IBM 

MP 

IBM 

other 

MP 

MP 

MP 

M 

M 

MP 

MP 

MP 

MP 

p 

p 

p 

p 

p 

h 

m 

m 

m 

p 

m 

m 

m 

p 

m 

m 

h 

p 

m 

p 

m 

1 

1 

m 

h 

1 

1 

m 

m 

p 

p 

m 

m 

m 

m 

m 

m 

p 

p 

m 

1 

p 

p 

p 

p 

p 

m 

m 

m 

p 

m 

m 

1 

p 

p 

p 

p 

p 

p 

p 

m 

m 

3 

2 

1# 

1$# 

1$# 

2 

1$# 

1$# 

1$# 

1# 

1# 

3# 

2 

3 

1$# 

1$ 

3$ 

1 

3 

3$ 

3 

1$ 

1$ 

3 

1$ 

1$ 

1$ 

1$ 

1$ 

1$# 

I 

1 

3 

1$ 

1$ 

1$ 

3 

3 

1$# 

1$# 

3 

3 

1$# 

1$# 

1$# 

1$# 

1$# 

1# 

1# 

1# 

(see next page for footnotes)

Type B Bellows spirometer 

C Ceramic element pneumotachometer 

D Dry rolling seal spirometer (Horizontal) 

F Fleisch-type pneumotachometer 

H Screen (Hans Rudolph-type) pneumotachometer 

M Mass flowmeter / Hot-wire anemometer 

0 Other flow-based spirometer 

R Nonwoven resistive element pneumotachometer 

S “Swirl meter” pneumotachometer 

T Turbine pneumotachometer / Rotameter 

V (Vertical) Dry rolling seal 

W Water seal spirometer 

Software Version 

Version identifier of the last version, if more than one tested 

Computer 

Size 

Source 

M 

APPLE 

CTNGEN 

IBM 

MP 

OTHER 

PC AT 

PPC 

TI 

Manual 

Apple II series 

Convergent Technologies N-Gen 

IBM Personal Computer, PC/XT, or compatible 

Microprocessor based (e.g. Z80, 6502) 

Other micro-, mini-, and mainframe computers 

IBM Personal Computer Model AT 

IBM Portable Personal Computer or compatible 

Texas Instruments Professional Computer 

h Handheld 

1 Large, generally not able to move 

m Movable, able to move on a small cart 

p Portable, able to be hand carried 

1 Manufacturer or Software Company 

2 Distributor 

3 User 

Paid/Observed 

$ Indicates testing paid for 

# Observed testing 

Many model names are trademarks of the company listed. 

panded the acceptable range for FVC and FEY, to 3.5 percent of 

the known value ( ± 70 ml for volumes less than 2,000 ml). The 

acceptable range for FEF25-75% was increased to 5.5 percent of 

the known value or ± 250 mI/s for flows less than 4,545 mI/s. The 

ATS Standardization of Spirometry- 1987 Updatebo subsequently 

adopted these expanded limits for future testing ofspirometers with 

waveform simulators. Since most of the testing was performed 

BEFORE the 1987 ATS Spirometry standardization update was 

published,’#{176} and since some of the testing was done as early as 

August 1985, some of the manufacturers had not updated their 

designs-most ofwhich were based on the 1979 ATS criteria.3 

Each waveform was injected into the spirometer at least once. If 

a value was outside the acceptable limit, the waveform in question 

was injected multiple additional times to assess whether the 

difference was a random occurrence or an inability ofthe spirometry 

system to correctly measure the particular waveform. If only the 

first measured parameter was outside the acceptable limits, the 

parameter was judged acceptable. 

For the three manual systems tested, waveforms were handmeasured 

by two technicians and rechecked if the volumes meas- 

ured by the technicians for FVC and FEV, did not agree to within 

± 20 ml or ± 1 percent of reading to eliminate human measurement 

errors. The FEF25-75% had to agree within ± 50 mI/s or ± 2 

percent ofreading. Only four of216 hand-measured values required 

rechecking. The results of the manual systems were then judged 

using the same criteria as the computerized systems. 

No spirometer readings were excluded from either the calibrated 

syringe or dynamic waveform performance testing unless a technical 

error was identified. Technical errors included such problems as 


not having connections properly attached or inadvertently initiating 

a spirometer’s sampling routine after waveform injection had begun. 

Spirometers that measured FVC,FEV1, and FEF25-75% for each 

ofthe 24 waveforms produced a total of72 results. Ifthe spirometer 

measured 68 or more parameters correctly, it was judged acceptsble. 

Three grades for classifying spirometer performance were 

used: 

Acceptable = 4 or fewer total errors 

Marginal=5 to 8 total errors 

Unacceptable 9 or more total errors 

Spirometers were obtained by sending a letter to 83 spirometer 

vendors in August 1985 explaining the test procedure and requesting 

spirometers for evaluation. Vendors that did not respond to the 

initial solicitation were contacted by telephone and follow up letters 

advising them of our intent to evaluate their spirometer. Vendors 

were invited to pay for the testing if they desired to remain 

anonymous or wanted testing results from their device prior to 

publication. Vendors were also given an opportunity to observe the 

testing for an additional fee. Some vendors chose not to pay for the 

testing but voluitarily sent spirometers for testing. Other spirom- 

eters were obtained from users or distributors. All spirometers, 

including those obtained from users rather than vendors, were 

calibrated and checked for proper operation prior to evaluation. All 

devices were tested with the same protocol. Manufacturers who 

paid for testing and sent a device that failed were given prompt 

feedback to assist them in correcting the problems. All manufacturers 

were given the opportunity to review their results and have 

their spirometer retested. Because several manufacturers chose to 

have their spirometers retested, the results reported here are from 

the most recent testing of each spirometer. 

RESULTS 

Sixty two spirometers were obtained from 37 sources 

comprising spirometers from the United States, 

United Kingdom, Germany, Holland, and Japan. At 

least one spirometer from each of the major United 

States vendors was tested. Thirty-three (53 percent) 

of the spirometers were volume-based, and 29 (47 

percent) were flow-based. Fifty-nine (95 percent) were 

computerized. 

Table 1 alphabetically lists the 62 spirometers tested 

and the source. The type of spirometer, software 

version, type of computer, and size of the spirometer 

are listed in Table 1. The results listed in the accom- 

panying tables apply ONLY to the spirometer model 

and software version listed in Table 1 . Other models 

or software versions may yield different results. 

Table 2 shows the results of the calibrated syringe 

testing. Forty-two (68 percent) of the 62 systems were 

evaluated using the calibrated syringe. For a variety 

of reasons, three manual and 18 automated systems 

were not tested with the calibrated syringe. The 

accuracy of the spirometers at fast and slow injection 

rates is reported. For calibrated syringe testing, the 

mean absolute value of the accuracy errors (absolute 

value of [syringe volume - measured volume]) of the 

remaining devices was 38.4 ml (SD 30.7) for “fast 

flows” and 36.0 ml (SD 28.6) for “slow flows.” The 

Kinetix Windmill was not included in the absolute 

value results because of the large errors associated 

with this device. Five (12 percent) of the 42 devices 


(No. 1, 14, 19, 45, 58) had volume errors greater than 

the ATS recommended ± 3 percent. All five of these 

devices were also found not to have acceptable dynamic 

waveform results. However, 19 spirometers had 

acceptable volume errors with the 3 L syringe but had 

unacceptable (12) or marginal (7) dynamic waveform 

results. Thirty-four spirometers were tested for stabil- 

AEC Spirocomp 

Spirometer 

Biotrine MultiSpiro PC 

CDX 110 

Chesebrough-Ponds Respiradyne 

Cybermedic CM5 

Cybermedic CM31O 

Cybermedic Moose 

Fukuda Sangyo ST200 

Gould 21E 

Gould 2180 

Gould Spiroscreen 

Infomed IFU 

Kinetix Windmill 

KL Engineering Pneumoscan 

LDSH Ohio 822 telemetry 

Medical Graphics 1070 

MedScience 3200 AT 

Omnitek w/Ohio 822 

PK Morgan Pocket Respirometer 

PK Morgan TTC 

Pneumedics Spiroloop 

Riko AS500 

S&M Inst w/lnfomed 

S&M Inst w/Jaeger Screenmate 

S&M Inst w/Ohio 822 

S&M Inst w/WEC Survey III 

SensorMedics Horizon 2 

Spirotech 5500 

Tenet w/PK Morgan Spiroflow 

VacuMed UCI 500 

Vertek VB5000 

Vitalograph Compact 

Vitalograph PVr II 


WE Collins DS-Plus 

-55 112U 93 

Mean difference between actual volume and measured volume for 10 calibrated syringe injections 

Syringe emptying time less than 1 sec-See methods 

Syringe emptying time ofmore than 6 sec-See methods 

Change in mean volume (ml) over at least 8 hour period 

For volume-based systems, volume linearity is the mean difference in volume between low (1 L) and high volumes (4 L) at start 

of test (ml); for flow-based systems, the difference in volume between fast and slow emptying rates (ml). 

U Unacceptable result (>3% error) with 3 L syringe 

*Excluded from absolute average and standard deviation values, error >2SD 

Models missing from this table were not tested using a calibrated syringe (see text). 

ity. The absolute value of the average change in the 

measurement of the 3 L syringe volume over eight 

hours was 37.9 ml (SD 28.2). The AEC Spirocomp 

had a stability error greater than 3 percent of the 

syringe volume. This device was also found to be 

unacceptable during dynamic waveform testing. 

The final column in Table 2 shows the largest change 

Table 2-Accuracy, Stability and linearity ofSpirometers Thsted Using 3 L Calibrated Syringe 

- 

Accuracy (ml) 

- 

Model Fast Slow Stability, ml Vol Linear, ml 

WE Collins Eagle!! w/pneumotach 

WE Collins MLA 3000 

Spirometer 2 

Spirometer 3 

Spirometer 7 

Spirometer 8 

Spirometer 9 

Accuracy 

Fast 

Slow 

Stability 

Vol Linear 

-39 

-51 

. -12 

-27 

-87 

-60 

55 

86 

-97U 

-21 

3185U 

-50 

-81 

-11 

-16 

-21 

-7 

-72 

-31 

28 

10 

61 

89 

13 

0 

8 

0 

47 

-94U 

-1 

-3 

-52 

-4 

-34 

-36 

-20 

99U 

39 

17 

39 

292 Performance Evaluation 01 Contemporary Spirometers (Nelson eta!) 


-16 

-51 

65 

-8 

-40 

-17 

-69 

44 

8 

- 126U 

-27 

-14845U 

-38 

11 

10 

6 

-67 

-26 

-10 

-13 

65 

23 

-79 

-27 

-48 

-38 

-43 

-10 

-55 

-36 

47 

11 

31 

-2 

6 

16 

29 

-67 

-21 

22 

-88 

-21 

-18 

18 

38 

84 

57 

16 

58 

72 

47 

54 

11 

40 

89 

47 

40 

16 

18 

3 

11 

8 

-22 

79 

-38 

34 

21 

9 

31 

82 

-14 

3 

8 

167 

23 

116 

81 

106 

26 

41 

119 

79 

104 

46 

163 

1802* 

17 

93 

10 

34 

98 

128 

34 

39 

34 

105 

18 

76 

61 

100 

42 

6 

188 

78 

30 

89 

23 

32 

114 

9 

85 

170 

106 

21 

13

Table 3-&#{231}formance Evaluation 

FVC FEy, FEF2S-75% Total Perf 

Spirometer Model No. No. No. No. Bating MilYr 

AEC Spirocomp 8D 1 10 19 U 1/86 

Biotrine MultiSpiro PC 0 OB 0 0 A 1L/85 

Brentwood Spiroscan 2000 0 0 0 0 A 10/88 

CDX1OI 3D 1 1 5 M 2/86 

CDX11O 2D OV 0 2 A 1186 

Chesebrough-Fbnds Respiradyne (1) ff1’ 1V 5 12 U 12/85 

Cybermedic CM5 OF 1 0 1 A 12/85 

Cybermedic CM31O iF 3 0 4 A 12/85 

Cybermedic Moose OF 1 0 1 A 12/85 

Fukuda Sangyo ST200 7D 2 5 14 U Sf86 

CouldMRl 2D 3 7 12 U 2/86 

Gould2lE 0 OV 0 0 A 3/86 

Gould2lflO 0 1V 2 3 A 3/86 

GouldSpiroscreen(2) 4 IV 1 6 M 3/86 

InfomedlFU 2T 3B 0 5 M 2/86 

Infomed MWS 6000 2T OB 0 2 A 3/86 

Kinetix Windmill 23 23 X 46 U 2/86 

KL Engineering Pneumoscan 12N 5 X 17 U 2/86 

LDSH Ohio 822 telemetry (3) 0 OB 0 0 A 7/85 

Med Equip Design Autospiro (4) 17 20 1 38 U 6/86 

Medical Graphics 1070 0 OB 0 0 A 12/85 

MedScience3200AT 0 OB 0 0 A 0/85 

NIOSH(5) 0 0 0 0 A 5(87 

Omnitek w/Ohio 822 (6) 0 1 0 1 A 1/86 

PK Morgan Pocket Respirometer (7) 17 13 X 30 U 1/86 

PK Morgan SPIROFLOW 12 0 0 0 0 A 8/88 

PK Morgan TTC 4 OB 3 7 M 

Pneumedics Dataloop 0 2 2 4 A 1/87 

Pneumedics Spiroloop 11 lB 8 20 U 1/86 

Puritan-Bennett VS400 2 2 3 7 M 4/86 

RikoAS500(8) 0 0 2 2 A 12/85 

S&Mlnstw/Infomed 0 4 2 6 M 10/85 

S&M Inst wfJaeger Screenmate 6 0 2 8 M 3/86 

S&M Inst w/Mijnhardt 0 1 0 1 A 3/87 

S&Mlnstw/Ohio822 0 0 0 0 A 9/85 

S&M Inst w/WE Collins Survey 3 0 0 1 1 A 9/85 

SensorMedics Horizon 2 0 OV 0 0 A 8/85 

Sherwood Medical Respiradyne II 0 3 0 3 A 7/88 

SpirometncsSMl3 0 0 1 1 A 1/87 

SpirotechS500 0 0 0 0 A 10/85 

Tenet wfPK Morgan Spiroflow 0 7 0 7 M 3/86 

VacuMedUCI500 0 1 2 3 A 3/86 

VertekVRS000 0 11 X 11 U 1/86 

Vitalograph Alpha 0 0 0 0 A 5/88 

Vitalograph Compact 3 1E 4 8 M 3/86 

Vitalograph PVf II 0 1 2 3 A 1/86 

WECollinsDS5ZO 1 OB 9 10 U 4/86 

WE Collins DS 560 0 0 1 1 A 4/86 

WE Collins DS-Plus 0 OV 0 0 A 

WE Collins Eaglell w/pneumotach 6N 0 6 12 AQ 

WE Collins Eaglell w/Survey 0 1 0 1 A 9/85 

WE Collins MLA 3000 0 lB 0 1 A 3/86 

WE Collins Respirometer 13.5L 0 2 0 2 A 4/86 

Spirometer 1 15 18 4 37 U 

Spirometer 2 13 10 3 26 U 

Spirometer 3 6 1V 8 15 U 

Spirometer4 13 8 5 26 U 

Spirometer5 3 8 7 18 U 

Spirometer6 7D 0 4 11 U 

Spirometer 7 21 14B 12 47 U 

Spirometer8 1 lB 1 3 A 2/86 

Spirometer 9 0 lB 2 3 A 2/86 


Test 

Date 

(see next page for footnotes) 


Notes: 

A Acceptable performance 

B Performs back-extrapolation, but does not indicate extrapolated volumes >10% 

D Has greater than average instability, should be rezeroed more frequently than once a day 

E Waveforms with large extrapolated volumes were marked unacceptable, values could not be saved or printed 

F Meets FVC accuracy recommendation only in the manual end-of-test mode 

M Marginal performance 

N Not accurate due to noise from simulator 

Q Qualified acceptable based on tests other than FVC 

R Chart recorder should be running before starting FVC 

T Terminates data collection at 10 seconds 

U Unacceptable performance 

V Either displays extrapolated volume or % for all tests, or warns ofextrapolated volumes greater than 5% of FVC 

X Not measured 

Date Period during which final version of software was tested 

(1) New version now manufactured by Sherwood Medical see Sherwood Medical Respirodyne II. 

(2) Also sold by Quinton and Chest (Japan) 

(3) LDSH = LDS Hospital (See Reference 16) 

(4) NIOSH = National Institutes ofOccupational Safety and Health-a specially designed computerized spirometer system. 

(5) Manufactured by Chest (Japan) 

(6) Now manufactured by Zao Medical Systems 

(7) Manufactured by Micro Medical Instruments (UK) 

(8) Also sold by Minata (Japan) 

The results listed apply ONLY to the spirometer model and software version lLsted in Table 1, other nwdets or software versions may yield 

different result& The ability to measure extrapolated volume was NOT used to rate spimmeters. Evaluation results and lLsting of brand names 

does not constitute endorsement or approval by the University of Utah, the authors, or the American College ofChest Physician& 

in the volume linearity test for each spirometer. The 

absolute value ofthe average difference over the tested 

range of flow or volume was 71 ml (SD 49. 1). Sixteen 

spirometers (38 percent) had differences greater than 

3 percent of the syringe volume. Five of these 16 

performed acceptably on the waveform evaluation. 

The results for dynamic waveform testing are shown 

in Table 3. Dynamic testing showed 35 (56.5 percent) 

spirometers acceptably measured the 24 standard 

waveforms. Nine (14.5 percent) were marginal and 18 

(29.0 percent) were unacceptable. Note that 15 spirometers 

measured the three spirometric parameters 

on all 24 waveforms without any errors. All 62 spirom- 

eters tested were capable ofmeasuring FVC and FEy,, 

and all but four spirometers measured FEF25-75%. 

Because ofhigh frequency noise that may have been 

generated by the waveform simulator, the WE Collins 

Eagle and the KL Engineering Pneumoscan, both 

flow-based devices, were unable to consistently determine 

the end-of-test for FVC. The WE Collins Eagle 

was given a qualified acceptable rating because it had 

no FEy, errors and the six FVC noise induced errors 

were the likely cause of the six FEF25-75% errors. 

These two spirometers did not appear to have adequate 

data smoothing functions, which are not addressed in 

the performance recommendations. However, good 

design practice usually filters out unwanted high 

frequency noise. In addition, the mechanical charac- 

teristics of rotameters of “stall before start and spin 

after stop” are ti’5 The pump created less 

than 1 ml bursts at low flows which could have caused 

an overestimation of the volume accumulated by 

rotameters. Two rotameter devices (No. 19 and 20), 

which each had more than 17 errors each with the 

dynamic waveforms, also failed the 3 L syringe test so 

their flaws were not due to test pump problems. The 

P.K. Morgan Pocket Respirometer had 30 errors and 

did poorly with the 3 L syringe testing. 

The Chesebrough-Ponds Respiradyne terminated 

data collection at eight seconds, and the two Infomed 

systems at ten seconds, which was not long enough to 

accurately measure the FVC of all 24 standard wave- 

294 Performance Evaluation of Contemporary Spirometers (Nelson a! a!) 


forms. 

Eleven of the 59 automated systems (20 percent) 

either displayed the actual extrapolated volume or 

marked tests as unacceptable when extrapolated volume 

exceeded 5 percent. Twenty-seven (49 percent) 

identffied extrapolated volumes which exceeded 10 

percent. Seventeen (31 percent) did not provide any 

warning about large extrapolated volumes. Recently, 

the ATS has suggested that the extrapolated volumes 

not exceed 5 percent of the FVC.’#{176}Ability to aecu- 

rately measure extrapolated volume was not used to 

rate the spirometers. 

Fifty-four spirometers (93 percent) measured 

FEF25-75%. Several spirometers had errors in 

FEF25-75% most likely because ofan insufficient data 

sampling rate.’#{176}-’’8 The FVC and FEy1 do not appear 

to be as sensitive to slow data sampling rates. Since 

FEF25-75% depends on accurate time and volume 

samples at two points, it is more easily affected by 

slow sample rates, typically

Table 4-Number ofErrors by Waveform and Parameter 

For 46 ofthe Spirometers Tested 

Waveform FVC FEVI FEF25-75% Total 

1 0 1 1 2 

2 0 1 5 6 

3 3 1 0 4 

4 1 2 1 4 

5 3 3 4 10 

6 4 5 7 16 

7 0 13 0 13 

8 5 5 4 14 

9 1 1 4 6 

10 3 3 2 8 

11 3 0 0 3 

12 4 1 5 10 

13 0 4 3 7 

14 2 3 2 7 

15 0 2 8 10 

16 1 5 1 7 

17 10 1 2 13 

18 0 0 0 0 

19 11 1 5 17 

20 2 1 0 3 

21 0 1 10 11 

22 6 1 4 11 

23 6 2 0 8 

24 2 0 0 2 

Total 67 57 68 192 

*Several of the spirometers were not included because they had 

such a large number of errors. 

some spirometers had such a large number of errors. 

Note that only one waveform (No. 18) had no errors 

for the three parameters measured. Also of interest is 

the fact that the errors were almost uniformly spread 

over the three parameters-67 for FVC, 57 for FEy1, 

and 68 for FEF25-75%. 

Fifty-seven automated spirometry systems (95 per- 

cent) measured peak flow (FEFmax). Volume-based 

spirometers that require manual calculations do not 

provide an accurate method of measuring instantane- 

otis flow;’7-’8 consequently, it was not measured in the 

manual devices. Peak flow is dependent on numerous 

factors that were not readily determinable, including 

the natural frequency of the system, data sampling 

rate, and filtering algorithms. Therefore, peak flow 

Type Total 

was not used in the qualification criteria. 

Table 5 shows performance by spirometer type. All 

bellows spirometers performed acceptably. The only 

water-seal spirometer found unacceptable used soft- 

ware written in 1978 (WE. Collins DS520). In addi- 

tion, at least 50 percent of the following spirometer 

types performed acceptably: horizontal dry rolling 

seal, Fleisch pneumotach, and ceramic pneumotachs. 

All turbine flowmeters performed 19 

The calibrated syringe and dynamic waveform test- 

ing results were examined to determine whether 

calibrated syringe testing alone was adequate to dif- 

ferentiate acceptable from unacceptable spirometer 

performance with dynamic waveform testing. None of 

the spirometers which failed the calibrated syringe 

testing passed the dynamic waveform testing. How- 

ever, testing with the 3 L syringe alone did not 

differentiate between all acceptable and unacceptable 

spirometers except for spirometers that exhibited 

gross errors. 

DISCusSIoN 

The performance criteria used were based on the 

ATS recommendations3 and earlier testing and stan- 

dardization criteria.4-5-7 The major changes from the 

earlier methods were additional waveforms, more 

sophisticated end of test criteria, and a relaxation of 

the limits of acceptability, which were all based on 

studies published since the 1979 ATS recommenda- 

tions.5’7 

Table 5-Pe,’formance by Spirometer Type 

An important addition to the earlier ATS recom- 

mendations was the allowance of a 5 percent random 

error rate. The authors felt that the 5 percent rate was 

necessary due to the complex interactions of the 

computer-controlled air pump and spirometry system. 

Much of the testing predated the acceptance and 

publication of the 1987 ATS spirometry standardiza- 

tion update.b0 Even though the waveforms recom- 

mended for testing spirometers were published in 

1982, most manufacturers had not tested their spi- 

rometers with the new waveforms. 

Compared to earlier testing studies, there was an 

overall improvement in spirometer performance. 1.4.6 

Acceptable 

(%) 

Marginal 

(%) 

Unacceptable 

Bellows 5 5 (100) 0 (0) 0 (0) 

Water seal 7 6 (86) 0 (0) 1 (14) 

Dry rolling seal (horizontal) 13 9 (69) 4 (31) 0 (0) 

Ceramic pneumotachometer 4 2 (50) 1 (25) 1 (25) 

Fleisch pneumotachometer 8 4 (50) 1 (13) 3 (38) 

Miscellaneous flowmeters 9 4 (44) 1 (11) 4 (44) 

Hans Rudolph (screen) pneumotach 5 2 (40) 1 (20) 2 (40) 

Vertical-Dry rolling seal 8 3 (38) 1 (12) 4 (50) 

Turbine flowmeter 3 0 (0) 0 (0) 3 (100) 

Total 62 35 (56.5) 9 (14.5) 18 (29.0) 


(%) 

CHEST I 97 I 2 1 FEBRUARY, 1990 295

The results shown here are probably indicative of the 

“best-case” attainable when a device is properly used 

and maintained by well-trained personnel. Histori- 

cally, flow-based spirometers have been less accurate 

than volume-based 1,4.6 The present study 

shows 12 of 29 flow-based spirometers (41 percent) 

met the criteria for acceptable performance. Twenty- 

four of the 34 volume-based spirometers (71 percent) 

performed acceptably. While flow-based spirometers 

have improved, as a group they did not perform as 

well as the volume-based devices. 

The flow-based spirometers found to be unaccept- 

able were all intended to be used as portable screening 

units. By contrast, all of the laboratory-based flow 

systems were found acceptable. Several recent authors 

have implied that cheap spirometers are “good 

enough” for bedside patient evaluation, regardless of 

their accuracy. Indeed, one paper shows numer- 

ous errors in FVC greater than 30 percent, then 

pronounces the spirometer being evaluated “a useful 

tool.” The present study shows that accurate flow- 

based spirometers can be manufactured inexpensively. 

Although using a 3 L syringe for quality control 

purposes is essential, simple testing of spirometers 

with a 3 L calibrating syringe for validation purposes 

was found to be an inadequate method of assessing 

spirometer performance when compared to dynamic 

waveform testing. Dynamic waveform testing is essen- 

tial to accurately measure and validate acceptability 

of spirometer system performance. We make this 

statement because we found spirometers that did not 

measure the 3 L syringe appropriately did not measure 

dynamic waveforms correctly either. Once a spirometer 

and its associated algorithths have been validated 

to be correct, quality control testing with a 3 L syringe 

will be able to monitor for calibration errors that arise 

due to aging, blocked sensors, leaks, etc. Testing with 

a calibrated syringe, however, cannot be used to 

determine whether the measurements other than total 

volume are accurate, since the syringe cannot recreate 

a test waveform reproducibly. 

Software problems, or “bugs:’ were found in 15 (25 

percent) ofthe 59 computerized systems. For example, 

some spirometers required a calibration syringe with 

exactly 3.000 L volume, and could not adapt to volumes 

such as the 3.021 L used in our testing. While 21 ml 

is a small volume, the error budget is only ± 90 ml 

and a 21 ml error is 23 percent of the error budget. 

One spirometer had an inadequate sample rate, caus- 

ing FEFma, to be overestimated by more than 200 

percent. The software used by the Vitalograph Com- 

pact was a European version, not intended for distri- 

bution in the United States and used a method for 

determining start- and end-of-test different from the 

ATS recommendations. The results therefore may not 

be indicative of the performance of the device cur- 

rently being marketed in the United States with the 

same name. 

The authors believe that most of the nine spirome- 

ters with marginal performance could be brought to 

an acceptable level of performance with more careful 

software design and testing. The potential to correct 

these problems was evident by the number of devices 

that met performance criteria after vendors were 

allowed to make changes in software and resubmit 

their device for testing. 

Other important properties of spirometers which 

could be of importance which were not exhaustively 

tested included the following: (1) Air leaks-we made 

certain there were none when the testing was done. 

Air leaks can cause major errors in the clinical 

situation. (2) BTPS correction factors -Our testing 

was done with ambient room air. BTPS correction 

factors can cause errors of more than 10 percent and 

are complex to study because ofvarying tubing lengths 

before the flow or volume transducers and because of 

ambient temperature effects.23 (3) Backpressure 

caused by the resistance of the flow or volume 

transducer-we tested about ten different types of 

devices and found for waveform No. 2 that the peak 

pressure was about 1 cm H2O/L/s, well below the ATS 

recommendation of 1.5 cm H2O/LIs. (4) Effect of 

manufacturing variability-We cannot be sure how 

well a different spirometer of the same model from 

the same manufacturer will perform because we tested 

only one instrument for each model. (5) Performance 

over time - several of the devices tested had been in 

field use, but we did not test most devices after they 

had been in long term use. None of the spirometers 

was tested over time while in clinical use. 

In conclusion, we found that only 35 of 62 (56.5 

percent) spirometers performed acceptably. With such 

a high failure rate, spirometer purchasers should be 

certain that the spirometers they acquire meet ATS 

recommendations as validated by independent labo- 

ratories. We observed software problems in 25 percent 

of the spirometers. These problems were fixed before 

final testing. These software problems included large, 

complex spirometry systems as well as inexpensive 

handheld units. We also found that validating the 

accuracy of a spirometer with a calibrated 3 L syringe 

alone was not sufficient to demonstrate how well a 

spirometer would perform with dynamic patient wave- 

296 Performance Evaluation of Contemporary Spirometers (Nelson eta!) 


forms. 

Substantial time has passed since the initial testing 

was completed. It is likely that changes have been 

made in basic equipment and computer systems. Test 

results may, therefore, not apply to current devices. 

Spirometers should meet the 1987 ATS recommenda- 

tions which are set as a lower limit of acceptable 

performance and not as an upper limit or design 

criteria.

ACKNOWLEDGMENTS: We thank the spirometer manufacturers, 

vendors and users who participated in our testing. Also we thank 

Steven L. Berlin for technical assistance in doing some of the more 

recent testing. Device 28 was provided to us by the Harvard School 

of Public Health and is being used for the Acid Aerosol Health 

Effects in North American Children Study.” We thank them for 

making the equipment available. 

REFERENCES 

1 FitzGerald MX, Smith AA, Gaensler EA. Evaluation of “elec- 

tronic” spirometers. N EngI J Med 1973; 289:1283-86 

2 Morgan WKC, Chairman. Committee recommendations: The 

assessment ofventilatory capacity: Statement ofthe committees 

on environmental health and respiratory physiology. Chest 1975; 

67:95-97 

3 Gardner RM, Chairman. ATS Statement-Snowbird workshop 

on standardization of spirometry. Am Rev Respir Dis 1979; 

119:831-38 

4 Gardner RM, Hankinson JL, West BJ. Evaluating commercially 

available spirometers. Am Rev Respir Dis 1980; 121:73-82 

5 Association for the Advancement of Medical Instrumentation. 

Standards for spirometers (Draft). AAMI, Arlington, VA, 1980 

6 Wever AMJ, Britton MG, Hughes DTD, van der Plas KH, 

Wever-Hess J. Clinical evaluation of five spirometers. Eur J 

Respir Dis 1981; 62:127-37 

7 Hankinson JL, Gardner RM . Standard waveforms for spirometer 

testing. Am Rev Respir Dis 1982; 126:362-64 

8 Sobol BJ. Spirometry and forced flow studies. In: Chusid EL. 

The selective and comprehensive testing of adult pulmonary 

function. Mount Kisco, NY: Futura Publishing, 1983 

9 Enright PL, Hyatt RE. Office spirometry. Philadelphia: Lea & 

Febiger, 1987 

10 Gardner RM, Hankinson JL, Clausen JL, Crapo RO, Johnson 

RL Jr. Epler GR. ATS Statement on Standardization of Spirometry- 

1987 Update. Am Rev Respir Dis 1987; 136:1285-98 


11 Glindmeyer HW, Harwood C. A compact computer printer for 

quality control spirometric data collection. J Occup Med 1982; 

24:1003-07 

12 Morris All, Kanner RE, Crapo RO, Gardner RM. Clinical 

pulmonary function testing. 2nd ed. Salt Lake City, UT: 

Intermountain Thoracic Society, 1984 

13 Nelson SB, Gardner RM . Computer-controlled spirometry wave 

form simulator. Respir Care 1986; 31:964 

14 Nelson SB. Commercially available spirometers: a performance 

evaluation. MS Thesis, University of Utah, Salt Lake City, 1987 

15 Yeh MP, Adams TD, Gardner RM, Yanowitz FG. Turbine 

flowmeter vs Fleisch pneumotachometer: A comparative study 

for exercise testing. J Appi Physiol 1987; 63: 1289-95 

16 Ostler DV, Gardner RM, Crapo RO. A computer system for 

analysis and transmission ofspirometry waveforms using volume 

sampling. Comptit Biomed Res 1984; 17:229-40 

17 Gardner RM, Crapo RO, Billings RG, Shigeoka JW, Hankinson 

JL. Spirometry: what paper speed? Chest 1983; 84:161-65 

18 Nelson SB, Gardner RM, Crapo RO, Jensen RL. Variability of 

instantaneous flow measurements in spirometers. Am Rev Respir 

Dis 1987; 135:68 

19 Yeh MP, Adams TD, Gardner RM, Yanowitz FG. Rotameter 

flowmeter: its implications to exercise testing. J Appl Physiol 

1987; 63:1289-95 

20 Gs,nawardena KA, Houston K, Smith AP Evaluation of the 

turbine pocket spirometer. Thorax 1987; 42:689-93 

21 Zimnicki GL, Kline JL, MacDonell RJ. Evaluation of the 

Respiradyne pulmonary function monitor. Respir Care 1987; 

32:261-67 

22 Hess D, Chieppor PJ, Johnson K. An evahiation of the Respiradyne 

II spirometer. Respir Care 1987; 32:1123-30 

23 Hankinson JL, Viola JO. Dynamic BTPS correction factors for 

spirometric data. J Appl Physiol: Respir Environ Exercise 

Physiol 1983; 44:1353-60

Performance Evaluation of Contemporary Spirometers - Chest

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?