Comparison of clinical and analytical performance of the Abbott ...

JCM Accepts, published online ahead of print on 23 March 2011
J. Clin. Microbiol. doi:10.1128/JCM.00012-11
Copyright © 2011, American Society for Microbiology and/or the Listed Authors/Institutions. All Rights Reserved.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Comparison of clinical and analytical performance of the Abbott
RealTime High Risk HPV test and Hybrid Capture 2 in
population-based cervical cancer screening
Running title: Abbott RealTime High Risk HPV in primary screening
Mario Poljak, 1 * Anja Oštrbenk, 1 Katja Seme, 1 Veronika Učakar, 2 Peter Hillemanns, 3
Eda Vrtačnik Bokal, 4 Nina Jančar 4 and Irena Klavs 2
Institute of Microbiology and Immunology, Faculty of Medicine, University of Ljubljana,
Slovenia 1 ,
National Institute of Public Health of Slovenia, Ljubljana, Slovenia 2 ,
Clinic of Obstetrics and Gynecology, Hannover Medical School, Hannover, Germany 3 ,
Department of Obstetrics and Gynecology, University Medical Centre Ljubljana, Ljubljana,
Slovenia 4
*Corresponding author. Prof. Mario Poljak, MD, PhD. Mailing address: Institute of
Microbiology and Immunology, Faculty of Medicine, University of Ljubljana Zaloška 4, 1000
Ljubljana, Slovenia. Phone: +386 1 543 7453. Fax: +386 1 543 7418. E-mail:
mario.poljak@mf.uni-lj.si
1

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Abstract
The clinical performance of the Abbott RealTime High Risk HPV test (RealTime) and Hybrid
Capture 2 HPV DNA Test (hc2) was prospectively compared in the population-based cervical
cancer screening setting. In women above 30 years (N=3,129), the clinical sensitivity for
detection of cervical intraepithelial neoplasia (CIN) grade 2 or worse (38 cases) and clinical
specificity for lesions less than CIN2 (3,091 controls) of RealTime were 100% and 93.3%,
respectively, and of hc2 97.4% and 91.8%, respectively. A noninferiority score test showed
that the clinical specificity (P

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
High-risk genotypes of alpha human papillomaviruses (hrHPV) are etiologically linked to
virtually all cervical carcinomas and their immediate precursors - high-grade cervical
intraepithelial neoplasia (CIN) lesions (49). HPV DNA testing has therefore become an
important part of cervical carcinoma screening and management algorithms in several
countries (reviewed in 10, 41). The four main clinical applications of HPV DNA testing at
present are: (i) triage of women with equivocal screening cytology results, in order to
determine which patients should be referred to colposcopy; (ii) follow-up of women with
abnormal screening cytology results who are negative at initial colposcopy/biopsy; (iii)
prediction of the therapeutic outcome after treatment of high-grade CIN and (iv) primary
screening of women aged 30 years and more in combination with Pap smear to detect cervical
cancer precursors (1, 3, 10-11, 13, 15).
Several in-house and more than 35 commercial assays for the detection of hrHPV are
currently available (reviewed in 11, 40, 44). These assays have significantly different clinical
performance for high-grade CIN detection and are not necessarily useful for primary
screening purposes (21, 40). The Hybrid Capture 2 HPV DNA Test (hc2) (Qiagen, Hilden,
Germany) is the most frequently used diagnostic HPV assay worldwide, which has been used
in the majority of key trials that have proved the clinical value of hrHPV testing (summarized
in 10, 12-13, 32). In order to assess suitability and facilitate the acceptance of novel hrHPV
assays for primary cervical cancer screening, it has been recently recommended that candidate
HPV assays should show similar clinical characteristics as hc2 i.e. clinical sensitivity, clinical
specificity and reproducibility, before they can be used for cervical cancer screening purposes
(32). Based on reported clinical characteristics of hc2, the requirement has been set that the
candidate assay should have a clinical sensitivity and clinical specificity for high-grade CIN
or worse (CIN2+) not less than 90% and 98% of the hc2 test, respectively, in a population of
women above 30 years (32). A candidate assay should be robust and display high intra-
3

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
laboratory reproducibility and inter-laboratory agreement with a lower confidence boundary
not less than 87% (32). Stoler at al. recently proposed that any novel HPV assay intended for
use for cervical screening should have a clinical sensitivity and clinical specificity for CIN3+
of 92% ± 3% and at least 85%, respectively (46). The common idea behind all currently
proposed recommendations is that a clinically useful HPV assay should achieve an optimal
balance between clinical sensitivity and clinical specificity for detection of CIN2+, in order to
detect virtually all women with cervical cancer or immediate precursors and, at the same time,
minimize redundant or excessive follow-up procedures for hrHPV positive women with
transient hrHPV infections and/or without cervical lesions (19, 27, 32, 46).
Several recent studies have clearly shown that a negative hrHPV result provides more
reassurance against cervical precancerous lesions and cancer than a negative cervical cytology
result, and therefore, safely permits longer intervals between screens (4, 8, 12-13, 16, 31, 33).
However, among hrHPV positive women, only a small proportion will have a concurrent
clinically-relevant disease, creating a dilemma of how to identify the subset of women that
require further immediate clinical attention, such as colposcopy (2, 8). One approach to
improving the positive predictive value for disease among hrHPV positive women is the use
of limited HPV genotyping for identification of the two most oncogenic HPV types: HPV16
and HPV18 (9, 26). A new generation of commercial assays that test for the most important
hrHPV types and, in addition have the potential to separate cytologically negative/hrHPV
positive women at highest risk for CIN3+ (HPV16 or HPV18 positive) from those at lower
risk (HPV16 and HPV18 negative), has been recently introduced (reviewed in 40).
One of the next-generation tests, the Abbott RealTime High Risk HPV test (RealTime)
(Abbott, Wiesbaden, Germany) detects a pool of 12 carcinogenic HPV genotypes in
aggregate, with concurrent, separate detection of HPV16 and HPV18. The assay was
launched in Europe in January 2009. In previous evaluations, RealTime demonstrated
4

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
superior analytical specificity and similar analytical sensitivity as hc2 (23, 39). The clinical
sensitivity of RealTime for CIN2+ lesions in six published studies performed on different
study populations was at least comparable with hc2 (14, 20, 22, 25, 39, 47). The clinical
specificity and positive predictive value for either CIN2+ or CIN3+ in a triage of women with
abnormal cervical cytology smears were comparable between RealTime and hc2 in one study
(22), while RealTime performed slightly better in another study (14).
We present here the results of the first comparative evaluation of the clinical performance
of RealTime and hc2 in the population-based cervical cancer screening setting. The study was
primarily designed and statistically powered to determine the clinical specificity of RealTime
for the detection of lesions less than CIN2 in women above 30 years, although applicable data
were also obtained related to the clinical sensitivity of RealTime for the detection of CIN2+
lesions. In addition, the analytical performance of the two HPV assays were compared on the
largest sample collection to date (4,479 samples) and the first data on intra-laboratory
reproducibility and inter-laboratory agreement of RealTime are provided.
MATERIALS AND METHODS
Study population and sample collection. During the period from December 2009 to
August 2010, we prospectively enrolled all women attending the routine organized national
cervical screening program within a network of 16 outpatient gynecology services with a
nationally wide geographical coverage. In the Slovenian National Cervical Cancer Screening
Program, which started in 2003, each woman between ages 20 and 64 is invited to have a
preventive gynecological examination, together with a PAP smear, once every 3 years (after
two consecutive negative smears taken one year apart) (35). All smear and histology reports
are gathered in the central database, which is linked to the Central Population Registry. The
present study (Slovenian HPV Prevalence Study) was conducted in accordance with the
5

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Helsinki Declaration and was approved by the National Medical Ethics Committee at the
Slovenian Ministry of Health. Women were eligible for inclusion if they were attending
routine organized Slovenian national cervical cancer screening program. The exclusion
criteria were: attendance for a gynecological examination after an atypical/abnormal cytology
result, history of CIN of any grade or treatment for cervical disease in the preceding year,
hysterectomy, menstruating or pregnancy at presentation (36). A total of 34 gynecologists
were responsible for patient recruitment and management. Written informed consent was
obtained from all women by the participating gynecologists. Patient identity was blinded to all
study participants except the participating gynecologist. During the gynaecological
examination, the cervix was visually inspected and a sample was taken for routine cervical
cytology, following the procedures normally used in each gynaecological practice. Samples
were most often taken with a wooden or plastic spatula or with an endocervical brush,
smeared onto a microscope slide and spray fixed. In addition to this standard procedure at
gynecology examination for cervical cancer screening purposes, a second sample was
obtained for HPV DNA testing using either a Cervex-Brush (Rovers Medical Devices, Oss,
the Netherlands) (in 87.6% of cases) or a Pap Perfect Plastic Spatula and Cytobrash Plus GT
Gentle Touch (Medscand Sample Collection Kit, Medscand Medical, Berlin, Germany) (in
12.4% of cases) and placed into ThinPrep PreservCyt Solution (Hologic, Marlborough, MA).
Coded ThinPrep vials and all accompanying data collection forms were transported to the
laboratory on a weekly basis. Immediately on arrival at the laboratory, the specimens were
split into several aliquots. The first two aliquots were used alternately for hc2 and RealTime
testing and remainder aliquots were stored at -70 o C.
RealTime HPV testing. RealTime assay was performed on the fully automated nucleic
acid preparation instrument m2000sp and the real-time PCR instrument m2000rt (Abbott),
following the manufacturer’s instructions, as previously described (20, 23). The assay uses
6

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
four channels for the detection of fluorescent signals; one for the detection of an internal
process control (136-bp region of human beta-globin) for sample adequacy, DNA extraction
and amplification, a second one for the detection of HPV16, a third for the detection of
HPV18 and a fourth for the aggregate detection of the 12 HPV types: HPV31, HPV33,
HPV35, HPV39, HPV45, HPV51, HPV52, HPV56, HPV58, HPV59, HPV66 and HPV68
(23). The PCR amplification of HPV targets was achieved using a modified GP5+/6+ primer
mix consisting of three forward and two reverse primers (24). The assay cutoff is set at a fixed
cycle threshold (Ct) value of 32. According to the manufacturer’s instructions, the assay was
repeated on samples that showed initial invalid results for internal control and, additionally, at
our discretion, on samples that showed some degree of HPV-specific positive amplification
signal(s) but Ct values were above the manufacturer’s fixed assay cutoff cycle and on samples
that showed an initial HPV-negative result but had been defined as cases (CIN2+) during
evaluation of clinical performance.
hc2 HPV testing. hc2 is a hybridization assay designed for aggregate detection of 13 HPV
types: HPV16, HPV18, HPV31, HPV33, HPV35, HPV39, HPV45, HPV51, HPV52, HPV56,
HPV58, HPV59 and HPV68, using a mixture of unlabeled single-stranded full-genomic-
length RNA probes (30, 37). Testing was performed following the manufacturer’s
instructions. Samples with a relative light unit per cut-off (RLU/CO) ratio higher than 2.50
were considered positive and samples with a RLU/CO value of less than 1.00 were considered
negative. According to the manufacturer’s instructions, all samples with a RLU/CO ratio
between 1.00 and 2.50 were retested and the results interpreted according to the
manufacturer’s instructions. Additionally, hc2 was repeated at our discretion on samples that
showed an initial hc2 borderline HPV-negative result (RLU/CO ratio from 0.80-0.99) and on
samples that showed an initial hc2 negative result but had been defined as cases (CIN2+)
during evaluation of clinical performance.
7

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
HPV genotyping and discordant analysis. To detect HPV type(s) present in the sample,
all samples with RealTime/hc2 concordant positive results and all samples with discordant
results were additionally tested using the Linear Array HPV Genotyping Test (Linear Array)
(Roche Molecular Diagnostics, Branchburg, NJ), which is capable of recognizing 36 different
HPV types and one HPV subtype (including all 13 assay-common HPV types, i.e., types
targeted by both RealTime and hc2), following the manufacturer’s instructions (45). Samples
with a positive Linear Array HPV52 cross-reactive probe signal were additionally tested with
an HPV52 type-specific real-time PCR assay, as previously described (29). All Linear Array
HPV-negative samples and all samples in which no assay-common HPV types were identified
by Linear Array, were additionally tested with the INNO-LiPA HPV Genotyping Extra Test
(INNO-LiPA) (Innogenetics, Gent, Belgium), which is capable of recognizing 28 different
alpha-HPV types (including all 13 assay-common types), following the manufacturer’s
instructions. Finally, all INNO-LiPA HPV-negative samples and all samples in which no
assay-common types were identified by INNO-LiPA, were tested using an in-house
GP5+/GP6+ PCR assay targeting a 150-bp fragment in HPV L1 gene with additional HPV68
specific primers, as previously described (18, 34). Direct sequencing of the GP5+/GP6+ PCR
products with the same primers was used for genotyping, as previously described (28). All
HPV types identified by any of the genotyping tests were considered when interpreting the
RealTime/hc2 discordant results. The analytical reliability of the applied three-step
genotyping strategy was verified using the HPV DNA Proficiency 2010 Panel prepared by the
World Health Organization HPV Laboratory Network – LabNet (17). The panel consisted of
46 coded samples with a titration series of purified plasmids of 16 different HPV types
(including all 13 assay-common types), in concentrations ranging from 5-500 IU of HPV16 or
HPV18 DNA and 5-500 genome equivalents of the other 14 HPV types. The genotyping
8

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
strategy used in this study showed 100% specificity and 100% sensitivity for a total of 72
HPV types present in 46 proficiency panel specimens.
Cytological examination. All cervical smears were examined under routine screening
conditions by certified cytologists normally used by each participating gynecology practice
who were blinded to HPV results.
Colposcopic referral. According to the criteria of the Slovenian National Cervical Cancer
Screening Program, women were called for immediate colposcopy using a cytology threshold
of atypical squamous cells - cannot exclude high-grade lesion (ASC-H)/atypical glandular
cells (AGC) or worse. In addition, irrespective of cytology result, according to our study
protocol, women were also invited for colposcopy if they were positive for HPV16 or HPV18.
In women positive for hrHPV other than HPV16 and HPV18, an immediate colposcopy was
performed at the physician’s discretion; otherwise the woman was invited to a control
gynaecological examination after six months to one year. Colposcopy was performed by
certified colposcopists according to standard operating procedures, using the international
nomenclature (48). During colposcopy, punch biopsy specimens were taken from any regions
suspicious for CIN. No biopsy was taken from women with normal colposcopy, since this is
considered unethical in Slovenia. The three-tier CIN nomenclature was used for biopsy
classification and the most severe abnormality was selected for final histopathological
diagnosis. An expert histopathology system was used for histopathological assessment: all
biopsies were first examined by a certified pathologist with more than 20 years of experience
in gynecological pathology, followed by independent, blinded histopathological review. In
discrepant cases, the final diagnosis was the consensus reached by a panel of three
pathologists. Pathologists performing histopathological assessments were blinded to the HPV
status but did have access to concurrent cytology results.
9

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Intra-laboratory reproducibility and inter-laboratory agreement of RealTime. To
assess RealTime intra-laboratory reproducibility in time, a total of 500 samples (167
randomly selected HPV positive and 333 randomly selected HPV negative samples) were
retested after 61 to 226 days (median 73 days) from initial testing, as recently recommended
(32). In addition, two sets of coded 0.7 ml ThinPrep aliquots (2 x 500 samples) were prepared
from the same 167 HPV positive and 333 HPV negative samples and shipped on dry ice to a
collaborative laboratory in Hannover (Germany), where two additional HPV testing rounds
were performed. The obtained results were used to calculate intra-laboratory reproducibility
in time in the two participating laboratories, as well as inter-laboratory agreement between the
Ljubljana and Hannover laboratories. Reproducibility testing was performed using coded
samples and the technicians performing the assay in the two laboratories were completely
blinded to the HPV status of the samples.
Statistical analysis. For assessment of the clinical performance of RealTime and hc2,
cases were defined as women with high-grade cervical disease (CIN2+) and controls as
women without high-grade cervical disease (less than CIN2). The clinical performance of
both assays was compared using a noninferiority score test, as recently recommended (32).
The thresholds used for noninferiority were specificity for the detection of lesions less than
CIN2 of at least 98% and sensitivity for the detection of CIN2+ lesions of at least 90%
relative to the results of hc2, as previously described (32). Since we enrolled more than 2,500
women above 30 years, for clinical specificity the power of the study was more than 99%
(32). The analytical performance of the two HPV assays was determined against hrHPV
status, which was defined by the concordance between RealTime and hc2 and, for discordant
specimens, by genotyping results. Genotyping results were designated as hrHPV positive
when at least one of the 13 assay-common HPV types was detected: HPV16, HPV18, HPV31,
HPV33, HPV35, HPV39, HPV45, HPV51, HPV52, HPV56, HPV58, HPV59 and HPV68.
10

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Sensitivity, specificity, negative predictive value and positive predictive value were calculated
using the conventional contingency tables and 95% confidence intervals (95% CI) were
computed using exact binomial methods. The level of agreement between tests was assessed
by the Kappa statistics. The Chi-square test was used for intercomparison of proportions. All
statistical analyses were performed using R software vs. 2.12.0 (Free Software Foundation,
Boston, MA). The level of statistical significance was set at a value of 0.05.
RESULTS
Between December 2009 and August 2010, 4,602 eligible women were invited to
participate in the study, of which 88 (1.9%) declined to participate for various reasons. Seven
women were excluded from the study due to ThinPrep sample spill out during transport (3
women) or missing ThinPrep samples (4 women). HPV testing was thus finally performed
using RealTime and hc2 on a total of 4,507 women.
RealTime was repeated on a total of 41 samples. The assay was repeated according to the
manufacturer’s instructions on 10 samples due to initial invalid results for internal control; all
10 samples also had repeated invalid results and were excluded from the study. All excluded
samples were repeatedly hc2 negative and were obtained from women who had normal
cytology. Of 30 samples that initially showed RealTime HPV-specific positive amplification
but Ct values were above the manufacturer’s fixed assay cutoff cycle and were repeated at our
discretion, eight samples turned out to be HPV-positive after repeat testing and 22 samples
again showed HPV-specific Ct values above the manufacturer’s cutoff. A single sample that
showed an initial RealTime HPV-negative result but was defined as a case (CIN2+) during
evaluation of clinical performance (repeated at our discretion), was again tested RealTime
HPV-negative on repeat testing.
11

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Hc2 was repeated on a total of 137 samples. The test was repeated according to the
manufacturer’s instructions on 101 samples due to an initial RLU/CO ratio between 1.00 and
2.50 (borderline HPV-positive results), among which 43 samples had to be retested twice. Of
these 101 samples, after discordant analysis and HPV genotyping, 41 samples were finally
considered to be hc2 analytically true positive (the sample contained at least one targeted
HPV type) and 60 samples to be hc2 analytically false positive (no targeted HPV types were
detected). All 101 women with hc2 initially borderline HPV-positive samples were defined as
controls (less than CIN2+) in clinical performance assessment. Of 32 samples that showed an
initial hc2 borderline HPV-negative result (RLU/CO ratio between 0.80-0.99) and were
repeated at our discretion, seven samples turned out to be hc2 positive after repeat testing and
25 samples again had a RLU/CO ratio below 1.00. Of the four samples that showed an initial
hc2 negative result but were defined as cases (CIN2+) during evaluation of clinical
performance, and were repeated at our discretion, three samples again tested hc2 negative on
repeat testing (initial RLU/CO ratios 0.29, 0.42, 0.55; repeat RLU/CO ratios 0.43, 0.74, 0.73,
respectively) and one sample turned out to be hc2 positive after repeat testing (initial
RLU/CO ratio 0.84; repeat RLU/CO ratio 1.44).
Clinical performance of RealTime and hc2. Of 4,497 women who had valid HPV results
in both assays, 14 women were excluded from assessment of clinical performance of
RealTime and hc2 due to missing cytology results and 51 women eligible for colposcopy
(according to the protocol criteria) were excluded because they did not respond in time to
repeated invitations for colposcopy or they refused colposcopy. The clinical performance of
the HPV assays was finally assessed on a total of 4,432 women using two study groups:
women above 30 years and all participating women (the number of women in age groups ≤29,
30-39, 40-49, 50-59 and ≥60 years were 1,304, 1,528, 976, 542 and 82).
12

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Clinical performance of RealTime and hc2 in women above 30 years. Women above
30 years represented our primary study group for evaluation of the clinical performance of
RealTime and hc2 (N=3,129 women; mean age 41.5 years; median age 40 years). The overall
prevalence of HPV infection in women above 30 years assessed by RealTime and hc2 was
7.8% (243/3,128; 95% CI, 6.9 to 8.8%) and 9.3% (290/3,128; 95% CI, 8.3 to 10.4%),
respectively, and the HPV prevalence in women with a cytology result negative for
intraepithelial lesion and malignancy (NILM) was 6.1% (180/2,974; 95% CI, 5.2 to 7.0%) and
7.4% (219/2,974; 95% CI, 6.5 to 8.4%), respectively. Table 1 shows the RealTime results in
women above 30 years, stratified for cases and controls, in comparison to the hc2 findings. A
total of 38 cases were identified in women above 30 years: 18 CIN2 lesions, 16 CIN3 lesions,
one carcinoma in situ and three invasive carcinomas and 3,091 women were classified as
controls. The clinical sensitivity for the detection of CIN2+, clinical specificity for the
detection of lesions less than CIN2, PPV and NPV of RealTime and hc2 at cutoff values of
1.00 and 2.50 RLU/CO are shown in Table 2. The single CIN2+ case missed by hc2 (at a
cutoff value of 1.00 RLU/CO) had RLU/CO values of 0.42 and 0.74 in initial and repeat hc2
testing, respectively. A noninferiority score test performed to determine whether the clinical
specificity of RealTime for the detection of lesions less than CIN2 and the clinical sensitivity
of RealTime for the detection of CIN2+ lesions were noninferior to those of the hc2 (cutoff
1.00 RLU/CO) at recommended thresholds of 98% and 90% (32), respectively, showed that
both clinical specificity (P < 0.0001) and clinical sensitivity (P = 0.011) of RealTime were
noninferior to that of hc2.
Clinical performance of RealTime and hc2 in the total study population. The total
study population represented our secondary study group for evaluation of the clinical
performance of RealTime and hc2 (N=4,432 women; mean age 36.6 years; median age 35
years). The overall prevalence of HPV infection assessed by RealTime and hc2 were 11.6%
13

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
(515/4,431; 95% CI, 10.7 to 12.6) and 13.3% (589/4,431; 95% CI, 12.3 to 14.3%),
respectively. The prevalence of HPV infection in women with NILM cytology assessed by
RealTime and hc2 were 9.7% (411/4,217; 95% CI, 8.9 to 10.7%) and 11.2% (474/4,217; 95%
CI, 10.3 to 12.2%), respectively. As shown in Table 1, a total of 57 cases were found in the
total study population: 31 CIN2 lesions, 22 CIN3 lesions, one carcinoma in situ and three
invasive carcinomas and 4,375 women were classified as controls. The clinical sensitivity,
clinical specificity, PPV and NPV of RealTime and hc2 at cutoff values of 1.00 and 2.50
RLU/CO are shown in Table 2. A noninferiority score test showed that both clinical
specificity (P < 0.0001) and clinical sensitivity (P = 0.015) of RealTime were also noninferior
to that of hc2 in the total study population.
Analytical performance of RealTime and hc2. Of 4,497 women who had valid HPV
results in both assays, 18 women were excluded from an assessment of analytical
performance because HPV genotyping showed the presence of HPV66 alone or in
combination with other non-targeted HPV genotypes. All excluded HPV66-positive samples
tested RealTime positive and 16 out of 18 samples tested hc2 positive (although HPV66 is not
targeted by hc2). The analytical performance of the HPV assays was finally assessed on a
total of 4,479 specimens. As shown in Table 3, after hc2 repeat testing of 101 samples
according to the manufacturers’ instructions (initial results), excellent agreement between
RealTime and hc2, with a kappa value of 0.83 (95% CI, 0.80 to 0.85) and a percentage of
total agreement of 96.0% (4,304/4,479; 95% CI 95.5 to 96.6%), was obtained. After repeat
testing of 62 samples (30 samples by RealTime and 32 samples by hc2) at our discretion
(resolved results; see reasons for repeated testing in Materials and Methods), excellent
agreement with a kappa value of 0.84 (95% CI, 0.82 to 0.87) and a percentage of total
agreement of 96.4% (4,319/4,479; 95% CI 95.8 to 96.9%) was obtained.
14

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
HPV types determined in 160 samples with analytically discordant results between
RealTime and hc2 are presented in Table 4. Of the 44 samples detected by RealTime and not
by hc2 (resolved results), 38 samples were positive for at least one assay-common HPV type
and were considered to be analytically RealTime true positive/hc2 false negative. The hc2
most frequently missed HPV51 (in nine samples), followed by HPV16 and HPV18 (in six
samples each) (Table 4). Five RealTime positive/hc2 negative samples tested HPV DNA
negative using all three broad-range PCR-based genotyping methods applied and only low-
risk HPV6 was found in one RealTime positive/hc2 negative sample. These six samples were
considered to be analytically RealTime false positive/hc2 true negative samples.
Of the 116 samples detected by hc2 and not by RealTime (resolved results), only non-
targeted HPV types were identified in 62 samples and 24 samples tested HPV DNA negative
using all three broad-range PCR-based genotyping methods applied (Table 4). These 86
samples were considered to be analytically hc2 false positive/RealTime true negative. The
most frequently identified non-targeted HPV type causing an hc2 false positive result was
HPV53 (in at least 15 samples), followed by HPV67 and HPV70 (in at least five samples
each). In 30 RealTime negative/hc2 positive samples, genotyping showed the presence of at
least one assay-common HPV type and these samples were considered to be analytically hc2
true positive/RealTime false negative. Of these 30 samples, 16 samples repeatedly showed
some degree of RealTime HPV-specific amplification but Ct values were above the
manufacturer’s fixed assay cutoff (cycle of 32 nd ) and 14 samples were RealTime HPV non-
reactive. Among these RealTime negative/hc2 positive samples, the RealTime most
frequently missed HPV68 (in ten samples).
Comparing RealTime and hc2 results against resolved HPV status, the analytical
sensitivity of RealTime was 94.8% (543/573; 95% CI, 92.6 to 96.4%) and analytical
specificity 99.8% (3,900/3,906; 95% CI, 99.7 to 99.9%). By comparison, these analytical
15

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
values for hc2 at a cutoff value of 1.00 RLU/CO were 93.4% (535/573; 95% CI, 91.0 to
95.3%) and 97.8% (3,820/3,906; 95% CI 97.3 to 98.2%), respectively. The analytical
accuracy of RealTime and hc2 in detecting 13 HPV types targeted by the two assays was
99.2% (4,443/4,479; 95% CI, 98.9 to 99.4%) and 97.2% (4,355/4,479; 95% CI, 96.7 to
97.7%), respectively; the values were significantly different (P < 0.0001). The analytical
accuracy of RealTime in detecting HPV16 and HPV18 at manufacturer’s fixed assay cutoff
(cycle of 32 nd ) was 99.8% (4,471/4,479; 95% CI, 99.6 to 99.9%) and 99.8% (4,473/4,479;
95% CI, 99.6 to 99.9%), respectively.
Intra-laboratory reproducibility of RealTime. In Ljubljana, there was excellent
agreement of overall RealTime HPV result between the two rounds of testing, with a kappa
value of 1.0 (95% CI, 0.98-1.0), percentage of agreement of 100% (500/500; 95% CI, 99.0 to
100.0%) and percentage of positive agreement of 100% (167/167; 95% CI, 97.1 to 100.0%).
After stratification of RealTime HPV-positive results into three categories (HPV16 positive,
HPV18 positive and positive for the other 12 HPVs), the percentage of agreement was 99.0%
(495/500; 95% CI, 97.5 to 99.6%), the percentage of positive agreement was 97.0% (162/167;
95% CI, 92.7 to 98.8%), with the kappa value being 0.98 (95% CI, 0.96-0.99). All HPV type-
specific discordant results were obtained in samples containing several HPV types (mixed
HPV infection): additional HPV type(s) were detected in four samples in the second testing
round (3x HPV18, 1x other hrHPVs) and in one sample in the first testing round (other
hrHPVs).
In Hannover, there was also excellent agreement of overall RealTime HPV result between
the two rounds of testing, with a kappa value of 0.99 (95% CI, 0.98-1.0), percentage of
agreement of 99.8% (499/500; 95% CI, 98.7 to 99.9%) and percentage of positive agreement
of 99.4% (166/167; 95% CI, 96.2 to 99.9%). One sample was positive for HPV16 in the first
testing round (Ct = 30.21) and HPV negative in the second, but with HPV16-specific
16

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
amplification near cutoff (Ct = 32.28). After stratification of RealTime HPV-positive results
into three categories, the percentage of agreement was 99.0% (495/500; 95% CI, 97.5 to
99.6%), the percentage of positive agreement was 97.0% (162/167; 95% CI, 92.7 to 98.8%),
with the kappa value being 0.98 (95% CI, 0.96-0.99). HPV type-specific discordant results
were obtained in a total of five samples: in a previously described sample with an HPV16
discordant result and in four samples with mixed HPV infection: additional HPV type(s) were
detected in two samples in the first testing round (1x HPV18, 1x other hrHPVs) and in two
samples in the second testing round (2x HPV18).
Inter-laboratory agreement of RealTime. There was excellent agreement of overall HPV
result between Ljubljana and Hannover laboratories in the first and second testing rounds,
with kappa values of 1.0 (95% CI, 0.98-1.0) and 0.99 (95% CI, 0.98-1.0), respectively,
percentage of agreement of 100.0% (500/500; 95% CI, 99.0%-100.0%) and 99.8% (499/500;
95% CI, 98.7%-99.9%), respectively, and percentage of positive agreement of 100.0%
(167/167; 95% CI, 97.1%-100.0%) and 99.4% (166/167; 95% CI, 96.2%-99.9%),
respectively. In the second testing round, one sample was positive for HPV16 in Ljubljana (Ct
= 30.15) and HPV16 negative when tested in Hannover but with HPV16-specific
amplification near the cutoff (Ct = 32.28). After stratification of RealTime HPV-positive
results into three categories, in the first and second testing rounds the percentage of agreement
was 99.2% (496/500; 95% CI, 97.8%-99.7%) and 98.2% (491/500; 95% CI, 96.4%-99.1%),
respectively, percentage of positive agreement 97.6% (163/167; 95% CI, 93.5%-99.2%) and
94.6% (158/167; 95% CI, 89.7%-97.3%), respectively, with kappa values of 0.98 (95% CI,
0.96-0.99) and 0.96 (95% CI, 0.94-0.98), respectively. In the first testing round, all HPV type-
specific discordant results were obtained in samples with mixed HPV infection: additional
HPV type(s) were detected in three samples in Ljubljana (1x HPV16, 2x HPV18) and in one
sample in Hannover (1x other hrHPVs). In the second testing round, HPV type-specific
17

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
discordant results were obtained in a total of nine samples: in a previously described single
sample with HPV16 discordant result; in one sample in which different HPV types were
detected (in Ljubljana HPV16, in Hannover other hrHPVs); and in seven samples with mixed
HPV infection: additional HPV type(s) were detected in six samples in Ljubljana (5x HPV18,
1x other hrHPVs) and in one sample in Hannover (1x HPV18).
DISCUSSION
The most important consideration when evaluating an assay for routine detection of
hrHPVs in cervical specimens is the clinical accuracy for the detection of cervical high-grade
lesions (10, 27, 32, 46). A clinically useful hrHPV assay should have balanced clinical
sensitivity and clinical specificity for CIN2+ lesions to ensure reliable detection of women
with high-grade disease and to minimize HPV positive results in those with minimal risk of
disease (19, 32, 46). In recent years, it has become clear that many currently available
commercial hrHPV assays are not very useful for primary cervical cancer screening, mainly
as a result of misguided attempts to achieve perfect clinical sensitivity via increasing analytic
sensitivity (27, 32, 40). Such analytically highly sensitive HPV assays, although capable of
recognizing almost all women with underlying high-grade disease, usually yield a large
number of clinically insignificant positive results, which cause unnecessary clinical follow-
up, unnecessary diagnostics procedures and unnecessary treatment of healthy women (27). In
order to facilitate the evaluation and acceptance of novel hrHPV assays, guidelines describing
requirements for the use of HPV assay for primary cervical cancer screening have recently
been provided with European-North American collaboration (32). These guidelines
recommend the use of a so-called clinical validation strategy, based on analysis of the
equivalence of the result of the candidate hrHPV assay relative to that of an already clinically
validated reference HPV assay such as hc2, with clinical samples that originate from a
18

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
population-based cervical cancer screening, as assessed by the use of a noninferiority score
test (21, 32). According to the guidelines, three characteristics of the candidate hrHPV assay
should be assessed during the clinical validation process, using predetermined thresholds:
clinical sensitivity, clinical specificity and reproducibility (32).
RealTime is a recently launched next-generation HPV assay designed to detect a pool of 12
carcinogenic HPV genotypes in aggregate, with concurrent, separate detection of HPV16 and
HPV18. In the past, the clinical performance of RealTime has been assessed on preselected
archived cervical samples obtained from women with histologically confirmed cervical high-
grade lesions (25, 39, 47) or in triage settings on cervical samples obtained from women with
abnormal cytology referred for colposcopy (14, 20, 22). The clinical sensitivity of RealTime
for cervical high-grade lesions in these studies performed on a total of 1,481 histologically
confirmed CIN2+ lesions was above 96% in all six studies and noninferior to hc2 (14, 20, 22,
25, 39, 47). In the present study, we assessed for the first time the two remaining requirements
for the use of RealTime for primary cervical cancer screening: clinical specificity for the
detection of lesions less than CIN2 in women above 30 years in the population-based primary
cervical cancer screening setting and intra-laboratory reproducibility/inter-laboratory
agreement. Although our study was primarily designed and statistically powered to determine
the clinical specificity of RealTime (with power greater than 99%), applicable data were also
obtained relevant to clinical sensitivity of RealTime. In our primary study group for
evaluation of the clinical performance of RealTime (3,128 women above 30 years; 38 cases
and 3,091 controls), both the clinical specificity of RealTime for the detection of lesions less
than CIN2 and the clinical sensitivity of RealTime for the detection of CIN2+ lesion were
noninferior to those of the clinically validated reference HPV assay (hc2) with the use of
predetermined thresholds of 98% and 90% (relative specificity and relative sensitivity of
RealTime versus hc2), respectively. The favorable clinical performance of RealTime was also
19

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
confirmed in our secondary study group - total study population - which comprised 4,432
women 20-64 years old (57 cases and 4,375 controls). Some previous studies have shown that
hc2 might benefit from adjusting the cutoff value to 2.0-2.5 RLU/CO, which improved hc2
clinical specificity and PPV, while the effect on clinical sensitivity was minimal (42-43). This
was not confirmed in our study, since adjustment of the hc2 cutoff value from 1.00 to 2.5
RLU/CO resulted in a marginal improvement of hc2 clinical specificity but a substantial
reduction of its clinical sensitivity. This was probably the result of an specific distribution of
hc2 RLU/CO values among cases, i.e., three women with CIN2+ had RLU/CO values
between 1.00 and 2.00: 1.18, 1.44 and 1.67.
The clinical specificity of the hc2 assessed in our study is in agreement with results
obtained in similar previous studies. Recent meta-analysis of HPV primary screening trials
thus showed a pooled clinical specificity of hc2 in North American and European trials of
91.3% (95% CI, 89.5 to 93.1%; range: 85 to 95%)(1,32). In European HPV primary screening
studies, hc2 and GP5+/6+ PCR pooled clinical specificity was 93.3% (95% CI, 92.9 to
93.6%) for women 35–49 years of age and 90.7% (95% CI, 90.4 to 91.1%) for all women
(12). In recent trials, hc2 clinical specificities were 93.2% (95% CI, 92.8 to 93.6%) for
women 35–60 years of age (41) and 94.1% (95% CI, 93.4 to 94.8%) for women 30–69 years
of age (31). Due to the fact that the hrHPV prevalence among Slovenian women is still
relatively high in the age group 30-34 years (12.8% by Abbott; 14.4% by hc2), based on our
results HPV primary screening in our country would be feasible only if it starts at the age of
35 years. Similar findings have been also recently been described in other European countries
(1, 12, 16, 41). The clinical specificity of the two HPV assays in our study improved
substantially in women above 35 years: RealTime had a clinical specificity of 94.4%
(2,188/2,317; 95% CI, 93.4 to 95.3%) and hc2 had a clinical specificity at a cutoff value of
1.00 RLU/CO of 93.0% (2,154/2,317; 95% CI, 91.9 to 94.0%).
20

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
In addition to the assessment of clinical performance, six previously published RealTime
evaluations have also assessed some analytical characteristics of the novel assay, mainly in
comparison to hc2 (14, 20, 22, 25, 39, 47) and one study examined RealTime analytical
performance in details (23). In these seven studies, RealTime showed comparable analytical
sensitivity to hc2 but superior analytical specificity. These findings were also confirmed in the
present study, in which the analytical performance of the two HPV assays were compared on
the largest sample collection to date (4,479 samples). Excellent analytical agreement between
RealTime and hc2, with a kappa value of 0.84 and a total agreement of 96.4% was obtained in
our study but the analytical accuracy of RealTime in detecting 13 assay-common HPV types
was significantly higher than that of hc2. The vast majority of 116 RealTime negative/hc2
positive samples were considered to be analytically hc2 false positive and were probably the
consequence of previously described hc2 probe cocktail cross-reactivity with untargeted HPV
types (7, 38). Similarly to previous findings, the four most frequently identified non-targeted
HPV types causing hc2 false positive results in our study were HPV66, HPV53, HPV67 and
HPV70. One quarter of the RealTime negative/hc2 positive samples were considered to be
analytically RealTime false negative results: approximately half of these samples repeatedly
showed some degree of RealTime HPV-specific amplification but with a cycle number
beyond the assay cutoff and approximately half of samples were RealTime HPV non-reactive.
RealTime most often missed HPV68, probably due to the lower ability of the GP5+/6+ primer
mix to detect the HPV68 prototype. This problem is not limited only to GP5+/6+ based assays
such as RealTime. PGMY-primer based assays such as Linear Array, designed to detect
HPV68 subtype b, also cannot detect the HPV68 prototype because of several mismatches
(34). HPV68 was the least commonly detected hrHPV type in the recent 2009 and 2010 HPV
DNA Proficiency Panels prepared by the World Health Organization HPV Laboratory
Network – LabNet; this HPV type was correctly identified by fewer than 38% of participating
21

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
laboratories (17). Of the 44 samples detected in our study by RealTime and not by hc2, 38
(86.3%) were considered to be analytically RealTime true positive. hc2 most often missed
HPV51, HPV16 and HPV18, probably due to the presence of low levels of the HPV target,
which may not be reliably detected by hc2; i.e., approximately half of these samples had
repeated hc2 RLU/CO values between 0.60-0.99. Five RealTime positive/hc2 negative
samples tested HPV DNA negative using all three broad-range PCR-based genotyping
methods applied and only low-risk HPV6 was found in one RealTime positive/hc2 negative
sample. These six samples were considered to be analytically RealTime false positive; the
most probable reason for false positivity was amplicon contamination (all samples had late
Ct).
In the present study, intra-laboratory reproducibility and inter-laboratory agreement of
RealTime were assessed for the first time in two laboratories (Ljubljana and Hannover),
following the requirements set in guidelines for the evaluation of candidate HPV assays for
cervical cancer screening purposes (32). According to this document, intra-laboratory
reproducibility in time and inter-laboratory agreement should be determined by evaluation of
at least 500 samples, 30% of which tested positive in a reference laboratory using a clinically
validated assay. This should result in an agreement with a lower confidence boundary not less
than 87% (kappa value of at least 0.5 in this series of samples including 30% positives) and
the same intra-laboratory reproducibility should be achieved after testing the same set of
samples several weeks later. Our evaluation showed that RealTime can be considered to be a
reliable and robust HPV assay, since intra-laboratory and inter-laboratory kappa and
agreement values exceeded those set in the recommendations and those obtained in similar
previous evaluations of hc2 (5-6). As expected, the vast majority of otherwise infrequent HPV
type-specific discordant RealTime results were obtained in samples containing several HPV
types (mixed HPV infection), probably due to amplification competition. However, HPV
22

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
type-specific discordant results did not influence RealTime overall HPV positivity with the
exception of a single sample in the Hannover laboratory, in which presumably the low
quantity of HPV16 present in the sample produced a discordant borderline positive and
borderline negative result in the first and second testing rounds, respectively.
In summary, the evaluation of RealTime in the population-based cervical cancer screening
setting showed that the clinical performance of RealTime is not inferior to that of the
clinically validated reference HPV assay hc2, in women above 30 years and in women 20-64
years. Excellent analytical agreement between RealTime and hc2 results was obtained while
the analytical accuracy of RealTime was significantly higher than that of hc2. The typing
information for HPV16 and HPV18 provided by RealTime could serve as a valuable
additional tool in patient risk stratification and management. RealTime displayed high intra-
laboratory reproducibility and inter-laboratory agreement. According to our results and the
results of previous studies, RealTime can be considered to be a reliable and robust HPV assay
clinically comparable with hc2 for the detection of CIN2+ lesions in population-based
cervical cancer screening setting.
ACKNOWLEDGEMENT
Abbott Molecular, the National Institute of Public Health of Slovenia and the Institute of
Microbiology and Immunology, Faculty of Medicine, University of Ljubljana provided
financial support for the assays and logistical conduct of the study. Abbott Molecular was not
involved in the study design, data collection, data analysis and interpretation, or writing the
manuscript. The authors would like to thank clinical colleagues: Petra Bavčar, Irena Begič,
Lara Beseničar Pregelj, Martina Bučar, Simona Čopi, Petra Eržen Vrlič, Andreja Gornjec,
Mojca Grebenc, Mojca Jemec, Jožefa Kežar, Tatjana Kodrič, Zdravka Koman, Jasna
Kostanjšek, Jasna Kuhelj Recer, Zlatko Lazić, Sonja Lepoša, Mili Lomšek, Sladjana Malić,
23

1
2
3
4
5
6
7
8
9
Petra Meglič, Maja Merkun, Aleksander Merlo, Anamarija Petek, Suzana Peternelj Marinšek,
Igor Pirc, Uršula Reš Muravec, Filip Simoniti, Lucija Sorč, Tina Steinbacher Kokalj, Mateja
Darija Strah, Vesna Šalamun, Ksenija Šelih Martinec and Andrej Zore for patient recruitment
and management, Petra Markočič for study management, Petra Čuk, Robert Krošelj, Boštjan
J. Kocjan and Mateja Jelen for excellent laboratory assistance, Jasna Šinkovec, Marja Lenart,
and Boštjan Luzar for cytology and histology review, Matthias Jentschke for reproducibility
testing, Johannes Berkhof for help with noninferiority test calculations and Miha Pirc for
sample transportation.
24

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
REFERENCES
1. Arbyn, M., P. Sasieni, C. J. Meijer, C. Clavel, G. Koliopoulos, and J. Dillner. 2006.
Chapter 9: Clinical applications of HPV testing: a summary of meta-analyses. Vaccine.
24(Suppl. 3):78-89.
2. Arbyn, M., G. Ronco, J. Cuzick, N. Wentzensen, and P. E. Castle. 2009. How to
evaluate emerging technologies in cervical cancer screening? Int. J. Cancer. 125:2489-2496.
3. Barzon, L., C. Giorgi, F. M. Buonaguro, G. Palù, and Italian Society for Virology.
2008. Guidelines of the Italian Society for Virology on HPV testing and vaccination for
cervical cancer prevention. Infect. Agent Cancer. 3:14.
4. Bulkmans, N. W., J. Berkhof, L. Rozendaal, F. J. van Kemenade, A. J. Boeke, S. Bulk,
F. J. Voorhorst, R. H. Verheijen, K. van Groningen, M. E. Boon, W. Ruitinga, M. van
Ballegooijen, P. J. Snijders, and C. J. Meijer. 2007. Human papillomavirus DNA testing
for the detection of cervical intraepithelial neoplasia grade 3 and cancer: 5-Year follow-up of
a randomised controlled implementation trial. Lancet. 370:1764–1772.
5. Carozzi, F. M., A. Del Mistro, M. Confortini, C. Sani, D. Puliti, R. Trevisan, L. De
Marco, A. G. Tos, S. Girlando, P. D. Palma, A. Pellegrini, M. L. Schiboni, P. Crucitti, P.
Pierotti, A. Vignato, and G. Ronco. 2005. Reproducibility of HPV DNA Testing by Hybrid
Capture 2 in a Screening Setting. Am. J. Clin. Pathol. 124:716–721.
6. Castle, P. E., C. M. Wheeler, D. Solomon, M. Schiffman, C. L. Peyton, and ALTS
Group. 2004. Interlaboratory reliability of Hybrid Capture 2. Am. J. Clin. Pathol. 122:238–
245.
7. Castle, P. E., D. Solomon, C. M. Wheeler, P. E. Gravitt, S. Wacholder, and M.
Schiffman. 2008. Human papillomavirus genotype specificity of hybrid capture 2. J. Clin.
Microbiol. 46:2595-2604.
8. Castle, P. E., A. C. Rodríguez, R. D. Burk, R. Herrero, S. Wacholder, M. Alfaro, J.
Morales, D. Guillen, M. E. Sherman, D. Solomon, M. Schiffman, and Proyecto
Epidemiológico Guanacaste (PEG) Group. 2009. Short term persistence of human
papillomavirus and risk of cervical precancer and cancer: population based cohort study. Br.
Med. J. 339:b2569.
9. Castle, P. E., M. Sadorra, T. Lau, C. Aldrich, F. A. Garcia, and J. Kornegay. 2009.
Evaluation of a prototype real-time PCR assay for carcinogenic human papillomavirus (HPV)
detection and simultaneous HPV genotype 16 (HPV16) and HPV18 genotyping. J. Clin.
Microbiol. 47:3344-3347.
10. Cox, J. T. 2009. History of the use of HPV testing in cervical screening and in the
management of abnormal cervical screening results. J. Clin. Virol. 45(Suppl. 1):S3-S12.
11. Cuschieri, K. S., H. A. Cubie. 2005. The role of human papillomavirus testing in cervical
screening. J. Clin. Virol. 32(Suppl. 1):S34-42.
25

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
12. Cuzick, J., C. Clavel, K. U. Petry, C. J. Meijer, H. Hoyer, S. Ratnam, A. Szarewski,
P. Birembaut, S. Kulasingam, P. Sasieni, and T. Iftner. 2006. Overview of the European
and North American studies on HPV testing in primary cervical cancer screening. Int. J.
Cancer. 119:1095-1101.
13. Cuzick, J., M. Arbyn, R. Sankaranarayanan, V. Tsu, G. Ronco, M. H. Mayrand, J.
Dillner, and C. J. Meijer. 2008. Overview of human papillomavirus-based and other novel
options for cervical cancer screening in developed and developing countries. Vaccine.
26(Suppl. 10):K29-41.
14. Cuzick, J., L. Ambroisine, L. Cadman, J. Austin, L. Ho, G. Terry, S. Liddle, R. Dina,
J. McCarthy, H. Buckley, C. Bergeron, W. P. Soutter, D. Lyons, and A. Szarewski. 2010.
Performance of the RealTime RealTime high-risk HPV test in women with abnormal cervical
cytology smears. J. Med. Virol. 82:1186-1191.
15. Desai, M.S., and H. A. Cubie. 2005. The HPV test in cervical screening: a brave new
world? Cytopathology. 16:3-6.
16. Dillner, J., M. Rebolj, P. Birembaut, K. U. Petry, A. Szarewski, C. Munk, S. de
Sanjose, P. Naucler, B. Lloveras, S. Kjaer, J. Cuzick, M. van Ballegooijen, C. Clavel, T.
Iftner, and Joint European Cohort Study. 2008. Long term predictive values of cytology
and human papillomavirus testing in cervical cancer screening: Joint European cohort study.
Br. Med. J. 337:a1754.
17. Eklund, C., T. Zhou, J. Dillner, and WHO Human Papillomavirus Laboratory
Network. 2010. Global proficiency study of human papillomavirus genotyping. J. Clin.
Microbiol. 48:4147-4155.
18. Evans, M. F., C. S. Adamson, L. Simmons-Arnold, and K. Cooper. 2005. Touchdown
General Primer (GP5+/GP6+) PCR and optimized sample DNA concentration support the
sensitive detection of human papillomavirus. BMC Clin. Pathol. 5:10.
19. Gravitt, P. E., F. Coutlée, T. Iftner, J. W. Sellors, W. G. Quint, and C. M. Wheeler.
2008. New technologies in cervical cancer screening. Vaccine. 26(Suppl. 10):K42-52.
20. Halfon, P., D. Benmoura, A. Agostini, H. Khiri, G. Penaranda, A. Martineau, and B.
Blanc. 2010. Evaluation of the clinical performance of the RealTime RealTime High-Risk
HPV for carcinogenic HPV detection. J. Clin. Virol. 48:246-250.
21. Hesselink, A. T., D. A. Heideman, J. Berkhof, F. Topal, R. P. Pol, C. J. Meijer, and P.
J. Snijders. 2010. Comparison of the clinical performance of PapilloCheck human
papillomavirus detection with that of the GP5+/6+-PCR-enzyme immunoassay in populationbased
cervical screening. J. Clin. Microbiol. 48:797-801.
22. Huang, S., B. Erickson, N. Tang, W. B. Mak, J. Salituro, J. Robinson, and K.
Abravaya. 2009. Clinical performance of RealTime RealTime High Risk HPV test for
detection of high-grade cervical intraepithelial neoplasia in women with abnormal cytology. J.
Clin. Virol. 45(Suppl. 1):S19–23.
26

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
23. Huang, S., N. Tang, W. B. Mak, B. Erickson, J. Salituro, Y. Li, E. Krumpe, G.
Schneider, H. Yu, J. Robinson, and K. Abravaya. 2009. Principles and analytical
performance of RealTime RealTime HR HPV test. J. Clin. Virol. 45(Suppl. 1):S13–17.
24. Jacobs, M. V., P. J. F. Snijders, A. J. C. van den Brule, T. J. M. Helmerhorst, C. J. L.
M. Meijer, and J. M. M. Walboomers. 1997. A general primer GP5+/GP6+-mediated PCRenzyme
immunoassay method for rapid detection of 14 high-risk and 6 low-risk human
papillomavirus genotypes in cervical scrapings. J. Clin. Microbiol. 35:791-795.
25. Kaliterna, V., S. Z. Lepej, and A. Vince. 2009. Comparison between the RealTime
RealTime High Risk HPV assay and the Hybrid Capture 2 assay for detecting high-risk
human papillomavirus DNA in cervical specimens. J. Med. Microbiol. 58:1662-1663.
26. Khan, M. J., P. E. Castle, A. T. Lorincz, S. Wacholder, M. Sherman, D. R. Scott, B. B.
Rush, A. G. Glass, and M. Schiffman. 2005. The elevated 10-year risk of cervical precancer
and cancer in women with human papillomavirus (HPV) type 16 or 18 and the possible utility
of type-specific HPV testing in clinical practice. J. Natl. Cancer Inst. 97:1072-1079.
27. Kinney, W., M. H. Stoler, and P. E. Castle. 2010. Special commentary: Patient safety
and the next generation of HPV DNA tests. Am. J. Clin. Pathol. 134:193-199.
28. Kocjan, B. J., M. Poljak, K. Seme, M. Potocnik, K. Fujs, and D. Z. Babic. 2005.
Distribution of human papillomavirus genotypes in plucked eyebrow hairs from Slovenian
males with genital warts. Infect. Genet. Evol. 5:255-259.
29. Kocjan, B. J., M. Poljak, and K. Seme. 2010. Universal ProbeLibrary based real-time
PCR assay for detection and confirmation of human papillomavirus genotype 52 infections. J.
Virol. Methods. 163:492-494.
30. Lörincz, A. T. 1996. Hybrid Capture method for detection of human papillomavirus DNA
in clinical specimens: a tool for clinical management of equivocal Pap smears and for
population screening. J. Obstet. Gynaecol. Res. 22:629-636.
31. Mayrand, M. H., E. Duarte-Franco, I. Rodrigues, S. D. Walter, J. Hanley, A.
Ferenczy, S. Ratnam, F. Coutle´e, E. L. Franco, and Canadian Cervical Cancer
Screening Trial Study Group. 2007. Human papillomavirus DNA versus Papanicolaou
screening tests for cervical cancer. N. Engl. J. Med. 357:1579–1588.
32. Meijer, C. J., J. Berkhof, P. E. Castle, A. T. Hesselink, E. L. Franco, G. Ronco, M.
Arbyn, F. X. Bosch, J. Cuzick, J. Dillner, D. A. Heideman, and P. J. Snijders. 2009.
Guidelines for human papillomavirus DNA test requirements for primary cervical cancer
screening in women 30 years and older. Int. J. Cancer. 124:516-520.
33. Naucler, P., W. Ryd, S. Törnberg, A. Strand, G. Wadell, K. Elfgren, T. Radberg, B.
Strander, O. Forslund, B. G. Hansson, E. Rylander, and J. Dillner. 2007. Human
papillomavirus and Papanicolaou tests to screen for cervical cancer. N. Engl. J. Med.
357:1589–1597.
34. Nazarenko, I., L. Kobayashi, J. Giles, C. Fishman, G. Chen, and A. Lorincz. 2008. A
novel method of HPV genotyping using Hybrid Capture sample preparation method combined
27

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
with GP5+/6+ PCR and multiplex detection on Luminex XMAP. J. Virol. Methods. 154:76-
81.
35. Nicula, F. A., A. Anttila, L. Neamtiu, M. P. Zakelj, R. Tachezy, A. Chil, M. Grce, and
V. Kesić. 2009. Challenges in starting organised screening programmes for cervical cancer in
the new member states of the European Union. Eur. J. Cancer. 45:2679-2684.
36. Petry, K. U., S. Menton, M. Menton, F. van Loenen-Frosch, H. de Carvalho Gomes,
B. Holz, B. Schopp, S. Garbrecht-Buettner, P. Davies, G. Boehmer, E. van den Akker, T.
Iftner. 2003. Inclusion of HPV testing in routine cervical cancer screening for women above
29 years in Germany: results for 8466 patients. Br. J. Cancer. 88:1570-1577.
37. Poljak, M., A. Brenčič, K. Seme, A. Vince, and I. J. Marin. 1999. Comparative
evaluation of first- and second-generation Digene Hybrid Capture assays for detection of
human papillomaviruses associated with high or intermediate risk for cervical cancer. J. Clin.
Microbiol. 37:796-797.
38. Poljak, M., I. J. Marin, K. Seme, and A. Vince. 2002. Hybrid Capture II HPV test
detects at least 15 human papillomavirus genotypes not included in its current high risk
cocktail. J. Clin. Virol. 25(Suppl. 3):S89-S97.
39. Poljak, M., A. Kovanda, B. J. Kocjan, K. Seme, N. Jancar, and E. Vrtacnik-Bokal.
2009. The RealTime RealTime High Risk HPV test: comparative evaluation of analytical
specificity and clinical sensitivity for cervical carcinoma and CIN 3 lesions with the Hybrid
Capture 2 HPV DNA test. Acta Dermatovenerol. Alp. Panonica Adriat. 18:94-103.
40. Poljak, M., and B. J. Kocjan. 2010. Commercially available assays for multiplex
detection of alpha human papillomaviruses. Exp. Rev. Anti. Infect. Ther. 8:1139-1162.
41. Ronco, G., M. van Ballegooijen, N. Becker, A. Chil, M. Fender, P. Giubilato, J.
Kurtinaitis, L. Lancucki, E. Lynge, A. Morais, M. O'Reilly, P. Sparen, O. Suteu, M.
Rebolj, P. Veerus, M. P. Zakelj, and A. Anttila. 2009. Process performance of cervical
screening programmes in Europe. Eur. J. Cancer. 45:2659-2670.
42. Sargent, A., A. Bailey, A. Turner, M. Almonte, C. Gilham, H. Baysson, J. Peto, C.
Roberts, C. Thomson, M. Desai, J. Mather, and H. Kitchener. 2010. Optimal threshold for
a positive hybrid capture 2 test for detection of human papillomavirus: data from the
ARTISTIC trial. J. Clin. Microbiol. 48:554-558.
43. Seme, K., K. Fujs, B. J. Kocjan, and M. Poljak. 2006. Resolving repeatedly borderline
results of Hybrid Capture 2 HPV DNA Test using polymerase chain reaction and genotyping.
J. Virol. Methods. 134:252-256.
44. Snijders, P. J., D. A. Heideman, and C. J. Meijer. 2010. Methods for HPV detection in
exfoliated cell and tissue specimens. APMIS. 118:520-528.
45. Stevens, M. P., S. M. Garland, and S. N. Tabrizi. 2006. Human papillomavirus
genotyping using a modified linear array detection protocol. J. Virol. Methods.135:124-
126.
28

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
46. Stoler, M. H., P. E. Castle, D. Solomon, and M. Schiffman. 2007. The expanded use of
HPV testing in gynecologic practice per ASCCP-guided management requires the use of wellvalidated
assays. Am. J. Clin. Pathol. 127:335–337.
47. Tang, N., S. Huang, B. Erickson, W. B. Mak, J. Salituro, J. Robinson, and K.
Abravaya. 2009. High-risk HPV detection and concurrent HPV 16 and 18 typing with
RealTime RealTime High Risk HPV test. J. Clin. Virol. 45(Suppl. 1):S25–29.
48. Walker, P., S. Dexeus, G. De Palo, R. Barrasso, M. Campion, F. Girardi, C. Jakob,
M. Roy, and Nomenclature Committee of the International Federation for Cervical
Pathology and Colposcopy. 2003. International terminology of colposcopy: an updated
report from the International Federation for Cervical Pathology and Colposcopy. Obstet.
Gynecol. 101:175-177.
49. zur Hausen, H. 2009. Papillomaviruses in the causation of human cancers - a brief
historical account. Virology. 384:260-265.
29

1
2
3
4
5
6
7
8
TABLE 1. Comparison of RealTime and hc2 resolved results stratified for cases (CIN2+
lesions) and controls (less than CIN2) in women above 30 years (N=3,129) and total study
population (N=4,432).
Study group and
RealTime results
Women above 30 years
Controls
30
hc2 results
No. (%) of samples
- + Total
- 2,816 (91.1) 69 (2.2) 2,885 (93.3)
+ 21 (0.7) 185 (6.0) 206 (6.7)
Total 2,837 (91.8) 254 (8.2) 3,091
Cases
Total study population
- 0 (0.0) 0 (0.0) 0 (0)
+ 1 (2.6) 37 (97.4) 38 (100.0)
Total 1 (2.6) 37 (97.4) 38
Controls
- 3,800 (86.9) 115 (2.6) 3,915 (89.5)
+ 39 (0.9) 421 (9.6) 460 (10.5)
Total 3,839 (87.8) 536 (12.2) 4,375
Cases
- 1 (1.8) 0 (0.0) 1 (1.8)
+ 2 (3.5) 54 (94.7) 56 (98.2)
Total 3 (5.3) 54 (94.7) 57

1
2
3
TABLE 2. Comparison of clinical sensitivity for the detection of CIN2+, clinical specificity for the detection of lesions less than CIN2, positive
predictive value and negative predictive value of RealTime and hc2 at cutoff values of 1.00 and 2.50 RLU/CO in women above 30 years and in
total study population.
Women above 30 years
RealTime 100.0%
Clinical sensitivity Clinical specificity Positive predictive value Negative predictive value
(38/38; 95% CI, 86.5-100.0%)
hc2 (1.00 RLU/CO) 97.4%
(37/38; 95% CI, 86.2-99.9%)
hc2 (2.50 RLU/CO) 92.1%
Total study population
(35/38; 95% CI, 78.6-98.3%)
RealTime 98.2%
(56/57; 95% CI, 90.6-100.0%)
hc2 (1.00 RLU/CO) 94.7%
(54/57; 95% CI, 85.4-98.9%)
hc2 (2.50 RLU/CO) 89.5%
(51/57; 95% CI, 78.5-96.0%)
93.3%
(2,885/3,091; 95% CI, 92.4-94.2%)
91.8 %
(2,837/3,091; 95% CI, 90.8-92.7%)
92.9%
(2,872/3,091; 95% CI, 92.0-93.8%)
89.5%
(3,915/4,375; 95% CI, 88.5-90.4%)
87.7 %
(3,839/4,375; 95% CI, 86.7-88.7%)
89.1%
(3,900/4,375; 95% CI, 88.2-90.0%)
31
15.6%
(38/244; 95% CI, 11.3-20.7%)
12.7%
(37/291; 95% CI, 9.1-17.1%)
13.8%
(35/254; 95% CI, 9.8-18.6%)
10.9%
(56/516; 95% CI, 8.3-13.9%)
9.2%
(54/590; 95% CI, 7.0-11.8%)
9.7%
(51/526; 95% CI, 7.3-12.6%)
100.0%
(2,885/2,885; 95% CI, 99.8-100.0%)
99.9%
(2,837/2,838; 95% CI, 99.8-100.0%)
99.9%
(2,872/2,875; 95% CI, 99.7-100.0%)
100.0%
(3,915/3,916; 95% CI, 99.9-100.0%)
99.9%
(3,839/3,842; 95% CI, 99.8-100.0%)
99.8%
(3,900/3,906; 95% CI, 99.7-99.9%)

1
2
3
4
5
6
7
8
9
10
TABLE 3. Analytical results of testing for 13 assay-common HPV types using RealTime and
hc2 on 4,479 samples after repeat testing of 101 samples by hc2 according to the
manufacturers’ instructions (initial results) and after repeat testing of 62 samples at our
discretion (30 samples by RealTime and 32 samples by hc2) (resolved results).
Testing result Initial results
No. (%) of samples
32
Resolved results
No. (%) of samples
RealTime positive/hc2 positive 490 (10.9) 505 (11.2)
RealTime negative/hc2 negative 3,814 (85.2) 3,814 (85.2)
RealTime positive/hc2 negative 51 (1.1) 44 (1.0)*
RealTime negative/hc2 positive 124 (2.8) 116 (2.6)**
TOTAL 4,479 4,479
*38 samples considered to be analytically RealTime true positive and 6 samples to be
RealTime false positive.
**86 samples considered to be analytically hc2 false positive and 30 samples to be hc2 true
positive.

1
2
3
TABLE 4. HPV types determined in 160 analytically discordant samples based on the
resolved hc2/RealTime results. Assay-common HPV types are shown in bold.
Sample no hc2 RealTime HPV type(s) Interpretation Frequency
result result
1-6 negative positive 18 RealTime TP 6
7-11 negative positive 51 RealTime TP 5
12-15 negative positive 16 RealTime TP 4
16-18 negative positive 45 RealTime TP 3
19-21 negative positive 52 RealTime TP 3
22-23 negative positive 31 RealTime TP 2
24-25 negative positive 59 RealTime TP 2
26 negative positive 35 RealTime TP 1
27 negative positive 6, 16 RealTime TP 1
28 negative positive 16, 55 RealTime TP 1
29 negative positive 39, 66 RealTime TP 1
30 negative positive 45, 89 RealTime TP 1
31 negative positive 51, 73 RealTime TP 1
32 negative positive 51, 89 RealTime TP 1
33 negative positive 56, 74 RealTime TP 1
34 negative positive 35, 53, 61 RealTime TP 1
35 negative positive 40, 52, 89 RealTime TP 1
36 negative positive 51, 53, 54 RealTime TP 1
37 negative positive 59, 62, 66 RealTime TP 1
38 negative positive 51, 56, 59, 89 RealTime TP 1
39-43 negative positive neg RealTime FP 5
44 negative positive 6 RealTime FP 1
45-68 positive negative neg RealTime TN 24
69-83 positive negative 53 RealTime TN 15
84-88 positive negative 67 RealTime TN 5
89-93 positive negative 70 RealTime TN 5
94-97 positive negative 42 RealTime TN 4
98-101 positive negative 69/71 RealTime TN 4
102-104 positive negative 82 RealTime TN 3
105-107 positive negative 89 RealTime TN 3
108-110 positive negative 53, 55 RealTime TN 3
33

1
2
111-112 positive negative 55 RealTime TN 2
113-114 positive negative 61 RealTime TN 2
115 positive negative 40 RealTime TN 1
116 positive negative 67 RealTime TN 1
117 positive negative 81 RealTime TN 1
118 positive negative 84 RealTime TN 1
119 positive negative 6, 81 RealTime TN 1
120 positive negative 6, 84 RealTime TN 1
121 positive negative 42, 82 RealTime TN 1
122 positive negative 42, 89 RealTime TN 1
123 positive negative 53, 62 RealTime TN 1
124 positive negative 53, 70 RealTime TN 1
125 positive negative 54, 62 RealTime TN 1
126 positive negative 62, 67 RealTime TN 1
127 positive negative 67, 89 RealTime TN 1
128 positive negative 6, 53, 62 RealTime TN 1
129 positive negative 54, 61, 62 RealTime TN 1
130 positive negative 42, 44, 70, 83 RealTime TN 1
131-140 positive negative 68 RealTime FN 10
141-145 positive negative 31 RealTime FN 5
146-148 positive negative 58 RealTime FN 3
149-150 positive negative 52 RealTime FN 2
151 positive negative 16 RealTime FN 1
152 positive negative 59 RealTime FN 1
153 positive negative 16, 67 RealTime FN 1
154 positive negative 31, 39 RealTime FN 1
155 positive negative 31, 53 RealTime FN 1
156 positive negative 31, 68 RealTime FN 1
157 positive negative 42, 45 RealTime FN 1
158 positive negative 45, 53 RealTime FN 1
159 positive negative 39, 53, 73 RealTime FN 1
160 positive negative 51, 67, 82 RealTime FN 1
TP, true positive; FP, false positive; TN, true negative; FN, false negative.
34