Abstract

1108

annsaudimed

Annals of Saudi Medicine

Ann Saudi Med

King Faisal Specialist Hospital and Research Centre

PMC6074396

6074396

27478909

10.5144/0256-4947.2016.245

Predictive value of p16/Ki-67 immunocytochemistry for triage of women with abnormal Papanicolaou test in cervical cancer screening: a systematic review and meta-analysis

Chen

Cheng-Chieh

aHuang

Lee-Wen

bBai

Chyi-Huey

cLee

Chin-Cheng

a✉

aDepartment of Pathology and Laboratory Medicine bDepartment of Obstetrics and Gynecology, Shin Kong Wu Ho-Su Memorial Hospital cDepartment of Public Health, College of Medicine, Taipei Medical University, Taipei, Taiwan

✉

Correspondence: Dr. Chin-Cheng Lee, Department of Pathology and Laboratory Medicine, Shin Kong Wu Ho-Su Memorial Hospital, No. 95, Wen Change Road, Shih Lin District, Taipei City, Taipei, 111 Taiwan, T: 886-2-28332211 ext. 2121, F: 886-2-28389360, M002058@ms.skh.org.tw, ORCID: http://orcid.org/0000-0002-3064-5016

✉

Corresponding author.

Jul-Aug2016

364245

245–251

2192018

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

AbstractBACKGROUND

The Papanicolaou (Pap) test is one screening strategy used to prevent cervical cancer in developed countries. The p16/Ki-67 immunocytochemistry is a triage test performed on Pap smears in women with atypical squamous cells of undetermined significance (ASCUS) or low grade squamous intraepithelial lesion.

OBJECTIVE

Our objective was to review studies investigating the diagnostic performance of p16/Ki-67 dual stain for triage of women with abnormal Pap tests.

DESIGN

We conducted a systematic review and meta-analysis of diagnostic test accuracy studies.

SETTINGS

We followed the protocol of systematic review of diagnostic accuracy studies.

PATIENTS AND METHODS

We searched PubMed, The Cochrane Library, BioMed Central, and ClinicalTrials.gov for relevant studies. We included research that assessed the accuracy of p16/Ki-67 dual stain and high risk human papillomavirus testing for triage of abnormal Pap smears. Review articles and studies that provided insufficient data to construct 2×2 tables were excluded. Data synthesis was conducted using a random-effects model.

MAIN OUTCOME MEASURES

Sensitivity and specificity.

RESULTS

In seven studies encompassing 2628 patients, the pooled sensitivity and specificity of p16/Ki-67 for triage of abnormal Pap smear results were 0.91 (95% CI, 0.89 to 0.93) and 0.64 (95% CI, 0.62 to 0.66), respectively. No study used a case-control design. A subgroup analysis involving liquid-based cytology showed a sensitivity of 0.91 (95%CI, 0.89 to 0.93) and specificity of 0.64 (95%CI, 0.61 to 0.66).

CONCLUSIONS

Our meta-analysis of p16/Ki-67 dual stain studies showed that the test achieved high sensitivity and moderate specificity for p16/Ki-67 immunocytochemistry for high-grade squamous intraepithelial lesion and cervical cancer. We suggest that p16/Ki-67 dual stain might be a reliable ancillary method identifying high-grade squamous intraepithelial lesions in women with abnormal Pap tests.

LIMITATIONS

No study in the meta-analysis examined the accuracy of the p16/Ki-67 dual stain for interpretation of glandular neoplasms.

status

released

display-pdf

yes

is-olf

is-manuscript

is-preprint

is-journal-matter

is-scanned

is-retracted

The Pap test, introduced by George Papanicolaou, is a cytological screening examination that detects abnormal epithelial cells scraped from transformation zone of uterine cervix.¹ Use of the technique in cervical cancer screening reduces the incidence and mortality of cervical cancer in developed countries.²^,³ However, women with equivocal cytological results usually undergo repeat Pap smears to obtain a definite diagnosis, which may cause anxiety. Immunocytochemistry for p16 protein has been shown as an effective approach for triage in women with cytological abnormalities, such as atypical squamous cells of undetermined significance (ASCUS).⁴ The p16 protein, a cyclin dependent kinase inhibitor, decelerates the cell cycle by facilitating the re-binding of retinoblastoma protein (Rb) and E2F transcription factor. The E7 oncoprotein, which is the product of E7 oncogene of hr-HPV interrupts the linkage between Rb and E2F transcription factor, resulting in disruption of Rb/E2F pathway.⁴ Therefore, overexpression of p16 protein in a dysplastic cervical epithelial cell is indicative of hr-HPV induced transformation.⁵ The Ki-67 antigen is a nuclear protein that expresses during all phases of the cell cycle except G0. Under normal physiological conditions, its expression is limited in basal layer squamous epithelium of uterine cervix. A staining protocol that simultaneously detects p16 protein and Ki-67 antigen in the same cervical epithelial cell has been established.⁶ Co-expression of p16 protein (tumor suppressor marker) and Ki-67 antigen (proliferative marker) in the same cervical epithelial cell indicates deregulation of the cell cycle and is supposed to be a positive test result.⁶ The hr-HPV testing has also been proposed as a tool for triage of equivocal or low-grade cytological abnormalities as its role in cervical cancer screening evolves.⁷^,⁸ A systematic review and meta-analysis showed that p16 immunocytochemistry has higher specificity than hr-HPV testing to detect underlying high-grade squamous intraepithelial lesions (HSIL) in triage of ASUCS or low-grade squamous intraepithelial lesions (LSIL).⁹ Estimates of p16/Ki-67 dual stain in triage of abnormal Pap smear differ from study to study. The aim of the review was to evaluate the accuracy of p16/Ki-67 dual stain for triage of women with abnormal Pap smear results.

PATIENTS AND METHODS

We followed the review protocol based on the guideline for systematic review of diagnostic accuracy studies and used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement as framework for reporting the review.¹⁰^,¹¹

Literature search

We searched PubMed, The Cochrane Library, BioMed Central, and ClinicalTrials.gov using Boolean logic of the following search terms: patients (cervical intraepithelial neoplasia OR CIN OR cervical dysplasia OR HSIL OR LSIL OR squamous intraepithelial lesion OR precancer), intervention (cytology OR pap smear OR pap test) (p16 or p16 protein or Ki-67 or Ki-67 protein or immunocytochemistry or p16/Ki-67 dual stain), and outcome (sensitivity OR diagnostic accuracy). The search strategy in PubMed is in the Appendix. No publication date restriction was applied. English language studies were reviewed. The last search was performed on October 4, 2014.

Study selection and quality assessment

We included studies that addressed the test performance of p16/Ki-67 immunocytochemistry and hr-HPV testing, as a comparator, for triage of women with cytological abnormalities. We also included primary studies focused on p16/Ki-67 dual stain performed on conventional smear or liquid-based cytology. Either prospective trials or retrospective studies were included. We excluded review articles and research that reported insufficient data for calculating sensitivity and specificity. One reviewer initially screened titles and abstracts for potentially relevant studies. Two reviewers independently examined full-text articles to determine eligibility. Disagreements were resolved by discussion. We assessed the methodological quality of the included studies using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2).¹² The QUADAS-2 consists of four domains: patient selection, index test, reference standard, and flow and timing. Each domain has “signaling questions” that help in judging the bias and applicability of studies. A study is considered to be high quality if it used consecutive recruitment of participants, all participants used the same reference standard, index test interpretation was done without knowledge of the reference standard, and all recruited participants were included in analyses.

Data extraction and synthesis

We extracted data from each study to construct a 2×2 table of true positives, true negatives, false positives, and false negatives to calculate sensitivity and specificity of both the p16/Ki-67 dual stain and hr-HPV testing. Sensitivity is the proportion of true positive results out of all who have the target condition (illness) whereas specificity is the proportion of true negative results out of all who do not have the target condition (illness). Diagnostic odds ratio (DOR) was calculated as sensitivity/(1- sensitivity) over (1- specificity)/specificity. ¹³ A diagnostic test is thought to be discriminative if it has a DOR greater than one.¹⁴ We defined p16/Ki-67 immunocytochemistry performed on abnormal Pap smear as the index test. We considered molecular testing for hr-HPV as the comparator. A biopsy sample taken from uterine cervix was regard as the reference standard.

We conducted a meta-analysis of p16/Ki-67 results extracted from included studies to generate pooled sensitivity, specificity, and DOR using a random-effects model. We plotted the summary receiver operating characteristic (SROC) curve to demonstrate overall test performance of p16/Ki-67 dual stain. The closer the curve approaches the upper left-handed corner, the better overall diagnostic performance of the test.¹⁵ We also carried out a meta-analysis for hr-HPV assays using a random-effects model to produce summary sensitivity and specificity. Pooled estimates including sensitivity, specificity, and DOR were generated along with associated 95% confidence interval (95% CI). Owing to inconsistent study design and cytological preparation, we investigated heterogeneity across studies. I² is one of the indicators of quantitative evaluation of heterogeneity. The I² expresses the percentage of the total variation across studies due to true differences rather than chance. Values of 25%, 50%, and 75% represent low, moderate, and high level of heterogeneity, respectively. ¹⁶ All analyses were performed by using MetaDiSc 1.4 software package.¹⁷

RESULTSOverall accuracy of p16/Ki-67 dual stain and hr- HPV assay

Seven eligible studies¹⁸^–²⁴ comprising 2628 participants were identified (Figure 1). Five studies were carried out with prospective study design (Table 1).¹⁸^–²² Table 2 summarizes the test results for each study. In the two retrospective studies, one retrieved Pap smears from their departmental files and the other used the residual specimens collected from the authors’ previous research.²³^,²⁴ One study used conventional smears and the other used liquid-based cytology. Schmidt et al reported two sets of estimates based on triages of ASCUS and LSIL.²⁴ To avoid double counting of results from the same participants, we extracted data for the LSIL triage. The summary sensitivity and specificity of p16/Ki-67 dual stain for triage of abnormal Pap smear results were 0.91 (95% CI, 0.89 to 0.93) and 0.64 (95% CI, 0.62 to 0.66), respectively (Figure 2, Figure 3). The meta-analysis for p16/Ki-67 dual stain yielded a pooled DOR of 15.35 (95% CI, 8.2 to 28.74) (Figure 4), showing the discriminatory power of the dual stain. We obtained pooled hr-HPV test estimates that had sensitivity of 0.94 (95%CI, 0.92 to 0.96) and specificity of 0.35 (95% CI, 0.32 to 0.37). The area under the curve (AUC) of an SROC plot presents the overall accuracy of a diagnostic test. A perfect test shows an AUC equal to 1. The AUC of an excellent test should be in the region of 0.97 or above. An AUC ranging from 0.93 to 0.96 is very good; an AUC of 0.75 to 0.92 is good.²⁵ The SROC curve (Figure 5) of p16/Ki-67 showed an AUC of 0.87, indicating that the overall test performance of p16/Ki-67 was good. The Youden index is defined as (sensitivity + specificity – 1).²⁶ A good diagnostic test has a Youden index over 0.5. The Youden index of p16/Ki-67 dual stain was 0.55. The meta-analysis, which was conducted using a random-effects model, demonstrated that p16/Ki-67 dual stain had comparable sensitivity and higher specificity, compared with hr-HPV testing.

Figure 1

Flow diagram of literature search.

Table 1

Characteristics of the seven included studies.

Study	Study design	No. of participant (total/data extraction)	Mean age (range)	Triage	Specimen type	Index test	Comparator	Detection	Reference standard

Ordi et al¹⁸	Prospective, consecutive	1169/1123	35.8 (18–91)	abnormal Pap test	ThinPrep	CINtec PLUS	Hybrid Capture 2	CIN2+	Biopsy, endocervical curettage
Killeen et al¹⁹	Prospective, consecutive	515/232	35.8 (21–86)	ASCUS, LSIL, ASC-H, HSIL	ThinPrep	CINtec PLUS	Cervista	CIN2/3	Biopsy
Vrdoljak-Mozetic et al²⁰	Prospective	155/145	34.9 (19–69)	CIN1	Conventional smear	CINtec PLUS	Hybrid Capture 2	CIN2+	Biopsy
Byun et al²¹	Prospective	56/56	46 (25–83)	ASC-H, LSIL-H	SurePath	CINtec PLUS	Cytoactive	CIN2+	Histology
Waldstroem et al²²	Consecutive	469/469	32.3 (16–25)	LSIL	ThinPrep	CINtec PLUS	APTIMA	CIN2+/CIN3+	Histology
Loghavi et al²³	Retrospective	188/188	median: 30 (15–77)	ASCUS, LSIL	SurePath	CINtec PLUS	Cervista	CIN2+	Histology
Schmidt et al²⁴	Retrospective, consecutive	810/415	>18	LSIL	ThinPrep	CINtec PLUS	Hybrid Capture 2	CIN2+/CIN3	Biopsy

ASCUS= atypical squamous cells of undetermined significance; ASC-H= atypical squamous cells cannot exclude high grade squamous intraepithelial lesion; CIN= cervical intraepithelial neoplasia; CIN 2+ = cervical intraepithelial neoplasia 2 or worse; CIN 3+ = cervical intraepithelial neoplasia 3 or worse; HSIL= high-grade squamous intraepithelial lesion; LSIL= low-grade squamous intraepithelial lesion; LSIL-H= low-grade squamous intraepithelial lesion cannot exclude high grade squamous intraepithelial lesion. Manufacturers are as follows (listed in alphabetical order by product): APTIMA, Gen-Probe company; Cervista, Hologic; CINtec PLUS, Roche; Cytoactive, Cytoimmun Diagnostics GmbH; Hybrid Capture 2, QIAGEN; SurePath, Becton, Dickinson and Company; ThinPrep, Hologic

Table 2

Summary of test results for each study in the meta-analysis.

Study	True positive	False positive	False negative	True negative	Total

Ordi et al¹⁸	360	203	36	524	1123
Killeen et al¹⁹	33	75	2	122	232
Vrdoljak-Mozetic et al²⁰	16	36	4	89	145
Byun et al²¹	16	17	5	18	56
Waldstroem et al²²	77	186	10	196	469
Loghavi et al²³	30	74	1	83	188
Schmidt et al²⁴	129	89	8	189	415

Figure 2

Forest plot showing the pooled sensitivity estimate.

Figure 3

Forest plot showing the pooled specificity estimate.

Figure 4

Forest plot showing the pooled diagnostic odds ratio.

Figure 5

The SROC curve for p16/Ki-67 dual stain. Individual studies are shown as circles whose area is proportionate to the sample size of the study.

Investigation of heterogeneity

The I² index (45.3%) of the meta-analysis in sensitivity is lower than that (90.4%) in specificity. The Cochran’s Q test for the pooled DOR yielded a result of 20.34 (P=.0024), indicating that there was heterogeneity across studies. We performed a subgroup analysis to investigate heterogeneity. Study design might have an impact on accuracy of p16/Ki-67 dual stain. The prospectively conducted studies had a pooled sensitivity and specificity of 0.90 (95%CI, 0.87 to 0.93) and 0.69 (95% CI, 0.67 to 0.72), respectively. The pooled sensitivity and specificity of the index tests in studies using liquid-based cytology were 0.91 (95%CI, 0.89 to 0.93) and 0.64 (95%CI, 0.61 to 0.66). The results were nearly identical to these of all included studies. Two studies which addressed the triage of atypical squamous cells could not exclude high-grade squamous intraepithelial lesion (ASC-H) for cervical intraepithelial neoplasia 2 or worse (CIN2+) and had slightly decreased sensitivity of 0.88 (95%CI, 0.76 to 0.95) and specificity of 0.60 (95%CI, 0.54 to 0.67).¹⁹^,²¹

DISCUSSION

Our study showed high sensitivity and moderate specificity for the p16/Ki-67 dual stain for triage of abnormal Pap smears. The presence of illness could be ruled out if a diagnostic test that has high sensitivity for the condition yields a negative result. The meta-analysis generated the pooled estimates, indicating that the p16/Ki- 67 dual stain was superior to hr-HPV testing. Arbyn et al performed a meta-analysis that showed that two types of hr-HPV assays were less specific in detecting CIN 2+ in women with ASCUS or LSIL, compared with our study.²⁷ According to consensus guidelines developed by the American Society for Colposcopy and Cervical Pathology, hr-HPV testing is the management of choice in women with either ASCUS or LSIL.²⁸ We suggested that the p16/Ki-67 dual stain might be an alternative procedure in the management of patients with abnormal cervical cytology. Laboratory professionals could conduct a p16/Ki-67 dual stain by using the residual specimen after liquid-based cytology preparation. The potential benefit of the p16/Ki-67 dual stain for women with an abnormal Pap test is the reduction in repeat cytology which is a worrisome problem. Colposcopy is one way of managing women with LSIL cytology.²⁸ Negative p16/Ki-67 immunocytochemistry performed on an abnormal or equivocal Pap smear, which could be free of HSIL, might lead to decreased colposcopic referral, easing the gynecologists’ workload. Fujii et al stated that p16/Ki-67 immunocytochemistry was a more accurate triage test for identifying CIN2+ in LSIL or ASCUS specimen, compared with hr-HPV testing.²⁹ Their finding was compatible with our study outcome. The majority of HPV infections in young women are transient.³⁰ About 90% of women with new HPV infections resolve within two years.³⁰ A retrospective study of 680 biopsy-proven LSIL patients demonstrated that 49 percent regressed to negative at six months.³¹ Such phenomenon might lead to lower specificity of hr-HPV testing in triage of LSIL. Therefore, we prefer p16/Ki-67 dual stain to hr-HPV testing.

No study had a case-control study design and four studies reported consecutive enrollment of participants. ¹⁸^,¹⁹^,²²^,²⁴ Three studies reported that p16/Ki-67 dual stain was interpreted without knowledge of the result of pathological examination.¹⁸^,²²^,²³ One study recorded that the biopsy result was interpreted without knowledge of p16/Ki-67 dual stain.¹⁸ Three studies reported that they analyzed all participants.²¹^–²³ There were no concerns about patient selection, index test, or reference standards. Regarding applicability, the studies had a low risk of bias. Four studies had a high risk of bias owing to incomplete patient analyses (Table 3).¹⁸^–²⁰^,²⁴ The specificity estimates of the included studies ranged from 0.51 to 0.72. The meta-analysis of the p16/Ki-67 dual stain in specificity generated an I2 of 90.4%, indicating that study design, patient spectrum, triage, and specimen type across studies appeared to have greater impact on pooled specificity than on pooled sensitivity.

Table 3

Quality of the seven studies included in the meta-analysis.

Study	Risk of bias				Applicability concerns
Study	Patient selection	Index test	Reference Standard	Flow and timing	Patient selection	Index test	Reference standard

Ordi et al¹⁸	LR	LR	LR	HR	LR	LR	LR
Killeen et al¹⁹	LR	UR	UR	HR	LR	LR	LR
Vrdoljak-Mozetič et al²⁰	UR	UR	UR	HR	LR	LR	LR
Byun et al²¹	UR	UR	UR	UR	LR	LR	LR
Waldstroem et al²²	LR	LR	UR	LR	LR	LR	LR
Loghavi et al²³	UR	LR	UR	LR	LR	LR	LR
Schmidt et al²⁴	LR	UR	UR	HR	LR	LR	LR

HR=high risk; LR=low risk; UR=unknown risk

The strength of evidence of our study relied on a rigorous methodological quality assessment based on QUADAS-2. Regarding applicability, patient selection, index test, and the reference standards of studies had a low risk of bias. Two reviewers independently reviewed titles, abstracts, and full-texts. A meta-epidemiologic analysis has indicated that MEDLINE alone is sufficient for systematic reviews of diagnostic accuracy.³²

A limitation of our study is that no studies in the meta-analysis examined the accuracy of the p16/Ki-67 dual stain for interpretation of glandular neoplasms. Six studies did not report whether the reference standard results were interpreted without knowledge of the index tests.¹⁹^–²⁴ Four studies stated that not all patients were included in the analyses.¹⁸^–²⁰^,²⁴ Lijmer et al reported that studies using case-control study design might produce three-fold overestimation of diagnostic accuracy effect, compared with studies with clinical cohort.³³ No study in this review used a case-control study design.

In summary, our investigation yielded robust summary estimates of the p16/Ki-67 dual stain, indicating that it could be an accurate triage test to identify HSIL in abnormal Pap smear cytology. Further high quality studies using consecutive patient enrollment and complete participant analysis are required to improve understanding of the applicability of p16/Ki-67 dual stain in triage of women with abnormal Pap smear results. Future systematic reviews on the diagnostic accuracy of the p16/Ki-67 dual stain for triage of endocervical glandular abnormalities detected on Pap smears are also needed to explore the overall performance of the technique.

Appendix.Detailed search strategy in PubMed

Key words	Search details
cervical intraepithelial neoplasia	“cervical intraepithelial neoplasia”[MeSH Terms] OR (“cervical”[All Fields] AND “intraepithelial”[All Fields] AND “neoplasia”[All Fields]) OR “cervical intraepithelial neoplasia”[All Fields]
CIN	“Comput Nurs”[Journal] OR “cin”[All Fields] OR “Comput Intell Neurosci”[Journal] OR “cin”[All Fields] OR “Comput Inform Nurs”[Journal] OR “cin”[All Fields]
cervical dysplasia	“uterine cervical dysplasia”[MeSH Terms] OR (“uterine”[All Fields] AND “cervical”[All Fields] AND “dysplasia”[All Fields]) OR “uterine cervical dysplasia”[All Fields] OR (“cervical”[All Fields] AND “dysplasia”[All Fields]) OR “cervical dysplasia”[All Fields] OR “cervical intraepithelial neoplasia”[MeSH Terms] OR (“cervical”[All Fields] AND “intraepithelial”[All Fields] AND “neoplasia”[All Fields]) OR “cervical intraepithelial neoplasia”[All Fields] OR (“cervical”[All Fields] AND “dysplasia”[All Fields])
cytology	“cytology”[Subheading] OR “cytology”[All Fields] OR “cytological techniques”[MeSH Terms] OR (“cytological”[All Fields] AND “techniques”[All Fields]) OR “cytological techniques”[All Fields] OR “cytology”[All Fields] OR “cell biology”[MeSH Terms] OR (“cell”[All Fields] AND “biology”[All Fields]) OR “cell biology”[All Fields]
pap smear	“papanicolaou test”[MeSH Terms] OR (“papanicolaou”[All Fields] AND “test”[All Fields]) OR “papanicolaou test”[All Fields] OR (“pap”[All Fields] AND “smear”[All Fields]) OR “pap smear”[All Fields]
pap test	“papanicolaou test”[MeSH Terms] OR (“papanicolaou”[All Fields] AND “test”[All Fields]) OR “papanicolaou test”[All Fields] OR (“pap”[All Fields] AND “test”[All Fields]) OR “pap test”[All Fields]
sensitivity	“sensitivity and specificity”[MeSH Terms] OR (“sensitivity”[All Fields] AND “specificity”[All Fields]) OR “sensitivity and specificity”[All Fields] OR “sensitivity”[All Fields]
diagnostic	“diagnosis”[MeSH Terms] OR “diagnosis”[All Fields] OR “diagnostic”[All Fields]
p16 protein	“cyclin-dependent kinase inhibitor p16”[MeSH Terms] OR (“cyclin-dependent”[All Fields] AND “kinase”[All Fields] AND “inhibitor”[All Fields] AND “p16”[All Fields]) OR “cyclin-dependent kinase inhibitor p16”[All Fields] OR (“p16”[All Fields] AND “protein”[All Fields]) OR “p16 protein”[All Fields]
protein	“proteins”[MeSH Terms] OR “proteins”[All Fields] OR “protein”[All Fields]
immunocytochemistry	“immunohistochemistry”[MeSH Terms] OR “immunohistochemistry”[All Fields] OR “immunocytochemistry”[All Fields]
stain	“staining and labeling”[MeSH Terms] OR (“staining”[All Fields] AND “labeling”[All Fields]) OR “staining and labeling”[All Fields] OR “stain”[All Fields]

Footnotes

Conflicts of Interest

The authors have no conflicts of interest relevant to this article.

REFERENCES

Burd EM. Human papillomavirus and cervical cancer. Clin Microbiol Rev. 2003;16(1):1–17. doi: 10.1128/CMR.16.1.1-17.2003.

Aklimunnessa K, Mori M, Khan MM, Sakauchi F, Kubo T, Fujino Y, et al. Effectiveness of cervical cancer screening over cervical cancer mortality among Japanese women. Jpn J Clin Oncol. 2006;36(8):511–8. doi: 10.1093/jjco/hyl060.

Nygård JF, Skare GB, Thoresen SØ. The cervical cancer screening programme in Norway, 1992–2000: changes in Pap smear coverage and incidence of cervical cancer. J Med Screen. 2002;9(2):86–91. doi: 10.1136/jms.9.2.86.

Nieh S, Chen SF, Chu TY, Lai HC, Lin YS, Fu E, et al. Is p16(INK4A) expression more useful than human papillomavirus test to determine the outcome of atypical squamous cells of undetermined significance-categorized Pap smear? A comparative analysis using abnormal cervical smears with follow-up biopsies. Gynecol Oncol. 2005;97(1):35–40. doi: 10.1016/j.ygyno.2004.11.034.

Klaes R, Friedrich T, Spitkovsky D, Ridder R, Rudy W, Petry U, et al. Overexpression of p16(INK4A) as a specific marker for dysplastic and neoplastic epithelial cells of the cervix uteri. Int J Cancer. 2001;92(2):276–84. doi: 10.1002/ijc.1174.

Wentzensen N, Schwartz L, Zuna RE, Smith K, Mathews C, Gold MA, et al. Performance of p16/Ki-67 immunostaining to detect cervical cancer precursors in a colposcopy referral population. Clin Cancer Res. 2012;18(15):4154–62. doi: 10.1158/1078-0432.CCR-12-0270.

Cotton S, Sharp L, Little J, Cruickshank M, Seth R, Smart L, et al. The role of human papillomavirus testing in the management of women with low-grade abnormalities: multicentre randomised controlled trial. BJOG. 2010;117(6):645–59. doi: 10.1111/j.1471-0528.2010.02519.x.

Whitlock EP, Vesco KK, Eder M, Lin JS, Senger CA, Burda BU. Liquid-based cytology and human papillomavirus testing to screen for cervical cancer: a systematic review for the U.S. Preventive Services Task Force. Ann Intern Med. 2011;155(10):687–97. W214–5. doi: 10.7326/0003-4819-155-10-201111150-00376.

Roelens J, Reuschenbach M, von Knebel Doeberitz M, Wentzensen N, Bergeron C, Arbyn M. p16INK4a immunocytochemistry versus human papillomavirus testing for triage of women with minor cytologic abnormalities: a systematic review and meta-analysis. Cancer Cytopathol. 2012;120(5):294–307. doi: 10.1002/cncy.21205.

10.

Leeflang MM, Deeks JJ, Gatsonis C, Bossuyt PM Cochrane Diagnostic Test Accuracy Working Group. Systematic reviews of diagnostic test accuracy. Ann Intern Med. 2008;149(12):889–97. doi: 10.7326/0003-4819-149-12-200812160-00008.

11.

Moher D, Liberati A, Tetzlaff J, Altman DG The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. BMJ. 2009;339:332–6.

12.

Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529–36. doi: 10.7326/0003-4819-155-8-201110180-00009.

13.

Glas AS, Lijmer JG, Prins MH, Bonsel GJ, Bossuyt PM. The diagnostic odds ratio: a single indicator of test performance. J Clin Epidemiol. 2003;56(11):1129–35. doi: 10.1016/s0895-4356(03)00177-x.

14.

Birim O, Kappetein AP, Stijnen T, Bogers AJ. Meta-analysis of positron emission tomographic and computed tomographic imaging in detecting mediastinal lymph node metastases in nonsmall cell lung cancer. Ann Thorac Surg. 2005;79(1):375–82. doi: 10.1016/j.athoracsur.2004.06.041.

15.

Chartrand C, Leeflang MM, Minion J, Brewer T, Pai M. Accuracy of rapid influenza diagnostic tests: a meta-analysis. Ann Intern Med. 2012;156(7):500–11. doi: 10.7326/0003-4819-156-7-201204030-00403.

16.

Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557–60. doi: 10.1136/bmj.327.7414.557.

17.

Zamora J, Abraira V, Muriel A, Khan K, Coomarasamy A. Meta-DiSc: a software for meta-analysis of test accuracy data. BMC Med Res Methodol. 2006;6:31. doi: 10.1186/1471-2288-6-31.

18.

Ordi J, Sagasta A, Munmany M, Rodríguez-Carunchio L, Torné A, del Pino M. Usefulness of p16/Ki-67 immunostaining in the triage of women referred to colposcopy. Cancer Cytopathol. 2014;122(3):227–35. doi: 10.1002/cncy.21366.

19.

Killeen JL, Dye T, Grace C, Hiraoka M. Improved abnormal Pap smear triage using cervical cancer biomarkers. J Low Genit Tract Dis. 2014;18(1):1–7. doi: 10.1097/LGT.0b013e31828aeb39.

20.

Vrdoljak-Mozetič D, Krašević M, Verša Ostojić D, Stemberger-Papić S, Rubeša-Mihaljević R, Bubonja-Šonje M. HPV16 genotype, p16/Ki-67 dual staining and koilocytic morphology as potential predictors of the clinical outcome for cervical low-grade squamous intraepithelial lesions. Cytopathology. 2015;26(1):10–8. doi: 10.1111/cyt.12121.

21.

Byun SW, Lee A, Kim S, Choi YJ, Lee YS, Park JS. Immunostaining of p16(INK4a)/Ki-67 and L1 capsid protein on liquidbased cytology specimens obtained from ASC-H and LSIL-H cases. Int J Med Sci. 2013;10(12):1602–7. doi: 10.7150/ijms.6526.

22.

Waldstrøm M, Christensen RK, Ørnskov D. Evaluation of p16(INK4a)/Ki-67 dual stain in comparison with an mRNA human papillomavirus test on liquid-based cytology samples with low-grade squamous intraepithelial lesion. Cancer Cytopathol. 2013;121(3):136–45. doi: 10.1002/cncy.21233.

23.

Loghavi S, Walts AE, Bose S. CINtec® PLUS dual immunostain: a triage tool for cervical pap smears with atypical squamous cells of undetermined significance and low grade squamous intraepithelial lesion. Diagn Cytopathol. 2013;41(7):582–7. doi: 10.1002/dc.22900.

24.

Schmidt D, Bergeron C, Denton KJ, Ridder R European CINtec Cytology Study Group. p16/ki-67 dual-stain cytology in the triage of ASCUS and LSIL papanicolaou cytology: results from the European equivocal or mildly abnormal Papanicolaou cytology study. Cancer Cytopathol. 2011;119(3):158–66. doi: 10.1002/cncy.20140.

25.

Jones CM, Athanasiou T. Summary receiver operating characteristic curve analysis techniques in the evaluation of diagnostic tests. Ann Thorac Surg. 2005;79(1):16–20. doi: 10.1016/j.athoracsur.2004.09.040.

26.

Ruopp MD, Perkins NJ, Whitcomb BW, Schisterman EF. Youden Index and optimal cut-point estimated from observations affected by a lower limit of detection. Biom J. 2008;50(3):419–30. doi: 10.1002/bimj.200710415.

27.

Arbyn M, Roelens J, Cuschieri K, Cuzick J, Szarewski A, Ratnam S, et al. The APTIMA HPV assay versus the Hybrid Capture 2 test in triage of women with ASC-US or LSIL cervical cytology: a meta-analysis of the diagnostic accuracy. Int J Cancer. 2013;132(1):101–8. doi: 10.1002/ijc.27636.

28.

Massad LS, Einstein MH, Huh WK, Katki HA, Kinney WK, Schiffman M, et al. 2012 updated consensus guidelines for the management of abnormal cervical cancer screening tests and cancer precursors. Obstet Gynecol. 2013;121(4):829–46. doi: 10.1097/AOG.0b013e3182883a34.

29.

Fujii T, Saito M, Hasegawa T, Iwata T, Kuramoto H, Kubushiro K, et al. Performance of p16INK4a/Ki-67 immunocytochemistry for identifying CIN2+ in atypical squamous cells of undetermined significance and low-grade squamous intraepithelial lesion specimens: a Japanese Gynecologic Oncology Group study. Int J Clin Oncol. 2015;20(1):134–42. doi: 10.1007/s10147-014-0688-0.

30.

Steben M, Duarte-Franco E. Human papillomavirus infection: epidemiology and pathophysiology. Gynecol Oncol. 2007;107(2 Suppl 1):S2–5. doi: 10.1016/j.ygyno.2007.07.067.

31.

Bansal N, Wright JD, Cohen CJ, Herzog TJ. Natural history of established low grade cervical intraepithelial (CIN 1) lesions. Anticancer Res. 2008;28(3B):1763–6.

32.

van Enst WA, Scholten RJ, Whiting P, Zwinderman AH, Hooft L. Meta-epidemiologic analysis indicates that MEDLINE searches are sufficient for diagnostic test accuracy systematic reviews. J Clin Epidemiol. 2014;67(11):1192–9. doi: 10.1016/j.jclinepi.2014.05.008.

33.

Lijmer JG, Mol BW, Heisterkamp S, Bonsel GJ, Prins MH, van der Meulen JH, et al. Empirical evidence of design-related bias in studies of diagnostic tests. JAMA. 1999;282(11):1061–6. doi: 10.1001/jama.282.11.1061.