Ann G. Zauber, PhD

		The Cancer Intervention and Surveillance Modeling Network (CISNET) Colorectal Cancer (CRC) Working Group generated outcomes on benefits, harms, and burdens of several CRC screening strategies defined by the US Preventive Services Task Force. 

		Annual fecal occult blood testing has been shown to be effective in randomized controlled trials and all 3 CISNET models recommended annual fecal immunochemical testing over all other stool-based testing strategies. 

		Berger et al eliminated annual stool-based CRC screening strategies from consideration and re-did the analysis conducted by the CISNET CRC Working Group. With this exclusion, they found that multi-target stool DNA (mt-sDNA) every 3 years would be recommendable according to 1 of the 3 CISNET models.

Karen M. Kuntz, ScD; Ann G. Zauber, PhD; Amy B. Knudsen, PhD; Carolyn M. Rutter, PhD; and Iris Lansdorp-Vogelaar, PhD

The US Preventive Services Task Force (USPSTF) released draft recommendations for colorectal cancer (CRC) screening in October 2015.

 As part of that effort, the USPSTF requested simulation modeling to inform their recommendations. The Cancer Intervention and Surveillance Modeling Network (CISNET) CRC Working Group—a group of modelers with 3 independently developed microsimulation models of CRC&mdash;generated outcomes on benefits, harms, and burdens. They did so for a set of screening strategies defined by the Task Force in terms of age to begin screening, age to end screening, screening interval, and screening modality.

 The modeling study found that annual fecal immunochemical testing was the model-recommendable strategy among stool-based tests. In the February 2016 issue of 

, Berger et al used the output generated by the CISNET models to conduct a post hoc analysis, and they reported that their results “meet the USPSTF criteria for a recommendation for [multi-target stool DNA] mt-sDNA 3y for routine screening.”

 This letter serves to clarify several points.

In their post hoc analysis, Berger et al eliminated one set of strategies—namely, annual screening with a stool test of any kind&mdash;that are typically recommended by several organizations, including the USPSTF. The authors were able to “demonstrate that the modeling performed by CISNET supports a recommendation of mt-sDNA testing every 3 years (3y) for routine screening based on the USPSTF’s criteria, in light of data demonstrating that patients are unlikely to adhere to annual testing.” However, the original CISNET analysis was meant to inform population guidelines; therefore, in consultation with the USPSTF, the CISNET CRC Working Group assumed perfect adherence to screening regimens, including receipt of all screening, diagnostic follow-up (eg, for positive stool tests), and surveillance tests. This assumption enabled the models to predict the maximum achievable benefit for each screening strategy. Eliminating a valid screening strategy for which randomized controlled trials of its efficacy are available

 because its adherence is presumed to be lower than other strategies in the population is not rational within this context, especially when continuing to assume perfect adherence for the other strategies.

Berger et al stated that they used the same algorithm adopted by CISNET researchers, which was outlined in the report, to select the recommendable strategy among the set of stool-based strategies; however, we believe deviation from the algorithm may have occurred. Specifically, one of the criteria for deeming a strategy “recommendable” was that it had to be at least 90% as effective as the colonoscopy strategy with the same screening start and stop ages. Although the 3y mt-sDNA strategy met this criterion in 1 model, it did not in the other 2 models. Thus, on a “majority rules” basis, this criterion was not met; in other words, across the 3 models, the average life-years gained by 3y mt-sDNA was 87.3% of the life-years gained by screening with colonoscopy every 10 years (model-specific estimates were 84.0%, 87.0%, and 90.8%), which is lower than the a priori criterion of 90%.

Finally, Berger et al frame their analysis under the assumption that the USPSTF recommendations process is one that is based solely on the analysis performed by the CISNET modelers. However, we would like to emphasize that the criteria in question were used by the CISNET modeling group to present the results in a meaningful way in order to narrow down the over 200 strategies to a set of potentially recommendable strategies. The modeling results were only one consideration among many used by the USPSTF when making their draft, and now their final recommendations.

 In addition, these criteria were established by the CISNET CRC Working Group prior to doing any analysis, and they were vetted with the USPSTF. The post hoc analysis conducted by Berger et al showed that if one eliminates annual stool-based testing from consideration as a screening strategy, then a strategy of mt-sDNA 3y would be considered recommendable according to 1 of the 3 CISNET models.

Barry M. Berger, MD FCAP, and Bernard Levin, MD, FACP

Kuntz et al have detailed their objections to our elaboration of their own data in our recent article. We appreciate the fact that they do not dispute our analysis but consider it post hoc. Given that these are their data and that their rules were set a priori, we extracted the information from the Cancer Intervention and Surveillance Modeling Network (CISNET) data tables to more fully report CISNET’s own findings. We expanded their modeling conclusions to be more consistent with current clinical practice, which does not routinely achieve the “optimal” annual screening strategies they recommend.

	The CISNET group indicates that the US Preventive Services Task Force (USPSTF) defined the scope of the microsimulation modeling by specifying age to begin screening, age to end screening, screening interval, and screening modality. These strategies were then evaluated in 3 separate microsimulation models.

 In January 2014, the USPSTF provided a research framework for public comment for its review of colorectal cancer (CRC) screening. However, that public comment did not specify the type of modeling or the rules of modeling that were to be used; such specification would have facilitated appropriate comment from technical and medical experts to inform the work of modelers, thereby ensuring the most accurate and clinically useful range of modeled outcomes. Although CISNET has a concern that our analysis was post hoc, we are concerned that their a priori rulemaking was deficient given that the models do not incorporate the SSA pathway and do not accurately account for differences in stool-based test performance. The new rules consider burdens (colonoscopy volume but not fecal immunochemical test (FIT)/fecal occult blood test [FOBT] volume) rather than harms (complications) as the comparator for benefits (life-years gained). We elaborate on these concerns below.

	CISNET utilized models that were out-of-date with respect to the biology of colorectal carcinogenesis and which insufficiently distinguish the performance differences between FOBT/FIT as a single marker and multi-target stool DNA (mt-sDNA)—referred to as “FIT + DNA” in the CISNET models). These models credited FOBT/FIT-based screening with the detection of sessile serrated adenomas (SSA) 1 cm or larger with the same sensitivity as FOBT/FIT-detected conventional advanced adenomas. However, the evidence shows that FOBT/FIT is almost completely insensitive to such SSAs (5% sensitivity), while methylated stool DNA markers can identify 42% of SSAs 1 cm or larger in size even in the absence of any other advanced adenoma

; SSAs may be the precursor lesions for up to 33% of colorectal carcinomas.

 Furthermore, interval and missed cancers on colonoscopy commonly have a molecular DNA profile that indicates their frequent origin from these precursor lesions.

 Three of the 4 strategies recommended by CISNET modeling to the USPSTF cannot detect proximal colonic SSA’s directly, because they either include only annual FIT/FOBT testing or annual FIT with 10-year flexible sigmoidoscopy, which itself doesn’t reach the proximal colon. Approximately 1% (99 of 9989) of average-risk patients were found to have SSAs 1 cm or larger in size with no additional advanced adenomas.

 These lesions would only be found by random detection due to specificity failure of FIT/FOBT in 3 of the 4 strategies recommended to the USPSTF by CISNET.

	Secondly, high-grade dysplasia—the most biologically and clinically critical premalignant change&mdash;is detected significantly better by stool DNA-containing strategies (69%) than by FIT/FOBT as a single marker (46%).

 Finally, the detection of surgically curable stage CRC (stage I + II) is also significantly better with stool DNA containing strategies (94%) than FIT/FOBT as a single marker (70%).

Thus, the models did not account for the differences in detection of SSA’s, high-grade dysplasia, and CRC by stage—the 3 key distinguishing features that lead to CRC prevention and to surgical cure of CRC and that separate stool DNA performance from FIT/FOBT as a single marker. These omissions biased the model outcomes toward the FIT/FOBT strategies previously recommended by the USPSTF for stool-based testing.

	Updating models requires significant work and clinical data with which to accurately calibrate their outputs. We appreciate that CISNET was limited by the models available to them; however, they could have accounted for this deficiency by grouping the stool tests in a more relevant way that reflected differences in test biology and patient-related factors by placing multi-target stool DNA (mt-sDNA) in its own group, which then could have been examined for the effect of interval testing.

	Without consultation outside the USPSTF internal process, the USPSTF and CISNET determined that the “burden” of FOBT, FIT, and “FIT + sDNA” (mt-sDNA) were equal. They then analyzed 9 stool strategies as single group (FIT 1y, 2y, and 3y; hsFOBT 1y, 2y, and 3y; mt-sDNA 1y, 3y, and 5y). This grouping is neither justified clinically nor biologically, and takes the simplistic view that the measure of burden is simply providing a stool sample. Lessened patient burden is associated with other factors related to mt-sDNA tests, including peace of mind from superior lesion detection, longer screening interval (3y), specifically designed collection kits to minimize patient interaction with stool, and an embedded nationwide 24/7 patient navigation system to ensure the highest completion levels of successful screening. The mt-sDNA test was designed to diminish burdens on patients, providers, and health systems, which supported grouping and analyzing mt-sDNA separately from FIT/FOBT.

	Further, the mt-sDNA test was a priori designed for longer-interval testing with high sensitivity and somewhat lower point specificity. CISNET found annual stool DNA testing not as efficient as annual FIT/FOBT with respect to colonoscopies generated. This was hardly surprising as mt-sDNA was never intended to be used annually. Annual FIT/FOBT testing, in typical clinical practice, has been well documented as a failed strategy

 in all but the most highly organized health systems

 who are willing to invest in the high cost of patient navigation—recently reported as $153 per person screened for patient support.

 In contrast, every patient in the United States who uses mt-sDNA has the benefit of an embedded patient navigation system, which significantly lowers patient and system burden and raises compliance. A recent study of mt-sDNA screening in previously screening-noncompliant Medicare patients showed 88% intent to screen compliance and 90% compliance with diagnostic colonoscopy in mt-sDNA—positive patients.

 Another study showed that 42% of mt-sDNA users aged 50 to 74 years had never been previously screened with FIT/FOBT or endoscopy.

	Therefore, we do not consider our analysis “post hoc,” so much as revealing the underlying analysis provided by CISNET for the 6 member group of multi-year stool tests (FIT 2y and 3y, hsFOBT 2y and 3y, and mt-sDNA 3y and 5y). The 3 annual strategies were eliminated in our grouping because annual testing is not successfully executed in general clinical practice and does not reflect how the mt-sDNA test was designed. Biennial testing with FIT/FOBT is a widely used strategy that more accurately reflects real-life FIT/FOBT test use and is supported by 5 controlled trials showing a mortality benefit.

 Annual testing would be optimal for FOBT and FIT because they are only of modest sensitivity for curable stage cancer, poorly sensitive for advanced adenoma, and insensitive to SSAs. By contrast, mt-sDNA 3y, as designed, provides superior CRC-related incidence and mortality reduction and more life-years gained than either biennial or triennial FIT/FOBT, according to CISNET. Providing “optimal strategies alone” is not helpful when the limitation of the models precludes identifying those strategies alone as optimal.

	We did not suggest eliminating the annual FIT/FOBT strategy from the recommendation; rather, we highlighted the multi-year FIT/FOBT strategies, supported by RCTs, which CISNET did not in the recommendation. We demonstrated that when the 6 multi-year stool test strategies are more properly grouped together for evaluation, the CISNET data reveal the mt-sDNA 3y to be a strategy more beneficial than FIT/FOBT, as most commonly used in actual daily practice. When examined in light of complications rather than number of colonoscopies generated, mt-sDNA has as good or better benefits to harms ratios as other recommended strategies. Complications are more directly reflective of patient harm than colonoscopy volume itself. Although colonoscopy may be a burden according to the USPSTF, it is not a significant harm per se absent complications.

	Kuntz et al suggest we may have deviated from their a priori “rules.” We conservatively reported CISNETs own findings in the Table. There was no “majority rules” rule pre-specified by CISNET in their report.

 In addition to mt-sDNA 3y being the only multi-year modality to achieve 90% of the life-years gained by colonoscopy 10y in at least 1 of the models, the key finding is that in all 3 CISNET models, mt-sDNA 3y significantly out-performs FIT 2y and 3y and FOBT 2y and 3y, with respect to CRC incidence and mortality reduction and life-years gained. Although Kuntz et al might argue that annual FIT/FOBT is optimal for FIT/FOBT use, while we don’t disagree, we do argue that this is functionally irrelevant to the majority of individuals who use FIT/FOBT intermittently as their only CRC screening approach.

	We recognize that other factors enter into USPSTF recommendations, although the Task Force has not openly specified what all of those factors might be.

 Given the reluctance of the USPSTF to engage in transparent scientific and clinical discussion outside its own selected group of advisors, CISNET modeling takes on a principal role “to inform their recommendations.” The issues we raised with the CISNET analysis arose because CISNET established their rulemaking without significant input from outside groups. A more collegial process could have better informed CISNET or highlighted the significant questions raised here, and thus, may have led CISNET to a different a priori analytic structure and technical report discussion. We stand by our analyses and hope this exchange of opinions will enhance the CISNET modeling process.

	Kuntz and colleagues provided the data that illustrate the benefit of mt-sDNA 3y over current clinical practice of FIT and FOBT 2y, 3y. We reported out CISNET’s data in the context of multi-year testing, a strategy used in many countries outside the United States and not specific to mt-sDNA. We are committed to providing relevant data for clinicians and policy makers to use in making continued improvements in healthcare for their patients.

 University of Minnesota (KMK), Minneapolis, MN; Memorial Sloan Kettering Cancer Center (AGZ), New York, NY; Massachusetts General Hospital (ABK), Boston, MA; RAND Corporation (CMR), Santa Monica, CA; Erasmus Medical Center (IL-V), Rotterdam, The Netherlands.

 The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (KMK, AGZ, ABK, CMR, IL-V); acquisition of data (KMK, AGZ, ABK, CMR, IL-V); analysis and interpretation of data (KMK, AGZ, ABK, CMR, IL-V); drafting of the manuscript (KMK); critical revision of the manuscript for important intellectual content (KMK, AGZ, ABK, CMR, IL-V); administrative, technical, or logistic support (AGZ); and supervision (KMK).

 Karen M. Kuntz, ScD, Division of Health Policy and Management, University of Minnesota, School of Public Health, 420 Delaware St SE, Minneapolis, MN 55455. E-mail: kmkuntz@umn.edu. 

 Exact Sciences Corporation (BMB, BL), Madison, WI.

 Barry M. Berger, MD is an employee of Exact Sciences Corporation, Bernard Levin, MD has previously served on the Exact Sciences Scientific Advisory Board.

 Concept and design (BB, BL); drafting of the manuscript (BB, BL); critical revision of the manuscript for important intellectual content (BB, BL).

	1. Draft recommendation statement: colorectal cancer—screening [table: recommended screening strategies for colorectal cancer]. US Preventive Services Task Force website. http://www. uspreventiveservicestaskforce.org/Page/Document/draft-recommendation-statement38/colorectal-cancer-screening2#table-recommended-screening-strategies-for-colorectal-cancer. Published October 2015. Accessed March 15, 2016.

	2. Zauber A, Knudsen A, Rutter CM, et al; Writing Committee of the Cancer Intervention and Surveillance Modeling Network (CISNET) Colorectal Cancer Working Group. Technical report—evaluating the benefits and harms of colorectal cancer screening strategies: a collaborative modeling approach [AHRQ pub. No. 14-05203-EF-2]. US Preventive Services Task Force website. http://tinyurl.com/hw9b9zl. Published October 2015. Accessed March 15, 2016.

	3. Berger BM, Parton MA, Levin B. USPSTF colorectal cancer screening guidelines: An extended look at multi-year interval testing. 

	4. Mandel JS, Bond JH, Church TR, et al. Reducing mortality from colorectal cancer by screening for fecal occult blood. Minnesota Colon Cancer Control Study. 

	5. US Preventive Services Task Force. Screening for colorectal cancer: US Preventive Services Task Force recommendation statement. 

 Barry M. Berger, MD FCAP, Chief Medical Officer, Exact Sciences Corporation, 441 Charmany Dr, Madison, WI 553719. E-mail: bberger@exactsciences.com.

1. Winawer SJ, Fischer SE, Levin B. Evidence-based reality-driven colorectal cancer screening guidelines: the critical relationship of adherence to effectiveness. 

. 2016;315(19):2065-2066. doi:10.1001/jama.2016.3377.

2. Zauber A, Knudsen A, Rutter CM, et al; Writing Committee of the Cancer Intervention and Surveillance Modeling Network (CISNET) Colorectal Cancer Working Group. Technical report—evaluating the benefits and harms of colorectal cancer screening strategies: a collaborative modeling approach [AHRQ pub. No. 14-05203-EF-2]. US Preventive Services Task Force website. http://tinyurl.com/hw9b9zl. Published October 2015. Accessed January 28, 2016.

3. Imperiale TF, Ransohoff DF, Itzkowitz SH, et al. Multitarget stool DNA testing for colorectal-cancer screening. 

. 2014;370(14):1287-1297. doi: 10.1056/NEJMoa1311194.

4. Rex D, Ahnen D, Baron JA, et al. Serrated lesions of the colorectum: review and recommendations from an expert panel. 

. 2012;107(9):1315-1329; quiz 1314, 1330. doi: 10.1038/ajg.2012.161.

5. Cyhaniuk A, Coombes ME. Longitudinal adherence to colorectal cancer screening guidelines. 

6. Gellad ZF, Stechuchak KM, Fisher DA, et al. Longitudinal adherence to fecal occult blood testing impacts colorectal cancer screening quality. 

. 2011;106(6):1125-1134. doi: 10.1038/ajg.2011.11.

Articles

This is a letter clarifying some points in an article published in the February 2016 issue of AJMC by Berger et al on colorectal cancer
screening guidelines.

Ann G. Zauber, PhD

Articles

A Restricted Look at CRC Screening: Not Considering Annual Stool Testing as an Option