The American Journal of Managed Care July 2016
A Restricted Look at CRC Screening: Not Considering Annual Stool Testing as an Option
- The Cancer Intervention and Surveillance Modeling Network (CISNET) Colorectal Cancer (CRC) Working Group generated outcomes on benefits, harms, and burdens of several CRC screening strategies defined by the US Preventive Services Task Force.
- Annual fecal occult blood testing has been shown to be effective in randomized controlled trials and all 3 CISNET models recommended annual fecal immunochemical testing over all other stool-based testing strategies.
- Berger et al eliminated annual stool-based CRC screening strategies from consideration and re-did the analysis conducted by the CISNET CRC Working Group. With this exclusion, they found that multi-target stool DNA (mt-sDNA) every 3 years would be recommendable according to 1 of the 3 CISNET models.
Karen M. Kuntz, ScD; Ann G. Zauber, PhD; Amy B. Knudsen, PhD; Carolyn M. Rutter, PhD; and Iris Lansdorp-Vogelaar, PhD
The US Preventive Services Task Force (USPSTF) released draft recommendations for colorectal cancer (CRC) screening in October 2015.1 As part of that effort, the USPSTF requested simulation modeling to inform their recommendations. The Cancer Intervention and Surveillance Modeling Network (CISNET) CRC Working Group—a group of modelers with 3 independently developed microsimulation models of CRC—generated outcomes on benefits, harms, and burdens. They did so for a set of screening strategies defined by the Task Force in terms of age to begin screening, age to end screening, screening interval, and screening modality.2 The modeling study found that annual fecal immunochemical testing was the model-recommendable strategy among stool-based tests. In the February 2016 issue of The American Journal of Managed Care, Berger et al used the output generated by the CISNET models to conduct a post hoc analysis, and they reported that their results “meet the USPSTF criteria for a recommendation for [multi-target stool DNA] mt-sDNA 3y for routine screening.”3 This letter serves to clarify several points.
In their post hoc analysis, Berger et al eliminated one set of strategies—namely, annual screening with a stool test of any kind—that are typically recommended by several organizations, including the USPSTF. The authors were able to “demonstrate that the modeling performed by CISNET supports a recommendation of mt-sDNA testing every 3 years (3y) for routine screening based on the USPSTF’s criteria, in light of data demonstrating that patients are unlikely to adhere to annual testing.” However, the original CISNET analysis was meant to inform population guidelines; therefore, in consultation with the USPSTF, the CISNET CRC Working Group assumed perfect adherence to screening regimens, including receipt of all screening, diagnostic follow-up (eg, for positive stool tests), and surveillance tests. This assumption enabled the models to predict the maximum achievable benefit for each screening strategy. Eliminating a valid screening strategy for which randomized controlled trials of its efficacy are available4 because its adherence is presumed to be lower than other strategies in the population is not rational within this context, especially when continuing to assume perfect adherence for the other strategies.
Berger et al stated that they used the same algorithm adopted by CISNET researchers, which was outlined in the report, to select the recommendable strategy among the set of stool-based strategies; however, we believe deviation from the algorithm may have occurred. Specifically, one of the criteria for deeming a strategy “recommendable” was that it had to be at least 90% as effective as the colonoscopy strategy with the same screening start and stop ages. Although the 3y mt-sDNA strategy met this criterion in 1 model, it did not in the other 2 models. Thus, on a “majority rules” basis, this criterion was not met; in other words, across the 3 models, the average life-years gained by 3y mt-sDNA was 87.3% of the life-years gained by screening with colonoscopy every 10 years (model-specific estimates were 84.0%, 87.0%, and 90.8%), which is lower than the a priori criterion of 90%.
Finally, Berger et al frame their analysis under the assumption that the USPSTF recommendations process is one that is based solely on the analysis performed by the CISNET modelers. However, we would like to emphasize that the criteria in question were used by the CISNET modeling group to present the results in a meaningful way in order to narrow down the over 200 strategies to a set of potentially recommendable strategies. The modeling results were only one consideration among many used by the USPSTF when making their draft, and now their final recommendations.5 In addition, these criteria were established by the CISNET CRC Working Group prior to doing any analysis, and they were vetted with the USPSTF. The post hoc analysis conducted by Berger et al showed that if one eliminates annual stool-based testing from consideration as a screening strategy, then a strategy of mt-sDNA 3y would be considered recommendable according to 1 of the 3 CISNET models.
REPLY FROM THE AUTHORS
Barry M. Berger, MD FCAP, and Bernard Levin, MD, FACP
Kuntz et al have detailed their objections to our elaboration of their own data in our recent article. We appreciate the fact that they do not dispute our analysis but consider it post hoc. Given that these are their data and that their rules were set a priori, we extracted the information from the Cancer Intervention and Surveillance Modeling Network (CISNET) data tables to more fully report CISNET’s own findings. We expanded their modeling conclusions to be more consistent with current clinical practice, which does not routinely achieve the “optimal” annual screening strategies they recommend.1
The CISNET group indicates that the US Preventive Services Task Force (USPSTF) defined the scope of the microsimulation modeling by specifying age to begin screening, age to end screening, screening interval, and screening modality. These strategies were then evaluated in 3 separate microsimulation models.2 In January 2014, the USPSTF provided a research framework for public comment for its review of colorectal cancer (CRC) screening. However, that public comment did not specify the type of modeling or the rules of modeling that were to be used; such specification would have facilitated appropriate comment from technical and medical experts to inform the work of modelers, thereby ensuring the most accurate and clinically useful range of modeled outcomes. Although CISNET has a concern that our analysis was post hoc, we are concerned that their a priori rulemaking was deficient given that the models do not incorporate the SSA pathway and do not accurately account for differences in stool-based test performance. The new rules consider burdens (colonoscopy volume but not fecal immunochemical test (FIT)/fecal occult blood test [FOBT] volume) rather than harms (complications) as the comparator for benefits (life-years gained). We elaborate on these concerns below.
CISNET utilized models that were out-of-date with respect to the biology of colorectal carcinogenesis and which insufficiently distinguish the performance differences between FOBT/FIT as a single marker and multi-target stool DNA (mt-sDNA)—referred to as “FIT + DNA” in the CISNET models). These models credited FOBT/FIT-based screening with the detection of sessile serrated adenomas (SSA) 1 cm or larger with the same sensitivity as FOBT/FIT-detected conventional advanced adenomas. However, the evidence shows that FOBT/FIT is almost completely insensitive to such SSAs (5% sensitivity), while methylated stool DNA markers can identify 42% of SSAs 1 cm or larger in size even in the absence of any other advanced adenoma3; SSAs may be the precursor lesions for up to 33% of colorectal carcinomas.4 Furthermore, interval and missed cancers on colonoscopy commonly have a molecular DNA profile that indicates their frequent origin from these precursor lesions.4 Three of the 4 strategies recommended by CISNET modeling to the USPSTF cannot detect proximal colonic SSA’s directly, because they either include only annual FIT/FOBT testing or annual FIT with 10-year flexible sigmoidoscopy, which itself doesn’t reach the proximal colon. Approximately 1% (99 of 9989) of average-risk patients were found to have SSAs 1 cm or larger in size with no additional advanced adenomas.3 These lesions would only be found by random detection due to specificity failure of FIT/FOBT in 3 of the 4 strategies recommended to the USPSTF by CISNET.2
Secondly, high-grade dysplasia—the most biologically and clinically critical premalignant change—is detected significantly better by stool DNA-containing strategies (69%) than by FIT/FOBT as a single marker (46%).3 Finally, the detection of surgically curable stage CRC (stage I + II) is also significantly better with stool DNA containing strategies (94%) than FIT/FOBT as a single marker (70%).3
Thus, the models did not account for the differences in detection of SSA’s, high-grade dysplasia, and CRC by stage—the 3 key distinguishing features that lead to CRC prevention and to surgical cure of CRC and that separate stool DNA performance from FIT/FOBT as a single marker. These omissions biased the model outcomes toward the FIT/FOBT strategies previously recommended by the USPSTF for stool-based testing.
Updating models requires significant work and clinical data with which to accurately calibrate their outputs. We appreciate that CISNET was limited by the models available to them; however, they could have accounted for this deficiency by grouping the stool tests in a more relevant way that reflected differences in test biology and patient-related factors by placing multi-target stool DNA (mt-sDNA) in its own group, which then could have been examined for the effect of interval testing.
Without consultation outside the USPSTF internal process, the USPSTF and CISNET determined that the “burden” of FOBT, FIT, and “FIT + sDNA” (mt-sDNA) were equal. They then analyzed 9 stool strategies as single group (FIT 1y, 2y, and 3y; hsFOBT 1y, 2y, and 3y; mt-sDNA 1y, 3y, and 5y). This grouping is neither justified clinically nor biologically, and takes the simplistic view that the measure of burden is simply providing a stool sample. Lessened patient burden is associated with other factors related to mt-sDNA tests, including peace of mind from superior lesion detection, longer screening interval (3y), specifically designed collection kits to minimize patient interaction with stool, and an embedded nationwide 24/7 patient navigation system to ensure the highest completion levels of successful screening. The mt-sDNA test was designed to diminish burdens on patients, providers, and health systems, which supported grouping and analyzing mt-sDNA separately from FIT/FOBT.