Currently Viewing:
The American Journal of Managed Care September 2014
Impact of Atypical Antipsychotic Use Among Adolescents With Attention-Deficit/Hyperactivity Disorder
Vanja Sikirica, PharmD, MPH; Steven R. Pliszka, MD; Keith A. Betts, PhD; Paul Hodgkins, PhD, MSc; Thomas M. Samuelson, BA; Jipan Xie, MD, PhD; M. Haim Erder, PhD; Ryan S. Dammerman, MD, PhD; Brigitte Robertson, MD; and Eric Q. Wu, PhD
Effective Implementation of Collaborative Care for Depression: What Is Needed?
Robin R. Whitebird, PhD, MSW; Leif I. Solberg, MD; Nancy A. Jaeckels, BS; Pamela B. Pietruszewski, MA; Senka Hadzic, MPH; Jürgen Unützer, MD, MPH, MA; Kris A. Ohnsorg, MPH, RN; Rebecca C. Rossom, MD, MSCR; Arne Beck, PhD; Kenneth E. Joslyn, MD, MPH; and Lisa V. Rubenstein, MD, MSPH
Is All "Skin in the Game" Fair Game? The Problem With "Non-Preferred" Generics
Gerry Oster, PhD, and A. Mark Fendrick, MD
Targeting High-Risk Employees May Reduce Cardiovascular Racial Disparities
James F. Burke, MD, MS; Sandeep Vijan, MD; Lynette A. Chekan, MBA; Ted M. Makowiec, MBA; Laurita Thomas, MEd; and Lewis B. Morgenstern, MD
HITECH Spurs EHR Vendor Competition and Innovation, Resulting in Increased Adoption
Seth Joseph, MBA; Max Sow, MBA; Michael F. Furukawa, PhD; Steven Posnack, MS, MHS; and Mary Ann Chaffee, MS, MA
Out-of-Plan Medication in Medicare Part D
Pamela N. Roberto, MPP, and Bruce Stuart, PhD
Currently Reading
New Thinking on Clinical Utility: Hard Lessons for Molecular Diagnostics
John W. Peabody, MD, PhD, DTM&H, FACP; Riti Shimkhada, PhD; Kuo B. Tong, MS; and Matthew B. Zubiller, MBA
Long-term Glycemic Control After 6 Months of Basal Insulin Therapy
Harn-Shen Chen, MD, PhD; Tzu-En Wu, MD; and Chin-Sung Kuo, MD
Characteristics Driving Higher Diabetes-Related Hospitalization Charges in Pennsylvania
Zhen-qiang Ma, MD, MPH, MS, and Monica A. Fisher, PhD, DDS, MS, MPH
Quantifying Opportunities for Hospital Cost Control: Medical Device Purchasing and Patient Discharge Planning
James C. Robinson, PhD, and Timothy T. Brown, PhD
Effects of a Population-Based Diabetes Management Program in Singapore
Woan Shin Tan, BSocSc, MSocSc; Yew Yoong Ding, MBBS, FRCP, MPH; Wu Christine Xia, BS(IT); and Bee Hoon Heng, MBBS, MSc
Predicting High-Need Cases Among New Medicaid Enrollees
Lindsey Jeanne Leininger, PhD; Donna Friedsam, MPH; Kristen Voskuil, MA; and Thomas DeLeire, PhD
Cost-effectiveness Evaluation of a Home Blood Pressure Monitoring Program
Sarah J. Billups, PharmD; Lindsy R. Moore, PharmD; Kari L. Olson, BSc (Pharm), PharmD; and David J. Magid, MD, MPH

New Thinking on Clinical Utility: Hard Lessons for Molecular Diagnostics

John W. Peabody, MD, PhD, DTM&H, FACP; Riti Shimkhada, PhD; Kuo B. Tong, MS; and Matthew B. Zubiller, MBA
The authors describe 5 basic requirements for planning, implementing, and proving clinical utility for diagnostic tests, drawing on recent reimbursement decisions.
Clinical simulation, already used widely in clinical performance measurement,5 offers a simple and cost-efficient way to do an early clinical utility study, even before validity studies are complete; in fact, it can be used to determine the parameters of validity studies. In our own experience using Clinical Performance and Values (CPV) vignettes, we have captured clinical behavior change, and once such changes are established in silico, it is possible and plausible to assert that when the test is launched and used broadly, there will be commensurate improvements in outcomes.6 A randomized controlled study of a recently approved multi-biomarker diagnostic assay, for example, used simulations successfully. The company demonstrated that when its test indicated a change in disease activity (ie, the test was validated), rheumatologists who used this test made the correct assessments and treatment decisions for simulated rheumatoid arthritis cases. Importantly, the measured change was not an outcome or a patient health measure, but the clinical decision to treat (or not treat). One clear advantage of CPV simulations is that they have been validated against actual practice,7-9 they are straightforward (randomly assigned, impartial physicians are asked to make hypothetical clinical decisions based upon having or not having the test results), and they remove patient-level variation. Another advantage is that they can assert that the test has utility before the validation studies are completed or even begun, thus accelerating the C&R process.

Starting early with simulations means starting data generation for utility at a much lower cost than would be the case with a full clinical equipoise study. In the event the test does not change clinical practice (ie, a negative utility study), the company has the opportunity to revisit its technology and seek another aspect of clinical practice the test might be able to change beneficially. Correspondingly, if the test does indicate a change in clinical practice, the company is appropriately encouraged to use the experimental sample frame and examine the impact on patients of the providers who use the test, as there is an expected cascading impact on actual patient outcomes stemming from clinical practice change. This links the early adopters of diagnostic tests to hard patient outcomes.

Lesson 3. Learn from successes (and failures). While the parameters of exactly what defines clinical utility are broad and may appear amorphous, MolDX makes clear that at a minimum, clinical utility is composed of good science, patient impact, and practice change. MolDx guidelines suggest 2 well-designed controlled experiments published in peer-reviewed journals, a significant number of subjects to establish clinical significance (including Medicare population in the study group), and demonstrated changes in physician treatment behavior based on the assay results and/or improved patient outcomes. Companies too often appear to simply check appropriate boxes on a form, believing that their work is done when they can present 2 studies, some patient results, and their earnest assurance that practitioners value the test. What they must realize, however, is that they need to be involved in serial evaluation of clinical utility (see Table).

Lesson 4. Determine clinical utility with rigorous science. For too long, companies have relied on retrospective studies, anecdotes, testimonials, and non-randomized studies to try to demonstrate clinical utility. Clinical utility, like clinical validity, can and must be determined experimentally. The Center for Medical Technology Policy, which develops and publishes methodological standards, has established guidelines on the design of prospective studies on clinical utility.10 They argue, as we do, that clinical utility must be examined in a scientifically rigorous manner and must be considered early on, with an analysis plan in place. What is scientifically “rigorous enough” is a common question, especially since randomized controlled trials may mean involving multiple sites and outcomes.

For many companies, developing experimental studies on utility means, to start, (re)thinking how a business creates an ongoing clinical utility research plan. Such a plan focuses on how a test will change care, its most effective and selective clinical uses, and its potential economic benefits (or costs), and it ultimately demonstrates—through a series of studies—how clinical utility can be generalized to different populations. A company’s leadership choices—perhaps made previously with expertise in validity in mind—may have to be tweaked when expertise in utility, and openness to new ways of thinking, is desired as well.

With ongoing reimbursement changes expected, stricter and more narrowly specified demands for experimental evidence on clinical utility are likely. Non-randomized trials, for example, fail to meet the new evidence standards. The better strategy is to conduct smaller, well-designed randomized controlled trials to identify exactly the clinical outcomes that will build the evidence chain for clinical utility. In contrast, a large clinical equipoise study early on in test development is fraught with too much sample error and too many uncertainties related to power estimations and variations in practices and patients to be an efficient use of resources. The prudent initial smaller-study approach, however, requires close attention to sample size, variance, and effect size calculations.

Another important avenue to consider is Coverage with Evidence Determination (CED),11 under which CMS coverage is given to effective but unproven diagnostic

tests, contingent on providing evidence to support clinical utility and demonstrating that the principal purpose of the study is to test whether a particular intervention improves health outcomes. The experience of Iverson Genetics is a good example of how CED has been and can be used. Iverson’s panel test for genetic variants in the CYP2C9 and VKORC1 genes is used to determine the best dose of warfarin, an anticoagulant frequently used in the prevention of thrombosis and thromboembolism. Warfarin has a narrow therapeutic window, which means physicians often have to adjust its dose to avoid serious adverse events, such as excessive bleeding and blood clots in patients. However, despite strong evidence of an association between genetic variants and stable warfarin dose, Iverson’s initial clinical studies did not prove to CMS′ satisfaction that testing for the genetic variants CYP2C9 and VKORC1 actually improved health outcomes.12

Subsequently, Iverson and others have conducted additional randomized controlled clinical utility studies under a CED arrangement with CMS in an effort to demonstrate clinical utility. Results from these randomized controlled trials have been mixed,13,14 leading to further reassessment of the clinical utility of warfarin pharmacogenetics.15 The most recent draft policy from Palmetto GBA expresses this uncertainty explicitly for use in warfarin dosing, specifying that CYP2C9 genotype testing is still pending consideration under CED, but that VKORC1 genotype testing has been deemed to have insufficient clinical utility data and thus will not be covered.16

Lesson 5. Understand that clinical utility studies may need to involve payers and providers from the start. Reimbursement is made by payers, of course, who ultimately decide on coverage and reimbursement. National spending for molecular diagnostics and genetic testing totaled about $5 billion in 2010, which is about 8% of national spending on clinical laboratory services.17 Much of the reimbursement for diagnostic services is provided to nonelderly females, because of the wide variety of tests available for breast and ovarian cancers. Clearly, although the field of molecular diagnostics has grown rapidly, its use in clinical practice remains limited.18,19 Whether the slow rates of adoption beyond breast and ovarian cancer testing come from lack of awareness, lack of infrastructure (eg, some tumor markers require specific technologies), lack of knowledge on the part of physicians, limited availability, or limited effectiveness/utility in real-world settings, is difficult to determine.18 Payers, however, are sensitive to uptake and utilization; thus, they want to ensure that the diagnostic tests they do cover are indeed critical to clinical decision making. With the adoption of Z-Codes, for example, payers can be involved earlier in the process and perhaps enhance the collection of data by sharing the claims information relevant to the test and patient outcome.

Companies can consider engaging commercial payers by involving them in the design plan(s) for a clinical utility study. When a company tells a payer about studies that will assess the utility of a new diagnostic test, decision makers have the opportunity to comment and guide clinical utility validation. We at QURE Healthcare often do this by convening a panel of commercial payers and providers as we move from study protocol to ethics review. Panels contextualize the anticipated findings and offer opportunities to hear from other stakeholders. Most importantly, by starting early; by designing an experimental study with explicit, identified outcomes; and by committing ex ante to findings that may or may not be favorable, a company can expect results that are both anticipated and credible. By inviting an impartial outside party to moderate the panel, the likelihood of payer adoption is enhanced. MolDx, on the other hand, does not like the idea of early engagement. Its preference is to engage companies only after a full application/dossier has been submitted for their full review. Companies often err by submitting an incomplete dossier to MolDx, which leads to delays and frustration on both sides.


Coverage and reimbursement for diagnostic tests is shifting from relatively low entry barriers to much higher, evidence-based barriers that will require test developers to generate evidence of net clinical benefits before widespread clinical use. As clinicians increasingly rely on these tests, patients and the test manufacturers are increasingly concerned about payment coverage. This has created a market that beguiles and worries payers and test makers alike.

Copyright AJMC 2006-2020 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up