Understanding the Role of Big Data in the CV Risk Calculator Controversy
Published Online: January 20, 2014
Medical scholars on both sides of the very public dispute over new disease prevention guidelines from the American College of Cardiology (ACC) and the American Heart Association (AHA) expected controversy.
Major revisions to treatment protocols typically generate at least some points of genuine dispute, leaving individual professionals to weigh the evidence for themselves until new research and analysis answers questions definitively.
In this particular case, the points of actual contention are few, though serious. The vast majority of the guidelines, along with the vast majority of the procedures for creating them, have won widespread support among those who study atherosclerotic cardiovascular disease. Indeed, even the most famous critics of the new recommendations have praised them for their emphasis
on preventing stroke as well as heart disease, their efforts to differentiate risk by race and gender, and their efforts to translate research into suggestions1:
• Studies correlating low-density lipoprotein (LDL) cholesterol and disease find no specific thresholds separating safety from danger, so the guidelines omit any mention of specific LDL targets along the lines of <70 mg/dL or <100 mg/dL.
• Drug trials have found that many nonstatin medications can lower LDL levels, but no such drug has demonstrated the ability to lower the risk of stroke or heart disease like statins, which is why the new guidelines rarely recommend considering them, not even for statin-intolerant patients with dangerous LDL levels.
• Trials of statins, on the other hand, have distinctly demonstrated that they prevent strokes and myocardial infarction associated with hypertension, diabetes, and other risk factors beyond LDL levels; thus, the new guidelines suggest statins for many patients with healthy cholesterol profiles.2,3
• Analyses that found no evidence of benefit to statins given to older patients have also been incorporated into the new guidelines, which do not recommend them for patients over age 75 years. Such recommendations have raised objections among conservative clinicians who think the changes too radical and from people who wanted more emphasis on lifestyle and less on medication.
Among thought leaders, however, all the suggestions above have received a warm reception. The only real controversial points are formulas that combine nearly a dozen factors to estimate each patient’s 10-year risk of atherosclerotic cardiovascular disease and a recommendation that any patient whose calculated 10-year risk exceeds 7.5% consider statin therapy.
A viewpoint article published by The Lancet on November 19, 2013, just a few days after the ACC and AHA unveiled the new guidelines, began the debate on both those points, starting with the decision to base discussions about statin use on a combined calculation. “No trial of statin therapy has ever used a global risk prediction score as an enrollment criterion, so basing prescriptions on such a metric might be difficult to defend in an evidence-based climate,” wrote Paul Ridker, MD, and Nancy Cook, PhD, both professors at Harvard University’s Brigham & Women’s Hospital.
“In addition, trial data contradict statements that high absolute risk always predicts statin efficacy… The CORONA4 and AURORA5 trials enrolled individuals with very high vascular risk in the settings of heart failure or renal failure and found no evidence of event reduction with statin therapy despite large reductions in LDL cholesterol.”
Other critics have expressed particular frustration that high risk scores generated almost entirely by a patient’s age can easily cross the 7.5% threshold and thus trigger serious consideration of statin therapy.6 After all, they argue, no trial has ever found that statins prevent atherosclerotic cardiovascular disease associated with aging.
Donald Lloyd-Jones, MD, ScM, who co-chaired the risk assessment work group, thinks many of the critics simply misunderstand, or haven’t read, the guidelines. “The guidelines don’t order anyone to use statins. They merely indicate when it makes sense for doctors and patients to consider statins because there is evidence of benefit for patients in those groups,” said Lloyd-Jones, Eileen M. Foell Professor and chairman of the Department of Preventive Medicine at Northwestern University’s Fienberg School of Medicine.
“If your total 10-year risk is lower than 7.5%, the costs associated with statins may outweigh the benefits for many patients, so there may be no reason to use them. Once your risk exceeds 7.5%, though, it may make sense to use them, depending on why your risk exceeds 7.5%.
“If hypertension or diabetes or high LDL contributes to your risk, then statins probably make sense. If age is your only risk factor, then statins may not make sense and these guidelines aren’t pushing statins on you. I just don’t understand how anyone who had bothered to read the documents in full rather than news reports could think otherwise.”
The other major criticism of the guidelines concerns the accuracy of the risk calculator and, again, traces its origin to the Lancet piece. In it, Ridker and Cook report on their efforts to validate the risk calculator on 3 large medical data sets: the Women’s Health Study (WHS), the Physicians’ Health Study (PHS), and the Women’s Health Initiative Observation Study (WHI-OS).
The risk calculator received the initial information from each cohort participant and made predictions that Ridker and Cook then compared against actual patient results. The calculator overestimated risk by 75% to 150% in all 3 cohorts, Ridker and Cook wrote, predicting roughly double the actual amount of observed disease across the 3 groups. “It is possible that as many as 40–50% of the 33 million middle-aged Americans targeted by the new ACC/AHA guidelines for statin therapy do not actually have risk thresholds that exceed the 7.5% threshold suggested for treatment. Miscalibration to this extent should be reconciled and addressed in additional external validation cohorts before these new prediction models are widely implemented.”
Those charges have made much news, but the risk calculator’s creators and backers stand firmly behind their work.
To build the calculator, the ACC/AHA committee started with a list of factors proved to increase the risk of atherosclerotic cardiovascular disease and found several major medical cohorts that collected all the relevant data on large numbers of typical American patients for at least a decade: the Framingham Heart Study, Atherosclerosis Risk in Communities, Coronary Artery Risk Development in Young Adults, and the Cardiovascular Health Study.
PDF is available on the last page.