Cost vs Accuracy: Choosing the Best ML Model for Vascular Calcification in CKD

While multiple algorithms offered similar predictive performance, the study revealed stark differences in the costs associated with their use.

image credit: SOMKID - stock.adobe.com

Logistic regression has been deemed the superior machine learning model for classifying medial vascular calcification in a recent cost-effective analysis.¹ Demonstrating a mean incremental cost-effectiveness ratio (ICER) of $278, this approach utilizes only 5 readily available and low-cost features, in contrast to the support vector machine model, which, despite employing 21 features, recorded a significantly higher ICER.

Medial vascular calcification, a pathology associated with elevated cardiovascular risk in chronic kidney disease (CKD), requires cost-effective diagnostic approaches. Machine learning algorithms are being increasingly used in disease detection through the integration of multiple biomarkers. Recent research highlights their application for atrial fibrillation,² skin cancer,³ and HIV.⁴ However, the researchers of the current study stated that economic considerations are often overlooked.¹

To assess cost-effectiveness, the researchers calculated the ICER across different assumptions regarding biomarker availability, disease prevalence, and quality-adjusted life years (QALYs) gained. The pilot study included 152 patients with CKD undergoing living donor kidney transplantation, and Spearman correlation revealed collinearity among some machine learning model features. In univariable logistic regression, several variables were associated with medial vascular calcification, but after adjusting for multiple comparisons, only age, male sex, copeptin, insulin-like growth factor-1 (IGF-1), osteoprotegerin, and body mass index (BMI) remained significant.

Evaluating Algorithms for CKD Biomarker Detection

Despite their differences, the classifiers showed comparable diagnostic accuracy. Five classification models were applied: logistic regression, support vector machine, random forest, elastic net, and regularized least squares. Age and copeptin were the only features selected by all models. Logistic regression showed the highest area under the curve (AUC, 0.85; 95% CI, 0.78–0.90), followed closely by regularized least squares (0.84; 95% CI, 0.77–0.90).

Cost-effectiveness analysis using the ICER highlighted logistic regression as the most efficient model. With a mean ICER of $278, logistic regression required only 5 features, most of which are routinely available and low cost. Support vector machine, which used 21 features, had the highest ICER ($769). These results held across varying assumptions about biomarker availability, disease prevalence, and QALYs gained, supporting logistic regression as the most practical and cost-effective choice for medial vascular calcification classification.

While multiple algorithms offered similar predictive performance, the study revealed stark differences in the costs associated with their use, leading researchers to call for greater attention to cost-effectiveness in AI-based diagnostics.

Clinical Implications and Potential for Pre-Screening

The study emphasizes that machine learning models, particularly simpler ones like logistic regression, could help reduce reliance on costly and invasive CT scans to detect medial vascular calcification. While none of the models is yet suitable to replace imaging outright, they could serve as pre-screening tools to triage patients who are less likely to have the condition, reducing health care expenditures and radiation exposure.

However, the authors caution that performance metrics alone should not guide model selection. Instead, health care providers should consider the trade-offs between diagnostic accuracy, cost, and operational feasibility.

The study's retrospective design, relatively small sample size, and lack of external validation limit the generalizability of the findings. Some potentially relevant biomarkers, such as N-terminal pro B-type natriuretic peptide, were not analyzed, and cost estimates may vary across health care systems. Still, the researchers believe their framework offers a promising foundation for future investigations.

Biomarker Insights and Future Directions

In addition to validating known predictors like age and BMI, the study identified copeptin as a consistently important biomarker across all models, reinforcing prior research linking it to vascular calcification. Osteoprotegerin and sclerostin were also frequently selected and may warrant inclusion in future biomarker panels.

“The accuracies of the models were not perfect, underscoring that there is still much to uncover regarding the biomarkers associated with [medial vascular calcification] and that machine-learning-based algorithms cannot serve as a standalone method for assessing [medial vascular calcification] presence in CKD patients,” the authors wrote. “However, they can help reduce the frequency of performing unnecessary CT scans for individuals who are found to be less likely to have the pathology, based on the initial assessment of the biomarkers. This reduction can lead to significant savings in healthcare costs, limit radiation exposure, and decrease the time required for diagnostic procedures.”

References

1. Bialonczyk U, Debowska M, Dai L, et al. Balancing accuracy and cost in machine learning models for detecting medial vascular calcification in chronic kidney disease: a pilot study. Sci Rep. 2025;15(1). doi:10.1038/s41598-025-02457-2

2. Kaltwasser J. Novel AI-based score could pinpoint AF risk in patients with CLL on BTKis. AJMC^®. May 15, 2025. Accessed May 28, 2025. https://www.ajmc.com/view/novel-ai-based-score-could-pinpoint-af-risk-in-patients-with-cll-on-btkis

3. Kaltwasser J. Noninvasive skin cancer detection method shows promise. AJMC. May 18, 2025. Accessed May 28, 2025. https://www.ajmc.com/view/novel-non-invasive-skin-cancer-detection-method-shows-promise
4. Bonavitacola J. Machine learning capable of predicting hyperlipidemia in people with HIV. AJMC. May 28, 2025. Accessed May 28, 2025. https://www.ajmc.com/view/machine-learning-capable-of-predicting-hyperlipidemia-in-people-with-hiv