Hadi Kharrazi, MD, PhD, MHI

 This exploratory study used outpatient laboratory test results from electronic health records (EHRs) for patient risk assessment and evaluated whether risk markers based on laboratory results improve the performance of diagnosis- and pharmacy-based predictive models for healthcare outcomes.

 Observational study of a patient cohort over 2 years.

 We used administrative claims and EHR data over a 2-year period for a population of continuously insured patients in an integrated health system who had at least 1 ambulatory visit during the first year. We performed regression tree analyses to develop risk markers from frequently ordered outpatient laboratory tests. We added these risk markers to demographic and Charlson Comorbidity Index models and 3 models from the Johns Hopkins Adjusted Clinical Groups system to predict individual cost, inpatient admission, and high-cost patients. We evaluated the predictive and discriminatory performance of 5 lab-enhanced models.

 Our study population included 120,844 patients. Adding laboratory markers to base models improved R

 predictions of costs by 0.1% to 3.7%, identification of high-cost patients by 3.4% to 121%, and identification of patients with inpatient admissions by 1.0% to 188% for the demographic model. The addition of laboratory risk markers to comprehensive risk models, compared with simpler models, resulted in smaller improvements in predictive power.

 The addition of laboratory risk markers can significantly improve the identification of high-risk patients using models that include age, gender, and a limited number of morbidities; however, models that use comprehensive risk measures may be only marginally improved.

Laboratory tests that are frequently ordered by physicians in outpatient practices contain valuable data for individual risk assessment.

Ranges of blood chemistries and hematology results define a set of model markers that have clinical face validity and potential utility for care management.

Adding the laboratory-based markers to risk levels derived from claims, prescriptions, and enrollment data improves the prediction of individual cost, the prediction of inpatient admission, and the prospective identification of high-cost patients.

For practices, a simple model that includes demographics and laboratory information may provide a basic tool to evaluate patient panels.

Most predictive models in healthcare have relied upon diagnosis information from health insurance claims or other administrative data. Such claims-based predictive models have been used extensively by health plans and government agencies for provider profiling and payment, underwriting, and prioritizing patients for care management.

 Although claims remain an important source of risk data, the widespread implementation of electronic health records (EHRs) and other clinical information technology systems offers a new source of data on disease severity and health status, as most EHRs contain information not captured in claims, such as laboratory values, vital signs, and clinical assessments.

In the inpatient setting, laboratory tests have been used to assess the risk of mortality across a range of conditions, including acute myocardial infarction, congestive heart failure, diabetes, ischemic and hemorrhagic stroke, pneumonia, and septicemia.

 These predictive assessments of mortality risk have incorporated blood chemistries, hematology, and blood gases. Predictive models for mortality performed better after adding laboratory risk markers, but similar models predicting 30-day readmission did not improve as much.

Another case for laboratory data has been made for case-mix adjustment of inpatient admissions using diagnosis-related groups (DRGs).

 Clinical laboratory results combined with inpatient administrative data incrementally improved the ability of DRGs to explain the length of inpatient stays; however, Medicare Severity DRGs and other DRG versions do not incorporate laboratory data for inpatient classification.

Laboratory tests can be powerful predictors among certain patient populations. For example, patients with diabetes who maintained reduced glycated hemoglobin (A1C) levels (ie, had better glycemic control) had lower annual costs than patients with higher levels.

The goal of this study was to develop and evaluate an approach for transforming common outpatient laboratory tests into risk measures that could be useful when added to population-level predictive models. Our objective was to determine result ranges for several candidate blood tests that were associated with increased costs in the year after the tests were performed. We hypothesized that certain ranges of component results from blood tests in the base year would be associated with higher healthcare costs and increased inpatient utilization during the subsequent year. We also hypothesized that laboratory risk markers based on component ranges would improve predictive risk models for these outcomes, including models with demographic and Charlson Comorbidity Index (CCI) risk markers and 3 models from the Johns Hopkins Adjusted Clinical Groups (ACG) system.

We obtained data from HealthPartners, Inc (Bloomington, Minnesota), a health insurer and large integrated delivery system. Its database contains structured EHR data, including encounter diagnoses and laboratory test results; administrative data that included benefit eligibility files; and claims data with 

International Classification of Diseases, Ninth Revision, Clinical Modification

) diagnoses, Current Procedural Terminology (CPT) procedure codes from inpatient and outpatient settings, and filled prescriptions with National Drug Codes from outpatient pharmacies. HealthPartners provided these data for patients who were receiving care at facilities owned by the healthcare system.

Our study population included 120,844 patients who were continuously enrolled in 2012 and 2013 and had at least 1 visit to 1 of 5 HealthPartners outpatient clinics in the Minneapolis-St. Paul metropolitan area in 2012.

To harmonize the coding of test orders across HealthPartners’ entities, we mapped the internal HealthPartners codes to Logical Observation Names Identifiers and Codes (LOINC). LOINC is a common language for identifying health measurements, observations, and documents, and it is commonly used for laboratory orders and findings.

The assignment of LOINC was a 2-step process. We first used the Regenstrief LOINC Mapping Assistant to suggest potential LOINC, which were turned over to a pathologist for final review in the second step.

 All laboratory tests that we selected for this study were mapped to LOINC.

Selection of Laboratory Tests and Creation of Risk Markers

We identified 23 blood chemistries and hematology counts from 4 test panels (ie, the basic metabolic panel, lipid panel, liver function tests, complete blood counts) and extracted A1C, alanine aminotransferase (ALT), albumin, alkaline phosphatase (ALK), aspartate aminotransferase (AST), bicarbonate, blood urea nitrogen (BUN), calcium, chloride, creatinine, glucose, hematocrit, hemoglobin, high-density lipoprotein cholesterol (HDL-C), low-density lipoprotein cholesterol (LDL-C), platelet, potassium, sodium, total bilirubin, total cholesterol, total protein, triglycerides, and white blood cell results from the EHR data. Our literature review suggested that these tests are commonly ordered in office-based clinical practices, and our study population confirmed this theory, as 49% had at least 1 result for any of the 23 tests in 2012. We extracted CPT codes for test orders from claims submitted to HealthPartners and confirmed that essentially all results for the tests of interest were present in the EHR data. We compared results with reference ranges for healthy persons,

 manually excluded implausible results that were extremely far outside the reference ranges, and selected the most recent results to create patient-level risk markers. Patient-level annual costs were calculated from claims incurred in 2013.

We used a 3-step process to develop laboratory-based risk markers. First, we conducted a regression tree analysis for each of the 23 tests to determine result ranges that were prospectively associated with increased annual costs using the caret package in R software.

A strategy of individually analyzing laboratory covariates has been used to discriminate the risk of inpatient mortality.

 We minimized the impact of high-cost claimants by truncating individual costs at $250,000, which was equivalent to the 99.9th percentile in the study population. Second, several regression tree analyses generated multiple result ranges, and we condensed them into ordinal “low” and “high” levels to create binary markers. High levels of risk consisted of low test results, high values, or both. To prevent model overfitting, we required the high-risk groups to contain at least 1% of patients who had results for a test. Third, we created binary markers for low and high risk levels. Tested patients had either a low- or a high-risk marker assigned to their test results. High risk indicated a potential for high cost in the future period; low risk indicated that a patient’s condition was nonsevere or under control or that the test was performed for diagnostic reasons. Patients who did not have tests had all markers set to 0 so that we were able to evaluate the joint impact of laboratory-based risk markers in the entire patient population.

Our prospective outcome measures were individual total annual claims costs, presence of inpatient hospitalization, and high-risk patient status, defined as being among the top 5% of claimants in 2013. We developed 5 predictive models for these 3 outcomes.

The demographic model included age and gender only; the Charlson model included age, gender, and 17 morbidity categories from the CCI, which is widely used to predict high-cost patients.

 The ACG system contributed 3 predictive models; all included morbidity markers based on diagnoses, and 1 included morbidity markers that were derived from prescription data.

 We first combined the demographic variables with 32 Aggregated Diagnosis Groups (ADGs; ie, types of morbidity from the ACG system) into an ADG model. Other researchers have validated a similar ADG model for predicting mortality.

 Second, we implemented the ACG-Dx predictive model with age, gender, 24 selected ACGs, and 116 Expanded Diagnosis Clusters (EDCs) for the patient population.

 ACGs are mutually exclusive groups that are based on patterns of ADG morbidity types and have similar resource use; EDCs are clinical groupings of diseases based on 

International Classification of Diseases, Tenth Revision, Clinical Modification 

codes. A patient may have multiple ADGs and EDCs. Third, we implemented an ACG-DxRx model that contained all markers from the Dx version and included 65 Rx-MGs (ie, morbidity types based on prescription data). ACG-PM predictive models have been validated for predicting high-risk patients and inpatient hospitalization.

We used the health system’s data to generate individual risk scores. The models were run with and without the laboratory markers included. Our objective was to evaluate the contribution of new laboratory markers to the performance of predictive models with different levels of complexity as indicated by the number of morbidity markers.

We calibrated models for costs using ordinary least squares (OLS) and generalized linear regression and chose to report the coefficient of determination (R

) for OLS models. Models for inpatient hospitalization and high-risk outcomes were calibrated using logistic regression, and we computed sensitivity, specificity, area under the receiver operating characteristic curve (AUC), and integrated discrimination improvement (IDI) statistics to quantify the improvement in discriminatory performance due to laboratory markers. We included IDI because the original models that included ACG system variables showed high AUC values; therefore, meaningful improvements in discriminatory power might not be captured by measuring only AUC. The properties of the IDI statistic are well understood, and this statistic is increasingly used to evaluate markers that are introduced into predictive models.

The study population consisted of 120,844 patients who had at least 1 ambulatory care visit at a HealthPartners-owned clinic in 2012. The mean (SD) age was 37.6 (19.2) years, 20.6% were younger than 18 years, 3.3% were 65 years or older, and 57.1% were women (

). Almost all patients (99.9%) had at least 1 type of morbidity recorded, with an average (SD) of 5.9 (3.4) ADGs, and 20.0% had at least 1 comorbid condition that was included in the CCI. Mean (SD) patient total annual claims costs were $5732 ($20,208), and 5.1% were admitted to a hospital in 2013.

We extracted test orders from all outpatient medical service claims and measured laboratory data completeness. A1C results were 92% complete; calcium results, 96% complete; and all other test results, more than 98% complete.

We determined whether any result ranges for the candidate tests were associated with increased costs in the year after the tests were performed. Our analysis indicated separations between “low-cost” and “high-cost” risk groups for 12 of the 23 tests. These 12 tests included sodium, chloride, bicarbonate, glucose, and calcium from the basic metabolic panel; total protein and albumin from the liver function tests; hemoglobin, hematocrit, and platelets from the complete blood count; and total cholesterol and LDL-C from the lipid panel. The other 11 laboratory tests—A1C, ALT, ALK, AST, BUN, creatinine, HDL-C, potassium, total bilirubin, triglycerides, and white blood cell results&mdash;did not show an association with low- or high-cost risk and were excluded from the predictive modeling.

High-risk group sizes ranged from 1% of patients with hematocrit results to 21% of patients with albumin results. The average costs in these 2 groups were $32,695 and $24,234, respectively. Other high-risk groups showed similar increased average annual costs. The cost separation between high- and low-risk groups ranged from $4943 for LDL-C to $23,492 for hematocrit (

Predictive Model Performance Improvement With Added Laboratory Markers

Laboratory markers increased the prospective R

 of the demographic model for costs more than 2-fold from 2.2% to 5.9%; the IDI measures for inpatient and top-cost claimant identification were 121% and 188%, respectively (

 for cost increased from 10.3% to 11.4%, and the identification of inpatients and top-cost claimants, as measured by the IDIs, improved by 40% and 14%, respectively.

Overall, ACG-PM models exhibited higher prospective R

 values and showed less improvement with added laboratory markers compared with the demographic and Charlson models. The ACG-Dx and ACG-DxRx models predicted 22.2% and 24.7% of cost variation, respectively. Laboratory markers added small improvements to predicting costs across all 3 ACG system models; the R

The lab-enhanced ADG model had an AUC of 0.820 and an IDI of 4.8% for the identification of top-cost claimants. Similarly, lab-enhanced ACG-Dx and ACG-DxRx models had AUCs (IDIs) of 0.835 (1.5%) and 0.847 (1.0%) for high-risk identification, respectively. For hospitalization predictions, the AUCs across lab-enhanced ACG system models ranged from 0.789 (ADG) to 0.799 (ACG-DxRx); IDIs ranged from 8.4% for the ADG model to 3.4% for the ACG-DxRx model.

We developed high-cost risk markers using commonly ordered outpatient laboratory test results and evaluated how these markers improved individual predictions of healthcare costs, hospitalization, and high-risk status. This analysis extends previous research that used laboratory test results to predict clinical outcomes, such as mortality and hospital admission.

 We explored the potential value of these new commonly available clinical data sources for population-based predictive models as applied to care management.

We transformed test results that were extracted from an outpatient EHR into risk markers that could be replicated in a health system; organizations should be able to derive risk thresholds that are similar to those used in this research. Our thresholds were outside the reference ranges for apparently healthy persons, although less extreme than those used in previous inpatient mortality models.

Among the basic metabolic panel, 5 of 8 candidate tests were associated with high-cost risk. Abnormalities in electrolytes (sodium, potassium, chloride, bicarbonate, BUN) can occur in patients with congestive heart failure and kidney disease, and both conditions are linked with higher costs.

 Although creatinine is used clinically to determine chronic kidney disease stages, we found an association between creatinine and higher costs for less than 1% of tested patients, which was lower than our threshold. Hyperglycemia, as demonstrated by elevated glucose, was associated with increased cost risk, but A1C showed no association. Some tests that are commonly used to stage disease or guide treatment (eg, creatinine and A1C) were not predictive of prospective cost in our analysis. A previous analysis found that these tests were associated with 5-year costs among patients with diabetes

; our results may be accounted for by the 2-year duration of this study.

Among tests from the liver function test panel, 2 of 6 tests were associated with high-cost risk. Low levels of total protein and albumin are linked with liver disease and malnutrition.

 Three components from the complete blood count were associated with high risk in the subsequent year. Low hemoglobin and hematocrit values indicate anemia, and low values of platelets can be diagnostic of thrombocytopenia. Among the 4 tests included in the lipid panel, low total cholesterol and LDL-C levels were associated with high risk. Several conditions, including cancer, may contribute to low cholesterol levels.

 We did not identify high total cholesterol and LDL-C levels, 2 clinical risk factors for coronary artery disease,

 as ranges that contribute to high-cost risk. Patients who had lipid tests may have already been treated with statins, so they had decreased risk.

Our second objective was to examine whether laboratory-based risk markers improved predictive models for total healthcare costs, top-cost claimants, and inpatient hospitalization. We explored the added value to models that varied in complexity in terms of the number and scope of morbidity markers, ranging from a demographic-only model to a Charlson model with 17 morbidity categories to 3 complex models from the ACG system. This approach enabled us to examine the impact of laboratory-based risk markers across a range of models and inform organizations that may have access to sources of data with different limitations (eg, stand-alone laboratory centers may not have the complete clinical picture of referred patients). In all cases, laboratory-based markers improved the prediction of costs and the identification of high-cost claimants and patients with inpatient admission compared with original models. Model performance improved greatly when laboratory risk markers were added to the demographic and Charlson models and modestly when laboratory-based markers were added to ACG-Dx and ACG-DxRx models with large sets of morbidity markers derived from diagnoses and medication data found in claims or EHRs. Importantly, for health systems or healthcare practices with limited resources for predictive modeling, our results demonstrate that a simple model with laboratory markers may provide a tool to evaluate individuals and patient panels.

Our research has several limitations, including that (1) the development of laboratory-based risk markers could be refined by integrating patient characteristics (ie, age and sex) and multiple tests in regression tree analyses; (2) our study population contained mainly working-age insured patients; therefore, our exploratory research should be replicated in other populations (eg, elderly patients); (3) temporal changes in test results could contain additional risk information for patients who have multiple laboratory tests in a year; (4) additional risk information could be gathered from tests that are less frequently ordered in outpatient settings, including tests that would inform about diagnoses that are potentially underreported in EHRs and claims; and (5) the model fit could conceivably be improved somewhat with alternative statistical techniques.

Articles

An evaluation of the added value of risk markers derived from ambulatory laboratory tests in the prediction of healthcare costs and identification of high-risk patients.

Hadi Kharrazi, MD, PhD, MHI

Articles

Assessing Markers From Ambulatory Laboratory Tests for Predicting High-Risk Patients