Jonathan Hatoun, MD, MPH, MS; Emily K. Trudell, MPH; and Louis Vernacchio, MD, MS
Asthma is a common and costly pediatric condition that has proven difficult to control despite the availability of efficacious treatments.1,2
In-office prescription management and education are insufficient to adequately achieve control in many patients with asthma.3
Home visitation, case management, and some population health management strategies are effective but costly and therefore should be targeted to the most at-risk patients. However, determining asthma risk is difficult. Many organizations and quality improvement programs use the National Committee on Quality Assurance’s Healthcare Effectiveness Data and Information Set (HEDIS) criteria to identify patients with persistent asthma.4
The HEDIS definition of persistent asthma uses insurance claims to identify patients who have emergency department (ED) or inpatient stays for asthma, are frequently seen in the outpatient setting for asthma, or fill multiple asthma medications. However, this definition of persistent asthma identifies just the sickest patients with asthma. Many more who do not meet this definition may still be at risk of exacerbation and could benefit from monitoring and proactive management.5
To better focus health efforts on the population of at-risk children with asthma, a more inclusive claims-based definition of asthma is needed: one that identifies the majority of patients who will have an asthma exacerbation in the near future but does not overidentify patients at low risk, for whom monitoring and interventions may be wasteful. We pursued this definition by creating multiple claims-based definitions of asthma and determining the sensitivity and specificity of each definition to identify patients who went on to have exacerbations. We hypothesized that the varying test characteristics of numerous definitions would allow for the construction of a receiver operating characteristic (ROC) curve. Such a chart could help population health managers choose a definition of at-risk asthma that identifies patients with a reasonable probability of having an exacerbation without expending resources on low-risk patients.
The Pediatric Physicians’ Organization at Children’s is an independent practice association of more than 90 pediatric primary care practices affiliated with Boston Children’s Hospital in Massachusetts. We analyzed paid insurance claims data from 4 commercial insurance companies that share data with our organization for quality improvement purposes. Using this database, we identified all patients aged 2 to 18 years with continuous enrollment during the measurement year and preceding 2 years.
We developed a number of claims-based definitions of asthma and compared their performance with that of the HEDIS definition of persistent asthma. Elements involved in the construction of the novel definitions included: clinician-assigned diagnosis of asthma (International Classification of Diseases, Ninth Revision
] codes 493.0x, 493.1x, 493.8x, 493.9x), wheeze (ICD-9
code 786.07), or cough (ICD-9
code 786.2); medication fill patterns (HEDIS medication set asthma medication ratio)6
; and prior exacerbations of asthma (defined as an admission with a primary diagnosis of asthma, an ED visit with a primary diagnosis of asthma, or an outpatient visit for asthma with an oral steroid prescribed).7
Definitions evaluated outpatient, ED, and inpatient claims for clinician-assigned diagnoses, and many were revised iteratively throughout the analysis to capture a range of sensitivities and specificities. Each definition under consideration was applied to the claims database using a 2-year lookback (2012-2013) and a single-year lookback (2013), and each evaluated just the first 10 diagnoses of any claim. Some definitions evaluated just the primary diagnosis, whereas others searched for diagnoses in any position. Patients meeting a particular definition’s criteria were then evaluated for an asthma exacerbation in 2014. The sensitivity and specificity of each definition for exacerbation detection was then plotted to create an ROC curve, with the area under the curve (AUC) and its 95% CI calculated for each point.
All analyses were performed in SAS version 9.4 (SAS; Cary, North Carolina). The project was approved by the Boston Children’s Hospital Committee on Clinical Investigation. A waiver of informed consent was obtained because the research could not practicably be conducted without access to the protected health information and the risk to individual patients was no more than minimal.
A cohort of 28,363 children with continuous enrollment from 2012 to 2014 was identified, with a median age of 11 years as of the anchor date, December 31, 2013; 49.2% of the cohort were female. Asthma exacerbations were found in 390 children (1.4%) during the measurement year (2014). The many potential definitions of patients at risk of an asthma exacerbation are shown in the Table
, along with the discriminating characteristics (sensitivity, specificity, and AUC) of each definition.
The formal HEDIS definition of persistent asthma, which requires meeting the HEDIS definition in each of the 2 years prior to the measurement year (2012 and 2013 in our sample), identified a cohort of 434 patients (1.5% of the total population), 17.7% of whom went on to have an exacerbation in 2014 (sensitivity, 0.20; specificity, 0.99). Patients meeting this definition accounted for 77 of the 390 patients who experienced an exacerbation in 2014. A more relaxed version of these criteria, applying the inclusion criteria just to claims in the prior year (2013 in our sample), identified 751 patients, 13.8% of whom had an exacerbation in 2014 (sensitivity, 0.27; specificity, 0.98). All other definitions were examined similarly, and test characteristics were plotted on an ROC curve (Figure
The AUC of each definition was significantly different from 0.5, and the definition identifying patients who had been given a clinician diagnosis of asthma or wheeze on 1 or more claims in the prior 2 years (definition No. 18) had the highest AUC, indicating an efficient balance of sensitivity and specificity. Definition No. 18 identified a population of 4822 individuals (17.0% of the cohort), 6.3% of whom had an exacerbation in 2014 (sensitivity, 0.78; specificity, 0.84). Patients meeting this definition accounted for 305 of the 390 patients experiencing an exacerbation in 2014. Definition No. 17 (1 or more clinician diagnoses of asthma in the prior 2 years) performed nearly as well, with a sensitivity of 0.75 and a specificity of 0.86.
Creating disease-specific patient registries for population health initiatives increasingly requires organizations to “think outside the problem list” to find patients at risk. This work demonstrates such an approach for pediatric asthma using a claims-based definition of asthma that can efficiently identify patients at risk of an asthma exacerbation with reasonably high sensitivity and specificity. Our definition No. 18, which identifies patients with 1 or more paid claims for a clinician visit with asthma or wheeze as a diagnosis in the past 2 years, had the largest AUC—substantially larger than that of the formal HEDIS definition for persistent asthma.
The HEDIS definition can be useful in defining the denominator for certain quality measures, such as the asthma medication ratio, or as a research definition, but it identifies just a small and very high-risk population of children with asthma and excludes many children who will go on to suffer an exacerbation in the coming 12 months. Indeed, in our sample, the formal HEDIS definition excluded 80% of children who suffered an exacerbation in the measurement year. The HEDIS-identified cohort of patients in this study made up just 1.5% of the entire population, far below the established population prevalence estimates for pediatric asthma, such as the CDC’s estimate of pediatric asthma prevalence nationally (8.6%)8
or in Massachusetts (9.8%),9
and even further below the school-reported prevalence in Massachusetts (10.9%).10
In order to approach such population estimates, a broader definition is needed. Our definition No. 18 had the largest AUC but identified a cohort representing 17.0% of the total population, well above the population estimates. An individual organization balancing the desire to identify all patients at risk of an asthma exacerbation with the need for judicious allocation of resources may choose to use an alternative definition that is more or less sensitive, depending on its resource availability.
Increasingly, organizations are taking a proactive approach to managing patients with asthma by reaching out to their families to optimize preventive strategies, educate about the importance of adherence to an action plan, promote influenza vaccination, and reduce environmental triggers, among other evidence-based strategies.2,11,12
The cost of proactively managing at-risk children with asthma is high. Nonetheless, if applied to the right population of children with asthma, costly interventions can provide a positive return on investment.13
Building an accurate registry of patients with asthma for population health management purposes is difficult due to the inherent unpredictability of the disease, however. A claims-based identification rule may be optimal, because relying on provider memory is inaccurate,14
and assuming that any documented respiratory complaint represents asthma overestimates the desired population.15
Although any loss of specificity from the HEDIS definition means that a larger proportion of the patients identified will not actually go on to have an exacerbation, there are clear benefits to a more inclusive identification definition that promotes outreach to a larger but still substantially at-risk population. There may be long-term benefits that are difficult to quantify for families who receive asthma education and mitigate environmental triggers in the home, such as by reducing exposure to environmental tobacco smoke. Given that genetic factors contribute to the development of asthma,2
such interventions can also benefit siblings. Additionally, missing fewer school days and workdays benefits families in innumerable ways.
This study does not help clinicians determine which children with wheeze will go on to become patients with persistent asthma, and it was not designed to assess the economic impact of an expanded registry of patients with asthma. Using a claims database also has the limitation of relying on assigned visit diagnoses to summarize nuanced clinical situations and lacks valuable demographic information. Although our study used ICD-9
codes because of the time period of our data, the current International Classification of Diseases, Tenth Revision
) convention offers a more refined spectrum of asthma codes that incorporate severity, which could more accurately determine a patient’s risk. Although this analysis used a 2-year lookback period, it is effectively a cross-sectional study. A longitudinal approach may further refine the performance of various definitions, but the time period in this analysis adequately accounts for seasonal variation, and analyses of more recent data were limited by the conversion to ICD-10
coding in fall 2015.
In order to monitor and further characterize the patients in our network with asthma at risk of exacerbation, we have begun to use a definition of asthma that identifies patients with asthma or wheezing as any diagnosis on a paid insurance claim in the preceding 2 years (definition No. 18). This definition may serve as a more appropriate criterion for inclusion into the denominator of certain quality measures for children with asthma compared with the HEDIS definition of persistent asthma. Work is ongoing to determine which characteristics of patients with asthma portend a greater likelihood of an asthma exacerbation and what the relative contributions of these factors are to a patient’s risk of exacerbation.