Asthma, a common chronic condition, has substantial effects on individuals, the healthcare system, and society, accounting annually for 2 million emergency department (ED) visits, more than 400 000 hospitalizations, 4500 deaths, 2 million lost work days, and a cost of $6 billion.1-4 The enormous burden of asthma on patients and society in part reflects the under treatment of patients at risk of poor outcomes.5,6 Therefore, there is a need for screening tools to identify patients needing closer scrutiny of their therapeutic regimen.7
Screening populations to identify higher-risk individuals for disease management programs can improve efficiency of resource use and lead to cost savings.8-10 In managed care, there are opportunities to improve asthma care, particularly with medication use.7 Programs that identify high-risk patients and direct them to certain interventions have achieved improved outcomes.8,11,12 Most risk models for asthma have focused on using physiologic and clinical data to predict clinical outcomes, including mortality, hospitalization, ED use, and relapse.13-15 Some recent work has focused on developing tools to predict patient-reported outcomes, including health-related quality of life, adherence, satisfaction, and work disability.16-24 This emphasis is important, as the perspectives of varied stakeholders, such as employers, patients, clinicians, disease management companies, and health plans, support the need for developing models that predict a broader range of outcomes. Managed care organizations (MCOs) have a financial incentive to intervene to prevent hospitalizations and ED use.25 Employers may find lost work days to be the most relevant outcome.26 Patients' quality of life and satisfaction with care may be affected most by symptom reduction.27
The objective of this study was to develop a brief set of patient-reported questions that could be used to predict a broad array of outcomes, including severe symptoms, reduced activities, ED use, and hospitalization. We developed and assessed the performance of 5 asthma risk models, based on a common set of survey questions, to predict these outcomes in patients enrolled in managed care.
METHODS
The risk models were developed using data from the Managed Health Care Association Outcomes Management System Asthma Project, a collaboration among 12 large employers and their managed care partners. The project was designed to test the feasibility and usefulness of information on adult asthma for improvement of the quality of patient care.28,29 Sixteen MCOs participated in a prospective cohort study that included an initial patient baseline survey and 2 annual follow-up surveys. A 58-item questionnaire,30 which included the generic Medical Outcomes Study 36-Item Short-Form Health Survey and condition-specific measures, was used for the study. This instrument was used for the baseline survey and the 1-year follow-up to develop the final risk models. Some of the items in the questionnaire were the following:
Symptoms
In the past 4 weeks, how often did you have asthma attacks? By "asthma attack," we mean increased difficulty breathing that may be accompanied by cough, wheezing, chest tightness, or other symptoms. Not at all, less than once a week, once or twice a week, 3 or more times a week
In between the times when you have asthma attacks, how is your breathing? No problems, some symptoms on some days, some symptoms on most days requiring an inhaler for relief, symptoms most of the time
During the past 4 weeks, how much did your asthma cause you to cancel or rearrange your normal activities? Not at all, a little bit, some, quite a bit
Access to Care
Did any of the following happen to you in the past 12 months?
-
You had a problem with your asthma but had trouble reaching a doctor or nurse on the phone. Yes, No
-
You had a problem with your asthma but had trouble getting an appointment to see your doctor. Yes, No
-
You needed medicine for your asthma but had trouble getting it. Yes, No
Use
Over the past 12 months, how many times have you gone for care to a hospital emergency room for your asthma? ___ Number of times
Over the past 6 months, how many times have you gone for care to a doctor, nurse, or other health professional in an office or clinic for your asthma? ___ Number of times
When was the last time you were admitted to a hospital for your asthma? Month and year ___
The study design has been described in detail elsewhere.11,31 Briefly, in 1993, patients (n = 10 539) were sampled using claims data from 16 MCOs. The questionnaires were accompanied by a disclosure letter, and consent was indicated by return of the questionnaire. The study was approved by the institutional review board of the Bloomberg School of Public Health. Eligible patients met 3 criteria: 2 or more asthma encounters (visits or hospitalizations, International Classification of Diseases, Ninth Revision, Clinical Modification [ICD-9-CM] code 493.X) during the previous 2 years; age 18 years or older on September 1, 1993; and enrollment in the MCO at the time of sampling. The sample was stratified so that approximately 40% of the patients had a hospitalization or ED visit for asthma in the past 2 years. Stratification was intended to assure a sufficient number of patients with moderate to severe asthma who might benefit from quality improvement efforts. Individuals were excluded if they denied having asthma or if they had disenrolled. At baseline, 6612 questionnaires were completed (response rate, 77%); 4895 completed an identical questionnaire 1 year later (response rate, 79%). The study sample includes the 4888 patients who completed baseline and 1-year questionnaires. Nonrespondents at 1 year were more likely to be younger, nonwhite, and less satisfied at baseline with their asthma care (data not shown).
Development of Risk Models
The objective was to identify a parsimonious group of variables from the baseline survey to predict 1-year outcomes. Dependent variables were selected based on literature review 14-17,19,26,29,32 and priorities expressed by employer and MCO members of the Managed Health Care Association Outcomes Management System Asthma Project. Candidate predictors were identified from the literature and using clinical judgment and were conceptualized within 6 domains: demographic characteristics, generic health status, asthma-specific health status, asthma treatment, asthma service use, and access to care.21,33-38 We selected 4 dichotomous outcome indicators from the 1-year follow-up survey: (1) hospitalization for treatment of asthma in the prior year (hospital); (2) ED treatment for asthma in the prior year (ED); (3) reduced activities for 7 or more days due to asthma, or missed work 5 or more days in the past month (lost days); and (4) experiencing 5 or more asthma attacks per week in the past month or having symptoms most of the time between attacks (symptoms).
Preliminary analysis suggested that 3 of the dependent variables–hospital, ED, and lost days–would be predict- ed by a similar set of independent variables. Therefore, a fifth outcome, a composite variable defined as the occurrence of any 1 of these 3 outcomes, was created; this type of outcome may represent the shared perspective of the administrative and employer groups. Odds ratios for predictor variables associated with each of the 5 outcomes are presented separately for patients in the inpatient/ED stratum and for those in the outpatient stratum.
Management of Missing Data for Independent Variables
The results in this article are presented with substitutions made for missing values. We developed an algorithm for imputing missing data based on multiple conditional imputation.39 Approximately one quarter of the respondents had at least 1 missing response. The candidate variables with the largest percentage of missing responses were corticosteroid use (12%), ED visits in the past year (7%), age 35 years to 65 years or older (5%), β-agonist inhaler use more than 8 puffs per day (4%), and trouble getting medications (3%). Sensitivity of the analyses to missing values was evaluated by comparing the difference between odds ratios in the complete and imputed datasets. There were no important changes to the models with and without substitution.
Statistical Analysis
A multistep process was used to select candidate variables for the 5 final predictive models. First, the relationship of each candidate predictor variable to 1-year outcomes was assessed using a χ2 test. Because these relationships differed for patients recruited from the hospital or ED and outpatient strata, model development proceeded separately for each stratum. Variables with a χ2 greater than 2 times the df were retained in the multiple regression models.40 Multiple logistic regression analysis was performed to identify independent predictors (P < .05) for each outcome. Age, sex, and race were included in all models. For each outcome, variables that were significant independent predictors in the inpatient/ED model or the outpatient model were tested in the final models. The final product was 2 sets of multivariable predictive models, one for patients from the inpatient/ED stratum and one for the outpatient stratum. Odds ratios and 95% confidence intervals (CIs) were obtained for each predictor variable. The analyses were performed using SAS version 6.07 (SAS Institute, Cary, NC).
To ensure against overfitting, that is, spurious regularity in the data, multiple imputation was used to generate 5 complete datasets, fit the selected risk models, and estimate 5 sets of odds ratios. Because there was little variability across the 5 datasets, we present the pooled odds ratio for each predictor variable and outcome. Confidence intervals were calculated taking into account the within-dataset variability and betweendataset variability of the odds ratios.
Performance of the Predictive Models
Performance of the models was assessed by comparing them on rankings of candidate variable odds ratios, area under the receiver operating characteristic (ROC) curve, and diagnostic accuracy with sensitivity, specificity, and positive and negative predictive values. We also explored the predicted risk in 3 subgroups by stratifying the sample by sex, race, and education. For the composite model, we further assessed the range of possible cutpoints and tested their diagnostic accuracy.
RESULTS
Most patients in this study were white (83% [13% were African American]), female (69%), educated (89% were high school graduates), and employed full-time (66%) or part-time. The mean age was 45 years, and nearly half (46%) of the patients reported being current or former smokers. Approximately 39% of patients were in the inpatient/ED sample, and 61% in the outpatient sample. Frequencies of the 5 outcome variables were similar at baseline and at 1 year: (1) hospitalization for asthma (baseline and 1 year, 12% and 9%, respectively), (2) ED in the past year (43% and 35%), (3) reduced activities or missing work (38% and 36%), and (4) severe symptoms (57% and 53%). More than half of the patients reported 1 or more of the outcomes that comprise the composite measure (baseline and 1 year, 61% and 54%, respectively).
Significant Predictor Variables
Table 1 shows each of the 5 risk models separately for the inpatient/ED and outpatient strata. Also shown are the 12 independent variables (not counting demographics and comorbidities) identified in the bivariate comparisons as likely predictors of the outcomes. These 12 variables were tested in all 5 risk models in each of the recruitment strata. The odds ratios and 95% CIs represent an unweighted mean of results obtained from analyses of each of the 5 datasets, including imputed values. Between 4 and 11 predictors were significant independent predictors of the various 1-year outcomes. Emergency department visits in the year before baseline were significant in all 5 models; all the other variables were significant in at least 1 model.
As anticipated, different variables emerged as independent predictors in each of the 5 outcome models, and for patients in the inpatient or ED vs outpatient strata. Although comorbidity was a strong predictor in 4 of the 5 models, the specific conditions involved varied as follows: ulcer for the hospital model, myocardial infarction and chronic obstructive pulmonary disease for the ED model, chronic obstructive pulmonary disease alone for the symptoms model, and all 3 conditions for the composite model. Canceling activities because of asthma was important in 4 of the models (all except the symptoms model). Smoking was not statistically significant in any of the 5 predictive models, although it had shown a trend in the bivariate comparisons.
Predictive Discrimination
Areas under the ROC curve for the 5 outcome models ranged from 0.67 to 0.78, suggesting adequate discrimination. There was no significant variation in the ROC area for sex, race, and education for those younger than 65 years; however, the ROC area increased to 0.86 for those 65 years of age or older, although this age group was represented in low numbers. Receiver operating characteristic areas in the composite model were 0.78 for the inpatient/ED stratum and 0.73 for the outpatient stratum, and in the symptom model, 0.76 for inpatient/ED and outpatient strata, indicating fairly good discrimination.
Further examination of the composite model shows the ability to capture approximately three fourths of the high-risk population with good accuracy at a cutpoint of 0.7 (sensitivity , 77%; 95% CI, 74%-79%; specificity , 63%; 95% CI, 59%-67%; positive predictive value , 82%; 95% CI, 78%-85%; and negative predictive value , 56%; 95% CI, 53%-59%).
Using the variables in the composite model, Table 2, demonstrates how a scoring system could be used to characterize patient risk, with 14 items producing 22 possible points. For patients with a score of 0, for example, the proportion experiencing 1 or more of the adverse outcomes in the next year would be 23%. Similarly, 35% of patients with scores of 2 would have at least 1 adverse outcome, 49% of those with scores of 3, 70% of those with scores of 6, and 100% of those with scores of 12 or more.
DISCUSSION
We demonstrated that patient self-report using a limited number of questions can satisfactorily predict diverse asthma outcomes in a nationwide population of patients in managed care. The questions were selected from factors previously shown to be associated with asthma outcomes, including patient demographics, asthma symptom severity, comorbid illnesses, tobacco exposure, and current use of medications. The resulting set of 14 self-administered questions could be applied for screening of large populations of patients. Advantages of using this instrument include the options of predicting specific outcomes relevant to different stakeholders in asthma care, as well as prediction of an aggregate outcome. Screening is conducted using an easily administered survey format. Depending on a healthcare organization's priorities and available resources, these questions could be used to stratify patients by risk for the outcomes of greatest interest, so those at higher risk could receive additional attention.
Most risk models for asthma have used clinical and physiologic data to predict clinical outcomes such as hospitalization and ED visits.13-15 Our hospitalization and ED outcomes were predicted with good accuracy by patient-reported data. Models for severe symptoms and lost work days extend prediction to include outcomes important from the patient and societal perspectives. Our composite model allows prediction of occurrence of any 1 of several adverse outcomes, including hospitalization, ED visits, and lost work days, providing an aggregate outcome that broadly captures the effect of disease.
A self-reported history of ulcer or gastrointestinal bleeding was related to bad patient outcomes. Although the question used in the survey asked simultaneously about gastrointestinal bleeding and ulcer disease, we suspect that most of these patients may actually have had ulcer disease or gastroesophageal reflux, because frank gastrointestinal bleeding is uncommon. National asthma guidelines recommend assessment of many comorbid conditions, including reflux, which are known or suspected to worsen asthma control. Therefore, it was not surprising to see some relationship of this question to worse asthma outcomes.
For prediction of the aggregate outcome, our approach resulted in good discriminative accuracy, with increased sensitivity for screening patients at the expense of decreased specificity. In contrast, a model to predict hospitalization or ED use developed by Lieu and colleagues25 for pediatric populations achieved greater specificity at the expense of lower sensitivity. That model used predictors based on medication use and previous healthcare use. In our models, medication use was also predictive of outcomes, including theophylline to predict hospitalization and oral corticosteroids to predict ED use. However, these were not the strongest predictors in our model, perhaps because medication use was related to the presence of other patient characteristics available from our survey.
Other patient survey–based tools have been developed to predict adult asthma outcomes. The Perceived Control of Asthma Questionnaire is an 11-item survey that assesses self-efficacy. In a population of patients seen by asthma specialists in California, a score derived from the survey was shown to be associated with prior hospitalization, restricted activity, and cessation of employment.41 Prediction of future outcomes has not been established. Blanc and colleagues 22 developed an asthma severity measure that uses information from patient interview and medical chart review to predict work disability. This measure has been validated in cross-sectional and longitudinal analyses.22,23 Vollmer and colleagues42 developed and tested an index of asthma control that was significantly related to cross-sectional measures of healthcare use and generic and asthma-specific quality of life. Prospective investigations have shown that the index predicts healthcare use 43 and may be useful for monitoring patient self-management.44
An important feature of our approach is that all of the data can be obtained directly from patient surveys, without requiring medical chart review or other data sources. Many large MCOs use administrative data (ICD-9-CM codes and prescription records) to identify members with asthma and to monitor quality of care. However, administrative data do not provide the kind of detail that is possible using survey methods. Therefore, it is likely that our methods would have greater ability to discriminate between patients at higher and lower risk for bad outcomes. Patient surveys are more costly to mount than use of administrative data. However, many organizations already survey their members on a periodic basis. The addition of questions such as ours would add little to patient burden or incurred costs.
Our study has some limitations. We relied on patient-reported measures, without comparison to clinical or administrative data. However, whenever possible, we used questions from existing, previously tested instruments. For some data elements such as perceived access, satisfaction, and symptoms, patient reports are the gold standard,45-47 and previous studies48-51 have supported the validity of patient reports in related applications. Generalizability of our findings may be limited, as the education and race demographics are not typical of the entire US adult asthma population; our results are most applicable to patients in employer-based managed care. As MCOs begin to care for more vulnerable populations, it would be important to test tools in higher-risk disadvantaged populations. There are other outcomes that would be valuable to predict. For example, acute care visits to primary care physicians and the use of antibiotics make up substantial and avoidable costs. Finally, our data were collected a decade ago, during a period of declining use of xanthines for treatment of asthma.52 Although these medications are used with decreasing frequency, the question about use may continue to be relevant, as rare use may identify particularly difficultto- control subjects. However, exclusion of this variable would not change substantially the main findings of the study concerning the value of patient-reported data for predicting different outcomes. Future studies should attempt to replicate these findings, which would likely result in modifications in our models.
How could these models be used? Screening instruments can be adapted by healthcare organizations as case-finding tools for referral of patients to more intensive management programs. Unless an organization was concerned about a specific adverse outcome, it might be reasonable for it to apply the composite model. Patients to be surveyed would be selected using administrative data indicating previous visits for asthma and would be stratified based on whether they had recently been hospitalized or visited an ED. Some organizations might choose to survey only the outpatient stratum, which would otherwise be presumed to be at lower risk for adverse outcomes. Table 2 illustrates how a scoring system could help translate risk assessment data into practice. A healthcare organization that wanted to identify the 10% of patients at greatest risk of future adverse outcomes could identify all patients whose score was greater than 8. In individual practice, a primary care physician could use the screening questions to obtain relevant patient history during a patient visit.
Managed care organizations are in a unique position to provide integrated care, assisted by the assessment of patient-centered outcomes.34 National and international asthma guidelines have emphasized patient education, environmental control, periodic physiologic measurement, and a step-care approach to pharmacologic treatment of patients with asthma.5,53 These guidelines link processes of care and subsequent outcomes in asthmatics and function as a quality-of-care standard.54-56 Risk models such as those described herein provide a step toward linking current clinical quality process indicators with meaningful patient outcomes.
In conclusion, the questionnaire-based risk models developed in this project identified asthmatics at risk for increased symptoms, disability, and avoidable healthcare use. Risk models based on patient-reported data could be used to target individuals for early intervention and could help to prevent adverse outcomes.
Acknowledgments
We wish to acknowledge Jewel Crum-Freeman for her assistance in preparing the manuscript. |
Previous Article
