Objective: To test several patient-oriented asthma outcome measures and the Healthcare Effectiveness Data and Information Set (HEDIS) measure of appropriate medication for persistent asthma to determine the most useful quality indicator of asthma care.

Design: Prospective mail survey of adult employees and dependents with asthma.

Methods: The medical and pharmacy claims of all subjects from 12 months before and after the survey were abstracted. Outcomes measures included the Asthma Control Test (ACT), workday loss, unscheduled healthcare utilization (emergency department and inpatient care), and satisfaction with care.

Results: Although 81% of all responders had wellcontrolled asthma, persistent asthma was uncontrolled in 28%. Only 64.5% received appropriate controller medication. Well-controlled asthma is associated with a high degree of satisfaction, less workday loss, fewer prednisone bursts, and minimal unscheduled healthcare utilization. Except for a reduced incidence of more than 2 oral corticosteroid dispensings (6.4% vs 13.6%, P = .012), compliance with the HEDIS appropriate medication for asthma was not positively associated with any of the patient-centered outcomes studied.

Conclusions: Asthma control was the most useful patient outcome quality indicator in this study. Compliance with the HEDIS asthma measure in this population was not associated with a better patient-oriented outcome. This finding may be different with different levels of asthma control. The positive association between well-controlled asthma and patient satisfaction, minimal unscheduled healthcare utilization, and low workday loss suggests that asthma control as measured by ACT may be a better performance measure in asthma.

(Am J Manag Care. 2008;14(8):487-494)

Healthcare providers and organizations have responded to the public demand for excellence in medical care^1,2 by reporting performance based on disease-specific quality indicators. There are many types of quality indicators,³ and the choice is crucial because what is measured should be important to patients, payers, and providers. Which quality indicator is adopted influences the tactical approach or focus of healthcare providers and organizations. Physicians are skeptical of performance measures because of previous experience with inadequately calibrated and poorly validated measurements.^4,5 This is especially true if the quality indicator chosen is variably linked to better patient outcome. Most physicians subscribe to the “I believe, therefore I practice” philosophy.⁶

Asthma is one of the priority chronic diseases earmarked for quality improvement.7 The Healthcare Effectiveness Data and Information Set (HEDIS) asthma measure^8-10 is a “process of care” measure (ie, appropriate controller medication prescription for patients with persistent asthma). It is controversial because compliance with the HEDIS measure has never been prospectively linked to a better health outcome for the patients.^11-15 The current scientific literature has shifted focus from severity categorization to asthma control as an outcome measure.^16-20 Other relevant patient-oriented outcomes cited have included patient satisfaction,^21-23 unscheduled healthcare utilization, asthma-related quality of life,^24-26 and workday loss.²⁷ We sought to assess which quality indicators are most useful for quality improvement in asthma care in our institution.

METHODS
The Mayo Clinic Rochester (MCR) is a multispecialty practice that self-insures employees and dependents. A cross-sectional survey was conducted to simultaneously measure asthma control, patient satisfaction, healthcare utilization, and workday loss, and to determine the effectiveness of the HEDIS asthma measure. Our survey was limited to our employees and dependents (primary care patients for practical reasons, because we had access to full claims data regardless of where they received their care). We investigated medication and healthcare use by using actual pharmacy and billing claims submitted. Patients were categorized according to the HEDIS definition of persistent asthma, and severity risk adjustment was performed by medication intensity scale.²⁸

Study Population
The eligible study cohort was identified by using the Rochester Medical Index database.²⁹ The Rochester Medical Index classified asthma patients’ problems by using an internal coding system based on the Hospital Adaptation of the International Classification of Diseases, Eighth Revision. Patients’ problem lists were refined by applying a second database with International Classification of Diseases, Ninth Revision codes from billing data for clinic and hospital visits (see eAppendix Table A at www.ajmc.com). The resulting dataset was cross-referenced to a personnel and billing database to identify participants who were MCR employees, including retirees and/or dependents. All MCR employees and dependents more than 17 years of age at the time of the first mailing on May 5, 2005, with physician- diagnosed asthma were considered eligible. The dataset was further refined by including only active subjects (defined as those with any billing for medical care in the last 5 years) because many trainees and dependents of employees may have moved out of the Rochester area. The survey was posted to all eligible subjects. Seven subjects were found to have died before the initial posting of the survey, leaving 3137 subjects eligible for participation.

Demographic information on all potential subjects was extracted from the Mayo Clinic Subject Registration Database. Subjects without prior research authorization on record were excluded (n = 49). The survey was conducted with the approval of and in accordance with the policies of the Mayo Clinic Institutional Review Board. The first posting was on May 5, 2005, with a second mailing to nonresponders in July 2005. The study was closed in September 2005.

The medical and pharmacy claims of all eligible subjects were abstracted for the time period from May 2004 through May 2006 to encompass both 12 months before and 12 months after the survey. The pharmacy data included the National Drug Code number, information on dispensing, size of dispensing, number of canisters, and days supply. These were used to compute milligrams per day. The HEDIS definition of persistent asthma,⁸ the Global Initiative for Asthma (GINA)Medication Intensity Score,30 and the Kaiser Permanente Medication Intensity Scale²⁸ were based on claims data 12 months before the survey and were applied to characterize responders and nonresponders. We used the HEDIS definition of controller medication to classify medications (ie, long-acting bronchodilators were not included). All emergency department (ED) visits and hospitalizations were based on claims data from the previous 12 months before the survey.

Survey Instrument Development
The Mayo Clinic Employee and Dependent Asthma Survey (MEDAS) was developed by a collaborative interdisciplinary team. The resulting MEDAS questionnaire is a 4-page, 29-item survey tool incorporating the Asthma Control Test (ACT; permission obtained from QualityMetric, Lincoln, RI). There were questions regarding tobacco use, current medications, oral corticosteroid use, unscheduled healthcare utilization, workday loss and absenteeism, and satisfaction with access and asthma care in the past 12 months. Satisfaction with access to care, quality of care, and willingness to recommend were scored along a 5-point scale. There are 5 questions in the ACT, with a Likert scale from 1 to 5. The higher the score, the better the asthma control. A cumulative ACT score of ≥20 is the threshold for well-controlled asthma. The workday loss rate is calculated using 260 workdays per person per year.

Statistical Analysis
All continuous variables were presented as mean ± standard deviation and compared between groups with 2 sample t tests. Categorical variables were expressed as actual numbers as well as percentages and analyzed by the χ² test as appropriate. All statistical tests were 2-sided, and significance was set at P ≤.05. Statistical analysis was performed with SAS statistical software, version 8.1 (SAS Institute Inc., Cary, NC) and S-PLUS version 7.0.6 for Unix (Insightful Corporation, Seattle, WA). The satisfaction indicator was dichotomized to greater satisfaction (very satisfied/somewhat satisfied) versus lesser satisfaction (neither satisfied nor dissatisfied/somewhat dissatisfied/very dissatisfied). The overall care satisfaction was dichotomized to greater satisfaction (excellent/very good) and lesser satisfaction (good/fair/poor). Unscheduled healthcare utilization was defined by ED visit or hospitalization because MCR does not have an urgent care service and the billing data do not discern between different appointment types (ie, unscheduled visit or same-day visit).

RESULTS
Demographic Features of Responders and Nonresponders
There were 1056 responders out of 3137 (33.7%) eligible adult subjects. Most responders were women (70.5%), Caucasian (71.2%), and nonsmokers (95.9%); their mean age was 41 ± 13 years. A comparison of responders with the 2081 nonresponders revealed that the latter were significantly more likely to be men and younger (≤35 years of age) compared with responding counterparts (see Table 1). Many subjects had mixed heritage or did not provide this informaand were marked in the ethnicity category as unknown. Among responders, only 4.1% were current smokers. Only 30% of the total patients met HEDIS criteria for persistent asthma.

Comparison of Responders and Nonresponders: GINA Score Severity, Healthcare Utilization, and Medication Intensity Responders were more likely to meet the HEDIS criteria for persistent asthma than nonresponders (41.1% vs 24.4%, P <.001; Table 2). This difference was driven primarily by the percentage of subjects with 4 or more asthma medications (39.7% vs 23.0%, P <.001). Among potentially eligible subjects who met the HEDIS criteria for persistent asthma, the survey response rate was 46.1%. There was no difference in healthcare utilization (ED, inpatient, or ambulatory care visits) between responders and nonresponders. Responders were significantly more likely to have severe asthma by the GINA Medication Intensity Score (step 4; P <.001). The Kaiser Permanente Medication Intensity Scale did not differ between responders and nonresponders. Only a handful had an excessive requirement for beta agonists or frequent oral corticosteroid use.

The ACT Score Distribution
Among responders, 854 (81%) adults had an ACT score of 20 or higher, with 45.2% having an ACT score of 24 or 25 (see Figure). There was no significant difference in ACT scores between sex and age groups for this cohort (data not shown).

ACT Score, Appropriate Controller Medication Use, and Other Outcome and Utilization Measures A higher proportion of subjects with uncontrolled asthma as measured by the ACT required more than 2 oral corticosteroid bursts (8.4%; 17/202) than those with well-controlled asthma (2.9%; 25/854) (χ2 test; P <.001). Although there were too few events to detect a significant difference, 3 of 202 (1.5%) patients with ACT scores lower than 20 reported hospitalizations, whereas no (0/854) patients with well-controlled asthma had hospitalizations. Six percent (12/202) of patients whose asthma was not well controlled had an ED visit, whereas 0.2% (2/854) of those patients whose asthma was well controlled had an ED visit (χ2 test; P <.001).

A total of 94% of responders reported no workday loss in the preceding 12 months because of asthma. Well-controlled asthma was associated with low workday loss (χ2 test; P <.001). Only 7.9% of subjects with well-controlled asthma reported work/school days lost compared with 27% of the subjects whose asthma was not well controlled (χ2 test; P <.001). Those who reported lost workdays had a significantly lower mean ACT score than those with no lost workdays (16.3 vs 22.2, respectively; P <.001). This difference is even greater when we compare subjects who lost 10 or more work/school days with subjects who lost no work/school days (ACT score 16.3 ± 4.3 vs 22.2 ± 3.3; P <.001). Subjects whose asthma was under control reported more satisfaction with overall care than those whose asthma was not under control (80.4% vs 72.3%; χ2 test; P = .0105). Well-controlled asthma (ACT >20) was positively associated with satisfaction, fewer requirements for prednisone bursts, less workday loss, and less unscheduled healthcare use.

HEDIS Measures, Appropriate Controller Medication Use, and Other Outcome and Utilization Measures
Of the 1056 survey responders, 434 (41.1%) met the HEDIS definition for having persistent asthma (Table 2 and Table 3). Only 64.5% of subjects with persistent asthma were appropriately receiving controller medication by the HEDIS asthma measure. Asthma was significantly less well controlled in those who met the HEDIS definition of persistent asthma than in those who did not (71.9% vs 87.2%; P <.001). As shown in Table 3, 120 (27.7%) of those with persistent asthma had ACT scores below 20, whereas only 75 (12.1%) of those not meeting the HEDIS definition of persistent asthma had ACT scores below 20. ACT scores also were significantly better for all patients on controller medications (inhaled corticosteroids and/or leukotrienes) compared with those not on controller medications (ACT score of 22.4 ± 3.2 vs 21.4 ± 3.7; P <.001). Of those who met the HEDIS criteria for persistent asthma, 9% required more than 2 oral corticosteroid bursts, in contrast to 0.48% of those who met HEDIS criteria for nonpersistent asthma. Among those who met HEDIS criteria for persistent asthma, 15% reported workdays lost because of asthma, whereas only 7.7% of those who met HEDIS criteria for nonpersistent asthma reported workdays lost (P <.001).

Requiring more than 2 oral corticosteroid dispensings was significantly less likely (P = .012) in HEDIS patients who received controllers (6.4%) than in HEDIS patients who did not receive controllers (1.6%). However, compliance with the HEDIS performance measure (use of controller medications) had no positive association with other outcomes. Of all subjects who met the HEDIS definition of persistent asthma, 280 were on controller medications and 154 were not on controller medications. The ACT scores were 21.2 ± 3.8 and 20.7 ± 3.6, respectively (P = .1673). The 95% confidence interval for the difference in means between the 2 groups extended from -1.26 to 0.22, indicating no difference between the groups. Of the HEDIS persistent-asthma group, 17.5% of those on controller medication reported lost  workdays, whereas only 10.4% of those not on controller medication reported lost workdays (P = .047). There also was no difference in satisfaction among HEDIS persistent-asthma patients whether they were on controller medication or not (11.8% vs 16.2% dissatisfied subjects; χ2 test; P = .19). Those who were satisfied with their overall care were more likely to have well-controlled asthma (χ2 test; P <.01) and HEDIS nonpersistent asthma (χ2 test; P <.001). The numbers of ED visits for asthma and hospitalizations were too low for statistical testing to evaluate whether compliance with appropriate medication was associated with a better outcome.

Satisfaction With Access and Asthma Care
The majority (90.2%; 883/979) were satisfied with their ability to get an appointment to see their primary care provider, but only 77.9% (563/723) were satisfied with their ability to see an asthma specialist (see eAppendix Tables B and C at www.ajmc.com). There was a high degree of overlap between subjects reporting satisfaction with access to both their primary care provider and their asthma specialty care physician (539/715). However, responders who reported being dissatisfied with access to either type of provider were 5.8 times as likely to be dissatisfied with access to their asthma specialty care physician than to their primary care provider (P <.001). This is of interest because a copayment requirement was introduced the year before the survey. Most of the subjects (83.4%) rated the care received from their asthma care provider as satisfactory, and 86.3% rated their overall asthma care as very good or excellent. Last, 97.2% of patients said they were willing to recommend the MCR to their family and friends for asthma care. There was a high degree of agreement between asthma control when crossed with patients’ satisfaction with their usual asthma care provider (χ2 test; P <.001).

Emergency Department, Inpatient Care, and Lost Productivity, and ACT Score
Although 17% self-reported the use of oral corticosteroids, only 2% and 2.3% required an ED visit or inpatient care for asthma, respectively. Ninety percent of subjects without additional healthcare use reported no lost workdays; conversely, only 33.3% of subjects with additional healthcare utilization reported any lost workdays. Conservatively estimated, 374 days were lost because of asthma among those surveyed. Using 260 workdays per year per person, the potential workday loss to asthma is 0.14%, and the average workday loss per patient per year in the study is 0.35. 

DISCUSSION
Although the majority of patients surveyed had mild asthma and a high degree of asthma control, more than a quarter with persistent asthma continued to have burdensome symptoms (ACT <20). A third in this subgroup could have benefitted from being on a controller medication. There is a high degree of agreement between asthma control and other outcome measures, including patient satisfaction, minimal unscheduled healthcare utilization, and low workday loss. Compliance with the HEDIS appropriate medication measure for asthma was not positively associated with other patient-oriented outcome measures. In considering which quality indicator to use in our primary care practice, we found that unscheduled healthcare utilization and workday loss may have a floor effect (ie, low and infrequent), whereas satisfaction with access and willingness to recommend care may have a ceiling effect. In our institution, these may be less amenable as targets for improvements in asthma care but may be important as quality indicators of service and for cost considerations.

In recent years, asthma control has emerged as a major end point of therapy.³¹ Asthma control must be differentiated from asthma severity.18 The latter is a reflection of the intensity of the underlying disease process, whereas the former reflects how well the clinical manifestations of the disease have been minimized. Asthma control is easily measured, and several validated instruments are available for routine use in the ambulatory setting.^32-34 Asthma control not only is associated with improved quality of life³⁵ and reduced healthcare utilization,^25,26,34 but it also can be used as an end point for active disease management and intervention.³⁶ Current treatment guidelines recognize that regardless of disease severity, it is possible with proper medical care to achieve the same degree of asthma control.^18,26,32,37 The latest GINA has completely eschewed the use of severity  categorization in favor of asthma control assessment for monitoring and adjusting therapy.¹⁷ Based on the literature and our present study, asthma control assessment is better than the current HEDIS asthma measure. It also is convenient, provides realtime management relevancy to the provider, and is an excellent measure for quality assurance and for improving performance. Since the submission of the manuscript, the Expert Panel Report has endorsed the use of asthma control as one of the major end points in therapy.¹⁶

Although this degree of asthma control in our institution appears to be much better than that in pharmacy-based reports³⁸ and Web-based reports,³⁹ there is substantial room for practice improvement. The observed level of control may have been influenced by the demographic features of the responders, who had milder asthma and were predominantly nonsmokers and of Northern European descent. The HEDIS definition of persistent asthma does stratify patients in terms of severity for assessment. As a performance measure (ie, use of controllers among patients with persistent asthma), it correlated only with a reduced need for oral corticosteroids. This may be related to the high degree of asthma control and the low rate of ED use and hospitalization observed in this study population. Whether the HEDIS asthma measure has a better association with other patient-centered outcomes in populations whose asthma is not as well controlled remains to be seen. As a performance measure, asthma control appears to be the dominant variable.

The present study has several advantages for assessing our institutional performance in providing asthma care. These include the simultaneous assessment of several patient-centered outcomes (ie, asthma control, patient satisfaction, lost productivity, unscheduled healthcare utilization); a prospective, cross-sectional design; and a large clinical sample.

The results of this study also should be viewed in the context of its limitations. The first potential limitation of our study is that, despite enlisting measures previously shown to promote a survey response such as a personalized letter, stamped return envelopes, and a second mailing to nonresponders,⁴⁰ the response rate was 34%. This low rate may be in part because of the length and complexity of the survey questionnaire (4 pages needing 15-25 minutes to complete) and the lack of a monetary incentive.40 However, there is strong evidence that it is becoming more difficult to obtain a response rate of a historically acceptable level of 60% to 70% because participation in general population surveys has been decreasing for some time, despite the deployment of the aforementioned heroic measures to ensure response.^41-43 Alreck and Settle have commented on the problem of low response rate to mail surveys, with most response rates often falling below 30%.⁴⁴ Moreover, recent discussion suggests only a weak relationship between a survey’s response rates and response bias.^45,46 We sought to mitigate potential bias resulting from a low response rate by examining demographic features as well as disease severity through risk adjustment from billing and pharmacy data for both responders and nonresponders. Our analysis of responders and nonresponders revealed underrepresentation of men and the young (age ≤35 years) in our responding sample, and overrepresentation of persistent asthma in the responder group. This finding, coupled with evidence from a previous study showing that men and younger individuals tended to report less daytime and nocturnal symptoms compared with women and older individuals,⁴⁷ suggests that generalization of our findings to other populations should be made with caution and that we may be underestimating the percentage of patients with good asthma control in the population.

A second limitation beyond our control is the demographic composition of our employees, the majority of whom are Caucasian and nonsmokers. In addition, the survey was conducted in a population of healthcare workers and dependents, who may have had better adherence to therapy and better health status (ie, less tobacco use) and consequently may have had less severe asthma.⁴⁸ Nonetheless, we believe that this degree of asthma control with minimal requirement for unscheduled healthcare utilization and minimal loss of workdays can be achieved in any insured population with no barrier to healthcare. From the perspective of a single employer, it is important to use clinically relevant as well as achievable benchmarks to compare outcomes across institutions and longitudinally within each institution.

The third potential limitation is the use of an unvalidated patient satisfaction survey. The patient satisfaction questions were taken from the survey tool used by our institution to monitor service quality. We acknowledge that the preferred methodological tack in this instance would have been to utilize satisfaction questions that underwent the rigorous piloting and psychometric testing associated with other instruments such as the surveys developed by the Agency for Healthcare Research Consumer Assessment of Healthcare Providers and Systems (CAHPS) program. However, one of the analytical objectives of the investigation was to compare and trend internal data on satisfaction of patients with asthma vis-à-vis other chronic diseases seen within the institution. As such, we decided to utilize the items from our own internal satisfaction survey. It should be noted that this instrument has been used at our institution for over a decade with tens of thousands of data points. Moreover, although the developmental process for our institutional survey is not as rigorous as that deployed for CAHPS, the survey itself has undergone pilot testing under the auspices of our Mayo Clinic Survey Research Center and with oversight by its survey methodological staff.