Currently Viewing:
The American Journal of Managed Care March 2015
Evaluation of Care Management Intensity and Bariatric Surgical Weight Loss
Sarit Polsky, MD, MPH; William T. Donahoo, MD; Ella E. Lyons, MS; Kristine L. Funk, MS, RD; Thomas E. Elliott, MD; Rebecca Williams, DrPh, MPH; David Arterburn, MD, MPH; Jennifer D. Portz, PhD, MSW; and Elizabeth Bayliss, MD, MSPH
Potential Savings From Increasing Adherence to Inhaled Corticosteroid Therapy in Medicaid-Enrolled Children
George Rust, MD, MPH, FAAFP, FACPM; Shun Zhang, MD, MPH; Luceta McRoy, PhD; and Maria Pisu, PhD
Innovation in Plain Sight
Karen Ignagni, MBA, President and Chief Executive Officer, America's Health Insurance Plans
Early Changes in VA Medical Home Components and Utilization
Jean Yoon, PhD, MHS; Chuan-Fen Liu, PhD, MPH; Jeanie Lo, MPH; Gordon Schectman, MD; Richard Stark, MD; Lisa V. Rubenstein, MD, MSPH; and Elizabeth M. Yano, PhD, MSPH
Currently Reading
Are Healthcare Quality "Report Cards" Reaching Consumers? Awareness in the Chronically Ill Population
Dennis P. Scanlon, PhD; Yunfeng Shi, PhD; Neeraj Bhandari, MD; and Jon B. Christianson, PhD
Insurance Impact on Nonurgent and Primary Care-Sensitive Emergency Department Use
Weiwei Chen, PhD; Teresa M. Waters, PhD; and Cyril F. Chang, PhD
Cost Differential by Site of Service for Cancer Patients Receiving Chemotherapy
Jad Hayes, MS, ASA, MAAA; J. Russell Hoverman, MD, PhD; Matthew E. Brow, BA; Dana C. Dilbeck, BA; Diana K. Verrilli, MS; Jody Garey, PharmD; Janet L. Espirito, PharmD; Jorge Cardona, BS; and Roy Beveridge, MD
The Combined Effect of the Electronic Health Record and Hospitalist Care on Length of Stay
Jinhyung Lee, PhD; Yong-Fang Kuo, PhD; Yu-Li Lin, MS; and James S. Goodwin, MD
Strategy for a Transparent, Accessible, and Sustainable National Claims Database
Robin Gelburd, JD, BA
Treatment Patterns, Healthcare Utilization, and Costs of Chronic Opioid Treatment for Non-Cancer Pain in the United States
David M. Kern, MS; Siting Zhou, PhD; Soheil Chavoshi, MS; Ozgur Tunceli, PhD; Mark Sostek, MD; Joseph Singer, MD; and Robert J. LoCasale, PhD
Trends in Mortality Following Hip Fracture in Older Women
Joan C. Lo, MD; Sowmya Srinivasan, MD; Malini Chandra, MS, MBA; Mary Patton, MD; Amer Budayr, MD; Lucy H. Liu, MD; Gene Lau, MD; and Christopher D. Grimsrud, MD, PhD
Long-Term Outcomes of Analogue Insulin Compared With NPH for Patients With Type 2 Diabetes Mellitus
Julia C. Prentice, PhD; Paul R. Conlin, MD; Walid F. Gellad, MD, MPH; David Edelman, MD; Todd A. Lee, PharmD, PhD; and Steven D. Pizer, PhD
Factors Affecting Medication Adherence Trajectories for Patients With Heart Failure
Deborah Taira Juarez, ScD; Andrew E. Williams, PhD; Chuhe Chen, PhD; Yihe Goh Daida, MS; Sara K. Tanaka, MPH; Connie Mah Trinacty, PhD; and Thomas M. Vogt, MD, MPH

Are Healthcare Quality "Report Cards" Reaching Consumers? Awareness in the Chronically Ill Population

Dennis P. Scanlon, PhD; Yunfeng Shi, PhD; Neeraj Bhandari, MD; and Jon B. Christianson, PhD
Using longitudinal data, this paper examines to what extent comparative quality information in the form of healthcare "report cards" is reaching chronically ill adults.
Background: Significant investments have been made to provide comparative healthcare quality information (CQI) to the public, but whether these efforts are increasing awareness of CQI is unknown.

Objectives: To provide regional estimates of change in awareness of CQI among the chronically ill population residing in 14 geographic regions of the United States between 2008 and 2012. Additionally, to examine its correlation with the changes in the availability of quality reports.

Study Design: Data from 2 waves (2008 and 2012) of a random-digit dial survey of 11,896 adults with chronic illness.

Methods: Regression-adjusted change in the percentage of respondents aware of physician and hospital CQI, and Pearson correlations between regional change in awareness of CQI and regional change in availability of quality reports.

Results: While the number of reports on both hospital quality and physician quality increased between 2008 and 2012, there was significant change in awareness of only physician CQI (12.8% to 16.2%, regression-adjusted change of 3.7 percentage points; P <.05). No significant correlation was found between the change in awareness of CQI and the change in availability of hospital quality reports or physician quality reports.

Conclusions: Awareness of physician CQI among the chronically ill increased modestly between 2008 and 2012, but no significant increase in awareness of hospital quality was observed. As efforts to report CQI accelerate, more attention to approaches to dissemination may be warranted in order to increase awareness in the chronically ill population.

Am J Manag Care. 2015;21(3):236-244
  • Federal investment in providing comparative quality information (CQI) has been substantial and increasing as CMS’s “Physician Compare” website is now online.
  • There is little information about consumer awareness, especially among those with chronic conditions.
  • This is the first study to examine awareness of CQI with longitudinal data.
  • Our results suggest that awareness of healthcare quality reports remains low, with modest increases between 2008 and 2012.
  • There is also substantial variation across regions in terms of baseline levels of awareness and longitudinal changes.
Public reporting of healthcare provider quality measures has emerged as a major reform strategy. Reporting efforts have been propelled by an increasing recognition that healthcare markets vary on the dimensions of prices and quality, may lack transparency on these dimensions, and may fail to foster incentives that will lead to better outcomes.1-4 The availability of quality information may lead to consumer “shopping” for higher-quality providers (“consumer pathway”), incentivizing providers to improve the quality of care they provide. However, this strategy assumes awareness and use of quality reports by consumers.5,6

Harris and Buntin (2008) define comparative quality information (CQI) as including “all materials disseminated by public and private entities that are intended to inform consumers’ decisions in choosing healthcare providers.”7 This includes quality information that may be seen by consumers at their workplace, gathered from traditional printed sources (eg, newspapers, books, magazines, journals), or via the Internet (eg, social media, Web searches, e-mail).

The variety and scope of CQI has grown substantially, with information coming from many sources offering a range of measures, and making quality comparisons among provider entities.8 There is regional variability in CQI accessible to consumers in terms of content, credibility, and applicability of information to local consumer needs.9 A substantial proportion of CQI is targeted toward the chronically ill,9 a population group that is the focus of the chronic care model and the patient-centered medical home model,10,11 with particular emphasis on disease self-management. Despite the abundance of CQI, it is unclear whether consumer awareness has kept pace with its growth, especially over the last few years.

The federal and state governments, regional coalitions, and employers and health plans have invested significant resources in making CQI available to the public.12 While quality information for Medicare-certified hospitals has been publicly available to consumers since 2005, the Affordable Care Act mandated that CMS report performance data about Medicare-enrolled physicians on the “Physician Compare” website, which is now available online.13 Despite these expanding policy initiatives, little empirical information on consumer awareness of CQI exists. Some large nationally representative surveys that preceded many current reporting efforts found that less than 1 in 5 consumers are aware of hospital or physician CQI.14-16 A more recent study found higher awareness (69%) of online physician ratings17; however, none of these studies looked at the awareness of CQI among chronically ill patients, which is a high-priority group for healthcare reform. Moreover, data on regional variation in consumer awareness of CQI is unavailable, as is information on whether the awareness among the chronically ill is related to the variation in the amount of CQI available by type of chronic disease.

We provide overall and regional estimates of awareness of CQI among the chronically ill population residing in 14 regions of the United States for 2 different periods: 2008 and 2012. These regions are sites for multi-stakeholder alliances in Aligning Forces for Quality (AF4Q), a national program funded by the Robert Wood Johnson Foundation (RWJF), and aimed at realizing community-wide improvements in health and healthcare delivery.18 We also correlated regional estimates of availability of quality reports with regional awareness of these reports.



We used information from 2 data sources: the AF4Q Community Quality Reporting Tracking Database (AF-4QTD) and the Aligning Forces for Quality Consumer Survey (AF4QCS). AF4QTD tracks the number and contents of physician and hospital quality public reports released in the 14 AF4Q regions. Research staff members systematically review the websites of various public and private organizations that are sponsoring public reports in these regions, and they conduct interviews with their program personnel to request additional details as well as to gather specific information about these reports.9

The first wave of AF4QCS was a random-digit dial (RDD) survey conducted between June 2007 and August 2008 for chronically ill adults (18 years or older) in the 14 AF4Q regions across the nation, including whole states and metropolitan and rural areas.18 All respondents had visited healthcare professionals during the previous 2 years for the care of at least 1 of the following 5 conditions: diabetes, hypertension, asthma, chronic heart disease, and depression. 19 The same respondents were attempted again in the second wave between July 2011 and November 2012. To compensate for attrition and to account for potential change in the population between the 2 periods, additional respondents were surveyed in the second wave using RDD.

Our study sample includes 7139 individuals from the first wave and 8362 from the second wave. The response rate of the first wave was 27.6% based on the American Association of Public Opinion Research (AAPOR) method, and 45.8% based on the Council of American Survey Research Organizations (CASRO) method. In the second wave, the panel response rate was 63.3%; the overall response rate—calculated as the weighted average of the panel and the new RDD samples—was 39.7% based on the AAPOR method, and 42.1% based on the CASRO method.

Our response rates are in line with the general declining trend in survey response rates.20-22 Recent studies have shown substantial variation in reported response rates depending on the methods of calculation23,24 and the lack of association between response rates and nonresponse bias.20,24 Moreover, surveys with lower response rates do not lead to significantly different estimates,25-27 and using extra resources to achieve a better response rate may actually introduce bias.28-30 We compared AF4QCS to the 2008 and 2011 National Health Interview Survey, which has a greater than 90% response rate because of its continuous annual sampling design and personal household interviews by trained staff. No significant differences were found in terms of respondents’ demographic characteristics and prevalence of chronic conditions.

Availability of Physician and Hospital CQI

The variation in the availability of physician and hospital CQI is measured by the number of physician quality reports and the number of hospital quality reports accessible to the public, using information at 2 points in time from the AF4QTD.9 Reports produced by local and regional health plans were included, but only if they were available to the public and not to plan enrollees exclusively. Additionally, we used a directory created by the RWJF to systematically capture physician and hospital quality reports that are national in scope.31 The database includes reports produced by a variety of organizations, including CMS, the Leapfrog Group, National Committee for Quality Assurance, and Bridges to Excellence, in addition to a number of health plans that operate in many markets nationally. These reports vary significantly in the percentage of hospitals and doctors included in the reports, as well as whether participation in the report is voluntary. For example, in our counts we excluded reports that provide comparative scores for a narrow or selected group of hospitals or physicians, sometimes due to the voluntary decision of the hospital or physician practice to provide data. We also estimated the percentage of the population with access to a health plan-sponsored report using plan enrollment estimates from Interstudy.

Awareness of Physician and Hospital CQI

The awareness of physician and hospital CQI is measured by the responses to consecutive survey questions in AF4QCS. The first question asks respondents whether they had seen information comparing physicians, hospitals, or health plans during the past 12 months. Among those who answer yes, subsequent questions separately ask if the respondent had seen information comparing physician quality and hospital quality during the past 12 months. The response was coded as 1 if the respondent said yes to the first and subsequent questions, and 0 if the respondent said no to either the first or subsequent questions. Survey questions focused on comparative information or reports that provided data for all or most providers that might be selected by consumers in a particular healthcare market. It should be noted that “seeing” is a specific yet important aspect of “awareness”—respondents may be aware of CQI without seeing it, but the survey question was designed to be more specific (see) rather than general (aware) to limit the possibility of socially desirable responses and to reduce measurement error. Moreover, actual visual exposure to CQI is more likely to have a direct and meaningful impact on individuals.

Statistical Analysis

We calculated the percentage of respondents who were aware of physician and hospital CQI. To understand whether certain regions differed from the others in terms of the population level awareness, for each of the 2 periods, we conducted a 2-sample proportion z test for each region versus the other regions combined. We then calculated regression-adjusted change of awareness over the 2 waves of AF4QCS for all regions combined and for each region separately. The (linear) regression adjustments allowed us to control for sociodemographic characteristics of survey respondents. To see if the response varied based on the likelihood that a report card contained measures associated with one’s specific health conditions, we conducted sensitivity analysis using a subsample of respondents with diabetes and a subsample of respondents with depression, since the majority of physician report cards contain diabetes measures while few contain depression measures. To examine whether there was an association between the amount of available CQI and awareness of CQI, we calculated Pearson correlation coefficients between the change in population awareness of CQI and the change in regional availability of publicly available quality reports, and the change in the percentage of the population with access to a health plan-sponsored report.


Study Population

Table 1 shows the demographic profile of survey respondents in 2012. Most of the sample respondents were older, white, female, college-educated, and had some form of public insurance, consistent with other studies of the adult chronically ill.32 About half of sample respondents had 2 or more chronic conditions, with hypertension and diabetes being the most prevalent. About 9% of our 2012 sample was uninsured, and most had an annual household income between $30,000 and $60,000.

Availability of Comparative Quality Reports

Copyright AJMC 2006-2020 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up