Seth W. Glickman, MD, MBA

The Affordable Care Act directs the Secretary of Health and Human Services to compare individual physicians using patient experience measures. This policy initiative will utilize the Clinician and Group Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS) survey program. It will impact over 700,000 eligible physicians and will be tied to reimbursement and the Centers for Medicare & Medicaid Services’ Physician Compare reporting feature starting in 2015. We believe that the relevance of this framework to today’s clinical environment is a critical issue to address before implementing this regulatory mandate. In this article we discuss our concerns about tying individual physician performance to CG-CAHPS scores, including: 1) intrinsic versus extrinsic approaches to assessing the patient experience, 2) measurement issues, and 3) unintended consequences. We also suggest an alternative pathway and opt-out mechanism to facilitate more rapid translation of service excellence into clinical practice.

The Affordable Care Act directs the Secretary of Health and Human Services to compare individual physicians using patient experience measures.

		This initiative could impact over 700,000 eligible physicians and will be tied to reimbursement and the Centers for Medicare & Medicaid Services’ physician compare reporting feature starting in 2015.

		We have policy concerns which center around 3 areas: 1) intrinsic versus extrinsic approaches to assessing the patient experience, 2) measurement issues, and 3) unintended consequences.

		An opt-out pathway allowing organizations to assume accountability for patient experience measurement should be considered.

The United States is poised to embark on one of the largest measurement efforts in the history of medicine—the assessment of patient satisfaction with physician performance using the Clinician and Group Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS) surveys.

The Affordable Care Act (ACA) directs the Secretary of Health and Human Services to compare individual physicians using patient experience measures.

 This initiative will impact over 700,000 eligible physicians and will be tied to reimbursement and the Centers for Medicare & Medicaid Services’ (CMS’) Physician Compare reporting feature starting in 2015 for provider groups and for all individual physicians by 2017.

While acknowledging the role of CMS in catalyzing an increasing focus on the patient experience in provider organizations, we  believe that the CG-CAHPS construct for evaluating individual physician performance is flawed. The final regulations to implement  this section of the ACA for individual providers have not yet been written, thus the importance of raising our concerns about this process to the physician community now. Our concerns about tying individual physician performance to CG-CAHPS data center around 3 areas: 1) intrinsic versus extrinsic approaches to assessing the patient experience, 2) measurement issues, and 3) unintended consequences.

Intrinsic Versus Extrinsic Approaches to Assessing the Patient Experience

Since the inception of CAHPS, we have seen profound improvement in our understanding of the concept of service operations in healthcare.

 In a learning healthcare system, an information feedback loop to clinicians on their activities and outcomes is critical in achieving the best possible care.

 Service transformation is a dynamic process that requires actionable data for physicians and managers and the flexibility to focus on operational issues unique to each clinical environment. However, CG-CAHPS was not designed for such a framework. In fact, the explicit goal of CAHPS is to “develop standardized patient surveys that can be used to compare results across sponsors and over time” where priority is on public reporting rather than facilitating improvement at the practice level.

By analogy, one of the key methodologies in the service operations field is the Toyota Production System.

 Here, information is developed and assessed in real time by all members of the production team. This approach cannot work when data points are 1) determined by an external group, 2) not designed for the specific environment at hand, and 3) not available in real time. Additionally, Toyota does not wait for the annual J.D. Power survey of consumers to decide how to improve its automobiles. Contrary to the  Toyota model, CMS prevents physicians from taking ownership of measurement of the patient experience. For example, the surveys are generated by an external body in a standardized fashion which does not allow modification of the core measures. Further, in an  effort to preserve the integrity of the external measurement process, CMS requires that its surveys be administered before any internally informed patient survey.

 This complexity makes it very difficult for a provider organization to collect its own patient  experience data. Of note, the delay in CAHPS survey administration may be up to 6 weeks and responses cannot be tracked to individual patients, thereby limiting the utility and actionability of the data. There is no reason to believe the guidance on this issue will be any different for CG-CAHPS than it is for CAHPS Hospital Survey (HCAHPS).

This is not to argue the potential benefits to extrinsic measurement of the patient experience, such as greater transparency and accountability. However, we believe that a preferred  approach to measurement would be intrinsic, with a more limited approach to external evaluation that does not inhibit internal  innovation within provider organizations. In fact, CMS recognizes the importance of  this intrinsic approach, stating “HCAHPS survey items complement the data hospitals currently collect to support improvements in  internal customer services and quality related activities.”

However, current regulations do not allow sufficient flexibility around this framework. Later in this paper we will discuss ways in which CMS could consider implementing pay for performance concepts focused on enhancing internal performance efforts.

Measurement Challenges and Methodologic Issues

Specific constructs within CAHPS are indeed evidence-based and in cross-sectional studies patient experience measures are associated with improved clinical performance and outcomes.

However, there is a gap in effective strategies to translate solutions to improving the patient experience into clinical practice. The creation of financial penalties and public reporting programs without a more robust understanding f the science of improving the patient experience in provider organizations may fail to achieve the policy goal of enhancing patient care or may have unintended consequences.

 Previous work suggests that substantial change within provider organization around the patient experience will only occur with significant changes in organizational strategy, leadership, and culture.

These organizational challenges require actionable data to help managers lead this transformation.

CG-CAHPS proposes to capture data on at least 300 patient encounters per practice and 45 surveys per physician. Currently, participation in the program is the sole criterion established by CMS for receipt of payment updates, but in the hospital realm, these data have been used to “incentivize” performance as well. Without a specific construct of how these data will be used in physician payment, we envision  several major challenges. In measuring performance of 700,000 physicians, minute differences on a 10-point likert scale will be statistically significant. If the data are used to rank physicians (assuming a score distribution similar to HCAHPS), a change of 100,000 places in ranking would result from a difference of only 0.1 on an overall score. Moreover, ceiling effects could exacerbate these measurement issues within and across practices, especially as they develop strategies to improve scores to try to achieve payment updates.

In many situations, physicians are now employees of larger systems without direct control of their practice environment. Will the CG-CAHPS measure be used to update professional services, technical/facility fees, or both? In fact, physicians in a larger system might correctly argue they have minimal control over the overall patient experience, and that their survey data reflect system versus individual performance. For many physicians in academic medicine, patient care is melded into a life that also includes teaching and research. It may not be possible to attain the number of encounters for those whose practice may be limited to 1 or 2 specialty clinics per week.

The reported patient experience is often a factor in determining compensation for senior managers to the detriment of using assessment of the patient experience for improving service operations. Rather than a positive achievement, we fear we have institutionalized administrative aspects of assessment of the patient experience rather than the concept of improving service operations. In fact, the constant “strive for five” throughout the healthcare system is evidence of how measurement of patient experience has been coopted into a performance metric for senior managers.

 Strive for five is a marketing campaign for higher scores, not a commitment to a better patient experience. Previous work has shown that hospitals that focus on administrative components of performance measures actually deliver worse clinical outcomes.

The emphasis on public reporting may also cause providers to shy away from treating certain “high-risk” patients or make them more likely to acquiesce to patient requests, leading to unnecessary healthcare utilization (ie, prescription drugs, imaging) and higher costs.

From a policy perspective, the original goal of CAHPS to develop comparative information for consumers in an era of solo practices has been eclipsed over time. To this end, the evidence seems to suggest that health systems, not patients, are most impacted by publically reported performance measures.

These organizations are now struggling with the concept of how to create robust measurement tools for individual practices within the overall system, again an effort very different from that proposed by CMS through the CAHPS program.

There is an urgent need to move the science of measurement of the patient experience forward.

 We believe that the time is right to consider approaches that incentivize the transformation of healthcare consistent with the Toyota Production System model, where service operations is nurtured as a management construct. To address this policy goal, we propose a CG-CAHPS opt-out alternative for organizations willing to take responsibility for development and implementation of their own patient experience measurement. CMS could require confirmation that such a measurement process is in place and actively being used by the management of organizations in the opt-out pathway. Further, there could be a requirement that each “self-managed” site must develop an annual report of achievements (submitted to hospital or practice management, a public website, or CMS). This report will require oversight to ensure that organizations are using the opt-out mechanism to develop actionable data and that they are building management capacity to respond to the data they receive. From a financial perspective, those sites that choose this path and complete the required reporting would then be considered in the to tier of any pay-for-performance criteria. From a policy perspective, CMS should aggressively consider this type of optout pathway for other clinical performance measurement approaches through the Centers for Medicare & Medicaid Innovation in order to move the science of performance measurement forward. We believe this alternative pathway should apply to future CAHPS surveys (eg, emergency department CAHPS [ED CAHPS]) and other related quality measurement efforts.

We need to carefully reconsider the overall CG-CAHPS framework and its associated measurement issues before this policy experiment becomes a reality for individual physicians and their practices all over the country. In addition, we propose an opt-out pathway allowing organizations to assume accountability for assessment of the patient experience to facilitate more rapid translation of service excellence into clinical practice. 

From Department of Emergency Medicine, University of North Carolina School of Medicine, Chapel Hill (SWG), Chapel Hill, NC; Duke Clinical Research Institute, and Department of Medicine, Duke University School of Medicine (KAS), Durham, NC.

 Drs Glickman and Schulman are co-founders, equity holders, and board members of Bivarus, Inc. They report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Concept and design (SWG, KAS); acquisition of data (SWG, KAS); analysis and interpretation of data (SWG, KAS); drafting of the manuscript (SWG, KAS); critical revision of the manuscript for important intellectual content (SWG, KAS); statistical analysis (SWG, KAS); provision of study materials or patients (SWG, KAS); administrative, technical, or logistic support (SWG, KAS); and supervision (SWG, KAS).

Seth W. Glickman, MD, MBA, University of North Carolina, 170 Manning Dr, CB #7594, Chapel Hill, NC 27599. E-mail: seth_glickman@med.unc.edu.1. Surveys and tools to advance patient care: about CAHPS. http://cahps.ahrq.gov/about.htm. Accessed February 18, 2013.

2. The Patient Protection and Affordable Care Act (Pub. L. 111—148). 2010. Section 10331(a).

3. Federal Register. Department of Health and Human Services Centers for Medicare & Medicaid Services 42 CFR Parts 410, 414, 415, et al. Medicare Program; Revisions to Payment Policies Under the Physician Fee Schedule, DME Face-to-Face Encounters, Elimination of the Requirement for Termination of Non-Random Prepayment Complex Medical Review and Other Revisions to Part B for CY 2013; Final Rule. http://www.gpo.gov/fdsys/pkg/FR-2012-11-16/pdf/2012-26900.pdf. Accessed June 13, 2013.

4. Kenagy JW, Berwick DM, Shore MF. Service quality in health care. JAMA. 1999;281:661-665.

5. Institute of Medicine of the National Academies. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, DC: The National Academies Press; 2012.

6. Culig MH, Kunkle RF, Frndak DC, Grunden N, Maher TD Jr, Magovern GJ Jr. Improving patient care in cardiac surgery using Toyota production system based methodology. Ann Thorac Surg. 2011;91:394-399.

7. The CAHPS Hospital Survey Quality Assurance Guidelines, Version 7.0. http://www.hcahpsonline.org/files/HCAHPS%20Quality%20Assurance%20Guidelines%20V7.0%20March%202012.pdf. Published March 2012. Accessed April 18, 2013.

8. Hospital Care Quality Information from the Consumer Perspective. http://www.hcahpsonline.org. Accessed June 11, 2013.

9. Sequist TD, Schneider EC, Anastario M, et al. Quality monitoring of physicians: linking patients’ experiences of care to clinical quality and outcomes. J Gen Intern Med. 2008;23:1784-1790.

10. Manary MP, Boulding W, Staelin R, Glickman SW. The patient experience and health outcomes. N Engl J Med. 2013;368:201-203.

11. Auerbach AD, Landefeld CS, Shojania KG. The tension between needing to improve care and knowing how to do it. N Engl J Med. 2007; 357(6):608-613.

12. Davies E, Shaller D, Edgman-Levitan S, et al. Evaluating the use of a modified CAHPS survey to support improvements in patient-centered care: lessons from a quality improvement collaborative. Health Expect. 2008;11:160-176.

13. Lee F. If Disney Ran Your Hospital: 9-1/2 Things You Would Do Differently. Bozeman, MT: Second River Healthcare Press; 2004.

14. Glickman SW, Boulding B, Staelin R, Roos MT, Schulman KA. An alternative pay for performance scoring method: implications for quality improvement and patient outcomes. Medical Care. 2009;47:1062-1068.

15. Fenton JJ, Jerant AF, Bertakis KD, Franks P. The cost of satisfaction: a national study of patient satisfaction, health care utilization, expenditures, and mortality. Arch Intern Med. 2012;172:405-411.

16. Ketelaar NA, Faber MJ, Flottorp S, et al. Public release of performance data in changing the behaviour of healthcare consumers, professionals or organisations. Cochrane Database Syst Rev. 2011;9;(11): CD004538.

17. Berenson RA, Pronovost PJ, Krumholz HM. Achieving the potential of health care performance measures: the Robert Wood Johnson Foundation. http://www.rwjf.org/content/dam/farm/reports/reports/2013/rwjf406195. Published May 2013. Accessed June 12, 2013.

Articles

We discuss our concerns about tying physician performance to CG-CAHPS scores and suggest an alternative approach to facilitate translation of service excellence into clinical practice.

The Mis-Measure of Physician Performance

To determine whether hospitals where patients report higher overall satisfaction with their interactions among the hospital and staff and specifically their experience with the discharge process are more likely to have lower 30-day readmission rates after adjustment for hospital clinical performance.

 Among patients 18 years or older, an observational analysis was conducted using Hospital Compare data on clinical performance, patient satisfaction, and 30-day risk-standardized readmission rates for acute myocardial infarction, heart failure, and pneumonia for the period July 2005 through June 2008.

 A hospital-level multivariable logistic regression analysis was performed for each of 3 clinical conditions to determine the relationship between patient-reported measures of their satisfaction with the hospital stay and staff and the discharge process and 30-day readmission rates, while controlling for clinical performance.

 In samples ranging from 1798 hospitals for acute myocardial infarction to 2562 hospitals for pneumonia, higher hospital-level patient satisfaction scores (overall and for discharge planning) were independently associated with lower 30-day readmission rates for acute myocardial infarction (odds ratio [OR] for readmission per interquartile improvement in hospital score, 0.97; 95% confidence interval [CI], 0.94-0.99), heart failure (OR, 0.96; 95% CI, 0.95-0.97), and pneumonia (OR, 0.97; 95% CI, 0.96-0.99). These improvements were between 1.6 and 4.9 times higher than those for the 3 clinical performance measures.

 Higher overall patient satisfaction and satisfaction with discharge planning are associated with lower 30-day risk-standardized hospital readmission rates after adjusting for clinical quality. This finding suggests that patient-centered information can have an important role in the evaluation and management of hospital performance.

	Hospitals routinely use patient satisfaction surveys to assess the quality of care, although it remains unclear whether patient satisfaction data provide valid information about the medically related quality of hospital care.

		Higher patient satisfaction with inpatient care and discharge planning is associated with lower 30-day readmission rates even after controlling for hospital adherence to evidence-based practice guidelines.

		Patient-centered information can have an important role in the evaluation and management of hospital performance.

	Hospital readmission rates are an important measure of the quality and costs of healthcare. Recent estimates suggest that almost one-fifth of Medicare beneficiaries discharged from a hospital are readmitted within 30 days, resulting in an estimated annual cost of unplanned readmissions of $17.4 billion.

 Although factors outside of the hospital contribute to unplanned readmissions,

the fact that one-quarter of readmissions occur within 30 days of discharge suggests that there is room for improvement in the quality of inpatient care and discharge planning. Therefore, understanding the factors associated with hospital readmission has important implications for managing the provision of healthcare.

	Until recently, the measurement of hospital quality has focused on how often the hospital delivers evidence-based clinical care. In June 2009, Medicare released the Hospital Care Quality Information from the Consumer Perspective (HCAHPS), a large database of information on patients’ perceptions of their hospital experiences and, in particular, their interactions with the hospital’s staff.5,6 It is unknown whether patients can “sense” from these interactions and experiences if they are getting highquality care even if they do not have deep medical knowledge about the proper courses of treatment. Even if they can form beliefs about the appropriateness of the treatments, it is unclear whether their responses to the HCAHPS capture these beliefs. It is also unclear whether these patient satisfaction data provide information about the overall quality of inpatient care beyond that obtained from commonly accepted clinical performance measures that also are used to assess the quality of a hospital’s care.

	We sought to address these questions by studying hospital-level patient perceptions of their inpatient care and discharge planning at approximately 2500 hospitals in the United States for which we also have clinical performance measures and 30-day readmission rates for the following 3 clinical areas within the hospital: acute myocardial infarction, heart  failure, and pneumonia. Specifically, we sought to determine whether hospitals where patients reported higher satisfaction with inpatient care and discharge planning were more likely to have lower 30-day readmission rates for these 3 clinical areas after adjustment for hospital clinical performance.

	Our goal was to obtain measures of each hospital’s quality of care, as well as good indicators of the hospital’s objective clinical performance and patients’ perceptions of this performance. To do this, we used 2 major data sources.

	The first major data source was the June 2009 release of the Hospital Compare database by the US Department of Health and Human Services.

 It contained a 3-year aggregated mean of a hospital’s 30-day risk-standardized readmission rates for 3 clinical areas (acute myocardial infarction, heart failure, and pneumonia) for the period July 2005 through June 2008. We also used this data source to obtain the annualclinical process-of-care performance for the same 3 clinical areas for the same 3 years. We then combined these 3 years of data to form a 3-year mean for the same period for each hospital for each of the 3 clinical areas. We used the readmission rates to measure the hospital’s quality of care and the clinical process-of-care data to measure the hospital’s objective clinical performance.

	The second major data source was the HCAHPS patient satisfaction survey for the period July 2007 through June 2008. We used this data source to measure patients’ perceptions of a hospital’s clinical performance. Patients included in the satisfaction survey were 18 years or older, stayed at least 1 night in the hospital, and had a nonpsychiatric diagnosis at discharge. The surveys covered admissions for medical and surgical care and were initiated between 48 hours and 42 days after discharge. Hospital-level means were adjusted by the Centers for Medicare & Medicaid Services to account for factors known to affect patient responses. These factors include the mode of survey delivery (eg, mail vs phone), patient mix (eg, self-reported health and time between discharge and survey completion), and nonresponse percentages.

	These data were supplemented by data on hospital structural characteristics. These were obtained from the database of the American Hospital Association.

	It should be noted that these data sources do not allow us to link individual patients to the objective clinical performance or their readmission. Instead, these should be viewed as fallible measures of a hospital’s objective quality of care (ie, readmission rates) and the performance of in-hospital care provided to the hospital’s patients in general (ie, process-of-care and patient satisfaction scores).

	We identified 4469 hospitals that reported 30-day risk-standardized readmission rates, 4488 hospitals that collected clinical performance measures, 3746 hospitals that collected HCAHPS surveys, and 6338 hospitals in the American Hospital Association database. Using the hospital as the unit of analysis for a given clinical area (eg, acute myocardial infarction, heart failure, pneumonia), we included all hospitals that had complete information for readmission rates, clinical performance measures, patient satisfaction scores, and American Hospital Association hospital structural characteristics. This process resulted in a sample of 1798 hospitals for acute myocardial infarction, 2561 hospitals for heart failure, and 2562 hospitals for pneumonia. The clinical performance data were based on 430,982 patients with acute myocardial infarction (mean, 240 per hospital); 1,029,578 patients with heart failure (mean, 402 per hospital); and 912,522 patients with pneumonia (mean, 356 per hospital).

	There were 18 clinical performance measures in the 3 clinical categories (7 for acute myocardial infarction, 4 for heart failure, and 7 for pneumonia). Using the composite scoring method by the Centers for Medicare & Medicaid Services, we calculated hospital-level scores for each clinical category by dividing the number of times the procedures in a category were followed by the total number of eligible times associated with those measures.

	The HCAHPS database contains patient assessments of 10 dimensions of patient care derived from 18 of 22 individual survey questions. Most of the 10 dimensions of patient care were highly correlated. Based on prior work on customer satisfaction, we used 2 hospital-specific questions (“How do you rate the hospital overall?” and “Would you recommend the hospital to friends and family?”) to assess patients’ overall satisfaction with their hospital experience.

 We postulated that this overall patient satisfaction measure would be an excellent (albeit fallible) measure of patients’ observations of the performance of the hospital’s staff and would be an important predictor of readmission rates. Note that such patient observations do not require literacy in medicine but only an ability to know if the service provider “cares” and shows some  concern. We also postulated that patient satisfaction with a hospital’s discharge process would be a good indicator of the hospital’s adherence to good discharge policies and predict readmission rates for each of the clinical areas. We  captured these perceptions using the following 2 questions from the HCAHPS: “During this hospital stay, did doctors, nurses or other hospital staff talk with you about whether you would have the help you needed when you left the hospital?” and “During this hospital stay, did you get information in writing about what symptoms or health problems to look out for after you left the hospital?”

	We transformed the HCAHPS information on each hospital into overall satisfaction and discharge satisfaction scores as follows. The HCAHPS database reported the total number of patients surveyed and the percentage of patients who responded to the different levels of the particular question. For the 2 overall satisfaction questions, the database provided 3 levels (ie, a satisfaction rating of 1-6 [low], 7-8 [medium], or 9-10 [high]). We multiplied the percentage of patients who responded to a given level by the numerical values of 0, 0.5, and 1 for low, medium, and high, respectively, to obtain scores between 0 and 1, where 1 indicates that all patients gave a high response and 0 indicates that all patients gave a low response to the particular question. The hospital-level overall patient satisfaction score is the mean of these 2 numerical values. For the 2 discharge questions, we converted the  reported percentages into numerical values by assigning the percentage of “no” responses the value of 0 and the percentage of “yes” responses the value of 1 and averaging the 2 questions across respondents. Note that the Hospital Compare documentation does not provide patient satisfaction information for specific diagnosis related groups but instead reflects patient responses for several other units, as well as the 3 units we analyze. Therefore, the patient satisfaction scores used for analyzing readmission rates for acute myocardial infarction, heart failure, and pneumonia are the same for a given hospital.

	The hospital-level 30-day risk-standardized readmission rates and sample sizes were obtained directly from the Hospital Compare database, and our measures of hospital  structural characteristics came directly from the American Hospital Association database. These measures included the number of beds, medical school affiliation, geographic region, and the presence of adult interventional cardiac catheterization facility, medical, and surgical intensive care units.

	Our primary objectives were to determine the association of hospital-level 30-day risk-standardized readmission rates with (1) hospital-level clinical performance as measured by the guideline adherence score in each clinical area and (2) hospital-level overall perception among patients of their hospital stay and interactions with the hospital staff and their view of the hospital’s discharge process. We performed 3 separate logistic regression analyses in which the dependent measures were based on the risk-standardized hospital readmission rates for each of the 3 clinical areas.13 Specifically, we converted the readmission rates to 1 or 0 to reflect whether patients were readmitted. Therefore, positive coefficients indicate higher readmission rates. The unit of analysis was the hospital; therefore, hospitals with more patients were weighted more heavily. The independent variables were hospital-level clinical performance, overall patient satisfaction, and patient satisfaction with discharge planning. We also included hospital structural characteristics to control for fixed effects that might influence the outcome measures.

	To help inform the policy implications of the results, we performed sensitivity analyses to determine the change in predicted risk-standardized 30-day readmission rates associated with a change in hospital score from the 25th percentile to the 75th percentile for the overall patient satisfaction score and for the patient satisfaction with discharge planning score, while keeping the hospital-level clinical composite score fixed. Conversely, we also examined the effect of the same interquartile change in hospital-level clinical composite score, while keeping the patient satisfaction measures fixed.

	Finally, we calculated pairwise Pearson product moment correlation coefficients between the overall patient satisfaction score and the 8 other HCAHPS-reported dimensions of quality. This was to assess which dimensions were most associated with the patients’ overall satisfaction with the hospital’s quality of care.

	We used JMP version 7.0.2 (SAS Institute Inc, Cary, North Carolina) for all statistical analyses. P <.05 was considered statistically significant.

 gives the characteristics of the study hospitals. Although hospitals in the sample tended to be larger and better resourced than hospitals in the total sample of American   Hospital Association acute care hospitals, the 3 samples represent a broad cross-section of US hospitals. 

 gives the distributions of the variables of interest, including the scores for overall patient satisfaction and patient satisfaction with discharge planning, the clinical composite score, and 30-day risk-standardized readmission rates. There was considerable variability in patient-reported measures and clinical measures across hospitals. Note that the mean 30-day risk-standardized readmission rates are approximately 20% for all 3 clinical areas.

gives the correlations among the variables. The 2 hospital-level patient-reported measures were not highly correlated with the hospitals’ clinical performance measures.  Overall patient satisfaction and patient satisfaction with discharge planning were negatively and significantly correlated with higher 30-day risk-standardized readmission rates for all 3 clinical conditions. In addition, all 3 clinical composite scores were negatively and significantly correlated with higher 30-day risk-standardized readmission rates, although these correlations are smaller than those associated with the patient satisfaction scores.

 gives the results of the multivariable logistic regression analyses for the variables of interest. All 3 clinical performance measures were negatively associated with higher 30-day risk-standardized readmission rates, although the acute myocardial infarction and heart failure measures were not statistically significant (P = .16 and P = .06, respectively).  Higher overall patient satisfaction scores also were associated with lower 30-day risk-standardized readmission rates for all 3 clinical conditions. In this case, all 3 measures were highly statistically significant (P <.001). Finally, scores for patient satisfaction with discharge planning were associated with lower 30-day risk-standardized readmission rates for all 3 clinical areas and were statistically significant for heart failure and for pneumonia (P <.001 and P = .02, respectively).

shows that the odds of 30-day risk-standardized readmission were associated with interquartile improvements in hospitals’ patient total satisfaction scores (ie, overall patient satisfaction and patient satisfaction with discharge planning), while holding the clinical composite scores fixed, and vice versa. Interquartile improvements in patient total satisfaction scores were associated with significantly lower predicted 30-day risk-standardized readmission rates for acute myocardial infarction (odds ratio [OR] = 0.97; 95% confidence interval [CI], 0.94-0.99), heart failure (OR = 0.96; 95% CI, 0.95-0.97), and pneumonia (OR = 0.97; 95% CI, 0.96-0.99). Also shown are the interquartile improvements in the 3 clinical performance measures. The improvements in 30-day risk-standardized readmission rates associated with interquartile improvements in the patient total satisfaction scores for heart failure, acute myocardial infarction, and pneumonia were 4.9, 2.2, and 1.6 times higher, respectively, than those associated with interquartile improvements in the same 3 clinical composite scores.

gives the correlations of the overall patient satisfaction measure with each HCAHPS question category. Quality of communication by nurses had the strongest correlation with overall patient satisfaction, followed by several other measures that capture the patient’s interaction with the hospital staff. Patient satisfaction with discharge planning was seventh of the 8 questions in terms of correlation, indicating that it captured a different dimension from that captured by overall patient satisfaction. Also low in terms of correlation with overall patient satisfaction were the 2 questions concerning the hospital facilities (ie, cleanliness and noise level), again highlighting that overall patient satisfaction seems to be capturing the patients’ interactions with the hospital staff.

	A substantial proportion of Medicare beneficiaries experience an unplanned hospital readmission within 30 days of discharge. In this study, we found that patients’ stated overall satisfaction score and their perception of the hospital’s discharge process were significantly and negatively correlated with the hospital’s 30-day readmission rates in the 3 clinical areas studied. Moreover, these 2 patient-related measures were more predictive than the objective clinical performance measures often used to assess the quality of hospital care.  Although the key drivers of hospital readmission are complex, our findings suggest that patients’ perspectives on inpatient care and discharge planning provide important insights  into hospital performance with respect to quality. Moreover, because the overall satisfaction score is most highly correlated with factors associated with the patients’ interaction with  the hospital staff, our findings are consistent with the observation by the Institute of Medicine that high-quality care is “patient centered” and responsive to patients’ preferences,  eeds, and values.

 More generally, given the association between these patient perceptions and better outcomes, our findings suggest that patient-centered information can be  used to assess the degree to which patients will be more likely to experience better health outcomes, at least as measured by hospital readmission rates.

Higher overall patient satisfaction with inpatient care and discharge planning is associated with lower 30-day readmission rates after adjusting for clinical quality.

Relationship Between Patient Satisfaction With Inpatient Care and Hospital Readmission Within 30 Days

Pay-for-performance (P4P) programs have the potential to improve overall quality of care by narrowing gaps between what national care guidelines recommend and those treatments actually delivered in routine community practice. P4P is also viewed as a tool to promote more efficient use of healthcare resources while improving patient outcomes. P4P provides financial incentives for quality of service instead of quantity of service. Despite the promise of healthcare quality, concerns have been raised that P4P may have potential unintended consequences for patients, physicians, and hospitals. The shortcomings of many traditional P4P programs have fueled the emergence of new and innovative models of payment reform. P4P and newer models that link reimbursement with quality and efficiency show promise to improve patient outcomes and lower costs, but multiple approaches are needed to ensure that future initiatives provide value for key stakeholders, including patients, providers, and payers.

The concept of linking financial incentives to the quality of healthcare provided has been termed 

, or P4P. P4P has received significant national attention as a potential means of improving overall quality of care by narrowing gaps between what national care guidelines recommend and those treatments actually delivered in routine community practice.

 In an era of rapidly rising healthcare costs, P4P is also viewed as a tool to promote more efficient use of healthcare resources while improving patient outcomes. Over the past decade, commercial and federal payers have implemented a vast array of P4P initiatives. The structure of these programs is remarkably diverse, spanning a variety of payment models, intended targets (eg, hospitals, physician groups, individual providers), and clinical conditions. While a number of programs have shown promise, additional work is necessary to determine whether they achieve their intended long-term effects. The purpose of the article is to provide a brief overview of current P4P initiatives, discuss the evidence regarding their effectiveness, and provide insight into newer, innovative payment models that have emerged.

More than half of commercial health plans in the United States currently use P4P incentives in their provider contracts.

 Many of these programs involve joint efforts among employers, health management organizations, pharmaceutical companies, physician groups, academia, as well as for-profit and not-for-profit organizations (

 Complementing efforts by the private sector, the Centers for Medicare & Medicaid Services (CMS) has sponsored P4P demonstration projects in a variety of clinical settings, including physician practices, acute care hospitals, dialysis facilities, nursing homes, as well as programs to increase the adoption of information technology and disease management. The largest demonstration project to date is the Hospital Quality Improvement Demonstration (HQID) Project, which offers financial incentives to hospitals based on the inpatient quality of care for 5 clinical conditions-acute myocardial infarction, heart failure, pneumonia, coronary artery bypass surgery, and hip and knee replacement. In addition to these US programs, P4P has gained significant traction overseas.

 For example, the National Health Service (NHS) in the United Kingdom has invested massive resources in P4P initiatives. The NHS's Quality and Outcomes Framework, which provides financial incentives to primary care physicians for 146 quality indicators related to chronic disease and patient experience, has distributed over £2 billion to providers since 2004.

Compensation models that link financial incentives to performance have been widely implemented in other industries and are a powerful lever to influence behavior. P4P is seen as a way to create a "business case" for quality by better aligning payment with quality of service instead of quantity of service.

 This helps address the issue that hospitals and physicians are not necessarily rewarded for delivering high-quality care. For example, hospitals that produce better health outcomes may paradoxically face lower margins through phenomena such as diagnosis-related group switching and a reduction in unplanned rehospitalizations for the chronically ill.

 P4P also holds promise because many of the traditional approaches to improving quality, such as physician education, provider certification, and consumerism, have failed, largely due to the fragmented nature of our healthcare delivery system.

 Linking financial incentives to quality is also viewed by many as a more palatable approach than traditional managed care models where financial incentives are provided to physicians to limit referrals and see more patients per day.

Despite the promise of healthcare quality, concerns have been raised that P4P may have potential unintended consequences for patients, physicians, and hospitals.

 For example, most P4P programs reward providers based on evaluation of a limited number of process performance metrics. If hospitals and physicians become too focused on these metrics, they may lose sight of the global goals of healthcare (analogous to students studying just what is on the test).

 Some have argued that forcing hospitals and providers to follow select process patterns could stifle innovation and the ability or willingness of organizations to develop creative solutions to improving quality.

 Others have worried that the large fixed costs required to support P4P data collection and quality improvement programs could deviate important resources away from patient care and have unintended consequences.

 In a similar manner, the financial incentives in P4P could paradoxically exacerbate healthcare disparities-either by financially penalizing hospitals that treat underserved populations or by prompting caregivers to avoid sick and "high-risk" patients from their practice.

Evidence Regarding the Effectiveness of P4P

While many have speculated on both the potential positives and negatives of P4P programs, the evidence regarding its impacts has been mixed.

 A systematic review conducted in 2006 identified 17 studies on P4P published between 1980 and 2005.

 These studies focused primarily on programs targeting preventive care services. The studies tended to be inconclusive due in part to their small sample size, specialized setting, and short-term follow-up. More recently, however, there have been investigations on the impact of large P4P initiatives sponsored by the US federal government and the United Kingdom. These studies have generally found that P4P programs were associated with modest improvements in process of care measures, yet none of these programs had an impact on patient outcomes or efficiency of care. For example, Lindenauer et al found that hospitals engaged in both public reporting and CMS' HQID P4P program achieved modestly greater improvements in quality than did hospitals engaged only in public reporting.

 The estimated incremental effect on a composite quality score for cardiac and pneumonia care ranged from approximately 2.8% to 4.3%. Glickman et al evaluated CMS' HQID program in a cohort of 500 hospitals participating in a voluntary quality improvement initiative for acute myocardial infarction and found that financial incentives were not associated with a significant incremental improvement in quality or reduction in mortality.

 Evidence from the large P4P program in the United Kingdom for asthma, diabetes, and coronary heart disease suggests that family practitioners achieved high levels of achievement in the first year of the program.

 Over the ensuing 3 years, however, improvements in the quality of care slowed and actually declined for conditions not linked to financial incentives.

 Unfortunately, few large-scale, randomized controlled trials have evaluated the effectiveness of P4P interventions.

Issues Involved in the Methodology for P4P Measurement and Provider Ratings

A key challenge to implementing P4P programs is selecting valid and reliable measures of quality and performance. The largest P4P programs to date have focused primarily on processes of care (ie, adherence to evidence-based treatment guidelines). For example, the 2009 CMS Physician's Quality Reporting Initiative is tracking 153 quality measures spanning multiple therapeutic areas.

 A number of professional organizations, including the National Quality Forum, the American Heart Association, and the American College of Cardiology developed consensus methodology for the selection and creation of performance measures.

 Performance measures must be valid (eg, have a robust evidence base supporting their use), accurate and reliable, easy to interpret, and allow for reliable comparisons among providers. Other important considerations are the clinical relevance of the outcome, adherence to the process measure, and variability in baseline adherence to the measure (ie, is there already a ceiling effect in performance?).

Despite the popularity of using process measures to gauge quality, such measures present significant challenges. Selecting performance measures that do not meet evidence-based criteria may negate their intended effects of improved efficiency and patient outcomes.

 For example, a recent study of quality measures for acute myocardial infarction in 1351 hospitals found that in a resource-constrained environment, hospitals that focus on "administrative" process measures (eg, smoking cessation counseling or discharge instructions) at the expense of clinical interventions (eg, aspirin or angioplasty) have worse patient outcomes.

 Another issue is the possibility of obtaining stable estimates of performance for small hospitals and physician groups or whether additional techniques are needed to account for small and unequal denominators. A recent study by O'Brien et al demonstrated that high-volume hospitals had better performance on average, but were significantly less likely to be identified as "top" hospitals (ie, top decile).

There are also challenges in finding ways to combine performance measures to create a valid aggregate measure of hospital or physician performance. Composite scores that combine several performance measures into a single ranking are commonly used to assess hospital performance. Yet, existing methods used to create composite scores are highly variable in their weighting of process versus outcomes metrics, which can, in turn, lead to highly divergent provider rankings.

 Composite scores are typically weighted by the total number of treatment opportunities, although evidence suggests that weighting based on how hospitals organize care or the range for possible improvement in scores may provide more useful information.

 Finally, although performance measures are not typically risk-adjusted, a recent study by Mehta et al found that accounting for hospital case mix (such as age, sex, race, insurance status, and medical comorbidities) can dramatically affect a hospital's ranking and financial benefits in P4P programs.

In addition to process measures, P4P programs can also reward patient outcomes (eg, inpatient mortality, 30-day mortality, 30-day hospital readmission), patient perceptions of care, measures of efficiency (eg, ratio of observed to expected costs), and hospital and physician group structural characteristics (eg, adoption of information technology). Outcomes performance assessment, however, can be challenging due to the potential confounding influence of patient case mix on provider outcomes, as well as the instability of hospital outcome estimates due to low event rates.

New Care Models With Financial Incentives

The shortcomings of many traditional P4P programs, including their limited impact on provider behavior, value, and health outcomes, have fueled the emergence of new and innovative models of payment reform.

 Primary care capitation models that include large performance and efficiency bonuses are currently being piloted in Massachusetts. Other groups are moving toward episode-based payments, which reimburse providers on the basis of expected costs for clinically defined episodes of care. The Geisinger Health System recently implemented a bundled payment for coronary artery bypass grafting (ProvenCare) that covers all care 30 days before and 90 days after the procedure, including any complications, hospital readmissions, and follow-up care.

 This model has been associated with a significant reduction in hospital readmissions and hospital charges, and has been expanded to a number of other clinical conditions, including angioplasty and hip replacement surgery. In California, P4P programs have shifted toward efficiency measurements using bundled payment for acute surgical and medical interventions.

One of the most promising payment models to emerge is the accountable care organization (ACO).

 This concept is rooted in the observation that there are dramatic differences in Medicare spending, by both region and hospital. Regions with lower per beneficiary spending in Medicare achieve equal or better quality and health outcomes than their counterparts. The goal of the ACO model is to create organizational accountability for quality, improve coordination of care for Medicare beneficiaries, and reward innovations that simultaneously improve healthcare quality and reduce costs.

In 2005, Medicare implemented the Physician Group Practice (PGP) demonstration program in large group practices that serve at least 5000 Medicare patients. The program allows group practices to share the cost savings they achieve in caring for their patients if they simultaneously meet quality improvement targets. The cost savings are calculated by comparing actual spending to a target. The target is determined by the PGP's base year per capita expenditures trended forwarded by a per capita growth rate from the same area. Medicare savings in excess of 2% are shared with CMS depending on whether process performance targets are achieved. In the first 2 years of the program, there was overall improvement in quality-of-care targets and lower risk-adjusted expenditure growth rates for several of the participating groups. A recent study demonstrated that more than 75% of Medicare beneficiaries nationally receive care in ACO-eligible networks (>5000 beneficiaries) and that widespread adoption of ACOs would lead to considerable savings in Medicare spending.

 Yet, there are important challenges to widespread adoption of ACOs. These include fragmentation of provider efforts by competing P4P programs, applicability of ACOs to patients living in rural and underserved areas, and concerns that a primary focus in efficiency may undermine the credibility of these programs among physicians and other stakeholders.

Another promising option is to provide financial incentives for patients to modify their behavior.

 Targeting unhealthy patient behaviors such as smoking, poor diet, and physical inactivity may yield important public health benefits and reduce overall healthcare costs. This is because unhealthy patient behaviors may account for as much as 40% of premature deaths in the United   States, whereas inefficiencies in healthcare delivery account for only 10%. In a recent study of 878 employees at a large company, the group that received a $750 financial incentive for smoking cessation had a significantly high rate of smoking cessation up to 18 months after enrollment compared with those in the group who received no incentives (9.4% vs 3.6%, respectively).

 P4P for patients merits additional study, and future programs will need to draw on insights from behavioral economics in order to optimize the structure of these programs.

P4P and newer models of payment reform hold tremendous

promise to improve healthcare quality and reduce costs.

Multiple approaches are needed to ensure that future initiatives

provide value for key stakeholders, including patients,

providers, and payers. In general, the goal should be to foster

innovative approaches to improving quality patient outcomes,

promote accountability for quality on the part of both

patients and providers, and improve the value of services purchased

It is important to continue to build consensus about the selection of evidence-based performance measures, including measures that reward technical aspects of care, outcomes, efficiency, and patient-centered care. Given the traditionally long lag time between the translation of new evidence into clinical practice, it will be critical to find ways to rapidly incorporate validated process measures into everyday use. Greater focus on outcome and efficiency measures appears warranted. Further refinement of risk-adjustment methods is needed to standardize outcome and cost-efficiency metrics and allow for valid benchmarking across providers.

The creation of a more compelling "business case" and value proposition for quality on the part of key stakeholders is essential to creating sustainable quality improvement efforts.

 Adoption of well-designed patient outcome and efficiency measures would encourage providers and hospitals to develop innovative solutions for improving value. In contrast, promulgation of too many measures that focus on technical aspects of care may distract providers and lead to fragmented care. As we transition toward greater use of outcome and efficiency measures, additional investments in comparative effectiveness and health services research will be needed to help payers and providers identify and implement higher-quality, more cost-effective treatment approaches. These efforts would be facilitated by greater investments in large quality improvement registries and patient information systems.

P4P programs will need to be sensitive to hospitals and provider groups that care for vulnerable patient populations, including the underinsured, racial and ethnic minorities, and patients living in rural areas. In addition, further efforts are needed to better understand hospital structural characteristics that facilitate high-quality healthcare delivery, including the role of information technology, management, culture, and organizational structure.

P4P and newer models that link reimbursement with quality and efficiency show promise to improve patient outcomes and lower costs. Future success hinges on collaboration among key stakeholders including patients, physicians, payers, and policymakers.

From the Duke Clinical Research Institute and the Department of Medicine, Duke University Medical Center, Durham, NC (SWG, EDP), and the Department of Emergency Medicine, University of North Carolina School of Medicine, Chapel Hill (SWG).

Financial support for this work was provided by GlaxoSmithKline.

Seth W. Glickman, MD, MBA

Articles

The Mis-Measure of Physician Performance

Relationship Between Patient Satisfaction With Inpatient Care and Hospital Readmission Within 30 Days

Innovative Health Reform Models: Pay-for-Performance Initiatives