Denise D. Quigley, PhD

 To examine how primary care practices use the Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey and its patient-centered medical home (PCMH) items during their PCMH transition.

 Qualitative study of practices’ use of patient experience data during PCMH transformation, based on a random sample of primary care practices engaged in PCMH transformation, stratified by region, practice size, PCMH recognition history, and use of the CAHPS PCMH survey.

 We interviewed 105 practice leaders from 294 sampled practices (36% response rate) and used content analysis to identify themes about uses of patient experience data for practice improvement during PCMH transformation.

 Patient experience data were used primarily to assess and track improvements toward PCMH goals and as quality improvement (QI) metrics. CAHPS measures were used most often to discuss best practices, share data with patient advisory councils, and improve provider performance. The CAHPS PCMH survey helped practices improve patient-centeredness, particularly in coordinating care and supporting patient self-management and communication. The CAHPS PCMH items that assisted practices most were about patient self-management and provider referrals. Most practice leaders using the CAHPS PCMH survey felt that its items were actionable for standardizing PCMH changes or making structural changes. Practices administering CAHPS surveys focused on a more diverse set of QI areas.

 CAHPS surveys were considered actionable for PCMH transformation and used in standardizing and coordinating care. The CAHPS PCMH items were considered integral to the continuous QI needed for moving beyond formal PCMH recognition and maximizing transformation. This supports the National Committee for Quality Assurance’s recommendation to administer the CAHPS or CAHPS PCMH survey for PCMH transformation.

Prior research has assessed how providers undergoing patient-centered medical home (PCMH) transformation use patient experience data to improve care experiences and make care more patient centered, but little research has examined the use of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) PCMH survey.

The CAHPS PCMH survey was used in standardizing and coordinating care.

Practices used patient experience data most often to assess improvements in patient experience related to PCMH goals; CAHPS measures were used to discuss best practices, share data with patient advisory councils, and improve provider performance.

Practices administering the CAHPS PCMH survey focused on a diverse set of patient experience improvement areas and found CAHPS data actionable.

The patient-centered medical home (PCMH) may help reduce the fragmentation and inefficiency of the US health care system.

 The PCMH model focuses on team-based care to address patient needs and improve care experiences and outcomes while reducing costs.

Although several organizations provide PCMH recognition,

 more than 13,000 US primary care practices, or 15% to 18% of all primary care practices, are recognized as PCMHs under the medical home standards of the National Committee for Quality Assurance (NCQA).

 PCMH implementation requires changes to multiple aspects of primary care practice.

 and requires resources from leaders and staff.

 Adopting the PCMH model entails a fundamental shift in orientation and culture.

NCQA included performance and quality improvement (QI) measurements as part of its sixth standard to drive more comprehensive measurement and use of patient experience data. Patient experience has been assessed using the Consumer Assessment of Healthcare Providers and Systems Clinician and Group (CG-CAHPS) survey supplemented with PCMH items—that is, the CAHPS PCMH survey. This survey includes questions assessing specific aspects of PCMH delivery such as access to care and self-management support.

Research has assessed how health care providers and systems use patient experience survey data to improve patient care experiences

 and make care more patient centered. Studies have documented that organizations and health care leaders encounter challenges in driving change. Case studies have shown how large systems use CAHPS data primarily for QI and have focused PCMH transformation on care delivery.

However, there has been little research examining how practice leaders use CAHPS and CAHPS PCMH survey data to improve patient experiences and support PCMH transformation. This paper examines the PCMH transformation experiences of a nationwide sample of 105 primary care practices. We investigate how practices administering different patient experience surveys used the resulting data for PCMH transformation and related QI efforts.

We created a stratified random sample of 294 practices that had applied for NCQA PCMH recognition, and 105 participated (36% response rate).

 Participating sites differed significantly from other eligible NCQA sites on use of CAHPS surveys, PCMH Level 3 history, number of physicians, and state, but nonresponse weights increased noise without reducing differences.

We obtained consent and collected information on practices in hour-long phone interviews from June 2017 to June 2018 with practice leaders knowledgeable about the practice’s PCMH history and patient experience data. We discussed the practice’s PCMH history, PCMH transformation decisions, and change efforts. We asked how they selected and used patient experience data. We recorded and transcribed the interviews and provided a $75 honorarium to each participant.

We entered transcripts into Dedoose (SocioCultural Research Consultants), a web application for analyzing qualitative data. We established structural codes that mapped to the research questions.

 We developed a code structure and codebook using systematic, inductive procedures

 and content analysis to develop emerging themes.

 We coded early transcripts independently, noting topics related to the research questions.

Our coding team used meetings to reach consensus on topics, identify discrepancies, refine concepts, and define and add codes.

 Coders resolved discrepancies through interrater reliability exercises and obtained a pooled κ coefficient of 0.93, indicating “very good” coding agreement.

 We employed ongoing training among the coding team on emerging subcodes using the Dedoose training module.

Study protocols were approved by RAND’s Human Subjects Protection Committee (IRB Assurance No. FWA00003425; IRB No. IRB00000051) and the Office of Management and Budget (No. 0935-0236).

The 105 practices were distributed throughout the United States and had a range of PCMH history (

). The pattern of practices’ PCMH history suggests that practices adopt the CAHPS PCMH survey during the critical transformation period that leads to the practice receiving PCMH Level 3 certification, and then practices drop the CAHPS PCMH items as their PCMH transformation status matures.

Patient Experience Survey Characteristics

Overall, 31% administered the CAHPS PCMH survey, 29% administered a “homegrown” survey, 22% administered another standardized survey, and 17% administered the core CG-CAHPS survey. 

 provides aggregated detail on practices’ patient experience surveys and the 

) shows further refined information about these surveys.

Of the 30 practices administering a homegrown survey, 13 (43%) included CAHPS items, either “exactly plagiarizing CAHPS” (5 practices) or by adapting them (8 practices). Eight of all 105 practices (8%) reported administering the CAHPS PCMH survey and other surveys: Three fielded a Medicaid survey, 3 fielded an additional homegrown survey (eg, focused on visit type or procedure), and 2 fielded an additional standardized survey (eg, BluePrint survey).

Four main themes emerged on how practices used patient experience data for PCMH transformation.

Theme 1: Practices used patient experience data most often to assess improvements related to PCMH goals; CAHPS measures were used to discuss best practices, share data with patient advisory councils, and improve provider performance.

The most common uses of patient experience data across all practices were monitoring and assessing changes in patient experience related to the PCMH change process (90%). This aligns with the PCMH standard for “performance measurement and quality improvement.”

 This was similar for practices currently, previously, or never administering the CAHPS PCMH survey. One leader described how they share their data:

We monitor trending reports on patient experience quarterly, comparing overall CAHPS scores and picking out particular items that we really want to focus on improving.…[T]hose CAHPS scores are disseminated across the entire practice and discussed at monthly staff meetings.…We also share them with our QI collaborative, which has representatives from all our primary care clinics, specialty clinics, and from throughout the hospital.…[That] is a nice opportunity for people [who] are not right in the thick of it to have a look at CAHPS data and make suggestions.…We also share these data at the clinic level.…[W]e have an expectation that patient experience is always on the agenda. – Current_1071

Another leader noted using CAHPS to start QI efforts:

We monitor the information on the CAHPS PCMH survey including the comments. The first place I send the data is to the CEO.…Then [they go] to the monthly [quality assurance]/QI committee where we compare [them with data from] previous surveys.…CAHPS data help us get things set up by asking, “What do you identify from the trends and what do we need to work on?” Then we always pick several areas we desire to increase and start new QI efforts. We do this for at least 3 or 4 findings from the CAHPS PCMH, as these data really jumpstart our QI process and QI changes touch every department. – Current_18415

 lists quotes on how practices use patient experience data to meet PCMH standards. 

 lists uses of patient experience data, including sharing performance trends with patient advisory councils (55%) and leadership (20%). These uses vary by CAHPS PCMH survey administration. Practices currently using the CAHPS PCMH survey were most likely to use them for monitoring and QI but also to discuss best practices with staff or to improve provider performance. Practices that previously used the CAHPS PCMH survey were more likely than others to track performance.

Theme 2: Practices used patient experience data for QI.

All practices reported using patient experience data for QI, PCMH changes, and tracking their progress. Most often, practices—including 90% of those previously using the CAHPS PCMH survey and 67% of those currently or never using it—used a general QI methodology to implement and track changes over time. Twenty percent also reported using a specific QI method such as Plan-Do-Study-Act (PDSA) cycles, root-cause analysis, Pareto analysis, or Lean methodologies for implementing PCMH. One leader reported:

Every practice gets a monthly CAHPS report including responses to each question. Every site has a multidisciplinary site-level QI team that works on specific quality measures for NCQA, and CAHPS metrics are the measures they work on improving. So every month, they’d see the CAHPS data and talk about [them], come up with an improvement plan, and roll the plan out in their practice and track future trend data…using a PDSA cycle approach. – Previous_4139

Another leader noted using patient narrative comments to assess whether practice changes improved care:

We’ve been monitoring our newly implemented PCMH processes for wait times…adjusting and tweaking processes as we need to as part of QI. We look at our patient experience data regularly, also looking at any comments to see if we’re still getting the same written comments from patients. – Never_11421

All practices reported using their patient experience data extensively for QI during PCMH transformation.

Theme 3: Practices had various reasons for choosing their patient experience survey.

Most practices (81%) reported that leadership, rather than those in charge of implementing PCMH changes, selected their patient experience survey. Twenty percent of practice leaders also reported that their practice chose their patient experience survey to meet the quality measurement PCMH standard. This was most common in practices that previously implemented the CAHPS PCMH survey and for practices that never had administered the CAHPS PCMH survey but administered their own homegrown surveys.

Practices that never implemented or previously implemented the CAHPS PCMH survey were more likely to report using the core CAHPS survey before PCMH implementation. Practices currently administering the CAHPS PCMH survey chose to pursue PCMH in tandem with CAHPS PCMH survey administration rather than independently.

More than 70% of practices chose to change their patient experience survey during their PCMH transformation. Practices that previously administered the CAHPS PCMH survey were the most likely to change which survey they used. Many practices that previously implemented the CAHPS PCMH survey were part of programs or initiatives that determined which survey they would use. Once those programs ended, or once those programs required practices to pay for survey administration costs, these practices chose to drop the CAHPS PCMH survey or switch to a new survey altogether. One leader said:

We do our own surveys. We did do the CAHPS PCMH when we were still within the [state program], but we graduated from that group.…They were the ones to actually send those CAHPS surveys out.…[Now] we have our own patient surveys that we give to our patients, usually twice a year, with obviously different questions to see how we’re doing; some of those are similar to the CAHPS questions anyway. – Previous_4170

Many practices that currently administer the CAHPS PCMH survey reported that they chose to also administer a second patient experience survey. The most common reason for administering a second survey was the desire to obtain more frequent data to use in implementing and monitoring PCMH changes. Some practices also added a survey to target specific aspects of patient experience, such as probing more closely on wait times or laboratory test turnaround. One leader discussed administering a supplemental patient survey:

CAHPS PCMH is only done once a year. Our [supplemental survey] is done at every visit or procedure for all patients.…It’s more comprehensive, more timely, and provides data on a monthly basis. – Current_4064

Practices that were responsible for paying survey administration costs noted a desire to change from the CG-CAHPS or CAHPS PCMH survey because of the high cost of survey administration. Practices for which a state program or network paid survey administration costs did not report issues with the cost of administering CAHPS surveys. One respondent summarized the balance between costs and quality by noting:

We have done an internal patient satisfaction survey for 22 years. It’s fairly robust, but it wasn’t evidence-based or vetted through random-controlled studies, so we started Hospital CAHPS in the hospital and had our care coordination department participating in that. We actually hired a company to administer the HCAHPS survey and we liked that very much. Unfortunately, it was costing us $20,000 to $25,000 a year. – Never_4273

Theme 4: Practices administering the CAHPS PCMH survey focused on a diverse set of patient experience improvement areas and reported that the CAHPS data were actionable.

Practices currently administering the CAHPS PCMH survey focused QI efforts on provider-patient communication (30%), shared decision-making (30%), coordinating patient care (30%), and nurse-patient communication (29%) in addition to overall ratings of providers (27%) and the clinic (30%). Practices that previously used the CAHPS PCMH survey focused on 2 issues: nurse-patient communication (29%) and overall rating of providers (26%). Practices that never used the CAHPS PCMH survey focused on 4 aspects of care: provider-patient communication (29%), self-management support (24%), coordinating patient care (22%), and overall rating of the clinic (23%).

Among practices that had used CAHPS PCMH items, 88% of current users and 39% of previous users thought these items were actionable. When asked how these items were actionable, 30% said they helped standardize and formalize PCMH change processes, 18% said they helped in structural/environmental decisions, and another 15% said they were useful for monitoring specific areas of patient experience such as coordinating patient care and enhancing patient self-management support.

Forty-five percent of those currently using the CAHPS PCMH survey indicated that the PCMH-specific items helped them achieve recognition in specific PCMH standards or goals. These practices pointed to 2 specific items: “patients caring for their own health” (45% of practices currently administering the CAHPS PCMH survey) and “gaining access to care from other providers” (eg, via referrals) (30% of practices currently administering the CAHPS PCMH survey). One leader noted:

To confirm if there [are] any barriers that would keep the patient from meeting the goals that we need them to meet [to be] able to manage themselves, we specifically used the CAHPS PCMH items about taking care of [their] own health and talking about medication with the patient.…With the older population, who are on more medicines, it helps place priority on making sure they don’t have any barriers to taking medication…and making sure that they are on some type of regimen [for] exercise and diet. – Current_4356

Forty-seven percent of practices discussed how the CAHPS PCMH items added value to their PCMH transformation journey. For example, 41% of practices described how the CAHPS PCMH items helped improve provider access by highlighting issues related to scheduling, clinic hours, and questions being answered. One leader said:

One of the biggest areas of opportunity [we] found from using CAHPS PCMH was that a high percentage of our patients did not know how to receive care on the weekends or after hours.…[T]he clinic put together a QI plan to increase the awareness and really increase the percentage of patients [who] knew how to receive care after hours. We put together a PCMH brochure which was shared with all patients and was displayed in our exam rooms. As part of our Patient and Family Advisory Council, the patients decided to create a newsletter. So that served as a communication channel not just to patients on the council but to all patients here at the practice. – Current_11279

Similarly, practices reported using CAHPS PCMH items to identify a range of complex gaps in care and track progress in meeting patient needs for medication management and behavioral health integration. One provider noted:

The CAHPS PCMH medication question was fairly easy to deal with, simply because it involved our medical assistants.…They do medication reconciliations at every visit, but somehow that information wasn’t getting through to patients.…[W]e changed our scripting for how that works.…[T]here wasn’t some miraculous big change, but over time, there will be. – Current_1115

Furthermore, 11% of providers relayed how they improved their process of providing patient test results and follow-up. One provider said:

The CAHPS PCMH access question about after hours and weekends was especially useful for our practice, [as were] the medication question and the question under care coordination about patients receiving test results in a timely fashion. We talked about those test results data quite a bit and did scripting with staff.…Scripting helped give the patient a reference as to when they might know something. And that prevents the patient from undue worry [and] the staff from getting multiple calls.…When we use scripting, staff can say that X particular test is going to take 3 days, so that what we convey to the patient is specific and helpful. – Previous_4127

 by specifically examining use of patient experience data for a nationwide sample of practices. We found that practices engaged in PCMH transformation used patient experience data to standardize patient-centered care and to identify and assess improvements related to PCMH goals. The most common use of patient experience data across all practices was to monitor progress. Data were also shared widely within organizations, ranging from CEOs to advisory councils, and used for discussions about best practices with staff, providers, or other network practices. Han et al also found that one-third of PCMH practices engaged patients via advisory groups and patient experience survey feedback.

 Patient experience data were also commonly used to specifically improve provider-patient interactions (eg, 1-on-1 counseling based on CAHPS performance).

Studies specific to health systems using CAHPS data found that practice leaders used CAHPS data to implement QI, develop a shared vision, and coach providers and staff on performance.

 We extend this evidence by identifying that practices administering the CAHPS PCMH survey used these data as guideposts for providing patient-centered care and focused on a broader range of patient experience–related improvement projects than practices not using CAHPS data. The CAHPS PCMH items helped practices standardize how care is delivered, share best practices, provide care that is coordinated with other providers, and focus on supporting patients’ self-management needs. Practices also primarily used the CAHPS PCMH items during the critical transformation period that leads to the practice receiving PCMH Level 3 certification, and then many practices dropped the CAHPS PCMH items as their PCMH transformation status matured. Data from CAHPS surveys were considered actionable by practice leaders, particularly for provider-patient and nurse-patient communication, shared decision-making, coordinating patient care, and improving global ratings.

Although we studied a large, varied national set of practices, the sample is not nationally representative. We did not include practices that sought or gained PCMH certification under different recognition programs. Also, because this study is not a survey of practices’ activities using patient experience data, it does not contain an exhaustive exploration of all uses of patient experience surveys from all stakeholders.

Practice leaders most often used their patient experience data to assess QI and PCMH-related improvements. Those who administered CAHPS surveys used the data to discuss best practices, share data with leaders and patient advisory councils, and improve provider-patient interactions. Practice leaders used both CAHPS and non-CAHPS patient experience data as part of QI activities, with practices administering the CAHPS PCMH survey focused on a more diverse set of patient experience improvement areas. Importantly, CAHPS PCMH patient experience data were reported as actionable in making improvements in patient-centeredness. Specifically, practice leaders indicated that CAHPS PCMH survey data helped them improve care coordination, self-management support, communication, and standardizing and coordinating the provision of care. CAHPS data were integral to the continuous QI needed for moving beyond formal PCMH recognition and maximizing primary care medical home transformation. This supports the recommendation of NCQA to administer CAHPS or CAHPS PCMH items as part of PCMH transformation.

 RAND Corporation (DDQ, NQ, LA, RDH), Santa Monica, CA; Division of General Internal Medicine & Health Services Research, University of California, Los Angeles (RDH), Los Angeles, CA.

 This work was supported by a cooperative agreement from the Agency for Healthcare and Research Quality (contract No. U18-HS016980 and U18-HS025920).

 The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (DDQ); acquisition of data (DDQ, NQ, LA); analysis and interpretation of data (DDQ, NQ, LA, RDH); drafting of the manuscript (DDQ, LA, RDH); critical revision of the manuscript for important intellectual content (DDQ, LA, RDH); statistical analysis (DDQ, NQ, LA); provision of patients or study materials (DDQ); obtaining funding (DDQ, RDH); administrative, technical, or logistic support (DDQ); and supervision (DDQ).

 Denise D. Quigley, PhD, RAND Corporation, 1776 Main St, Santa Monica, CA 90407-2138. Email: quigley@rand.org.

1. Perry RJ, McCall N, Wensky SG, Haber SG. 

Continuity in a Patient-Centered Medical Home Setting

. RTI Press; 2016. Accessed April 16, 2020. https://www.ncbi.nlm.nih.gov/books/NBK532528/

2. Timbie JW, Hussey PS, Setodji CM, et al. Association between patient-centered medical home capabilities and outcomes for Medicare beneficiaries seeking care from federally qualified health centers. 

. 2017;32(9):997-1004. doi:10.1007/s11606-017-4078-y

3. Pines JM, Keyes V, van Hasselt M, McCall N. Emergency department and inpatient hospital use by Medicare beneficiaries in patient-centered medical homes. 

. 2015;65(6):652-660. doi:10.1016/j.annemergmed.2015.01.002

4. Fifield J, Forrest DD, Burleson JA, Martin-Peele M, Gillespie W. Quality and efficiency in small practices transitioning to patient centered medical homes: a randomized trial. 

. 2013;28(6):778-786. doi:10.1007/s11606-013-2386-4

5. van Hasselt M, McCall N, Keyes V, Wensky SG, Smith KW. Total cost of care lower among Medicare fee-for-service beneficiaries receiving care from patient-centered medical homes. 

. 2015;50(1):253-272. doi:10.1111/1475-6773.12217.

6. Accreditation with medical home. Accreditation Association for Ambulatory Health Care. Accessed April 16, 2020. https://www.aaahc.org/accreditation/medical-home/

7. Primary care medical home certification program. The Joint Commission. Accessed April 16, 2020. https://www.jointcommission.org/en/accreditation-and-certification/certification/certifications-by-setting/hospital-certifications/primary-care-medical-home-certification/

Internal Data From NCQA’s Monthly Progress Reports: Recognition Programs Operations. 

National Committee for Quality Assurance; 2015.

National Committee for Quality Assurance; 2017.

10. Wagner EH, Coleman K, Reid RJ, Phillips K, Abrams MK, Sugarman JR. The changes involved in patient-centered medical home transformation. 

. 2012;39(2):241-259. doi:10.1016/j.pop.2012.03.002

11. Sugarman JR, Phillips KE, Wagner EH, Coleman K, Abrams MK. The safety net medical home initiative: transforming care for vulnerable populations. 

. 2014;52(11 suppl 4):S1-S10. doi:10.1097/MLR.0000000000000207

12. Qureshi N, Quigley DD, Hays RD. Nationwide qualitative study of practice leader perspectives on what it takes to transform into a patient-centered medical home. 

. 2020;35(12):3501-3509. doi:10.1007/s11606-020-06052-1

13. Stout S, Weeg S. The practice perspective on transformation: experience and learning from the frontlines. 

2014;52(11 suppl 4):S23-S25. doi:10.1097/MLR.0000000000000239

14. Cronholm PF, Shea JA, Werner RM, et al. The patient centered medical home: mental models and practice culture driving the transformation process. 

. 2013;28(9):1195-1201. doi:10.1007/s11606-013-2415-3

15. Friedberg MW, Coltin KL, Safran DG, Dresser M, Zaslavsky AM, Schneider EC. Associations between structural capabilities of primary care practices and performance on selected quality measures. 

. 2009;151(7):456-463. doi:10.7326/0003-4819-151-7-200910060-00006

16. Friedberg MW, Safran DG, Coltin KL, Dresser M, Schneider EC. Readiness for the patient-centered medical home: structural capabilities of Massachusetts primary care practices. 

. 2009;24(2):162-169. doi:10.1007/s11606-008-0856-x

17. Friedberg MW, SteelFisher GK, Karp M, Schneider EC. Physician groups’ use of data from patient experience surveys. 

. 2011;26(5):498-504. doi:10.1007/s11606-010-1597-1

18. Geissler KH, Friedberg MW, SteelFisher GK, Schneider EC. Motivators and barriers to using patient experience reports for performance improvement. 

. 2013;70(6):621-635. doi:10.1177/1077558713496319

19. Quigley DD, Mendel PJ, Predmore ZS, Chen AY, Hays RD. Use of CAHPS patient experience survey data as part of a patient-centered medical home quality improvement initiative. 

20. Case studies of leading primary care practice facilitation programs: lessons learned from leading models of practice facilitation. agency for healthcare research and quality. January 2013. Accessed September 10, 2020. https://pcmh.ahrq.gov/sites/default/files/attachments/LessonsLearned_022813comp_0.pdf

Articles

The Consumer Assessment of Healthcare Providers and Systems (CAHPS) Patient-Centered Medical Home survey assists practices by targeting improvement, standardizing care delivery, discussing best practices, improving provider-patient communication, coordinating care, and supporting patient self-management.

Using CAHPS Patient Experience Data for Patient-Centered Medical Home Transformation

 Knowing which patient-centered medical home (PCMH) care delivery changes and quality improvement (QI) practices further PCMH implementation is essential.

We used the 2008-2017 National Committee of Quality Assurance (NCQA) PCMH directory of 15,188 primary care practices that received Level 1, 2, or 3 NCQA PCMH recognition to construct a stratified national sample of 105 practices engaged in PCMH transformation. We examined their QI practices and PCMH changes associated with PCMH transformation.

We derived QI practice and PCMH change variables from semistructured interviews. Practice leaders completed the PCMH Assessment (PCMH-A) measuring the practice’s degree of PCMH implementation, which is a proxy for patient-centeredness. Controlling for practice characteristics, we regressed PCMH-A scores on QI practice and PCMH change variables.

 Practices undergoing PCMH transformation nationwide most commonly made care delivery changes in access and continuity of care. To improve quality, practices most commonly engaged in discussing and targeting areas of patient experience improvement, trending performance, and conducting targeted QI. However, practices lower in patient-centeredness as measured by the PCMH-A were more likely to engage in efforts to improve patient experiences, such as reviewing patient experience data or engaging in 1-on-1 provider counseling related to patient interactions. Mature PCMH practices focused on changes in continuity of care.

 Practices undertake a wide variety of care delivery changes and QI practices simultaneously to meet PCMH requirements. The patient experience–specific QI practices and PCMH care delivery changes that practices make to improve patient-centeredness differ by years of PCMH recognition.

Evidence shows that practice leaders who implement the patient-centered medical home (PCMH) care model encounter challenges in driving change. Understanding which PCMH changes and quality improvement (QI) practices further PCMH implementation is essential.

Practices implementing PCMH made a wide range of changes in care delivery and QI practices.

The most common changes were in access and continuity of care. QI practices used most often were discussing and targeting areas of patient experience improvement and trending performance.

Mature PCMH practices focused on changes to continuity of care, whereas practices lower in patient-centeredness focused on improving patient experiences. These differences provide guidance for practices undergoing PCMH implementation.

Health care organizations in the United States face challenges in delivering high-quality primary care.

 For reform measures to successfully improve health, affordability, and patient experience, as well as meaning in work for providers, the care delivery system must be transformed.

 The patient-centered medical home (PCMH) model is a leading effort to improve quality of care

 by reorganizing a practice’s care delivery process to promote comprehensive care, patient-centeredness, care coordination, accessible services, safety, and high-quality care.

 The model offers many pathways to reduce costs and improve quality, but evidence of its impact is mixed.

 Due in part to variations in the amount and types of practices implemented, PCMH implementation varies significantly by site, with practices undergoing many changes.

 Although there is no single way to achieve PCMH transformation, there is concordance in the main implementation steps.

PCMH recognition and certification require that core criteria are satisfied; they also provide various optional criteria.

 Practices target PCMH changes that are required for recognition and utilize a variety of quality improvement (QI) activities for PCMH transformation. Practices are required to collect and use patient experience surveys and other data to improve their care and services. Which QI activities and PCMH changes are most useful is unclear.

 Research has examined organizational climate,

 structural factors associated with PCMH changes,

 and which PCMH implementation resources are widely used,

 but it has not studied these factors alongside a validated measure of PCMH implementation with established psychometric qualities. One such measure is the PCMH Assessment (PCMH-A), a proxy for patient-centeredness that measures the degree of PCMH implementation at the practice level.

 PCMH-A scores are higher for recognized PCMH practices than for nonrecognized practices.

 In this study, we investigate QI practices and PCMH changes associated with higher levels of PCMH transformation using the PCMH-A.

The National Committee for Quality Assurance (NCQA) is the largest program to certify PCMH practices. To select our sample, we used NCQA’s PCMH directory of 15,188 primary care practices that had applied for and received Level 1, 2, or 3 NCQA PCMH recognition between 2008 (when NCQA PCMH began) and 2017. We excluded practices that lacked a current NCQA PCMH status, were pediatric only, were in Puerto Rico, or provided care to the Armed Forces.

From the remaining 10,712 practices, we developed a stratified sample of 900 practices by US Census region, whether a statewide PCMH initiative existed, physician count, level and years of PCMH recognition, and use of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) PCMH survey. Of these, 55 were ineligible (eg, merged, closed, HIV only, child only), leaving 845 practices. We drew a stratified random sample of 294 practices, contacted them via email and phone, and received completed responses from 105 (36% response rate). 

 summarizes final sample characteristics. Details of the sample and qualitative findings are described elsewhere.

Four researchers (2 coauthors [D.D.Q., N.Q.], 2 analysts) conducted semistructured, hour-long interviews with practice leaders in charge of PCMH transformation. Each interview discussed practice characteristics, such as PCMH history (ie, level and years of PCMH recognition), motivation for PCMH transformation, areas of care delivery change targeted for PCMH implementation, and QI practices used to make PCMH changes. Practice leaders completed the PCMH-A after the interview. We recorded and transcribed the phone interviews and paid participants a $75 honorarium.

We entered transcripts into Dedoose (SocioCultural Research Consultants), a web application for analyzing qualitative data. We created codes for care delivery changes identified in the NCQA PCMH standards and related QI practices using content analysis to identify key topics and develop a codebook.

 We developed the code structure using systematic, inductive procedures to generate insights from responses and mapped codes to research questions.

 We used content analysis to identify themes.

The 4 interviewers coded transcripts individually, using team meetings to reach consensus. We mapped emerging topics and processes to the 2017 NCQA PCMH standards and designated whether areas of improvement were part of core or optional criteria for PCMH recognition.

 Practices must meet core criteria but have leeway on optional criteria. We also coded QI practices discussed during the interviews.

Interrater reliability was 0.93, indicating “very good” agreement.

 We employed ongoing training among the coding team on emerging subcodes using the Dedoose training module. Study protocols were approved by RAND’s Human Subjects Protection Committee (IRB Assurance No. FWA00003425; IRB No. IRB00000051) and the Office of Management and Budget (OMB No. 0935-0236).

PCMH-A scores range from 1 to 12, with higher scores indicating greater implementation of PCMH recommended practices. Scores assess 36 PCMH practices and behaviors associated with 8 PCMH change concepts. The overall score is the mean of the 8 change-concept scores. PCMH-A scores are sensitive to practice change over a time period as short as 6 months,

 and we used the overall score as our dependent variable.

QI practice and PCMH change variables were elicited from semistructured interview questions about practice changes related to PCMH or its requirements. The 

) provides quotes and terms to establish these indicators. Responses were coded to align with NCQA PCMH standards and to note whether changes were core or optional.

We derived QI practice indicators from questions on PCMH implementation and on efforts to improve patient experiences. The 4 general QI practices were (1) tracking performance and trending data over time, (2) discussing best practices with other providers or practices, (3) using staff meetings to review performance data, and (4) starting or continuing QI initiatives for PCMH change. The 5 patient experience QI practices were (1) sharing patient experience survey scores with leadership, (2) benchmarking and comparing patient experience scores over time, (3) discussing/targeting areas of needed improvement related to patient experiences of care, (4) reviewing provider performance on patient experience surveys, and (5) 1-on-1 counseling with low-performing providers. We scored these QI practice measures as dichotomous variables. We also created 2 count measures of the number of general (0-4) or patient-specific (0-5) QI practices.

The 7 core PCMH changes were for access, communicate lab test results, continuity of care, flow of patient visit, referrals to specialists, team-based care, and team huddles. The 4 optional PCMH changes were for behavioral health

 use of electronic health records (EHRs) to support patient care, and population health management. We analyzed dichotomous indicators of whether the practice implemented each of these changes and also total counts of the 7 core and 4 optional changes made by a practice.

We classified practices’ PCMH history into 3 groups: (1) practices that had not yet achieved Level 3 PCMH recognition (ie, received NCQA Level 1 or Level 2), (2) practices that had achieved Level 3 PCMH recognition within the past 3 years, and (3) practices that had held Level 3 PCMH recognition for more than 3 years.

We examined the percentages of practices reporting each QI and PCMH initiative overall and by PCMH history (

). We fit separate multiple linear regression models for the 9 QI practices, 11 PCMH changes, and 4 count variables, controlling for PCMH history, CAHPS PCMH survey status, location, and number of physicians. Next, we fit these same models with an interaction term between each QI practice or PCMH change and PCMH history. We added the interaction terms because PCMH history is a potential effect modifier. Because access and continuity of care changes were endorsed by nearly all practices, we did not include them in the interaction model.

From these models, we calculated estimated marginal means (EMMs) for all practices (using the noninteraction model) and by practice subgroups (using the interaction model). For the 4 count variables, we calculated the EMM to compare practices that had a sum of 1 with those that had a sum of 0. In calculating the EMM, we set all other control variables to their mean values. All analyses used R version 3.6.1 (R Foundation for Statistical Computing), including the 

Table 1 summarizes characteristics of the 105 practices. About half the practices were in the Northeast, and about half had Level 3 certification for at least 3 years. Most had fewer than 10 physicians on staff. The mean PCMH-A score for practices with Level 3 PCMH recognition for 3 or more years (9.6) was higher than the overall mean score of 9.2. It was also higher for non–PCMH initiative states (8.8), other Northeast states (10.0), and sites that were primary care only (9.4). Seventy-eight percent of practice leaders were present since the start of PCMH implementation (described elsewhere).

Table 2 shows the proportion of the 105 practice managers reporting the implementation of QI practices and PCMH changes. The most widely implemented PCMH changes were changes to continuity of care and access to care, followed by use of the EHR for patient-centered care, conducting team huddles, and changing the flow of a patient visit. The least implemented PCMH change was investing in population health management activities. Sixteen percent of practices had implemented all 11 changes; the mean number of core changes was 6.0 (of 7) and the mean number of optional changes was 2.9 (of 4).

Eighty-eight percent of practices employed at least 1 of the 9 QI practices during PCMH implementation, but none employed all 9 practices. Sixty-three percent of practices engaged in at least 1 general QI practice and 60% in at least 1 patient experience–specific QI practice. The most-used QI practices were discussing and targeting areas of patient experience improvement (34%), tracking performance and trending data over time (33%), and starting or continuing QI activity for PCMH change areas (30%). The least-used practices were discussing best practices with other providers or practices (10%), benchmarking or comparing patient experience data over time (15%), and 1-on-1 counseling with low-performing providers (15%). The mean count of general QI practices was 0.9 (of 4) and of patient experience–specific practices was 1.1 (of 5).

Our noninteraction models show that review of provider performance on patient experience, 1-on-1 counseling for patient experience, and referrals to specialists were associated with lower mean PCMH-A scores (

). Use of more patient experience–specific QI practices was also associated with lower mean PCMH-A scores.

Our exploratory interaction models for each group of practices by PCMH maturity indicate that among practices with less than 3 years of PCMH recognition, those that engaged in benchmarking and comparing data over time or 1-on-1 counseling for patient experience had lower PCMH-A scores than those that did not (

). In addition, a greater number of QI-specific practices or optional PCMH changes were associated with lower PCMH-A scores. The only reported practice type associated with significantly higher PCMH-A scores was those with 3 or more years of PCMH recognition that worked on continuity of care.

Implementing the PCMH model of care includes a wide range of changes in care delivery, particularly changes related to continuity of care and access. This is not surprising, because access and continuity of care are core aspects of PCMH recognition and transformation. Many sites across the 3 groups of PCMH transformation reported implementing the core set of NCQA PCMH changes. This is consistent with attempts to implement the standards and elements of the PCMH model. There is variation in how practices chose to implement optional criteria, with, for example, 80% implementing referrals to specialists and 44% implementing population health practices. Our findings support the adage that what is required is implemented and what is measured is managed.

Our results confirm that QI is a key activity of PCMH implementation, with the majority of practices engaging in combinations of multiple QI practices. PCMH transformation most often included the general QI practices of tracking and trending performance over time and starting or continuing a QI activity, as well as the more patient experience–specific QI practices of sharing patient experience data with leadership and discussing and targeting areas of patient experience improvement. Tracking and trending performance can identify target areas for QI efforts. Discussing and targeting areas of patient experience improvement indicates that patient experience data, metrics, and improvement are key for practices as they become more patient centered. However, practices undertook each individual QI practice much less often than making one of the central changes to care delivery outlined to achieve required PCMH standards.

Controlling for site characteristics, we found that practices using 1-on-1 counseling to improve patient experience scores or reviewing provider performance on patient experience had significantly lower PCMH implementation than others. Reviewing provider performance on patient experience and using 1-on-1 counseling for providers can identify and target modifiable provider behaviors.

 Such coaching has been effective in building and maintaining competencies among providers and in increasing compliance with practice guidelines.

Furthermore, among practices with the same stage of PCMH implementation from our exploratory analysis, we find that sites with less than 3 years of PCMH Level 3 recognition significantly differ in their use of 1-on-1 counseling, benchmarking and comparing data, and utilizing an increased number of patient experience–specific QI practices. Controlling for site characteristics, those using 1-on-1 counseling for patient experience performance improvement, benchmarking and comparing data over time, and more QI practices related to patient experience have lower PCMH implementation. This suggests that sites lacking PCMH care delivery (ie, low PCMH implementation indicating low patient-centeredness) focus more on benchmarking and comparing data over time, 1-on-1 counseling for patient experience performance improvement, and QI activities specific to improving patient experience. That is, practices recognize that they are lacking in patient-centeredness and search for ways to improve patient experiences of care through identifying low performers and using targeted, individual counseling as a means of improving quality.

Our study extends previous evidence on PCMH implementation. Solberg et al

 found among 123 primary practices in Minnesota that setting goals and benchmarking performance at least yearly worked well for PCMH implementation. We found that benchmarking practice-level performance was used most often during the early stages of PCMH transformation.

Previous research found that the culture of quality improvement and specifically conducting QI efforts during PCMH implementation is important; however, studies that investigated QI as part of PCMH implementation were a single health care organization

 To our knowledge, the only nationwide PCMH implementation study was a cross-sectional study of 6464 Veterans Health Administration primary care personnel within clinics nested within 135 hospitals.

 That study found that team huddles were most widely used and QI was least frequently used.

Additionally, we found that practices with mature PCMH recognition achieve higher levels of PCMH implementation by focusing on changes to continuity of care. We examined the association of team huddles and changes to continuity of care or access to care in addition to the role of QI practices in assessing a site-level measure of PCMH implementation. Although we did not find a significant association between most PCMH changes and QI practices and higher levels of PCMH implementation, we did find that practices lacking in PCMH implementation engaged more in benchmarking and comparing data over time and more QI activities related to improving patient experience, specifically 1-on-1 counseling. When needing to improve patient-centeredness, practices focused on improving their referrals to specialists and engaging in benchmarking and comparing data over time. They did not engage in general QI but instead in specific QI practices for improving patient experience.

The lack of significant findings for PCMH changes by levels of implementation may be because of the high endorsement of PCMH changes, yielding little variation. Or, the nonsignificant findings for conducting QI may result from the wide range of general QI efforts that practices can undertake during PCMH transformation, with each connected to PCMH implementation. Further work should characterize general PCMH QI initiatives during PCMH transformation and investigate their association with PCMH history and implementation.

Our study has limitations. Although the study design included a national, large, stratified set of primary care practices, it was not nationally representative. Also, although we recruited and identified leaders most knowledgeable and responsible for PCMH changes in the practice, we typically interviewed 1 representative per organization. There may be additional perspectives within practice leadership that we did not capture, and leaders with more positive feelings toward PCMH may have chosen to participate. Our participation rate was 36% and may not have been random within each stratum. Nevertheless, this study explored a rich, wide range of perspectives from practice leaders about PCMH implementation. It was able to show which PCMH changes were more common and provide evidence that practices do engage primarily in making core PCMH changes and that they vary in making optional PCMH changes. Our data are limited in that they are cross-sectional, but we were able to control for practice characteristics including PCMH history. The PCMH-A was completed after the interview, which may have resulted in higher PCMH-A scores. Because the PCMH-A is meant to be an inventory of all PCMH changes, these scores likely reflect more accurate assessments than if the interviews were conducted after PCMH-A completion. Finally, we focused on practices that obtained PCMH recognition from NCQA, the most widely used organization to provide PCMH recognition, and did not include practices that sought PCMH recognition from different programs (eg, The Joint Commission).

Implementation of the PCMH model includes a wide range of changes in care delivery and a varied set of QI practices. Practices that recognize they are lacking in patient-centeredness seek to specifically improve patient experiences of care; they review patient experience scores or engage in 1-on-1 counseling of providers to improve their patient experience scores. Practices that have recently obtained PCMH Level 3 recognize their lack of patient-centeredness (ie, low PCMH implementation) and engage in ways to improve their implementation. This includes increasing the number of optional PCMH changes, benchmarking and comparing data over time, and increasing QI-specific practices related to patient experience, particularly 1-on-1 counseling for patient experience improvement. Such actions can advance the PCMH transformation process, whereas practices with mature PCMH recognition can achieve higher implementation scores by focusing on changes to continuity of care.

Practices undertake a wide variety of care delivery changes and QI practices simultaneously to meet PCMH requirements. Given limited resources, it is important to know what care delivery changes or QI practices are needed when PCMH implementation is lacking and when those changes or practices should be implemented. Our study provides considerations for which PCMH changes and what QI practices, particularly those related to patient experience performance, should be instituted to improve PCMH implementation. Differences in timing of care delivery changes during the course of PCMH recognition provide guidance for practices in PCMH implementation.

The authors acknowledge the time and support of the interview participants in this study.

 RAND Corporation (DDQ, MS, NQ, MNE, RDH), Santa Monica, CA; University of California, Los Angeles (RDH), Los Angeles, CA.

 This work was supported by a cooperative agreement from the Agency for Healthcare Research and Quality (contract No. U18 HS016980 and U18 HS025920).

 Concept and design (DDQ); acquisition of data (DDQ, NQ); analysis and interpretation of data (DDQ, MS, NQ, MNE); drafting of the manuscript (DDQ, MS, NQ); critical revision of the manuscript for important intellectual content (DDQ, NQ, MNE, RDH); statistical analysis (MS, NQ); provision of patients or study materials (DDQ, NQ); obtaining funding (DDQ, MNE, RDH); administrative, technical, or logistic support (DDQ, NQ); and supervision (DDQ, RDH).

 Denise D. Quigley, PhD, RAND Corporation, 1776 Main St, Santa Monica, CA 90406. Email: quigley@rand.org.

1. Berwick DM. A user’s manual for the IOM’s ‘Quality Chasm’ report. 

. 2002;21(3):80-90. doi:10.1377/hlthaff.21.3.80

Crossing the Quality Chasm: A New Health System for the 21st Century

3. Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. 

Practices undertake many care delivery changes and quality improvement practices to become patient-centered medical homes (PCMHs), and these differ by years of PCMH recognition.

Denise D. Quigley, PhD

Articles

Using CAHPS Patient Experience Data for Patient-Centered Medical Home Transformation

Practices and Changes Associated With Patient-Centered Medical Home Transformation

Patient-Reported Denials, Appeals, and Complaints: Associations With Overall Plan Ratings

Tools to Gauge Progress During Patient-Centered Medical Home Transformation