Zachary S. Predmore, AB

To review tools designed to evaluate and improve the extent of patient-centered medical home (PCMH) implementation.

Literature search and review of tools to evaluate PCMH “medical homeness” and track progress toward practice transformation.

We conducted a literature search to identify tools designed for evaluation and quality improvement during the PCMH change process. We identified and reviewed the content of 5 publicly available PCMH survey tools used by an administrator or clinical lead to collect data at the practice level for evaluation and/or quality improvement during PCMH implementation. We assessed each tool’s coverage of PCMH content, standards, and requirements. 

 We found that 3 tools (Patient-Centered Medical Home Assessment [PCMH-A], Primary Care Assessment Tool—Facility Edition, and Medical Home Care Coordination Survey–Healthcare Team [MHCCS-H]) are actionable for quality improvement. PCMH-A assesses the broadest array of practice capabilities and includes items pertaining to all National Committee for Quality Assurance PCMH standards. MHCCS-H was the only tool to contain items on comprehensiveness of care. There was variation in emphasis on main domains, with some content areas covered by only 1 tool. 

There is currently little evidence on which PCMH tools are associated with improved quality outcomes, as relatively few longitudinal studies have been conducted. Of the 5 tools we reviewed, only PCMH-A and MHCCS-H impose a light administrative burden (less than 10 minutes to complete) and can identify specific actions to improve a given practice capability. Each tool is lacking in a particular content area: PCMH-A, for example, lacks items on comprehensiveness of care, whereas MHCCS-H lacks items addressing access to care.

The American Journal of Accountable Care. 2017;5(4):e8-e18

The patient-centered medical home (PCMH) model is promoted as a way to transform US primary care practices.

 It incorporates current best practices for access to care, prevention, chronic disease management, care coordination, and responsiveness to patients.

 Monitoring the transformation of primary care practices to PCMHs requires understanding the context, change process, patient experiences of care, and outcomes of care.

 Even after obtaining recognition from one of the several organizations that grant “official” PCMH status, practices continue their transformation into a PCMH.

Practices, networks, and federally qualified health centers need an evaluative and quality improvement (QI) tool to collect data and conduct real-time analyses that help them understand their level of “medical homeness,” track progress toward transformation goals, and identify possible improvements. Practice coaches, payers, and pilot or demonstration sponsors may also use these tools. Most primary care practices cannot engage in a third-party evaluation or large-scale demonstration with medical record audits, site visits, and staff surveys, but need a practical survey tool they can use to gauge achievement of PCMH standards. 

Despite their importance, information about tools for assessing PCMH progress is limited. Malouin et al

 found that the Primary Care Assessment Tool (PCAT), available in multiple formats, was the only tool that scored high on primary care features, as it was designed to assess both structural and process features of primary care, but their evaluation was conducted early in PCMH adoption and focused on a narrower set of components than currently delineated in PCMH definitions and standards.

 included tools primarily used for the PCMH recognition process, which itself requires adequate and appropriate documentation, rather than for practice self-improvement, research, evaluation, or quality measurement. 

This article reviews PCMH tools designed for evaluative and QI purposes to assist practices and researchers wanting to select a survey tool to track outcomes of PCMH transformation. We also assess how well the identified PCMH tools align with the 2014 and 2011 standards of the National Committee for Quality Assurance (NCQA), the organization responsible for the largest share of PCMH recognition across the United States.

Using traditional bibliographic databases (PubMed, Google Scholar, and PsycInfo), we systematically reviewed the peer-reviewed literature on patient-centered healthcare to identify surveys or tools designed to evaluate or monitor the process of becoming a PCMH. We also searched the Agency for Healthcare Research and Quality’s online primary care measure directory. Search terms included “patient-centered medical home” OR “PCMH” OR “medical home” OR “care coordination” OR “team-based care” AND “questionnaire*” OR “telephone” OR “phone” OR “survey” OR “surveys” OR “tool” OR “tools.” We limited our search to English-language articles published since 1990 on adult populations. We identified additional resources through reference citations. 

Our primary inclusion criteria were that a tool: 1) was designed as a survey with questions and response options, 2) was publicly available, 3) measured half or more of the PCMH domains (ie, access, team-based care [including continuity of care], population health management, care management and support [including shared decision making and self-care/self-management support], care coordination [including care transitions], and performance measurement and QI), 4) was based on 2014 PCMH standards of care, 5) was designed for QI or evaluation (not for PCMH recognition), 6) focused on the practice (not the health system) as the unit of change, and 7) did not require an external surveyor or site visit but could be administered by practice site administrators or evaluators.  

We identified 20 potential survey tools and abstracted details on each tool’s identifying information, including whether it was designed for recognition (for a national organization like NCQA or state agencies), evaluation, or QI; whether it assessed at the practice level; form (eg, paper or web); who completes the tool; and publications documenting its reliability. We estimated time burden of a tool by counting the number of questions overall and by type (eg, closed- or open-ended questions).

 Once tools not meeting these initial criteria were excluded, we coded each question in the remaining tools by content domain and related NCQA PCMH standard and element.

 We chose the NCQA PCMH standards because they are the most widely used standards, with recent estimates suggesting 15% to 18% of primary care physicians work in a NCQA-recognized PCMH.

 We created a summary worksheet to record the percentage of items in each content domain and the percentage of items related to each of the current (2014) and previous (2011) NCQA PCMH standards and elements. The project was approved by RAND Corporation’s institutional review board (Number: FWA00003425).

 provides inclusion and exclusion information on all 20 identified tools. Of these, we excluded 3 that were not publicly available, 1 that was based on 2008 NCQA PCMH standards,

 4 that were not designed for evaluation, 1 that was not a survey, 3 that focused on the health system rather than the practice, 2 that had limited PCMH content, and 1 that required an external surveyor and site visit. 

We were left with 5 current publicly available tools designed to collect data at the practice level by an administrator or clinical lead for practice improvement, research, evaluation, or quality measurement: Medical Home Index-Long Version (MHI-LV) and -Short Version (MHI-SV), Medical Home Care Coordination Survey—Healthcare Team (MHCCS-H), Primary Care Assessment Tool–Facility Edition (PCAT-FE), and Patient-Centered Medical Home Assessment (PCMH-A) tool. 

shows details of these 5 tools. All except PCAT-FE have a light (less than 10 minutes to complete) or medium (approximately 20 minutes) administrative burden, with zero monetary cost and web-downloaded availability. Based on the Berry

 method of estimation, completion times for the tools range from 8 to 37 minutes (for 36 to 167 items). None require self-reported documentation or verification of responses. For 4 of these tools, a key clinical lead at the practice responds; for the PCAT-FE, a system administrator does so. Both the MHCCS-H and the PCMH-A also recommend that a multidisciplinary team of practice staff (including primary care physicians, physician assistants, nurses, and clinical administrative staff) collectively complete 1 survey per practice. 

Answer format varies by tool, leading to different levels of data actionability. To be actionable, a survey question must outline specific practices or elements that can be identified and changed/improved, link the specific information to a numerical value/score, and cover no more than 1 practice or element.

We judged that the PCMH-A, PCAT-FE, and MHCCS-H are generally actionable for QI, whereas the other 2 tools have actionability limitations. The PCMH-A uses a numerical 1 to 12 rating based on the extent to which recommended practices and behaviors are implemented, divided across 4 levels with specific actions or practices described for each item; it is also sensitive to practice change over a time period as short as 6 months.

 The PCAT-FE and MHCCS-H both have items that are specific to 1 action and align with the response scales, making them fully actionable. The MHCCS-H has only 1 ambiguous item (regarding primary care team roles). By contrast, the MHI-LV and MHI-SV do not help the user determine specific practices to implement. For example, the MHI-LV asks a respondent to rate whether there is “partial” or “complete” implementation of an item with several components of patient- and family-centered care; an answer of “partial” does not allow understanding of which specific component needs further work. Burton et al

 similarly documented these actionability problems with the Medical Home Index (MHI) tool. 

All 5 tools have detailed scoring instructions and include a summary-score calculation. The PCMH-A asks teams to rate their care delivery in 36 key areas associated with the 8 change concepts. Each item is scored from 1 to 12 based on the extent to which recommended practices and behaviors are implemented. The MHI-LV has groups of items that are considered “themes,” with a total of 25 items using an 8-point scale where “partial” mastery of the most basic item equals 1 point and “complete” mastery of the most advanced of the 4 items equals 8 points. The MHI-SV is a 10-item survey that scores a practice on a continuum of care across 3 levels reflecting the degree to which it has achieved components of a medical home, scored on a 1 to 5 scale. Level 1 is “good, responsive primary care”; Level 2 is “pro-active primary care (in addition to Level 1)”; and Level 3 is “primary care at the most comprehensive levels” (in addition to Levels 1 and 2). The reporter is asked to respond to each of the 10 indicators with: Level 1, Level 2—partial, Level 2–complete, Level 3–partial, or Level 3–complete. The MHCCS-H has 8 care coordination domains and a total of 35 items (34 items plus 1 global-rating item), with 5-point categorical-response scales, plus 2 responder-characteristic items. The PCAT-FE is scored by taking the average score (on a 1 to 4 scale) of each domain.

Reliability and validity have been estimated for the MHI-LV

 examined interrater reliability between 2 project staff and between practices and project staff and the internal consistency of MHI domains and themes. The intraclass correlation coefficients between the summary scores of the interviewers was .98 and between the scores of the 2 interviewers and scores of the practices, .97. Standardized coefficient alphas for the 6 domains of the MHI ranged from .81 to .91, and the overall standardized alpha was .96. In the sample of practices studied, the MHI was an internally consistent instrument with acceptable reliability and support for validity for use by pediatric primary care practices to assess PCMH implementation. For the MHCCS-H, Zlateva et al

 conducted analysis to examine internal consistency, reliability and validity of the MHCCS-H using structural equation modeling. All domains had Cronbach’s coefficient alphas above 0.80.

described a longitudinal study of 65 safety-net practices from 5 states participating in a national demonstration program for PCMH transformation that compared 15 practices that achieved PCMH recognition with 29 practices that were actively pursuing NCQA recognition practice (but had not achieved it) and 21 practices that were not pursuing NCQA recognition but that used the PCMH-A. They found that mean overall PCMH-A scores increased overtime, indicating that the PMCH-A was sensitive to change over time (7.2 in March 2010 to 9.1 in September 2012; 

 <.01). Increases were statistically significant for each change concept (

 <.05). Implementers of PCMH change (ie, facilitators) agreed with scores 82% of the time. NCQA-recognized sites (n = 15) had significantly higher PCMH-A scores than sites that were not yet recognized. Sites that had completed more transformation activities and progressed over defined tiers reported significantly higher PCMH-A scores. They concluded that the PCMH-A was sensitive to change over time and provided an accurate reflection of practice transformation.

 shows the emphases given to different content domains in the PMCH tools. We started with the content areas identified by the Burton

 review of PCMH tools and added several newly identified domains. There were 24 content areas identified across the PCMH tools. Content domains receiving the most emphasis were: 1) use of care plan in care management, 2) coordination of care, 3) provide self-care support and community resources, 4) quality improvement, and 5) enhance access and continuity using medical records. Each tool had a main emphasis: PCMH-A targeted continuity of care; MHI-LV, community resources; MHI-SV, coordination of care; and MHCCS-H, the care plan. Some content areas were only covered by 1 tool: comprehensiveness of care items (defined as the breadth of services a practice offers to address any health problem at any given stage of patient’s life [eg, the primary care practice/health center has behavior change interventions readily available for patients as part of routine care, using a 5-point agreement scale]) by MHCCS-H; empanelment, evidence-based care, quality measurement, and quality reporting by PCMH-A; and standard non—PCMH care provided (used as a base level to PCMH care for patient and family involvement) and adherence to current laws by MHI-LV. 

Our review mapped PCMH tools to both the NCQA 2011 and 2014 standards. 

 shows the relative content emphases for each of the 2011 NCQA PCMH standards and elements. The 2011 NCQA PCMH standards receiving the most emphasis across tools were: a) Standard 4: Provide Self-Care and Community Support; b) Standard 3: Plan and Manage Care; c) Standard 1: Enhance Access and Continuity; d) Standard 5: Track and Coordinate Care; e) Standard 6: Measure and Improve Performance; and f) Standard 2: Identify and Manage Patient Populations. 

Although many standards and elements remained the same from the 2011 to the 2014 NCQA PCMH standards, there are new areas of focus. The 2014 version focuses more on care management. In previous standards, practices needed to identify high-risk patients; 2014 requires also focusing on patients who would benefit from care management. NCQA 2014 includes a new standard on team-based care, giving more emphasis to providing patients with information—proactively orienting them to the practice, helping them choose a physician, and explaining the practice’s use of evidence-based guidelines. This team-based care element not only ensures that the whole care team works at the top of their licensure (included in 2011), but also improves the focus on the individual patient. 

The population health standard in 2014 incorporates many Stage 2 Core and Menu Meaningful Use requirements, use of electronic medical records, and a comprehensive health assessment. This is not a new NCQA standard, but the documentation requirements have changed. For the 10 factors, NCQA requires the practice either has a report or conducts chart audits using NCQA’s tool, the Record Review Workbook. Practices had been allowed to show examples of their comprehensive health assessments. Behavioral health has been integrated into numerous standards, where it had previously not been a focus. 

In 2011, NCQA wanted practices to develop the habit of measuring performance. As a result, the 2014 standards identify more areas for improving practice performance. Additionally, in the 2014 standards, NCQA dropped the 2011 element 4B (Provide Referrals to Community Resources); the 2011 element 6F (Report Data Externally) was incorporated into the 2014 element 6F (Report Performance); and the 2011 element 6A (Measure Performance) was divided into 2 separate elements: 2014 element 6A (Measure Clinical Quality Performance) and element 6B (Measure Resource Use and Care Coordination). NCQA has published a full crosswalk of 2011 to 2014 standards.

characterizes how well each assessment covers the 2014 PCMH standards. Those receiving the greatest emphasis across tools were: a) Standard 4: Care Management and Support; b) Standard 2: Team-Based Care; c) Standard 3: Population Health Management; d) Standard 5: Care Coordination and Care Transitions; e) Standard 6: Performance Improvement and Quality Improvement; and f) Standard 1: Patient-Centered Access. The PCMH-A had the broadest array of practice capabilities assessed (Table 3) and contained items pertaining to all 6 2014 NCQA PCMH standards and 27 elements (Table 5). The MHI-LV, MHI-SV, and MHCCS-H lacked items addressing the “measurement and improvement of performance” (PCMH Standard 6). The MHCCS-H lacked items addressing “access to care” during office hours and those addressing after-hours or electronic access within “enhance access and continuity” (PCMH Standard 1).

The differences in the 2011 and 2014 standards did not influence our content coding or coverage of content to PMCH standards, except for behavioral health, which was new to the 2014 standards. Items related to behavioral health were included in the PCMH-A and PCAT-FE, but not in the MHI or MHCCS-H. 

We searched for associations of the tools to other quality measures. We found studies using the PCMH-A to track PCMH transformation,

 but no articles specifically tying PCMH-A results to other quality measures. This may be because there has been insufficient time since its development for these studies to be performed. For the MHI-LV, we found limited and mixed results from 3 studies

; 2 showed associations with fewer hospitalizations and lower nonurgent preventable emergency department (ED) use by children and 1 showed associations with higher chronic care management subscores to better provider communication and follow-up on test results but worse access to care among adults. Another study

 found higher levels of “medical homeness” (based on the MHI) associated with lower nonurgent, preventable, or avoidable ED use by publicly insured children. To date, there is little evidence showing which PCMH recognition tool(s) may be associated with improved quality outcomes.

Many healthcare organizations and payers are investing in a PCMH tool for QI or evaluation of medical homeness and progress toward PCMH standards. With little evidence about which PCMH-recognition tool is associated with improved quality outcomes, payers and practices must decide what role quality measurement should play in their PCMH journey, how much administrative burden to place on practices to collect data, what data to use for QI related to PCMH, and how to track practice-level progress toward PCMH transformation. 

We found that the MHI-LV, PCMH-A, and MHCCS-H tools have been subjected to relatively more evaluation and validation and impose only a light or medium administrative burden to complete. The PCMH-A, PCAT-FE, and MHCCS-H (except for 1 of its 37 items) are actionable for quality improvement, whereas the 2 other tools have limitations in their ability to identify specific improvements. The PCMH-A is documented

 to be sensitive to changes in practice behavior over a time period as short as 6 months, indicating it can track QI progress and a practice’s trajectory of PCMH change. All 5 tools have detailed scoring instructions, including a summary-score calculation.

No single tool covers all relevant PCMH content, and some content is only found on 1 tool. Practice capabilities in coordination of care, care plans, and support of self-management and community resources receive the greatest emphasis. None of the reviewed tools covers all 4 main attributes of primary care: comprehensiveness of care, coordination of care, continuity, and access to care.

Because there is no evidence on what combination of practice capabilities produces the best outcomes for patients, practices and payers must decide which medical home aspects they most wish to measure and whether an available tool contains those specific practice capabilities. They must weigh these content decisions against the administrative burden they impose on physicians and clinical staff.

The decision on content includes not only practice capabilities and content areas covered in the PMCH tool but also whether the tool aligns with recognition standards and criteria a practice is pursuing. The NCQA standards are the most widely followed PCMH recognition standards. The PCMH-A had the broadest array of practice capabilities assessed and contains items pertaining to all 6 of the 2011 NCQA PCMH standards/elements and all 6 of the 2014 standards/elements. The MHI-LV and the MHCCS-H lacked items addressing the measurement and improvement of performance (PCMH Standard 6). The MHCCS-H also lacked items addressing access to care during office hours and after hours or electronic access within “enhance access and continuity (2011)/Patient-centered access (2014)” (PCMH Standard 1). Altogether, the PCMH-A could be the most widely applicable tool for most practices transforming into PCMHs, given its alignment with NCQA standards and broad content. As noted previously, the findings of this study are applicable to practices using the NCQA standards, which are the most widely used; practices using other sets of standards may see slight differences in content alignment.  

The decision on which tool to administer for tracking PCMH transformation must weigh administrative burden (ie, cost, time, need for documentation), reliability and validity of data collected, actionability of data, practice capabilities measured, and alignment with PCMH recognition standards. The PCMH-A and MHCCS-H have a light administrative burden and are actionable for quality-improvement purposes. The MHCCS-H has data available on its reliability, whereas the PCMH-A has shown sensitivity to changes over time in practices that have achieved (or have not achieved) NCQA PCMH recognition. Each of the tools we reviewed has a different constellation of practice capabilities and emphases. The PCMH-A assesses the broadest array of capabilities, whereas the MHCCS-H is the only tool with items on capability to provide comprehensiveness of care. 

RAND Corporation (DDQ), Santa Monica, CA; RAND Corporation (ZSP), Boston, MA; Division of General Internal Medicine and Health Services Research, University of California, Los Angeles (RDH), Los Angeles, CA.

This work was supported by a cooperative agreement from the Agency for Healthcare Research and Quality (AHRQ) [Contract number U18 HS016980].

 The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Concept and design (DDQ); acquisition of data (DDQ, ZSP, RDH); analysis and interpretation of data (DDQ, ZSP); drafting of the manuscript (DDQ, ZSP); critical revision of the manuscript for important intellectual content (DDQ, ZSP, RDH); obtaining funding (RDH); and administrative, technical, or logistic support (DDQ, ZSP).

Denise D. Quigley, PhD, RAND Corporation, 1776 Main St, Santa Monica, CA 90407-2138. E-mail: quigley@rand.org.

1. Bitton A, Martin C, Landon BE. A nationwide survey of patient centered medical home demonstration projects. 

. 2010;25(6):584-592. doi: 10.1007/s11606-010-1262-8.

Dentzer S. Reinventing primary care: a task that is far ‘too important to fail’. 

. 2010;29(5):757. doi: 10.1377/hlthaff.2010.0410.

Bodenheimer T, Lorig K, Holman H, Grumbach K. Patient self-management of chronic disease in primary care. 

Ferrer RL, Hambidge SJ, Maly RC. The essential role of generalists in health care systems. 

Grumbach K, Bodenheimer T. A primary care home for Americans: putting the house in order. 

Articles

We reviewed operational details and content of tools designed to evaluate patient-centered medical home (PCMH) transformation. These tools assist practice leaders in understanding specific information about the process and progress of becoming a PCMH.

Zachary S. Predmore, AB

Articles

Tools to Gauge Progress During Patient-Centered Medical Home Transformation