To review tools designed to evaluate and improve the extent of patient-centered medical home (PCMH) implementation.

Literature search and review of tools to evaluate PCMH “medical homeness” and track progress toward practice transformation.

We conducted a literature search to identify tools designed for evaluation and quality improvement during the PCMH change process. We identified and reviewed the content of 5 publicly available PCMH survey tools used by an administrator or clinical lead to collect data at the practice level for evaluation and/or quality improvement during PCMH implementation. We assessed each tool’s coverage of PCMH content, standards, and requirements. 

 We found that 3 tools (Patient-Centered Medical Home Assessment [PCMH-A], Primary Care Assessment Tool—Facility Edition, and Medical Home Care Coordination Survey–Healthcare Team [MHCCS-H]) are actionable for quality improvement. PCMH-A assesses the broadest array of practice capabilities and includes items pertaining to all National Committee for Quality Assurance PCMH standards. MHCCS-H was the only tool to contain items on comprehensiveness of care. There was variation in emphasis on main domains, with some content areas covered by only 1 tool. 

There is currently little evidence on which PCMH tools are associated with improved quality outcomes, as relatively few longitudinal studies have been conducted. Of the 5 tools we reviewed, only PCMH-A and MHCCS-H impose a light administrative burden (less than 10 minutes to complete) and can identify specific actions to improve a given practice capability. Each tool is lacking in a particular content area: PCMH-A, for example, lacks items on comprehensiveness of care, whereas MHCCS-H lacks items addressing access to care.

The American Journal of Accountable Care. 2017;5(4):e8-e18

The patient-centered medical home (PCMH) model is promoted as a way to transform US primary care practices.

 It incorporates current best practices for access to care, prevention, chronic disease management, care coordination, and responsiveness to patients.

 Monitoring the transformation of primary care practices to PCMHs requires understanding the context, change process, patient experiences of care, and outcomes of care.

 Even after obtaining recognition from one of the several organizations that grant “official” PCMH status, practices continue their transformation into a PCMH.

Practices, networks, and federally qualified health centers need an evaluative and quality improvement (QI) tool to collect data and conduct real-time analyses that help them understand their level of “medical homeness,” track progress toward transformation goals, and identify possible improvements. Practice coaches, payers, and pilot or demonstration sponsors may also use these tools. Most primary care practices cannot engage in a third-party evaluation or large-scale demonstration with medical record audits, site visits, and staff surveys, but need a practical survey tool they can use to gauge achievement of PCMH standards. 

Despite their importance, information about tools for assessing PCMH progress is limited. Malouin et al

 found that the Primary Care Assessment Tool (PCAT), available in multiple formats, was the only tool that scored high on primary care features, as it was designed to assess both structural and process features of primary care, but their evaluation was conducted early in PCMH adoption and focused on a narrower set of components than currently delineated in PCMH definitions and standards.

 included tools primarily used for the PCMH recognition process, which itself requires adequate and appropriate documentation, rather than for practice self-improvement, research, evaluation, or quality measurement. 

This article reviews PCMH tools designed for evaluative and QI purposes to assist practices and researchers wanting to select a survey tool to track outcomes of PCMH transformation. We also assess how well the identified PCMH tools align with the 2014 and 2011 standards of the National Committee for Quality Assurance (NCQA), the organization responsible for the largest share of PCMH recognition across the United States.

Using traditional bibliographic databases (PubMed, Google Scholar, and PsycInfo), we systematically reviewed the peer-reviewed literature on patient-centered healthcare to identify surveys or tools designed to evaluate or monitor the process of becoming a PCMH. We also searched the Agency for Healthcare Research and Quality’s online primary care measure directory. Search terms included “patient-centered medical home” OR “PCMH” OR “medical home” OR “care coordination” OR “team-based care” AND “questionnaire*” OR “telephone” OR “phone” OR “survey” OR “surveys” OR “tool” OR “tools.” We limited our search to English-language articles published since 1990 on adult populations. We identified additional resources through reference citations. 

Our primary inclusion criteria were that a tool: 1) was designed as a survey with questions and response options, 2) was publicly available, 3) measured half or more of the PCMH domains (ie, access, team-based care [including continuity of care], population health management, care management and support [including shared decision making and self-care/self-management support], care coordination [including care transitions], and performance measurement and QI), 4) was based on 2014 PCMH standards of care, 5) was designed for QI or evaluation (not for PCMH recognition), 6) focused on the practice (not the health system) as the unit of change, and 7) did not require an external surveyor or site visit but could be administered by practice site administrators or evaluators.  

We identified 20 potential survey tools and abstracted details on each tool’s identifying information, including whether it was designed for recognition (for a national organization like NCQA or state agencies), evaluation, or QI; whether it assessed at the practice level; form (eg, paper or web); who completes the tool; and publications documenting its reliability. We estimated time burden of a tool by counting the number of questions overall and by type (eg, closed- or open-ended questions).

 Once tools not meeting these initial criteria were excluded, we coded each question in the remaining tools by content domain and related NCQA PCMH standard and element.

 We chose the NCQA PCMH standards because they are the most widely used standards, with recent estimates suggesting 15% to 18% of primary care physicians work in a NCQA-recognized PCMH.

 We created a summary worksheet to record the percentage of items in each content domain and the percentage of items related to each of the current (2014) and previous (2011) NCQA PCMH standards and elements. The project was approved by RAND Corporation’s institutional review board (Number: FWA00003425).

 provides inclusion and exclusion information on all 20 identified tools. Of these, we excluded 3 that were not publicly available, 1 that was based on 2008 NCQA PCMH standards,

 4 that were not designed for evaluation, 1 that was not a survey, 3 that focused on the health system rather than the practice, 2 that had limited PCMH content, and 1 that required an external surveyor and site visit. 

We were left with 5 current publicly available tools designed to collect data at the practice level by an administrator or clinical lead for practice improvement, research, evaluation, or quality measurement: Medical Home Index-Long Version (MHI-LV) and -Short Version (MHI-SV), Medical Home Care Coordination Survey—Healthcare Team (MHCCS-H), Primary Care Assessment Tool–Facility Edition (PCAT-FE), and Patient-Centered Medical Home Assessment (PCMH-A) tool. 

shows details of these 5 tools. All except PCAT-FE have a light (less than 10 minutes to complete) or medium (approximately 20 minutes) administrative burden, with zero monetary cost and web-downloaded availability. Based on the Berry

 method of estimation, completion times for the tools range from 8 to 37 minutes (for 36 to 167 items). None require self-reported documentation or verification of responses. For 4 of these tools, a key clinical lead at the practice responds; for the PCAT-FE, a system administrator does so. Both the MHCCS-H and the PCMH-A also recommend that a multidisciplinary team of practice staff (including primary care physicians, physician assistants, nurses, and clinical administrative staff) collectively complete 1 survey per practice. 

Answer format varies by tool, leading to different levels of data actionability. To be actionable, a survey question must outline specific practices or elements that can be identified and changed/improved, link the specific information to a numerical value/score, and cover no more than 1 practice or element.

We judged that the PCMH-A, PCAT-FE, and MHCCS-H are generally actionable for QI, whereas the other 2 tools have actionability limitations. The PCMH-A uses a numerical 1 to 12 rating based on the extent to which recommended practices and behaviors are implemented, divided across 4 levels with specific actions or practices described for each item; it is also sensitive to practice change over a time period as short as 6 months.

 The PCAT-FE and MHCCS-H both have items that are specific to 1 action and align with the response scales, making them fully actionable. The MHCCS-H has only 1 ambiguous item (regarding primary care team roles). By contrast, the MHI-LV and MHI-SV do not help the user determine specific practices to implement. For example, the MHI-LV asks a respondent to rate whether there is “partial” or “complete” implementation of an item with several components of patient- and family-centered care; an answer of “partial” does not allow understanding of which specific component needs further work. Burton et al

 similarly documented these actionability problems with the Medical Home Index (MHI) tool. 

All 5 tools have detailed scoring instructions and include a summary-score calculation. The PCMH-A asks teams to rate their care delivery in 36 key areas associated with the 8 change concepts. Each item is scored from 1 to 12 based on the extent to which recommended practices and behaviors are implemented. The MHI-LV has groups of items that are considered “themes,” with a total of 25 items using an 8-point scale where “partial” mastery of the most basic item equals 1 point and “complete” mastery of the most advanced of the 4 items equals 8 points. The MHI-SV is a 10-item survey that scores a practice on a continuum of care across 3 levels reflecting the degree to which it has achieved components of a medical home, scored on a 1 to 5 scale. Level 1 is “good, responsive primary care”; Level 2 is “pro-active primary care (in addition to Level 1)”; and Level 3 is “primary care at the most comprehensive levels” (in addition to Levels 1 and 2). The reporter is asked to respond to each of the 10 indicators with: Level 1, Level 2—partial, Level 2–complete, Level 3–partial, or Level 3–complete. The MHCCS-H has 8 care coordination domains and a total of 35 items (34 items plus 1 global-rating item), with 5-point categorical-response scales, plus 2 responder-characteristic items. The PCAT-FE is scored by taking the average score (on a 1 to 4 scale) of each domain.

Reliability and validity have been estimated for the MHI-LV

 examined interrater reliability between 2 project staff and between practices and project staff and the internal consistency of MHI domains and themes. The intraclass correlation coefficients between the summary scores of the interviewers was .98 and between the scores of the 2 interviewers and scores of the practices, .97. Standardized coefficient alphas for the 6 domains of the MHI ranged from .81 to .91, and the overall standardized alpha was .96. In the sample of practices studied, the MHI was an internally consistent instrument with acceptable reliability and support for validity for use by pediatric primary care practices to assess PCMH implementation. For the MHCCS-H, Zlateva et al

 conducted analysis to examine internal consistency, reliability and validity of the MHCCS-H using structural equation modeling. All domains had Cronbach’s coefficient alphas above 0.80.

described a longitudinal study of 65 safety-net practices from 5 states participating in a national demonstration program for PCMH transformation that compared 15 practices that achieved PCMH recognition with 29 practices that were actively pursuing NCQA recognition practice (but had not achieved it) and 21 practices that were not pursuing NCQA recognition but that used the PCMH-A. They found that mean overall PCMH-A scores increased overtime, indicating that the PMCH-A was sensitive to change over time (7.2 in March 2010 to 9.1 in September 2012; 

 <.01). Increases were statistically significant for each change concept (

 <.05). Implementers of PCMH change (ie, facilitators) agreed with scores 82% of the time. NCQA-recognized sites (n = 15) had significantly higher PCMH-A scores than sites that were not yet recognized. Sites that had completed more transformation activities and progressed over defined tiers reported significantly higher PCMH-A scores. They concluded that the PCMH-A was sensitive to change over time and provided an accurate reflection of practice transformation.

 shows the emphases given to different content domains in the PMCH tools. We started with the content areas identified by the Burton

 review of PCMH tools and added several newly identified domains. There were 24 content areas identified across the PCMH tools. Content domains receiving the most emphasis were: 1) use of care plan in care management, 2) coordination of care, 3) provide self-care support and community resources, 4) quality improvement, and 5) enhance access and continuity using medical records. Each tool had a main emphasis: PCMH-A targeted continuity of care; MHI-LV, community resources; MHI-SV, coordination of care; and MHCCS-H, the care plan. Some content areas were only covered by 1 tool: comprehensiveness of care items (defined as the breadth of services a practice offers to address any health problem at any given stage of patient’s life [eg, the primary care practice/health center has behavior change interventions readily available for patients as part of routine care, using a 5-point agreement scale]) by MHCCS-H; empanelment, evidence-based care, quality measurement, and quality reporting by PCMH-A; and standard non—PCMH care provided (used as a base level to PCMH care for patient and family involvement) and adherence to current laws by MHI-LV. 

Our review mapped PCMH tools to both the NCQA 2011 and 2014 standards. 

 shows the relative content emphases for each of the 2011 NCQA PCMH standards and elements. The 2011 NCQA PCMH standards receiving the most emphasis across tools were: a) Standard 4: Provide Self-Care and Community Support; b) Standard 3: Plan and Manage Care; c) Standard 1: Enhance Access and Continuity; d) Standard 5: Track and Coordinate Care; e) Standard 6: Measure and Improve Performance; and f) Standard 2: Identify and Manage Patient Populations. 

Although many standards and elements remained the same from the 2011 to the 2014 NCQA PCMH standards, there are new areas of focus. The 2014 version focuses more on care management. In previous standards, practices needed to identify high-risk patients; 2014 requires also focusing on patients who would benefit from care management. NCQA 2014 includes a new standard on team-based care, giving more emphasis to providing patients with information—proactively orienting them to the practice, helping them choose a physician, and explaining the practice’s use of evidence-based guidelines. This team-based care element not only ensures that the whole care team works at the top of their licensure (included in 2011), but also improves the focus on the individual patient. 

The population health standard in 2014 incorporates many Stage 2 Core and Menu Meaningful Use requirements, use of electronic medical records, and a comprehensive health assessment. This is not a new NCQA standard, but the documentation requirements have changed. For the 10 factors, NCQA requires the practice either has a report or conducts chart audits using NCQA’s tool, the Record Review Workbook. Practices had been allowed to show examples of their comprehensive health assessments. Behavioral health has been integrated into numerous standards, where it had previously not been a focus. 

In 2011, NCQA wanted practices to develop the habit of measuring performance. As a result, the 2014 standards identify more areas for improving practice performance. Additionally, in the 2014 standards, NCQA dropped the 2011 element 4B (Provide Referrals to Community Resources); the 2011 element 6F (Report Data Externally) was incorporated into the 2014 element 6F (Report Performance); and the 2011 element 6A (Measure Performance) was divided into 2 separate elements: 2014 element 6A (Measure Clinical Quality Performance) and element 6B (Measure Resource Use and Care Coordination). NCQA has published a full crosswalk of 2011 to 2014 standards.

characterizes how well each assessment covers the 2014 PCMH standards. Those receiving the greatest emphasis across tools were: a) Standard 4: Care Management and Support; b) Standard 2: Team-Based Care; c) Standard 3: Population Health Management; d) Standard 5: Care Coordination and Care Transitions; e) Standard 6: Performance Improvement and Quality Improvement; and f) Standard 1: Patient-Centered Access. The PCMH-A had the broadest array of practice capabilities assessed (Table 3) and contained items pertaining to all 6 2014 NCQA PCMH standards and 27 elements (Table 5). The MHI-LV, MHI-SV, and MHCCS-H lacked items addressing the “measurement and improvement of performance” (PCMH Standard 6). The MHCCS-H lacked items addressing “access to care” during office hours and those addressing after-hours or electronic access within “enhance access and continuity” (PCMH Standard 1).

The differences in the 2011 and 2014 standards did not influence our content coding or coverage of content to PMCH standards, except for behavioral health, which was new to the 2014 standards. Items related to behavioral health were included in the PCMH-A and PCAT-FE, but not in the MHI or MHCCS-H. 

We searched for associations of the tools to other quality measures. We found studies using the PCMH-A to track PCMH transformation,

 but no articles specifically tying PCMH-A results to other quality measures. This may be because there has been insufficient time since its development for these studies to be performed. For the MHI-LV, we found limited and mixed results from 3 studies

; 2 showed associations with fewer hospitalizations and lower nonurgent preventable emergency department (ED) use by children and 1 showed associations with higher chronic care management subscores to better provider communication and follow-up on test results but worse access to care among adults. Another study

 found higher levels of “medical homeness” (based on the MHI) associated with lower nonurgent, preventable, or avoidable ED use by publicly insured children. To date, there is little evidence showing which PCMH recognition tool(s) may be associated with improved quality outcomes.

Many healthcare organizations and payers are investing in a PCMH tool for QI or evaluation of medical homeness and progress toward PCMH standards. With little evidence about which PCMH-recognition tool is associated with improved quality outcomes, payers and practices must decide what role quality measurement should play in their PCMH journey, how much administrative burden to place on practices to collect data, what data to use for QI related to PCMH, and how to track practice-level progress toward PCMH transformation. 

We found that the MHI-LV, PCMH-A, and MHCCS-H tools have been subjected to relatively more evaluation and validation and impose only a light or medium administrative burden to complete. The PCMH-A, PCAT-FE, and MHCCS-H (except for 1 of its 37 items) are actionable for quality improvement, whereas the 2 other tools have limitations in their ability to identify specific improvements. The PCMH-A is documented

 to be sensitive to changes in practice behavior over a time period as short as 6 months, indicating it can track QI progress and a practice’s trajectory of PCMH change. All 5 tools have detailed scoring instructions, including a summary-score calculation.

No single tool covers all relevant PCMH content, and some content is only found on 1 tool. Practice capabilities in coordination of care, care plans, and support of self-management and community resources receive the greatest emphasis. None of the reviewed tools covers all 4 main attributes of primary care: comprehensiveness of care, coordination of care, continuity, and access to care.

Because there is no evidence on what combination of practice capabilities produces the best outcomes for patients, practices and payers must decide which medical home aspects they most wish to measure and whether an available tool contains those specific practice capabilities. They must weigh these content decisions against the administrative burden they impose on physicians and clinical staff.

The decision on content includes not only practice capabilities and content areas covered in the PMCH tool but also whether the tool aligns with recognition standards and criteria a practice is pursuing. The NCQA standards are the most widely followed PCMH recognition standards. The PCMH-A had the broadest array of practice capabilities assessed and contains items pertaining to all 6 of the 2011 NCQA PCMH standards/elements and all 6 of the 2014 standards/elements. The MHI-LV and the MHCCS-H lacked items addressing the measurement and improvement of performance (PCMH Standard 6). The MHCCS-H also lacked items addressing access to care during office hours and after hours or electronic access within “enhance access and continuity (2011)/Patient-centered access (2014)” (PCMH Standard 1). Altogether, the PCMH-A could be the most widely applicable tool for most practices transforming into PCMHs, given its alignment with NCQA standards and broad content. As noted previously, the findings of this study are applicable to practices using the NCQA standards, which are the most widely used; practices using other sets of standards may see slight differences in content alignment.  

The decision on which tool to administer for tracking PCMH transformation must weigh administrative burden (ie, cost, time, need for documentation), reliability and validity of data collected, actionability of data, practice capabilities measured, and alignment with PCMH recognition standards. The PCMH-A and MHCCS-H have a light administrative burden and are actionable for quality-improvement purposes. The MHCCS-H has data available on its reliability, whereas the PCMH-A has shown sensitivity to changes over time in practices that have achieved (or have not achieved) NCQA PCMH recognition. Each of the tools we reviewed has a different constellation of practice capabilities and emphases. The PCMH-A assesses the broadest array of capabilities, whereas the MHCCS-H is the only tool with items on capability to provide comprehensiveness of care. 

RAND Corporation (DDQ), Santa Monica, CA; RAND Corporation (ZSP), Boston, MA; Division of General Internal Medicine and Health Services Research, University of California, Los Angeles (RDH), Los Angeles, CA.

This work was supported by a cooperative agreement from the Agency for Healthcare Research and Quality (AHRQ) [Contract number U18 HS016980].

 The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Concept and design (DDQ); acquisition of data (DDQ, ZSP, RDH); analysis and interpretation of data (DDQ, ZSP); drafting of the manuscript (DDQ, ZSP); critical revision of the manuscript for important intellectual content (DDQ, ZSP, RDH); obtaining funding (RDH); and administrative, technical, or logistic support (DDQ, ZSP).

Denise D. Quigley, PhD, RAND Corporation, 1776 Main St, Santa Monica, CA 90407-2138. E-mail: quigley@rand.org.

1. Bitton A, Martin C, Landon BE. A nationwide survey of patient centered medical home demonstration projects. 

. 2010;25(6):584-592. doi: 10.1007/s11606-010-1262-8.

Dentzer S. Reinventing primary care: a task that is far ‘too important to fail’. 

. 2010;29(5):757. doi: 10.1377/hlthaff.2010.0410.

Bodenheimer T, Lorig K, Holman H, Grumbach K. Patient self-management of chronic disease in primary care. 

Ferrer RL, Hambidge SJ, Maly RC. The essential role of generalists in health care systems. 

Grumbach K, Bodenheimer T. A primary care home for Americans: putting the house in order. 

Articles

The American Journal of Accountable Care

We reviewed operational details and content of tools designed to evaluate patient-centered medical home (PCMH) transformation. These tools assist practice leaders in understanding specific information about the process and progress of becoming a PCMH.

Tools to Gauge Progress During Patient-Centered Medical Home Transformation

Optimal transitions of care involve team-based collaboration across inpatient and outpatient settings and have the potential to help achieve the Quadruple Aim.&ensp;his article describes a quality improvement pilot implemented by a safety-net academic inpatient service and partner community-based health centers to improve patient outcomes and provider experience in the care of underserved hospitalized adults.

 Inefficient nonstandardized care coordination led to collaborative quality improvement between an inpatient service and partner primary care clinics in the San Francisco Health Network. Email-based care transitions were implemented to facilitate care team communication and improve linkage to primary care for hospitalized patients. Patient outcomes and provider work experience were assessed over 2 years.

 Sustained improvements in care coordination among the intervention cohort were observed. Patients demonstrated a statistically significant 14% increase in attendance at follow-up appointments within 7 days of discharge.&ensp;cross the health system, follow-up within 7 days of discharge reduced 30-day all-cause readmissions by 58% during the study period. A majority of providers reported that the intervention improved the ease and efficiency of care transitions and increased their ability to address patients’ multidisciplinary needs. Among a survey cohort of outpatient providers involved in the pilot, 93% recommended that all hospital services adopt a similar care transitions system.

A pilot of e-mail—based care transitions increased linkage to primary care following hospitalization and improved providers’ experience and satisfaction with care transitions. This process was recommended for systemwide use by over 90% of surveyed outpatient providers. An electronic health record–triggered care transitions system is in development for use by all hospital services.

The American Journal of Accountable Care. 2017;5(4):e19-e22 

Over the past decade, team-based models of care have become a best practice in primary care

 Care coordination among teams has been cited as a critical factor in achieving the Triple Aim.

 As the numbers of tasks and care providers increase, timely and accurate communication becomes both more critical and more difficult to execute,

 with potential negative effects on workforce satisfaction—the fourth component of the Quadruple Aim.

Although recent study findings support the use of electronic care coordination

 electronic health record (EHR) “alert fatigue” has been described as a negative outcome among primary care providers (PCPs).

 The importance of early postdischarge follow-up for high-risk patients has also been recently described.

 We report on an email-based care transitions intervention that has improved linkage to primary care after discharge and providers’ experiences across the inpatient and outpatient settings.  

Our health system, the San Francisco Health Network (SFHN), cares for medically and socially vulnerable patients across a countywide system of geographically and operationally diverse inpatient and outpatient settings.

 The University of California San Francisco (UCSF) Family Medicine Inpatient Service (FMIS) is an academic medical inpatient service based at Zuckerberg San Francisco General Hospital (ZSFG), the acute care hospital of the SFHN. FMIS has an ongoing partnership with 6 SFHN primary care health centers and cares for more than 1500 socially vulnerable and medically complex inpatients each year. Thirty percent of inpatients have a primary admitting diagnosis of organ failure, more than 50% have at least 1 chronic organ failure, and nearly 100% are affected by 1 or more of the following: food insecurity, economic poverty, homelessness or housing instability, cognitive impairment, substance use disorders, severe mental illness, and community-level violence (UCSF FMIS, unpublished demographic data, FY2015-2016). 

Prior to the intervention described in this article, there was no standard workflow for postdischarge linkage to primary care in SFHN or for communication between inpatient and outpatient providers during admission to ZSFG. Care transitions in SFHN are further complicated by different noninteroperable EHRs in the hospital and at primary care clinics, increased frequency of patient handoffs among hospital housestaff as a result of Accreditation Council for Graduate Medical Education duty hours reform, and a complex interrelation of providers from the academic and public health sides of the health network.

At baseline, FMIS housestaff were spending more than 2 hours per patient on basic care coordination tasks, including 30 to 45 minutes arranging a primary care follow-up appointment by phone (UCSF FMIS, unpublished service workflow data, July 2013). Inpatient and outpatient physicians consistently noted difficulty with real-time communication of feedback to the inpatient service medical directors. Consistently less than 40% of patients from the inpatient service attended follow-up visits after hospital discharge (ZSFG Care Transitions Taskforce, unpublished data, CY2013-2014).

To improve care transitions, a quality improvement process was initiated through key stakeholder interviews with inpatient, primary care, and health system leadership. Due to noninteroperable inpatient and outpatient EHR systems, secure team-based email was selected as the method of care coordination and follow-up scheduling. Care transitions emails are sent by the inpatient team at admission and discharge through a secure server, and the recipients include inpatient team members and primary care team members. Each primary care clinic selected its own recipient list, which included the PCP plus other designated staff (eg, clerical, nursing, behavioral health, etc).

]) includes patient information (name and medical record number), brief history and plan of care, expected date of discharge, and recommendation for timing of primary care follow-up.

 Primary care clinic staff, often in consultation with the PCP, reply to the admission email with a scheduled follow-up appointment based on the estimated date of discharge. This discharge email includes patient information, brief description of hospitalization, key medication changes,

 and items for immediate provider follow-up.

 Both emails contain inpatient providers’ email addresses and pager numbers to allow outpatient teams the option of asynchronous or real-time communication about further details of care coordination. 

Data on postdischarge follow-up and readmissions were extracted from the hospital EHR (Invision LCR; Siemens Corporation, Washington, DC). Statistical significance of the impact of standardized workflow on 7-day postdischarge follow-up was analyzed using χ

 tests to show overall difference between pre- and postintervention groups and McNemar’s test to show pre- and postintervention (in-group) differences to emphasize the impact of the intervention. A voluntary electronic survey was conducted among outpatient PCPs and inpatient housestaff to assess impact on workflow, including questions on ease, efficiency, time spent, and provider experience related to transitions of care. The survey response rates and data are reported directly. 

The email-based care transitions system was rolled out on March 3, 2014. Rates of attendance at postdischarge follow-up visits for the intervention cohort (FMIS patients; email-based care transitions) and the control cohort (all other hospital inpatient services; no email-based care transitions) were compared. Following implementation, more than 94% of patients in the intervention cohort were provided with a scheduled primary or specialty care follow-up appointment prior to discharge. During the second year of implementation, mean rates of patient attendance at follow-up within 7 days of discharge had increased to 53% from a baseline of 39% in the intervention cohort (n = 1301) (

). Rates of follow-up in the control cohort (n = 5322) were 32% at baseline and 38% during the intervention period. Using the χ

 method, email-based care transitions were associated with a statistically significant improvement in 7-day postdischarge follow-up (

 = 5.95 with df = 1). The difference between cohorts during the pre-intervention period was not significant (

 = 2.25). McNemar’s test showed a statistically significant difference between the pre- and post intervention groups of patients in the intervention cohort (

= .02; 95% CI, 1.002-1.8) but not in the control cohort (

 = 0.11; 95% CI, 0.9-1.6). A systemwide analysis of all-cause 30-day readmission rates during the intervention period showed that patients who followed up with primary care within 7 days of discharge had 58% fewer readmissions compared with those who did not (5% vs 12%).

Surveys were performed to assess the workflow and experience of inpatient housestaff (n = 7 respondents of 19 surveys sent) and outpatient providers (n = 30 respondents of 84 surveys sent) caring for patients in the intervention cohort with email-based care transitions. Inpatient residents used the system more than 80% of the time during the first month and gradually increased their usage to 97% with training and reminders. They reported improvements in ability to give the patient specific follow-up information (86%), communication with the outpatient team (67%), and ease of scheduling (80%) and a 50% decrease in time spent on care coordination. 

Eighty-six percent of outpatient PCPs reported receiving communication about admission and discharge within 1 to 2 days of each event. Primary care team members reported improvements in ease of communication with the inpatient team (82%), scheduling of follow-up appointments (72%), and ability to meet patients’ multidisciplinary needs (73%). Ninety-three percent of outpatient providers recommended that all inpatient services adopt this system of communication.

Following integration of email-based care transitions, we observed improvements in patient outcomes and provider work experience. Using standardized care transitions communication beginning at admission, inpatient providers were able to collaborate with primary care teams to create postdischarge plans, including follow-up appointments, and provide patients with a specific follow-up plan prior to discharge. This inpatient—outpatient collaboration resulted in a statistically significant improvement in attendance rates at postdischarge follow-up, which has a considerable potential impact on readmissions. During the intervention period, a separate analysis showed that SFHN patients with an established PCP who attended follow-up within 7 days of discharge had a 58% decreased rate of all-cause 30-day readmission (5% vs 12.3%). 

The use of this system also improved work efficiency and provider satisfaction related to care transitions among both inpatient and outpatient teams. A majority of inpatient and outpatient providers reported improvements in care coordination and in their ability to contact colleagues across the health system and a decrease in the amount of time spent doing this work. These gains may be of particular importance in health systems where distributed multidisciplinary teams collaborate to care for complex patients. We hypothesize that the improvements in ease and efficiency account for the high rate of uptake: the process incentivizes participation. Further investigation is needed to assess how efficiency gains in team-based care coordination might improve provider satisfaction and reduce burnout.

Throughout the study period, there was ongoing work across all SFHN primary care clinics to improve visit attendance, particularly for patients following discharge. The effect of such interventions should be seen for all patients and appear to be reflected in the overall trend toward improvement in both the intervention and control groups seen in the Figure. The statistically significant improvement in the intervention cohort likely reflects the impact of email-based care transitions. This is supported by the fact that statistically significant differences were observed 1) between control and intervention groups in the postintervention period and 2) between the cohorts of patients on the FMIS in the pre- and postintervention periods, but not between observed groups in the pre-intervention period, nor cohorts of patients in the control services before and during the intervention.

Potential advantages of this system include privacy, information access, and cost. Existing email servers provided secure institutional email in compliance with Health Insurance Portability and Accountability Act regulations. Email as a method is ubiquitous and provides an electronic record, which providers can return to after the hospitalization. Perhaps most importantly, this intervention succeeded for highly vulnerable patients in a large, geographically distributed public safety-net health system without an enterprise EHR system and did not require the multiple years or many millions of dollars required to fully implement such a system. 

Potential disadvantages of this system include the lack of direct integration into the EHR and the potential for “inbox fatigue” among providers. Lack of EHR integration was a motivating factor in creating this intervention, due to the use of separate noninteroperable EHRs in the inpatient and outpatient settings—and may be a common issue, especially in resource-limited systems that do not yet have an enterprise EHR. Although we did not directly assess provider burnout, and it would be useful to do so in future analyses, over 90% of responding outpatient providers recommended adopting this care transitions

intervention systemwide, signaling a high level of satisfaction.

Limitations of this pilot analysis include the lack of analysis and comparison of readmission rates among patients in the intervention and control cohorts. On multiple occasions during the study period, the SFHN conducted an inpatient repatriation program to ZSFG for patients hospitalized at out-of-network facilities. This program was not implemented consistently across patient cohorts and thus skewed readmission rates in a wide variation during multiple noncontiguous time periods, making it very difficult to control for effects. A future analysis should include readmission rates. An additional limitation of this initial pilot is the lack of assessment of more complex outcomes of care coordination, such as content of conversations between PCPs and inpatient teams, measures of trust between providers, presence of burnout symptoms, and overall job satisfaction. These would be useful areas for future investigation into the impact of this type of intervention. A third potential limitation is the lack of patient satisfaction or patient experience data, which may be relevant in assessing the impact of team communication on the inpatient care plan and robust follow-up from patients’ points of view.

A structured multidisciplinary email-based care transitions system was associated with multiple positive outcomes, including a statistically significant improvement in attendance at follow-up within 7 days of hospital discharge, improved provider-reported ease and efficiency of care team communication, and greater provider satisfaction with care coordination for hospitalized patients. Based on the success of this pilot, we have led the effort to integrate this workflow into the inpatient EHR, allowing providers across San Francisco’s safety-net medical community to participate in this communication system regardless of their home EHR (

). We believe that the practical application of cost-effective technology to care coordination has the potential to help the US healthcare system better achieve the Quadruple Aim of improving patient experience, increasing population health, improving the work experience of healthcare professionals, and reducing costs.

The authors wish to thank Dr. Danielle Hessler Jones for reviewing the statistical methods used in this article.

Zuckerberg San Francisco General Hospital (JC, KO, SG), San Francisco, CA; Department of Family and Community Medicine (JC), and Department of Medicine (KO, SG), University of California, San Francisco, CA.

No external funding was used in the creation of this article or in the quality improvement process described.

 Ms Oza receives salary support from Gilead Sciences for projects unrelated to the subject matter of this article. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article. 

Concept and design (JC, SG); acquisition of data (JC, KO); analysis and interpretation of data (JC, KO); drafting of the manuscript (JC); critical revision of the manuscript for important intellectual content (JC, KO); statistical analysis (JC); administrative, technical, or logistic support (JC, SG); and supervision (SG). 

Jack Chase, MD, FAAFP, FHM, Zuckerberg San Francisco General Hospital, 995 Potrero Ave, Ward 83, San Francisco, CA 94110. E-mail: jack.chase@ucsf.edu. 

1. Bodenheimer T, Ghorob A, Willard-Grace R, Grumbach K. The 10 building blocks of high-performing primary care. 

. 2014;12(2):166-171. doi: 10.1370/afm.1616. 

2. O’Leary KJ, Sehgal NL, Terrell G, Williams MV; High Performance Teams and the Hospital of the Future Project Team. Interdisciplinary teamwork in hospitals: a review and practical recommendations for improvement. 

3. Hansen LO, Greenwald JL, Budnitz T, et al. Project BOOST: effectiveness of a multihospital effort to reduce rehospitalization. 

. 2013;8(8):421-427. doi: 10.1002/jhm.2054.

4. Jack BW, Chetty VK, Anthony D, et al. Reengineered hospital discharge program to decrease rehospitalization: a randomized trial. 

. 2009;150(3):178-187. doi: 10.7326/0003-4819-150-3-200902030-00007.

5. Craig C, Eby D, Whittington J. Care coordination model: better care at lower cost for people with multiple health and social needs. Institute for Healthcare Improvement website. ihi.org/resources/Pages/IHIWhitePapers/IHICareCoordinationModelWhitePaper.aspx. Published 2011. Accessed March 27, 2017.

6. Roberts SR, Crigler J, Ramirez C, Sisco D, Early GL. Working with socially and medically complex patients: when care transitions are circular, overlapping, and continual rather than linear and finite. 

. 2015;37(4):245-265. doi: 10.1097/JHQ.0000000000000006.

7. Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. 

. 2014;12(6):573-576. doi: 10.1370/afm.1713.

8. Walsh C, Siegler EL, Cheston E, et al; Informatics Intervention Research Collaboration (I2RC). Provider-to-provider electronic communication in the era of meaningful use: a review of the evidence. 

. 2013;8(10):589-597. doi: 10.1002/jhm.2082.

9. Horwitz LI, Jenq GY, Brewster UC, et al. Comprehensive quality of discharge summaries at an academic medical center. 

. 2013;8(8):436-443. doi: 10.1002/jhm.2021.

10. Lindquist LA, Yamahiro A, Garrett A, Zei C, Feinglass JM. Primary care physician communication at hospital discharge reduces medication discrepancies. 

. 2013;8(12):672-677. doi: 10.1002/jhm.2098.

11. Murphy DR, Reis B, Sittig DF, Singh H. Notifications received by primary care practitioners in electronic health records: a taxonomy and time analysis. 

. 2012;125(2):209.e1-7. doi: 10.1016/j.amjmed.2011.07.029.

Jackson C, Shahsahebi M, Wedlake T, DuBard CA. Timeliness of outpatient follow-up: an evidence-based approach for planning after hospital discharge. 

. 2015;13(2):115-122. doi: 10.1370/afm.1753.

13. Goldman LE, Sarkar U, Kessell E, et al. Support from hospital to home for elders: a randomized trial. 

. 2014;161(7):472-481. doi: 10.7326/M14-0094.

A pilot of email-based care transitions between hospital and primary care teams improved patient attendance at follow-up visits, provider satisfaction, and work efficiency.

Email-Based Care Transitions to Improve Patient Outcomes and Provider Work Experience in a Safety-Net Health System

Clinical and economic burdens are placed on the US healthcare system when target goals are not met in diabetes care. In 2012, total costs (direct and indirect) associated with diabetes in the United States were $245 billion dollars.

 In a presentation at the 2017 Fall ACO & Emerging Healthcare Delivery Coalition

on October 26th, 2017, Kari Uusinarkaus, MD, discussed the economic impact of type 2 diabetes (T2D). This presentation focused on the prevalence, costs, and consequences of not meeting glycemic goals in patients with T2D.

Approximately 16.5 million people in the United States have T2D.

 The majority of adult patients (90%-95%) with diabetes have T2D.

 Risk factors associated with the occurrence of T2DM include ethnicity (eg, American Indians, African Americans, Hispanics/Latinos, Asians, Native Hawaiians, Pacific Islanders), male gender, older age, obesity, family history, gestational diabetes, impaired glucose metabolism, and physical inactivity. 

Diabetes increases the risk of developing complications (eg, neurological, peripheral vascular, cardiovascular, renal, endocrine/metabolic, ophthalmic).

 More than 60% of patients with T2D die from cardiovascular disease.

 From 1998 to 2011, the overall death rate among patients with T2D was 38.64 per 1000 person-years.

 The risk of death in patients with T2D increased in patients who were diagnosed at a younger age, who lacked glycemic control, and/or who experienced greater severity of renal complications.

Both the American Diabetes Association (ADA) and the American Association of Clinical Endocrinologists/American College of Endocrinology (AACE/ACE) have established target goals to decrease cardiovascular comorbidities associated with T2D.

 These recommended targets are described in 

Achieving glycemic goals reduces long-term cardiovascular disease and diabetes-related complications. A prospective UK database study found that intensive glycemic control (to achieve glycated hemoglobin [A1C] goals <7%) significantly decreased rates of microvascular and neuropathic complications in patients with T2D.

16% reduction in cardiovascular disease events (combined fatal or nonfatal myocardial infarction and sudden death)

13% reduction in myocardial infarction after 10 years of follow-up 

27% reduction in all-cause mortality after 10 years of&thinsp;follow-up.

An alarming number of patients with T2D are not meeting glycemic goals and other important goals (eg, blood pressure, body mass, cholesterol). According to a study reported in 

that focused on patients taking metformin and a sulfonylurea, 31% to 92% of patients failed to reach recommended glycemic goals.

 In particular, more than 70% of patients were not meeting the recommended A1C goal of less than 7%. Data from the National Health and Nutrition Examination Survey found that only 30% to 50% of US patients with diabetes met the individualized targets for glycemic control, blood pressure, and/or lipid control.

 Additionally, 40% to 50% of patients were not taking advantage of preventive measures (eg, diabetes education, vaccinations, and annual dental examinations). 

Hypoglycemia is a major rate limiting factor in the glycemic management of T2D.

 Diabetes control and quality of life can be severely compromised in patients with hypoglycemia unawareness or hypoglycemia-associated autonomic failure. For example, a history of severe hypoglycemia among older adults with T2D is associated with increased risk of dementia. In 2017, the ADA altered the glycemic target to 80 to 130 mg/dL (4.4-7.2 mmol/L) as a method to limit overtreatment and to provide a safety margin for patients titrating glucose-lowering drugs such as insulin.

Improved diabetes control with oral therapies can lead to decreased costs of care.

, goal attainment was associated with significantly lower total diabetes-related healthcare costs per person per year.

 The results were also consistent in patients who were older than 65 years. As shown in 

, a meta-analysis of 56 studies from 20 countries also found that many of the ADA-recommended interventions were cost-saving or cost-effective.

T2D affects more than 16 million patients in the United States and the total economic burden is high.

 Achieving glycemic and related diabetes goals decreases long-term cardiovascular disease and complication rates.

 An increase in goal attainment is associated with decreased diabetes-related healthcare costs.

1. National diabetes statistics report, 2014. CDC website. stacks.cdc.gov/view/cdc/23442/cdc_23442_DS1.pdf. Published July 2014. Accessed December 4, 2017.

2. Dall TM, Zhang Y, Chen YJ, Quick WW, Yang WG, Fogli J. The economic burden of diabetes. 

 2010;29(2):297-303. doi: 10.1377/hlthaff.2009.0155.

3. American Diabetes Association. Economic costs of diabetes in the U.S. in 2012. 

2013;36(4):1033-1046. doi: 10.2337/dc12-2625.

4. Duckworth W, Abraira C, Moritz T, et al; VADT Investigators. Glucose control and vascular complications in veterans with type 2 diabetes. 

2009;360(2):129-139. doi: 10.1056/NEJMoa0808431.

5. Tancredi M, Rosengren A, Svensson AM, et al. Excess mortality among persons with type 2 diabetes. 

2015;373(18):1720-1732. doi: 10.1056/NEJMoa1504347.

6. Handelsman Y, Bloomgarden ZT, Grunberger G, et al. American Association of Clinical Endocrinologists and American College of Endocrinology - clinical practice guidelines for developing a diabetes mellitus comprehensive care plan - 2015. 

2015;21(suppl 1):1-87. doi: 10.4158/EP15672.GL.

7. American Diabetes Association. 5. glycemic targets. 

. 2016;39(suppl 1):S39-S46. doi: 10.2337/dc16-S008.

8. UK Prospective Diabetes Study (UKPDS) Group. Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33) [published erratum appears in 

1998;352(9131):837-853. doi: 10.1016/S0140-6736(98)07019-6.

9. Lafeuille MH, Grittner AM, Gravel J, et al. Quality measure attainment in patients with type 2 diabetes mellitus. 

10. Ali MK, Bullard KM, Saaddine JB, Cowie CC, Imperatore G, Gregg EW. Achievement of goals in U.S. diabetes care, 1999-2010. 

 2013;368(17):1613-1624. doi: 10.1056/NEJMsa1213829.

11. Li R, Zhang P, Barker LE, Chowdhury FM, Zhang X. Cost-effectiveness of interventions to prevent and control diabetes mellitus: a systematic review. 

2010;33(8):1872-1894. doi: 10.2337/dc10-0843.

A presentation at the fall live meeting of the ACO & Emerging Healthcare Delivery Coalition® focused on the clinical and economic consequences of not meeting glycemic goals in patients with type 2 diabetes.

When Goals Are Not Met in Diabetes Care

Institutions participating in the Medicare Bundled Payments for Care Improvement (BPCI) initiative invest significantly in efforts to reduce readmissions and costs for patients who are included in the program. Eligibility for the BPCI initiative is determined by diagnosis-related group (DRG) classification. The implications of this methodology for chronic diseases are not known. We hypothesized that patients included in a BPCI initiative for chronic obstructive pulmonary disease (COPD) would have less severe illness and decreased hospital utilization compared with those excluded from the bundled payment initiative. 

We sought to determine the clinical characteristics and outcomes of Medicare patients admitted to the University of Alabama at Birmingham Hospital with acute exacerbations of COPD between 2012 and 2014 who were included and excluded in a BPCI initiative. Patients were included in the analysis if they were discharged with a COPD DRG or with a non-COPD DRG but with an 

International Classification of Diseases, Ninth Revision

Diagnosis-related group coding determines eligibility for many Medicare bundled payment initiatives. This approach excluded many patients with chronic obstructive pulmonary disease likely to benefit while including others without the disease.

December 2017

Tools to Gauge Progress During Patient-Centered Medical Home Transformation

Email-Based Care Transitions to Improve Patient Outcomes and Provider Work Experience in a Safety-Net Health System

When Goals Are Not Met in Diabetes Care

Implications of DRG Classification in a Bundled Payment Initiative for COPD

Factors Associated With Timeliness in Academic General Surgery Clinics: A Prospective Quality Assessment

Transitioning Community Hospitals to Value-Based Care: Lessons From Massachusetts

Enhancing Dementia Care Through Digital Health