David W. Bates, MD, MSc

 To determine if electronic health record (EHR) tools and patient engagement can improve the quality of chronic kidney disease (CKD) care.

 We enrolled 153 primary care physicians caring for 3947 high-risk and 3744 low-risk patients with stage III CKD across 13 ambulatory health centers in eastern Massachusetts. Intervention physicians received a set of electronic alerts during office visits recommending risk-appropriate CKD care. Patients of intervention physicians also received tailored educational mailings. For high-risk patients, we assessed for a visit with a nephrologist and prescription of an angiotensin-converting enzyme (ACE) inhibitor or angiotensin receptor blocker (ARB) during the 12-month study period. For low-risk patients, we assessed for a urine microalbumin screening and prescription of an ACE inhibitor or ARB during the 12-month study period.

 Among high-risk patients, those in the intervention arm were significantly more likely to have an office visit with a nephrologist compared with those in the control arm (45% vs 34%; 

<.001). Among low-risk patients, those in the intervention arm were significantly more likely than those in the control arm to have received urine microalbumin testing (45% vs 21%; 

<.001). There was no difference between the intervention and control arms in rates of prescription of an ACE inhibitor or ARB in either the high-risk patient group (76% vs 79%; 

= .17) or the low-risk patient group (64% vs 65%; 

 A combined program of EHR tools and patient engagement improved some areas of CKD care, but substantial gaps remain.

We randomized 153 primary care physicians caring for nearly 8000 patients with chronic kidney disease (CKD) to receive an intervention that combined electronic decision support tools, patient engagement materials, and collaboration between primary care and nephrology. 

Awareness of CKD is low among patients and physicians.

The intervention was favorably received by physicians and patients. 

We developed a practical population-based approach to assist accountable care organizations that are seeking to engage collaboration between primary care and specialty care to improve the quality of CKD care.

Chronic kidney disease (CKD) affects more than 25 million Americans, or more than 10% of the adult population.

 Effective management of moderate-stage CKD is needed to reduce the high mortality rates and extensive costs associated with progression to more advanced kidney failure.

Many challenges exist to improving care for CKD,

 which remains a frequently unrecognized condition by both primary care physicians (PCPs)

 Just 12% of patients with stage III or IV CKD are aware of their diagnosis, and just 63% of PCPs can correctly identify the presence of CKD.

 However, monitoring for disease progression, using appropriate medications, and involving nephrologists early can improve CKD outcomes.

This highlights the need for healthcare systems to develop a systematic approach to treating this condition that supports primary care providers and nephrologists.

 Electronic health records (EHRs) present an opportunity to deliver appropriate care by identifying patients with CKD, stratifying the patient population, and facilitating tailored treatment and care coordination among patients, primary care, and nephrology.

 We conducted a randomized controlled trial to assess the impact of a comprehensive set of EHR tools and patient engagement materials to improve the management of CKD.

This 18-month trial was conducted from 2011 to 2013, with patient enrollment occurring during the initial 6 months and all patients followed for 12 months after enrollment. We randomly assigned PCPs to receive alerts within the EHR during office visits for patients with CKD and mailed educational materials to patients of physicians in the intervention arm. The Human Studies Committee at Brigham and Women’s Hospital approved the study protocol, and a waiver of informed consent was approved for physicians and patients. The trial was registered at ClinicialTrials.gov (ID NCT01203813).

We conducted our study at Harvard Vanguard Medical Associates, an integrated multispecialty group practice in eastern Massachusetts caring for approximately 300,000 adult patients. The system has significant experience in population health management, such as participation as a Pioneer Accountable Care Organization. The practices use a common EHR (Epic Systems; Verona, Wisconsin) that captures clinical notes, electronic diagnosis codes, specialty referrals, medication prescriptions, and laboratory test results. This system has delivered automated reporting of estimated glomerular filtration rate (eGFR), computed using the Modification of Diet in Renal Disease (MDRD) Study equation. The EHR does not provide decision support for patients with CKD. Nephrology services are provided by 8 nephrologists employed by the group practice.

We enrolled 153 physicians across 13 health centers and 7691 patients aged 18 to 80 years with an established diagnosis of stage III CKD (

). The diagnosis was based on meeting each of the following criteria: 1) the presence of an office visit with a PCP within the group practice within the prior 18 months, 2) the presence of at least 2 eGFR results between 30 and 60mL/min/1.73m

 within the prior 5 years, 3) the qualifying abnormal eGFR results were separated by at least 90 days, and 4) the most recent eGFR was less than 60 mL/min/1.73m

The intervention was randomized at the individual physician level. Within each health center, we paired clinicians based on their number of eligible patients with CKD and then randomly assigned 1 physician in each pair to the intervention arm.

Based on local consensus and emerging data on the importance of both eGFR and albuminuria,

 we stratified patients with stage III CKD according to their risk of complications and identified relevant treatment targets. Our local consensus was achieved prior to the publication of recent guidelines by the Kidney Disease: Improving Global Outcomes (KDIGO) initiative

 and involved gathering input from both primary care and nephrology leadership within our multispecialty group practice using current data on predictors of mortality for patients with CKD. “High-risk” patients were defined as those with either 1) at least 1 eGFR of less than 45 mL/min/1.73m

 in the prior 5 years or 2) at least 1 eGFR of at least 45 mL/min/1.73m

, in combination with the presence of diabetes or albuminuria (urine microalbumin >30 mcg/mg or spot protein to creatinine ratio >0.15 mcg/mg). All other patients—specifically, those with an eGFR of at least 45 mL/min/1.73m

 and no history of diabetes or albuminuria—were considered “low-risk”. We identified our treatment targets using the same process, and for high-risk patients, we recommended use of an angiotensin-converting enzyme (ACE) inhibitor or angiotensin receptor blocker (ARB), as well as referral to a nephrologist. For low-risk patients, we recommended use of an ACE inhibitor or ARB, as well as annual monitoring of urine microalbumin to assess for disease progression and risk stratification.

All EHR alerts were displayed when physicians accessed the electronic ordering module of the patient chart. During office visits with high-risk patients, PCPs received up to 2 alerts. The first alert recommended a referral to a nephrologist if no such specialty office visit had occurred in the prior 12 months (

]). The second alert recommended prescription of an ACE inhibitor or ARB if the patient carried a diagnosis of hypertension or albuminuria, had not been prescribed the medication in the last 12 months, and had no documented allergy to such medication. 

During office visits with low-risk patients, PCPs received up to 2 alerts. The first was the same ACE inhibitor alert used for high-risk patients. The second recommended overdue annual laboratory tests, including those for urine microalbumin, serum creatinine, low-density lipoprotein (LDL) cholesterol, 25-hydroxy (OH) vitamin D, parathyroid hormone, calcium, phosphorus, and hemoglobin.

Our intervention included a mailed outreach program to promote patient engagement. We encouraged PCPs in the intervention arm to enroll patients using 2 strategies. First, all electronic reminders also prompted physicians to enroll patients in the mailed outreach program (

). Second, for those physicians who did not respond to the request to enroll a patient via the electronic alert, we sent a follow-up postcard within 1 month of the office visit requesting them to enroll the patient. We required physicians to enroll patients to ensure that patients received a diagnosis of CKD from their care team prior to receiving any mailings.

The outreach program consisted of quarterly mailings to patients that provided tailored treatment recommendations based on detailed extracts from their EHR. The mailings were based on educational materials developed by the National Kidney Disease Education Program (

 These mailings provided recommendations specific to CKD for managing blood pressure, appropriate use of ACE inhibitors or ARBs, education on current medications, and recommendations for overdue laboratory tests or follow-ups on previously abnormal results.

Physicians in the control group received no EHR alerts, and their patients were not eligible to receive the mailed outreach program.

The primary study end points were based in part on the KDIGO guidelines and extracted from the EHR. Among high-risk patients, the primary end points included an office visit to a nephrologist during the 12-month study period and the prescription of an ACE inhibitor or ARB during the 12-month study period for those with hypertension and/or microalbuminuria and no documented allergy. The primary study end points among low-risk patients included the presence of a urine protein test during the 12-month study period and the prescription of an ACE inhibitor or ARB for those with hypertension and/or microalbuminuria and no documented allergy. We also assessed secondary outcomes of rates of annual serum creatinine, LDL cholesterol, hemoglobin, phosphorus, 25-OH vitamin D, calcium, and parathyroid hormone testing. Physicians and patients were not blinded to intervention status, although all outcomes data were collected without respect to intervention status.

We surveyed all patients in the intervention arm who were enrolled in the outreach program (n = 1002) by their PCP. Patients used a 4-point ordinal scale from “definitely yes” to “definitely no” to report on whether the mailings gave them choices to think about for treating CKD, helped them set specific CKD treatment goals, and helped them understand their medications for CKD. Patients also reported on whether their doctor or another health professional had told them that they had weak or failing kidneys, and they used a 5-point ordinal scale from “strongly agree” to “strongly disagree” to report agreement with their diagnosis of CKD. Finally, patients used a 5-point scale from “excellent” to “poor” to rate the CKD care they received. The survey was administered via a single mailing at the end of the intervention and achieved a 24% (n = 242) response rate. 

We surveyed 153 study physicians at the completion of the intervention. Physicians used a 5-point ordinal scale from “always” to “never” to report on the frequency with which they informed patients of a new diagnosis of CKD once they recognized it was present. Physicians also reported on the eGFR threshold at which they felt comfortable informing their patients of a diagnosis of CKD. Intervention physicians also rated the effectiveness of the electronic alerts, patient mailings, and collaboration with nephrology on improving the quality of CKD care among their patients (“very effective,” “somewhat effective,” or “not effective”). The survey was implemented via an initial paper mailing, followed by a reminder email to nonresponders and a final paper mailing at 4 weeks, achieving a 73% (n = 111) response rate.

Balance between patient demographic characteristics in the intervention and control arms was checked using a 

 test for patient age, Fisher exact tests for binary variables, and χ

 tests for categorical variables. We analyzed the impact of the intervention by fitting logistic regression models using the generalized estimating equation approach to account for clustering of patients within clinics, with performance of each of our prespecified outcomes as the dependent variable and intervention status as the primary independent variable. The models were implemented using the GENMOD procedure in SAS version 9.3 (SAS Institute; Cary, North Carolina).

We conducted post hoc analyses to understand the importance of exposure to the intervention components. These included the subset of patients in the intervention arm who received the outreach mailings, as well as patients with varying numbers of office visits (0, 1-3, and >3) to their PCP during the intervention period. For the outreach mailing analyses, we used propensity score stratification to compare the appropriate set of patients in the control arm with the subset of patients in the intervention arm who received mailings. A propensity score model was created separately for each clinic, using the following variables as predictors of receiving a mailing: patient sex; race/ethnicity; insurance type; prior nephrology visit; current treatment with an ACE inhibitor or ARB; baseline eGFR; and presence of diabetes, cardiovascular disease, or hypertension. Patients from the intervention arm who received a mailing were then compared, through stratification, with patients from the control arm who had a similar probability (ie, were within the same 5% propensity interval) of receiving a mailing. Outcomes among patient groups were compared using the same clustered logistic regression models described earlier, adjusting for correlation within clinicians, time on study, and propensity strata.

We randomized 153 PCPs caring for 7691 adult patients with stage III CKD, including 3947 high-risk patients and 3744 low-risk patients (

). The median number of patients enrolled per PCP was 47 (interquartile range, 26-69).

Among high-risk patients, those in the intervention arm were significantly more likely to have an office visit with a nephrologist during the 12-month study period compared with those in the control arm (45% vs 34%; 

). Among low-risk patients, those in the intervention arm were significantly more likely than those in the control arm to have received urine microalbumin testing in the prior 12 months (45% vs 21%; 

<.001). There was no difference between the intervention and control arms in rates of prescribing an ACE inhibitor or ARB in either the high-risk patient group (76% vs 79%; 

Among both high- and low-risk patients, those in the intervention arm had higher rates of annual testing compared with those in the control arm for phosphorus, vitamin D, and parathyroid hormone. High-risk patients also had higher annual testing rates for calcium in the intervention arm compared with the control arm (Table 2).

Intervention physicians enrolled 1002 (26%) patients into the patient mailing program, including 647 (32%) high-risk patients and 355 (19%) low-risk patients. With the exception of ACE inhibitor or ARB therapy and testing of microalbumin in high-risk patients, both high- and low-risk patients in the intervention arm who received patient mailings were significantly more likely than propensity-stratified control arm patients to achieve all primary and secondary study outcomes (

Among all study patients, 41% had 4 or more office visits to their PCP during the study period, 51% had 1 to 3 visits, and 7% had no primary care visits. Regardless of intervention status, rates of annual nephrology visits, receiving prescription of ACE inhibitor or ARB therapy, and annual urine protein monitoring were all significantly lower among patients with no primary care visits compared with either group of patients with at least 1 visit (

). In addition, the intervention effect varied according to the number of PCP office visits during the study period, demonstrating no significant intervention effect among those patients with 0 visits and larger intervention effect sizes for those with at least 1 visit.

More than half (61%; n = 138) of intervention patients who received outreach mailings reported being told by a doctor or health professional that they had weak or failing kidneys. In logistic regression models that considered patient age, sex, and race; comorbid conditions (diabetes, hypertension, and cardiovascular disease); and CKD features (high- vs low-risk status and nephrology consultation), the absence of diabetes (odds ratio [OR], 1.9; 95% CI, 1.1-3.2), a nephrology visit prior to the intervention period (OR, 2.6; 95% CI, 1.6-4.3), and a nephrology visit during the intervention period (OR, 3.5; 95% CI, 2.1-5.9) were all associated with patients reporting being told that they had weak or failing kidneys.

More than half (63%; n = 142) of intervention patients strongly or somewhat agreed with their diagnosis of CKD, whereas 18% (n = 41) strongly or somewhat disagreed with their diagnosis of CKD. Two-thirds (67%; n = 136) of intervention patients rated their care for CKD as excellent or very good. A majority (89%; n = 177) of patients reported that the outreach mailings definitely or somewhat gave them choices to think about for treating their CKD, 82% (n = 162) felt the mailings helped them set specific goals for CKD treatment, and 81% (n = 153) felt the mailings helped them understand their medications for CKD.

Intervention and control physicians were similarly likely to report that they always or usually informed their patients of a diagnosis of CKD (87% vs 75%; 

= .12). A higher percentage of intervention physicians compared with control physicians reported feeling comfortable establishing a diagnosis of CKD using a threshold eGFR of less than 60 (56% vs 39%; 

= .07, adjusted for within-clinic correlation), although the difference was not statistically significant. Three-quarters (75%) of physicians in the intervention group reported that our electronic reminders were somewhat or very effective at improving the quality of CKD care among their patients, 84% reported the patient mailings were somewhat or very effective, and 92% reported that collaboration with nephrology was somewhat or very effective.

Articles

Decision support tools, disease registries, and patient engagement materials can improve population-based chronic kidney disease care.

Physician and Patient Tools to Improve Chronic Kidney Disease Care

Using Behavioral Risk Factor Surveillance System (BRFSS) telephone survey data, we evaluated whether individuals who reported access to enhanced primary care features experienced improved quality of care.

We assessed a sample of 16,731 Massachusetts residents contacted by telephone using random-digit dialing, to complete the BRFSS in 2008. A randomized subset of 5693 respondents received an additional 5-question enhanced primary care assessment module. We defined an “enhanced” primary care group as those who reported having a regular, personal provider and responded that their provider “always” or “almost always” 1) had knowledge of their medical history, 2) gave them an appointment right away when necessary, 3) was up-to-date in their knowledge of the patient’s specialist care, and 4) asked them about all of their medications. Multivariable logistic regression was used to compare the “enhanced” versus “usual care” groups and assess several quality process measures.

Nearly one-third of participants endorsed having indicators of enhanced care, and this group reported significantly higher rates of diabetes process measures (56% vs 38%), cholesterol screening (89% vs 81%), influenza vaccination (57% vs 49%), pneumonia vaccination (51% vs 43%), and lower cost and/or access barriers to care (22% vs 33%).

Enhanced primary care was associated with improved self-reported quality outcomes in a statewide telephone survey. A brief, 5-question module provided a novel population measure of access to enhanced primary care. This is a scalable option for other states hoping to characterize their own primary care improvement efforts through the patient-centered medical home model.

	We evaluated Behavioral Risk Factor Surveillance System telephone survey data from Massachusetts residents that included a patient-centered medical home (PCMH) assessment module. 

		Patient-reported indicators of the PCMH correlated with improved quality process measures in diabetes, cholesterol screening, vaccination, and access to care.   

		A brief, 5-question module provided a novel population measure of access to enhanced primary care. 

		This may be a scalable option for other states hoping to characterize their own primary care usage and utility as health insurance expansion continues nationally.

Massachusetts was one of the leaders in insurance expansion with the 2006 Massachusetts Mandated Health Insurance Law.

 Recent reports have cited positive outcomes from the Massachusetts reform effort, such as increases in the total number of Massachusetts residents insured, growing rates of employer-based insurance coverage, strong public and physician support, reductions in disparities in access based on race or ethnicity, increasing amounts of preventive services, fewer low-income or chronically ill adults reporting unmet medical needs,

 and even lower mortality rate among those getting insurance.

 However, insurance reform only addresses a small part of the deficiencies in the larger healthcare delivery system. Patient-centered medical homes (PCMHs) are generating significant state and national attention as one solution in what is generally a costly and ineffective system plagued with disparities in health outcomes and significant barriers to high-quality, integrated medical care.

Most PCMH assessment surveys are based on individual practice or demonstration reports, and such surveys have been at the forefront of reports on the success of the PCMH model.

 Although such demonstrations are a critical part of the evaluation process of new care models, they are context-specific and take time to generate evaluable data. Another approach to evaluate the effect of the PCMH-associated components is to use publically available survey data and parsimonious definitions of what factors compose a PCMH. One such evaluation was performed on the 2007 to 2008 National Survey of Children’s Health, and it identified improvements in 6 of 10 quality measures among children in a PCMH, as defined by a 5-component measure.

 Such patient care experience surveys are often unavailable, however, and to date, there is no state-level evidence on the availability of PCMH components in primary care practices or their association with quality of care.

The Behavioral Risk Factor Surveillance System (BRFSS) was developed by the CDC as a random-digit-dial landline telephone survey.

 The CDC collects data monthly, using this survey in all states and territories of the United States. In order to evaluate the availability of primary care providers, PCMH components in primary care, and sociodemographic predictors of the preceding variables in Massachusetts, a 5-question module was added to the BRFSS for a subset of responders. Using the data provided by this module, we then evaluated the association of PCMH components with quality of care.

In preparation for statewide initiatives in medical homes, one of the authors, who was a member of the Massachusetts Executive Office of Health and Human Services, worked with experts in PCMHs to develop a 5-question short form survey module (

). The questions were adapted from work on the Primary Care Assessment Survey (PCAS) by Safran et al.

 Officials from the Massachusetts Department of Public Health, which administers the survey in the state, gave permission for the module to be piloted in a subset of participants in the 2009 Massachusetts BRFSS (MA-BRFSS) survey. 

The MA-BRFSS survey was completed in 3 waves, and the experimental module was used in the second wave. We used descriptive statistics to characterize the subset of responders and a 

 test to look for statistical variance between the experimental module—wave 2&mdash;and the other survey waves. We then evaluated whether the responder had a primary care physician according to their survey results and used a multivariable logistic regression to test for associated subject characteristics between the subset of responders with a primary care physician and the subset without.

To determine whether having an enhanced primary care experience was related to completion of healthcare process measures, we divided responders into 2 groups. One group, termed the “enhanced primary care” group, endorsed having a primary care physician and responded that their provider “always” or “almost always” 1) had a knowledge of their medical history, 2) gave them an appointment right away when necessary, 3) was up-to-date in their knowledge of the patient’s specialist care, and 4) asked them about all of their medications. A multivariable logistic regression was used to test for associated subject characteristics between the “enhanced primary care” group and the “usual care” group without these PCMH attributes. A 

test was used to characterize the 2 groups.

We performed a multivariable logistic regression analysis to assess differences between the enhanced and usual care groups in several quality measures. We selected these indicators because they are common measures of clinical quality in the management of chronic disease and preventive care. Quality measures included diabetes annual care processes (which includes an annual visit, eye exam, foot exam, cholesterol screening, and hemoglobin A1C testing); lipid screenings for patients with cardiovascular disease, diabetes, and hyperlipidemia; annual influenza vaccination in eligible patients (all patients aged ≥6 months)

; and pneumonia vaccinations in eligible patients (all adults ≥65 years and adults aged 19-64 years who smoke, have chronic pulmonary disease, or are immunodeficient).

An additional binary quality measure assessing cost barriers and access to care was created based on the responses to the following: “Was there a time during the last 12 months when you needed to see a doctor but could not due to the cost?” and “About how long has it been since you last visited a doctor for a routine checkup?” Subjects who responded that they did forgo a visit to the doctor due to cost in the past year or that they had not had a checkup within 1 year were given a positive value on the “cost/access” variable; subjects who did not meet one or both criteria were not given a positive value. The regressions were controlled for age, sex, income, education, and insurance status as covariates.

This study was reviewed by the Institutional Review Board at both the Massachusetts Department of Public Health and the CDC, and was considered exempt due to being a low-risk study without any patient identifiers.

Of the 16,731 respondents to the MA-BRFSS in 2009, 5693 respondents were randomized to receive the short form survey module. 

 summarizes subject characteristics for the respondents to the short form survey module (wave 2) compared with other respondents (waves 1 and 3). There were no significant demographic differences between the 2 groups, confirming that the random sampling for the survey module was successful.

Among participants, 29.8% answered “always” or “almost always” to all 5 PCMH module questions, indicating that they had an enhanced primary care experience. Subjects who were aged 50 to 74 years (

 <.001) were more likely to report having an enhanced primary care experience; subjects who were either uninsured (

 <.001) or were Asian, Native Hawaiian or Pacific Islander, American-Indian, multiracial, or selected “other” for their racial affiliation (

A multivariate logistic regression adjusted for age, sex, income, education, and insurance status found significant differences between the enhanced and usual primary care groups in all quality measures tested. Subjects receiving enhanced care were more likely to receive diabetes care

 = .002). Subjects in this enhanced group were also less likely to report experiencing barriers to access to care, measured by whether they reported an instance in the past 12 months where they needed to see a doctor but could not due to cost and by whether more than 1 year had elapsed since their last checkup (

Almost one-third of residents reported having access to a primary care practice with enhanced primary care attributes. Importantly, having enhanced primary care was associated with better performance on preventive, access, and chronic disease measures. Our findings have favorable implications for the wider adoption of the PCMH model—the main mode of primary care improvement and transformation nationwide in the United States. Our findings also contribute to a growing body of research showing that the PCMH-enabled primary care transformation may result in modest increases in quality measures early on by improving care processes, particularly in the area of prevention.

For example, a recent prospective multi-payer study in Hudson Valley, New York, found that PCMH practices improved significantly more (from 1% to 9%) on 4 of 10 quality measures than practices with electronic health records and paper records.

 In a single private payer demonstration project in New Jersey in 2011 with 8 medical home practices, mammography and nephropathy screening increased, although changes in 7 other HEDIS measures were not significant.

 A separate randomized controlled trial in New York evaluating PCMH transformation over 2008 to 2010 among 32 practices (18 intervention practices) found improvements in 2 of 11 quality indicators: hypertensive blood pressure control and breast cancer screening.

Notably, other PCMH studies have not shown significant impact on quality measures,

 enforcing the need for additional research in this area to understand why some initiatives are associated with quality improvement while others are not. Understanding how the PCMH model can impact quality of care is critical for optimal primary care design, especially given the enormous and rapid uptake of the PCMH model. From 2009 to 2014, the number of patients covered by PCMH initiatives increased from nearly 5 million to almost 21 million across more than 100 state and federal initiatives.

In addition, we need more efficient and effective methods to evaluate primary care and access to care, with a particular focus on the patient experience as medicine becomes increasingly patient-centered. A PCMH transformation study evaluating patient experiences in a pilot practice in eastern Massachusetts with Lean enhancement and payment reform found that patient experience was sustained or improved across PCMH domains including the personal physician and communication domain.

 Our study builds on this finding, utilizing a simple tool for evaluating access to enhanced primary care directly from patients in a large, state-based population survey of patient respondents. This 5-question survey module was a valuable addition to the BRFSS and provided a new population measure of access to enhanced primary care. This survey of critical primary care functions may be a replicable and scalable strategy that other states and healthcare systems can use to characterize their own primary care transformation efforts and reach. 

While it may be that the enhanced primary care components evaluated are inseparable from what might be simply termed high-quality traditional primary care, such a distinction may be irrelevant. The PCMH model provides structure and script for the delivery of better primary care. This is invaluable, as it allows struggling practices to implement a comprehensive plan for improvement rather than disparate quality improvement efforts that may fail due to their lack of integration into the central practice philosophy and administration. However, there is still great variability in how PCMHs are implemented from site to site,

 and evaluations at a population level are important. In addition, evaluations within a single system may reflect characteristics of that system only and may not be generalizable to PCMHs implemented in a broad array of practice types.

In spite of the demonstrable increases in quality and access to healthcare brought about by the Massachusetts healthcare reform law of 2006, healthcare costs continue to increase steadily. Reform efforts have not contributed to any additional rise in healthcare costs, but neither have they led to a significant decrease in overall costs.

 One of the largest drivers of healthcare expenditure is preventable hospitalizations and emergency department visits.

 There is evidence to suggest that these expenditures can be mitigated by high-quality, long-term, prevention-based primary care focused on managing the chronic conditions that often result in hospitalization when mis- or undermanaged.

 Delivering such care is the goal of the PCMH, although it remains to be seen whether the continued expansion of this model results in the expected and sustained cost reductions.

This study is a cross-sectional pilot study that includes only 1 year of data, although the sample is large and representative of the population of Massachusetts. A majority of the differences in quality outcomes in the enhanced primary care group were modest; however, the differences in diabetes care were quite large and may have been even larger had we not used a bundled care process measure, which can result in lower scores.

 Furthermore, the 5-question module used in this study was validated as a measure of primary care quality,

 but not as an indicator of whether a patient was treated by a PCMH. Thus, it is an indirect measurement of medical home attributes. As such, we cannot say with certainty that these differences in quality are a direct result of receiving enhanced care or are indicative of practices that are themselves medical homes, and we had no direct measure of which patients were receiving their care in a PCMH. The study is also limited by its reliance on self-report for all measures; however, this direct feedback from patients provides an important window into the patient perspective and experience. 

Evaluation of the PCMH and how it affects patient experience and quality measures on a population level is critical to monitor the progress of reform measures, such as the ongoing large-scale adoption of medical homes. Other major health reforms, such as increased insurance coverage, are being deployed on a state level, and we need new population-level tools to evaluate these initiatives. The pilot implementation of the 5-question enhanced primary care module in the Massachusetts BRFSS provides a simple, low-cost model, which other states may use to gather this crucial experience and quality data. We found that higher levels of enhanced primary care attributes at practices were associated with substantially improved quality for an array of measures, suggesting that this benefit may be realized even at the population level.

The authors would like to thank Bruce Cohen, PhD, Director of Research and Epidemiology and Bureau of Health Statistics, Research and Evaluation at the Massachusetts Department of Public Health; Liane Tinsley, MPH, for assistance with implementing the PCMH module into the BRFSS survey; and also Jennifer Kincheloe, PhD, MPH.

 Brigham and Women’s Hospital (AB, AWB, JSW, DWB), Boston, MA; Department of Health Care Policy, Harvard Medical School (AB, JSW), Boston, MA; Ariadne Labs (AB), Boston, MA; Veterans Affairs Boston Healthcare System (AWB), Boston, MA; Pennsylvania State University College of Medicine (SC), Hershey, PA.

 Dr Bitton is a board member of Health Leads and is a senior advisor for the Center for Medicare & Medicaid Innovation. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (AB, JSW, DWB); acquisition of data (AB, JSW); analysis and interpretation of data (AB, AWB. JSW, SC, DWB); drafting of the manuscript (AB, AWB, JSW, SC, DWB); critical revision of the manuscript for important intellectual content (AB, AWB, SC, JSW, DWB); statistical analysis (AB, SC); obtaining funding (AB, DWB); administrative, technical, or logistic support (AB, DWB, SC, AWB); and supervision (AB, JSW, DWB).

 Asaf Bitton, MD, MPH, Assistant Professor of Medicine and Health Care Policy, Division of General Medicine and Ariadne Labs, Brigham and Women’s Hospital, 1620 Tremont St, 3rd Fl, Rm 3-002P, Boston, MA 02120. E-mail: abitton@partners.org.

In a statewide telephone survey, patient-reported indicators of the patient-centered medical home correlated with improved process measures in diabetes, cholesterol screening, vaccination, and care access.  

Enhanced Primary Care and Impact on Quality of Care in Massachusetts

To determine whether clinical decision support (CDS) is associated with improved quality indicators and whether disabling CDS negatively affects these.

Using the 2006-2009 National Ambulatory and National Hospital Ambulatory Medical Care Surveys, we performed logistic regression to analyze adult primary care visits for the association between the use of CDS (problem lists, preventive care reminders, lab results, lab range notifications, and drug-drug interaction warnings) and quality measures (blood pressure control, cancer screening, health education, influenza vaccination, and visits related to adverse drug events).

There were an estimated 900 million outpatient primary care visits to clinics with EHRs from 2006-2009; 97% involved CDS, 77% were missing at least 1 CDS, and 15% had at least 1 CDS disabled. The presence of CDS was associated with improved blood pressure control (86% vs 82%; OR 1.3; 95% CI, 1.1-1.5) and more visits not related to adverse drug events (99.9% vs 99.8%; OR 3.0; 95% CI, 1.3-7.3); these associations were also present when comparing practices with CDS against practices that had disabled CDS. Electronic problem lists were associated with increased odds of having a visit with controlled blood pressure (86% vs 80%; OR 1.4; 95% CI, 1.3-1.6). Lab result notification was associated with increased odds of ordering cancer screening (15% vs 10%; OR 1.5; 95% CI, 1.03-2.2).

The use of CDS was associated with improvement in some quality indicators. Not having at least 1 CDS was common; disabling CDS was infrequent. This suggests that meaningful use standards may improve national quality indicators and health outcomes, once fully implemented.

		The use of some electronic health record clinical decision support (CDS) was associated with improved quality of care among national primary care outpatient visits from 2006 to 2009.

		Most practices were missing at least 1 CDS module considered essential by meaningful use standards.

		Meaningful use standards may have a significant impact on national quality of care and health outcomes once fully implemented.

The CMS Electronic Health Record Incentive Program, better known as “meaningful use,” provides initial monetary incentives for adoption of electronic health records (EHRs) and subsequently imposes monetary penalties if physician practices do not meet requirements. The meaningful use program has spurred a significant rise in the number of office-based physicians reporting EHR use, from 24% in 2005 to 48% in 2009 to 72% in 2012.

The meaningful use goals and objectives are divided into 3 stages. Stage 1 requirements include using a certified EHR to capture and share patient data by 2015; stage 2, beginning in 2014, includes use of advanced care processes with clinical decision support (CDS); and stage 3, to begin in 2016, will require providers to demonstrate outcome improvements. The time frame for stages 2 and 3 was recently extended.

 As of March 2013, 73% of eligible providers had registered to participate in the meaningful use incentive program and 44% of eligible providers had received incentive payment for meeting meaningful use objectives.

Although the meaningful use requirements have already been established, the evidence is inconsistent regarding improvement in healthcare processes or patient outcomes as a result of the implementation of general and individual EHR components. Prior studies have shown that EHR-based CDS is associated with improved prescribing safety,

 CDS has also been associated with some improvements in quality indicators,

 Despite having EHRs, many physicians report being unable to complete basic panel management activities,

which affects their ability to deliver high-quality care for patients with chronic conditions.

Notwithstanding the mandate to use certified EHRs, use of an EHR in the outpatient setting has not yet been shown unequivocally to improve quality indicators across a broad range of practices and EHRs, especially because there is substantial variability among the 1600 certified EHR platforms.

In addition to the large number of EHRs, they also can be customized to each practice’s needs and resource availability, so results will vary between practices in terms of impact on care even with the use of a single vendor’s practice. Certain types of practices, such as smaller community-based practices, may be less able to fully implement a complete EHR with clinical decision support, which is the kind postulated to have the greatest potential to improve quality.

 Decisions about choices around the CDS functions implemented may affect the quality of care delivered by that practice. To further inform the potential effect of meaningful use stages 2 and 3, we examined the association between having CDS, not having CDS, and disabling CDS, and quality of care indicators using nationally representative data just prior to the start of the meaningful use incentive program.

We performed a retrospective, cross-sectional analysis of adult primary care ambulatory clinic visits in the National Ambulatory Medical Care Survey (NAMCS) and National Hospital Ambulatory Medical Care Survey (NHAMCS) outpatient department records from 2006-2009 in order to analyze data with little impact from current meaningful use incentive payments. We examined a number of patient and practice characteristics to determine any differences in access to practices with EHRs based on prior work showing associations between these characteristics and EHR availability.

On the premise that most preventive care takes place through a primary care practice, we limited our sample to just those practices.

The NAMCS and NHAMCS are conducted by the Ambulatory Care Statistics Branch of the National Center for Health Statistics (NCHS) of the CDC. These surveys focus on outpatient visits to non—federally funded US medical practices.

The NCHS uses a complex multistage probability sampling design based first on geography, then physician specialty, and finally patient visits to individual practices. Information is collected at ambulatory visits by participating practices or US Census Bureau representatives. The NCHS assigned weights to each visit record to allow for estimation of national statistics. The NCHS institutional review board approves the protocols for the NAMCS and NHAMCS, including a waiver of informed consent for patients.

The NAMCS collects information at a visit level from community-based physician practices; from 2006 to 2009, approximately 30,000 community-based office visits were recorded annually. The NHAMCS collects information at a visit level from hospital-based physician practices; from 2006 to 2009, approximately 34,000 hospital-based office visits were recorded annually.

In all years included in the analysis, both the NAMCS and NHAMCS included questions in their intake surveys that asked whether the practice had an electronic medical record (for other than billing purposes), a computerized patient problem list, computerized warnings of drug interactions or contraindications, a computerized system to view lab results, a computerized system that highlights out-of-range lab values, and a computerized system for reminders for guideline-based interventions and/or screening tests. Response variables indicated that a practice had a function (“yes”), did not have a function (“no”), had the function but disabled it (“turned off”), or was unsure if the practice had the function (“don’t know” or “blank”).

The NAMCS and NHAMCS record health information for each visit sampled. The respondent indicates the top 3 reasons for the visit, which are then coded by NCHS; the patient’s vital signs, including blood pressure; whether any health education was ordered or provided during the visit; and whether an influenza vaccination was ordered or administered during the visit. The respondent also indicates any screening services ordered or performed during the visit. The respondent can record mammography, Pap smears, and scope procedures (NCHS records and codifiers, free text scope procedures, 1 of which is sigmoidoscopy/ colonoscopy), among others.

The NAMCS and NHAMCS collect patient and practice demographic information: patient age, patient race, patient income category (imputed based on patient zip code by NCHS), primary reason for the visit, region of the country, practice ownership, and number of physicians in the practice (solo or non-solo). Race data are collected in the manner already in use by the practice. Due to high nonresponse rates for race (24% to 33% in NAMCS, and 12% to 15% in NHAMCS for the years included), the NCHS also provides a race variable imputed based first on the patient’s location, then on physician specialty and 

International Classification of Diseases, Ninth Revision, Clinical Modification

 code for primary diagnosis, and finally on a random basis.

Our predictors of interest were EHR CDS modules mandated by stage 1 of meaningful use: electronic problem lists, preventive care reminders, lab result reporting, out-of-range lab notification, and drug-drug interaction warnings. We performed analyses with each predictor separately, and performed analyses combining them into a composite group of key EHR features. Based on prior work that examined associations between certain EHR features and clinical outcomes,

 we developed a matrix between CDS functions and the clinical outcomes of interest and performed analyses accordingly (

Our main outcomes of interest were based on national quality metrics related to adult primary care: blood pressure control (defined as a systolic blood pressure less than 140 mm Hg), age- and gender-appropriate cancer screening (mammography, Pap smear, and sigmoidoscopy or colonoscopy), health education for particular conditions, influenza vaccination during the months of October through March, and adverse drug events (as measured by visits coded to be related to an adverse drug event). We also combined the receipt of cancer screening, health education, and influenza vaccination into a composite “preventive care” outcome measure.

We were interested in the difference in outcomes based on whether a practice had a CDS function or not. Specifically, we examined whether there were significant differences in the rates of the clinical outcomes depending on whether a visit was to a practice with the CDS or without the CDS.

We categorized practices into 3 groups: all CDS tools active; without 1 or more CDS function; or any disabled CDS. We considered the practice to have all CDS active if they answered “yes” to having all 5 of the individual CDS. We considered the practice to be without 1 or more CDS functions if they answered “no,” “turned off,” don’t know,” or “blank” to any of the CDS functions. We considered the practice to have “disabled CDS” if they answered “turned off” for any of the CDS functions.

We collapsed practice ownership into 5 categories: physician or physician group, community health center, health maintenance organization, medical or academic health center, and other. All other covariates were used as provided by NCHS.

The unit of analysis was the patient visit. We followed NCHS guidelines based on the hierarchical sampling design. We performed all statistical tests on data with less than a 30% relative standard error (the standard error divided by the estimate itself) and more than 30 sample records.

We used multivariable logistic regression modeling to control for a number of patient and practice level covariates. We modeled age as a continuous variable, and all other covariates as categorical variables. We evaluated categorical variables with the χ2 test and continuous variables with the t test. We performed multivariable logistic regression modeling for each quality measure individually. The main predictors of interest were the status of the EHR CDS function. We considered P values of less than .05 to be significant.

We used SAS statistical software (version 9.3; SAS Institute, Cary, North Carolina) for all analyses.

From 2006-2009, the NAMCS/NHAMCS databases included 104,102 adult primary care visits, representing 2 billion adult primary care visits in the United States. Of these, 45% of visits were to practices with EHRs. Of those visits, 97% also had at least 1 of the 5 CDS functions of interest. Of visits using EHRs with at least 1 CDS function, 77% did not have (for any reason), and 15% had disabled (actively turned off), at least 1 of the 5 CDS functions of interest.

We compared patient and practice demographics for visits to clinics with all the CDS functions versus visits to clinics with at least 1 CDS function missing (

). Patients were of similar age (50.9 years vs 51.1 years,

 = 0.84). Rates of visits for flares of chronic problems (7%) and pre- or post surgery (2%) were similar between the 2 groups; rates of visits for new problems (39% vs 41%) and follow-up of chronic problems (32% vs 33%) were less common and visits for preventive care (17% vs 16%) were more common at clinics without at least 1 of the CDS functions versus clinics with all the CDS functions (

 <.0001). Of visits to practices with at least 1 CDS function missing versus those to practices with all the CDS functions, rates of visits to health maintenance organizations (3% vs 7%) were lower, and to physician groups (60% vs 50%), community health centers (5% vs 3%), or medical/academic health centers (18% vs 12%) were higher (

 <.0001). There were no differences in the distribution of visits to clinics without at least 1 of the CDS functions versus visits to clinics with all the CDS functions by gender, race, or region of the country.

Most patients who made visits (81%) had controlled blood pressure. Of eligible visits, there were low documented rates of age-appropriate cancer screening (13%) and influenza vaccination (4%). Approximately 0.2% of visits were related to an adverse drug event. Patients who made visits to clinics with all the CDS functions present versus those who visited clinics that were missing at least 1 of the CDS functions were more likely to have controlled blood pressure (86% vs 82%; OR 1.3; 95% CI, 1.1-1.5) and more likely to not have adverse drug event visits (99.9% vs 99.8%; OR 3.0; 95% CI, 1.3-7.3) (

We found similar results when specifically analyzing visits to clinics with all the CDS functions versus visits to clinics with any of the CDS functions disabled. Patients were more likely to have controlled blood pressure (86% vs 83%; OR 1.5; 95% CI, 1.02-2.3) and more likely to not have adverse drug event visits (99.9% vs 99.8%; OR 5.2; 95% CI, 1.4-19.0) (

We next examined only those CDS functions directly related to each clinical outcome. There were higher odds of receiving cancer screening if the visit was to a clinic with electronic lab results (15% vs 10%; OR 1.5; 95% CI, 1.03-2.2) or notification of acceptable lab ranges (16% vs 11%; OR 1.4; 95% CI, 1.03-1.8) than if the visit was to a clinic without these CDS functions. There were higher odds of having controlled blood pressure (86% vs 80%; OR 1.4; 95% CI, 1.3-1.6) if the visit was to a clinic with electronic problem lists rather than to a clinic without this EHR function (

Because of extremely small numbers and/or absence of outcomes in certain subgroups, we were unable to analyze the association of disabling particular CDS functions with specific clinical outcomes.

Using national surveys of ambulatory visits to primary care clinics, we found significant associations between the use of CDS and some, but not all, clinical quality measures before the enactment of meaningful use. Physicians reporting use of CDS had patients with better blood pressure control and fewer visits related to adverse drug events; these associations were present regardless of whether the physicians with CDS were compared with physicians without CDS or with physicians with disabled CDS. Physicians who reported having electronic problem lists had better rates of blood pressure control of their patients. Physicians with electronic lab results or notification of lab ranges were more likely to provide ageappropriate cancer screening. As we enter stages 2 and 3 of meaningful use in which clinicians must use CDS and then provide data about their performance with respect to patient outcomes, hopefully performance will continue to improve, though this remains to be demonstrated.

The incidence of not having at least 1 of the CDS functions was common, which is likely because the meaningful use mandate was not yet in place at the time of these national surveys. However, despite not having this mandate, the incidence of disabling any CDS was infrequent. Keeping decision support active is a “best practice” in clinical decision support,

but prior experience has demonstrated that many practices elected to disable specific types of decision support before meaningful use. These decisions can sometimes be indicated if the EHR is generating too many false positive warnings, which can create alert fatigue, although there typically are better solutions. The low rate of disabling CDS is reassuring and may indicate that the performance characteristics of CDS are improving. Notably, the odds of having controlled blood pressure and avoiding adverse drug events were higher at clinics with active CDS compared with those who had disabled it.

Prior studies of EHR use and clinical quality outcomes have been inconsistent in their findings. One study using earlier data from the same surveys used here found almost no associations between CDS use and clinical quality measures.

A more recent study found a significant, although small, improvement in blood pressure control of patients visiting physicians with CDS functions in their EHR versus patients visiting physicians without EHRs or CDS.

 In our study, we found an association between general CDS use and blood pressure control, and more consistent associations between quality indicators and CDS functions that were clinically related to these measures. These associations may be present because EHRs and CDS are evolving in a positive direction, spurred in part by the requirements of certification and also those of meaningful use.

Despite the positive findings, we have also found a lack of association between general CDS use and cancer screening, influenza vaccination, or health education, just as many previously have shown a lack of association between general EHR use and certain health outcomes.

 These findings may be because it is harder to change outcomes than quality processes. Some diseases, such as diabetes and hypertension, may be more amenable to clinical decision support than others, in part because there are clear guidelines and metrics.

This study has several limitations. We could not directly measure actual use of CDS during a visit, as the survey consisted of self-reported metrics. Even when physician practices have EHR features at their disposal, clinicians find ways to bypass or ignore them.20,21 Of the small number of practices that chose to disable CDS functionality, we do not know the reasoning behind their decisions, which may, in some cases, have been justified. In addition, the distinction between not having CDS functions and having turned off CDS may be sufficiently nuanced as to lead survey respondents to misclassify their EHR, although this would not affect our analysis of the difference between visits to clinics with all 5 of the CDS functions versus those without CDS. In addition, there were low rates of cancer screening and influenza vaccination captured in this visit-based incident data. There were also low rates of adverse drug event outcomes.

In contrast with some other studies, we found that the use of CDS functions was associated with improvements in measures of health quality at a national level. In addition, decisions to disable these CDS functions, although infrequent, negated these increases in care quality. With the continued implementation of meaningful use, there should be a further decline in practices without EHRs or that choose not to implement or to disable CDS. Overall, meaningful use standards that include CDS appear to have a significant positive effect on some national quality-of-care indicators and health outcomes. It will be important to evaluate the evolving impact of meaningful use as the stages continue to be more widely implemented and better integrated with care processes; we anticipate further gains in healthcare quality indicators and outcomes as a result.

 Division of General Medicine and Primary Care, Brigham Women's Hospital (RGM, JAL, DWB, AB); Brigham and Women’s Hospital (RGM, JAL, AB); Harvard Medical School (RGM, JAL, DWB, AB); Harvard Medical School Center for Primary Care (AB); Harvard School of Public Health (DWB), Boston, MA.

 Dr Mishuris is supported by an Institutional National Research Service Award to the Harvard Medical School Fellowship in General Medicine and Primary Care (T32 HP10251). No sponsor had a role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review or approval of the manuscript; or decision to submit the manuscript for publication.

Dr Bates holds a minority equity position in the privately held company Medicalis, which develops Web-based decision support for radiology test ordering, and he has served as a consultant to Medicalis. Dr Bates serves as an advisor to Calgary Scientific, which makes technologies that enable mobility within electronic health records. He is on the clinical advisory board for Patient Safety Systems, which provides a set of approaches to help hospitals improve safety. He has received funding support from the Massachusetts Technology Consortium. Dr Bitton serves part-time as a senior advisor to the Center for Medicare and Medicaid Innovation (CMMI). The research presented in this paper was conducted solely within the scope of his faculty appointments and research at Brigham and Women’s Hospital and Harvard Medical School, and in no way reflects any official position of CMMI. The authors report no other conflicts of interest.

 Concept and design (RGM, JAL, DWB, AB); acquisition of data (RGM, AB); analysis and interpretation of data (RGM, JAL, DWB, AB); drafting of the manuscript (RGM, JAL, AB); critical revision of the manuscript for important intellectual content (RGM, JAL, DWB, AB); statistical analysis (RGM, JAL, AB); administrative, technical, or logistic support (DWB); and supervision (JAL).

Using EHR clinical decision support is associated with improved quality of care. Most primary care practices are missing at least 1 "meaningful use" clinical decision support module.

Using Electronic Health Record Clinical Decision Support Is Associated With Improved Quality of Care

 To examine changes in patient experience across key domains of the patient-centered medical home (PCMH) following practice transformation with Lean quality improvement methodology inclusive of payment reform.

Pre-intervention/post-intervention analysis of intervention with a comparison group, a quasi-experimental design. We surveyed patients following office visits at the intervention (n = 2502) and control (n = 1622) practices during the 15-month period before and 14-month period after PCMH Lean transformation (April-October 2009).

We measured and compared pre-intervention and post-intervention levels of patient satisfaction and other indicators of patient-centered care. Propensity weights adjusted for potential case-mix differences in intervention and control groups; propensity-adjusted proportions accounted for physician-level clustering.

More intervention patients were very satisfied with their care after the PCMH Lean intervention (68%) compared with pre-intervention (62%). Among control patients, there was no corresponding increase in satisfaction (63% very satisfied pre-intervention vs 64% very satisfied post-intervention). This comparison resulted in a statistical trend (

 = .10) toward greater overall satisfaction attributable to the intervention. Post-intervention, patients in the intervention practice consistently rated indicators of patient-centered care higher than patients in the control practice, particularly in the personal physician and communication domain. In this domain, intervention patients reported superior provider explanations, time spent, provider concern, and follow-up instructions compared with control participants, whereas control group ratings fell in the post-intervention period (

 In a pilot PCMH transformation including Lean enhancement with payment reform, patient experience was sustained or improved across key PCMH domains.

Using a quasi-experimental design, we evaluated patient experience following patient-centered medical home (PCMH) practice transformation with Lean enhancement coupled with payment reform.

		Compared with controls, intervention participants were more likely to report faster speed of registration (

 = .04) and provide higher ratings across the personal physician and communication domain (

		62% of intervention participants were very satisfied pre-intervention versus 68% post-intervention (

		Future studies are needed to determine if physicians, over time, will thrive under an adjusted salary-based reimbursement system in the PCMH.

The patient-centered medical home (PCMH) has emerged as a patient-focused, quality-driven, cost-effective model to revive primary care. A growing body of literature supports the notion that health systems emphasizing the role of primary care can achieve superior outcomes at a lower cost.

 The PCMH joint principles were developed into a set of best practices by the National Committee for Quality Assurance and have been used as the blueprint for organizational change for many recent PCMH demonstration projects. As preliminary results of PCMH demonstration projects emerge, successes have shown that the PCMH has the potential to reduce cost and improve quality,

 although transformation is often challenging

In a pilot patient-centered medical home transformation including Lean quality improvement methodology with payment reform, patient experience was sustained or improved across key domains.

David W. Bates, MD, MSc

Articles

Physician and Patient Tools to Improve Chronic Kidney Disease Care

Enhanced Primary Care and Impact on Quality of Care in Massachusetts

Using Electronic Health Record Clinical Decision Support Is Associated With Improved Quality of Care

Patient-Centered Medical Home Transformation With Payment Reform: Patient Experience Outcomes

Active Pharmacovigilance and Healthcare Utilization