) strives to provide its readers with information that will support the transition to a more value-oriented healthcare system. Doing so involves knowledge from several different types of studies. At the center are studies that provide clinical information on value, such as the paper by Cohen-Mekelburg et al. This paper quantifies the readmission rates (1 in 7) associated with inflammatory bowel disease and suggests that more attention to postdischarge care for these patients may help reduce one source of readmissions. Oftentimes, the clinical evidence is nuanced: Klink et al study treatment for rheumatoid arthritis, noting the distinction in outcomes for different subsets of patients and arguing that managerial structures will be better suited to produce value if they can be more flexible. In this spirit, Sexton Ward et al assess the social value of prophylaxis of febrile neutropenia using a microsimulation model. Although they find that prophylaxis can add considerable value, a large challenge is determining who captures that value. Finally, Charlie Lin et al make a broader point that for some specialties, defining and measuring value is more complex.

A particular interest of the journal is providing evidence regarding managerial tools and methods that may promote value. For example, Liu et al examine ways to improve screening and follow-up for patients with unhealthy alcohol use. Vandergrift and Gray study the relationship between care quality and practice infrastructure. They note the importance of infrastructure in quality and that this effect is strongest for physicians who scored well on their Maintenance of Certification exam. Efforts to promote better infrastructure may be leveraged if we can find ways to support physicians in the lower tail of exam performance. Schwartz et al provide a more direct examination of the way in which infrastructure promotes value, noting that electronic health record systems are not designed to allow for the efficient ordering of high-value clinical services. We thus have a long way to go to adopt information technology systems that actually promote value. Finally, En-Ju Lin et al explore how artificial intelligence tools can help systems deal with the nuances of value by stratifying patients using sophisticated analytics.

A third category of research examines the financial relationships that drive value. Chambers et al report that health plans vary widely in the extent to which they restrict access to orphan drugs. Given the high price of prescription drugs and imperfection in the market, such restrictions are to be expected, but given this variation, more needs to be learned about their clinical and economic impact. 

 has been continually interested in evaluations of broader value-based payment reforms and, in that spirit, Yoon et al evaluate Oregon’s coordinated care organization program, an example of a population-based payment model in Medicaid, finding some positive impact on reducing preventable hospitalizations, consistent with other work.

’s efforts to provide insights useful to the wide range of stakeholders reached by the journal, hopefully supporting their daily efforts to improve healthcare in the United States.

Articles

From The Editorial Board

From the Editorial Board: Michael E. Chernew, PhD

 The use of electronic health record (EHR) systems by US clinicians is nearly ubiquitous. One motivation for EHR implementation is the ability to increase provider efficiency and improve patient-centered outcomes. There are no data examining how EHR design aligns with the ordering of high- and low-value clinical services.

 A survey of outpatient providers utilizing various EHR systems.

 Five high-value and 5 low-value services that would typically be ordered in a primary care setting were identified. Providers using different EHR systems quantified the number of computer clicks required to order each service.

 Five unique EHR systems representing those used by nearly two-thirds of health systems were included. No correlation was found between the ease of EHR ordering and the value of the clinical service. Three of the 5 services that were easiest to order were low value, and 3 high-value services were among the most difficult to order.

 In EHR systems used nationwide, no association existed between the clinical value of a service and the ease of ordering. This disconnect suggests that EHR redesign can significantly improve clinician workflow to facilitate the use of more high-value care and fewer low-value services.

Electronic health record (EHR) systems currently are not designed to allow for the efficient ordering of high-value clinical services. Our study evaluates if higher-value services are easier than low-value services to order.

There was no difference in the number of clicks required to order commonly used high- or low-value clinical services.

EHR systems could be better designed to facilitate the increased use of high-value clinical care and reduce low-value services.

Three-quarters of Americans feel that the United States does not get good value from its healthcare spending. This concern is partially driven by the systematic underuse of high-value care, coupled with the widespread provision of medical care services that do not improve patient-centered outcomes.

 One proposed strategy to enhance efficiency is the use of electronic health records (EHRs). Since the passage of the Health Information Technology for Economic and Clinical Health Act in 2009, EHR systems have become nearly ubiquitous in US health systems.

 EHR adoption was intended to both decrease costs and improve patient-centered outcomes via the use of clinical support tools, medication reconciliation, reduced duplication of services, and enhanced patient—provider communication.

 Although recent study results have demonstrated that EHR implementation has led to improvements in guideline adherence, patient satisfaction, and medical spending,

 the totality of available data is mixed.

Given that every aspect of an EHR is subject to human design, there is ample opportunity to create a system that encourages high-value care. Informed by the field of behavioral economics, which seeks to understand the rationality behind seemingly irrational human behaviors, several other studies have examined the inherent design of EHRs. For example, one nationwide survey of emergency departments looked at the default number of pills when a provider prescribed an opiate and noted default pill numbers as high as 90 tablets.

 Another study developed various “nudges” within the EHR to encourage less-intensive glycated hemoglobin (A1C) targets in appropriate patients.

Our study sought to evaluate whether any EHR system appears to be designed to facilitate the ordering of higher-value care. A behavioral economics model would suggest that making higher-value care easier to order would provide an intrinsic

incentive to deliver this care. However, we suspect that most current EHR systems do not facilitate efficient ordering behavior. Accordingly, the aim of this study was to quantify the ease of EHR ordering of specified high-value and low-value services.

From April to June 2018, a survey was sent to medical institutions to identify a sample of EHR systems. Prior to the survey being sent, the study was approved for institutional review board (IRB) exemption by the Michigan Medicine institutional IRB. Respondents who agreed to participate in the study were sent a standardized form that listed 5 high-value and 5 low-value services that would be frequently ordered in an outpatient primary care setting. Rationale for their inclusion was derived from published national guidelines (

). The services were listed in random order, in that the form did not specify whether the services were considered high or low value. Participants were instructed to count the number of EHR clicks required to order each of the services. When typing was required, it was predetermined that any typing would count as a single click, with the rationale being that number of characters typed would be highly variable among providers but that typing of any amount represented additional provider documentation.

Participants from 7 medical institutions, representing 5 unique EHR systems (Epic, CPRS, Practice Fusion, Allscripts, and Cerner), were identified and agreed to complete the survey. Responses were received and included from 5 systems: 3 academic institutions, 1 Veterans Affairs hospital, and 1 community hospital. Nearly all order entries (47/50) required some typing in addition to clicking. Glucometers were ordered outside the EHR workflow in 2 systems; flu shots were ordered outside the EHR in 1 system.

Little correlation was seen between the value of a service and ease of ordering (

). Of the 5 easiest services to order, 2 were designated high value (flu shot [9.0 clicks], A1C testing [9.8 clicks]) and 3 were deemed low value (prostate-specific antigen screening [10.0 clicks], vitamin D screening [10.2 clicks], lumbar x-ray for back pain [11.2 clicks]). Similarly, of the 5 most difficult to order, 3 were designated as high value (glucometer [12.0 clicks], pulmonary rehabilitation [13.6 clicks], colonoscopy [13.6 clicks]) and 2 as low value (upper endoscopy for gastric reflux [14.4 clicks], computed tomography angiogram for coronary calcium [16.28 clicks]).

Prior to the widespread adoption of EHRs, a proposed advantage of their adoption was to enhance provider efficiency. Our study sought to quantify the degree to which current EHR systems are designed to enhance the efficiency of providing high-value care. Based on our findings of a national sampling of EHR systems, representing nearly two-thirds of EHR users in hospitals,

 there was little correlation between the ease of ordering a particular clinical service and the strength of the evidence guiding the rationale for its use. Additionally, most of the high-value services required typing in all EHRs, which may further slow clinician workflow.

EHR vendors have promoted tools such as best practice advisories and other similar alerts to increase utilization of high-value services and reduce low-value care.

 Our study did attempt to quantify the use of these alerts, but only 2 systems were noted to contain any alerts for the given services. Therefore, although these alerts may provide useful clinical reminders,

 they do not appear to be a widespread tool for supporting high-value care based on our findings. Moreover, an increased reliance on alerts and reminders may increase alert fatigue and provider frustration with EHR systems.

 Results from several studies demonstrate that current EHR systems have increased clerical burden and frustration for clinicians and that this increased burden is associated with higher rates of burnout.

 Simplifying the ordering process for clinicians who are ordering established high-value services could decrease the workload for these clinicians as well as incentivize higher-value care by other clinicians.

Our study does have several limitations, including that not all EHR systems are represented. Additionally, we looked only at a small fraction of potential orders available to a primary care provider. Further, our study evaluated the ability to order services through a standard ordering process. However, many EHR systems also contain redundancy and customizability features. This may potentially affect the ordering of services in individual hospital systems.

To our knowledge, this is the first study to explicitly quantify how current EHR design fails to facilitate efficient ordering behavior. These findings imply that there is significant potential for EHRs to better support clinicians by making it easier to order evidence-based care and more difficult to access low-value care.

The authors would like to thank Jason Buxbaum, MHSA, and Chris Petrilli, MD, for their help in inspiring this work. Additionally, they would like to thank all who participated in our survey: Matt Heavner, MD; R.J. Schildhouse, MD; Jodie Barkin, MD; John Fleagle, MD; and Matthew Carey.

 Michigan Medicine (ES, AR), Ann Arbor, MI; Genesys Regional Medical Center (MK), Grand Blanc, MI; University of Michigan Center for Value-Based Insurance Design (AMF), Ann Arbor, MI.

 Dr Fendrick has been a consultant for AbbVie, Amgen, Bayer, Centivo, Community Oncology Alliance, Exact Sciences, Freedman Health, Lilly, Mallinckrodt, MedZed, Merck, Risalto, Sempre Health, State of Minnesota, Takeda, Wellth, and Zansors; has performed research for the Agency for Healthcare Research and Quality, Boehringer Ingelheim, Gary and Mary West Health Policy Center, Laura & John Arnold Foundation, National Pharmaceutical Council, Patient-Centered Outcomes Research Institute, PhRMA, Robert Wood Johnson Foundation, and State of Michigan/CMS; and holds outside positions as co-editor-in-chief of 

, member of the Medicare Evidence Development & Coverage Advisory Committee, and partner in V-BID Health, LLC. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (ES, AR, AMF); acquisition of data (ES, AR, MK); analysis and interpretation of data (ES, AR, MK, AMF); drafting of the manuscript (ES, AR, MK, AMF); critical revision of the manuscript for important intellectual content (ES, AR, MK, AMF); statistical analysis (ES, AR); and supervision (AMF).

Eric Schwartz, MD, Michigan Medicine, 1500 E Medical Center Dr, Ann Arbor, MI 48109. Email: schwarte@med.umich.edu.

1. Lind KD. How big is the problem of low-value health care service use? AARP website. aarp.org/content/dam/aarp/ppi/2018/03/how-big-is-the-problem-of-low-value-health-care-service-use.pdf. Published March 2018. Accessed September 15, 2018.

2. Henry J, Pylypchuk Y, Searcy T, Patel V. Adoption of electronic health record systems among U.S. non-federal acute care hospitals: 2008-2015. Office of the National Coordinator for Health Information Technology website. dashboard.healthit.gov/evaluations/data-briefs/non-federal-acute-care-hospital-ehr-adoption-2008-2015.php. Published May 2016. Accessed September 15, 2018.

3. Hillestad R, Bigelow J, Bower A, et al. Can electronic medical record systems transform health care? potential health benefits, savings, and costs. 

. 2005;24(5):1103-1117. doi: 10.1377/hlthaff.24.5.1103.

4. Adler-Milstein J, Everson J, Lee SY. EHR adoption and hospital performance: time-related effects. 

. 2015;50(6):1751-1771. doi: 10.1111/1475-6773.12406.

5. Campanella P, Lovato E, Marone C, et al. The impact of electronic health records on healthcare quality: a systematic review and meta-analysis. 

. 2016;26(1):60-64. doi: 10.1093/eurpub/ckv122.

6. Heekin AM, Kontor J, Sax HC, Keller MS, Wellington A, Weingarten S. Choosing Wisely clinical decision support adherence and associated inpatient outcomes. 

7. Blutinger EJ, Shofer FS, Meisel Z, Perrone J, Engel-Rebitzer E, Delgado MK. Variability in emergency department electronic medical record default opioid quantities: a national survey [published online March 12, 2019]. 

8. Chokshi SK, Belli HM, Troxel AB, et al. Designing for implementation: user-centered development and pilot testing of a behavioral economic—inspired electronic health record clinical decision support module. 

. 2019;5:28. doi: 10.1186/s40814-019-0403-z.

9. Final update summary: colorectal cancer: screening. US Preventive Services Task Force website. uspreventiveservicestaskforce.org/Page/Document/UpdateSummaryFinal/colorectal-cancer-screening. Updated July 2015. Accessed October 3, 2018.

10. Standards of Medical Care in Diabetes—2016: summary of revisions. 

. 2016;39(suppl 1):S4-S5. doi: 10.2337/dc16-S003.

11. Handelsman Y, Bloomgarden ZT, Grunberger G, et al. American Association of Clinical Endocrinologists and American College of Endocrinology—clinical practice guidelines for developing a diabetes mellitus comprehensive care plan&mdash;2015&mdash;executive summary. 

. 2015;21(4):413-437. doi: 10.4158/EP15672.GL.

12. Who should and who should not get a flu vaccine. CDC website. cdc.gov/flu/protect/whoshouldvax.htm. Published September 24, 2018. Accessed October 3, 2018.

13. Wedzicha JA, Miravitlles M, Hurst JR, et al. Management of COPD exacerbations: a European Respiratory Society/American Thoracic Society guideline. 

. 2017;49(3). doi: 10.1183/13993003.00791-2016.

14. American Association of Family Physicians. Don’t do imaging for low back pain within the first six weeks, unless red flags are present. Choosing Wisely website. choosingwisely.org/clinician-lists/american-academy-family-physicians-imaging-low-back-pain. Published April 4, 2012. Accessed October 3, 2018.

15. American College of Physicians. Don’t obtain imaging studies in patients with non-specific low back pain. Choosing Wisely website. choosingwisely.org/clinician-lists/american-college-physicians-imaging-for-non-specific-low-back-pain. Published April 4, 2012. Accessed October 3, 2018.

16. Final recommendation statement: vitamin D deficiency: screening. US Preventive Services Task Force website. uspreventiveservicestaskforce.org/Page/Document/RecommendationStatementFinal/vitamin-d-deficiency-screening. Published December 2016. Accessed October 3, 2018.

17. Final recommendation statement: prostate cancer: screening. US Preventive Services Task Force website. uspreventiveservicestaskforce.org/Page/Document/RecommendationStatementFinal/prostate-cancer-screening1. Published October 2018. Accessed October 3, 2018.

18. Final recommendation statement: cardiovascular disease: risk assessment with nontraditional risk factors. US Preventive Services Task Force website. uspreventiveservicestaskforce.org/Page/Document/RecommendationStatementFinal/cardiovascular-disease-screening-using-nontraditional-risk-assessment. Published July 2018. Accessed October 3, 2018.

19. Muthusamy VR, Lightdale JR, Acosta RD, et al; ASGE Standards of Practice Committee. The role of endoscopy in the management of GERD. 

. 2015;81(6):1305-1310. doi: 10.1016/j.gie.2015.02.021.

20. Peckham C. Medscape EHR report 2016: physicians rate top EHRs. Medscape website. medscape.com/features/slideshow/public/ehr2016#page=6. Published August 25, 2016. Accessed October 3, 2018.

21. Riaz H, Krasuski RA. Best practice advisories should not replace good clinical acumen. 

. 2017;130(3):245-246. doi: 10.1016/j.amjmed.2016.08.035.

TRENDS FROM THE FIELD

This study evaluates the ease of ordering high- and low-value clinical services in a national sample of electronic health records.

Ease of Ordering High- and Low-Value Services in Various Electronic Health Records

 To understand if and how one dimension of physician skill, clinical knowledge, moderates the relationship between practice infrastructure and care quality.

 We included 1301 physicians who certified in internal medicine between 1991 and 1993 or 2001 and 2003 and took the American Board of Internal Medicine (ABIM)’s Maintenance of Certification (MOC) exam and completed ABIM’s diabetes or hypertension registry during their 10-year recertification period between 2011 and 2014. Composite quality scores (overall, process, and intermediate outcome) were based on chart abstractions. Practice infrastructure scores were based on a web-based version of the Physician Practice Connections Readiness Survey. Our measure of clinical knowledge was drawn from MOC exam performance.

 We regressed a physician’s composite care quality scores against the interaction between their practice infrastructure and MOC exam scores with controls for physician, practice, and patient panel characteristics.

 We found that a physician’s exam performance significantly moderated the association between practice infrastructure and care quality (

 for interaction = .007). For example, having a top quintile practice infrastructure score was associated with a quality care score that was 7.7 (95% CI, 4.3-11.1) percentage points (

 <.001) higher among physicians scoring in the top quintile of their MOC exam, but it was unrelated (0.7 [95% CI, —3.8 to 5.3] percentage points; 

 = .75) to quality among physicians scoring in the bottom quintile on the exam.

 Physician skill, such as clinical knowledge, is important to translating patient-centered practice infrastructure into better care quality, and so it may become more consequential as practice infrastructure improves across the United States.

Physician clinical knowledge significantly moderated the relationship between patient-centered practice infrastructure and care quality.

We found that better practice infrastructure was associated with higher-quality care, especially performance on process measures, only among physicians scoring in the top 2 or 3 quintiles on their Maintenance of Certification exam.

These data suggest that individual physician skill may become more important as practice infrastructure improves across the country.

In addition, these findings highlight that it is important for patients to be informed about both practice infrastructure quality and a physician’s skill when selecting a doctor.

About a decade ago, several important policies were initiated that promoted practice infrastructure to support patient-centered care.

 In response, the adoption of basic electronic health records (EHRs) increased from 20% to more than 50% for office-based physicians between 2009 and 2015, and the number of providers participating in medical home initiatives, which utilize practice infrastructure to promote patient-centered care, increased from 14,000 to more than 63,000 during this period.

 Underlying these policies is the assumption that better practice infrastructure would lead to better patient care, although subsequent research consisting mostly of patient-centered medical home evaluations has yielded mixed evidence regarding the effectiveness of improving practice infrastructure.

 That said, heterogeneity observed across medical home interventions suggests that idiosyncratic aspects of the practices themselves may be critical to translating infrastructure improvements into higher-quality care.

One such factor that may be important to effectively leveraging infrastructure is the underlying skill of the physicians whose care is supported by practice infrastructure. Conceivably, practice infrastructure could preferentially benefit physicians with either higher or lower skill. For example, it might be that lower-skilled physicians benefit more from infrastructure supports, such as embedded clinical reminders or guideline standards, when developing care plans. Alternatively, synergy could exist between a physician’s skills and the ready access to patient information and electronic tools that are embedded in higher-quality systems, and so higher-skilled providers might deliver better care than lower-skilled providers when supported by practice infrastructure. However, we are unaware of any studies examining whether and how physician skill affects the relationship between practice infrastructure and care quality.

We address this gap by examining whether one dimension of physician skill, namely a physician’s clinical knowledge as measured by performance on the American Board of Internal Medicine (ABIM)’s Maintenance of Certification (MOC) examination, moderates the relationship between practice infrastructure and the quality of diabetes or hypertension care among general internists.

We identified 1301 primary care physicians (ie, nonsubspecializing general internists) who initially certified between either 1991 and 1993 or 2001 and 2003 and participated in either a diabetes or hypertension Process Improvement Module (PIM) registry from 2011 through 2014.

 PIM registries were completed during either the first or second 10-year recertification cycle and involved physicians recording data used to assess their care quality and completing a web-based version of the Physician Practice Connections Readiness Survey (PPC-RS) developed by the National Committee for Quality Assurance.

 Data were compiled and merged using ABIM administrative identifiers, which were removed prior to analysis. The study was deemed exempt from review by the Advarra institutional review board.

Each physician in the study abstracted information from 25 sequential (or randomly chosen) charts for patients aged 15 to 90 years with a visit in the past year and noted as having the applicable condition for at least a year. Prior research has demonstrated that patients treated by physicians who elected to complete the PIMs are representative of the patients with diabetes and hypertension typically treated by board-certified internists.

 From each chart, physicians abstracted the patient’s receipt of screening/monitoring tests, laboratory values, and demographics, which were used to assess concordance with process and intermediate outcome quality measures.

 Quality measures were used to construct 3 composites using an algorithm designed by an expert panel—(1) an overall quality composite, (2) a process composite, and (3) an intermediate outcomes composite&mdash;that each reflect the weighted average compliance across the individual measures (ranging from 0% to 100%; see 

] for individual measure criteria and scoring details).

As part of the PIM, each physician completed a web-based version of the PPC-RS.

 This survey is a self-assessment tool designed to enable physicians to benchmark their practice infrastructure across 7 subscores: (1) quality measurement and improvement; (2) patient data-tracking systems to follow patients with conditions identified by the physician as important to their practice; (3) appropriate standards, such as guideline reminders; (4) proactive management of important conditions; (5) patient-centered self-care support and education; (6) patient access and communication; and (7) systematic processes such as e-prescribing, test tracking, referral tracking, and interoperable EHRs. The overall score ranges from 0 to 100 and was computed using the National Committee for Quality Assurance scoring algorithm.

Our measure of physician clinical knowledge was their performance on ABIM’s MOC exam. Exam scores were equated to account for differences in difficulty between exam administrations and scored on a 200- to 800-point scale.

Physician control variables included their gender, whether they were born and/or attended medical school in the United States, the year and type of PIM completed, and whether they completed the PIM during their first or second round of MOC. Practice characteristics used included practice setting (ie, academic, group, or other), size, and HHS region.

 We also included zip code—level median household income and hospital service area–level age-, sex-, and race-adjusted Medicare mortality rate to account for the general well-being of the community in which each physician practices.

 Patient panel characteristics, collected in the PIMs for risk adjustment, were used as control variables in the analysis, including the percentage of women, percentage in each racial group (white non-Hispanic, black non-Hispanic, Hispanic, Asian or Pacific Islander, or other), percentage in each age group (<45, 45-54, 55-64, 65-74, or >74 years), and percentage with barriers to care (due to language or insurance/payment issues).

We used a share regression (ie, fractional response probit regression) to measure associations with our care quality measures.

 The advantage of this modeling approach is that it accounts for our dependent measure being a percentage score bounded by 0 and 1 (inclusive) and so, unlike linear regression, ensures that all predictions lie within this interval. Applying this model, we regressed each of the 3 quality composites against practice infrastructure score quintile indicators, MOC exam score quintile indicators, and the measures of physicians’ practice setting, training, demographic characteristics, and patient panel characteristics described previously. All controls were included in the final model, regardless of their statistical significance. To evaluate whether the relationship between quality care and practice infrastructure was moderated by physician clinical knowledge, we include interactions between the practice infrastructure score quintile indicators and MOC exam score quintile indicators. However, as we are using a nonlinear regression model, Ali and Norton point out that the magnitude, sign, and statistical significance of the interaction effect cannot be directly determined from the coefficients on the interaction terms.

 Karaca-Mandic and colleagues demonstrate that an appropriate method for calculating effects for such interactions is the cross partial derivative method.

 function in Stata version 14 (Stata Corp; College Station, Texas) to estimate the percentage-point difference in quality score (ie, the marginal effect based on the average derivative across the sample) between each practice infrastructure quintile indicator and the lowest practice infrastructure quintile at each quintile of exam performance.

 value, which tests the hypothesis that the value of the infrastructure—exam score quintile associations increases with higher value of both the exam score quintiles and practice infrastructure quintiles by applying a contrast test of the linear trend in predicted quality score marginal means across levels of both increasing practice infrastructure and increasing exam score quintiles using Stata’s 

 Lastly, we conducted a policy simulation for each exam score quintile to estimate the overall change in quality scores that might be expected if we assumed each physician moved from their observed quintile of practice infrastructure to the top quintile level of practice infrastructure.

In addition to examining the overall practice infrastructure score, we also conducted a secondary analysis in which we regressed the overall quality composite score against practice infrastructure subscores controlling for the remaining infrastructure score (ie, the total infrastructure score minus the subscore).

We found that practice infrastructure was more strongly associated with the overall quality score among physicians who performed better on their MOC exam (overall 

). For example, as shown in Table 1, the regression-adjusted difference in the overall quality score between the top and bottom practice infrastructure quintiles is 7.7 percentage points (95% CI, 4.3-11.1; 

 <.001) among physicians who scored in the top quintile of their MOC exam compared with 0.7 percentage points (95% CI, —3.8 to 5.3; 

 = .75) among physicians who scored in the bottom quintile of their MOC exam. The 

 demonstrates that despite a negative correlation existing between the MOC exam and practice infrastructure scores (

 = .004), a sizable portion of high and low exam performers worked in both high- and low-infrastructure practices, suggesting enough variation existing within levels of exam performance to evaluate the interaction. See 

 for the main effect associations without any interactions, 

 for the probit regression coefficients, and 

 for mean values of the control variables.

 describes the results of the policy simulation in which we estimated the potential increase in the overall quality score associated with moving every physician from their observed practice infrastructure quintile to the top practice infrastructure quintile. Among physicians scoring in the top exam quintile, transitioning each physician from their observed practice infrastructure quintile to the top quintile predicts an overall increase in the average overall quality score of 3.7 percentage points (95% CI, 1.4-6.0; 

 = .001). This increase in quality would account for a 12% reduction in the gap in quality between the average quality score (70.1%) and the maximum possible quality score. These simulations were not significant for physicians scoring in the bottom quintile (0.4 percentage points; 95% CI, —2.3 to 3.0; 

 describes the associations when we separately examine the process or intermediate outcome quality scores. The interaction between practice infrastructure and physician clinical knowledge was statistically significant only for the process composite score (overall 

 value of the interaction = .006). For example, among physicians with a top-quintile exam score, practicing in a top- versus bottom-quintile infrastructure system was associated with a process quality score that was 14.1 percentage points higher (95% CI, 9.0-19.1; 

 <.001). This association was 1.8 percentage points and not statistically significant (95% CI, —4.8 to 8.5; 

 = .59) for physicians who performed in the bottom quintile on their MOC examination.

 delineates analyses examining the 7 practice infrastructure subscores. We found that physician exam performance significantly moderated the association between infrastructure and overall quality for only subscore 7 (having systematic processes such as e-prescribing, test tracking, referral tracking, and interoperable EHRs). The association between a 5-point increase in this subscore was associated with an increase of 1.0 percentage points (95% CI, 0.1-2.0; 

 = .03) in the overall quality score among physicians in the top quintile of exam performance, but it was not statistically significant among physicians with bottom-quintile exam scores (—1.0 percentage points; 95% CI, –2.2 to 0.3; 

We found that individual physicians may be key to translating high-quality practice infrastructure into better care quality experienced by patients, as measured by each physician’s performance across an amalgam of individual process and intermediate outcome measures. In particular, we observed that better practice infrastructure was associated with higher care quality only among physicians with better clinical knowledge, as indicated by their scoring in either of the top 2 quintiles on ABIM’s MOC exam. In general, the observed association with care quality was largely driven by better performance on process-of-care measures.

Why might practice infrastructure matter for physicians who performed well on their MOC exam but not for physicians who performed more poorly? To help address this question, we examined the individual subscores that comprise the overall practice infrastructure score. We discovered significant differences in the practice infrastructure—quality relationship between top- and bottom-quintile exam performers for the practice infrastructure subscore related to having systematic processes for managing health information. This includes infrastructure elements such as having an EHR that stores structured patient data, including from external organizations; a system for managing and tracking referrals; and a system for ordering tests, retrieving results, and tracking whether they have been completed. This suggests that infrastructure elements that provide ready access to patient information and track whether care plans are being followed may be important drivers of quality among physicians with higher clinical knowledge. For example, a physician with a higher level of clinical knowledge may be able to take advantage of these infrastructure elements to ensure that their patients are following through on care plans that may be more comprehensive than for a physician with a lower level of clinical knowledge. However, if these 2 physicians work in a system where they have difficulty accessing patient records or tracking their patients’ health data, then neither might be able to consistently deliver high-quality care, regardless of how comprehensive their care plans are, because they may be limited in their ability to delineate patients who would benefit from more follow-up.

 Another explanation for our findings is that a dimension of physician skill other than knowledge, such as professionalism, may be correlated with both clinical knowledge and diligence in applying the information and tools accessible in high infrastructure systems. For example, it could be that physicians with a greater commitment to professionalism may be both more committed to maintaining their clinical knowledge, and so score better on their MOC exam, and more motivated to use higher-quality infrastructure to improve their practice, and so take the time to experiment with and optimize use of these systems for improving care. However, in the absence of high-level practice infrastructure, these physicians may be hamstrung in their ability to improve quality. Supporting this, Haber and colleagues have found that performance on ABIM’s certification examination is correlated with other Accreditation Council for Graduate Medical Education physician core competencies, including patient care and professionalism.

 Furthermore, Pham and colleagues report based on interviews with physicians that those physicians who scored well on ABIM’s examination tended to be more reflective in discussing their clinical quality, often describing their “good performance as an amalgamation of skills.”

 In contrast, physicians who scored lower on their MOC exams tended to be less reflective and typically framed their quality in terms of their numerical scores compared with their peers.

These findings suggest that evaluations of interventions designed to improve practice infrastructure should consider characteristics of the providers who comprise these practices, regardless of whether they are driven by differences in clinical knowledge or another correlated dimension of physician skill.

 This is consistent with a recent review of medical home evaluations that reported a great deal of heterogeneity in effectiveness across interventions, which the authors partially attributed to local contextual factors.

 In part, this could be due to differences in the medical home interventions themselves, such as variable implementation strategies or placing more or less emphasis on particular practice infrastructure elements.

 However, our findings suggest that characteristics of the physicians comprising a practice may be important to successfully translating medical home interventions into higher-quality care and so should be considered by policy makers looking to encourage such practice transformations. Further, that we found physicians who performed slightly less well on their MOC exam to have better practice infrastructure suggests that opportunities exist to more efficiently distribute infrastructure investment. These data suggest that it would be more efficient if high-quality infrastructure were positively correlated with exam performance, given our observation that the mixture of higher-quality infrastructure and better exam performance is associated with higher-quality care.

Several limitations must be considered when applying our results. First, because we rely on cross-sectional comparisons, we cannot account for reverse causality in the practice infrastructure and care quality relationship. As such, the overall positive correlation observed between practice infrastructure and care quality might simply be the result of high-quality practices being more likely to improve their infrastructure. However, if these results were the product of high-quality practices acquiring better practice infrastructure, then one would expect the relationship between practice infrastructure and quality to be the same regardless of how the physicians who practice in these systems performed on their MOC exam. That we found the practice infrastructure—quality relationship to be moderated by a physician’s clinical knowledge suggests that for reverse causality, or an unobserved confounding factor, to explain our findings, it would have to be present only among physicians who scored well on their MOC exam and work in a high-infrastructure practice but not present among physicians who (1) scored well but work in a low-infrastructure practice, (2) scored poorly but work in a high-infrastructure practice, or (3) scored poorly and work in a low-infrastructure practice.

Another limitation is that these data reflect only physicians who completed a diabetes or hypertension PIM. In order to complete either PIM, a physician needs a patient panel that is comprised of enough patients with either condition. However, these conditions are widely prevalent among adults in the United States

 and so presumably most primary care physicians would have a practice capable of supporting either of these PIMs. Further, prior research using PIM data has found that the patients treated by physicians who completed a diabetes or hypertension PIM are representative of the patients typically cared for by primary care physicians certified in internal medicine.

 That said, these are all physicians who decided to maintain their certification and so these findings may not generalize to physicians who do not participate in MOC.

We found that better practice infrastructure was associated with higher-quality care only among physicians who performed well on their MOC exam. This finding implies that individual physician skill may be important to successfully translating infrastructure improvements into higher-quality care and that future research is needed to understand what types of support are needed to help lower-skilled physicians leverage practice infrastructure. This is especially important given that these latter physicians are likely to treat underserved populations.

 These findings also highlight that it is important for patients to be informed about both practice infrastructure quality and physician skill when selecting a doctor.

 American Board of Internal Medicine (JLV, BMG), Philadelphia, PA.

 Financial and material support was provided by the American Board of Internal Medicine. Data used in the current study were collected by the American Board of Internal Medicine.

These data were in part previously presented at the 2016 Academy Health Annual Research Meeting.

 The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (JLV, BMG); analysis and interpretation of data (JLV, BMG); drafting of the manuscript (JLV, BMG); critical revision of the manuscript for important intellectual content (JLV, BMG); and statistical analysis (JLV, BMG).

 Jonathan L. Vandergrift, MS, American Board of Internal Medicine, 510 Walnut St, Ste 1700, Philadelphia, PA 19106. Email: Jvandergrift@abim.org.

1. American Academy of Family Physicians; American Academy of Pediatrics; American College of Physicians; American Osteopathic Association. Joint principles of the patient-centered medical home. American College of Physicians website. acponline.org/system/files/documents/running_practice/delivery_and_payment_models/pcmh/demonstrations/jointprinc_05_17.pdf. Published March 2007. Accessed August 26, 2019.

2. Rittenhouse DR, Shortell SM, Fisher ES. Primary care and accountable care—two essential elements of delivery-system reform. 

. 2009;361(24):2301-2303. doi: 10.1056/NEJMp0909327.

3. Patient-centered medical home (PCMH). National Committee for Quality Assurance website. ncqa.org/programs/recognition/practices/patient-centered-medical-home-pcmh. Accessed August 26, 2019.

4. Washington V, DeSalvo K, Mostashari F, Blumenthal D. The HITECH era and the path forward. 

. 2017;377(10):904-906. doi: 10.1056/NEJMp1703370.

5. Bitton A, Martin C, Landon BE. A nationwide survey of patient centered medical home demonstration projects. 

. 2010;25(6):584-592. doi: 10.1007/s11606-010-1262-8.

6. Edwards ST, Mafi JN, Landon BE. Trends and quality of care in outpatient visits to generalist and specialist physicians delivering primary care in the United States, 1997-2010. 

. 2014;29(6):947-955. doi: 10.1007/s11606-014-2808-y.

POLICY

Patient-centered practice infrastructure was associated with better care quality only among physicians who scored well on their Maintenance of Certification exam.

Physician Clinical Knowledge, Practice Infrastructure, and Quality of Care

 To compare coverage of orphan and nonorphan drugs, to examine variation in orphan drug coverage across the largest US private plans, and to evaluate factors influencing coverage decisions.

 We analyzed a data set of private health plan coverage decisions for specialty drugs (N = 5000) in 3 ways. First, we compared the frequency with which plans applied restrictions in their decisions for orphan and nonorphan drugs. Second, we examined variation in the frequency with which 17 of the largest 20 private plans applied coverage restrictions for orphan drugs. Third, we used multivariate regression to examine factors associated with restricted orphan drug coverage.

 Health plans are less likely to restrict orphan drugs compared with nonorphan drugs. Of orphan drug decisions (n = 2168), plans did not apply coverage restrictions in 70% of cases, applied restrictions in 29%, and did not cover in 1%. In contrast, of nonorphan drug decisions (n = 2832), plans did not apply coverage restrictions in 53% of cases, applied restrictions in 41%, and did not cover in 6%. The frequency of restrictions for orphan drugs varied from 11% to 65% across plans. The attributes of orphan drugs that were more likely to be associated with restrictions than others included drugs for noncancer diseases, drugs with alternatives, self-administered drugs, drugs indicated for diseases with a higher prevalence, and drugs with higher annual costs (all 

 Health plans restrict access to orphan drugs approximately one-third of the time, and restrictions vary considerably across plans. Plans more often add restrictions for orphan drugs that are indicated for diseases with a higher prevalence and that have higher annual costs.

Private US health plans less often apply restrictions in their coverage decisions for orphan drugs than in their coverage decisions for nonorphan drugs (30% vs 47% of the time, respectively).

Health plans vary in the frequency with which they apply restrictions in their orphan drug coverage decisions.

Private US health plans more often add orphan drug coverage restrictions (eg, patient subgroup restrictions, step therapy protocols) for drugs for diseases other than cancer, drugs with alternatives, self-administered drugs, drugs indicated for diseases with a higher prevalence, and drugs with higher annual costs.

Orphan drugs (ie, drugs indicated for rare diseases) pose a singular challenge for health plans. Although they target small patient populations, their high costs, rising numbers, and strong backing by patient groups force plans to confront ever-more-delicate decisions about how to provide access, even in the face of sometimes highly uncertain evidence.

 The challenge is intensifying: From 1997 to 2017, spending on orphan drugs grew from 4% to 10% of total prescription drug spending, with spending on orphan drugs totaling $43 billion in 2017.

 An estimated 25 million to 30 million Americans live with an orphan disease.

Eighty-seven percent of orphan drug spending is made up of specialty drugs, which are typically complex large-molecule drugs that require special administration, monitoring, and handling, although high-priced small-molecule drugs are also often considered specialty drugs.

 Specialty drugs are also used to treat a range of nonorphan conditions, including rheumatoid arthritis, hepatitis C, Crohn disease, and multiple sclerosis.

Research has demonstrated that health plans sometimes restrict patient access to orphan drugs. Results of one study found that private plans apply restrictions in roughly a quarter of orphan drug coverage decisions.

 Another study reported that Medicare and private plans often place orphan drugs on the highest co-payment tier of their formularies.

However, the circumstances under which plans restrict access to orphan drugs have not been reported. It is unclear, for example, whether plans are more likely to restrict orphan drugs with larger budget impacts. Knowledge of these circumstances would help patients, providers, and policy makers better understand the coverage decisions affecting access. We analyzed a database of private health plan coverage decisions to (1) compare coverage of orphan and nonorphan drugs, (2) examine variation in orphan drug coverage across the largest US private plans, and (3) evaluate factors influencing orphan drug coverage decision making.

We identified orphan drug coverage decisions using the Tufts Medical Center Specialty Drug Evidence and Coverage (SPEC) database. SPEC includes publicly available specialty drug coverage decisions issued by 17 of the 20 largest US private health plans. SPEC includes coverage decisions relevant to the included plans’ commercial lines of business. Of the 3 largest plans excluded from SPEC, 1 does not make its coverage decisions publicly available and 2 focus exclusively on Medicare or Medicaid populations.

SPEC includes information on health plans’ specialty drug coverage decisions and the supporting evidence that they cite. SPEC details drug-level information, including the number of years since a drug’s FDA approval and whether a drug is indicated for a pediatric population. Roughly half (51%) of drugs in SPEC are physician-administered and covered through the plans’ medical benefit. The remaining drugs are self-administered drugs, which plans cover through their pharmacy benefit.

We define a coverage restriction as a health plan decision to reduce the size of the eligible patient population compared with the population described in the FDA label. We further classify restrictions into categories: (1) patient subgroup (requirement for patients to meet certain clinical criteria, such as disease severity), (2) step therapy protocol (requirement for patients to first fail an alternative treatment before gaining access to a drug), (3) prescriber restriction (requirement for a certain type of physician—for example, a rheumatologist&mdash;to prescribe the drug), (4) combination therapy (requirement for a drug to be used concomitantly with another medication), and (5) other.

 SPEC includes 204 drugs. Because a drug can be indicated for multiple diseases, the database includes 409 unique drug—indication pairs (186 orphan drug–indication pairs; 223 nonorphan drug–indication pairs). For example, because ruxolitinib is indicated for myelofibrosis and polycythemia vera (both orphan diseases), it appears twice in the database. SPEC includes 5000 coverage decisions (2168 for orphan drugs; 2832 for nonorphan drugs) across the 17 health plans. Included decisions are current as of March 2018.

The dependent variable is dichotomous (yes/no) and describes whether the health plan’s coverage decision restricted patients’ access to a drug beyond the drug’s FDA label. “No restrictions” means a plan covered the drug consistent with its label. Coverage restrictions included decisions in which the plan did not cover the drug or the plan applied some restriction (for example, by step therapy protocol).

 We include 10 independent variables that we selected either because they have been previously shown to be associated with restricted drug coverage

 or because we consider them likely to be associated with coverage restrictions. 

 is a dichotomous variable capturing whether a drug is indicated to treat cancer. 

 is a dichotomous variable capturing whether a drug has any therapeutic alternatives (ie, whether the FDA has approved another drug for the same indication). 

 is a dichotomous variable accounting for whether a drug is indicated for pediatric or adolescent patients. Examples include nusinersen for spinal muscular atrophy and velaglucerase alfa for Gaucher disease type 1. 

 is a continuous variable accounting for the number of years since a drug’s FDA approval. When a drug is approved for multiple orphan indications, we consider the approval date for each indication separately. For example, tocilizumab was approved to treat polyarticular juvenile idiopathic arthritis in 2013 and approved for cytokine release syndrome in 2017; thus, the time since FDA approval differs for these indications. 

 is a dichotomous variable describing whether the drug has been flagged by the FDA for a safety concern (specifically, a black box safety warning or implementation of a Risk Evaluation and Mitigation Strategy program).

 is a dichotomous variable accounting for whether the FDA included the drug in one of its expedited approval programs, including priority review, fast track, accelerated approval, or breakthrough therapy.

 is a dichotomous variable capturing whether the drug is self- or physician-administered. We categorize tablets, capsules, and subcutaneously self-injected drugs as being self-administered, and drugs injected or infused in inpatient or outpatient settings as being physician-administered. 

 is a dichotomous variable accounting for situations in which an orphan drug is also indicated for a nonorphan indication. For instance, adalimumab is approved for orphan indications (eg, pediatric Crohn disease) and nonorphan indications (eg, rheumatoid arthritis). 

 is a categorical variable accounting for the prevalence of the indicated disorder. We obtained estimates of US disease prevalence from a review of the medical literature and the websites of national clinical organizations. To construct the variable, we divided observations into quartiles based on prevalence magnitudes. 

 is a categorical variable capturing the yearly per-patient drug cost (and dividing observations into quartiles). We estimated these costs using dosing information from the drug’s label and the Federal Supply Schedule (FSS) price (federal purchaser price based on a drug’s average sales price inclusive of rebates).

 If the FSS drug price was unavailable, we relied on alternative sources, including the CMS Average Sales Price (a product’s average sales price as submitted to CMS) and the IBM Micromedex RED BOOK (a proprietary source of drug pricing).

We analyzed the data in 3 ways. First, we compared the frequency with which plans applied coverage restrictions for nonorphan and orphan drugs. Second, we evaluated variation in orphan drug coverage across health plans. Third, we developed a multilevel, random-effect, logistic regression model to estimate the relationship between the independent variables and the likelihood of restrictive orphan drug coverage. We used a multilevel model to estimate the average effect of an independent variable on coverage across health plans while accounting for differences among them. We report associations between independent variables and coverage restrictiveness using odds ratios, which describe how much larger or smaller the odds of restricted coverage are for an independent variable (eg, drugs with alternatives compared with drugs with no alternatives). We considered 

Health plans restrict orphan drug coverage less often than nonorphan drug coverage. However, the frequency of restrictions varies considerably across plans.

October 2019

From the Editorial Board: Michael E. Chernew, PhD

Ease of Ordering High- and Low-Value Services in Various Electronic Health Records

Physician Clinical Knowledge, Practice Infrastructure, and Quality of Care

Variation in US Private Health Plans’ Coverage of Orphan Drugs

Inflammatory Bowel Disease Readmissions Are Associated With Utilization and Comorbidity

The Long-term Social Value of Granulocyte Colony-Stimulating Factors

A Deep Learning Model for Pediatric Patient Risk Stratification

Low Screening and Follow-up for Unhealthy Alcohol Use Among Health Plan Beneficiaries

Patients’ Expectations of Their Anesthesiologists

Can Accountable Care Divert the Sources of Hospitalization?

Real-World Outcomes Among Patients With Early Rapidly Progressive Rheumatoid Arthritis