Currently Viewing:
The American Journal of Managed Care January 2015
Disease-Modifying Therapy and Hospitalization Risk in Heart Failure Patients
Fadia T. Shaya, PhD, MPH; Ian M. Breunig, PhD; and Mandeep R. Mehra, MD, FACC, FACP, FRCP
Frequency and Costs of Hospital Transfers for Ambulatory Care-Sensitive Conditions
R. Neal Axon, MD, MSCR; Mulugeta Gebregziabher, PhD; Janet Craig, PhD, RN; Jingwen Zhang, MS; Patrick Mauldin, PhD; and William P. Moran, MD, MS
Celebrating Our 20th Anniversary
A. Mark Fendrick, MD, and Michael E. Chernew, PhD Co-Editors-in-Chief, The American Journal of Managed Care
Value-Based Insurance Design: Benefits Beyond Cost and Utilization
Teresa B. Gibson, PhD; J. Ross Maclean, MD; Michael E. Chernew, PhD; A. Mark Fendrick, MD; and Colin Baigel, MBChB
Currently Reading
Changing Physician Behavior: What Works?
Fargol Mostofian, BHSc; Cynthiya Ruban, BSc; Nicole Simunovic, MSc; and Mohit Bhandari, MD, PhD, FRCSC
Relationship of Diabetes Complications Severity to Healthcare Utilization and Costs Among Medicare Advantage Beneficiaries
Leslie Hazel-Fernandez, PhD, MPH; Yong Li, PhD; Damion Nero, PhD; Chad Moretz, ScD; S. Lane Slabaugh, PharmD, MBA; Yunus Meah, PharmD; Jean Baltz, MMSc, MSW; Nick C. Patel, PharmD, PhD; and Jonathan R. Bouchard, MS, RPh
Revisiting Hospital Length of Stay: What Matters?
Mollie Shulan, MD; and Kelly Gao
Medical Homes: Cost Effects of Utilization by Chronically Ill Patients
Jason Neal, MA; Ravi Chawla, MBA; Christine M. Colombo, MBA; Richard L. Snyder, MD; and Somesh Nigam, PhD
Value-Based Insurance Design and Medication Adherence: Opportunities and Challenges
Kevin A. Look, PharmD, PhD
New Start Versus Continuing Users on Aripiprazole: Implications for Policy
Rashid Kazerooni, PharmD, BCPS; Joseph B. Nguyen, PharmD, BCPS; Mark Bounthavong, PharmD, MPH; Michael H. Tran, PharmD, BCPS; and Nermeen Madkour, PharmD, CSP
Multiple Chronic Conditions in Type 2 Diabetes Mellitus: Prevalence and Consequences
Pei-Jung Lin, PhD; David M. Kent, MD, MSc; Aaron Winn, MPP; Joshua T. Cohen, PhD; and Peter J. Neumann, ScD
Prognostic Factors of Mortality Among Patients With Severe Hyperglycemia
Ya-Wun Guo, MD; Tzu-En Wu, MD, MS; and Harn-Shen Chen, MD, PhD
Survey Nonresponders Incurred Higher Medical Utilization and Lower Medication Adherence
Seppo T. Rinne, MD, PhD; Edwin S. Wong, PhD; Jaclyn M. Lemon, BS; Mark Perkins, PharmD; Christopher L. Bryson, MD; and Chuan-Fen Liu, PhD
Using Financial Incentives to Improve the Care of Tuberculosis Patients
Cheng-Yi Lee, MS; Mei-Ju Chi, PhD; Shiang-Lin Yang, MS; Hsiu-Yun Lo, PhD; and Shou-Hsia Cheng, PhD

Changing Physician Behavior: What Works?

Fargol Mostofian, BHSc; Cynthiya Ruban, BSc; Nicole Simunovic, MSc; and Mohit Bhandari, MD, PhD, FRCSC
The authors evaluate methods for implementing clinical research and guidelines, in order to change physician practice patterns, in surgical and general practice.
There are various interventions for guideline implementation in clinical practice, but the effects of these interventions are generally unclear. We conducted a systematic review to identify effective methods of implementing clinical research findings and clinical guidelines to change physician practice patterns, in surgical and general practice.

Study Design
Systematic review of reviews.

We searched electronic databases (MEDLINE, EMBASE, and PubMed) for systematic reviews published in English that evaluated the effectiveness of different implementation methods. Two reviewers independently assessed eligibility for inclusion and methodological quality, and extracted relevant data.

Fourteen reviews covering a wide range of interventions were identified. The intervention methods used include: audit and feedback, computerized decision support systems, continuing medical education, financial incentives, local opinion leaders, marketing, passive dissemination of information, patient-mediated interventions, reminders, and multifaceted interventions. Active approaches, such as academic detailing, led to greater effects than traditional passive approaches. According to the findings of 3 reviews, 71% of studies included in these reviews showed positive change in physician behavior when exposed to active educational methods and multifaceted interventions.

Active forms of continuing medical education and multifaceted interventions were found to be the most effective methods for implementing guidelines into general practice. Additionally, active approaches to changing physician performance were shown to improve practice to a greater extent than traditional passive methods. Further primary research is necessary to evaluate the effectiveness of these methods in a surgical setting.

Am J Manag Care. 2015;21(1):75-84
This paper compares implementation methods that are currently dispersed among literature and specialties. Findings show that commonly used passive interventions in practice today (eg, printed educational material) are less effective than active methods (eg, continuing medical education workshops and tailored multifaceted interventions).
  • Our study indicates that practices should focus on implementation of active methods to change physician behavior and limit use of passive dissemination of educational material or formal didactic conferences.
  • Future research should focus on testing and adapting these implementation methods to specific environments, such as surgery.
  • The cost-effectiveness of these interventions should be studied in future research.
There is increasing recognition of the difficulty in translating research evidence and clinical guidelines into practice, which has resulted in the development of many active dissemination and implementation strategies.1 Although there is a substantial amount of primary research evidence concerning the effectiveness of various implementation methods, it is extensively dispersed amongst medical specialties.1

Research evidence should ideally inform the development of clinical practice guidelines (CPGs).2 CPGs are systematically developed and updated, and are evidence-based.3 They provide physicians with a framework for diagnosing, assessing, and treating clinical conditions commonly encountered in practice, and are developed to promote best practices for patient populations. The implementation of these guidelines is important to help improve the quality and consistency of care in clinical situations by changing physician practice patterns.3

The Figure demonstrates the steps involved in implementing research findings into medical practice. After dissemination of CPGs, there are 6 main factors specific to healthcare providers that effect guideline adoption into practice and physician behavior: guideline implementation, characteristics of practice, laws and incentives, patient characteristics/problems, social norms, and knowledge and skills.4,5 Understanding the relative effectiveness of guideline implementation methods is necessary to changing physician behavior for the better and improving patient outcomes.5

Overall, just a small number of reviews unify different methods and evaluate the implementation methods proven most successful in changing physician behavior.2 Moreover, there is a lack of research on intervention methods specific to surgery.1 We have undertaken a systematic review to inform changing physician practice patterns by evaluating methods for implementing clinical research and guidelines. More specifically, we addressed the following research question: in surgical and general practice, through what methods are clinical research results, as well as guidelines, best implemented to change physician practice patterns? A secondary focus of this study was to determine implementation methods shown to be effective in a surgical setting, specifically in orthopedics.


Assessment of Eligibility

Reviews fulfilling the following criteria (1-4 and either condition 5 or 6) were eligible for inclusion: 1) topic linking research/guideline to practice, 2) education or other implementation method of guidelines, 3) systematic reviews, 4) English language, 5) all surgery types and postoperative care, and 6) general practice (hospital and private).

Reviews were excluded for the following reasons: 1) published before 1970; 2) guidelines publications; 3) conference and letter reviews; 4) reviews focused on nonsurgical topics (ie, those that are too specialized to be considered general practice, as they only focus on a specific condition [eg, stroke/cardiology, urology, gynecology, pathology/bacterial infection, dentistry, rehab, nursing, palliative care, pharmaceuticals/analgesics, psychiatric, vaccination, and diabetes]); and 5) literature reviews.

Recent changes to medicine and technology rendered any evidence published before 1970 of limited relevance to current and future practice. Reviews were screened independently in duplicate at the title, abstract, and full-text stage based on the eligibility criteria. All disagreements were resolved by a consensus process that required the reviewers to discuss their rationale for their decisions.

Identification of Reviews

In order to identify appropriate search terms, 2 reviewers (whose anonymity will be maintained) each conducted an independent preliminary electronic search and selected 5 to 6 reviews appropriate to the topic. Key terms were identified from these reviews to develop a broad search strategy.

We searched the electronic databases EMBASE, MEDLINE, and PubMed for relevant articles published prior to October 6, 2012, using a combination of the identified search terms (eAppendix Table 1, available at

Critical Appraisal Score

The methodological quality of each included review was independently graded, using an adapted version of the AMSTAR critical appraisal tool.6 The AMSTAR tool has been shown to have good construct validity.6 A copy of this adapted appraisal score tool can be found in eAppendix Table 2.

For the critical appraisal scores, a “yes” to an assessment question was given a score of 1. The maximum score for the adapted AMSTAR questionnaire was 11, and the minimum score was 0. We specified that a score of 9 or higher indicated high methodological quality, 5 to 8 moderate quality, and 4 or less low quality. The reviewers resolved discrepancies for each item through discussion and re-evaluation of the study methodology until a consensus was reached.

Assessment of Agreement

Inter-rater agreement for each screening step was conducted using a weighted kappa (κ) statistic. Cohen’s κ values of less than 0 were rated as less than chance agreement; 0.01-0.20, slight agreement; 0.21-0.40, fair agreement; 0.41-0.60, moderate agreement; 0.61-0.80, substantial agreement; and >0.80, high agreement.7

The inter-rater agreement for the critical appraisal score was determined using an intraclass correlation coefficient (ICC; 2-way mixed model, single measure). ICC values were interpreted as follows: 0 to 0.2, poor agreement; 0.3 to 0.4, fair agreement; 0.5 to 0.6, moderate agreement; 0.7 to 0.8, strong agreement; and >0.8, high agreement.

All statistical analyses were conducted using SPSS v18.0 (IBM Corp, Armonk, New York).

Data Collection and Data Abstraction

Two reviewers independently abstracted data from the full-text articles using a previously piloted data abstraction form. They abstracted information on the form of implementation method, the subject focus group (eg, general physicians, surgeons, medical students, etc), number of included studies’ outcome measures (physician performance outcomes and patient outcomes), definition of effectiveness, and the most effective and least effective implementation methods recommended by the review.

After data abstraction, the reviews were grouped based on implementation method: audit and feedback, continuing medical education, other interventions, and comparison of interventions. The “other interventions” category included reviews that analyzed incentives, decision support systems, journals, and printed educational materials (PEMs) as single interventions. The last category consisted of reviews that compared the effectiveness of 2 or more interventions.

Finally, a spectrum of least effective to most effective intervention methods was created to summarize the findings of this paper. To create this spectrum, the relative effectiveness of the different implementation methods was compared by considering the AMSTAR scores of the reviews. Results from studies with high methodological quality were given greater weight than results of studies with lower methodological quality.8

Statistical Analysis

Descriptive statistics were presented for all systematic reviews that reported them. For reviews that compared 2 or more interventions, we reported adjusted relative risk (ARR) and adjusted risk difference (ARD) values if included in the review. For reviews assessing a single intervention, we presented the percentage of compliance of physicians with guidelines or the percentage of studies included in the review that showed significant change in outcome measure. Lastly, implementation methods were deduced highly effective, moderately effective, and ineffective based on description by the authors of the original reviews.


Included Studies

Our literature search identified 1592 potentially relevant citations, of which, 14 reviews were included (eAppendix Figure).

Inter-rater agreement was fair for the title screening stage (κ = 0.480; 95% CI, 0.439-0.52) and moderate for the abstract screening stage (κ = 0.726; 95% CI, 0.661-0.791) and full text screening stage (κ = 0.560; 95% CI, 0.454-0.665).

Study Quality

We judged 5 reviews to be of high methodological quality,9-13 5 reviews to be of moderate quality,14-18 and the remaining 4 reviews to be low quality (Table 1).2,5,19,20 There was high agreement between reviewers for the critical appraisal score (0.946; 95% CI, 0.848-0.974).

Study Characteristics

Various intervention methods were used, including audit and feedback, computerized decision support systems, continuing medical education, financial incentives, local opinion leaders, marketing, passive dissemination of information, patient-mediated interventions, reminders, and multifaceted interventions. These interventions and additional terminology used in this paper are defined in Table 2. Table 1 also summarizes some of the general study findings. Six reviews had qualitative measures and did not define specific parameters on how they identified implementation methods as highly effective, moderately effective, or ineffective.2,5,14,16,18,20

Audit and Feedback

Three reviews discussed the effectiveness of audit and feedback in changing physician performance.10,17,20 The outcome measures were based on patient outcomes (eg, blood pressure) and physician performance (eg, prescribing). Two reviews by Jamtvedt et al (118 studies) evaluated multifaceted interventions, including audit and feedback, as more effective (ARR of physician compliance ranged from 0.99 to 1.30) compared with no intervention or audit alone or feedback alone.10,17 In one review, Jamtvedt et al reported that the ARD of audit and feedback versus no intervention varied from –0.16 (a 16% decrease in compliance) to 0.70 (a 70% increase in compliance) in different studies; thus, the results were inconclusive.10 The review by Mugford et al20 showed that feedback alone is most effective when presented close to the time of decision making in clinical practice.

Continuing Medical Education

Copyright AJMC 2006-2019 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up