Currently Viewing:
The American Journal of Accountable Care December 2015
The Need to Level the Playing Field Between Accountable Care Organizations and Medicare Advantage
David Introcaso, PhD
The Debate Over Drug Costs: Instead of "How Much" We Spend, Let's Focus on What We Get in Terms of Health
A. Mark Fendrick, MD Co-Editor-in-Chief, The American Journal of Managed Care Professor of Medicine, School of Medicine Professor of Health Management and Policy, School of Public Health Director, Cen
The Pediatric ACO: A New Frontier in Accountable Care
Anthony D. Slonim, MD, DrPH
Revisiting the Role of Academic Medical Centers in Medicare Shared Savings Program ACOs
Benjamin M. Gerber, JD
The Emerging Business Models and Value Proposition of Mobile Health Clinics
Khin-Kyemon Aung AB; Caterina Hill, MSC, MA; Jennifer Bennet, BS; Zirui Song, MD, PhD; and Nancy E. Oriol, MD
Now That ACOs Are Engaged, What About Those Patients?
Mary K. Caffrey
It Is Time to Ask Patients What Outcomes Are Important to Them
Leif I. Solberg, MD; Stephen E. Asche, MA; John C. Butler, MD; David Carrell, PhD; Christine K. Norton, MA; Jeffrey G. Jarvik, MD, MPH; Rebecca Smith-Bindman, MD; Juliana O. Tillema, MPA; Robin R. Whi
Continuous Quality Improvement Program, Based on Lean Concepts, Allows Emptying of Emergency Department Corridors
Enrique Casalino, MD, PhD; Christophe Choquet, MD; Mathias Wargon, MD, PhD; Romain Hellmann, MD; Michel Ranaivoson, MD; Luisa Colosi, MD; Gaƫlle Juillien, MD; and Julien Bernard, MD
Bundled Payments for Diabetes Care and Healthcare Costs Growth: A 2-Year Follow-up Study
Sigrid Mohnen, PhD; Caroline Baan, PhD; and Jeroen Struijs, PhD
Currently Reading
Using Guidelines: When Is It Appropriate?
Maude St-Onge, MD, PHD, FRCPC

Using Guidelines: When Is It Appropriate?

Maude St-Onge, MD, PHD, FRCPC
This article aims to review guideline development methodology to inform users on key aspects to consider when judging if a practice should be adapted or changed.

Guidelines are concise, usable messages that are intended to influence behaviors. This article aims to review guideline development methodology to inform guide­line users, decision makers, and patients regarding key aspects to consider when both appraising the quality of this important knowledge tool and when judging if a practice should be adapted or changed based on the recommendations. Given the potential guideline benefits and harms, it is of primary importance to ensure a rigorous development process. Moreover, guideline users, decision makers, and patients should be able to evaluate the following: if the guideline is needed; if the scope, analytical framework, and key questions are relevant to their clinical reality; if the recommendations are applicable; and if the process of recommen­dations development is reliable and valid. Appraising guidelines will allow those key stakeholders to judge when a recommendation is applicable or not to their specific circumstances in order to avoid an inappropriate use.
Graham et al1 developed the knowledge-to-action pro­cess to represent how knowledge is created, dissemi­nated, and implemented to improve the quality of care. Knowledge creation comprises knowledge inquiry (eg, primary studies), knowledge synthesis, and knowledge tools—such as guidelines2—which are concise and usable messages that are in­tended to influence behaviors.

Guidelines are defined as, “systematically developed state­ments to assist practitioner and patient decisions about appro­priate healthcare for specific clinical circumstances.”3 Guidelines are developed with the intent to improve the quality of care and outcomes, reduce inappropriate variation in practice, summarize evidence, favor an efficient resource use, make clinical decisions more transparent, identify areas of knowledge for which research should be prioritized, empower patients, inform public policy, and support quality control activities.2

This article aims to review guideline development methodol­ogy to inform users, decision makers, and patients regarding key aspects to consider both when appraising the quality of this im­portant knowledge tool and when judging if a practice should be adapted or changed based on the recommendations. After outlin­ing the potential benefits and risks of guidelines, aspects related to appropriate guideline development and use will be detailed.


Potential Benefits and Harms of Guidelines

Woolf et al4 identified the potential benefits of guidelines for patients, healthcare professionals, medical researchers, and healthcare systems. For patients, one important benefit is the promotion of therapies that have the potential for better out­comes; clear guidelines can also improve consistency of care. For healthcare professionals, guidelines clarify which interventions are better supported by the evidence, help to explain the risks and benefits to the healthcare consumer, and can constitute a point of reference for quality improvement activities. For medi­cal researchers, guidelines can emphasize gaps in knowledge. Ad­ditionally, healthcare systems may also benefit from guidelines that highlight cost-effective interventions.

In spite of the numerous benefits, if evidence is misleading or misinterpreted, or if the patients’ needs are not prioritized, guidelines have the potential to cause harm as well.4 First, pa­tients can be negatively impacted if guidelines do not allow cli­nicians to tailor care to specific circumstances. Guidelines are also influenced by the opinions and experiences of development workgroup members, which can have an impact on the priori­ties being considered. For example, members may be distracted from focusing on patients’ needs if they prefer to build their recommendations based on costs. Second, healthcare profession­als can be affected if the recommendations are wrong, outdat­ed, or contradict other recommendations, which might result in the unfair judgment of clinicians. Third, if further research is inappropriately discouraged, this may negatively impact medical researchers. Finally, the healthcare system may also be compro­mised if guideline implementation generates a waste of resources by recommending unnecessary interventions.

Appropriate Guideline Development

Recognizing the potential for harm stresses the importance of following a rigorous, transparent, and credible guideline develop­ment process. In 2000, Graham et al5 compared different practice guideline appraisal instruments and identified 15, containing 8 to 14 questions/statements each. The authors grouped the state­ments into 10 important attributes to assess the methodological rigor for guideline development: validity, reliability/reproducibil­ity, clinical applicability, clinical flexibility, multidisciplinary pro­cess, clarity, schedule review, dissemination, implementation, and evaluation. Each instrument was independently examined. Only 2 were validated: the Cluzeau instrument6 received the best eval­uation, and constituted the basis of the Appraisal of Guidelines for Research and Evaluation in Europe (AGREE II) instrument,7 and the Shaneyfelt instrument8 was the other tool validated. More recently, Schünemann et al9 systematically compiled a com­prehensive checklist of items that guideline development tools could consider. This checklist included 18 topics to serve as a resource for guideline developers.

Appropriate Guideline Use

References targeting guideline workgroups provide an exhaustive list of what must be considered in the development process.5-9 However, from the perspective of guideline users, decision mak­ers, and patients, a thoughtful appraisal of guidelines should be undertaken in order to apply this knowledge tool in appropriate circumstances. We propose asking the following questions: 1) Is there a need for a guideline? 2) Are the scope, analytical frame­work, and key questions relevant to my clinical reality? 3) Are the recommendations applicable in my clinical reality? 4) Is the process of recommendations development reliable and valid?

Is There a Need for a Guideline?

In order to evaluate if there is a need for a guideline, decision makers and patients should answer the following questions: What are the importance and burden of the disease? Is there a care gap based on evidence, and can outcomes be improved? Is there practice variation or clinical uncertainty?

In 2012, Atkins et al10 conducted a literature review to iden­tify the criteria for whether a topic should be considered for guideline development. While they found only a few studies that directly addressed this issue, they did identify a substantial con­sensus regarding the general factors that should be considered. The Institute of Medicine11 outlined 6 criteria: prevalence, illness burden, potential to improve outcomes (care gap), management costs, cost reduction, and practice variation.

To document the prevalence and burden of a given illness, Atkins et al10 recommended the use of national data or review articles. The authors also suggested ad hoc stakeholders’ con­sultations to evaluate practitioner interest. To document the availability of evidence, Atkins et al10 recommended that existing reviews be reviewed and that preliminary literature searches be performed. Nonetheless, the question remains as to whether ob­servational studies, case series, animal studies, and case reports are sufficient to develop a clinical practice guideline. Random­ized controlled trials are not always possible. For example, in the EXTRIP (EXtracorporeal TReatments In Poisoning) guideline methodology,12 the authors mention: “The guidelines are intend­ed to rely on evidence whenever possible. However, (...) extracor­poreal therapies have been recognized as the gold-standard treat­ment of several specific poisonings (...) elements of standard care in the management of poisonings and clinicians support its use overwhelmingly. In such instances a randomized trial would not be feasible or ethical.”

When the evidence is low or very low, the transparency of the consensus process is of primary importance, and the benefits of making recommendations (ie, decreased practice variation, facil­itation of future research, decreased costs) should outweigh the risk of harm (ie, recommending a harmful intervention).


Are the Scope, Analytical Framework, and Key Questions Relevant to My Clinical Reality?

After ensuring that there is a need for a guideline, decision mak­ers and patients should evaluate if the scope, analytical frame­work, and key questions are relevant to their clinical reality. They should answer the following questions: Is it the appropriate pop­ulation of patients and providers? Are all relevant options taken into account? Are all relevant key questions considered? Are pa­tient-centered health outcomes considered?

After ensuring that a guideline has been developed to answer a specific need (eg, to decrease burden of illness, decrease care gap, decrease practice variation), guideline users, decision mak­ers, and patients should evaluate the circumstances in which it applies. The scope of a guideline describes the populations of patients and providers, the type of services, and the sites for which the recommendations are intended.13 Some authors have raised concerns that guidelines may not properly consider the care of patients with comorbidities.14,15 Therefore, investigators from the Improving Guidelines for Multimorbid Patients Study Group established consensus-based recommendations for each step of guideline development to address this issue. Guideline developers should consider how disease–disease, disease–treat­ment, and treatment–treatment interactions impact clinical man­agement and outcomes.

Are the Recommendations Applicable to My Clinical Reality?

After ensuring that a necessary guideline targets a scope and key questions relevant to their clinical reality, guideline users, deci­sion makers and patients should evaluate if the recommenda­tions are applicable, and answer the following questions: Are the recommendations based on evidence relevant to my reality? Do the recommendations take into account the resources I have? Do the recommendations consider the values and preferences of my patients and their relatives?

First, guideline users, decision makers, and patients should as­sess if they agree with the proposed evaluation of quantity and quality of evidence. The AGREE collaboration7 suggests that guideline workgroups use systematic methods to search for evi­dence related to each key question of the analytical framework, clearly describe the criteria for selecting the evidence, and pro­vide a detailed description of the strengths and limitations of the body of evidence. If a systematic review has already been pub­lished, the guideline developers should appraise its quality based on the Assessment of Multiple Systematic Reviews (AMSTAR).16 In circumstances where there are no available systematic reviews, guideline developers should perform a new synthesis of the ev­idence.

When a new systematic review is required, the American Heart Association Clinical Practice Guideline Methodology Group17 recommends incorporating the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) method­ology to evaluate the quality of evidence for each outcome across studies.18 The quality can be evaluated as “high” when there is confidence that the true effect lies close to that of the estimate effect (as often occurs for randomized trials), “moderate” when the true effect is likely to be close to the estimate of the effect but there is a possibility that it is substantially different, “low” when the true effect may be substantially different (as is often the case for observational studies), and “very low” when there is little confidence in the effect estimate.19

Copyright AJMC 2006-2019 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up