Maria E. Otto, MD; Carlin Senter, MD; Ralph Gonzales, MD, MSPH; and Nathaniel Gleason, MD
Referral rates in the ambulatory setting doubled from 1999 to 2009,1
with specialty visits now constituting over half of all ambulatory care visits and more than two-thirds of all ambulatory care expenditures in the United States.2
Judicious and coordinated use of this specialty care is increasingly recognized as essential to high-value healthcare.3,4
The “Patient-Centered Medical Home Neighbor” (PCMH-N), a framework conceptualized by the American College of Physicians Council of Subspecialty Societies to address the relationship between primary care offices (PCMHs) and specialty/subspecialty practices, highlights the referral process as fundamental to effective utilization and coordination of specialty care in the ambulatory setting.5
With the advent of electronic health records (EHRs) and integrated referral platforms, we now have the opportunity to provide decision support and improve specialty care coordination at the point of referral. At University of California, San Francisco (UCSF) Health, we developed condition-specific referral guidelines and electronic consultations (eConsults) to support providers at the point of referral to medicine specialty practices6,7
). These programs incorporate PCMH-N principles by facilitating the transfer of clinical information, eliciting a clinical question, conveying recommendations about tests and treatments prior to referral, and defining expectations about management roles.5,8,9
eConsults facilitate timely specialist consultation via the EHR for clinical questions that do not require an in-person evaluation.
The opportunity to embed disease-specific recommendations at the point of referral raises the need for a process to define these recommendations. Practice guidelines are rarely designed to support the referring provider at this point in the patient’s care. One of the most frequent referrals from primary care to specialist care is for a patient with a musculoskeletal problem,1,10,11
and referrals to orthopedic care are an important example of this gap. Professional society practice guidelines exist for many musculoskeletal diseases,12-20
but do not address diagnostic or treatment modalities to attempt prior to referral or clinical questions that might be appropriate for an eConsult with an orthopedist in lieu of a face-to-face visit.
We therefore sought to develop local standards to guide primary care providers (PCPs) when referring patients for orthopedic care. We used the Delphi method, which has been successfully applied to create guidelines in other areas of healthcare21-24
and to build consensus among UCSF primary care and orthopedic clinicians. Specifically, we asked: 1) What tests and treatments should be performed in primary care prior to orthopedic consultation for specific common musculoskeletal problems? and 2) Which common musculoskeletal problems could be managed by the PCP with an eConsult by an orthopedist, in place of a face-to-face patient visit?
The UCSF Health System is a multi-site academic institution with 178 PCPs, 24 orthopedists, and approximately 65,000 primary care patients with 5000 referrals to orthopedics per year. We performed a 2-phase modified Delphi study25
using Web-based surveys to identify consensus between primary care and orthopedic clinicians at UCSF for common musculoskeletal problems.
All UCSF physicians, physician assistants, and nurse practitioners that provide care to adult patients in the UCSF orthopedic clinics were asked to participate at a faculty meeting and via e-mail (n = 24). PCPs in the fields of family medicine and internal medicine volunteered in response to a recruitment e-mail sent to all UCSF adult primary care clinicians (n = 178).
We performed an administrative review of final diagnoses for referrals from UCSF adult primary care to orthopedic care in order to identify the most commonly referred musculoskeletal problems. A small team of primary care and sports medicine physicians designed a survey composed of clinical scenarios for each of these commonly referred musculoskeletal problems. When clinical characteristics such as age, duration of problem, or athletic status were likely to affect management, multiple scenarios were constructed to differ by 1 variable at a time.
Each clinical scenario was followed by the question stem: “The following should occur prior to referral to the orthopedic clinic...” A list of specific questions about pre-referral evaluation, conservative treatment in advance of referral, and the appropriateness of non–face-to-face consultation (ie, eConsult) followed. The panel members rated their agreement with each question using a 5-point Likert scale (1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, and 5 = strongly agree). Evaluation and treatment questions were developed based on clinical practice guidelines, when available.12-20
We included questions about pre-referral x-ray and magnetic resonance imaging (MRI) in all scenarios for consistency throughout the survey. A sample survey question is seen in the eAppendix
(available at www.ajmc.com
). The clinical scenarios and survey questions were vetted with the UCSF orthopedic surgery division chiefs to ensure appropriateness and completeness prior to survey administration.
The study was conducted between November and December 2013. Two rounds of Web-based surveys (Qualtrics, Provo, Utah) were sent to panel members as an e-mail link. Panel members were given 2 weeks to respond to each round, with a reminder e-mail sent at 7 days. The research team took 2 weeks after the first round to collate responses and develop the second round. Consensus was defined as at least 70% of PCPs and at least 70% of orthopedists who strongly agree/agree or strongly disagree/disagree. All answers were weighted equally.
Primary care clinicians received all questions (n = 214) and received a $25 gift card for their participation. Orthopedic clinicians received only questions relevant to their orthopedic subspecialty.
The second round consisted of questions that did not reach consensus in the first round. For these remaining questions, panel members received feedback on each question in the form of a graph depicting the median PCP response and median orthopedist response from the first round. Panel members were then asked to re-answer these questions using the same 5-point Likert scale.
Characteristics of the Delphi Panel
The expert panel comprised 38 clinicians. Of the 178 PCPs invited, 21 volunteered to complete the survey. All 21 volunteers participated in round 1 of the survey and 20 participated in round 2. Of the 24 orthopedic specialists invited, 17 participated in round 1 and 16 participated in round 2 (Table 1
In total, there were 214 questions for 36 clinical scenarios. The expert panel reached consensus in 145 (68%) questions. Of these, a total of 110 (51%) questions reached consensus after round 1, and an additional 35 (16%) questions reached consensus after round 2 (Table 2
). Figure 2
depicts an example in which consensus was not reached in round 1 but was reached in round 2 in response to the prompt, “A patient with chronic shoulder pain consistent with frozen shoulder (limited active AND passive range of motion (ROM). The PCP should order and review the results of an MRI prior to referral.”
Clinicians agreed that “confirming patient interest in an orthopedic procedure” should be completed prior to referral in 81% of clinical scenarios, as well as specific conservative management steps in 80%, physical therapy in 60%, and x-ray prior to referral in 42% of scenarios. Clinicians agreed an MRI should be completed prior to referral in only a handful (14%) of clinical scenarios, including acute knee ligament or meniscal tear, acute and chronic full thickness rotator cuff tear, spinal stenosis, and chronic neck pain with radiating arm pain. Clinicians agreed that prior to referral (negative consensus), an x-ray should not be performed in 27% of clinical scenarios and an MRI should not be performed in 58% of clinical scenarios. The panel agreed that “non–face-to-face electronic consultation (eConsult)” could be appropriate in 39% of clinical scenarios and inappropriate in 1 of 36 (3%) clinical scenarios. The Delphi survey results, organized by clinical scenario, are presented in Table 3
Of the questions in which no consensus was reached (n = 69), 36% lacked consensus within both the PCP and orthopedist groups. For clinical scenarios in which no consensus was reached for x-ray (n = 10), orthopedists agreed that an x-ray should be performed prior to referral in 6 scenarios. For clinical scenarios in which no consensus was reached about “confirming patient interest in an orthopedic procedure” (n = 7), PCPs agreed that confirmation should be performed prior to referral in 5 scenarios. The results of the remaining questions were mixed.
Using the modified Delphi method, we characterized expectations of primary care and orthopedic clinicians at our institution in the management of common musculoskeletal problems at a specific moment in patient care—the point at which the PCP considers referral to a specialist. In the absence of national guidelines delineating which diagnostic or treatment modalities should be completed by the PCP prior to referral, a process for local guideline generation is needed in order to provide nuanced and detailed decision support at the point of referral; the Delphi method proved an effective process to achieve this end. Further, the Delphi method facilitated an assessment of PCP and specialist support for using eConsult—a new model of care—for specific clinical conditions in a field that relies heavily on expertise in physical examination.
In its application at UCSF, consensus was identified for imaging and treatment modalities that should be performed in primary care prior to orthopedic consultation in two-thirds of the items examined in the survey. PCPs and orthopedists agreed that specific conservative management measures and physical therapy should be performed prior to referral for most musculoskeletal problems, an x-ray should be completed prior to referral in less than half of clinical scenarios, and an MRI should be completed prior to referral in only a small number of clinical scenarios. The x-ray and MRI-related findings may help reduce overutilization of expensive tests and reduce unnecessary radiation exposure for patients, as well as identify those clinical cases for which imaging is essential for a successful initial visit with the orthopedist.
Incorporating these findings—along with the specific x-ray views recommended by orthopedists—into guidelines at the point of referral may further reduce unnecessary re-imaging. The near-universal consensus that PCPs should obtain “confirmation that the patient is amenable to orthopedic intervention prior to referral” is striking, and suggests an opportunity to better synchronize expectations. Finally, PCPs and orthopedists at our institution also agreed that “electronic consultation [eConsults]” with an orthopedist may be an appropriate next step for many of the commonly referred musculoskeletal problems.
The Delphi method is not without limitations. In addition to being time-consuming, study designers may impose preconceptions through the Delphi questions that prevent other perspectives from surfacing. Consensus may represent a middle-of-the-road compromise, eliminating extreme positions and potentially obscuring the best judgment26
—where consensus is not reached, no guideline results. Furthermore, panel members may have a conflict of interest or not be generally representative. In our study, we sought to ensure the representativeness of the panel by inviting all PCPs and orthopedists at UCSF to participate; among the PCP panel, a broad range of experience is represented, although internists were disproportionately represented over family practice providers. The small number of orthopedists available to participate in the survey for some of the joint-specific questions, due to the sub-specialized nature of orthopedic care, is also a limitation.
The specific findings in our health system are local and not necessarily generalizable. Reproduction of the process by individual institutions may have a greater impact than implementation of referral guidelines based on these specific clinical findings. Local implementation of the process has the advantage of engagement of local clinicians in the care delivery improvement effort, as well.
The referral process is fundamental to the thoughtful and appropriate use of specialty care in an integrated high-value healthcare system. With the advent of the EHR, we have the opportunity to provide decision support at the moment of referral. Existing guidelines do not typically address this point in the care of a patient and are difficult to create at a national level due to variation in practice patterns. The Delphi method proved to be a feasible and robust way to identify local guidelines for referral appropriateness.