Jeanie Lo, MPH

 To analyze inappropriate use of magnetic resonance imaging (MRI) for patients with low back pain in a healthcare system with no financial incentives for overuse.

 We used administrative data to assess the appropriateness of lumbar spine (LS) MRI in the Veterans Health Administration.

All veterans who received LS MRI in the outpatient setting in fiscal year 2012 were included. We based our assessments of appropriateness on CMS criteria, which have been endorsed by the National Quality Forum. Generalized estimating equations were used to evaluate characteristics of inappropriate scans.

Of the 110,661 LS MRIs performed, 31% were classified as inappropriate. Most scans that were considered appropriate were characterized as such because they were preceded by conservative therapy (53%). "Red flag" conditions were responsible for a much smaller percentage of scans being considered appropriate; 13% of scans were preceded by conservative therapy and were performed in patients with a red flag condition, while only 4% of scans were considered appropriate because of red flag conditions only. Scans ordered in the emergency department and in urgent care, primary care, and internal medicine clinics were most likely to be classified as inappropriate. Resident physicians were significantly less likely than other provider types to order inappropriate LS MRIs (odds ratio, 0.80; 

 <.0001). Approximately 24% of providers ordered 74% of inappropriate scans.

We found that 31% of LS MRIs were inappropriate in a healthcare system largely absent of financial and other incentives for ordering. The problem of inappropriate ordering of LS MRI is concentrated in a small number of providers; any provider-facing interventions to reduce inappropriate order should therefore be targeted, rather than aimed at all providers who order LS MRI.

	Using CMS criteria, we found that 31% of lumbar spine magnetic resonance imaging (LS MRI) performed in fiscal year 2012 was inappropriate. This high level of inappropriate ordering is particularly striking in a system largely free of financial and other incentives for ordering, suggesting that nonpecuniary factors play a role in these inappropriate ordering choices. We found a high concentration of inappropriate ordering among few providers. Results indicate that efforts to reduce inappropriate use of LS MRI should involve cultural change, as well as patient and provider education, rather than financial incentives alone, and that provider-facing efforts should be targeted instead of universal.

Most Americans will experience low back pain (LBP) at some point in their adult lives.

 Many patients with LBP receive imaging to explore the cause of their pain

; advances in imaging technology have revealed unprecedented anatomical detail for LBP patients. The underlying difficulty is that, for many patients, anatomical and functional abnormalities do not correlate well, and thus imaging can fail to guide treatment of the primary complaint.

 In one study of individuals who underwent magnetic resonance imaging (MRI), approximately 90% were detected to have a degenerated or bulging disc, 36% had a herniated disc, and 21% had spinal stenosis—but all patients were asymptomatic.

 For most patients, LBP resolves on its own

; even sophisticated imaging of the lumbar spine (LS) for nonchronic pain does not improve outcomes.

Overuse of LS MRI is suspected to be a large problem; multiple societies have prioritized it as a quality-of-care measure or a measure of prudent stewardship of resources. The American College of Physicians and the American Association of Neurological Surgeons recommend against the use of LS MRI for nonspecific or nonpersistent LBP in their Choosing Wisely campaign.

 The National Quality Forum (NQF), a nonprofit entity that sets national priorities for quality measurement, has endorsed a CMS measure regarding inappropriate use of imaging in LBP.

The reasons for overuse of imaging are not well understood. Fee-for-service financial incentives have been suggested as a potential reason for overuse of lumbar spine imaging,

 In this study, we explored inappropriate use of LS MRI in a US healthcare system—one largely free of financial incentives for overuse, and in which physicians are largely insulated from malpractice concerns&mdash;to understand whether inappropriate ordering occurs in the absence of these financial or legal influences. The federal government accepts liability for any instances of medical negligence or wrongdoing on the part of the Department of Veterans Affairs (VA) providers; concerns over malpractice or incentives to practice defensive medicine are therefore much lower for VA providers compared with providers in the larger US healthcare environment. It is also not well understood whether inappropriate ordering is equally distributed across providers, or concentrated among relatively few high-ordering providers; this information carries important implications for quality improvement activities. Here, we use NQF-endorsed CMS criteria to evaluate inappropriate ordering of LS MRI across the entire VA, including the patient-, provider-, and system-level factors associated with the practice.

To study the appropriateness of this imaging modality, we identified the population of veterans who received an outpatient LS MRI in VA in fiscal year (FY) 2012. If patients received more than 1 LS MRI, we retained information regarding the first MRI. Additionally, we searched VA inpatient and outpatient data from FY 2010 to FY 2012 to evaluate the care and condition of these veterans prior to receipt of LS MRI. CMS criteria pertain to LS MRIs with a diagnosis of LBP on the MRI claim; because VA MRIs do not have associated diagnosis codes, we evaluated all LS MRIs. We considered the clinic visit with the ordering provider that immediately preceded the scan to be the ordering visit.

We evaluated the appropriateness of LS MRI according to the NQF-endorsed CMS measure #0514 (“MRI Lumbar Spine for Low Back Pain”),

). This measure classifies an LS MRI as appropriate if it was: 1) preceded by conservative therapy, or 2) conducted in a patient who has a “red flag” condition. CMS considers physical therapy or chiropractic care in the 60 days prior to the scan, or an evaluation and management visit occurring between 28 to 60 days prior to the scan, to be conservative therapy. Physical therapy, chiropractic care, and evaluation and management visits were identified by Current Procedural Terminology codes specified by the NQF-endorsed CMS measure.

 We classified an evaluation and management (E&M) visit as conservative therapy only if the scan was not ordered during that visit.

We made minor modifications to the CMS criteria regarding red flag conditions. The CMS measure excludes LS MRIs occurring in patients with red flag conditions from the denominator; here, we included them but considered them to be appropriate in order to present a complete view of all LS MRIs in VA. CMS red flag conditions include trauma in the 45 days preceding the scan; LS surgery in the 90 days preceding the scan; or intravenous drug abuse, neurologic impairment, cancer, HIV diagnosis, or diagnosis of unspecified immune deficiency in the 365 days preceding the scan. The measure includes all cancer; we excluded primary skin and primary prostate cancer as these types of neoplasms would not cause lumbar compression and rarely have an impact on spine pain. Additionally, there are concerns regarding a high false-positive rate for prostate cancer due to overuse of prostate-specific antigen (PSA) screening. To the CMS red flag conditions, we added intraspinal abscess or compression fracture in the 30 days prior to the scan, and osteomyelitis in the previous 365 days. All red flag conditions were identified through

 International Classification of Diseases, Ninth Revision, Clinical Modification 

) codes in inpatient and outpatient data sets.

From these data, we identified which LS MRI scans were considered inappropriate. LS MRIs were considered inappropriate in patients who did not have a red flag condition in the time frame of interest and those who did not receive appropriate conservative therapy.

We also investigated patient-, provider- and facility-level characteristics associated with inappropriate scans, as well as any regional differences in proportion of inappropriate scans. Patient-level characteristics included age, gender, and race. Provider-level variables included ordering provider type (attending physician, resident, physician’s assistant, and advanced practice nurse/nurse practitioner) and the clinic in which the scan was ordered (internal medicine/primary care, physical medicine and rehabilitation; neurology, neurosurgery, orthopedics, pain medicine, “other” medicine services, miscellaneous other medicine services, chiropractic care, and emergency department [ED] and urgent care). Other medicine services refer to various primary and specialty care clinics separate from the other categories of care (eg, geriatrics, oncology, hospice, dementia clinic).  Miscellaneous other services refer to all ancillary and support services (eg, cast clinic, social work, telephone triage).  The facility-level variable included complexity, defined in 5 levels (1a, 1b, 1c, 2, and 3; from most to least complex).

Characteristics of inappropriate scans were evaluated using generalized estimating equations to account for clustering of patients within facilities. Our generalized estimating equations employed a logit link, a binomial distribution family, and an independent correlation structure. We ran sensitivity analyses in which we evaluated facility-level physical therapy penetration—defined as physical therapy visits per 1000 pro-rated patients&mdash;and in which, we evaluated alternative definitions of conservative therapy. Specifically, we required the conservative therapy visits to have an accompanying diagnosis of low back pain. Post estimation commands were used to compare outcomes among different levels of each categorical level; for example, to compare residents with each other provider type, physician’s assistants to each other provider type, etc. All analyses were conducted in SAS version 9.2 (SAS Institute, Inc, Cary, North Carolina). 

In FY 2012, 110,661 LS MRIs were ordered in the outpatient setting by 16,273 unique providers. Most patients received 1 LS MRI; only 3.7% of patients received more than 1. Each provider ordered a median of 3 LS MRIs and a mean of 6.8 LS MRIs. Of the 110,661 total scans, 33,998 (30.7%) were classified as inappropriate; these patients had no red flag conditions coded and did not receive conservative therapy in a VA facility prior to their scan (

). The majority of scans classified as appropriate were done so because the patient was deemed to have received prior conservative therapy (65.3%). Another 16.6% of LS MRIs were classified as appropriate because the patient had a red flag condition. (Note that the numbers add up to more than 100% because patients could have had both a red flag condition and have received conservative therapy.)

describes the demographics of the population, which on average, was male, white, and middle-aged. The greatest proportion of scans (42%) was ordered for patients seen at the most-complex (level 1a) facilities. However, the greatest proportion of inappropriate scans (36%) was ordered for patients seen at the least-complex facilities (level 3) (Table 2). The number of scans ordered varied greatly by clinic setting, with the greatest percentage of scans ordered by the primary care/internal medicine clinic (62%), and the fewest percentage ordered by neurosurgery and orthopedics (6% combined). The greatest percentage of total scans (69%) was ordered by medical doctors or doctors of osteopathy, and the lowest was ordered by physicians’ assistants (8%). We did not find meaningful regional variation in proportion to scans considered inappropriate.

Results from generalized estimating equations revealed that patients aged under 35 years were more likely to have an inappropriate scan than those of all other ages (

). Black patients were less likely to have an inappropriate scan compared with white patients (odds ratio [OR], 0.80; 

 <.0001). Scans ordered at the lowest-complexity facility (level 3) were more likely to be inappropriate compared with those ordered at the highest-complexity facilities (OR, 1.27; 

 = .0096). These multivariate results also show the ED or urgent care clinic to be the settings most strongly associated with an inappropriate scan (OR, 1.42; 

 <.0001, with reference category of primary care/internal medicine clinic). In fact, ED/urgent care providers were more likely than providers practicing in any other clinic to order inappropriate scans (Table 3). Primary care/internal medicine clinics were also associated with a high likelihood of ordering inappropriate scans; all other clinic settings were less likely to order inappropriate scans than the primary care/internal medicine clinic (Table 3). Neurology, neurosurgery, and orthopedic clinics all practiced similarly; after adjusting for covariates, each was equally likely to order an inappropriate scan compared with the others (Table 3). Medical doctors (MDs), doctors of osteopathy (DOs), nurses, and physician’s assistants were also equally likely to order an inappropriate scan; however, residents were less likely to order an inappropriate LS MRI scan compared with MD/DOs (OR, 0.80; 

Results also show that 10.9% of providers ordered 50.1% of inappropriate scans, and 23.6% of providers ordered 74% of inappropriate scans (

). There appeared to be a weak, volume-percent inappropriate relationship; that is, there was little evidence of a relationship between the volume of scans ordered by a provider and the percentage of those scans deemed inappropriate.

We ran sensitivity analyses in which we modified the categorization of E&M codes. CMS considers any E&M visit between 28 and 60 days prior to the MRI to be conservative therapy, as long as the MRI was not ordered during that visit. Using this definition in our base-case, 63% of scans were conducted in patients who had a qualifying E&M visit. In sensitivity analyses, we restricted the E&M visit to have a diagnosis of LBP. Using this more restrictive definition, we found only 34% of LS MRI scans had a qualifying E&M visit, and thus, 53% of LS MRIs were considered inappropriate.

We also ran other sensitivity analyses. We tested a model with a covariate of facility-level physical therapy penetration. This variable was not a significant predictor of inappropriate LS MRI. Additionally, we ran a model on the subset of scans ordered by providers who ordered 5 or more scans per year, to avoid drawing conclusions based on a large number of providers each ordering few scans. Results did not change with respect to direction or significance. Third, we tested the inclusion of all prostate cancer in the red flag conditions, thus considering all LS MRI imaging in patients with a prostate cancer diagnosis in the previous 365 days to be appropriate. This resulted in slightly fewer (28.9%) LS MRIs being considered inappropriate.

We estimated the cost of providing each LS MRI in VA using the Decision Support System, the activity-based cost allocation system used by VA. We estimated that VA spent $44.4 million on LS MRIs in FY 2012, including $13.6 million on inappropriate scans.

The extent of overuse of LS MRI has been the subject of much debate. Here, we find that, despite a strong body of evidence regarding the poor utility of LS MRI in nonchronic LBP, and the identification of LS MRI as a low-value service by multiple specialty societies,

 31% of LS MRIs conducted in VA in FY 2012 were inappropriate.

Estimates of inappropriateness of LBP imaging vary in the literature. The 2000 to 2006 claims-based analysis of Pham and colleagues found that 29% of Medicare beneficiaries received x-ray, computed tomography (CT), or MRI inappropriately.

 A 2004 to 2006 claims-based analysis of privately insured patients found 9% of patients with LBP had an MRI within 14 days of initial diagnosis, and approximately 60% of patients had MRI, CT, or x-ray on the same day as the first diagnosis of LBP.

 These variations in the ordering of imaging could be due to increasing supply of MRI scanners from 2000 to 2012,

 different ratios of acute and chronic LBP patients in study samples, and varying definitions of inappropriate.

We found inappropriate use of MRI in VA, even though its providers have little to no financial incentive to order more scans and have less malpractice liability than physicians in private practice—2 factors often cited as reasons for overuse of imaging.

 This suggests that nonpecuniary factors play an important role in ordering behavior in VA. Thus, it also suggests that changes in financial incentives alone will not be sufficient to reduce inappropriate ordering of LS MRI, and that provider and patient education may be necessary. Further research to explore the reasons for inappropriate ordering in this healthcare system, including the relative importance of lack of awareness of guidelines and concerns about patient satisfaction or reassurance

 will be informative for quality improvement. Reducing inappropriate ordering of LS MRI also may be a question of educating patients, as well as providers. One randomized controlled trial found that patients receiving MRI were more satisfied than those receiving x-ray, independent of functional outcome.

 This tactic may be increasingly important as organizations move to include patient satisfaction metrics in their measures of quality of care.

This is, to the best or our knowledge, the first analysis of NQF-endorsed CMS criteria to evaluate inappropriate ordering of LS MRI. Our analysis indicated that providers in primary care and internal medicine clinics, EDs, and urgent care clinics were most likely to order inappropriate LS MRIs. MDs, DOs, nurses, and physician’s assistants were equally likely to order inappropriately. Of note, we found that residents were significantly less likely than all other provider types to order inappropriately. Although resident behavior has been relatively understudied in the literature, Pitts and colleagues found that ED visits with residents were more likely to have resulted in advanced imaging orders compared with ED visits without residents.

 Our dissimilar results may be due to questions of appropriateness; Pitts and colleagues did not evaluate the appropriateness of the imaging. We also found a weak relationship between the volume of scans ordered by a provider and the percentage of those scans considered inappropriate; high-ordering providers were not necessarily inappropriately ordering providers. Rather, we found that inappropriate LS MRIs were concentrated among relatively few providers. Approximately a quarter of providers (24%) ordered roughly three-fourths of all inappropriate scans (74%). Most providers (58%) ordered 3 or fewer scans per year, accounting for 60% of all LS MRIs ordered in this year. This suggests that the majority of providers do not need to be involved in interventions to reduce inappropriate use.

Our results indicate that interventions to reduce inappropriate ordering will be most appropriately directed at providers who order 18 or more scans per year, at providers and managers at the lowest-complexity facilities, and at providers in primary care/internal medicine clinics. Interventions aimed at providers who order 18 or more scans per year represent an opportunity to have a substantive impact on reducing the absolute number of inappropriate MRIs ordered. Although ED and urgent care services have the greatest percentage of inappropriate scans (41% in univariate analysis), intervening in these clinics will have modest effects, as they account for only 4% of all scans. Primary care/internal medicine clinics accounted for 62% of all scans. Scans ordered in these settings were significantly more likely to be inappropriate, making them likely targets for intervention.

It is possible that scans are being ordered in these primary care/internal medicine clinics to meet the scheduling requirements of specialists; specialty clinics may require an LS MRI before seeing a patient. We found that orthopedics and neurosurgery, together, ordered only 6% of all LS MRIs in VA, suggesting this may be a factor. Further research is needed to determine if scans attributed to primary care/internal medicine providers were ordered to secure referral to specialty clinics; this research should inform any interventions to reduce inappropriate ordering.

Our sensitivity analyses showed that changes in the definition of evaluation and management—which are a major component of what constitutes conservative therapy&mdash;have a large influence on estimates of appropriateness. Using a more restrictive definition of evaluation and management reduced the number of scans with conservative therapy and almost halved the percentage of scans regarded as appropriate. We used the most generous definition of evaluation and management in our base-case analyses. Our choice of a generous definition was two-fold: We wished to be consistent with the definition developed by CMS, and we prioritized identifying sensitivity (true appropriateness) over specificity (true inappropriateness). Maximizing sensitivity represents a cautious approach that avoids erroneously categorizing scans as inappropriate.

Use of imaging that provides no benefit to patients, and may in fact cause harm through catalyzing medical interventions for common and asymptomatic abnormalities, represents waste within the healthcare system. With healthcare expenditures of $2.6 trillion in 2010 and a consistently positive rate of growth in healthcare costs,

 waste is something the US healthcare system and US economy can ill afford. Our analysis indicates an excess annual cost of $13.7 million for LS MRI in VA. Outside of VA, it has been estimated that if all US adults were treated in accordance with imaging guidelines for LBP, the healthcare system would save at least $300 million annually (in 2009 dollars),

 with even greater cost savings achieved if accounting for costs associated with treating asymptomatic disease.

This study was a retrospective analysis of administrative VA data and is subject to certain limitations. We used administrative data to assess the appropriateness of LS MRI. Administrative data do not contain the complete details of the content of care; there may be data captured through history and physical (eg, regarding duration of treatment) that was not assigned an 

code. For example, it is possible that patients received some conservative therapy in a non-VA setting prior to their VA MRI, or that physicians had some rationale for ordering the MRI that was not captured in the coding of care such as progressively worsening symptoms; in this case, our analysis would overestimate the proportion of scans that were inappropriate. However, we employed CMS-generated and NQF-endorsed administrative data-based algorithms that were generous in their categorization of appropriate scans in order to mitigate the likelihood of this. Additionally, MRIs conducted at VA do not contain diagnosis codes; our analysis thus evaluated all LS MRIs. It is possible that we included LS MRIs that were conducted for reasons other than low back pain. This may overestimate the proportion of LS MRIs considered inappropriate. However, the appropriateness of LS MRIs is a function of both the reason for the LS MRI (eg, low back pain), as well as the reasons for which the LS MRI may be warranted (ie, prior failed conservative therapy or a patient with a red flag condition). As we used the least restrictive definition of conservative therapy (that which did not require the conservative therapy visit to have a diagnosis of low back pain) to evaluate care prior to the scan within VA, it is more likely that our analysis actually underestimates the proportion of scans that were inappropriate.

VA is a unique healthcare setting; our results may differ from those of studies of other US providers. While our analysis has limited generalizability outside of VA, analyzing this population provides important insight, by illuminating that even in a healthcare system largely absent of financial and other incentives for overuse, inappropriate ordering of LS MRI is still a substantial problem. Furthermore, our results may be of particular interest to accountable care organizations, whose financial incentives and organizational structure more closely resemble those of VA.

Despite recommendations from specialty societies that LS imaging should be performed only in patients who have severe or progressive neurological deficits or signs and symptoms of serious underlying conditions, we found that 31% of LS MRIs performed in a US healthcare system largely absent financial incentives for overuse—were inappropriate. This work reveals 2 important insights that will be informative for quality improvement purposes. First, our work suggests that nonpecuniary factors have a role to play in inappropriate ordering of imaging for low back pain; efforts to reduce inappropriate ordering cannot rely on financial incentives alone. Second, we found a high concentration of inappropriate ordering among few providers, indicating that provider-focused efforts to reduce inappropriate orders should be targeted rather than universal. Further work should be conducted to assess the proportion of inappropriate LS MRI in environments where defensive medicine or fee-for-service incentives may be at play. 

 VA Health Economics Resource Center (RG, PS, JL, PGB) and VA Center for Innovation to Implementation (RG, TA, SMA), VA Palo Alto Health Care System, Menlo Park, CA; Division of General Medical Disciplines (SMA) and Department of Health Research and Policy (PGB), Stanford University School of Medicine, Stanford, CA.

 This work was supported by VA QUERI Grant: RRP 12-442 Decision Support to Decrease Overuse of Imaging: Usability and Measurement; the funding source played no role in study design or interpretation of results. Dr Gidwani had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

 The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (RG, PS, SMA, PGB); acquisition of data (RG, JL, PGB); analysis and interpretation of data (RG, PS, TA, JL, SMA, PGB); drafting of the manuscript (RG, PGB); critical revision of the manuscript for important intellectual content (RG, PS, TA, SMA, PGB); statistical analysis (RG, JL, PGB); obtaining funding (SMA, PGB); administrative, technical, or logistic support (TA); and supervision (RG, SMA, PGB).

 Risha Gidwani, DrPH, VA Health Economics Resource Center, VA Center for Innovation to Implementation, 795 Willow Rd, 152 MPD, Menlo Park, CA 94025. E-mail: Risha.Gidwani@va.gov.

1. Deyo RA, Mirza SK, Martin BI. Back pain prevalence and visit rates: estimates from U.S. national surveys, 2002. 

2. Di Iorio D, Henley E, Doughty A. A survey of primary care physician practice patterns and adherence to acute low back problem guidelines. 

Articles

This article analyzes use of lumbar spine magnetic resonance imaging in a national sample of patients with low back pain.

Inappropriate Ordering of Lumbar Spine Magnetic Resonance Imaging: Are Providers Choosing Wisely?

In 2010, the Veterans Health Administration (VA) began national implementation of its patient-centered medical home (PCMH) model, called Patient Aligned Care Teams (PACTs), to improve access, coordination, and patient-centered care. We evaluated changes in reported implementation of PCMH components in all VA primary care clinics, and patients’ utilization of acute and non-acute care and total costs after 2 years.

 Longitudinal study of 2,607,902 patients from 796 VA primary care clinics.

Clinics were surveyed for their implementation of PCMH components. Patient outcomes were measured by outpatient visits for primary care, specialty care, telephone care, and emergency department (ED) care; hospitalizations for an ambulatory care—sensitive condition (ACSC); and costs of VA care in fiscal years (FYs) 2009 and 2011. Multi-level, multivariable models predicted changes in utilization and costs, adjusting for patients’ health status, clinic PCMH component scores, and a patient fixed effect.

 Clinics reported large improvements in adoption of all PCMH components from FY 2009 to FY 2011. Higher organization of practice scores was associated with fewer primary care visits (

= .012). Greater care coordination/transitions was modestly associated with more specialty care visits (

= .018), but quality/performance improvement was associated with more ED visits (

= .032). None of the PCMH components were significantly related to telephone visits, ACSC hospitalizations, or total healthcare costs.

 Improvements under organization of practice and care coordination/transitions appear to have impacted outpatient care, but reductions in acute care were largely absent.

Take-up of medical home components appeared to be high after Patient Aligned Care Teams (PACTs) began in the Veterans Health Administration. However, while improvements under organization of practice and care coordination/transitions seemed to have impacted outpatient care, reductions in acute care were largely absent.

		Identifying areas that would benefit from ongoing improvement is critical for the long-term success of PACTs.

		Better measures of PACTs implementation are also needed to evaluate the longer-term consequences of PACTs.

The spread of the patient-centered medical home (PCMH) model to numerous healthcare systems can be attributed to the promise of revitalized primary care delivery through team-based care aiming to improve continuity, coordination, quality, and access, as well as to reduce costs. Based on these principles, the Veterans Health Administration (VA) began a vast initiative in 2010 to implement the PCMH model, called Patient Aligned Care Teams (PACTs), across the entire VA healthcare system, which included more than 800 primary care clinics caring for more than 5 million patients.

Several healthcare systems have implemented pilot programs of the PCMH and found significant improvements in quality of care, reductions in acute care, and cost savings.

 The PACT model is expected to experience similar results since it focuses on improving access to primary care, increasing telephone and other non—face-to-face care, as well as reducing unnecessary emergency department (ED) visits and hospitalizations. Lower costs are expected, in part, from a reduction in (often very costly) acute care episodes.

An early PACT evaluation found that after implementation, there were significant changes: greater use of telephone care, fewer face-to-face visits, improved appointment access and continuity, and better follow-up after hospital discharge.

However, PACT models encompassed a multiplicity of PCMH features that ranged from routinely scheduling same-day appointments to tracking specialty consultations, and from coordinating post discharge care to the use of electronic decision support systems. Baseline adoption of these features was highly variable across clinics

—and clinics may have implemented new PACT features to varying degrees&mdash;so it is unknown how differences in early implementation influenced utilization and costs.

This study evaluated the changes in adoption of different PCMH components by VA clinics, and the relationship to patients’ utilization of acute and non-acute care and total costs after 2 years. We focused on a cohort of primary care patients who used VA care from 2009 to 2011, and were therefore most likely to experience the effects of PACT. This examination can guide ongoing PACT implementation, suggesting what PCMH components and aspects of care might be emphasized most productively in later stages.

We conducted a longitudinal study of a panel of primary care patients using data from before PACT implementation (fiscal year [FY] 2009) and shortly after PACT implementation began (FY 2011); VA FYs begin the October 1 prior and end September 30, of the indicated year. We obtained VA patient-level utilization and cost records and linked them to clinic-level data for FYs 2009 and 2011. Patient data included sociodemographic characteristics, VA inpatient and outpatient utilization, diagnoses, lab test results, and costs of all care; clinic data included clinic characteristics and reporting of PCMH features. The study cohort was limited to patients who had at least 2 primary care visits in FY 2009 and used any VA outpatient care in FY 2011; altogether, 544,984 patients were excluded. Patients were linked to the clinic where they had a plurality of visits. The study cohort included 2,607,902 patients from 796 clinics; our cohort was older, predominantly male, more likely to be service-connected or below the means test, and sicker based on risk scores and chronic conditions compared with excluded patients. This study received approval from the Stanford University Institutional Review Board (protocol #20124).

Non-acute care outcomes were measured for each patient by number of primary care visits, specialty care visits, and telephone visits. Acute care outcomes were measured by ED visits and potentially avoidable hospitalizations. We also measured total annual costs of care in each study year. Outpatient visit data were obtained from the VA Medical Statistical Analysis System utilization files and categorized based on the visit location for outpatient services. Potentially avoidable hospitalizations were obtained from inpatient records and identified by primary diagnosis for an ambulatory care—sensitive condition (ACSC).

The total annual costs were obtained from Decision Support System records for services provided by the VA, and from Fee Basis records for services provided by non-VA providers and paid by the VA. Because of the short study period, we did not adjust costs for inflation.

Our main independent variable, reporting of PCMH features, came from the American College of Physicians Medical Home Builder (MHB) survey administered to all VA primary care clinics in 2009 and 2011. The MHB is a self-administered practice advisor used to support practices toward recognition of National Committee for Quality Assurance (NCQA) 2008 PCMH standards.

The MHB covers 7 PCMH components: patient-centered care and communication (support for patients’ self-management and decision making and staff communication training); access and scheduling (scheduling flexibility such as same-day appointments and non—face-to-face services); care coordination and transitions in care (coordinating visits to other physicians, creating individualized treatment plans, assessing treatment barriers); organization of practice (tracking procedures, test results, medication lists, team huddles); population management (patient registries, clinical guidelines, identifying unmet needs); quality improvement and performance improvement (performance measures, satisfaction surveys); and use of technology (practice management systems, electronic health records, decision support systems). These components overlap with 6 of NCQA’s standards for receiving PCMH certification, so the MHB captures elements similar to NCQA’s measures.

The MHB was self-administered by the clinic director or other clinic leader, and the survey response rate was 100% in 2009, and 98% in 2011. The sum of PCMH features within each component was obtained for the component scores. The component scores were grouped into low and high categories based on the baseline distribution below or above the median.

In multivariable models, we included measures of time-varying patient factors in both years. We measured the Charlson Comorbidity Index (CCI) score, a measure of comorbidity burden, for each patient based on diagnoses from inpatient and outpatient records.

We also created an indicator of risk based on available lab results reported during the study years. We chose lab tests that could, if increasing or decreasing during the study period, indicate higher risk of morbidity and mortality: cholesterol tests (>100 mg/dL), hemoglobin A1C test (>8%), anemia hemoglobin (<13 g/dL), and kidney function (eGRF <45).

For descriptive purposes, we measured several patient and clinic characteristics in the baseline year that were time invariant. Clinic measures included clinic rurality, assessed by metropolitan/nonmetropolitan codes from the Area Resource File.

Clinic type was categorized as based at a VA medical center (VAMC); VA-owned community-based outpatient clinic (CBOC)—owned by the medical center and staffed by VA providers; leased CBOC&mdash;staffed by VA providers with VA governance; or contracted CBOC&mdash;staffed by contract providers without VA governance.

Patient-level measures, including age, sex, race/ethnicity, marital status, service connection, means test, and insurance, were obtained from utilization records in FY 2009. Area income and education came from US Census data on income and maximum years of educational attainment linked to patients’ zip codes. We also estimated patients’ distance to their primary care clinic.

In bivariate analyses, we compared the mean number of primary care, specialty care, telephone care, and ED visits, and ACSC hospitalizations, and total costs of care per patient in FY 2009 and FY 2011 using 1-way ANOVA with year as the independent variable to determine significant changes over time. In multivariate analyses, we estimated the longitudinal association between changes in PCMH components scores with changes in utilization and cost measures using multi-level, linear regression models for each outcome. Regression models included a fixed effect for each patient and adjusted for time-varying patient factors (risk indicator and CCI score), dummy variables for high versus low clinic PCMH component scores, and an indicator for year. The regression model for costs used the log of costs as the dependent variable since costs had a skewed distribution.

In our models, the fixed effect controls for all factors fixed over time, such as patient sociodemographic factors and clinic characteristics (eg, rurality and size). The model estimates the “within” estimator; the coefficients represent the effect of a unit change in each time-varying predictor on patient outcomes.

The coefficient for the year indicator represented the time trend in outcomes that was not explained by changes in PCMH components or health status. Additionally, regression models adjusted standard errors to account for cluster (the intra-class correlations between patients within a primary care site).

We also conducted regressions using Poisson models for utilization measures since they were counts, but standard errors could not be adjusted for clustering within clinics since we used a fixed effect. The effect sizes were similar with linear models, and these results are presented in the 

). All regressions and data analyses were conducted using Stata 11.0 (StataCorp LP, College Station, Texas).

At baseline, most of the primary care patients in the study cohort were middle-aged (45%) or elderly (46%) (

). The study cohort was predominantly male (95%), a majority were white (67%), and a majority of patients were also married (58%). The largest proportion of veterans (42%) was eligible for VA care because of a service-connected disability, or they were within 5 years of military service. Relatively few patients (19%) had a disability rating greater than 50%, and many patients (40%) reported no other health insurance. Patients traveled a mean distance of 18 miles to their primary care clinic, the mean income of their area of residence was $40,541, and less than half of patients (46%) lived in areas where more than one-fourth of the population had a college degree.

The percent of patients with high risk (as measured by lab values) for cholesterol, diabetes, anemia, or kidney function decreased slightly from 61% of the cohort in FY 2009, to 56% in FY 2011. The CCI score, which measured total comorbidity burden, increased from 0.86 (SD = 0.84) to 0.90 (SD = 0.85) during that time.

In the baseline year, the low and high categories of each PCMH component represented roughly equal distributions of clinics; the exceptions were access and scheduling, which had more scores in the low category (

). The range of the component scores overall, and for low and high categories, is also shown in Table 2. The proportion of clinics in the high-scoring groups increased over time, and the majority of clinics reported PCMH component scores in the high-scoring groups for all components in FY 2011. Almost one-fifth of primary care clinics represented in the study sample were large VAMC-based clinics, and most were community-based outpatient clinics (CBOCs), with the majority of these being leased CBOCs. Almost one-third of clinics were in nonmetropolitan areas.

Changes in Utilization and Costs Over Time

During the study period from just prior to widespread PACT implementation to 2 years after PACT implementation began, the mean number of primary care visits decreased from 4.81 to 3.99 visits per patient, which represented a 17% decrease. The rate of specialty care visits was negligibly higher, and telephone visits rose by 85% (all 

). ED visits per patient rose slightly (7%), and ACSC hospitalizations per patient also rose from 0.02 to 0.03 per patient (all

<.001). The mean total costs of VA care increased from $8469 to $9887 per patient during the study period (

Increased Medical Home Features and Utilization and Costs

Multi-level, multivariable models adjusted for both time-varying factors and fixed effects for time-invariant factors, and estimated how changes in reported PCMH components in primary care clinics explained some of these increases or decreases in use of VA care over time (

). High organization of practice scores was significantly related to 0.13 fewer mean primary care visits per patient compared with patients in low-scoring clinics (

= .012). High care coordination and transitions in care scores were significantly related to 0.06 more specialty care visits compared with the low scores (

= .010), after controlling for patients’ health status and time trend. The change in telephone visits was significantly higher over time (

<.001), but changes due to increases from low to high PCMH scores across all components were not statistically significant. High scores in care coordination and transitions in care decreased the mean number of ED visits by 0.04 visits per patient (

= .018), but high quality and performance improvement increased ED visits by 0.03 visits per patient (

= .032), relative to low scores. Admissions for an ACSC also increased over time after adjusting for worse health status (both 

<.001), and none of the PCMH components were related to higher admissions. The only significant increase in costs was explained by high risk or worse comorbidity (

<.001), and again, none of the PCMH component scores were significantly related to total costs per patient.

VA primary care clinics reported large improvements in adoption of all PCMH components from FY 2009 to FY 2011, and there were significant changes in utilization and costs for a cohort of primary care patients during this time. While various PCMH components (eg, access and scheduling) were hypothesized to increase access to and frequency of primary care, the only component that was related in adjusted models was organization of practice. Features such as team huddles and tracking lab tests were actually associated with fewer primary care visits per patient, possibly through better efficiency of primary care practice. Greater specialty care visits were modestly related to higher care coordination/transitions in care scores, so better procedures to coordinate care appeared to facilitate referrals to specialty care.

While increased care coordination/transitions of care was significantly related to fewer ED visits, higher quality and performance improvement scores were related to more ED visits. Care coordination features that reduced problems due to transitions from hospitalization or referral appeared to reduce ED care that may be unnecessary. It is unclear why greater quality/performance improvement features would be related to more use of ED care by patients. There was a significant increase in telephone visits over time, and although it was not related to any specific changes from low to high PCMH components, the push overall to increase virtual care under PACT may have led to increased telephone care. None of the PCMH components were related to potentially avoidable hospitalizations for an ACSC, although the rates in both years were very low in this cohort. Changes in healthcare costs were also not associated with increased PCMH component scores.

Our findings contribute new evidence on the specific aspects of PCMH implementation that are related to different types of healthcare utilization, and some of these results are consistent with early evidence from a PCMH demonstration in Group Health, which showed that patients had higher rates of telephone visits and specialist visits, and lower rates of ED visits in PCMH sites after 1 year.

In addition, the Group Health demonstration showed significant reductions in hospitalizations and costs per patient after 2 years, which we did not observe.

 Unlike the Group Health demonstration, which was implemented in 1 clinic, VA’s PACT initiative was rolled out to all clinics—that have considerable heterogeneity in size, staffing, governance, quality improvement orientation, and patient populations served&mdash;which may partly explain differences in evaluation results.

An earlier VA study found that higher baseline adoption of PCMH components of access/scheduling and care coordination/transitions were significantly related to lower risk of avoidable hospitalizations among primary care patients.

While we found that increased scores of these PCMH components were not related to avoidable hospitalizations, there may be a lag between implementation of features and observing effects in patients’ outcomes. Moreover, reducing ACSC hospitalizations in a stable cohort of patients may be difficult to achieve.

It is also unknown to what extent individual PCMH features (eg, using an open-access model or maintaining a disease registry) were fully functioning. The percent of clinics in the high categories of all medical home components score jumped during the study period so that most clinics reported adopting a high number of possible PCMH features. Hence, some implementation may have been rudimentary. Preliminary evidence suggests wide variation in how VA practices adopted various PCMH features, such as enhancing open access through practice redesign, or new interventions

 with successful implementation influenced by local VA leadership engagement, staffing resources, and access to information and knowledge.

Additional challenges, including defining team roles and interpersonal confl ict, have also caused implementation barriers in some sites.

 In addition to site-specific factors affecting PACT implementation, the total number of VA enrollees increased from 5.1 million veterans in 2009, to 5.5 million veterans in 2011.15 The challenges of serving a larger volume of patients may have contributed to fewer face-to-face visits and placed additional burdens on providers in some sites.

The Medical Home Builder survey was self-reported, so we were not able to validate the adoption of reported PCMH features; because of ongoing PACT evaluation efforts, clinics may have been motivated to provide more positive responses as a result. While the statistical methods used here accounted for measurable and unmeasurable fixed factors over time, it did not account for any time-varying omitted factors, so if unmeasured time-varying factors such as some other technological change were related to reported PCMH features and patients’ utilization and costs, the effects of PCMH features would be biased.

We did not have reports of non-VA utilization, so if patients shifted to using more non-VA care such as ED visits, we could not measure this change. Finally, we focused on regular primary care users who continued to use VA outpatient care during the study period, so our results may not be generalizable to less frequent users of primary care or new users who were younger and had different demands for VA care.

These results document short-term changes during the early implementation of PACT that can provide feedback as PACT continues to be implemented and developed. While take-up of many PCMH features appeared to be high after the initiative began, certain improvements in care under organization of practice and care coordination/transitions appear to have affected only certain types of non-acute outpatient care; reductions in acute care ED visits and potentially avoidable hospitalizations largely remain to be seen. Identifying areas for ongoing improvement is critical to ensuring long-term success of care transformation under PACT. Refining PACT models in later stages may require more than adding new interventions; it may involve evaluation of staffing and other resource needs, better understanding of cost-effective care, ensuring support of clinical leadership, and widespread dissemination of successful practices. Better measures of PACT implementation are also needed to evaluate the longer-term consequences of PACT.

The authors wish to acknowledge Jack Needleman and Ciaran Phibbs for valuable comments on earlier drafts.

Health Economics Resource Center (JY, JL), and Center for Innovation to Implementation (JY), VA Palo Alto Health Care System, Menlo Park, CA; Center for Innovation for Veteran-Centered and Value-Driven Care, VA Puget Sound (C-FL), Seattle, WA; Department of Health Services, University of Washington (C-FL), Seattle, WA; Patient Care Services, Veterans Health Administration (GS, RS), Washington, DC; Center of Innovation for the Study of Healthcare Innovation, Implementation, & Policy, VA Greater Los Angeles (LVR, EMY), Sepulveda, CA; RAND Corp (LVR), Santa Monica, CA; Department of Health Policy and Management, Fielding School of Public Health (LVR, EMY), and School of Medicine (LVR), University of California, Los Angeles, CA.

This work was supported by the Department of Veterans Affairs, Veterans Health Administration, Patient Care Services (XVA 65-018). Dr Yano’s effort was covered by a VA HSR&D Senior Research Career Scientist Award (Project #RCS 05-195). The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States federal government.

 The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article. VA Patient Care Services designed and administered the survey, and Drs. Stark and Schectman from VA Patient Care Services contributed to the interpretation of results.

 Concept and design (JY, LVR, EMY, CL); acquisition of data (EMY); analysis and interpretation of data (JY, CL, JL); drafting of the manuscript (JY, CL); critical revision of the manuscript for important intellectual content (LVR, EMY, RS, GS); statistical analysis (JY, JL); provision of patients or study materials (RS, GS); obtaining funding (LVR, EMY); administrative, technical, or logistic support (JL); supervision (JY, LVR, EMY).

Jean Yoon, PhD, MHS, 795 Willow Rd (152 MPD), Menlo Park, CA 94025. E-mail: 

1. Reid RJ, Coleman K, Johnson EA, et al. The Group Health medical home at year two: cost savings, higher patient satisfaction, and less burnout for providers. 

2. Reid RJ, Fishman PA, Yu O, et al. Patient-centered medical home demonstration: a prospective, quasi-experimental, before and after evaluation. 

Significant changes were found in patients' utilization of healthcare related to early implementation of medical home components in VA primary care clinics.

Jeanie Lo, MPH

Articles

Inappropriate Ordering of Lumbar Spine Magnetic Resonance Imaging: Are Providers Choosing Wisely?

Early Changes in VA Medical Home Components and Utilization