To estimate the relationship between electronic medical record (EMR) use and efficiency of utilization and provider productivity during visits to US office-based physicians.

 Cross-sectional analysis of the 2006-2007 National Ambulatory Medical Care Survey.

The sample included 62,710 patient visits to 2625 physicians. EMR systems included demographics, clinical notes, prescription orders, and laboratory and imaging results. Efficiency was measured as utilization of examinations, laboratory tests, radiology procedures, health education, nonmedication treatments, and medications. Productivity was measured as total services provided per 20-minute period. Survey-weighted regressions estimated association of EMR use with services provided, visit intensity/duration, and productivity. Marginal effects were estimated by averaging across all visits and by major reason for visit.

 EMR use was associated with higher probability of any examination (7.7%, 95% confidence interval [CI] = 2.4%, 13.1%); any laboratory test (5.7%, 95% CI = 2.6%, 8.8%); any health education (4.9%, 95% CI = 0.2%, 9.6%); and fewer laboratory tests (−7.1%, 95% CI = −14.2%, −0.1%). During pre/post surgery visits, EMR use was associated with 7.3% (95% CI= −12.9%, −1.8%) fewer radiology procedures. EMR use was not associated with utilization of nonmedication treatments and medications, or visit duration. During routine visits for a chronic problem, EMR use was associated with 11.2% (95% CI = 5.7%, 16.8%) more diagnostic/screening services provided per 20-minute period.

 EMR use had a mixed association with efficiency and productivity during office visits. EMRs may improve provider productivity, especially during visits for a new problem and routine chronic care.

	Electronic medical record (EMR) use had a mixed association with efficiency and productivity during office visits, and the relationships varied by type of service and by the major reason for the visit.

		EMR users had higher intensity and productivity of diagnostic/screening services, especially during visits for a new problem and routine visits for a chronic problem.

		Use of EMRs may alter the content of office visits and improve a provider’s productivity, which might lead to cost savings and quality improvements.

		Contrary to expectation, EMR use had no association with efficiency or productivity during visits for preventive care.

	Health information technology is widely believed to enhance quality of care and patient safety and to lower costs and improve efficiency.

 Despite these potential benefits, adoption of electronic medical records (EMRs) by US physicians in office settings has been slow.

 Uncertain financial return and loss in productivity are often cited as barriers to adoption.

 Nonetheless, surveys report that physicians perceive that EMR use can improve work flow, the quality of clinical decisions, and the delivery of  preventive care.

	Empirical studies demonstrating the impact of EMR systems on the efficiency and quality of care in ambulatory settings have been limited.

 Most prior work has focused on the relationships between specific EMR functions and medication safety

 In contrast, relatively few studies have examined the association of EMR with efficiency of utilization and provider productivity.

 The economic benefits of integrated EMR functionality from commercial systems used in community settings remains uncertain.

	This study examined the association between EMR use and efficiency and productivity during office visits using a large-scale, nationally representative data set. The findings from this study provide important evidence of the value of health information technology in ambulatory care.

Electronic Medical Record Use and Efficiency of Utilization

	In theory, EMR use has the potential to improve efficiency of utilization. An EMR system may include clinical notes, problem/medication lists, and test results. These functions can provide information about chronic conditions and prior utilization, which might reduce redundant and inappropriate diagnostic/screening services and medications. Computer-generated care suggestions and automated reminders could improve adherence to evidence-based guidelines and might increase provision of some services. Thus, in theory, EMR might increase or decrease utilization of services depending on the EMR functionality and the reason for visit.

	Prior studies of the association of EMR use with the efficiency of utilization have been limited.

 Some evidence suggests that EMR use can improve care for chronic illness and preventive care.

 However, 3 large-scale studies found little relationship between EMR use and quality of care in ambulatory settings.

 Whether and to what extent that EMR use is associated with the level of utilization during visits remains uncertain.

Electronic Medical Record Use and Provider Productivity

	An EMR system can automate manual tasks, streamline documentation, and improve access to information. These EMR functions can support clinical decision making and might reduce physician time, at least in theory.

 Improvements in productivity could allow physicians to see more patients per day or to provide more services to the same patient during each visit.

	Evidence of the impact of EMR use on provider time efficiency is mixed,

 and few studies have examined the relationship between EMR use and visit duration in ambulatory settings.

 Prior studies of the association of EMR use with provider productivity have also been limited, and study designs have varied in their unit of analysis.

 Whether and to what extent that EMR use is associated with visit intensity, duration, and productivity per visit remains an open question.

	The study used data from the public use version of the National Ambulatory Medical Care Survey (NAMCS) from 2006 and 2007. Conducted by the National Center for Health Statistics of the Centers for Disease Control and Prevention, the NAMCS is a survey of visits to US nonfederal office—based physicians. The multistage probability design is based on a random sample of physicians stratified by geographic area and specialty. Patient visits during a randomly selected week were sampled for each participating physician. Patient characteristics, reason for visit, and utilization of medical services were reported on individual patient record forms. Information about the physician and their practice, including EMR use, were captured during a separate intake survey. The NAMCS included weights for visits and physicians that allowed for the generation of nationally representative estimates. The 2006 and 2007 surveys collected information on 62,170 visits to 2625 physician respondents, and the full sample was included in the analysis.

	The main variable of interest is whether the physician’s practice used an EMR system. The NAMCS asked “Does this practice use electronic medical records (not including billing records)?” The survey also asked whether the practice’s EMR system or another computerized system included any of 13 specific EMR functions (

). Nonresponse and survey responses of “turned off” and “unknown” were included as not having the EMR system or function. Based on definitions developed by a consensus panel,

 EMR use in this study was defined as a Basic or Fully Functional EMR system with at 

Efficiency of Utilization and Provider Productivity

	The NAMCS collected details on the number and type of medical services ordered or provided during each visit. The NAMCS allowed physicians to report 2 additional diagnostic/screening services, which were classified by 

International Classification of Diseases, Ninth Revision, Clinical Modification

) procedure code. These services were included in laboratory tests if the 

 code started with 90 or 91 and in radiology procedures if the 

code started with 87, 88, or 92. Measures of efficiency of utilization and provider productivity during office visits were specified based on prior literature

	The NAMCS captured information about the patient’s major reason for visit, and each visit was classified into 1 of 5 visit types. “New problem” included visits for conditions that occurred within 3 months of the visit. “Routine visit for a chronic problem” included visits to receive care or examination for a preexisting chronic condition, illness, or injury that occurred more than 3 months prior to the visit. “Preventive care” included visits for general medical examinations and routine periodic examinations. “Flare-up of a chronic problem” included visits primarily due to sudden exacerbation of a preexisting chronic condition. “Pre/post surgery” included visits scheduled primarily for care required prior to or following surgery.

	A cross-sectional analysis of pooled survey data was conducted. Two-part models of medical care utilization were specified to analyze the association of EMR use with efficiency of utilization.

 This allowed separate analyses of the relationship of EMR use with the probability of any use and the number of services provided, conditional on some utilization. All productivity measures had a highly skewed distribution and were log-transformed to approximate a normal distribution. Analyses were conducted by averaging across all visits and by major reason for visit.

	Estimation was performed using Stata 10.1 software that accounted for the complex survey design. Survey-weighted probit, Poisson, and ordinary least squares  regressions included patient, physician, and practice characteristics reported in 

. Marginal effects from probit/ordinary least squares regressions and semielasticities from Poisson regressions were calculated, and results can be interpreted as the percent change in the dependent variable associated with EMR use. 

	Descriptive statistics for patient, physician, and practice characteristics are reported in eAppendix A. Descriptive statistics for efficiency of utilization and provider productivity during office visits are presented in 

Electronic Medical Record Use in Physician Offices

	Table 1 presents nationally representative estimates of physician use of EMR systems and specific EMR functions. In 2006-2007, 32.5% of US office-based physicians reported the use of any EMR system in their practice. On average, EMR systems included 7.25 out of 13 functions. Conditional on some EMR use, patient demographic information (90.2%) and clinical notes (78.4%) were the most commonly used EMR functions. Test orders sent electronically (33.7%) and electronic images returned (28.5%) were the least commonly used.

	Although one-third of physicians used any EMR, only 10.9% of US office-based physicians reported the use of a Basic or Fully Functional EMR system, which included the minimum set of functions. On average, Basic/Fully Functional EMR systems included 11.11 out of 13 functions, with 94.2% having medical history and follow-up notes and  85.2% having computerized orders for tests.

 presents regression results of physician and practice characteristics associated with EMR use. Specialty, geographic region, practice size, ownership, and electronic billing/claims submission were significant predictors of EMR use. Relative to general/family practice, physicians in psychiatry (−7.7%) and pediatrics (−4.2%) had a lower probability of EMR use. Relative to the Midwest region, practices in the Northeast region were 4.7% less likely to use an EMR system. Solo practitioners had a 5.6% lower probability of EMR use than physicians in partnerships/group practices. Relative to practices owned by a physician/physician group, practices owned by an HMO and practices owned by a corporation/other were 43.8% and 12.2% more likely, respectively, to use EMRs. Practices with electronic billing/claims submission had a 5.4% higher probability of EMR use. Metropolitan status, the number of managed care contracts, and percentage of revenue from managed care had no significant association with EMR use.

	Patients of EMR users differed from nonusers in some characteristics (eAppendix A). Patient age, race, chronic conditions, and insurance status were different for  physicians with EMR use. Patients of EMR users were less likely to be children (aged 17 years and under) and to have Medicaid or self-pay insurance, but were more likely to be aged 18 to 44 years, to be other race, and to have hyperlipidemia, diabetes, and private insurance.

 presents regression results of the association of EMR use with efficiency of utilization during office visits. Use of EMRs had a strong relationship to any use of diagnostic/screening services. On average, EMR users had a higher probability of providing any examination ( 7.7%) and any laboratory test ( 5.7%), especially during routine visits for a chronic problem and visits for a new problem.

Articles

MANAGERIAL

The relationship between electronic medical record use and efficiency/productivity during office visits varied by type of service and reason for visit.

Electronic Medical Records and Efficiency and Productivity During Office Visits

To determine potential predictors of sustainability among community-based organizations that are implementing health information technology (HIT) with health information exchange, in a state with significant funding of such organizations.

 A longitudinal cohort study of community-based organizations funded through the first phase of the $440 million Healthcare Efficiency and Affordability Law for New Yorkers program.

 We administered a baseline telephone survey in January and February 2007, using a novel instrument with open-ended questions, and collectedfollow-up data  from the New York State Department of Health regarding subsequent funding awarded in March 2008. We used logistic regression to determine associations between 18  organizational characteristics and subsequent funding.

All 26 organizations (100%) responded. Having the alliance led by a health information organization (odds ratio [OR] 11.4, P = .01) and having performed a community-based needs assessment (OR 5.1, P = .08) increased the unadjusted odds of subsequent funding. Having the intervention target the long-term care setting (OR 0.14, P = .03) decreased the unadjusted odds of subsequent funding. In the multivariate model, having the alliance led by a health information organization, rather than a healthcare organization, increased the odds of subsequent funding (adjusted OR 6.4; 95% confidence interval 0.8, 52.6; P = .08).

Results from this longitudinal study suggest that both health information organizations and healthcare organizations are needed for sustainable HIT transformation.

	Implementing health information technology (HIT) is a national priority. However, the sustainability of community-based HIT implementation efforts is unclear.

		We conducted a cohort study of community-based organizations in Phase 1 of New York’s $440 million HIT initiative to determine associations between baseline organizational characteristics and subsequent funding.

		We considered 18 potential predictors, including characteristics of the grantees and characteristics of their planned interventions.

		Community-based efforts were 6 times more likely to receive subsequent funding if they were led by a health information organization rather than a healthcare organization.

	Through the American Recovery and Reinvestment Act of 2009, the federal government is providing approximately $17 billion in financial incentives to physicians and hospitals for meaningful use of electronic health records (EHRs) that support the exchange of data electronically.

 The combination of EHRs and health information exchange is expected to decrease fragmentation of healthcare and improve coordination of care.

However, the optimal strategy for how best to implement EHRs and health information exchange is not known. Moreover, early experiences with community-based strategies for achieving EHR implementation and health information exchange have been fraught with failures.

	New York State has been embarking on an ambitious and novel program since 2005 called the Healthcare Efficiency and Affordability Law for New Yorkers (HEAL NY), through which it is investing approximately $440 million in interoperable health information technology (HIT), with a focus on EHRs and health information exchange.

 This represents the largest state-based investment in the country.

In addition, New York’s initiative includes $280 million in matching funds from grantee communities and $120 million in matching funds from other state and federal programs.

 Beyond financial support, New York State is also providing strategic direction, collaborative assistance, and technical support.

	New York State’s program relies in large part on community-based alliances that come together for the purpose of EHR implementation and health information exchange. We found previously that 100% of alliances funded under the first phase of HEAL NY (N = 26) were in existence 2 years after the start of the program, which is a much higher rate of survival than that found nationally.

 However, only 11 of the 26 alliances received funding in the next phase of HEAL NY.

 Our previous study did not determine which alliances were most likely to receive funding in the second round. Understanding predictors of continued funding—as potential markers of early sustainability&mdash;could inform similar state-based and community-based initiatives nationwide.

	Our objective was to determine which baseline characteristics of HEAL NY grantees were associated with early sustainability, as demonstrated by receiving subsequent HEAL NY funding.

	We conducted a longitudinal study of the 26 community-based alliances funded through the first phase of HEAL NY. Baseline data were collected through a telephone survey conducted over a 5-week period in January and February 2007. (At that time, grantees had been notified of their awards but had not yet received funding from New York State.) We supplemented baseline survey data with publicly available data about the grantees from the New York State Department of Health Web site.

 We obtained follow-up data from the Department of Health Web site, which announced the status of funding for HEAL NY Phase 5 in March 2008.

 (Note that HEAL NY Phase 5 was the second phase of HIT funding, as HEAL NY Phases 2 and 4 were not related to HIT, and Phase 3 was withdrawn, with funds reallocated to HEAL 5.) The Institutional Review Board of Weill Cornell Medical College approved this study.

	The HEAL NY program selected grantees based on responses to public Requests for Grant Applications.

 Grantees were community-based alliances that included stakeholders of various types not under the same corporate structure (eg, hospitals, physician practices, health plans). The lead entity for such an alliance could either be a health information organization (eg, a regional health information organization) or a healthcare organization (eg, a hospital, an independent practice association). Grantees were required to provide funds to match any state grant in at least a 50:50 ratio for HEAL 1 and a 25:75 (grantee to state) ratio for HEAL 5.

	Projects for HEAL 1 were required to address adoption of EHRs, support of electronic prescribing, and/or development and implementation of community-wide clinical data exchanges. Projects for HEAL 5 were required to address adoption of EHRs, development and implementation of community-wide clinical data exchanges, and development and implementation of clinical information services. Criteria for receiving funding under HEAL 5, which were announced in October 2007 (after baseline data for this study were collected), were more stringent than those under HEAL 1, requiring more advanced governance structures, business models, and willingness to participate in a statewide collaboration process.

	This survey was conducted as part of a broader effort by the Health Information Technology Evaluation Collaborative (HITEC). HITEC is a multi-institutional academic collaborative established to conduct independent and rigorous evaluation of New York State’s HIT initiatives.

 The authors include directors and members of HITEC.

	We included all 26 HEAL 1 grantees. Using a standardized telephone script, we invited the leader of each grantee to participate. Prospective participants were told that HITEC was conducting a study “to tell the story of HEAL NY” and that responses were voluntary.

	No previously published instrument was suitable for this study. We therefore created a novel instrument with 26 openended questions about the activities of the grantees  (

). The instrument was pilot-tested internally and refined for clarity. The questions were divided into 9 sections: overview of the planned interventions (including project goals, clinical settings, clinical issues, and use case analysis), EHRs and electronic prescribing, health information exchange, organization and governance, privacy policies, timetables for implementation, financial sustainability, expected impact, and evaluation.

	Each telephone interview was conducted by 1 of 4 HITEC members (LMK, ABW, JS, KY-F). Each interview was confirmed in advance by an e-mail that included (1) the surveyinstrument, (2) disclosures that some of the members of HITEC were involved in organizations funded through HEAL 1 and some were involved in applications for HEAL 5, and (3) notification that representatives from the New York State Department of Health would have access to the data collected. Each telephone interview lasted 30 to 75 minutes and was audio-recorded.

	Two investigators (LMK, RK) created domains a priori for categorization of survey data. Each investigator who conducted interviews compiled typed notes of his/her interviews. One investigator (LMK) assembled these notes into a summary of results overall, including categorization of responses to questions into the previously  constructed domains. Categorization into multiple, non—mutually exclusive domains was permitted. The other interviewers and members of the study team reviewed this summary for clarity and face validity.

	In addition, we classified the grantees into 2 organization types, based on the type of organization that served as the grantee’s lead applicant: healthcare organization or health information organization.

	We considered 18 potential predictors of HEAL 5 funding, which are shown in the 

. We used bivariate and multivariate logistic regression to determine associations between the potential predictors and the receipt of HEAL 5 funding (yes/no). Potential predictors with bivariate P values of <.20 were entered into the multivariate model. We used backward stepwise elimination to generate the most parsimonious model. We excluded from the model the variable of “pursuit of health information exchange,” because this was true for all grantees and thus could not be used to distinguish grantees from one another. We also excluded the size of the HEAL 1 award as a potential predictor, because this variable was not conceptually informative; that is, a larger HEAL 1 award may have indicated a more successful organization, but it did not explain   why the organization was successful.

	Analysis was conducted using Excel (Microsoft Corp, Redmond, WA), Stata version 9 (StataCorp, College Station, TX), and SAS version 9.2 (SAS Institute, Cary, NC).

	All 26 HEAL 1 grantees participated (100% response rate). The grantees received awards ranging in size from $177,503 to $5,000,000 each, with a mean award of $1.8  million (median $1.7 million). Grantees were distributed across 6 geographical regions of New York State (Table).

	Respondents listed a variety of stakeholders as participants in their organizations, with the most common being physician practices (mean 5.6 practices per grantee), community hospitals (mean 2.4), community health centers (mean 1.5), health plans (mean 1.4), and long-term care facilities (mean 1.1). Two-thirds of grantees (69%) did not include an academic medical center, as defined as a tertiary care teaching hospital with an on-site medical school.

	Half (50%) of the 26 grantees described themselves as having a formal governance structure. Fifteen percent reported having no formal governance structure and no plans to create one. The remainder (35%) reported either that formal governance structures were in the process of being created or that the governance structure for the grantee organization was composed of 1 or more committees housed inside a much larger healthcare organization.

	Approximately two-thirds of grantees (65%) were led by healthcare organizations. The other one-third (35%) were led by health information organizations formed expressly for the purpose of implementing health information exchange among the stakeholder organizations.

	The majority of grantees (80%) reported working on developing appropriate privacy policies specific to their HEAL NY initiatives. Two (8%) reported already having created privacy policies specific to their HEAL NY initiatives. Three (12%) reported that they would rely on more general existing privacy policies rather than pursue creation of specific new policies.

	Nearly two-thirds of grantees (62%) reported having performed a community-based needs assessment, which was defined as “any analysis your organization has done to frame the clinical problem and the potential benefit offered by your project.” Examples given of such analyses included surveys of providers to assess clinical needs and work flow, surveys of patients to assess comfort with various technologies, and projected return-on-investment analyses by external consultants. The clinical problems that the grantees reported trying to address included the inability to access remote data, gaps in communication between providers, suboptimal healthcare quality, and inefficient care.

Characteristics of the Planned Interventions

	All 26 grantees (100%) reported their intent to implement health information exchange. In addition, 17 (65%) planned implementation of EHRs, 17 (65%) planned implementation of electronic prescribing, and 12 (46%) planned implementation of both EHRs and electronic prescribing.

POLICY

Communities led by health information organizations were more likely than those led by healthcare organizations to receive ongoing funding for implementing health information technology.

Community-Based Health Information Technology Alliances: Potential Predictors of Early Sustainability

 To describe variations in the structure of anticoagulation clinic (ACC) care within the Veterans Health Administration (VA) and to identify structures of care that are associated with better site-level anticoagulation control.

 Questionnaire correlated with automated clinical data.

We characterized 90 VA ACCs using a questionnaire administered by the VA Central Office. Site descriptors included staffing levels, provider training, visit modalities, quality improvement programs, documentation, and care coordination. Patient outcomes were measured by site mean risk-adjusted percentage time in therapeutic range, a measure of anticoagulation control over time. Our study was powered to detect a 3% difference in risk-adjusted percentage time in therapeutic range, a small-to-moderate effect size, between sites with and without a certain characteristic.

 We observed considerable variation in the structure of ACC care. For example, 48 sites had fewer than 400 patients per provider, 25 sites had 400 to 599 patients per provider, and 17 sites had 600 patients or more per provider. However, none of the site characteristics measured were significantly related to anticoagulation control.

We found substantial variation in guideline-targeted organizational and management features of ACC care within the VA. However, no single feature was associated with better anticoagulation control. Current guidelines for organizing an ACC may have limited relevance for improving patient outcomes.

	Dedicated anticoagulation clinics (ACCs) improve anticoagulation control and patient outcomes.

		Clinical guidelines recommend that ACCs should have certain features, but these recommendations are based on expert opinion rather than on empirical evidence of benefit.

		In our study, 90 ACCs in an integrated system of care differed widely in their organization and management; however, none of these differences were consistently related to ACC performance as measured by anticoagulation control, an important intermediate outcome of anticoagulation care.

		Current guidelines for organizing and managing an ACC may have limited relevance for improving outcomes.

	Millions of patients receive oral anticoagulation therapy to treat or prevent thromboembolic disease.

 While effective management of warfarin sodium therapy is not easy, better anticoagulation control can improve outcomes and reduce adverse events.

 Indeed, a growing body of literature has shown that patients whose care is managed in  dedicated anticoagulation clinics (ACCs) have better outcomes than those managed in usual care.

recommend that all ACCs should have certain features, including quality improvement programs, adequate documentation of care, sufficient training for providers, and a ratio of patients to providers of less than 400:1. However, there is little evidence as to which, if any, of these features contribute to the improved outcomes seen with ACC care.

	Although oral anticoagulation therapy has been used for more than 5 decades, quality measurement has had little penetration into this field.

 Our group recently undertook the first systematic effort to profile riskadjusted anticoagulation control in an integrated health system. Within the Veterans Health Administration (VA), a system in which essentially all anticoagulation therapy is managed in an ACC, the mean percentage time in therapeutic range (TTR) varied widely among sites,

 with some sites performing more than 10% better or worse than would be expected based on the risk-adjustment model. Variations of this magnitude are associated with important differences in rates of stroke, venous thromboembolism, and major hemorrhage.

 To help all sites in the VA improve their performance, it is necessary to understand the site-level correlates of better or worse anticoagulation control.

	Therefore, this study had the following 2 objectives: first, to describe differences in the organization and management of ACCs within the VA, and second, to assess whether these differences help explain sitelevel differences in anticoagulation control. We measured site characteristics using a survey of VA ACCs that examined structural features that seem likely to affect anticoagulation control, including staffing ratios, provider training protocols, and the existence of quality improvement programs.

 We hypothesized that these variables would be associated with anticoagulation control. In this study, either a positive finding or a negative finding would be useful because it can help determine whether enforcing conformity to any specific feature is likely to improve anticoagulation care and control in the VA.

	The data for this study have been described elsewhere. The Veterans Affairs Study to Improve Anticoagulation

 included all patients who received oral anticoagulation therapy from the VA between October 1, 2006, and September 30, 2008. The study was approved by the institutional review board of the Bedford VA Medical Center, Bedford, Massachusetts.

	We included international normalized ratio (INR) values when patients were “on warfarin” (ie, when a patient was in possession of warfarin or was having INR tests performed at least every 42 days). We defined the period of warfarin possession as the duration of the most recent VA prescription for warfarin, plus 30 days. We calculated percentage TTR using the method by Rosendaal et al,

which assigns an INR value to each day by linear interpolation between successively observed INR values. Gaps of 56 days or more between INR values are not interpolated. The patient’s TTR equals the percentage of days for which the interpolated INR values lie between 2.0 and 3.0 (from 0%-100%).

Independent Variables of Site-Level Characteristics

	There are 128 sites of care within the VA, each of which includes a hospital, an outpatient care center, and several outlying community-based clinics. Each site has a specialized ACC, which is usually run by clinical pharmacists under the supervision of a medical director.16 Therefore, essentially all anticoagulation care within the VA is delivered within specialized ACCs. In October 2006, VA Pharmacy Benefits Management surveyed all 128 VA sites of care for the organization and management of their ACCs. Topics included visit modalities (face to face, telephone, and mail), quality improvement programs, clinic staffing, provider training, documentation, and care coordination (the verbatim text of the questionnaire is given in the 

]). We abstracted our independent variables from the responses to the questionnaire.

Dependent Variable of Risk-Adjusted Anticoagulation Control

	Site mean TTR was adjusted for case mix using a model that incorporates patient demographics, comorbid conditions that have an adverse effect on TTR, and general measures of comorbidity, including the number of medications and the number of hospitalizations. The derivation and validation of this model, which achieved an R2 of 13.3%, has been described previously.

 The model was used to calculate the expected mean TTR for each site (“E”) based on patient characteristics. The expected mean TTR was compared with the observed mean TTR for each site (“O”). Therefore, each site’s performance was characterized by an O minus E (“O − E”) score, our measure of risk-adjusted TTR (RA-TTR).

	We began by first calculating the mean observed TTR and the mean expected TTR for each site of care and then computing an O − E score for each site. We compared O − E scores between sites with and without various organizational characteristics using unpaired t test and compared O − E scores for multilevel variables using analysis of variance. We did not adjust for multiple comparisons. With regard to statistical power, for a characteristic present at half of the sites, t test would have 80% power to detect a 3% difference in the O − E score (a small-to-moderate effect size). Analyses were performed using commercially available statistical software (SAS version 9.1; SAS Institute, Cary, North Carolina).

	Of 128 sites in the VA, 28 were excluded because their data were insufficiently complete to fully assess TTR. Of the remaining 100 sites, 5 did not respond to the  questionnaire. Five other sites included more than 1 ACC, and the responses we received indicated that practices at the multiple ACCs of these 5 sites were not uniform. However, the dependent variable (RA-TTR) was assessed at the level of the overall site. Because we were unable to match our data for structure and outcomes, we excluded these 5 sites as well. This left 90 sites with complete data on structure and outcomes of care. The mean (SD) number of patients managed at each site was 1244 (799). The site mean TTR ranged from 38% to 69% (median, 58%). Site O − E scores ranged from −17% (ie, 17% below the expected value) to 12%.

Structure of Care and Relationship With Performance

	We observed considerable variation in structure of care (

). About half of the sites (n = 50) conducted most of their visits face to face; at other sites, most care was provided by telephone or mail. Most sites (n = 59) had some sort of quality improvement program, although only 7 sites used formal plan-do-check-act cycles. Most sites (n = 77) had some sort of support; 43 sites had clerical support. About half of the sites (n = 41) had a formal protocol for training new clinic providers. Many sites (n = 48) adhered to the recommended staffing ratio of less than 400 patients per provider (

), although 17 sites had 600 or more patients per provider. Sites differed markedly for the perceived likelihood of being informed about a new drug-drug interaction.

	Despite these differences in structure of care, we found no statistically significant predictors of site-level performance. A finding of marginal statistical significance was that 8 sites that did not allow telephone follow-up visits had almost 3% worse TTR (P = .10). There was also a hint, although not statistically significant, that sites with fewer than 500 patients under management might have worse control (performance difference, −1.6%).

	We performed sensitivity analyses by altering the format of the dependent variable. To ensure that our risk-adjustment model was not obscuring relationships between  structure and outcomes, we also examined the same independent variables as predictors of the mean unadjusted TTR by site, with similar results (data not shown). We also tried categorizing site O − E scores into the top quintile, bottom quintile, and all others and repeated our analyses; the results were unchanged (data not shown).

	We also performed sensitivity analyses by altering the format of the independent variables. For example, we varied the cutoffs for what constituted a small site of care or for what constituted adequate staffing levels; the results were unchanged. Finally, we created a combination quality score by assigning a point for each putative quality indicator that a site fulfilled, to examine whether these factors predicted performance better in the aggregate than individually. For example, a site might receive 1 point for having a dedicated program to train providers, 1 point for having fewer than 400 patients per provider, and 1 point for using a computerized system for documentation and patient   tracking. We examined several different versions of this combination quality score, including various measures and assigning them variable relative weights, but none of  these combination scores predicted site RA-TTR.

	We examined site-level organizational factors (structure of care) as potential predictors of outpatient oral anticoagulation control (outcomes of care).

 Although our hypotheses were supported by consensus guidelines,

 the site-level characteristics that we studied were not associated with anticoagulation control. Many studies relating structure and processes of care have been performed within the VA among sample sizes similar to that of this study. One such study

 found that organizational culture and commitment to continuous quality improvement were not associated with better processes of care for depression. However, most other studies have found associations between various aspects of organizational culture and processes of care, including rates of cancer screening

 and recommended processes of care for heart failure,

 In the diabetes study, the authors also found associations between organizational culture and intermediate

 a relationship more directly comparable to our examination of structure and intermediate outcomes. However, unlike these previous studies, we directly measured structural aspects of care within the ACC itself, which we expected to more directly affect anticoagulation control. Therefore, our null findings are all the more surprising; furthermore, they suggest the need to rethink quality improvement guidelines in anticoagulation care. For example, while some VA sites seem to  emphasize face-to-face visits, this may not always be an efficient use of resources for the patient or for the provider, as telephone-based or mail-based care seemed to achieve similar results in this and other studies.

	Our study results do not suggest, nor do we believe, that efforts to improve the quality of oral anticoagulation therapy are futile. In fact, our group plans to implement a program to improve anticoagulation control in the VA, as we believe that this is feasible and important. Rather, our study findings suggest that we must look beyond the limited measures examined herein to find the true determinants of high-quality oral anticoagulation care. Which aspects of structure of care did this study not measure and what might be fitting targets for future studies? We did not directly investigate how warfarin dosages are managed or which dosing protocols or computerized dosing aids are used. A rich literature shows that computer-aided dosing of warfarin

 or simply judicious dosing in the absence of computerized support

 can improve anticoagulation control. Our study was adequately powered to detect a 3% difference in RA-TTR, which is modest. By comparison, using a computerized dosing algorithm improves TTR by approximately 10% over usual care,

 and more judicious dosing without computer assistance could improve TTR by approximately 6%.

 It seems likely that aspects of structure or process of care that relate to actual warfarin dosage management would help explain the wide variations in ACC performance within the VA. Also, more detailed assessments of ACCs through site visits and staff interviews might identify differences between high-performing and low-performing clinics.

	This study has considerable strengths. We used measures of structure of care that are supported by prominent guidelines

 and evaluated structure within the ACC itself rather than in the organizational culture of the entire medical center. In addition, our outcome measure (RA-TTR) has been carefully developed

and represents the state of the art in measuring intermediate outcomes of anticoagulation care.

 This study also has limitations. Responses to the survey were by self-report, and it is possible that some responses (eg, the number of providers) were not fully accurate. Also, as already discussed, this survey instrument may not have collected enough detailed information to adequately characterize some aspects of structure, such as the nature rather than the mere existence of a quality improvement program. Finally, while our risk-adjustment model for TTR controlled for multiple measures of comorbidity and achieved a high R2, it did not include data on adherence. However, part of the influence of good management may be expressed through improved adherence; adjusting for adherence could adjust away such an effect. Therefore, adherence arguably does not belong in the risk-adjustment model.

	In summary, our data suggest that high-quality anticoagulation care can be provided across a wide array of structures of care. A dedicated ACC is a necessary first step for improving the quality of oral anticoagulation care.

However, within the structure of an ACC, further quality improvement efforts should focus on aspects of care that have been shown to affect anticoagulation control, such as the judicious dosing of warfarin.

 A program of quality improvement based solely on the measures studied herein may not have the desired effect on outcomes.

	The authors thank Fran Cunningham, PharmD, of VA Pharmacy Benefit Management for providing the data from the 2007 survey of VA Anticoagulation Clinics.

	Author Affiliations: From the Center for Health Quality, Outcomes, and Economic Research (AJR, AO, JIR, DRB), Bedford VA Medical Center, Bedford, MA; Department of Medicine (AJR, EMH, ASA, DRB), Section of General Internal Medicine, Boston University School of Medicine, Boston, MA; Pharmacy Benefits Management (FC), VA Central Office, Washington, DC; Biostatistics Core (AO), Clinical Research Program, Children’s Hospital Boston, Boston, MA; Department of Quantitative Health Sciences (ASA), Division of Biostatistics and Health Services Research, University of Massachusetts School of Medicine, Worcester, MA; Manchester VA Medical Center (PC), Manchester, NH; Providence VA Medical Center (MG), Providence, RI; and Department of Health Policy and Management (DRB), Boston University School of Public Health, Boston, MA.

	Funding Source: This project was supported by a Career Development Award from the Veterans Affairs Health Services Research and Development Service. The opinions expressed herein do not necessarily represent the official views of the Department of Veterans Affairs.

CLINICAL

Anticoagulation clinics in an integrated healthcare system differed widely in their organization and management, but these differences were not consistently related to their performance.

Relevance of Current Guidelines for Organizing an Anticoagulation Clinic

To determine pneumonia admission care components that are most affected by emergency department (ED) crowding.

 Secondary analysis of a crosssectional observational survey.

 The setting was a 29-bed academic ED with 39,000 visits per year and state-mandated ratios of nurses to patients. The patients were ED admissions with pneumonia, January 1, 2004, to June 30, 2005. From ED medical records and databases, we abstracted the times of arrival, room placement, ordering of chest radiograph and when obtained, ordering of blood culture and when obtained, and ordering of antibiotic and when administered. We assessed associations between ED volume at the time of arrival of a patient with pneumonia and component durations using multivariate linear regression.

 For 407 ED admissions with pneumonia, the median component durations (in minutes) were as follows: 20 for arrival to room placement, 44 for arrival to chest radiograph order, 10 for chest radiograph order to radiograph obtained, 120 for room placement to antibiotic order, 10 for blood culture order to culture obtained, 30 for antibiotic order to antibiotic administered, and 195 for arrival to antibiotic administered. Sixtyone percent of patients received antibiotic within 4 hours. We estimate that for every 10 additional ED patients the time from arrival to ordering of a chest radiograph was prolonged by 14.3 minutes and from ordering of antibiotic to administration by 9.3 minutes.

Despite compliance with mandated ratios of nurses to patients, the time from antibiotic ordering to administration (a nursing task) was prolonged with higher ED volumes, as were throughput measures. Targeting these may expedite treatment under crowded ED conditions.

	Multivariate analyses associated increased emergency department (ED) volume with delayed nursing-dependent care for patients with pneumonia (time from antibiotic  ordering to administration), despite compliance with required ratios of nurses to patients.

		In our academic ED with strict ratios of nurses to patients, the time from antibiotic ordering to administration (which reflects ED nursing workload) was prolonged in association with increased ED volume.

		If these findings are reproduced in other settings, EDs may want to focus on nursing processes to find ways to maintain performance during periods of high ED volume, in addition to looking at measures to decrease ED crowding.

	Emergency department (ED) crowding delays the time to first antibiotic dose for patients admitted with communityacquired pneumonia.

 Pneumonia was one of the initial areas identified by The Joint Commission (TJC) and the Centers for Medicaid and Medicare Services (CMS) as a hospital core measure for quality care. In 2002, TJC and CMS established a target of time to first antibiotic dose within 4 hours of hospital arrival (changed to 6 hours in April 2007) for this publicly reported benchmark and have indicated that hospital and individual physician funding may someday depend on performance on this and other core measures.

 Hospitals and physicians attempting to improve performance need to direct limited resources to aspects of care requiring improvement, without sacrificing care provided to others or disrupting processes that work well. It is likely that determining which   admission care components are most affected by increasing volume will provide insight into areas for improvement for the care of all ED patients, not just those with pneumonia.

	Emergency departments are closely scrutinized for their ability to meet the target to first antibiotic dose, yet the effect of ED volume is just beginning to be recognized.

 If a connection between ED volume and particular care components can be established, specific processes could be targeted for quality improvement efforts, without requiring a large influx of resources. This is critical as EDs become increasingly responsible for more time-sensitive interventions (ie, those for patients with trauma, sepsis, stroke, and acute coronary syndrome) and as healthcare dollars shrink. The objective of this investigation was to determine care components provided to patients with pneumonia who were admitted through the ED that were most affected by increasing ED patient volume.

	This is a secondary analysis of a cross-sectional observational survey of patients admitted through a university tertiary care hospital ED and discharged between January 1, 2004, and June 30, 2005, after an inpatient hospitalization for pneumonia.

 We determined the effect of concurrent ED volume on the duration of care components provided to patients admitted with pneumonia. The study was approved by the committee on human research at our institution.

	Our 29-bed academic ED has an annual census of 39,000 and is staffed by emergency medicine, internal medicine, pediatric, and psychiatry residents, as well as nurse  practitioners and physician assistants. Patients are seen according to triage acuity. Patient care is supervised 24 hours a day by board-eligible and board-certified emergency physicians. Our ED adheres to a strict ratio of nurses to patients (1:4) as mandated by state law. Dedicated ED radiology technicians are present 24 hours per day. Emergency department triage and physician notes are electronically documented (File-Maker Pro, version 7; FileMaker, Inc, Santa Clara, California). Emergency department orders and nursing notes are handwritten. Physicians and clerical staff notify ED nurses of pending orders by posting an icon on the electronic patient tracking board (GE Centricity; GE Healthcare UK Ltd, Buckinghamshire, England) or by calling with portable phones. These handwritten orders and nursing notes are electronically scanned and stored after the ED visit and are accessible for quality assurance and research.

	As part of ongoing quality assurance and TJC/CMS core measure reporting at our institution, an outside vendor (University HealthSystem Consortium, Oak Brook, Illinois) (UHC) reviews medical records of inpatients eligible for TJC/ CMS core measure PN-5b4 (as detailed in a prior study1). The UHC reviewed medical records of all patients meeting

	the eligibility criteria from January 1, 2004, through December 31, 2004, in accord with TJC/CMS reporting requirements. Thereafter, the TJC/CMS reporting requirements were revised, and UHC selected a random sample of 75 patients per calendar quarter using a computerized random number generator (SAS version 9.1; SAS Institute Inc, Cary, North Carolina). The UHC excluded patients in accord with TJC/CMS core measure PN-5b exclusion criteria

). Medical records of all remaining patients were reviewed by UHC for antibiotic timing, and the results were reported to our institution. From this group of patients, we selected those admitted through the ED. Analyses were restricted to patients with available data on the time of antibiotic administration.

	Collected from administrative databases were patient demographic and presenting characteristics (age, sex, race/ ethnicity, and mode of arrival [self or ambulance]), triage acuity (1-4, where 1 is emergent), and level of care to which the patient was admitted (intensive care unit or not).

 Using a structured data collection form, we reviewed each patient’s ED medical record to obtain the following: date and time of ED presentation (from the ED triage notes), time of room placement (from the ED nursing notes), time when a chest radiograph was ordered (from the ED orders) and obtained (from the radiology records), time when a blood culture was ordered (from the ED orders) and obtained (from the ED  nursing notes), and time when an antibiotic was ordered (from the ED orders) and administered (from the ED nursing notes). For patients with missing or illegible scanned data, we made 3 attempts to obtain the archived paper medical record before reporting the data as unavailable. All data abstraction was performed by 2 of us (CF and CAM), who were not blinded to the study hypothesis but were blinded to ED volume data.

	An ED database permits calculation of the hourly ED census on any prior date. From this database, we determined the total number of ED patients who were present at the time of arrival of each patient with pneumonia.

	All data were entered into an electronic spreadsheet. Microsoft Excel 97 (Microsoft Corporation, Redmond, Washington) was used.

	We calculated the duration of the following care components: ED arrival to ED room placement, ED arrival to chest radiograph order, chest radiograph order to radiograph obtained, blood culture order to culture obtained, room placement to antibiotic order, and antibiotic order to antibiotic administered. We classified care components as related to system  hroughput (arrival to room placement and arrival to chest radiograph order) or by provider discipline as follows: radiology technician (chest radiograph order to radiograph obtained), physician (room placement to antibiotic order), and nurse (blood culture order to culture obtained and antibiotic order to antibiotic administered) (

). We chose not to calculate time from room placement to chest radiograph order (a potential marker of physician workload), as chest radiographs may be ordered by triage and bedside nurses and physicians at our institution by preestablished protocols; therefore, this care component does not solely reflect physician or nursing workload. Similarly, we did not include time from room placement to blood culture order because nurses often obtain (but do not send) blood cultures before the physician sees the patient and writes this order.

	Intervals longer than 600 minutes were analyzed as equal to 600 minutes to limit their influence on the overall results, while still including them as very long intervals. This value was chosen because it was the mean total length of stay of admitted ED patients. If the task was performed before the time of order, we set order times to equal the time of order completion to eliminate negative intervals.

	The primary outcome was the time from antibiotic order to antibiotic administered (a nursing task). We chose this a priori based on casual observation that nursing staff seemed busier than physician staff during times of ED crowding. Secondary outcomes included times to completion of each care component.

	We assessed the association of total ED volume with the duration of each care component using multivariate linear regression analysis to control for demographic and presenting characteristics. We also controlled for the time from arrival to the start of the care component to control for the possibility that subsequent intervals would be affected by the previous  intervals. For example, if providers were cognizant of delays preceding their involvement, those providers may have behaved differently, either to expedite care intervals to maintain compliance with the antibiotic timing measure or to direct care to sicker patients, having decided that the target could not be met. Patients who had missing values were excluded from the analysis.

	Analyses were conducted using commercially available statistical software (SAS version 9.1; SAS Institute, Cary, North Carolina). We used linear regression analysis because results are most meaningfully and interpretably estimated as effects on the mean duration of the measured care component. Because many care components had skewed distributions that resulted in violation of the normality assumption for residuals from the models, we used bias-corrected accelerated bootstrapping to obtain valid confidence intervals (CIs).

 We tested the linearity assumption for continuous predictors by adding quadratic terms. For predictors with strong evidence of nonlinearity, we categorized them into quartiles.

	A total of 731 of 898 patients discharged from our hospital with a primary or secondary diagnosis of pneumonia during the study period met eligibility criteria for measure PN-5b and were chosen by UHC for review (

). Of this total, 245 (33%) met exclusion criteria. For 79 patients, we could not ascertain the time of antibiotic administration, leaving 407 patients for the final analysis. The initial study

 for which these data were abstracted consisted of 405 patients. Further investigation resulted in 2 additional patients with known time to antibiotic administration. Demographic and presenting characteristics of the study sample are given in Table 1. The median number of ED patients present at the time of arrival of the patients with pneumonia was 24.5 (interquartile range, 17-30; range, 3-52).

Multivariate analyses associate increased emergency department volume with delayed nursing-dependent care for patients with pneumonia, despite high ratios of nurses to patients.

April 2011

Electronic Medical Records and Efficiency and Productivity During Office Visits

Community-Based Health Information Technology Alliances: Potential Predictors of Early Sustainability

Relevance of Current Guidelines for Organizing an Anticoagulation Clinic

Effect of Emergency Department Crowding on Pneumonia Admission Care Components

Primary Care and Communication in Shared Cancer Care: A Qualitative Study

Long-Acting Beta-Agonist Monotherapy Among Children and Adults With Asthma

A Multimodal Blood Pressure Control Intervention in 3 Healthcare Systems

Consumer Attitudes Toward Personal Health Records in a Beacon Community

Electronic Health Record Functions Differ Between Best and Worst Hospitals

Drug Testing, Chronic Pain, and Financial Conflicts of Interest