Evaluation of Electronic Medical Record Administrative Data Linked Database (EMRALD)

The combination of electronic medical record data and administrative data provides the fullest picture of patient health histories.
Published Online: January 14, 2014
Karen Tu, MD, MSc; Tezeta F. Mitiku, BSc, MSc; Noah M. Ivers, MD; Helen Guo, BSc, MSc; Hong Lu, PhD; Liisa Jaakkimainen, MD, MSc; Doug G. Kavanagh, BSEng, MD; Douglas S. Lee, MD, PhD; and Jack V. Tu, MD, PhD
Background: Primary care electronic medical records (EMRs) represent a potentially rich source of information for research and evaluation.

Objective: To assess the completeness of primary care EMR data compared with administrative data.

Study Design: Retrospective comparison of provincial health-related administrative databases and patient records for more than 50,000 patients of 54 physicians in 15 geographically distinct clinics in Ontario, Canada, contained in the Electronic Medical Record Administrative data Linked Database (EMRALD).

Methods: Physician billings, laboratory tests, medications, specialist consultation letters, and hospital discharges captured in EMRALD were compared with health-related administrative data in a universal access healthcare system.

Results: The mean (standard deviation [SD]) percentage of clinic primary care outpatient visits captured in EMRALD compared with administrative data was 94.4% (4.88%). Consultation letters from specialists for first consultations and for hospital discharges were captured at a mean (SD) rate of 72.7% (7.98%) and 58.5% (15.24%), respectively, within 30 days of the occurrence. The mean (SD)  capture within EMRALD of the most common laboratory tests billed and the most common drugs dispensed was 67.3% (21.46%) and 68.2% (8.32%), respectively, for all clinics.

Conclusions: We found reasonable capture of information within the EMR compared with administrative data, with the advantage in the EMR of having actual laboratory results, prescriptions for patients of all ages, and detailed clinical information. However, the combination of complete EMR records and administrative data is needed to provide a full comprehensive picture of patient health histories and processes, and outcomes of care.

Am J Manag Care. 2014;20(1):e15-e21
This study examined the comprehensiveness of primary care electronic medical record (EMR) data compared with administrative data.
  • Primary care EMRs have reasonably comprehensive capture of primary care physician visits, laboratory tests, and prescriptions by the EMR physician.

  • Laboratory tests and prescriptions by other providers may be missing; there is incomplete communication with specialists, and particularly poor communication between EMRs and hospitals.

  • Until integrated systems with open communication channels between all aspectsof the healthcare system are in place, the most comprehensive patient health histories can be achieved by combining information from complete EMR records and administrative data.
Although the United States and Canada have historically lagged behind other industrialized countries in the adoption of electronic medical records (EMRs) in primary care,1 with the introduction of the Health Information Technology for Economic and Clinical Health Act in 2009 in the United States2 and the establishment of Canada Health Infoway3 and provincial EMR adoption support programs in Canada, the uptake of EMRs in both countries is rapidly increasing.4 This development, especially in primary care physician practices, has resulted in a new, potentially rich source of clinical information not only for point-of-care clinical practice but also for secondary purposes such as research and quality performance evaluation.

Because of its single-payer healthcare system, Canada has comprehensive health-related administrative databases that cover the entire population. Complete, provincewide, population-level administrative databases have been shown to be highly accurate in capturing hospitalizations and prescriptions for Ontario residents 65 years and older.5 Also in these databases, physician billing data accurately capture frequency of patient encounters, but the depth and details of patient clinical encounters are unavailable. Indeed, a previous study in the United States found that EMR data in community health centers were more complete than Medicaid claims data for assessing diabetes preventive care.6

Because use of primary care EMR data in Canada for secondary purposes is in its relative infancy, we set out to determine the completeness and comprehensiveness of the EMR data compared with administrative data.


We developed the Electronic Medical Record Administrative data Linked Database (EMRALD) at the Institute for Clinical Evaluative Sciences (ICES). The Institute for Clinical Evaluative Sciences is a “prescribed entity” under provincial privacy legislation and thus is able to collect individual-level patient health information without patient consent based on policies and procedures in place to protect patient privacy and confidentiality.7 EMRALD was developed using data from family physicians (FPs) in Ontario using Practice Solutions EMR, the market-leading EMR software vendor in Ontario.8 All clinically relevant data are extracted from the participating physician’s EMR and each patient is anonymously linked, using their scrambled health card number, to the health-related administrative databases for the province of Ontario housed at ICES.

Physicians participate on a completely voluntary basis and are required to have had their EMR a minimum of 2 years to ensure that the EMR is adequately populated. Over the past decade Ontario has undergone major primary care reform such that the majority of all FPs in the province practice under one of the reform models that require “rostering” of patients (identification of patients with the 1 primary care provider who is most responsible for their care).9 All of the EMRALD FPs practice under one of the various primary care reform models of care. This analysis was confined to data that were captured in 2008 for rostered patients of participating FPs who had at least 1 visit to their physician in the 2 years before January 1, 2008.

Family Physician Clinic Visits

To determine the capture of all FP visits by patients rostered to EMRALD physicians (regardless of whether the visit was to an EMRALD physician), FP outpatient visits billed to the Ontario Health Insurance Plan (OHIP) physician billing database were compared with visits recorded in the EMR to determine the capture of FP visits by patients rostered to EMRALD physicians. Visit comparison between OHIP and EMRALD was confined to EMRALD physician OHIP visits only, to assess how completely EMRALD physicians were documenting patient visits within their EMR.

Specialist Consultations and Hospitalizations

To determine the capture of specialist visits and hospitalizations within EMRALD, OHIP billings by specialists performed in an office setting for an initial consultation and hospitalizations, as recorded in the Canadian Institute for Health Information Discharge Abstract Database, were compared with consultation letters and hospital discharge summaries in EMRALD within 14 days and within 30 days of the specialist billing date or the hospital discharge date.

Laboratory Data

A frequency count of OHIP laboratory fee codes identified the 20 most common laboratory tests ordered. Comparisons were made between the laboratory tests recorded on the same day in EMRALD and in OHIP.

Prescription Data

The EMR records when a drug is prescribed to the patient by the EMR physician, and the Ontario Drug Database (ODB) records when a drug was dispensed at a pharmacy (regardless of the prescriber). The EMR may also list drugs prescribed to a patient by other providers (ie, specialists), but only if such data are manually updated by the EMR-using physician; thus, the EMR is variably populated with these data. The prescription field within EMRALD and the prescriptions dispensed in ODB in 2008 were used to compile 2 drugs lists for the top 50 dispensed Canadian medications,10 grouped into 7 clinical categories. The 2 lists for each patient were compared. This analysis was confined to patients 65 years and older as of January 1, 2008, as the ODB only captures drugs universally for all Ontario residents 65 years and older.

Match Rates

Match rates were calculated for all parameters of comparison by calculating the mean match rate and standard deviation (SD) for all the clinics, giving each clinic equal weighting regardless of size.

This study received ethics approval from Sunnybrook Health Sciences Research Ethics Board.


Overall, there were 56,107 patients with an average age of 39.7 years from 54 physicians practicing in 15 geographically distinct clinics. The mean years since graduation of the physicians was 19.3 years (SD = 10.4 years). Seventy percent of the physicians were in urban practice, 56% were male, and 98% were in group practice. The average duration of time on the EMR was 4.5 years (SD = 2.6 years).

Family Physician Visits

Nearly 80% of the FP billings in OHIP that occurred in an office setting were captured in EMRALD. To determine whether the remaining billings were missing because patients were seen elsewhere (eg, a walk-in clinic) and therefore their visits were not recorded in the EMR, we limited the comparison to OHIP billings made by EMRALD physicians. In that case, we found that nearly all the billings for patients had a corresponding progress note entry in the EMR. Thus, we concluded that the approximately 15% of billings missing from the EMR were because patients were seen elsewhere and that the EMRALD participating physicians were fully using the EMR to record their patient encounters (Table 1).

Specialist Clinical Encounters and Hospitalizations

On average, just over two-thirds of initial specialist visits resulted in a consultation letter captured in EMRALD within 14 days of the visit; this percentage increased by less than 5% when the interval was expanded to 30 days (Table 1). Just over half of the hospital discharges had documentation of the hospitalization within EMRALD (Table 1).

Laboratory Tests and Prescriptions

We found that on average approximately three-fourths of the laboratory tests captured in EMRALD were also billed in OHIP, and most clinics had a higher match rate. When the laboratory tests recorded in OHIP were compared with what was recorded in EMRALD, approximately two-thirds of the tests billed to OHIP were captured (Table 1). Urinalysis tests had the poorest capture of all the laboratory tests in OHIP compared with EMRALD (Table 2).

Capture of prescriptions in EMRALD compared with drugs dispensed in ODB was high. However, only just over two-thirds of the drugs dispensed in ODB were captured in EMRALD (Table 1). Antibiotics (as represented by amoxicillin) were the drug class that had the poorest capture rate.  However, the capture rates for drugs to treat chronic conditions such as cardiovascular disease and endocrine diseases appeared to be relatively high (Table 3).


We found that primary care EMR data capture compared well with administrative data. Like previous US studies comparing EMR data with administrative data,6,11 our results support the idea that primary care process quality measures involving laboratory test ordering or prescriptions are better assessed through the EMR than through administrative data because the EMR is more likely to reflect what the primary care physician ordered or prescribed.

PDF is available on the last page.
Clinical Pathways Compendium
COPD Compendium
Diabetes Compendium
GI Compendium
Lipids Compendium
MACRA Compendium
Oncology Compendium
Rare Disease Compendium
Reimbursement Compendium
Know Your News
HF Compendium