Zachary Gelber, DDS

Osteoarthritis of the knee is a chronic disease associated with pain and reduced quality of life. The ability to reliably measure patient-reported symptoms is important for clinical decision making and evaluation of outcomes. Electronic and web-based tools can eliminate much of the labor-intensive aspects of questionnaire administration and enables both real-time evaluation of responses by physicians and integration of data from multiple sites. This article describes the results of implementing a single integrated electronic questionnaire system into routine orthopedic practice at 2 diverse institutions.

A web-based version of a general quality-of-life questionnaire (EuroQol 5-dimension [EQ-5D]) and the pain domain of a disease-specific questionnaire (Knee Osteoarthritis Outcome Score [KOOS]) were administered in the office waiting room to (n = 666) patients at 2 centers over a 9-month period using touchscreen devices. Data were analyzed and descriptive statistics were calculated to assess feasibility of integration into the distinct work flows and to assess the agreement of the results.  

 The electronic questionnaire had a completion rate of 93% to 95%. Average questionnaire completion times were 3 to 5 minutes at each institution. Mean EQ-5D and KOOS scores for patients pre- and postsurgery were also consistent with prior literature studies.

 Lessons learned for future adoption of questionnaire systems elsewhere include the need for baseline assessment of clinic work flows to identify the optimal point of administration and the need for IT support. This study demonstrates the feasibility of routinely collecting patient-reported data as part of standard care, which will become increasingly important as the nationwide emphasis on tracking quality and cost-effectiveness of treatments in orthopedics grows.

	Patient-reported outcomes (PROs), through the use of new technologic advances, can be successfully integrated into routine orthopedic practice and networked across distinct institutions.

		PRO data can be collected electronically and scored instantaneously, allowing for real-time feedback to providers.

		PRO data can be used to track patient progress over time and for quality assurance purposes.

		Important lessons learned include the need for baseline assessment of clinic work flows to identify the optimal point of administration and need for IT support.

		PRO collection will become increasingly important as the nationwide emphasis on tracking quality and cost-effectiveness of treatments in orthopedics grows.

Osteoarthritis (OA) is a chronic, disabling disease associated with aging, and it is the predominant reason that most total joint replacements are performed.

 The ability to reliably measure and track pain symptoms and quality of life are critical for measuring the success of various therapeutic strategies used to manage OA symptoms, and can also help inform decisions concerning the appropriate timing of surgery.

 Along with providing clinicians with an additional tool to guide treatment decisions, patient-reported outcome (PRO) data can also be used to compare the outcome associated with new techniques or procedures and to benchmark best practices, enabling ongoing quality improvement and identifying areas needing additional clinical research.

 Consequently, routine clinical practice can strongly benefit from the capture of PRO data to monitor the outcome of care for patients with OA. However, the infrastructure needed to achieve this goal is generally lacking, and there is an ongoing perception that collecting PRO data will disrupt practice flow and increase workload.

Numerous challenges and obstacles confront institutions attempting to incorporate PRO data collection into routine practice. Traditionally, patient data capture has relied on the distribution of paper questionnaires that are filled out by patients and stored in electronic format—a time-intensive process fraught with risk of information loss or error. Moreover, paper-based systems do not allow for scoring in real time, making it difficult to incorporate the information into the immediate patient—physician encounter. Technology solutions can improve upon the paper-based questionnaire process: specifically, touch-screen technology offers the potential to facilitate collection of data, save time on administration, scoring, and data entry, and increase utility by allowing physicians immediate access to results. In addition, appropriate data security controls allow for secure PRO data collection and transmission across institutions for multi-site research studies. Although previous studies have shown high-quality agreement between data collected from paper-based and electronic-based instruments and demonstrated patients’ willingness to use electronic tools for data collection,

 more work is needed to demonstrate the usability of such tools as part of routine orthopedic practice.

This article describes the results and experience of a pilot study to incorporate and integrate an electronic PRO data capture system into routine orthopedic practice at 2 large, geographically separated and distinct orthopedic centers. A unified data capture system was developed and implemented, and our objective was to evaluate the feasibility of routine coordinated PRO collection between these 2 institutions and to describe the challenges and innovations that made routine collection possible.

We conducted a prospective cohort study to evaluate the implementation of an electronic PRO process into routine orthopedic practice at 2 institutions. A secondary objective was to compare the patient-reported data with previously reported values to validate the use of the data for research.

The study was conducted in the orthopedic practices of 2 large academic medical centers: 1 urban (referred to as Center A) and 1 rural (referred to as Center B). The inclusion criteria included patients be aged at least 18 years with a knee-related complaint, seeing any of 5 participating orthopedic surgeons at Center A or 2 participating orthopedic surgeons at Center B, between August 30, 2010, and June 27, 2011. Patients were required to be able to read English or have a translator present. The institutional review board approved this study at both institutions.

Two PRO instruments were chosen, including 1 general quality of life instrument and 1 disease-specific instrument. The 5-item EuroQol (EQ-5D [3L]) was chosen as the general preference-based quality-of-life instrument because it can be used for economic evaluations such as cost per quality-adjusted life-year (QALY) and comparative effectiveness research.

 The EQ-5D index is scored on a scale of 0 to 1, with 1 being the best score, and includes 1 additional visual analogue-style question (EQ-VAS) asking the patient to rate their quality of life on a scale from 0 to 100, with 100 being the best score. The pain subscale of the Knee Osteoarthritis Outcome Score (KOOS) was chosen as the disease-specific instrument. This instrument uses the pain questions adapted from the widely used Western Ontario and McMaster Universities Arthritis Index (WOMAC) and is validated for both arthroplasty and non-arthroplasty patients.

 The KOOS pain index is scored from 0 to 100, with 100 as the best score indicating “no pain.”

These 2 questionnaires were combined into a 16-question instrument and converted into an electronic, Web-based touchscreen version using commercial software (DatStat Illume, Seattle, Washington). The questionnaire was hosted on a Web server at Center B, and a user management system was set up to restrict access to study personnel only. Scoring of the PRO instruments was performed instantly in real time by the questionnaire software, and scores along with individual question responses, patient identifiers, and completion times were saved to an SQL database. A separate, Web-based visual interface (VI) was also developed to electronically view the patient’s scores. In addition to summary scores, the VI allowed physicians to view responses to individual questions, trends over multiple visits, and normative literature-based ranges of values in order to enhance the physician’s understanding of the patient’s condition and any change from prior visits. This individual patient information was also viewable by the physician in real time, enabling physicians to use the PRO data during the visit at which the patient had completed the questionnaires.

The questionnaire completion work flow differed at each institution (

), so each site’s work flow was studied to determine how to integrate uniform PRO collection with minimal disruption to existing practice flow. At Center A, patients used a touchscreen tablet (iPad) in the waiting room to complete the questionnaires. This electronic version was configured so that there was only 1 question per screen with a large font size (the subject could also zoom in if needed). After registration at the front desk, research assistants obtained informed consent from each patient (described below) and distributed the tablets. Once completed, the research assistant collected the tablet devices, accessed the patient scores via the web application, printed the score summary, and placed this summary into the medical chart for the physician to review during the exam. In addition, the PRO score summary was also scanned to the patient’s electronic health record (EHR) chart after the visit. At Center B, patients used a PC with an LCD touch-screen monitor in the exam room to complete the questionnaires. The clinic nurse responsible for escorting the patient to the exam room opened the questionnaire via an embedded link in the patient’s EHR chart, and instructed the patient on how to use the touchscreen before leaving them to wait for the physician. After patients completed the questionnaire, the physician would access the VI using another embedded link in the EHR chart in the exam room and view the scores with the patient. Although the VI display of scores was not a part of the EHR itself, a copy-and-paste function allowed the physician to easily copy the scores from the VI into the EHR’s visit note without additional typing.

At Center A, because the electronic questionnaire responses were being transmitted to a server at GMC, patients in the study were required to sign an institutional review board-approved informed consent form. All information transmitted between institutions was HIPAA compliant as unique study identification numbers were used in place of any personally identifying information. At Center B, no data was being transmitted outside the institution, and completion of the questionnaires was considered a standard practice of care for the participating physicians.

The statistical analysis for this study was descriptive. In addition to the patient scores on the EQ-5D Index, EQ-VAS, and KOOS pain subscale, we also analyzed the time needed to obtain patients’ informed consent (at Center A) and the time needed for patients to complete the PRO instruments. For the subset of patients who had multiple visits during the study period, we also compared the completion times between the first and second visit to see whether patients were able to reduce their completion times after becoming familiar with the questionnaire. Lastly, for the subset of patients who had completed a questionnaire both before and after total knee replacement (TKR) surgery, we analyzed the change in scores within this patient subgroup.

A total of 519 questionnaires were completed by 410 unique patients at Center A during the study period. An additional 31 patients that met the inclusion criteria did not complete the questionnaire, giving an overall completion rate of 93% (as shown in 

). Females composed 64% (263) of the patients, and 34% (143) of patients were male; 2% did not disclosure their gender

The median age was 61 years (range = 18-96). Patients took an average of 3 minutes and 17 seconds (range = 1:05-18:42) to grant informed consent. Once consent was obtained, it took an average of 3 minutes and 19 seconds (range = 0:55-15:00) for patients to complete the questionnaire using the iPad. Eighty-two patients completed the PRO questionnaire on multiple visits, and in this subset of patients, the mean time to complete the questionnaire decreased by 39 seconds, from 3 minutes and 37 seconds at the first visit to 2 minutes and 58 seconds at the second visit. For preoperative visits, the mean EQ-5D Index was 0.60 (SD = 0.20), the mean EQ-VAS score was 71 (SD = 18), and the mean KOOS pain score was 41.8 (SD = 20.2). For visits at 6 months or greater following TKR surgery, the mean EQ-5D Index increased to 0.74 (SD = 0.22), the mean EQ-VAS score increased to 73 (SD = 23), and the mean KOOS pain score increased to 70.4 (SD = 18.4).

A total of 316 questionnaires were completed by 256 unique patients at Center B during the study period. An additional 14 patients that met the inclusion criteria did not complete a questionnaire, and 8 of the 256 patients who did complete 1 questionnaire opted not to fill out additional questionnaires at a subsequent visit during the period. Therefore, the overall completion rates were 95% of patients and 93% of visits. Females composed 64% (162) of the patients, and 36% (85) were male; the median age was 62 years (range = 21-92). The median time to complete the questionnaire was 4 minutes (range = 1-24). Forty patients completed the PRO questionnaire on multiple visits, and in this subset of patients, the mean time to complete the questionnaire decreased by 18 seconds, from 4 minutes, 57 seconds at the first visit to 4 minutes, 39 seconds at the second visit. For preoperative visits, the mean EQ-5D score was 0.61 (SD = 0.25), the mean EQ-VAS score was 63 (SD = 21), and the mean KOOS pain score was 50.6 (SD = 22.8). For visits at 6 months or greater following TKR surgery, the mean EQ-5D Index increased to 0.76 (SD = 0.20), the mean EQ-VAS score increased to 69 (SD = 27), and the mean KOOS pain score increased to 68.9 (SD = 27.1).

In addition, we compared the scores obtained from this study with previously reported EQ-5D and KOOS pain scores in the literature (as shown in 

) to evaluate whether the data obtained were similar to that obtained from previous research studies, and found that scores at both institutions were well within the expected ranges. Brazier et al,

Articles

Patient-reported outcomes, through the use of new technological advances, can be successfully integrated into routine orthopedic practice and shared across distinct institutions. 

Zachary Gelber, DDS

Articles

Feasibility of Integrating Standardized Patient-Reported Outcomes in Orthopedic Care