Using Health Outcomes to Validate Access Quality Measures

Medicare payment reforms require valid measures of high-quality healthcare. Different types of administrative wait time measures predicted glycated hemoglobin levels for new and returning patients.
Published Online: November 18, 2013
Julia C. Prentice, PhD; Michael L. Davies, MD; and Steven D. Pizer, PhD
Background: Medicare payment reforms will reimburse accountable care organizations (ACOs) for providing high-quality healthcare. Quality measures that reliably predict health outcomes are required.

Objectives: To compare the ability of alternative wait time measures to predict glycated hemoglobin (A1C) levels among diabetes patients.

Study Design: This retrospective observational study relied on Veterans Health Administration utilization data and Medicare claims data from 2005 to 2010.

Methods: Outcomes included an average A1C level and uncontrolled A1C. Heckman selection models simultaneously predicted the presence of an A1C value and its level. Models were riskadjustedfor prior individual health status. The main explanatory variables of interest were facility-level primary care wait times measured in days. Several measures were tested, including capacity measures and retrospective and prospective time stamp measures. The time stamp measures used either the date the appointment was created in the scheduling system (create date) or the date the patient or provider desired the appointment (desired date) as the start date for wait time computation. All measures were calculated separately for new and returning patients.

Results: New patient capacity and create date measures significantly predicted outcomes, but desired date measures did not. The returning patient retrospective create date and desired date wait  time measures significantly predicted higher and uncontrolled A1C, but capacity and prospective create date measures did not.

Conclusions: Different administrative wait times predict A1C for new and returning patients. To properly measure quality, ACOs should use wait time measures that demonstrate relationships with outcomes for subpopulations of patients. 

Am J Manag Care. 2013;19(11):e367-e377
Medicare payment reforms require measures that can reliably predict outcomes. This study compares the ability of alternative wait time measures to predict glycated hemoglobin (A1C) levels among diabetes patients.
  • New patient capacity and date when appointment was created in the scheduling system (create date) measures significantly predicted outcomes, but desired date measures did not.

  • Returning patient retrospective create date and desired date wait time measures significantly predicted A1C, but capacity and prospective create date measures did not.

  • Accountable care organizations should use access measures that demonstrate relationships with outcomes for patient subpopulations.
A crucial piece of the Affordable Care Act legislation is the implementation of accountable care organizations (ACOs) that financially reward a group of providers for providing highquality healthcare at low cost.1,2 ACOs may provide the greatest benefits to patients diagnosed with chronic disease due to their focus on providing high-quality, well-coordinated, and patient-centered care.1,2 Patients diagnosed with diabetes are likely to be targeted for disease  management by ACO managers because of the high prevalence and  significant monetary costs of managing diabetes complications. Approximately 17.5 million individuals in the United States have diabetes, and the total cost related to diabetes in 2007 was $174 billion.3

The success of ACOs relies on robust quality measurement.1,2 Policy makers have emphasized the need to complement process quality measures (eg, the percentage of patients with diabetes who receive glycated hemoglobin [A1C] tests) with patient-centered measures that demonstrate improved patient outcomes.2,4,5 Timely access to healthcare has a strong potential to meet this criterion.

Previous research suggests that access measures that do not rely on patient self-report reliably predict health outcomes. Studies using capacity wait time measures (eg, time to first next available appointment [FNA]) and comparing populations exposed to open access scheduling with controls have found patients who face shorter waits have higher primary care utilization, lower A1C levels, and a decreased risk of experiencing poor health outcomes such as mortality or preventable hospitalizations.6-10 Notably, in the context of ACOs, Prentice and colleagues6,7 found the effects of longer FNA waits were largest with the most vulnerable veterans—those who were older, had more comorbidities, or had higher A1C levels during baseline.

Adherence may be one factor that contributes to the observed relationship between wait times and outcomes. Several studies indicate that patients diagnosed with diabetes who are more adherent to their treatment plan experience better health outcomes and have lower healthcare cost.3,11,12 Cramer and colleagues13 found a “white coat” effect, with adherence to medication increasing right before and right after physician visits. Consequently, longer waits between visits may decrease a patient’s ability to get the most effective treatment and lead to lower overall medication adherence. Patients with diabetes often underestimate the importance of consistent treatment adherence,3,14 and longer waits between appointments will decrease opportunities for patient education.

Although the importance of timely access to healthcare is widely recognized, the best method of measuring timely access has yet to be determined. The Centers for Medicare & Medicaid Services are currently using patient surveys to evaluate ACOs’ ability to deliver healthcare as soon as it is desired.15 An alternative approach relies on administrative data from patient scheduling systems. For example, capacity measures (eg, the number of days until the first or third next available appointments) are commonly used when implementing Advanced Clinic Access.16,17 The Veterans Health Administration (VHA) found that capacity measures may not reflect the access limitations actually experienced, especially for returning patients who are typically trying to schedule follow-up appointments.18 Consequently, a variety of wait time measures have been developed. This study is the first to compare the abilities of these alternative measures of wait times to predict glycemic control among patients diagnosed with diabetes. Results could be used in Medicare and the private sector to improve upon the current quality metrics used by ACOs.


Study Population

This study used administrative data from a wide variety of VHA and Medicare data sets. Please refer to Appendix A for an overview of all the data used. Using the VHA Pharmacy Benefits Management file, we chose all individuals who were prescribed a glucose-lowering medication in 2005 or 2006. These years were used as the baseline year for risk adjustment. Outcomes were measured starting in the year following baseline (2006 or 2007). American Diabetes Association guidelines at the time of the study called for A1C levels to be tested biannually for individuals exhibiting glycemic control and quarterly for individuals not in control.19-21 We split the outcome period into ten 6-month observation periods starting with January to June 2006 and ending with July to December 2010.

To help ensure that we had complete claims data for baseline risk adjustment, we required the sample to be enrolled in Medicare fee-for-service plans. We excluded individuals who were enrolled in a Medicare health maintenance organization; veterans with higher priority levels (7 or 8), who might have had private insurance claims we could not access. Other exclusions included individuals who died during the outcome period, because A1C levels could have changed right before death; veterans in a Department of Veterans Affairs (VA) nursing home during the baseline period, who might not have relied on VA outpatient care; individuals with missing A1C levels during the baseline period; and individuals with missing race data. The final sample size was 195,842 people.

Types of Wait Time Measures

We obtained 5 distinct wait time measures from VHA scheduling system records from 2006 through 2010: (1) capacity (FNA); (2) retrospective create date (CD); (3) retrospective desired date (DD); (4) prospective CD; and (5) prospective DD. Table 1 describes how each of these measures was calculated, and Prentice and colleagues18 provide a detailed overview of the measures.

Briefly, the FNA uses the day an appointment is created as the starting point and measures the time between that day and the day the first available open appointment slot occurs. Individual patients may not actually want the FNA appointment because they are looking for a follow-up appointment in the future. This is likely to be common for returning patients who typically wish to schedule a follow-up. New patients are more likely to want to be seen as soon as possible.22

To overcome this limitation, the VHA developed time stamp measures that measured how long individual patients waited. Time stamp measures can use a CD or a DD as the date to start measuring waits (Table 1). The CD is the date that an appointment is created (ie, made) in the appointment system. The principal limitation of CD is that it measures the pattern of booking appointments. For example, suppose a patient comes in for a checkup and the patient and provider agree to schedule a follow-up appointment in 6 months. If the clinic creates the follow-up appointment on the day of the initial appointment (“on today”), the resulting measured wait time will be 6 months. Alternatively, the clinic might contact the veteran 5 months from “today” (1 month before the intended 6-month follow-up appointment) and create the intended 6-month follow-up appointment at that time, resulting in a measured wait time of 1 month. This limitation is surmounted through the DD time stamp measure that designates the ideal time “a patient or provider wants the patient to be seen.”23-25 In this example, the DD is the date of the 6-month follow-up appointment that the patient and provider agreed upon.

In addition to different start dates for CD or DD, time stamp measures can have different ending points. One ending point is the day an appointment is completed, resulting in a retrospective time stamp measure (completed appointment— CD or DD). A second ending point is a bimonthly snapshot of all pending appointments in the VHA, resulting in a prospective measure (pending appointment date—CD or DD).

Facility-Level Wait Time Measures Compared With Individual Wait Times

When computing any of these measures for use in outcomes models, it is tempting to calculate a wait time measure based on services an individual actually used. This approach is problematic because unobserved individual health status affects individual wait times as well as individual outcomes due to medical triage. Medical providers identify patients who are in poorer health when they call to request an appointment and refer these patients to clinics with shorter waits. Thus, individual health status is affecting individual wait times and potentially obscuring the effect of wait times on health status (for an example, see Prentice and Pizer9,10). Although statistical controls for observable differences in health status will reduce the severity of this problem, we are not able to measure health status precisely enough to eliminate it.6,7,9,10,26

To avoid this problem and isolate the effect of wait times on health, we computed facility-level averages for each wait time measure based on a fixed pattern of clinic utilization.6,7,9,10 Averages were calculated separately for new and returning patients. Missing wait times were imputed with zero when appropriate.6,7,9,10,26

Individual-Level Explanatory Variables for Risk Adjustment

PDF is available on the last page.
Recommended Articles
This week GOP presidential candidate hopefuls turned their attention to the 2016 election as the next best chance to repeal Obamacare, and CMS released data revealing $6.5 billion payments to healthcare providers from drug and medical device makers in 2014.
Dana Goldman, PhD, suggests that value for breakthrough therapies is most often associated with health benefits, health economics, and outcomes research; however, there are other sources of value that have not been taken into account yet.
The deal combines Humana's 3.2 million Medicare enrollees with Aetna's 1.26 million Medicare enrollees, giving the new combined company a strong position as the baby boomer population ages.
As co-pays and deductibles in Medicare and commercial health plans become more prevalent, so, too, does the temptation to waive them. But beware.
In addition to increasing insurance coverage, the Affordable Care Act also aims to improve population health and lower healthcare costs. However, not much attention has been paid to the quality of care the newly insured are receiving.