Currently Viewing:
The American Journal of Managed Care December 2012
Trends in Viral Hepatitis Cost-Effectiveness Studies
Sammy Saab, MD; Young Choi, BS; Harman Rahal; Kun Li, MS; and Myron Tong, MD
Communication About Diabetes Risk Factors During Between-Visit Encounters
Courtney R. Lyles, PhD; Lou Grothaus, MS; Robert J. Reid, MD, PhD; Urmimala Sarkar, MD, MPH; and James D. Ralston, MD, MPH
Population-Based Breast Cancer Screening in a Primary Care Network
Steven J. Atlas, MD, MPH; Jeffrey M. Ashburner, MPH; Yuchiao Chang, PhD; William T. Lester, MD, MS; Michael J. Barry, MD; and Richard W. Grant, MD, MPH
Care Coordination in Accountable Care Organizations: Moving Beyond Structure and Incentives
Matthew J. Press, MD, MSc; Marilyn D. Michelow, MD; and Lucy H. MacPhail, PhD
Prioritized Post-Discharge Telephonic Outreach Reduces Hospital Readmissions for Select High-Risk Patients
L. Doug Melton, PhD, MPH; Charles Foreman, MD; Eileen Scott, RN; Matthew McGinnis, BS; and Michael Cousins, PhD
Impact of MMRV Combination Vaccine on Childhood Vaccination Compliance
Bradley K. Ackerson, MD; Lina S. Sy, MPH; Janis F. Yao, MS; Craig T. Cheetham, PharmD, MS; and Steven J. Jacobsen, MD, PhD
Impact of Education Program on Influenza Vaccination Rates in Spain
Bernardino Roca, PhD; Elena Herrero, PhD; Elena Resino, MD; Vilma Torres, MD; Maria Penades, MD; and Carlos Andreu, PhD
Personalized Preventive Care Leads to Significant Reductions in Hospital Utilization
Andrea Klemes, DO, FACE; Ralph E. Seligmann, MD; Lawrence Allen, MD; Michael A. Kubica, MBA, MS; Kimberly Warth, BS, MPA; and Bernard Kaminetsky, MD, FACP
Association Between Different Types of Social Support and Medication Adherence
Danielle Scheurer, MD, MSCR; Niteesh Choudhry, MD, PhD; Kellie A. Swanton, BA; Olga Matlin, PhD; and Will Shrank, MD, MSHS
Currently Reading
The Validity of Claims-Based Risk Estimation in Underinsured Populations
Janice M. Moore, MSW; and Jean P. Hall, PhD

The Validity of Claims-Based Risk Estimation in Underinsured Populations

Janice M. Moore, MSW; and Jean P. Hall, PhD
This article demonstrates a threat to validity when using claims-based risk tools with chronically ill and/or underinsured populations whose underutilization masks actual risk or comorbidity.
Health Status Measures. The primary instrument for measuring health status was the SF-12v2, a scale derived from the SF-36.17 We also administered the World Health Organization Quality of Life brief form (WHOQOL-BREF), the World Health Organization Health and Work Performance Questionnaires (HPQs), standardized ADL-IADL measures,18 and asked for self-reported diagnoses and health conditions. All instruments were administered by telephone at baseline and at 8-month intervals over the 41-month study period (5 rounds). SF-12v2 data are reported below for the first 4 rounds (baseline plus 3) corresponding with the first 2 years of the study. Claims data for case mix analysis are summarized for the corresponding 2-year period.

To measure comorbidity burden (case mix), we used the ACG, Version 8.2 with DX-PM model and prior cost for a non-elderly population.19 We did not include prescription data because a preliminary analysis indicated it did not significantly affect outcomes for this study population. The ACG maps each beneficiary’s age, gender, and diagnostic codes to a single actuarial cell, or ACG, representing their estimated resource use. ACG reference unscaled concurrent weights, or case-mix weights, are ratios comparing estimated resource use for a given ACG with the average resource use of a nationally representative sample of insured individuals. The average case-mix weight for the national population is set at 1. Thus, someone with a case-mix weight of 2 has an estimated resource use twice the national average. Although ACG case-mix weights are measures of estimated resource use, they are also proxy measures of disease burden. Generally, weights greater than 1.0 indicate a population is sicker than the national average, and weights of less than 1.0 indicate it is healthier.19

ACG software also produces other measures of disease burden, including Aggregated Diagnostic Groups (ADGs), Expanded Diagnostic Clusters (EDCs), counts

of major EDCs (chronic condition counts), and resource utilization bands (RUBs). Recent versions have also included a predictive model algorithm for forecasting cost.

Our hypothesis was that SF-12v2 PCS and MCS scores would stabilize or decrease more slowly for the intervention than the control group, while ACG case-mix weights would steadily decrease for the intervention group, reflecting reduced disease burden and improved health.


The data presented here were collected as part of a randomized controlled study and reflect the subsample continuously enrolled for the first 2 years of the study. For the entire study population, over 32 months, intervention group SF-12v2 PCS scores remained relatively stable but control group scores significantly declined.12 For this subsample, the same trend was evident, even though the between-group difference did not approach statistical significance by the end of year 2. However, the within-group difference showed that the control group was declining at a significantly greater rate (P = .01 control group, P = .58 intervention, repeated measures analysis of variance [ANOVA], Table 3). PCS scores did not differ significantly by economic status, as measured by family income as a percent of federal poverty level.

Previous research on the Medical Outcomes Study, from which the SF-12v2 is derived, found that the average agerelated decline for healthy populations 45 to 64 years of age is 0.4 points per year.20 As Table 3 shows, PCS scores for the intervention group fluctuated plus or minus 1 point while control group scores declined by 2.45 points over a 24-month period. Survey data also showed different self-reported health status. At study end, 34% of the intervention group indicated their health had improved compared with 21% of the control group; conversely, only 19% of the intervention group reported worsening health compared with 31% of controls; approximately equal proportions (46% intervention vs 48% control) said their health had remained the same (P = .01, Pearson χ2).

In contrast, the trend for ACG case mix moved inversely (Table 4). At baseline, the groups did not differ significantly on any ACG measure. Both had case-mix weights approximately 3 times the national average (2.97 intervention vs 3.59 control). Both groups had 1.1 major illnesses (ADG) and 2.6 chronic conditions. Average costs for the control group were slightly, but not significantly, higher ($11,124 vs $8563).

By the end of year 1, intervention group case-mix weights had increased to 4.2 and by year 2 to 4.98 (P = .00, repeated measures ANOVA). Over the 2 years the percent of participants with high to very high resource use also increased from approximately one-third (31%) to one-half (52%) and average cost almost doubled, from $8563 to $16,725 (P = .00). On the other hand, there were no significant changes in any ACG metric within the control group.

As Table 5 demonstrates, a large shift to higher acuity for intervention group members occurred in years 1 and 2. Because ACGs are mutually exclusive actuarial cells, the proportion of beneficiaries with higher-acuity ACGs displaced those with lower-acuity ACGs. The proportions of intervention members with 10 or more diagnosis clusters grew 72% the first year and almost doubled by the second year, and the proportion with 6 to 9 diagnostic clusters grew 62% during the first year.


The comparative spike in case mix rates for the group that gained greater access to services illustrates an important limitation of risk tools based on administrative data. Greater utilization of care increases the chance that more conditions will be diagnosed, even those that are only suspected. More International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes drive up case mix, regardless of whether health status has changed. For our subjects, SF-12v2 PCS scores suggest that, over the first 24 months, the intervention group actually experienced better health status than the control group, or, at the very least, certainly no worse. Case-mix and other ACG scores, on the other hand, suggest the intervention group experienced a dramatic worsening of health status. The same measures for controls, which differed only in the level of services received, showed no significant change. Other studies similarly have found risk scores more dependent on utilization patterns than comorbidity. For instance, Song et al found that when Medicare beneficiaries from low-intensity practice regions moved to higher practiceintensity regions, their HCC risk scores increased comparatively more than those who moved to lower practice-intensity regions.21 Welch et al found that case-mix scores did not predict mortality. In fact, the case fatality rate among Medicare beneficiaries moved inversely with the mean number of serious conditions diagnosed in 306 hospital referral regions; paradoxically, the greater the number of serious conditions diagnosed, the lower the fatality rate.22

Several potential explanations exist for the case mix increase among intervention subjects. One possibility is that the latter scores represent a more valid estimate of comorbidity than at baseline because both groups had previously been underserved. This interpretation is supported by evidence from a variety of qualitative and quantitative sources. First, at least 82% met criteria to be considered underinsured relative to their ability to pay for care. Participants in 6 focus groups (n = 42) conducted during the first 2 years of our study reported forgoing or deferring care for complex and potentially serious conditions, such as kidney disease and gastrointestinal bleeding, and employing numerous strategies to minimize cost, such as saving up procedures until they had met deductibles and rationing medications.16 Once the DMIE program benefits removed the cost barriers, they reported that they were able to afford needed services. We examined the records of individuals with large changes in ACG weights and found baseline self-reported conditions, such as diabetes and a cancer history, for which they had no claims during the baseline year but which were treated after the intervention began.

The time-limited nature of the intervention may have also encouraged those individuals with pent-up need to consume high levels of service while the DMIE benefit was in place. Although all types of service consumption increased, we saw notable spikes in elective surgeries, such as joint replacements, expensive screening and diagnostic testing, and services not included or for which coverage was very limited under the basic state high-risk pool benefit. The latter include prescription coverage with cost sharing of 50% after deductible that decreased to a flat $3 per drug copay under the DMIE, a major benefit for individuals heavily dependent on high-cost prescriptions for conditions such as cancer and autoimmune diseases.

In addition, all intervention group members received telephonic case management from registered nurses, who assisted subjects in identifying unmet needs and provided prior approval of insurance coverage. Although this surveillance was intended to help subjects optimize their health status while preventing wasteful utilization, it also inevitably raised awareness of unmet needs and stimulated a demand for services. Physicians who became aware of the availability of increased coverage also may have made more recommendations for interventions during this time-limited benefit period.

Because of this spike in utilization, some of which represented pent-up demand, the resulting case-mix scores may be inflated. However, many of the conditions for which participants sought treatment are chronic and represent continued clinical and actuarial risk, whether or not they continue to be incorporated into risk scores. This temporal aspect of risk adjustment highlights another limitation: case-mix scores normally encompass only 1 year of data; pre-existing conditions that have not been treated during the past year will not be detected, and underlying comorbidity may be vastly underestimated.

Study Limitations

The small sample size and high comorbidity risk of this study population, consisting of underinsured individuals with intensive need for healthcare services, limits the generalizability to large health plans with more generous health benefits and healthier, or more typically representative, health status. In addition, this study was conducted in only 1 small state; however, similar case mix increases were observed in the Texas DMIE, which used the ACG with a previously uninsured group of 1400 individuals whose initial health status was similar to those in the Kansas DMIE study.23 The findings seem most relevant to populations with high unmet healthcare need who suddenly gain access to relatively comprehensive healthcare benefits.


The ACA proposes to use risk adjustment as a permanent mechanism for protecting insurance plans against adverse selection. The federal government will provide insurance exchanges with a risk-adjustment tool, such as the HCC, or allow a comparable substitute. But all of these tools share important vulnerabilities. Although intended to shield plans against adverse selection, these tools may inadvertently reward the inefficient at the expense of the efficient. At worst, they could encourage insurance industry “gaming” in order to maximize revenues and shareholder profits.24

Copyright AJMC 2006-2018 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up

Sign In

Not a member? Sign up now!