The Validity of Claims-Based Risk Estimation in Underinsured Populations

This article demonstrates a threat to validity when using claims-based risk tools with chronically ill and/or underinsured populations whose underutilization masks actual risk or comorbidity.
Published Online: December 18, 2012
Janice M. Moore, MSW; and Jean P. Hall, PhD
Objectives: To demonstrate a threat to validity in using claims-based risk tools with chronically ill, underinsured populations.

Study Design: We tracked disease burden of highrisk pool beneficiaries with potentially disabling health conditions receiving enhanced health insurance benefits through a federally funded research demonstration. At baseline, beneficiaries paid high premiums and cost sharing for risk pool coverage, and most met common criteria for underinsurance. Study benefits provided intervention group members premium and cost-sharing subsidies and additional coverage; control group members paid usual premiums and coinsurance and received usual benefits. We hypothesized that enhanced benefits for the intervention group would increase or stabilize health status measures and decrease case-mix weights, reflecting stabilized or reduced disease burden.

Methods: The SF-12v2 health survey was used to measure health status and the Johns Hopkins Adjusted Clinical Groups (ACGs), Version 8.2 with DX-PM model and prior cost for a non-elderly population, was used to measure disease burden.

Findings: Over a 3-year period, SF-12v2 scores showed stable health status for the intervention group and significant decline for the control group, while ACG case-mix weights, major illnesses, and chronic condition counts rose significantly for the intervention group but remained stable for the control group. Increased resource utilization for the intervention group appears to have driven increases in ACG measures.

Conclusions: When high cost-sharing constrains access to care, risk tools that rely on medical claims may not provide an accurate measure of disease burden.

(Am J Manag Care. 2012;18(12):e468-e476)

  •  Risk tools based on International Classification of Diseases, Ninth Revision, Clinical Modification codes assume that access to care is sufficient to adequately represent health status.

  •  Access barriers may mean that many conditions, even major comorbidities, are not coded.

  • While the Affordable Care Act (ACA) may bring coverage to millions of previously uninsured individuals, many will still face high cost sharing. In addition, they may have higher than average comorbidity because of lower socioeconomic status.

  •  Since the ACA proposes to use individual risk scores in adjusting risk across exchanges, and because claims-based risk tools are widely used for other purposes, the potential bias associated with underutilization should be recognized.
A number of administrative tools have been developed to measure comorbidity—or burden of illness—through claims data. Instruments such as the Diagnostic Cost Group/Hierarchical Condition Category (DCG/HCC),1 the Johns Hopkins Adjusted Clinical Groups (ACGs),2 and the Chronic Illness and Disability Payment System (CDPS),3 were created to measure actuarial risk but have been extended to numerous other purposes, such as targeting high-risk patients for case management, profiling physician performance, and conducting outcomes research.4 Most recently, the Affordable Care Act (ACA) proposes to use individual risk scores to determine or validate average plan risk within health insurance exchanges.5,6 Plan average risk scores, in turn, will be used to reallocate premiums within exchanges so that plans with a disproportionate share of sicker individuals will be compensated for losses. Thus, the issue of the validity of commonly used risk tools is assuming increased importance and examining potential vulnerabilities is an urgent issue.

In the absence of readily available standardized clinical information, administrative data are often the only practical source of information for estimating illness burden. Typically tools such as the ACG and CDPS calculate a case-mix score by combining diagnostic and/or prescription drug codes with demographic variables associated with variation in utilization, such as gender and age. Some tools use prescription drug codes instead of or in addition to diagnostic codes.

Various authors have noted the limitations of these tools.7-9 This article makes the case for another common, unrecognized vulnerability. Specifically, the validity of administrative data as a proxy for comorbidity requires that beneficiaries have adequate access to healthcare services in order to generate a reasonably complete diagnostic profile. As this paper will demonstrate, scant or missing diagnosis codes and cost data can severely bias case-mix scores, even for very sick populations. Arguably, tools that use pharmacy data may compensate somewhat, because many chronically ill individuals use prescriptions even when they do not regularly see a provider. However, these tools are limited by the lack of specificity in the uses of many drugs. In addition, because many individuals now obtain prescriptions from $4 generic drug retail programs, insurer prescription records can no longer be assumed to represent reliable records of utilization.10 We illustrate these vulnerabilities with the ACG by showing how scores were influenced by a dramatic increase in utilization among a group of chronically ill adults participating in a research demonstration.



The Demonstration to Maintain Independence and Employment (DMIE) was a study of disability outcomes sponsored by the Centers for Medicare & Medicaid Services (CMS) under the Ticket to Work and Work Incentives Improvement Act (TW-WIIA) of 1999 and independently conducted by 4 states between 2006 and 2009.11 The demonstrations tested the hypothesis that improved access to healthcare and personal supports would prevent or delay transition to reliance on Social Security disability benefits among a population with potentially disabling conditions. The intervention was targeted to uninsured or underinsured populations whose high healthcare costs may have resulted in their delaying or forgoing needed health services. We hypothesized that the intervention would work over time by reducing barriers to care seeking, thus improving health status and reducing the burden of illness. This paper describes the experience of the Kansas DMIE, which recruited participants from the state high-risk insurance pool. The case-mix findings reported below pertain to the subset of the study population that was continuously enrolled for the first 2 years of the study. A more complete description of the full study population and outcomes is reported elsewhere.12,13

As with most other state high-risk pools, the Kansas pool offers a benefit package modeled on nongroup coverage, which typically has less generous benefits and is more expensive than group coverage. At baseline, premiums were 125% of standard market rates. Deductibles ranged from $500 to $7500, with no benefits available prior to meeting the deductible, other than a $250 preventive benefit in some plans. Prescription coinsurance was 50%, and medical coinsurance was 30% in-network and 50% out-of-network, with out-ofnetwork charges not accruing toward the annual coinsurance maximum. Some plans had a $100,000 annual maximum, and all had a lifetime limit of $1 million. The coinsurance cap for in-network services was $5000 for single policies and $14,000 for family plans, with some plans having unlimited 10% coinsurance thereafter.

Study Design

The study design was a clinical trial with applicants randomly assigned to equal-sized intervention and control groups. Control group members received high-risk pool benefits as usual, plus cash stipends for participation in surveys ($1200 paid in progressively increasing installments) and focus groups ($25 per session).

The intervention consisted of premiums subsidized to a flat $152 per month; elimination of all deductibles and coinsurance; cost-sharing limited to a $3 copay per service and charges in excess of reasonable and customary for out-of-network services; and a $1 million lifetime limit for DMIE-paid services. The intervention also provided nursing case management to coordinate care and pre-authorize benefits.

To be eligible, participants had to be enrolled in the state high-risk pool at least 6 months prior to recruitment and be aged 18 to 60 years, so that no one would turn 65 years during the study and attain Medicare coverage. They had to experience 1 or more health conditions that represented a potential for disability, as designated by CMS and the state program administrators. Data collected included claims files, surveys including standardized health status assessments and other questions, and focus group discussions. This research design and evaluation plan was approved by the institutional review board affiliated with the authors.

Participant Recruitment

The program began recruiting participants in the fall of 2005 and offered intervention services from April 2006 through September 2009. Over this time a total of n = 508 subjects participated. The present analysis is limited to the subset of subjects who were continuously enrolled during the first 2 years of the study (n = 317, cohorts 1 and 2). The recruitment process is described in more detail elsewhere.12,13

Although intervention and control groups were equal sized at baseline, large control group attrition resulting from dropping high-risk pool coverage created an imbalance in group size over time (n = 184 intervention, n = 133 control by the end of the second year). Comparison of demographic and health status data, including interviews that tracked reasons for disenrolling and types of replacement coverage, showed that those who disenrolled did not differ significantly from those who remained. Most disenrollees obtained group coverage through their own or a spouse’s employment, where there was no underwriting for pre-existing conditions. The reason cited for leaving high-risk pool coverage was almost exclusively the unaffordability of coverage.

Sample Baseline Characteristics

Study participants were demographically similar to the overall high-risk pool population, except that they were on average slightly younger, reflecting the study’s eligibility cap. We found no statistically significant between-group differences in demographic variables or distribution of comorbidities (Tables 1, 2). Most participants were aged 50 to 60 years, married, well-educated, and owned homes. Mean incomes were approximately $50,000 for individuals and $71,000 for families, with median individual incomes of $30,000 and family incomes of $50,000; 57% were below 400% of poverty when family size was included and 21% had incomes below 200% of federal poverty level. Occupations included homebased service or manufacturing enterprises, farming, professional services, and small business ownership. About 70% of participants earned at least some income from self-employment, explaining this population’s lack of access to group health insurance.

Baseline claims and self-reported diagnoses showed that participants experienced numerous comorbid conditions (Table 2). Musculoskeletal pain, cardiovascular conditions, and diabetes were among the most common; 19% had cancer, either active or by history. Forty-five percent reported at least 1 activity of daily living (ADL) limitation and 46% at least 1 instrumental activity of daily living (IADL) limitation, most commonly those requiring physical strength and mobility. Based on self-reported height and weight, 75% were overweight to obese (body mass index [BMI] >25), including 32% obese (BMI >30-40) and 11% morbidly obese (BMI >40).

Baseline Health Cost Burden

PDF is available on the last page.
Adult ADHD Compendium
COPD Compendium
Dermatology Compendium
Diabetes Compendium
GI Compendium
Immuno-oncology Compendium
Lipids Compendium
MACRA Compendium
Oncology Compendium
Pain Compendium
Reimbursement Compendium
Rheumatoid Arthritis Compendium
Know Your News
HF Compendium
Managed Care PODCAST