Marc N. Elliott, PhD

 To develop an easy-to-interpret, patient-reported Functional Limitations Index (FLI) that can be used to assess and monitor the full spectrum of functioning in a community-dwelling population.

 Observational design using nationally representative survey data.

 We used self-rated health as a criterion for empirically assigning weights to 5 National Health Interview Survey items assessing difficulty with seeing, hearing, walking, cognition, and self-care. In addition to succinctly summarizing cumulative limitations, we addressed 2 main questions: (1) Which limitations have stronger associations with self-rated health? and (2) How does severity (from 0, no difficulty, to 3, unable to do) relate to self-rated health? We generated a respondent-level summary score based on a model predicting self-rated health from the 5 linearly scored (0-3) items and used splines to account for nonlinear severity—self-rated health associations.

 The strongest association of specific functional limitations with self-rated health involved mobility; the weakest associations involved sensory limitations. The association of severity with self-rated health was nonlinear and largest moving from 

. Nationally, 5% of noninstitutionalized adults were considered 

. Great mobility limitations (defined as 

) most distinguished limitation groups (present in 0% of 

 The FLI is an easy-to-administer, easy-to-interpret, and valid summary measure of disability that health plans and health care organizations can use for quality-of-care monitoring across a variety of settings to improve care for patients with disabilities.

The 5-item Functional Limitations Index is an easily administered and easily interpretable measure that distinguishes patients along the disability continuum (least to most limited).

The strongest associations of specific functional limitations with self-rated health involved mobility and self-care, possibly reflecting acute illness sequalae.

The weakest associations involved sensory limitations, possibly due to long-term adaptation through accommodations.

Greater severity was associated with worse self-rated health, with the largest increase from no difficulty to somewhat difficult, possibly reflecting challenging adjustment to new health problems.

Limitations groups were most distinguished by being unable to or having a lot of difficulty walking or climbing stairs.

An estimated 12% to 30% of Americans have a disability,

 In clinical populations, greater disability is associated with higher costs and lower quality of life.

 There is a national commitment to improving the health and health care of persons with disabilities,

 who make up one of the largest groups of underserved adults in the United States.

 This requires a consistent, efficient means of identifying and monitoring the spectrum of community-dwelling individuals with disabilities.

Some widely used measures of disability are used to classify working-aged adults as unable to work because of a health condition (eg, CMS original entitlement code); such measures do not capture individuals with disabilities who are able to work or who developed disabilities after age 65 years. Other measures involve a clinical diagnosis (eg, the Sheehan Disability Scale

) or are limited to 1 or 2 domains (eg, SF-36 Physical Function

 score of mobility), limiting their usefulness in the general population.

Responding to the need for a standardized, broadly applicable measure of disability,

 a group of disability measurement experts developed the International Classification of Functioning, Disability and Health (ICF) to classify the consequences of disease and complement the 

 functioning, defined here as activity limitations, results from dynamic interactions among health conditions, body function and body structure impairments, participation in social roles, and environmental and personal factors. The ICF model focuses on functioning independent of health condition.

 For example, a patient with severe pain from a back injury and a patient with cerebral palsy may both have limited mobility, whereas a full-time student with depression may experience concentration limitations and a part-time worker with depression may not. The ICF model treats the first 2 people the same and the second 2 as distinct.

In this paper, we describe the development of a brief, easy-to-administer, and easy-to-interpret summary measure of patient functioning that health care organizations and providers can use to assess and monitor a wide spectrum of functioning independent of clinical diagnoses. In keeping with the ICF model, we focus on functional limitations independent of disease or condition. We considered only items in the 6-item Short Set of Disability Questions included in the National Health Interview Survey (NHIS) since 2010, as this measure was based on the ICF model.

 This short set of questions reliably and validly

 measures limitations associated with undertaking basic activities.

 It represents the majority of individuals (but not all) with functional limitations and assesses the most frequently occurring (but not all) domains of functional limitation.

We describe an empirically informed method of combining the items in this set to preserve distinctions among different types and severities of functional limitations while maximizing the correlation to self-rated health. Typically, individuals are classified as having a disability if they have at least 1 serious functional limitation.

 Another approach would be to classify individuals experiencing difficulty with at least 2 domains or activities of daily living as having a disability.

 Both of these dichotomous approaches ignore the possibility that the severity and different types of limitations may be associated with different levels of well-being and medical need.

Because the effect of a disability on well-being and medical need is likely to fall on a continuum, a continuous measure is needed to better capture the effect of disability and its severity, even if the continuous score is ultimately categorized for some purposes. One way to create a continuous measure is to count the number of limitations that a person reports

; although simple, a count assumes that each functional domain implies the same level of medical need or requires the same degree of intervention. Alternatively, functioning domains could be weighted to reflect their associations with health, using published weights (eg, those used in the World Health Organization [WHO] Disability Assessment Schedule

) or specific weights for the specific items in a given survey. For example, Altman and Bernstein

 constructed a weighted summary measure of disability based on 8 functions from the 2001-2005 NHIS, based on untested assumptions about the relative effects of different functional limitations and levels of severity.

Here we describe the development and application of a weighted summary measure of functioning with empirically derived weights reflecting the association between several functional limitations and self-rated health. Self-rated health was chosen as a criterion variable because it predicts health care utilization, morbidity, illness recovery, functional decline, and mortality,

 and because it is widely available in survey data. To inform our model, we addressed 2 questions: (1) Which limitations have stronger associations with self-rated health? and (2) How does the association of limitations with health vary by level of severity? Measuring the importance of severity in different patient functioning domains has practical clinical and policy implications for monitoring and improvement of quality of care.

&#8203;&#8203;&#8203;&#8203;&#8203;&#8203;

Data are from the 2014-2015 NHIS Adult Functioning and Disability (AFD) supplement, fielded with the Sample Adult module to a randomly chosen subset of all sampled adults (18,303 in 2014 and 16,939 in 2015).

We excluded 1818 AFD respondents (5.2%) who were missing all 6 WHO functional limitation items. We imputed a 0 (no difficulty) for 91 (0.3%) respondents who completed some but not all functional limitation items, as it is common for respondents to misunderstand long sequences of similar items as a “check-all-that-apply” format.

 Sampling weights incorporating study design and nonresponse were applied to all analyses, and robust variance estimation was used.

The NHIS-AFD module includes the following 6 functional limitation questions: vision (difficulty seeing), hearing (difficulty hearing), mobility (difficulty walking or climbing steps), communication (difficulty communicating), cognition (difficulty remembering or concentrating), and self-care (difficulty washing all over or dressing). All items had response options of no difficulty (level 0), some difficulty (level 1), a lot of difficulty (level 2), and unable to do (level 3). We sought to create a brief index based on these 6 well-validated items.

Constructing a Summary Functional Limitations Score

We developed a linear regression model that predicted self-rated health from the functional limitation items. The final model retained 5 functional limitation items scored linearly (0-3). Communication was excluded from the final model because an initial model that included all 6 items yielded a coefficient for communication that did not significantly differ from zero (

]). For each functional domain, we added a linear spline term that allowed the estimated difference between 

 (level 1) to differ from the estimated difference between 

Covariate-adjusted self-rated health scores were estimated using recycled predictions.

 The covariate-adjusted score is the expected self-rated health score for a given person if the person’s sociodemographic characteristics corresponded to those of the average person in the population. The coefficients of the functional limitations from the final model shown in 

 can be used to derive a predicted self-rated health score for every adult AFD respondent based on their self-reported vision, hearing, mobility, cognition, and self-care items (eAppendix B). We created a standardized (

 score) score, the Functional Limitations Index (FLI),

by subtracting the sample mean from each score and dividing the difference by the sample SD. The mean index score was 0 with an SD of 1. We classified respondents into 1 of 3 limitation groups based on the index scores. The cut points defining these groups were 1 SD apart on the index. Because the groups were meant to distinguish those with limitations, all cut points were below 0 (

). These cut points were selected to meet the potentially competing goals of having reasonably sized limitation groups (to ensure reliable subgroup estimates) and representing distinct levels of limitations. Most respondents were without limitations because the NHIS represents the general population. Respondents with no or very little limitation were assigned 0 or just above 0 scores. Although a small proportion of individuals with more limitations had large negative scores, there were no large positive scores; thus, the distribution of scores on the index is left-skewed.

As a preliminary validation and illustration of our approach, we compared the functional limitations and sociodemographic characteristics of the representative noninstitutionalized US adult population who were categorized as 

 per our FLI. We also determined which socio&shy;demographic characteristics were associated with which limitation group using bivariate and multivariate logistic regression models to predict being classified 

Predicting Self-rated Health From Functional Limitations

Negative coefficients for each functional limitation indicated a negative association with self-rated health (0-100 scale, where 0 represented poor health and 100 excellent health). Each retained functional limitation main effect coefficient was negative and statistically significant (

&thinsp;<&thinsp;.05). The strongest predictor of self-rated health was mobility limitation (—15 points for each additional severity level), followed by self-care (–10 points), cognition (–7 points), vision (–5 points), and hearing (–3 points).

, the negative association with functional limitations diminished (Figure 2). For example, a respondent with a single level 1 (

) mobility limitation was predicted to have a 15-point lower self-rated health score than those with no such limitation, but a respondent with a single level 2 (

) mobility limitation was predicted to have one 22 points (—15&thinsp;×&thinsp;2&thinsp;+&thinsp;8) lower, and a respondent with a single level 3 (

) limitation was predicted to have one 29 points (—15&thinsp;×&thinsp;3&thinsp;+&thinsp;8&thinsp;×&thinsp;2) lower. This means that the difference in quality of life between reporting 

 (15 points) was greater than the difference associated with reporting 

 an activity (7 points). Although the association with self-rated health diminished with increased severity in general, the largest change in self-rated health associated with moving from 

We used the model predicting self-rated health to assign the FLI score and create 3 limitation groups: 5% of noninstitutionalized US adults, hereafter referred to as adults, were considered 

 score&thinsp;>&thinsp;—2 and&thinsp;≤&thinsp;–1), and 87% 

 score&thinsp;>&thinsp;—1). As shown in 

, the proportion of adults with no limitations (

) ranged from 82% (for hearing) to 97% (for self-care). The proportion reporting great limitations (

) ranged from less than 1% (for self-care) to 6% (for mobility). Great mobility limitations most distinguished limitation groups: They were present in 0% of the 

Distribution of the Number of Functional Limitations Among Limitation Groups

 presents the proportion of adults with any limitation or great limitations in none, 1, 2, and 3 or more areas of functioning, overall and by limitation group. Overall, 65% of adults had no limitations, 22% had 1 limitation, 9% had 2 limitations, and 4% had 3 or more limitations. About 75% of the 

 group had 0 limitations, 23% had a single limitation, and 2% had 2 limitations. Of the 

, 60% had 2 limitations, 30% had 1 limitation, and 10% had 3 or more limitations. Of the 

, 62% had 3 or more limitations and most of the remainder (31%) had 2 limitations.

Overall, 8% of adults had at least 1 great limitation and 1% had 2 or more. Of the 

, 2% had a single great limitation and the remainder had no great limitation. Of the 

, 31% had 1 great limitation and 2% had 2. Of the 

, 76% had at least 1 great limitation, 14% had 2 great limitations, and 3% had 3 or more great limitations.

Distribution of Types of Limitations Among Limitation Groups

 had any functional limitations, most often 

 in a single area of sensory functioning (hearing or vision, 9%-13%), and rarely any mobility or self-care limitation (<5%). The 

 all had at least 1 limitation, and most had 2 limitations (usually 

 but occasionally great limitation). A majority (96%) of the 

 had mobility limitations; when a great limitation existed, it was usually in the realm of mobility. Approximately one-third or more of the 

 group had a sensory limitation (33%-45% with any limitation) and rarely a self-care limitation (5%). A majority of the 

 had 3 or more limitations, including at least 1 great limitation. Nearly all (99%) of the 

 had mobility limitations; a majority had great mobility limitations along with cognitive, hearing, and vision limitations. More than half of the 

 group (60%-75% with any limitation) had a sensory limitation and approximately half (47%) a self-care limitation.

Sociodemographic Characteristics of the 3 Limitation Groups

 were more often male, younger, and Hispanic or Asian and lived in the Northeast (

 were more often married and socioeconomically advantaged; that is, they more often had a bachelor’s degree or more, reported an income of at least $75,000, and utilized private rather than public insurance.

The 5-item FLI is an easy-to-administer, easy-to-interpret, valid measure of functioning that health care organizations can use to identify and monitor the care and outcomes of individuals with disabilities of different severities independent of health conditions.

Our approach to scoring the FLI improves on other methods used to measure and summarize disability by utilizing patient function over clinical diagnoses and allowing empirical evidence to drive the weighting of different functional limitations and their severities for the summary score. By adjusting for socio&shy;demographic characteristics, the FLI represents variation in health solely due to functional limitations; it can be used as a continuous measure of disability or to create limitation groups for analysis.

Self-rated health has a strong negative association with each of 5 functional limitations, with mobility limitations having the strongest association, followed by limitations in self-care, cognition, vision, and hearing. Our results are consistent with those of Balestroni and Bertolotti,

 who found that mobility was a stronger predictor of the EuroQol EQ-5D, a health-related quality-of-life measure, than self-care or daily activities limitations.

The disability paradox is the phenomenon wherein individuals with disabilities report greater well-being than unimpaired individuals assume they would experience in the same situation. In a 2014 survey, largely unimpaired community-dwelling adults cited vision loss as the limitation they believed would have the greatest effect on their day-to-day life, followed by memory loss; they ranked hearing loss as least likely to have a large effect.

 Our finding that visual limitations are not strongly associated with self-rated health suggests that the disability paradox may be especially relevant for visual sensory limitation; the relatively small impact of hearing loss is consistent with unimpaired individuals’ assessment. The associations between sensory limitations (vision and hearing) and self-rated health may in part reflect long-term adaptation to accommodations (eg, eyeglasses and hearing aids). In contrast, mobility and self-care limitations may reflect acute illness sequelae, including physical deconditioning, early mortality, and falls resulting in loss of independence and ability to participate in social activities.

 The large coefficients corresponding to a change from 

 limitation may reflect the difficulty of adjusting to the new onset of a health problem, whereas the smaller coefficients corresponding to differences between 

 limitations may reflect some combination of worsening of an existing health problem and the success of environmental adaptations to functional limitations.

This study has some limitations. Although the 6 NHIS functional limitation items are likely to capture most adults with disabilities, they probably underrepresent adults with psychiatric and cognitive disabilities associated with higher-order functioning limitations (eg, learning, decision making).

 Mental health disability is potentially very important, especially among those who also report disability due to a chronic condition.

 Information on US adults with intellectual disabilities is not routinely collected, despite recognized disparities in care.

 We did not evaluate other ICF items (eg, anxiety, depression) given our interest in a function-based measure. The FLI does not distinguish between permanent and temporary disability.

 Finally, although the construction of the FLI was supported by a strong theoretical foundation and the pattern of associations between limitation groups and sociodemographic characteristics provides preliminary evidence of the validity of the measure, additional validation work (eg, investigation of the convergent, discriminant, and predictive qualities of the index) is needed prior to any high-stakes application.

Policy makers and clinical researchers must understand how varying levels of disability influence quality of life. As individuals experience disability at higher levels of severity, the negative association between limitations and self-rated health increases, but to a diminishing extent. Dichotomous or summed counts of disability ignore this phenomenon. A key strength of the FLI is that because it empirically summarizes multiple functional limitation items and severity levels on the same ordinal scale as a single dimension, it can meaningfully facilitate research into variability in quality of and access to care by improving the insight into the relative contributions of different areas of functioning, severity, and additivity. The FLI may also help assess heterogeneity in disability-related disparities for different population subgroups. This method can be used as a guideline and be adapted to other populations of interest, such as Medicare or Medicaid beneficiaries. Policy makers might also find the FLI useful for assessing programmatic outcomes. For example, the association between disability and general health might be used to monitor change over time related to policy changes, such as the Americans with Disabilities Act, or programs, such as the Social Security Disability Insurance program.&ensp;

The authors would like to thank Biayna Darabidian for help with preparation of the manuscript.

 RAND Corporation, Santa Monica, CA (MM, DA, MNE, NO, MKB), and Pittsburgh, PA (SCM); American Hospital Association (PG), Chicago, IL; National Committee for Quality Assurance (JHN), Washington, DC.

Articles

We describe an easy-to-interpret, patient-reported Functional Limitations Index that can be used to monitor care of patients along the disability continuum.

The Development of a Patient-Reported Functional Limitations Index

 Studies have identified potential unintended effects of not adjusting clinical performance measures in value-based purchasing programs for socioeconomic status (SES) factors. We examine the impact of SES and disability adjustments on Medicare Advantage (MA) plans’ and prescription drug plans’ (PDPs’) contract star ratings. These analyses informed the development of the Categorical Adjustment Index (CAI), which CMS implemented with the 2017 star ratings.

 Retrospective analyses of MA and PDP performance using 2012 Medicare beneficiary-level characteristics and performance data from the Star Rating Program.

 We modeled within-contract associations of beneficiary SES (Medicaid and Medicare dual eligibility [DE] or receipt of a low-income subsidy [LIS]) and disability with performance on 16 clinical measures. We estimated variability in contract-level DE/LIS and disability disparities using mixed-effects regression models. We simulated the impact of applying the CAI to adjust star ratings for DE/LIS and disability to construct the 2017 star ratings.

 DE/LIS was negatively associated with performance for 12 of 16 measures and positively associated for 2 of 16 measures. Disability was negatively associated with performance for 11 of 15 measures and positively associated for 3 of 15 measures. Adjusting star ratings using the CAI resulted in half-star rating increases for 8.5% of MA and 33.3% of PDP contracts that exceeded 50% DE/LIS beneficiaries.

 Increases in star ratings following adjustment of clinical performance for SES and disability using the CAI focused on contracts with higher percentages of DE/LIS beneficiaries. Adjustment for enrollee characteristics may improve the accuracy of quality measurement and remove incentives for providers to avoid caring for more challenging patient populations.

CMS implemented the Categorical Adjustment Index (CAI) as part of the Medicare Advantage and Part D Star Rating Program in 2017. These analyses informed its development.

Low socioeconomic status and disability are negatively associated with performance for most measures included in the Medicare Advantage Star Rating Program, controlling for between-contract effects.

The CAI most benefits contracts with at least 50% of enrollees who are dually eligible for Medicare and Medicaid or receive a Part D low-income subsidy.

Adjusting the star ratings using the recently implemented CAI changes the star ratings for a modest number of contracts by half a star.

Policy makers use quality measurement, public accountability, and financial incentives to induce health plans and providers to improve performance. Frequently, the process-of-care and intermediate outcome measures on which plans and providers are evaluated are not adjusted for differences in patient socioeconomic status (SES) across plans and providers. Pay-for-performance programs that do not account for differences in patients across providers in their quality measurement risk reducing funding to providers that treat medically complex, disabled, and socioeconomically disadvantaged patients, potentially reinforcing or exacerbating existing SES disparities, as more resources may be required to achieve high quality for such patients.

 Moreover, providers caring for disadvantaged patients tend to have fewer resources available to invest in quality improvement due to lower reimbursement rates compared with providers with predominantly commercially insured patients.

Low-SES individuals receive recommended care less often and experience worse health outcomes than those with higher SES,

 possibly because they have greater health burdens and barriers to care, including limited transportation and lower health literacy,

 which may discourage providers from treating disadvantaged patients. A recent examination of associations between social risk factors (dual eligibility [DE] for Medicare and Medicaid, black race, Hispanic ethnicity, disability, and rural residence) and performance on quality measures included in 9 Medicare value-based purchasing programs found that beneficiaries with social risk factors received recommended care less often.

 Providers disproportionately serving high-risk beneficiaries performed worse on average, even after controlling for beneficiary differences.

Policy makers have identified closing these quality gaps as a key policy priority.

 One approach to accomplish this is to adjust quality measures for socioeconomic factors. The National Quality Forum (NQF)

 have called on sponsors of value-based measurement and payment programs to determine whether quality measures should be adjusted for differences in providers’ patient mix. The National Academies of Sciences, Engineering, and Medicine developed criteria for determining which social risk factors to address.

Although adjustment of process-of-care and intermediate outcome measures for socioeconomic factors is rare, the use of SES-adjusted patient experience measures in the Medicare Advantage (MA), prescription drug plan (PDP), and Hospital Consumer Assessment of Healthcare Providers and Systems surveys are examples of nationwide implementation of such adjustment.

 Clinical measures in the Medicare Star Ratings Program are not currently adjusted for SES.

Annually, Medicare computes star ratings based on MA and PDP contract performance on clinical, patient experience, customer service, and complaint measures. The star ratings are reported on Medicare Plan Finder,

 determine MA quality-based bonus payments, and affect MA rebates and enrollment (see 

] for star ratings description). To address concerns that the star ratings disadvantage contracts serving low-SES and disabled beneficiaries, CMS implemented a Categorical Adjustment Index (CAI)

 beginning with the 2017 star ratings as an interim policy until measure developers evaluate which clinical measures should be adjusted. The CAI approximates the effect of case mix on star ratings, adjusting the underlying clinical measures for SES characteristics available in CMS administrative data (DE/low-income subsidy [LIS]) and disabled status.

We present analyses that informed the development of the CAI. Our analyses addressed 3 questions: (1) Do within-contract SES and disability performance disparities exist for clinical measures used in the Medicare Star Ratings Program?, (2) How consistent are within-contract disparities across contracts?, and (3) How does adjusting for SES differences affect the overall star rating of MA and PDP contracts, particularly for contracts serving a large portion of beneficiaries who are dually eligible for Medicare and Medicaid, receive a Part D LIS, or are disabled?

We used patient-level data from the 2014 star ratings (measurement year 2012) to assess the relationship of contract performance with SES (ie, DE/LIS) and disability and to develop the CAI. The 2014 star ratings used 48 Part D (prescription drug) and Part C (health plan) measures to rate MA prescription drug contracts, 36 Part C measures to rate MA-only contracts, and 15 Part D measures to rate PDPs. Analyses included all MA and PDP contracts eligible for star ratings. We excluded Puerto Rico contracts from analyses due to program differences.

We examined the effect of SES and disability adjustment for 16 (13 Part C and 3 Part D) clinical measures (

; see eAppendix for description). We excluded from evaluation those measures that were already adjusted for SES (n = 10 measures), being retired or revised (n = 6), used only for Special Needs Plans (SNPs; n = 3), addressing plan-level customer service (n = 12), or under direct provider or plan control (n = 1; high-risk medication). Measures were coded to indicate whether the beneficiary received the recommended care or achieved the measured outcome (0 = no; 1 = yes).

 Beneficiaries were classified as low SES if they were partially or fully dually eligible for Medicare and Medicaid as of December 2012 or if they applied and were approved for an LIS.

 Beneficiaries were classified as disabled based on their original reason for Medicare eligibility.

NQF recommends considering adjustment for within-provider disparities (the extent to which low-SES patients receive lower-quality care than high-SES patients within the same provider) while preserving between-provider differences in performance (the extent to which all patients of a given provider receive lower-quality care than others). Consistent with this recommendation, we assessed average within-contract DE/LIS disparities for each of the 16 measures by fitting logistic regressions predicting performance from the DE/LIS indicator, using fixed effects for MA and PDP contracts to control for between-contract performance differences (see eAppendix for additional detail). A sensitivity test examined the effect on DE/LIS and disabled effects after adjusting for Census-based SES characteristics (block group—level education and income/poverty; see eAppendix for additional detail). We performed similar analyses for the disability indicator.

Contract-level variation in disparity in performance for DE/LIS versus non-DE/LIS beneficiaries was estimated in percentage points using a linear mixed effects model that included DE/LIS as a predictor, mean-centered at the contract level, and random effects for contract and the contract-by-DE/LIS interaction, using empirical best linear unbiased predictions (BLUPs) to account for sampling error in contract-level disparity estimates. As expected from the sample sizes,

 results were insensitive to normality assumptions (not shown). We performed similar analyses for the disability indicator.

The CAI adjustment factor is applied to groups of contracts. Each contract is assigned to an adjustment group based on the percentage of its beneficiaries who are DE or LIS or disabled. The measure subset selected by CMS for the 2017 star ratings CAI is limited to measures with large and/or consistent within-contract disparities, as determined by measures for which the within-contract DE/LIS disparities based on BLUPs were large (median absolute difference in performance of 5 or more percentage points between DE/LIS and non-DE/LIS enrollees) or consistent (DE/LIS performed worse/better than non-DE/LIS enrollees in all contracts) in the 2012 measurement year data. Adjusted scores for the CAI measure subset were derived from logistic regression with contract fixed effects, DE/LIS, and disabled status as righthand-side variables. An adjusted overall star rating for each contract was simulated based on these adjusted measure scores plus all other star rating measure scores. The value of the CAI adjustment factor was computed as the average difference between contract-level adjusted and unadjusted overall star ratings within each CAI adjustment group. To simulate the effect of DE/LIS and disability adjustment on star ratings, we applied the CAI to the star ratings separately for MA and PDP contracts using the 2015 measurement year data (2017 star ratings). The results are summarized for contracts overall and stratified by contract percentage of DE/LIS beneficiaries (<50% DE/LIS and ≥50% DE/LIS).

The study was approved by RAND Corporation’s Human Subjects Protection Committee.

The analyses using the 2012 data included 620 MA and 76 PDP contracts. The number of MA contracts that met the denominator criteria for individual measures varied from 341 to 613 (Table 1). All PDP contracts met the denominator criteria for each of the included prescription drug event (PDE) measures. The average contract-level percentage of DE/LIS beneficiaries was 40.5% (SD = 38.7%), ranging from 0.4% to 100%, for MA contracts. PDP contracts averaged 22.1% DE/LIS (SD = 27.7%), ranging from 0.0% to 86.2%. Contracts with at least 1 SNP, which focus on specific subpopulations of Medicare beneficiaries, including those who are dual-eligible, have chronic conditions, or reside in institutions, had more DE/LIS beneficiaries than contracts without an SNP (59.0% vs 27.8%; 

 <.0001). Roughly one-third (34.4%) of MA contracts and one-fifth (21.1%) of PDP contracts had at least 50% beneficiaries who were DE/LIS. MA contracts averaged 19.8% (SD = 16.2%) disabled beneficiaries (ranging from 0.0% to 97.1%), whereas PDPs averaged 17.9% (SD = 15.2%) disabled beneficiaries (ranging from 0.0% to 54.7%). Contracts with SNPs enrolled more disabled beneficiaries than contracts without an SNP (29.4% vs 14.6%; 

 <.0001). Approximately one-fourth (23.2%) of MA contracts and one-third (31.6%) of PDP contracts had at least 25% disabled beneficiaries.

Within-Contract SES and Disability Performance Disparities

Controlling for between-contract differences, DE/LIS beneficiaries received significantly worse care for 12 of 16 MA measures (

), with odds ratios (ORs) ranging from 0.68 (95% CI, 0.66-0.70) to 0.94 (95% CI, 0.93-0.95). DE/LIS beneficiaries were more likely to have an adult body mass index assessment (OR, 1.10; 95% CI, 1.06-1.14) and have better performance on the measure reducing risk of falling (OR, 1.67; 95% CI, 1.60-1.74) than non-DE/LIS beneficiaries; there were not significant overall differences between DE/LIS and other beneficiaries for 2 measures (controlling high blood pressure and monitoring physical activity). Within PDPs, DE/LIS beneficiaries received significantly lower-quality care than other beneficiaries on all 3 PDE measures, with ORs ranging from 0.67 (95% CI, 0.66-0.67) to 0.81 (95% CI, 0.81-0.81). These results were not sensitive to further adjustment for Census-based SES characteristics (block group—level education and income/poverty) (eAppendix).

Controlling for contract, disabled beneficiaries received significantly less care for 11 of 15 MA measures (Figure 1), with ORs ranging from 0.56 (95% CI, 0.51-0.62) to 0.93 (95% CI, 0.91-0.96). Disabled beneficiaries were more likely to receive rheumatoid arthritis management (OR, 1.13; 95% CI, 1.10-1.17) and have better performance on the measures reducing risk of falling (OR, 1.32; 95% CI, 1.22-1.42) and monitoring physical activity (OR, 1.33; 95% CI, 1.26-1.40) than other beneficiaries; there were not significant overall differences between disabled and other beneficiaries for 1 measure (controlling high blood pressure). Within PDPs, disabled beneficiaries received significantly lower-quality care than other beneficiaries on all 3 PDE measures, with ORs ranging from 0.61 (95% CI, 0.61-0.61) to 0.74 (95% CI, 0.74-0.75).

Consistency of Within-Contract Disparities Across Contracts

 illustrates the heterogeneity of the within-contract difference in care received by DE/LIS beneficiaries relative to non-DE/LIS beneficiaries for each measure; 

 provides analogous information for disability. DE/LIS beneficiaries receive, on average, lower-quality care than non-DE/LIS beneficiaries in contracts.

For 3 measures, DE/LIS beneficiaries received lower-quality care than non-DE/LIS beneficiaries in all MA contracts. DE/LIS beneficiaries received lower-quality care in at least 90% of contracts for an additional 3 measures, but higher-quality care in all contracts for 1 measure. For PDPs, DE/LIS beneficiaries received lower-quality care in all contracts for 1 PDE measure and lower-quality care in 90% or more of PDPs for the remaining 2 Part D measures.

Disabled beneficiaries received lower-quality care than nondisabled beneficiaries in all MA contracts for 6 measures and received lower-quality care in at least 90% of contracts for 3 additional measures. They received higher-quality care in at least 90% of MA contracts for 2 measures. For PDPs, disabled beneficiaries received lower-quality care in all contracts for the 3 PDE measures.

Contract Star Ratings Following Adjustment for SES Differences Using CAI

There are 7 measures (6 MA and 1 PDP) for which the contract-level median absolute DE/LIS disparity is at least 5 percentage points or there are no contracts with DE/LIS scores equal to or higher than their non-DE/LIS scores. 

 shows the simulation of the overall star ratings when applying the CAI based on these 7 measures, with large and consistent DE/LIS disparities across contracts. Adjustment with the CAI changed the overall star ratings for 8.5% of contracts with 50% or more DE/LIS beneficiaries (Table 2). Gains in overall star ratings were concentrated in the high-DE/LIS group; 9 of 10 contracts that had higher overall star ratings following CAI had 50% or more DE/LIS. One contract that had less than 50% DE/LIS lost one-half star, while no contracts with 50% or more DE/LIS lost stars. No contract gained or lost more than one-half star.

Adjustment with the CAI changed the Part D ratings for 20.3% of PDPs (16.3% of contracts with <50% DE/LIS and 33.3% with ≥50% DE/LIS; Table 2). No contract gained or lost more than one-half star. Gains only occurred among contracts with 50% or more DE/LIS beneficiaries (n = 5; 33.3%), while losses only occurred among contracts with less than 50% DE/LIS (n = 8; 16.3%).

To address gaps in care, public and private payers have undertaken a variety of actions, including performance measurement, public reporting, and performance-based payments. Concerns have been raised that some program designs may create incentives for providers and plans to avoid more challenging patient populations.

 Adjusting performance for differences in the patient populations that plans and providers serve, to level the playing field, is one approach that has been suggested to address potential mismeasurement problems.

 Providers caring for low-SES patients may face communication challenges associated with lower education, English proficiency, and health literacy, as well as reduced patient access to care and compliance with medical regimes associated with limited transportation, residential instability, and other barriers.

 Accounting for these differences may reduce the likelihood that providers will avoid lower-SES patients in response to pay-for-performance programs.

We found within-contract disparities in performance on the clinical measures used to assess MA contract and PDP performance, predominantly reflecting lower odds of receiving recommended care for low-SES patients; the magnitude of within-contract disparities varied across measures and contracts. These findings are consistent with those of prior studies demonstrating associations among patient sociodemographic characteristics, including SES, for selected Healthcare Effectiveness Data and Information Set measures in commercially insured populations,

 for outcomes measures among the general population with cardiovascular disease or diabetes,

 and for medication adherence measures in the MA population.

Based on these analyses, CMS implemented the CAI with the 2017 star ratings. Adjustment of star ratings through CAI resulted in increased star ratings for some contracts with higher percentages of DE/LIS beneficiaries; 8.5% of MA contracts with 50% or more DE/LIS received half-star increases and none decreased, and 33.3% of PDPs with 50% or more DE/LIS received half-star increases and none decreased. Of contracts with less than 50% DE/LIS, less than 0.1% of MA contracts and 0% of PDPs had higher star ratings and less than 0.1% of MA and 16.3% of PDP contracts had lower star ratings.

Our study is the first to estimate the effects of adjusting the full set of clinical measures used in the Medicare Star Rating Program for SES factors and to simulate the effect of CAI adjustments for DE/LIS and disability on the star ratings used for quality bonus payments and public reporting. These results should inform future decisions about adjustment for SES.

Our study has several strengths. First, our analyses used patient-level measures of SES, in contrast to other studies that have used area-level estimates of SES from Census data as proxies for patient-level measures; these estimates measure a combination of the separate effects of the neighborhood in which a person resides and are a less accurate measure of person-level SES. Second, we measured the effect of SES adjustment for the universe of MA and PDP contracts. Third, we measured the effect of adjustment on all clinical measures contained in the Star Rating Program, rather than only a small subset of measures as has previously been reported.

 Fourth, inclusion of the contract fixed effects in our models allowed for adjustment for within-contract differences in quality for DE/LIS and disability, preserving quality differences between contracts and their affiliated providers that should be the target of improvement efforts.

 Fifth, we translate into policy-relevant terms the effect of risk adjustment at the measure level by examining its effect on the overall star rating used for quality bonus payment determination in MA contracts.

Our study also has several limitations. This study examined the effects of DE and/or receipt of LIS; although Medicaid eligibility varies by state, it is an important and widely available measure of low income and assets and has been recognized as the best proxy for income linkable to the Medicare beneficiary level.

 Furthermore, other measures of SES, such as housing stability, may be important markers of disparity; however, CMS and other payers would face challenges in collecting this measure of disadvantage. We believe that DE/LIS is a partial proxy for housing instability, as it measures the resources available to a beneficiary. This study was not designed to determine what factors allow some contracts to have small or zero disparities in care while others have sizeable disparities. Our findings are limited to beneficiaries in MA and PDP contracts, although other studies have found disparities in care in fee-for-service.

Policy makers, plans, and providers need to understand the effects of case mix on performance scores and to consider whether it is appropriate to adjust for differences. The overall impact of adjustment and the feasibility of adjustment are important considerations.

 In addition, even when risk adjustment does not lead to changes in performance scores for most providers, it provides face validity to the overall measurement effort in signaling to providers that their treatment of more challenging patients will be accounted for in performance assessment. It is important to design performance measures to influence plan and provider behavior in desired ways, and case-mix adjustment could guard against undesired behaviors, improve the accuracy of quality measurement, and increase the incentive for high-performing contracts to enroll low-income and disabled beneficiaries, which, in turn, might help reduce disparities in quality of care. Decisions about whether to adjust and the effects of adjustment will be a function of the existence of within-contract or within-provider disparity, the magnitude of disparity, and the structure of the scoring algorithm used to rate providers.

In addition to adjustment for SES, which primarily addresses issues of quality measurement, policy makers may consider other options to reduce disparities in health and healthcare, including enhancing data collection to better support reporting quality, specifically for patients with social risk factors; developing and including in value-based purchasing programs measures of health equity paired with incentives to improve performance on these measures; changing the payment structure of incentive programs to reward high performance and improvement among beneficiaries with social risk factors; providing support and technical assistance to providers that serve beneficiaries with social risk factors; developing demonstrations that focus on care innovations intended to achieve better outcomes for beneficiaries with social risk factors; and requiring the coordination of benefits between Medicare and Medicaid by contracts that serve dually enrolled beneficiaries.

RAND Corporation, Pittsburgh, PA (MES, AH), and Santa Monica, CA (SMP, CLD, MK, AT, MM, MNE).

CMS implemented the Categorical Adjustment Index as part of the Medicare Advantage and Part D Star Rating Program in 2017. These analyses informed its development.
Check out our website’s new table/figure pop-up feature! Click on the name of a table or figure in the text to see it in your browser.

Adjusting Medicare Advantage Star Ratings for Socioeconomic Status and Disability

 To assess whether Medicare patients’ reports of denied care, appeals/complaints, and satisfactory resolution were associated with ratings of their health plan or care.

 Retrospective analysis of 2010 Medicare Advantage Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey data.

 Multivariate linear regression of data from 154,766 respondents (61.1% response rate) tested the association of beneficiary ratings of plan and care with beneficiary reports of denied care, appeals, complaints, and complaint resolution, adjusting for beneficiary demographics.

 Beneficiaries who reported being denied needed care rated their plans and care significantly less positively, by 17.2 points (on a 100-point scale) and 9.1 points, respectively. Filing an appeal was not statistically significantly associated with further lower ratings. Beneficiaries who filed a complaint that was satisfactorily resolved gave slightly lower ratings of plans (—3.4 points) and care (–2.5 points) than those not filing a complaint (

 Lower ratings from patients reporting complaints and denied care may notably affect the overall 0-10 CAHPS ratings of Medicare Advantage plans. Our results suggest that beneficiaries may attribute the actions that lead to complaints or denials to plans more than to the care they received. Successful complaint resolution and utilization management review might eliminate most deficits associated with complaints and denied care, consistent with the service recovery paradox. High rates of complaints and denied care might identify areas that need improved utilization management review, customer service, and quality improvement. Among those reporting being denied care, filing an appeal was not associated with lower patient ratings of plan or care.

Patients who reported denials of needed care gave low ratings to their care, and especially their plan, regardless of whether they appealed the denial. Beneficiaries satisfied with complaint resolution provided only slightly lower ratings than those with no complaints.

Medicare beneficiaries may largely attribute the actions that lead to complaints or denials to their plans more than to the care they received.

Medicare plans should focus on successful complaint resolution, as it can substantially mitigate the low ratings patients give their plan that result from whatever motivated them to file a complaint.

Patients’ complaints about their health insurance plan and denials of care are important aspects of the patient experience. The extent to which specific care is necessary can be complex and contentious.

 Denials to cover care may reflect barriers to accessing needed care or appropriate curtailing of unnecessary care. Throughout this article, we will refer to a health plan’s decision to not cover care as “denial of care” as an imperfect shorthand for the patient’s report, recognizing that patients can choose to pay out-of-pocket or out-of-network for their care. Similarly, complaints about health insurance plans may indicate a misunderstanding, a lack of plan responsiveness, or appropriate voicing of patient concerns.

Managed care organizations have delivery systems that greatly influence interactions between physicians and patients.

 Their rules and restrictions on choice of providers, network coverage, review processes, preauthorization, and drug formularies may create conflicts with patients and providers. Through the denial and appeals process, patients, physicians, and health plans communicate and determine what care is medically necessary.

As required by CMS, all Medicare Advantage (MA) contracts (health plans) must provide beneficiaries a means to file a complaint or appeal a denial of coverage.

 Denied coverage for care may adversely affect MA enrollees’ perceptions of plans and care. Nineteen percent of 2010 Medicare beneficiary disenrollments were prompted by denials of coverage for medical services,

 and over one-third of callers to the Medicare Rights Center’s national helpline expressed difficulty managing coverage denials and appeals.

Health plans may benefit from how they respond to grievances, complaints, and service failures, known as service recovery.

 found that it is not the service failure that reduces customer ratings of their plan or care, but the service provider’s actions (ie, recovery) following the failure that matter. This phenomenon is the “service recovery paradox,” wherein those who complain and see their problems resolved by the offending organization may rate the organization higher than those who experience no problems.

Patient ratings of plans and care were lower among beneficiaries filing complaints or reporting denied care. Appeals did not further predict ratings, but successful complaint resolution did.

Marc N. Elliott, PhD

Articles

The Development of a Patient-Reported Functional Limitations Index

Adjusting Medicare Advantage Star Ratings for Socioeconomic Status and Disability

Patient-Reported Denials, Appeals, and Complaints: Associations With Overall Plan Ratings

Beyond Black and White: Race/Ethnicity and Health Status Among Older Adults