Hsueh-Fen Chen, PhD; J. Mick Tilford, PhD; Robert F. Schuldt, MA; and Fei Wan, PhD
We appreciate the opportunity to address the RTI team’s main concerns of our study. Our paper demonstrated that CMS Hierarchical Condition Categories (HCC) risk scores are related to patient experience in home health.1
These findings suggest that HCC risk scores should be considered when evaluating the performance of home health agencies. However, the RTI team raised 2 concerns regarding the validity of agency-level HCC risk scores. First, HCC risk scores are prone to upcoding by providers across the spectrum, resulting in an upward bias. Statistically, upcoding will change the HCC risk score but not the SD. Our findings are based on the SD of HCC risk scores and therefore unlikely to be affected by upcoding. Secondly, the RTI team indicated that agency-level HCC risk scores likely include outliers, whereas the random sampling used in the Home Health Care Consumer Assessment of Healthcare Providers and Services (HHCAHPS) is unlikely to be affected by outliers, due to the exclusion of patients with certain conditions.2
The exclusion criteria are not based on the HCC risk scores. Additionally, most patients treated by agencies still remain in the sampling pool, regardless of their outlier scores. Theoretically, a randomizing process that selects patients from each agency will yield the characteristics of the sample that are similar to those of the patients cared by the agency. Thus, the outliers at the agency level of HCC risk scores and the outliers of HCC risk scores from the randomized HHCAHPS sampling should not differ significantly.
The RTI team also identified 2 methodological issues: (1) the reporting of excessive correlation as it relates to the authors’ combined use of state fixed effects and race variables and (2) our use of controls for agency profit/nonprofit status and term of certification by Medicare and how these factors are reflected in the agency’s public reporting. The correlation of most of our independent variables was less than 0.20, with the exception of ownership and tenure years with CMS. Therefore, the correlation in our study is not a concern (Table 1
). States vary in racial composition, but the correlation between racial/ethnic composition and state fixed effects is not our primary focus. States certify and regulate home health agencies and through these policies affect the practice of home health agencies.3,4
Additionally, agencies with different ownership and CMS tenure years respond to public reporting differently.5
Thus, controlling for those confounders (ie, state fixed effects, ownership of agency, and number of tenure years) is necessary. We reanalyzed the data by excluding ownership, number of years certified by CMS, and state fixed effects and then estimated the association between HCC risk scores and patient experience (Table 2
). The coefficients in the model without those variables remain significant and are larger than those in our original paper.1
For the reporting issue about the effect size, the RTI team recommended 3 levels using a small size (difference of 1 point), a medium size (3 points), and a large size (≥5 points). Our presentation is based on the association between the change of 1 SD in HCC scores and the change in the percent patient experience. We believe that we should provide the association instead of arbitrarily deciding a small, medium, or large effect size for the readers.
Finally, although the RTI team is unable to adjust HCC scores for patient experience due to limitations of the data and time constraints, our findings, at least, provide evidence for stakeholders that HCC risk scores can influence patient experience at the agency level.