Published Online: November 19, 2013
Margaret M. Byrne, PhD; Christina Daw, PhD; Ken Pietz, PhD; Brian Reis, BE; and Laura A. Petersen, MD, MPH
Background: Publicly reported performance data for hospitals and nursing homes are becoming ubiquitous. For such comparisons to be fair, facilities must be compared with their peers.
Objectives: To adapt a previously published methodology for developing hospital peer groupings so that it is applicable to nursing homes and to explore the characteristics of “nearest-neighbor” peer groupings.
Study Design: Analysis of Department of Veterans Affairs administrative databases and nursing home facility characteristics.
Methods: The nearest-neighbor methodology for developing peer groupings involves calculating the Euclidean distance between facilities based on facility characteristics. We describe our steps in selection of facility characteristics, describe the characteristics of nearest-neighbor peer groups, and compare them with peer groups derived through classical cluster analysis.
Results: The facility characteristics most pertinent to nursing home groupings were found to be different from those that were most relevant for hospitals. Unlike classical cluster groups, nearest neighbor groups are not mutually exclusive, and the nearest-neighbor methodology resulted in nursing home peer groupings that were substantially less diffuse than nursing home peer groups created using traditional cluster analysis.
Conclusion: It is essential that healthcare policy makers and administrators have a means of fairly grouping facilities for the purposes of quality, cost, or efficiency comparisons. In this research, we show that a previously published methodology can be successfully applied to a nursing home setting. The same approach could be applied in other clinical settings such as primary care.
Am J Manag Care. 2013;19(11):933-939
Comparison of healthcare facilities can be done for assessment of efficiency, as a step in quality improvement, or for resource allocation purposes, to name a few reasons. Our methodology, which results in peer groups of facilities with similar characteristics, is a feasible alternative to classical cluster analysis.
The results provide managers with a scientific and flexible means of grouping their facilities for comparison purposes.
This methodology will allow for improved quality, better comparisons of efficiency, and better resource allocation, which will strengthen healthcare reform.
Given the competitive nature of the healthcare industry, as well as continually rising healthcare costs, administrators of healthcare systems are increasingly interested in evaluating the quality of care provided in their system, as well as the efficiency of resource use. To do these evaluations, administrators often compare quality of care or efficiency of performance across healthcare systems or across units within a single healthcare system. Comparisons across units within a system can be useful in determining where resources should be allocated, where incentives or bonuses might be warranted, or where quality improvement is needed. However, to ensure that comparisons across units are equitable, it is important that the units or facilities that are being compared are similar to one another (peer facilities or peers).
In the research presented here, we consider the identification of peers among nursing home facilities within the Department of Veterans Affairs (VA) healthcare system. We used a methodology that we have previously developed for identification of peers among VA hospitals, modified to accommodate the special characteristics of VA nursing homes.1 Department of Veterans Affairs nursing homes (VANHs) are quite divergent in their characteristics; thus, it is important that comparable peers be identified for equitable comparisons to be made across facilities. This can be done using a peer grouping methodology. By determining appropriate VANH peers, we can facilitate the equitable benchmarking of nursing homes in areas of quality or financial evaluation and comparisons.
Here we describe our “nearest-neighbor” methodology for developing VANH peer groupings. We then explore the characteristics of the VANH peer groups developed by using the nearest-neighbor methodology and compare them with peer groups developed by using traditional cluster analysis. We emphasis that this methodology is agnostic to healthcare setting (ie, VA vs non-VA) and can be applied to a wide variety of settings.
Development of Peer Facilities Using Nearest-Neighbor and Traditional Cluster Methodologies
We included all 130 nursing facilities in the VA in our analyses. To address the widely disparate scales of the measures that we
use, all variables are standardized by converting the measure to a z score for the population. With the nearest-neighbor methodology, the software program Clustan (Clustan Software, Edinburgh, UK) was used to calculate the general squared Euclidean distance (GSED) coefficient, which is a multidimensional distance between all pairs of facilities based on the selected variables. Clustan allowsfor a combination of both continuous and binary measures. The GSED coefficient for the distance between any 2 facilities j and i is calculated as:
dij 2 = Sk [ wijk ( Xik – Xjk)2 / Sk wijk ]
where xik is the value of variable k in case i, and wijk is a weight of 1 or 0 depending upon whether or not the comparison is valid for the kth variable. If differential variable weights are specified, it is the weight of the kth variable, or 0 if the comparison is not valid. We report the Euclidean distance, which is the square root of the GSED.
We identified the facilities most closely surrounding each reference facility in the multidimensional space, providing each facility with a customized group of similar facilities. In this way, there were the same number of peer groups as there were facilities.
For comparison with this methodology, we generated peer groups using a standard 2-stage cluster analysis technique.2 We used the same variables and data as in the nearest-neighbor approach. Ward’s method of hierarchical clustering was used to generate cluster seeds.3 We then used these seeds as input to the standard k-means iterative algorithm for cluster analysis. Based on the R2 statistics, which represent the proportion of variation in distance that is explained by the clustering, we elected to create 8 peer groups from the set of VANHs.
There are several differences between peers developed with classical cluster analysis and nearest-neighbor analysis. In classical cluster analysis, membership of a facility in peergroups is mutually exclusive, the size of the peer groups is determined by the results of the clustering, and a facility may be on the “edge” of the sole group that it is in with respect to a certain measure or measures. For example, a facility may have the largest number of beds of any facility in its peer grouping by a substantial margin. Our modification of cluster analysis addressed these concerns. With the nearest-neighbor method, the number of facilities that were designated as peers to the reference facility were selected depending on the needs of the analysis or comparison. Facilities could and did appear in more than 1 peer grouping, and each facility was by design the hub of its own peer grouping. Finally, the peer relationship was not commutative. If facility B was in facility A’s peer group, facility A might or might not have been in facility B’s peer group.
Identifying Measures to Use for Peer Groupings
A number of steps, outlined below, were taken to determine which variables to use in developing peer groups for this research. At each step in the process, we took the latest results to a panel of VA nursing home experts who provided feedback on whether our findings and conclusions mapped to their experience and observations.
Step 1. We examined literature on risk adjustment, casemix reimbursement, determinants of quality of care, and evaluations of quality improvement initiatives.4-15 We also queried our expert panel on variables that they viewed as important in categorizing nursing homes into peer groups. We included variables that were related to how many resources would be needed to run the facility or maintain a certain level of quality or treat patients; variables that characterized the facility and gave information about how costly it would be to treat patients there or about the facility’s environment; and importantly, variables related to facility characteristics that could not be easily changed by the administrators and affected by patient cost. For example, a care center in an urban area with an academic affiliation would be expected to have higher costs than a small rural facility.
We identified an extensive list of potential measures that might be important in developing peer groupings. These measures fell into the following domains, which we determined were critical for assessing similarities and differences among facilities: size, academic mission, workload/case mix, patient population, and community environment. The expert panel approved the complete list.
Step 2. We narrowed the variables to a working list to use in the peer group algorithm. For this task, we used a combination of technical and practical considerations. We gave precedence to variables that were continuous rather than binary or categorical, as continuous variables provide more refined information to use in the algorithm (technical consideration).
Step 3. We considered the availability of each variable, giving preference to more easily obtainable and more frequently updated data (practical consideration).
Step 4. For parsimony in our peer grouping model, we wanted to eliminate variables that did not add new or unique information on the nursing homes. Therefore, we estimated correlations among all of the candidate variables in each domain. Where correlations within a domain were high, we eliminated 1 of the correlated variables (mean correlation [standard deviation] of variables within same domain was 0.283 [0.218]; range, 0.005-0.910). The expert panel approved the reduced list.
Development of Measure Weights for Use in Peer Groupings
To calibrate the relative influence of the various measures, we weighted some of the model variables higher than others before inclusion in the algorithms, with a weight of 1 as the basis point. Selection of the variables to weight higher than 1 was done according to a number of considerations: salience in peer group development or nursing home literature, and information on the importance of the variable provided by our expert panel. The weights greater than 1 that were chosen are shown in Table 1 in parentheses. These weights ranged from 2, for total operating beds, to 6 for mean activities of daily living (ADLs). The expert panel concurred with the choice of weights.
Exploration of Peer Group Characteristics and Comparison With Traditional Cluster Analysis
We explored the attributes of the peer groups created with the nearest-neighbor methodology and compared them with peer groups formed using traditional cluster analysis. These analyses mirror closely those presented for hospitals as a whole in previous research.1
Nondiscrete Nature of Nearest-Neighbor Peer Groups. We present nearest-neighbor peer groups for 2 reference nursing facilities (here using 20 VANHs per group) to demonstrate the non–mutually exclusive nature of groups developed with this methodology.
PDF is available on the last page.