Currently Viewing:
Currently Reading
Consumer Rating Systems for Hospitals May Use Faulty Data Sets and Metric
October 13, 2016 – Jackie Syrop
State-Level Insurance Reform Improved Access for Patients With CRC in Massachusetts
October 06, 2016 – Surabhi Dangi-Garimella, PhD
PanCAN's Precision Medicine Trial Designed for Collaboration and Personalized Care
October 04, 2016 – Surabhi Dangi-Garimella, PhD
What We're Reading: US Healthcare System Ranked 50th of 55 in Efficiency
October 04, 2016 – AJMC Staff
Best, Worst Cities for People With Disabilities Based on Health, Cost, and Lifestyle
October 04, 2016 – Laura Joszt
Supporting Population Health and Reducing Health Inequalities
October 03, 2016 – Jackie Syrop
This Week in Managed Care: October 1, 2016
October 01, 2016
ICER Recommends Significant Discount on WAC for Checkpoint Inhibitors in NSCLC
September 30, 2016 – Surabhi Dangi-Garimella, PhD
New Anticancer Drugs Associated With Big Increases in Costs, but Also Life Expectancy
September 30, 2016 – Jackie Syrop

Consumer Rating Systems for Hospitals May Use Faulty Data Sets and Metric

Jackie Syrop
More transparency and validation is needed for consumer-based benchmarking methods, such as US News & World Report's Best Hospitals rankings, according to a new study.
More transparency and validation is needed for consumer-based benchmarking methods, according to a new study in Joint Commission Journal on Quality & Patient Safety. The study found discrepancies in data and high false-positive event rates, problems that especially affect high-transfer and high-volume hospitals.

After Rush University Medical Center in Chicago received the lowest ranking (1 out of 5) for patient safety in the 2015-2016 US News & World Report “Best Hospitals” rankings, Bala Hota, MD, MPH, and coauthors at Rush University Medical Center compared the Patient Safety Indicators (PSIs) used by US News with findings derived from Rush University Medical Center’s own internal billing data as well as sensitivity analyses using a simulated data set derived from the Healthcare Cost and Utilization Project state inpatient data sets.

The researchers reported that discrepancies were found for PSIs for pressure ulcer rates, perioperative hemorrhage or hematoma rate, and postoperative respiratory failure rate in the US News & World Report’s data compared with Rush University Medical Center’s data: an excess of 0.72, 0.63, and 0.26 cases of these three PSIs, respectively. The sensitivity analysis, which included missing present on admission (POA) flags and dates, resulted in an increase of rates by 1.83 (pressure ulcer rate), 2.72 (perioperative hemorrhage or hematomas), and 3.89 (postoperative respiratory failure) PSIs.

“People have argued and theorized that if you take care of a lot of very sick people or have a lot of transfers, then the data could be biased,” Hota said in a statement. “This study proves that. It shows that data is not perfect, and misunderstandings in the data can negatively portray a hospital.”

The researchers noted that their study’s findings as well as concerns raised by others resulted in US News & World Report in 2016 making changes to its methodology and data sources for the future issues of the magazine’s Best Hospitals rankings. The 2016-17 US News & World Report’s hospital ranking issue, released in August 2016, stated that the rankings help patients make more informed healthcare decisions, comparing nearly 5000 medical centers nationwide in 25 specialties, procedures, and conditions.

According to Becker’s Hospital Review, the new US News rankings methodology places less emphasis on the Agency for Healthcare Research and Quality’s (AHRQ) PSIs when assessing hospitals for quality. Other changes include exclusion of the PSI measure for pressure ulcers when assessing a hospital for safety because of billing inaccuracies related to this PSI, and a change in which the patient safety score’s weighted influence on overall quality will be reduced from 10% to 5%. The 5% weight reduction from PSIs will be offset with added weight to risk-adjusted survival, increasing this measure’s significance from 32.5% to 37.5%.

Hota believes that the rules used by hospital ratings systems are too complex for most consumers to navigate, which is why he and his colleagues urge ratings systems, such as US News to be more transparent about hospital quality ratings. In the meantime, Rush plans to continue measuring, tracking, and sharing patient safety data.

In an editorial that accompanied the study, authors David M. Shahian, MD, Elizabeth Mort, MD, MPH, and Peter J. Pronovost, MD, PhD, concluded, “Just as healthcare providers have ethical and moral responsibilities to the public they serve, rating organizations and journalists that grade providers have similar obligations—in their case, to ensure measure validity and methodological transparency.”

Copyright AJMC 2006-2020 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up