Caroline S. Carlin, PhD; John Kralewski, PhD; and Megan Savage, BS
The increased focus on use of market forces to contain costs and improve quality of healthcare has raised questions about the ability and willingness of consumers to make informed choices in this complex area. Significant variance in the cost and quality of care provided by physicians has been well documented,1-3
offering the potential for real choice on the part of patient-consumers and scope for the usefulness of cost and quality information. In an effort to bring these market forces to bear, policy makers have been encouraging the provision of cost and quality information, and the use of increased patient cost sharing to heighten awareness of cost differences.
For quality information to influence physician choice, (1) the patient must be aware of and have access to the information, (2) they must trust the information source, (3) it must be understandable, and (4) the information must be salient to the choice. Because of measurement issues, quality information has historically been focused at the health plan and hospital level, but physician-level information is becoming increasingly available,4
typically through the Internet. Despite its increasing prevalence, consumer awareness of physician quality information remain slow; the most recent update of the Kaiser Family Foundation survey on the topic5
found that only 12% of consumers were aware of a source of physician quality information. Others have found that employer communication efforts are effective in increasing awareness of provider quality information in employment-based insurance.6
Consumer trust of the information and the sponsors is an important issue. Patients consistently express strong trust in recommendations from personal sources, such as family or friends,7-9
with formal sources of quality information and professional recommendations ranked below personal sources. Of the formal sources of quality information described, Alexander et al8
found that 37% of consumers “trust information a lot” from a health insurance company, with an additional 44% expressing some trust in insurance companies as a source of quality information, expressing significantly more trust for insurers than for employers or government agencies. Because health insurers have access to both contractual costs and the ability to evaluate processbased measures of quality, they are poised to leverage this trust in their covered population.
There is a rich literature examining the impact of format on the use of quality information and the consumer’s ability to comprehend the information presented. Simmeasures such as star rankings are preferred.10,11
Providing a definition of quality and context for the quality information have also been found to be important,12
improving saliency and the value consumers place on the information. Additionally, providing cost and quality information simultaneously helps the consumer avoid interpreting higher costs as a signal for quality,13
improving the consumer’s ability to accurately identify high-value providers.
The interplay of cost incentives and quality information is key to the use of market forces to improve value in healthcare delivery. Harris14
found that privately insured respondents would accept a more restrictive provider network in exchange for higher quality of care, but suggests that the level of quality required for patients to accept some of the restrictions is probably unattainable. For example, disallowing self-referral to specialists required huge percentage point improvements in quality measures to compensate for the change, such as the fraction of patients extremely satisfied with their involvement in care decisions (+57%) or fraction of physicians rated excellent by medical experts (+52%). More recently, a stated choice experiment by Sinaiko9
looked at the combined impact of cost and quality information. This work suggests that copay differences can induce a modest amount of movement to a preferred physician tier, unless the patient has received a personal or professional referral for a physician in a nonpreferred tier.
Our study was designed to build on this previous work by analyzing the influence of a health plan–sponsored physician rating program on physician selection. In order to maximize the saliency of cost and quality information, we focused our attention on patients who were searching for an orthopedic surgeon for hip or knee replacement surgery in the months after this new Internet-based information source was publicized by both the health plan and the local newspaper. The information combines cost and quality information in a simple star rating, minimizing cognitive demands. This jointreplacement population was of interest, as these are often new patient–physician relationships and the procedure is rarely emergent, thus patients were more likely to search for and be influenced by cost and quality information.
Physician Rating Program
Our study evaluated the use of a physician rating program developed by a health plan that provided physician cost and quality of care measures to their enrollees during 2010. Physicians received 1 star if they were above the national mean for quality, and a second star if they were also below the local benchmark for costs. Of the 9472 doctors rated, 20% received 1 star and 49% received 2 stars. The following is an excerpt of the health plan’s description of the program: “Evaluation for quality compares a physician’s observed practice to a national rate among other physicians who are responsible for the same interventions. Evaluation for cost efficiency compares a physician’s observed episode costs to the risk-adjusted costs of their peers in the same specialty and market… Physicians must first be designated for quality in order to be designated for cost efficiency… Quality and cost-efficiency evaluations both incorporate adjustments for case mix and severity of illness where appropriate.”
Our sample included patients who had knee or hip replacement surgery shortly after the physician rating program was publicized. The first 400 patients from each category (hip/knee) were selected by date of service starting March 1, 2010. All of these enrollees received their services during 2010 three months after the rating program was initiated. The mailed surveys included a $2 bill to encourage participation; no follow-up with non-respondents was possible because the responses were anonymous. This project was reviewed and approved by the Western Institutional Review Board.
The survey questionnaire included the following questions:
Are you aware of the physician scorecard ratings on cost and quality performance provided by your health insurance plan?
If so did you use this information when selecting the surgeon for your knee/hip surgery? (This was specific to each respondent's group.)
If you did not use the physician rating information, how did you select your physician? (Family/friend information, physician referral, physician location, network restrictions, physician gender, language spoken by physician, or other; open response for specification of “other.”)
If you were offered a copay reduction to switch to a lower cost physician of equal or higher quality would you do so?
Do you have access to the Internet at home?
Do you have access to the Internet at work?
Response rates were adequate at 60%, and were fairly consistent across demographic groups, ranging from 55% to 67% when subdivided by age, sex, urban status, and procedure type. The only demographic group that dropped below a 55% response rate was the Medicaid population; the 36% response rate dropped the Medicaid population from 9% to 5% of the respondents. This lower response is not surprising given the more transient residential status of a Medicaid population.
From the health claims data, the health plan’s informatics team collected the following information to merge with the survey results: age, sex, procedure type (hip/knee), urban/rural indicator, type of insurance (commercial, Medicare, Medicaid, or Medicare/Medicaid dual eligible). In addition, the informatics team included the median education and income level in the respondent’s census tract, based on the 2010 US Census, stripping identifiers before delivering the data set to the research team. To preserve the deidentified status of the data, median income in the census tract was rounded to the nearest $500.
Median education was measured using the percentage distribution of residents in the census tract among 3 categories: no high school degree; high school degree but less than 4-year college degree; and 4-year college degree or higher. The extreme categories, no high school and 4-year degree or higher,were highly skewed. We parameterized census tract education as a trinomial variable to capture whether a census tract was in the tail of the skewed variables or not, selecting the following 3 categories: (1) high average education—more than 50% of the residents in the census tract have a 4-year degree; (2) low average education—more than 10% of the residents in the census tract did not achieve a high school degree; and (3) baseline average education—at least 10% have a high school degree and fewer than 50% have a 4-year degree.
Summary statistics for the independent and dependent variables are shown in Table 1
. There is a tendency for the average respondent to be female (64%), live in an urban area (66%), and have access to the Internet (69%). With an average age of 68 years (range 23-93), the bulk of the respondents (64%) are covered under a Medicare plan, which is to be expected given the nature of the surgeries. The majority of respondents live in census tracts that are in the baseline education category, with 28% in low-education census tracts and 19% in high-education census tracts. To capture income, we generated a categorical variable that indicated the quartiles of the census tract median income, with break points rounded to $47,000, $57,000, and $71,000.
Awareness of the new scorecard was still quite low, at 11%, and a small minority of these (only 7 respondents) said they used the scorecard information in the selection of their surgeon. There does appear to be a significant fraction of the population that has some price sensitivity, with 30% of the respondents indicating some willingness to switch physicians for a lower copayment, assuming quality doesn’t decline.
Although the survey asked the respondent to indicate a single factor influencing their choice of surgeon, a fraction (22%) of respondents checked more than 1 factor. When there were multiple responses, we randomly selected 1 response for our analysis. Very few respondents indicated that physician gender, physician language, or physician network status was important in their selection process, so we merged these respondents into the “other” category. Despite our efforts to find participants who were making a new physician selection, 10% of the respondents indicated in the open response that they had had a previous experience with the surgeon. Therefore, we identified this as a new category, resulting in these final response categories: family/friend information (29%), physician referral (37%), physician location (13%), previous experience (10%), and other (11%).
Regression Analysis and Results
Because so few actually used the scorecard, we did not investigate what characteristics were associated with scorecard use. Therefore, we focused on awareness of the scorecard, willingness to switch physician for a financial incentive, and the selection process. The first 2 questions are dichotomous yes/no answers, modeled as a probit equation. We used multinomial logit to model the physician choice process. Both probit and multinomial logit estimations were completed using Stata 12. We explored nesting in physician choice, but found no improvement in fit or evidence of nesting in the choice process. We also tested methods15 of accounting for potentially endogenous variables (eg, access to the Internet and awareness of the scorecard), but found no evidence of bias due to endogeneity.
When modeling awareness of the physician scorecard (Table 2
), we found that the participant characteristic that has the greatest statistical significance is access to the Internet. Holding all other characteristics at their true values, moving the population from “no Internet access” to “Internet access” nearly doubled the average predicted probability of awareness of the scorecard, leading to the +6.5 percentagepoint (pp) marginal effect (ME) shown in the second column of Table 2. We also identified marginal statistical significance in the census tract’s median income. When we computed the marginal effect of moving the population from the lowest to highest median income quartile, we found a +9.1 pp ME, more than doubling the probability of being aware of the scorecard.
Willingness to change surgeons (Table 3
) appears to be sensitive to respondent’s sex, their access to the Internet, and the education level in their census tract. Females were much less likely to be willing to switch; the average predicted probability of being willing to switch drops by a –9.9 pp ME when moving from a male to a female status. Those who live in a highly educated census tract were much less willing to switch; moving from a baseline education to a highly educated census tract caused the willingness to change surgeons to drop by a –14.4 pp ME. In contrast, those with access to the Internet were more flexible in provider choice. Moving from “no access” to “Internet access” generated a +8.5 pp ME.
Two of our respondent characteristics appear to have significantly impacted the physician selection process (Table 4
): rural/urban residence and access to the Internet both have a χ2 P value less than .05. The marginal effects in Table 5
show us that those who live in an urban area were much less likely to rely on referrals from families and friends (–11.2 pp ME), and much more likely to rely on physician referrals (+15.6 pp ME). Those with access to the Internet were much less likely to rely on the physician’s location in their selection (–12.5 pp ME), with an increased reliance on previous physician experience (+5.4 pp ME) and other factors (+5.5 pp ME) to influence their choice.
It is interesting that we found no predictive value of age or type of insurance product (commercial, Medicare, Medicaid) when modeling the 3 dependent variables—awareness of the scorecard, willingness to switch providers for a lower copay without sacrificing quality, and the factors influencing choice of surgeon. On the other hand, Internet access, a variable rarely included in this literature, was strongly associated with all 3 dependent variables. It is possible that this is a causal factor—the greater informational resources that come with access to the Internet may alter the choices and attitudes of the respondent. With Internet access, the consumer can more easily find cost and quality information,4
as well as search for other, non-clinical quality dimensions such as practice size, presence of urgent care facilities, and hospital affiliation. This suggests that improving access to the Internet, or reducing search costs in other ways, such as employer-facilitated provision of quality information,6
may improve awareness and use of quality information, leading to increased patient-driven market forces. Alternatively, Internet access may be a proxy for (latent) respondent characteristics such as generalized differences in the way they engage with their environment. This is an area that deserves additional investigation. If it is a causal factor, this has great implications for the impact of market forces as Internet access increases in prevalence, particularly through mobile devices.
This study is unusual in its inclusion of rural versus urban status, which we find is an important factor in the source of informationused by the respondent in physician selection. The greater importance of family and friend referrals in rural locations implies stronger reliance on social fabric in rural areas, while the urban respondents had a much greater tendency touse professional referrals in their choice of surgeon. This suggests that, as we seek to increase the impact of market forces to drive improvements in cost and quality, different strategies may be needed by geography.
By selecting first observances of a claim for hip and knee surgery, this project was designed to highlight instances of orthopedic surgeon selection when an existing relationship is unlikely. This was intended to maximize the probability that quality and cost data provided by the health plan would be salient, in a setting where the patient had time for research, physician cost and quality rating information had been recently publicized, and the data were easily accessible on the Internet. Yet only 11% of the respondents indicated that they knew about the physician rating program, consistent with surveys of awareness in a more general population.5
Since nearly 70% of the respondents indicated that they have access to the Internet at home or office, the media for transmission of the cost and quality information does not appear to be the primary limiting factor. Given these conditions, we must conclude that either the health plan enrollees in this study do not place a high value on cost and quality data when selecting their surgeon, or that the message regarding the availability of the information was ineffective.
The availability of “product quality” information is a key feature of a functioning market. Therefore, if healthcare cost and quality improvements are to be driven by market forces, patient awareness and use of physician quality information are necessary. This work identifies factors that are important in patient awareness of quality information, and explores the physician selection process more broadly. Our results suggest that easy access to information through the Internet is a key component of awareness of quality information and willingness to change providers, and reduces the patient’s reliance on proximity to physician. This implies that other efforts that reduce search cost may be important, such as employer provision of information orstate-based public and private initiatives, such as those tied to the Robert Wood Johnson Foundation’s Aligning Forces for Quality initiative. We also find that cultural differences, here captured in rural versus urban residence, should influence communication strategies as they influence which factors are important to decision makers.
Because we focused on orthopedic surgery, the population was fairly old (average age 68.6 years), so these results may not be generalizable to a younger population. Age and type of coverage (Medicare vs commercial insurance) were not found to be statistically significant in any model, but there may be a subtle impact not detected. To capture a more age-diverse population, we originally attempted to survey new mothers about their choice of pediatrician, because these were also new relationships formed in a non-emergent setting, but—perhaps due to the demands of new motherhood—the response rate for this population was abnormally low (under 10%) and the data for new mothers had to be excluded.