The American Journal of Managed Care June 2008
Comprehension and Choice of a Consumer-Directed Health Plan: An Experimental Study
Objectives: To examine the extent to which numeracy predicts consumer-directed health plan (CDHP) comprehension and health plan choice. Also, to test whether comprehension can be improved using different presentation approaches.
Study Design: We conducted an experimental laboratory study in which 303 adults viewed information about a hypothetical CDHP and a hypothetical preferred provider organization (PPO) presented in several different ways. Participants were randomized to view plan comparisons in a side-by-side or a common/unique format, and whether or not to view a framework.
Methods: Participants completed a survey that included comprehension items, numeracy and literacy assessments, and sociodemographics. Multivariate regression models were developed to examine the independent effects of numeracy and presentation approach on CDHP comprehension and choice. Interactions between numeracy and presentation approaches were tested.
Results: Although less numerate consumers understood less about CDHPs, they were substantially more likely to select the CDHP. Providing an overarching framework to highlight the differences between the CDHP and PPO boosted comprehension on items related to the framework message. However, it reduced comprehension on items that were not related to the framework, particularly among the less numerate. Participants reported that the common/unique presentation of comparative information was easier to understand, yet there was a trend toward less comprehension using that presentation approach.
Conclusions: This study highlights the difficulty many consumers have in understanding comparative plan information and in making informed healthcare choices. Findings also indicate that some presentation strategies may help the less skilled understand choices better.
(Am J Manag Care. 2008;14(6):369-376)
Comparing consumer-directed health plans with preferred provider organizations is challenging for consumers, particularly those with low numeracy skills. These findings highlight the complexity of supporting informed choices among consumers and the need to develop and test presentation formats.
- Frameworks that emphasize key conceptual information can improve comprehension across numeracy levels. However, frameworks should be used cautiously, because they may result in reduced comprehension of information not included in the framework.
- Information providers should not rely on consumers’ assessment of how easy information is to understand, as this assessment is not necessarily predictive of comprehension.
With greater exposure to healthcare costs, CDHP enrollees are expected to make more cost-effective healthcare choices, like comparison shopping for healthcare and reducing unnecessary medical care visits. Unspent funds in the savings account may be rolled over for use in future years, providing an incentive to conserve on healthcare. Because CDHPs typically have lower premiums than traditional insurance,1 some policymakers view them as a vehicle for reducing the number of uninsured.2,3 Medicaid agencies also are starting to pilot a version of CDHPs with low out-of-pocket costs, called Health Opportunity Accounts.4
Critics have raised a number of concerns over CDHPs. These include the plans’ complexity, the potential for consumers to cut back on “essential” medical services, their role in segmenting the risk pool, and the fact that CDHPs are not designed to reduce healthcare costs for those with the highest utilization.5-9 Some have raised the question of how vulnerable populations, such as low-income persons and racial minorities, will fare with CDHPs, due to their having less discretionary income, lower literacy skills, and more limited access to Internet-based information.10,11
This paper focuses on consumers’ understanding of CDHP design among a largely low-income population. Because CDHPs potentially require greater out-of-pocket spending than traditional health insurance, it is important for consumers to understand differences in plan design when selecting health plans. CDHPs also are more complex than other forms of health insurance once consumers are enrolled. Depending on enrollees’ needs for healthcare, they may experience up to 4 different stages of coverage within a CDHP. In the first stage, consumers can pay for healthcare costs using the savings account (or they may opt to save the account funds for the future by paying out-of-pocket). In the second stage, which occurs if the account is exhausted, healthcare costs are not covered and must be paid out-of-pocket. In the third stage, once the high deductible is met (typically $1500-$2000 for individuals),1 healthcare costs are covered by a health plan with cost-sharing. Finally, in the fourth stage, if an out-of-pocket maximum is reached, all healthcare costs are covered by the health plan. Enrollees are expected to use informational resources to make cost-effective decisions. This complexity may be more difficult for consumers with lower education and numeracy levels.12
To date, little research has examined consumers’ understanding of CDHPs. One study found that fewer than half of enrollees in a CDHP said they knew “a lot” about the plan prior to enrollment, though CDHP enrollees reported being more knowledgeable than those not selecting the CDHP.13 Another study found that the CDHP plan selection process differed considerably between salaried and hourly employees.14 Although some salaried employees described using computer spreadsheet programs to compare likely out-of-pocket costs in preferred provider organizations (PPOs) and CDHPs prior to enrollment, hourly enrollees more often described confusion about selecting the plan and how CDHPs work.
Two bodies of literature are related to the question of consumers’ understanding of CDHPs. The first focuses on consumer literacy (reading skills) and numeracy (numeric literacy). In the United States, many more people have low literacy and numeracy skills than in other industrialized countries.15 Based on the National Assessment of Adult Literacy, only 13% of Americans have the skills to comprehend lengthy prose or to synthesize multiple pieces of information in complex documents.16 We hypothesized that these skills, in particular numeracy, will be predictive of understanding the differences between CDHPs and conventional health plans. Prior research has found that literacy, numeracy, or a measure of health literacy combining the 2 are positively associated with similar types of health-related tasks, including consumers’ ability to extract quality information from comparative tables and confidence in making health coverage decisions.17-19
Emerging literature has sought to improve consumer comprehension and informed decision making by simplifying the presentation of comparative information. Experimental studies have documented that simpler displays of information— those containing less information and that require less synthesis—are in general easier for consumers to comprehend. A recent study, for example, found that when consumers were provided with quality indicators along with hospital characteristics (eg, the number of hospital beds), only 40% selected the highest-quality hospital. However, when only quality indicators were provided, simplifying the data display and reducing the number of factors to consider, 62% of consumers selected the highest-performing hospitals.12 Two other techniques to simplify information presentation (listing health providers by quality rankings rather than alphabetically, and framing indicators so that higher numeric scores indicate better performance) also substantially improved comprehension and informed decision making.12,20-23
There is evidence that information presentation may not have the same influence on all consumers. Peters and colleagues found that using a traffic light symbol helped highly numerate consumers select better-quality hospitals. However, it reduced the ability of less numerate consumers to select the best-quality hospital.12 The present study builds upon this emerging area of study.
This study’s first goal was to examine the effect of numeracy on comprehension of CDHP design and informed decision making among a largely low-income population. The second goal was to test whether comprehension could be improved by varying the way the information was presented. We randomized consumers to view different presentations of the same comparative plan information to test the impact of presentation across all respondents and for those with high and low numeracy skills.
We first tested whether presenting common/unique plan attributes influenced comprehension and decision making compared with listing plan attributes side by side. Our hypothesis was that the common/unique presentation approach would require consumers to synthesize less information, and therefore would result in higher comprehension levels across all levels of ability.
We also tested the influence on comprehension and decision making of a framework that details the advantages and disadvantages of a CDHP relative to a conventional PPO. Educational theory suggests that if we provide consumers with a framework for understanding a concept or the “big ideas,” they will be more likely to understand the “little ideas” and integrate new pieces of information into the framework.24 Because using a framework increases the amount of information that is presented, which may be difficult for those with lower numeracy skills, we opted to test longer and shorter versions of the framework.
To test these questions, we used an experimental study design with a convenience sample of 303 adults age 18-64 years in Oregon. To ensure variation in numeracy and literacy skills, we recruited our sample so that approximately half of the participants would have lower levels of education (a high school diploma or less). Participants were randomly assigned to view 1 of the following 6 formats comparing the same 2 hypothetical health plans:
1. Long framework, information side by side.
2. Short framework, information side by side.
3. No framework, information side by side.
4. Long framework, common/unique information.
5. Short framework, common/unique information.
6. No framework, common/unique information.
One of the hypothetical health plans was a CDHP with a $2400 deductible for individuals, and the other was a conventional PPO with a $400 deductible. For simplicity, we refer to these plans as the CDHP and the PPO, even though CDHPs generally rely on PPO coverage once the high deductible is reached.
Side-by-side presentation comparisons are common in presenting plan choices. We tested side-by-side presentation against an alternative approach that presented common/ unique plan characteristics parsed out for the user. The sideby-side presentation compared the CDHP with the PPO on 13 characteristics, including premiums, cost sharing (office visits, urgent care, outpatient services, emergency services, prescription drugs, and preventive care visits), deductible levels, and out-of-pocket maximums. The common/unique version compared the plans on the same 13 characteristics, but first presented the 5 characteristics that were similar in the 2 plans and then presented the 8 characteristics that differed. The 2 versions are shown in eAppendix Table A and eAppendix Table B (available at www.ajmc.com).
The frameworks summarized the advantages and disadvantages of the CDHP relative to the PPO, and showed the minimum and potential maximum costs in the 2 plans. The long framework listed 4 advantages and 4 disadvantages of the CDHP, whereas the short framework listed 2 of each. In both versions, the frameworks’ key messages were that the CDHP had lower monthly premiums, but enrollees might have higher out-of-pocket costs if they used a lot of medical services. The framework is provided in eAppendix Table C (available at www.ajmc.com).
The dependent variables in this study were comprehension, plan choice, and ease of understanding information. The key comprehension measure is an index made up of the number of correct responses to the following 6 questions comparing the CDHP (Plan A) and the PPO (Plan B):
1. Which plan is better for a person who needs a lot of healthcare?
2. Which plan is better for a person who needs very little healthcare?
3. Which plan has lower monthly premiums?
4. How much would it cost with Plan B to go to an urgent care facility?
5. What would be the out-of-pocket maximum cost per year for a single person with individual coverage under Plan A?
6. What would be the out-of-pocket maximum cost per year for a single person with individual coverage under Plan B?
Each question was followed by 2-4 response options, one of which was correct. For some analyses, we also used 2 subindices of comprehension: one that included the 3 questions that directly related to the key framework messages (questions 1-3), and one that included questions that did not directly relate to the framework information (questions 4-6).
The plan choice question asked participants which plan they would be more likely to choose for themselves. The easeof- understanding question asked participants to self-report “how easy or difficult it was to understand the information about the 2 health plans” using a 7-point scale (–3 = very difficult to +3 = very easy).