Objectives: To examine the extent to which numeracy predicts consumer-directed health plan (CDHP) comprehension and health plan choice. Also, to test whether comprehension can be improved using different presentation approaches.

Study Design: We conducted an experimental laboratory study in which 303 adults viewed information about a hypothetical CDHP and a hypothetical preferred provider organization (PPO) presented in several different ways. Participants were randomized to view plan comparisons in a side-by-side or a common/unique format, and whether or not to view a framework.

Methods: Participants completed a survey that included comprehension items, numeracy and literacy assessments, and sociodemographics. Multivariate regression models were developed to examine the independent effects of numeracy and presentation approach on CDHP comprehension and choice. Interactions between numeracy and presentation approaches were tested.

Results: Although less numerate consumers understood less about CDHPs, they were substantially more likely to select the CDHP. Providing an overarching framework to highlight the differences between the CDHP and PPO boosted comprehension on items related to the framework message. However, it reduced comprehension on items that were not related to the framework, particularly among the less numerate. Participants reported that the common/unique presentation of comparative information was easier to understand, yet there was a trend toward less comprehension using that presentation approach.

Conclusions: This study highlights the difficulty many consumers have in understanding comparative plan information and in making informed healthcare choices. Findings also indicate that some presentation strategies may help the less skilled understand choices better.

(Am J Manag Care. 2008;14(6):369-376)

Consumer-directed health plans (CDHPs), which were introduced in 2000, enrolled 10% of covered employees in 2007.¹ CDHPs are high-deductible health plans with an accompanying savings account. They were designed to provide enrollees with financial incentives to contain their own healthcare costs. Enrollees in CDHPs pay for their medical care with account funds or out of their own pocket until they reach the deductible level, at which point standard insurance coverage begins.

With greater exposure to healthcare costs, CDHP enrollees are expected to make more cost-effective healthcare choices, like comparison shopping for healthcare and reducing unnecessary medical care visits. Unspent funds in the savings account may be rolled over for use in future years, providing an incentive to conserve on healthcare. Because CDHPs typically have lower premiums than traditional insurance,¹ some policymakers view them as a vehicle for reducing the number of uninsured.^2,3 Medicaid agencies also are starting to pilot a version of CDHPs with low out-of-pocket costs, called Health Opportunity Accounts.⁴

Critics have raised a number of concerns over CDHPs. These include the plans’ complexity, the potential for consumers to cut back on “essential” medical services, their role in segmenting the risk pool, and the fact that CDHPs are not designed to reduce healthcare costs for those with the highest utilization.^5-9 Some have raised the question of how vulnerable populations, such as low-income persons and racial minorities, will fare with CDHPs, due to their having less discretionary income, lower literacy skills, and more limited access to Internet-based information.^10,11

This paper focuses on consumers’ understanding of CDHP design among a largely low-income population. Because CDHPs potentially require greater out-of-pocket spending than traditional health insurance, it is important for consumers to understand differences in plan design when selecting health plans. CDHPs also are more complex than other forms of health insurance once consumers are enrolled. Depending on enrollees’ needs for healthcare, they may experience up to 4 different stages of coverage within a CDHP. In the first stage, consumers can pay for healthcare costs using the savings account (or they may opt to save the account funds for the future by paying out-of-pocket). In the second stage, which occurs if the account is exhausted, healthcare costs are not covered and must be paid out-of-pocket. In the third stage, once the high deductible is met (typically $1500-$2000 for individuals),¹ healthcare costs are covered by a health plan with cost-sharing. Finally, in the fourth stage, if an out-of-pocket maximum is reached, all healthcare costs are covered by the health plan. Enrollees are expected to use informational resources to make cost-effective decisions. This complexity may be more difficult for consumers with lower education and numeracy levels.¹²

To date, little research has examined consumers’ understanding of CDHPs. One study found that fewer than half of enrollees in a CDHP said they knew “a lot” about the plan prior to enrollment, though CDHP enrollees reported being more knowledgeable than those not selecting the CDHP.¹³ Another study found that the CDHP plan selection process differed considerably between salaried and hourly employees.¹⁴ Although some salaried employees described using computer spreadsheet programs to compare likely out-of-pocket costs in preferred provider organizations (PPOs) and CDHPs prior to enrollment, hourly enrollees more often described confusion about selecting the plan and how CDHPs work.

Two bodies of literature are related to the question of consumers’ understanding of CDHPs. The first focuses on consumer literacy (reading skills) and numeracy (numeric literacy). In the United States, many more people have low literacy and numeracy skills than in other industrialized countries.¹⁵ Based on the National Assessment of Adult Literacy, only 13% of Americans have the skills to comprehend lengthy prose or to synthesize multiple pieces of information in complex documents.¹⁶ We hypothesized that these skills, in particular numeracy, will be predictive of understanding the differences between CDHPs and conventional health plans. Prior research has found that literacy, numeracy, or a measure of health literacy combining the 2 are positively associated with similar types of health-related tasks, including consumers’ ability to extract quality information from comparative tables and confidence in making health coverage decisions.^17-19

Emerging literature has sought to improve consumer comprehension and informed decision making by simplifying the presentation of comparative information. Experimental studies have documented that simpler displays of information— those containing less information and that require less synthesis—are in general easier for consumers to comprehend. A recent study, for example, found that when consumers were provided with quality indicators along with hospital characteristics (eg, the number of hospital beds), only 40% selected the highest-quality hospital. However, when only quality indicators were provided, simplifying the data display and reducing the number of factors to consider, 62% of consumers selected the highest-performing hospitals.¹² Two other techniques to simplify information presentation (listing health providers by quality rankings rather than alphabetically, and framing indicators so that higher numeric scores indicate better performance) also substantially improved comprehension and informed decision making.^12,20-23

There is evidence that information presentation may not have the same influence on all consumers. Peters and colleagues found that using a traffic light symbol helped highly numerate consumers select better-quality hospitals. However, it reduced the ability of less numerate consumers to select the best-quality hospital.¹² The present study  builds upon this emerging area of study.

This study’s first goal was to examine the effect of numeracy on comprehension of CDHP design and informed decision making among a largely low-income population. The second goal was to test whether comprehension could be improved by varying the way the information was presented. We randomized consumers to view different presentations of the same comparative plan information to test the impact of presentation across all respondents and for those with high and low numeracy skills.

We first tested whether presenting common/unique plan attributes influenced comprehension and decision making compared with listing plan attributes side by side. Our hypothesis was that the common/unique presentation approach would require consumers to synthesize less information, and therefore would result in higher comprehension levels across all levels of ability.

We also tested the influence on comprehension and decision making of a framework that details the advantages and disadvantages of a CDHP relative to a conventional PPO. Educational theory suggests that if we provide consumers with a framework for understanding a concept or the “big ideas,” they will be more likely to understand the “little ideas” and integrate new pieces of information into the framework.²⁴ Because using a framework increases the amount of information that is presented, which may be difficult for those with lower numeracy skills, we opted to test longer and shorter versions of the framework.

METHODS
To test these questions, we used an experimental study design with a convenience sample of 303 adults age 18-64 years in Oregon. To ensure variation in numeracy and literacy skills, we recruited our sample so that approximately half of the participants would have lower levels of education (a high school diploma or less). Participants were randomly assigned to view 1 of the following 6 formats comparing the same 2 hypothetical health plans:

1. Long framework, information side by side.
2. Short framework, information side by side.
3. No framework, information side by side.
4. Long framework, common/unique information.
5. Short framework, common/unique information.
6. No framework, common/unique information.

One of the hypothetical health plans was a CDHP with a $2400 deductible for individuals, and the other was a conventional PPO with a $400 deductible. For simplicity, we refer to these plans as the CDHP and the PPO, even though CDHPs generally rely on PPO coverage once the high deductible is reached.

Side-by-side presentation comparisons are common in presenting plan choices. We tested side-by-side presentation against an alternative approach that presented common/ unique plan characteristics parsed out for the user. The sideby-side presentation compared the CDHP with the PPO on 13 characteristics, including premiums, cost sharing (office visits, urgent care, outpatient services, emergency services, prescription drugs, and preventive care visits), deductible levels, and out-of-pocket maximums. The common/unique version compared the plans on the same 13 characteristics, but first presented the 5 characteristics that were similar in the 2 plans and then presented the 8 characteristics that differed. The 2 versions are shown in eAppendix Table A and eAppendix Table B (available at www.ajmc.com).

The frameworks summarized the advantages and disadvantages of the CDHP relative to the PPO, and showed the minimum and potential maximum costs in the 2 plans. The long framework listed 4 advantages and 4 disadvantages of the CDHP, whereas the short framework listed 2 of each. In both versions, the frameworks’ key messages were that the CDHP had lower monthly premiums, but enrollees might have higher out-of-pocket costs if they used a lot of medical services. The framework is provided in eAppendix Table C (available at www.ajmc.com).

The dependent variables in this study were comprehension, plan choice, and ease of understanding information. The key comprehension measure is an index made up of the number of correct responses to the following 6 questions comparing the CDHP (Plan A) and the PPO (Plan B):

1. Which plan is better for a person who needs a lot of healthcare?
2. Which plan is better for a person who needs very little healthcare?
3. Which plan has lower monthly premiums?
4. How much would it cost with Plan B to go to an urgent care facility?
5. What would be the out-of-pocket maximum cost per year for a single person with individual coverage under Plan A?
6. What would be the out-of-pocket maximum cost per year for a single person with individual coverage under Plan B?

Each question was followed by 2-4 response options, one of which was correct. For some analyses, we also used 2 subindices of comprehension: one that included the 3 questions that directly related to the key framework messages (questions 1-3), and one that included questions that did not directly relate to the framework information (questions 4-6).

The plan choice question asked participants which plan they would be more likely to choose for themselves. The easeof- understanding question asked participants to self-report “how easy or difficult it was to understand the information about the 2 health plans” using a 7-point scale (–3 = very difficult to +3 = very easy).

All participants completed numeracy and literacy assessments. The numeracy assessment included the 11 items from Lipkus and colleagues,²⁵ and 4 additional items developed by Peters et al.¹² We used Passage B of the Test of Functional Health Literacy to assess reading literacy, which was used successfully by Gazmararian and colleagues.²⁶ Participants also provided basic sociodemographic information. The survey packet included other unrelated studies and took about 1 hour to complete on average. Each participant received $20 for completing the survey.

Analytic Approach
To examine the influence of presentation approach (side by side vs common/unique; and short, long, or no framework) by numeracy level, we first conducted factorial analysis of variance (ANOVA) tests. Factorial ANOVAs tested 3 null hypotheses: that the presentation approach is not related to the dependent variable; that numeracy is not related to the dependent variable; and that there is no difference in the relationship between presentation approach and the dependent variable depending on the numeracy level. We focused primarily on the role of numeracy because distinguishing the 2 plans largely requires comparing costs and cost-sharing levels. For this analysis we dichotomized numeracy at the median score (0-9, 10-15).

We then developed multivariate regression models. We first examined the independent influences of numeracy, literacy, and the 2 presentation approaches on each dependent variable, controlling for sociodemographic  characteristics. The health plan choice model used logistic regression because the dependent variable was dichotomous. This model also controlled for health status and the number of chronic conditions, which we expected to influence health plan choice.

We then developed multivariate models to test whether there was an interaction between the presentation approaches and numeracy. To facilitate interpretation of the interaction terms, we interacted a dichotomized version of numeracy with each presentation approach. Because there was little difference in influence of the 2 framework types in the prior analysis, we collapsed the short and long frameworks together for the regression analyses.

RESULTS
Table 1 presents the sociodemographic characteristics of this largely low-income sample of adults. Three quarters of participants reported incomes of less than $20,000 a year and only 44% were employed. Educational achievement was slightly lower than national averages.²⁷ The sample was disproportionately young (46% were under age 35 years) and white (74%). Respondents were evenly split by sex.

Participants, on average, correctly answered 4 of the 6 health plan comprehension questions (Table 1). They assessed the comparative information as slightly easy to understand (mean of 1.0 on a scale of –3 to +3). Almost half (48%) said they would have chosen the CDHP.

Numeracy was strongly related to comprehension, ease of understanding the information, and choosing the CDHP (Figure 1). Although the less numerate consumers had lower comprehension levels (3.0 vs 4.5 for consumers with higher numeracy) and found the comparative information harder to understand than the more highly numerate consumers did (0.8 vs 1.2), they were substantially more likely to select the CDHP than the highly numerate consumers (56% vs 41%).

Figure 1 examines the influence of presenting comparative data side by side versus using a common/ unique approach for individuals with higher and lower numeracy. For both more and less numerate individuals, the side-by-side presentation resulted in slightly higher comprehension levels, though this relationship was shy of statistical significance. Presentation approach was not significantly related to ease of understanding the comparative information, nor was it related to selecting the CDHP.

The framework, however, had a different impact on comprehension for the less and more numerate participants (Figure 2). For the less numerate, the framework (both long and short versions) reduced comprehension, whereas it increased comprehension for the highly numerate. Regardless of numeracy level, all respondents reported that the frameworks were more difficult to understand. The framework had no influence on plan selection.

To better understand the influence of the frameworks, we divided the comprehension index into 2 subindices. Figure 3 shows that the framework helped all respondents improve comprehension on questions related to the framework’s core message, regardless of numeracy level. However, the framework decreased comprehension among  less numerate respondents on questions not directly related to the framework’s message; it did not influence the highly numerate respondents.

The multivariate models show largely the same patterns (Table 2). After controlling for literacy, presentation, and sociodemographic factors, numeracy is predictive of greater comprehension and ease of understanding comparative information. However, those with lower numeracy still were more likely to select the CDHP. We used the regression coefficients to derive predicted probabilities of selecting the CDHP at different levels of numeracy (using individuals’ observed literacy, sociodemographic, and health characteristics). With a low numeracy score of 5, the model predicted that 59% would select the CDHP, compared with 46% and 33% at numeracy scores of 10 and 15, respectively.

DISCUSSION
In this laboratory study we found that comprehending key differences between a CDHP and a conventional PPO was challenging for consumers, particularly those with lower numeracy. Less numerate consumers comprehended less about the CDHP and reported having more difficulty understanding plan differences than those higher in numeracy; yet they were substantially more likely to select the CDHP. This suggests that less numerate CDHP enrollees are at an information disadvantage at enrollment. Furthermore, once enrolled, less numerate enrollees also may have more difficulty making the type of cost-effective decisions that CDHPs were developed to foster. This situation should be monitored as more disadvantaged consumers enroll in CDHPs as part of Medicaid pilot programs.

The findings have particular relevance in a policy environment that is relying on consumers to make informed choices to stimulate the market to improve performance. This policy approach has been strongly adopted in the Medicare program, where older adults typically face choices among 50 or more prescription drug plans as well as managed care options.²⁸ Yet when consumers make choices based on information they do not fully comprehend, it is likely to result in suboptimal outcomes for the delivery system, the payer, and the individual consumer.

This type of laboratory study has high internal validity because subjects were randomized to the different study conditions. The use of a convenience sample, however, raised the question of the generalizability of the findings. Because the cognitive processes that we studied are considered fundamental to the way that human beings process information, there is no reason to believe that the effects we observed in this study would vary widely from those with other low-income samples.

Additionally, laboratory decision making may differ from real-life decision making. In laboratory studies, participants may pay more attention to comparative information because they have allocated time to do so and they are not interrupted. More importantly, the decisions made in laboratory studies are not binding, so they may not reflect what would be decided under realistic situations.

This study confirmed that information providers (eg, employers, public agencies, health plans) can play an important role in influencing consumer comprehension of comparative information. Providing consumers with a framework that highlighted the key differences between a CDHP and a PPO improved comprehension for all consumers. It was beneficial, though, only for comprehension items directly linked to the framework’s message. This suggests that information providers (1) need to be very deliberate in developing frameworks so that they emphasize essential information and (2) use frameworks cautiously, because they may result in reduced comprehension of information not included in the framework.

Our findings also confirmed that information presentation may help one group of consumers, but not necessarily another. In this case, the framework was more consistently helpful to the high numerate. It is important that future research seeks to identify presentation approaches that benefit the less numerate.

We also found that consumers’ self-reported ease of understanding of comparative information was not consistent with comprehension. Consumers reported that the common/unique presentation approach was easier to understand, yet it did not improve comprehension. Similarly, consumers reported the framework was more difficult, but it was  helpful to consumers’ comprehension in some contexts.

The findings underscore the importance of developing and testing comparative presentation approaches with consumers of varying numeracy and literacy skills. The findings also highlight the fact that even well-intentioned and thoughtful information providers may accidentally choose presentation methods that exacerbate rather than help the problem for individuals lower in numeracy.