Currently Viewing:

NBER's Invalidation of Wellness: Behind the Headlines

Al Lewis wears multiple hats, both professionally and also to cover his bald spot. As founder of Quizzify, he has married his extensive background in trivia with his 30 years experience in healthcare to create an engaging, educational, fully guaranteed and validated, question-and-answer game to teach employees how to spend their money and your money wisely. As an author, his critically acclaimed category-bestselling Why Nobody Believes the Numbers, exposing the innumeracy of the wellness field, was named healthcare book of the year in Forbes. As a consultant, he is widely acclaimed for his expertise in population health outcomes, and is credited by search engines with inventing disease management. As a validator of outcomes, he consults to the Validation Institute, part of an Intel-GE joint venture.
Some tourist attractions feature an “A” tour for newbies and then a “behind-the-scenes” tour for those of us who truly need lives. For instance, I confess to having taken Disney’s Magic Kingdom underground tour, exploring, among other things, the tunnels through which employees travel so as not to be seen out of costume in the wrong “Land.”

Likewise, there have been many reviews of the recent wellness study conducted by the National Bureau of Economic Research (NBER), the first-ever randomized control study of a wellness program. This, however, is the first review to go beyond the “A” tour of the headlines.

By way of background, the headline is that the mainstream wellness program the investigators examined at the University of Illinois did not noticeably move the needle on employee health. They didn’t address return-on-investment (ROI), because there obviously was none. Achieving a positive ROI would require moving the health risk needle—not just by a little, but rather by enough to significantly improve the health of many employees. Then, since wellness-related events, such as heart attacks, would not otherwise have befallen these employees immediately, this improvement would have to be sustained over several years before there was a statistical chance of some events being avoided.

Finally, the magnitude of this improvement would have to be great enough to violate the rules of arithmetic, because it is not mathematically possible to avoid enough medical events to break even on wellness. For instance, it actually costs about $1 million to avoid a heart attack through a screening program.

This finding, therefore, represents an existential threat to conventional wellness programs.

No surprise, then, that while virtually all the reviews of this study have been positive, wellness apologists have tried to discredit it 2 ways. First, they would say: “Yes, but it’s only 1 year.” That is true, but there is no reason to expect a “hockey stick” improvement in Year 2, when in Year 1 employees didn’t even increase their visits to the gym in Year 1, which would have been the easiest behavior improvement. As the lead researcher said: “We don’t see anything trending towards savings.”

Next, detractors would, ironically, say: “Everyone knows savings don’t start in the first year.” And yet the Koop Awards—the award given to the best-in-country wellness program by a group of wellness promoters that licensed the name of former Surgeon General C. Everett Koop, MD, (which had a “For Sale” sign on it later in his life)—almost invariably go to programs that show massive first-year savings by comparing active motivated participants to non-participants. In one award-winning case, those massive savings started 2 yearsi before the program was implemented.

Detractors would also say: “This study was misdesigned because they lumped nonparticipants and participants together. Obviously, nonparticipants would have no interest in this program.” That, of course, is a feature of this study design, not a bug. This study design controls for motivation, in order to isolate the program effect from the self-selection bias, such as the bias evident in that very same award-winning study.

All these negative reviews took place in LinkedIn groups that are generally closed to the public, and also to wellness critics. Despite multiple requests to comment, the Health Enhancement Research Organization (HERO, the wellness industry trade association) has offered no rebuttal, response, or even acknowledgement of the existence of this study. Perhaps it fears that its comments will draw more attention to the original, not just from trade publications, but also from right-wing or left-wing media outlets, neither of which have been supportive of workplace wellness.

The study could be criticized on 1 key point, a flaw that no detractor spotted. The 2 groups were compared on the basis of change in total cost, rather than (or in addition to) the change in the total number of wellness-sensitive medical events (WSMEs). The “gold standard” methodology of tallying WSMEs is described in my book, Why Nobody Believes the Numbers (chapter 2), in a seminal Health Affairs research study, and in HERO’s own outcomes measurement guidebook (chapter 2). It is required by the Validation Institute for outcomes validation for employee- and member-facing organizations by the Validation Institute. 

The reason to focus on WSMEs is that being screened for heart disease and diabetes, providing information on one’s diet and exercise, and having access to a gym could not be expected to reduce total costs, but rather just the cost of avoided WSMEs. Wellness programs aren’t going to get “medical costs to decrease significantly for neoplasms and digestive systems [sic], and for blood and blood-forming organs,” though a widely cited study made that exact claim. 


i The separation of participants (green line) and non-participants (orange line) took place in 2004. The program started in 2006. By the time the program started, participants had already “saved” almost $400, without even having a program to participate in.

Copyright AJMC 2006-2020 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up