Michael Adelberg, MA, MPP; Austin Frakt, PhD; Daniel Polsky, PhD; and Michelle Kitchman Strollo, DrPH, MHS
Enrollees in managed care plans expect reasonable access to healthcare providers, yet “reasonable access”—a standard mandated by the Affordable Care Act—is a vague term never defined in federal regulation. Regulators measure access differently, from national standards in Medicare Advantage (MA) plans to individualistic network access plans in several states. Once a provider network is judged adequate by the regulator, the plan’s provider directory is the document that enrollees use to find providers. However, information in provider directories is often incorrect. There are 3 types of errors: a listed “in-network” provider has errant information, a provider listed as in network is not, and a provider that is in network is omitted.
A recent report from CMS found that 52% of providers in MA provider directories included at least 1 inaccuracy.1
This MA error rate is consistent with those in exchange and California Medicaid directories.2
In the last few years, significant regulatory actions have been taken against a handful of health plans with patterns of inaccurate provider information. In the last year, lawsuits were filed against 3 health plans by consumers who allege that they selected their health plan based on false provider network information.3-7
Health plans struggle with provider network accuracy for a number of reasons. Providers are frequently indifferent to keeping directories current: The unglamorous task of notifying health plans of changes is often delegated to junior office staff or no one at all. The patchwork of regulatory definitions and standards for directories is another problem. Although some promising state- and vendor-led initiatives are now underway (see eAppendix A
[eAppendices available at ajmc.com
]), regulators are uncoordinated on definitions, requirements, and oversight approaches.
Machine-Readable Provider Directories
Provider directories are commonly posted on the internet in PDF or other “flat file” formats that defy easy downloading, aggregation, or analysis. In 2013, the state of California began requiring machine-readable (MR) health information. A number of newly established state-run health insurance exchanges required the use of MR provider directories in 2014, and CMS picked up the requirement for the federally facilitated exchanges in 2016.8,9
By requiring health plans to post their provider directories in a common MR format, directories can be easily downloaded to assess network adequacy against regulatory standards or network breadth among competing plans. In addition, MR directories can also populate physician finder consumer tools and improve provider data accuracy by flagging cases in which a provider is inconsistently listed across data sources. Here, we present data on the accuracy of MR directories in comparison with other sources of provider information.
The study was conducted using quantitative and qualitative analyses. Data were collected and analyzed through quantitative research to compare accuracy among MR provider directories on the exchanges, traditional flat file provider directories in MA plans, and additional sources of provider information. Interviews were simultaneously conducted with relevant federal and state officials and key industry stakeholders to capture their knowledge of and experience using both types of provider directories. This study did not require institutional review board review and approval.
Quantitative research questions.
Our quantitative research questions were (1) “How do electronically available data sources vary? Are the same data reported across sources?” and (2) “How consistent are the provider data found in electronic sources with information gathered from phone validations conducted with provider offices?”
A secondary research question (“What are the differences in level of effort [time and cost] to aggregate MR files versus other data types to conduct analyses of provider networks?”) is discussed in eAppendix B
We selected 5 US counties across the country with insurance carriers that offer both MA and exchange health plans. We selected these counties based on geography, market penetration, and the presence of an insurer serving both the MA and exchange markets.
We downloaded and aggregated 4 separate sources of provider data—CMS’ National Plan and Provider Enumeration System (NPPES) file, conventional MA directories found on health plan websites and other online sources, exchange MR directory files, and Google Places—into a single analysis database (Table 1
MA file compilation required 5 manual processes to download into the analysis database. Data quality issues such as misspellings and small inconsistencies in addresses were solved manually. The exchange data were in a uniform and standard format, but we still experienced some data challenges, which were solved via code and automation with little manual intervention.
We compiled the data into an analysis database, removing inconsistencies (eg, spacing and capitalization) and assigning each reported element to a category (ie, address or phone number) for validation and analysis across data sources. The analysis database contained all known information.
Summary of analysis methods.
Throughout the data aggregation process, we tracked time and effort in order to assess the differences among the data sources by recording the time it took staff to complete each step to move the data into the analysis database (see eAppendix B).
After all of the data were compiled into a single database, data were analyzed via queries and text matching. For the text matching, human coders examined text similarity and assigned them into categories of like and unlike data. Because providers can practice in multiple locations, our analysis focused on whether 2 data sources reported at least 1 similar value.
Finally, we validated the accuracy of the electronically reported elements based on phone interviews with provider offices. We drew a random sample of more than 50 unique providers per specialty based on the primary specialty reported in NPPES. We contacted the provider’s practice address to validate each reported element in the analysis database.
We conducted phone interviews with representatives of CMS, state regulators, health plans and trade associations, and other experts to determine whether these stakeholders believe that MR directories facilitate more accessible and reliable information than conventional directories. Interviews were conducted with 4 CMS officials from 2 different program components, 5 senior state regulators from 4 states in different regions of the United States, 6 staff from 3 health plans that offer plans in the health insurance exchanges and MA, 3 health plan trade association staff from 2 associations, and 3 vendors of provider data accuracy solutions. These numbers do not include 2 individuals who declined to be interviewed.
Participants were provided with discussion topics in advance of the interview. Interviewees were offered confidentiality in exchange for their candor but no other compensation for their time.
We found widespread inaccuracy in provider information across directory types. We also found that provider directories from insurers with MA and exchange plans did not report the same phone number 50% of the time and did not list the same address 31% of the time. As displayed in Figure 1
, provider directory addresses have a 30% inconsistency rate when a common provider is in the same company’s health plan networks across markets.
We also analyzed provider information accuracy through phone validation by calling provider offices. During these calls, we successfully validated 80% (2850 of 3562 calls) of information attributes attempted. As summarized in Figure 2
, although all data sources contained inaccuracies and differences were sometimes slight, 2 findings can be implied: (1) Google is more accurate than provider directories or the federal NPPES file for name, address, and phone number (statistically significant [P
<.01]); and (2) despite the advantages of MR directories, exchange provider directories are less accurate than conventional MA directories.
The higher inaccuracy rate of exchange provider directories was unexpected given the advantages of MR technology to improve accuracy. This is not necessarily an indictment of MR. The high inaccuracy rate of exchange provider directories is likely because MA plan sponsors (facing oversight from CMS) are investing resources in raising the accuracy of MA directories, whereas there is no equivalent pressure to improve directory accuracy in the exchanges.
Interviewees—whether they were from health plans, government agencies, or provider data vendors—all understood that provider directories are frequently inaccurate, but they differed in their perceptions of the problem and potential solutions. Interviewees who were familiar with MR directories understood their advantages over conventional directories, but only some acknowledged their potential value in improving accuracy. Contrasting responses are offered in Table 2
The information contained in provider directories is inaccurate across information types and markets (ie, the inaccuracy rate of provider addresses ranges from 27% to 35%, and the inaccuracy rate of provider phone numbers ranges from 25% to 48%). Although the facts are straightforward, the reasons behind them are complex: As noted by interviewees, providers often treat maintaining current directory information as a low priority; there is a lack of consistent standards or a common data dictionary for provider information; there is no central, reliable information source (“source of truth”) against which to assert accuracy; and there is no harmonized federal strategy to address the problem. As argued by CMS in its 2020 Call Letter, health plans cannot solve this inaccuracy problem on their own.10
As noted, promising initiatives are now underway in a few states, and a few vendors are now offering promising accuracy tools (see eAppendix A), but we will not know their results for years. In the interim, MR directories offer great advantages over conventional directories, including crowdsourcing to identify information that is likely erroneous and quick data aggregation for ongoing network analyses. MR directories are mandated in the health insurance exchanges and Medicaid, but not federally enforced. They are recommended as a best practice in MA, but there is no evidence that they are widely used by MA health plans.11
The loose regulation of MA directories contrasts with CMS’ affirmative regulation of this market in most other respects. CMS Deputy Administrator Demetrios Kouzoukas warned MA organizations about the need for directory accuracy at the May 10, 2018, CMS Medicare Advantage and Part D conference, but without recommending the use of MR directories.12
The lack of a harmonized position on MR provider directories across markets merits further consideration. CMS’ recent requirement for hospitals to post MR hospital pricing information “to further improve the public accessibility of charge information” demonstrates that it values the technology.13
Our analysis focused on whether MR directories can result in more accurate information on network providers. It did not focus on whether MR directories can be used to lessen the instances in which in-network providers are omitted or out-of-network providers are included. More research is needed on these topics.
Fully utilize MR directory advantages.
For CMS and other entities requiring MR directories, it is incumbent to utilize the advantages of machine readability. CMS required exchange plans to invest in machine readability and endure a bumpy rollout, but it has not yet leveraged all of the considerable benefits of machine readability. The advantage of the technology can be utilized in a nonpunitive manner by having CMS and health plans partner to improve directory accuracy.
Watch for emerging best practices.
Regulators and researchers should analyze state and vendor initiatives to improve provider data accuracy. California’s statewide provider network utility and New Hampshire’s use of claims data to determine actual provider network are particularly interesting initiatives. These and other potential long-term accuracy solutions are summarized in eAppendix A. They may subsume and surpass the advantages provided by MR directories.
Clarify federal role.
Federal policy makers should consider benefits of federal leadership in correcting provider directory inaccuracy. Our analysis suggests that Medicare’s NPPES file is less accurate than health plan provider directories. This required Medicare data source could be reimagined to become a source of provider information accuracy. More broadly, national and transmarket efficiencies could be realized by establishing a national data dictionary and requirements across markets. In this regard, initiatives by HHS’ Office of the National Coordinator for Health Information Technology and CMS merit watching, including a recent proposal to require MA and Medicaid health plans to make provider directories available in a common electronic format equivalent to MR.14,15
MR directories offer significant advantages over conventional directories. As noted in eAppendix B, data can be downloaded at roughly $0.01 per provider from an MR directory compared with $2.15 per provider from a conventional directory. This efficiency makes provider network aggregation and comparisons feasible for the first time. This, in turn, powers the potential of MR directories to improve the transparency and accuracy of provider information. However, machine readability does not correct inaccuracies by itself. We found that MR exchange plan directories are slightly less accurate than conventional MA directories. This is likely because MA plans (facing oversight from CMS) are working to raise the accuracy of MA directories, whereas there is no equivalent pressure to improve directory accuracy in the exchanges.
The authors thank Timothy Riddle, MSMIS, MBA, of NORC; Kacey Stotler, MSW, of Faegre Baker Daniels Consulting; and Tricia Beckmann, JD, of Faegre Baker Daniels Consulting, for their work on this project.