ACA Network Regulatory Filings Are Inaccurate, Poorly Match Provider Directories

ABSTRACT

Objectives: Access to mental health services has been shown to be particularly inadequate, with limited understanding of the efficacy of existing network adequacy regulations. State and federal regulations mandate insurance carriers to submit regulatory filings to help maintain network adequacy compliance, but the accuracy of these data remains unassessed.

Study Design: We employed a secret shopper survey to verify regulatory filings and assess the congruence between the filings and provider directory listings as well as appointment availability and wait time for 8306 mental health counselors submitted by all carriers participating in Pennsylvania’s Affordable Care Act (ACA) Marketplace for plan year 2024.

Methods: Descriptive analyses, with tests of proportion and t tests to assess differences between carriers and between adult and pediatric provider specialties.

Results: A total of 19.9% of filed regulatory listings (n = 1649) were not present in consumer-facing provider directories, and only 35.3% of filed listings (n = 2928) fully matched provider directory entries. Of the 2152 provider listings we were able to verify fully via secret shopper calls, 65.2% (n = 1404) exhibited at least 1 inaccuracy. Inaccurate phone number was the most common issue (56.6%; n = 1219). Appointments were available for only 321 of the 2152 providers (14.9%), with a mean of 33.2 days lapsed between call and scheduled appointment time. Although we identified substantial differences in appointment wait times by carrier, we found no difference between adult and pediatric providers.

Conclusions: ACA network adequacy assessments that rely on carrier regulatory filings and/or consumer-facing directories substantially overestimated provider availability and access to mental health services.

Am J Manag Care. 2025;31(9):In Press

Takeaway Points

Using a large secret shopper survey of mental health counselors available in Pennsylvania’s Affordable Care Act Marketplace, we found the following:

• Carrier regulatory filings often did not align with consumer-facing provider directories;
• Carrier regulatory filings were often inaccurate; and
• Few appointments were available, and wait times averaged more than 33 days.

Our findings add to concerns about timely access to mental health services as well as the use of regulatory and consumer directories to assess and monitor network adequacy.

Despite landmark policy changes to cover mental health services on par with medical and surgical benefits, insured consumers continue to face barriers in accessing mental health services.^1-3 Consumers are more likely to seek care outside their insurance networks for mental health concerns compared with physical health concerns, leading to out-of-pocket costs, care delays, or forgone treatment.^4-6 In light of these gaps, recent federal and state network adequacy regulations have sought to improve monitoring and oversight of consumer access to health care, including mental health services.^7-11 Regulators across all markets assess network adequacy, in part, by requiring insurance carriers to file detailed data, often yearly, of their provider networks, including the name, address, and specialty of each participating provider.^8,12

One important contributor to the access challenges facing mental health consumers may be that neither existing regulatory filings nor provider directories reflect real-world provider availability and accessibility. That is, the data upon which network adequacy assessments are generated may not accurately capture consumer access challenges. Carriers face substantial logistical and administrative difficulties in monitoring their provider networks,^2,5,13-16 with data management challenged by frequent organizational or workforce changes.¹⁷ Given the current regulatory environment, carriers also have few incentives to ensure a large degree of accuracy in their filings.^7,18 Thus, data inconsistencies may disguise the true extent of network inadequacies and render current oversight efforts ineffective.^7,19

To our knowledge, however, no prior study has directly evaluated the accuracy of network regulatory filings. To assess this important source of network information and whether consumers can access care in a timely manner, we first compared these filings with online consumer-facing provider directories (hereafter referred to as provider directories) for all Pennsylvania Affordable Care Act (ACA) Marketplace carriers in plan year (PY) 2024.¹² We subsequently verified the filings via a secret shopper survey to assess their accuracy as well as appointment availability and wait times.

METHODS

Data

Carriers participating in the ACA Marketplaces are required to “maintain a network that is sufficient in number and types of providers.”¹² To ensure compliance with all regulatory requirements and standards, carriers must provide an overview of their networks across markets and states.^8,20 In Pennsylvania, ACA carriers are required to submit this information yearly to the Pennsylvania Insurance Department (PID) to obtain network approval, with subsequent quarterly, more limited submission for monitoring purposes.¹² The files are subsequently analyzed by a PID network vendor for compliance with all federal and state adequacy standards. The yearly submission also contains a narrative that outlines access gaps and efforts to mitigate these gaps.

We obtained the annual filings for PY 2024 (submitted in May 2023) from the PID for outpatient mental health counselors, including licensed clinical social workers, licensed marriage and family therapists, and mental health counselors for both adults and children.²¹ Carriers submit unique files for adult and pediatric providers. We identified 11,685 adult providers and 19,423 pediatric providers among 31,108 total listings for all carriers (Ambetter Health, Capital BlueCross, Cigna, Geisinger, Highmark BlueCross BlueShield, Independence Blue Cross, Oscar Health, UPMC). Providers listed as seeing both pediatric and adult consumers were counted twice.

Paid student research assistants (callers) were randomly assigned a provider listing. Online provider directories for the respective carriers were used to search for each provider by name. If a given provider was not located, another provider was assigned. For provider listings that were successfully identified in online directories, available contact information (street, city, state, zip code, and phone number) was compared against PID filings. Callers then proceeded to contact the provider, adopting a scripted role as an individual seeking an appointment for a family member or close friend. Callers sought to verify network participation, specialty, and phone numbers (encountering cases such as the number being disconnected, only a fax number was reached, a place other than a medical office was reached, or the provider never worked in the office) based on data from the PID filings.^2,22-24 To maintain callers’ disguise as real shoppers, calls ended once the first inaccuracy was identified. If they connected with an accurately listed provider, callers also asked for the next available appointment and recorded the appointment date without scheduling it. Overall, callers searched for 8306 mental health counselors and sought to contact the 6657 individuals they were able to match between PID filings and online provider directories (eAppendix 1 [eAppendices available at ajmc.com]). Data collection and verification of the annual network adequacy filings occurred from August 16, 2023, to February 27, 2024.

Analyses

We performed descriptive statistics comparing match rates between PID filings and provider directory listings. For sensitivity analyses, we also reanalyzed all data accounting for the passage of time (omitted). We compared findings by adult and pediatric provider types and across carriers using tests of proportion and t tests, as appropriate. The study was approved by the Texas A&M institutional review board.

RESULTS

Accuracy of Carriers’ PID Filings vs Provider Directory Information

Resource constraints allowed us to randomly survey 8306 providers (3170 adult and 5136 pediatric) from the overall population of 31,108 providers (26.7%). The sample size exceeded the requirements for a margin of error equal to or less than 1% with a 95% confidence level. Of these 8306 providers, we identified 6657 providers (80.1%) in the carriers’ corresponding online provider directories (eAppendix 2). Match rates for listings between PID filings (eAppendix 3) and online directories ranged from a low of 47.4% (292 of 616) for carrier D to a high of 88.3% (1207 of 1367) for carrier F (P < .001). Match rates were higher for pediatric (82.7%) than for adult (76.0%; P < .001) providers.

We further identified substantial differences in providers’ contact information between PID filings and online provider directories (Figure 1 and eAppendices 4 and 5). Only 44.0% of providers (2928 of 6657) had complete matches for street, city, state, zip code, and telephone number between the 2 sources. Across carriers, contact information congruence between PID filings and online provider directories ranged from 8.1% (98 of 1207) for carrier F to 82.2% (240 of 292) for carrier D (P < .001). Match rates were higher for adult (eAppendix 6) compared with pediatric providers (eAppendix 7) (46.3% vs 42.7%; P = .004). Detailed information about individual contact information components is provided in the eAppendix (eAppendices 8-13).

Accuracy of Carriers’ PID Filings vs Secret Shopper Calls

Of the 6657 providers sampled from carrier PID filings identified in online directories, callers successfully contacted 2152 providers but could not verify information for the remaining 4505 listings because they were unable to reach a person, they were on hold for longer than 5 minutes, or the provider’s office declined to provide any information (eAppendices 14-17). Of the providers listed in PID filings that callers successfully contacted, 65.2% (1404 of 2152) exhibited at least 1 of the 3 inaccuracies assessed (eAppendix 18), ranging from a low of 45.2% (42 of 93) for carrier D to a high of 90.4% (94 of 104) for carrier A (P < .001).

Phone numbers were the most commonly identified inaccuracy (56.6%; 1219 of 2152) (eAppendix 19), ranging from 35.5% (33 of 93) for carrier D to 90.4% (94 of 104) for carrier A (P < .001). Inaccurate network status was listed for 7.8% (167 of 2152) of providers (eAppendix 20). Carrier A did not have any errors related to network status, whereas 20.1% (41 of 204) of in-network listings for carrier G (P < .001) were actually out of network. Lastly, 6.4% of listings (137 of 2152) had incorrect specialty information (eAppendix 21), ranging from 0.0% for carrier A to 9.5% (16 of 168) for carrier H (P = .001). Overall error rates (62.8% vs 68.8%; P = .043) and incorrect phone numbers (62.0% vs 53.1%; P < .001) were lower for pediatric compared with adult mental health counselors, and pediatric mental health counselors also were less likely to have incorrect specialty listings (3.8% vs 8.0%; P < .001), incorrect in-network status (5.8% vs 9.1%; P = .006), and incorrect information on new patient acceptance (10.3% vs 13.4%; P = .035).

Access to Appointments and Wait Times

Callers were able to obtain appointments for 14.9% (321 of 2152) of successfully contacted providers (Figure 2 and eAppendices 22 and 23), with an additional 195 providers (9.1%) indicating that they were not offering appointments to new patients. Appointment rates ranged from 5.8% (6 of 104) for carrier A to 31.2% (29 of 93) for carrier D (P < .001). Appointments were available for 14.0% (120 of 860) of adult providers and 15.6% (201 of 1292) of pediatric providers (P = .306 for difference). Put differently, callers were able to get appointments with 3.9% (321 of 8306) of the providers sampled from the networks submitted to PID.

The mean wait time from call to appointment availability was 33.2 days (median, 27; range, 1-609) (Figure 2 and eAppendices 22 and 24), without statistically significant differences across carriers. Mean wait time was 35.1 days (median, 25.5; range, 1-609) for adult appointments and 32.1 days (median, 28; range, 1-251) for children (P = .590 for difference).

DISCUSSION

Provider directories have been shown to be highly inaccurate,^2,15,23-25 particularly for mental health services. Whether these inaccuracies are also present in regulatory filings—commonly used in regulatory assessments of network adequacy—has been unknown. Despite this information presumably originating from the same carrier sources, we found that nearly 20% of providers submitted to the PID were not identified in consumer-facing provider directories and, hence, were unlikely to be accessible to consumers seeking care despite being counted toward meeting network adequacy standards. Although a certain degree of missingness is inevitable, our findings indicate that consumers may only be truly accessing 80% of the “network” submitted to regulators—even before accounting for any additional inaccuracies. Moreover, secret shopper verification indicated substantial inaccuracies in carrier PID filings, with more than 65% of providers in regulatory filings exhibiting at least 1 inaccuracy. These inaccuracies were most commonly related to phone numbers (57%), network status (8%), and specialties (6%), all of which have previously been identified by CMS as potential barriers to patient care navigation and access.²⁶ These findings suggest that regulatory filings are similarly error prone as has been previously demonstrated in consumer-facing provider directories.

Moreover, because of the role of PID filings in regulatory oversight, findings suggest that inaccurate PID filings could be overestimating provider network breadth and underestimating true consumer access challenges. Importantly, the reasons for these discrepancies are yet unclear and warrant further investigation. It is also worth noting that, despite originating from the same source (ie, the carriers), substantial discrepancies in contact information existed between PID filings and provider directories. These internal inconsistencies highlight the inherent challenges carriers face in providing accurate information about their provider networks. They also raise questions about carrier network management and operational processes. The extent to which these findings are specific to or exacerbated in mental health remains unknown, although prior work on consumer-facing provider directories has found that mental health listings may have higher inaccuracies than other specialties and that commercial filings may be slightly more accurate than Medicaid.^2,14,22

Our second major set of findings is similarly concerning: We were able to access only 321 appointments out of 2152 successfully contacted mental health counselors (14.9%), with wait times in excess of 32 days. This finding underlies a stark reality of consumer experience in navigating an increasingly narrow funnel of providers. Moreover, observed wait times do not align with consumer expectations of access to care²⁷ nor with general regulatory requirements,^8,28 including the CMS standards for behavioral health of 10 calendar days.²⁸ These findings confirm well-documented mental health access challenges in the setting of high demand, particularly in the aftermath of the COVID-19 pandemic.^2,15,29

Limitations

This study has several limitations. First, we assessed mental health networks in the Pennsylvania ACA market only, which may not be representative of other markets such as Medicaid or other states. However, Pennsylvania is a large state with a diverse population, and several carriers operate regionally or nationally. Moreover, although our analysis focused on ACA Marketplace plans, the carriers in our sample also have substantial regional and national market scope, increasing the likelihood that these findings may be generalizable to other markets. Second, to maintain secret shoppers’ disguises, we ended calls when the first inaccuracy was identified, and thus our findings may underestimate the extent of specific inaccuracies. Third, we relied on carriers’ annual submission files to the PID because these data serve as the foundation for network adequacy monitoring and oversight. Although updated filings may have slightly altered our findings, recent analyses of the Pennsylvania insurance market found that inaccuracies persist over long periods of time.^23,24 Moreover, findings were robust to sensitivity analyses assessing the effects of time on outcomes (omitted). Fourth, to best capture the consumer experience, we only sought to verify providers from PID listings that were also identified in provider directories. Lastly, we were unable to contact 4505 providers to verify their information.

CONCLUSIONS

Despite these limitations, our findings have substantial policy implications that extend the current literature on network adequacy monitoring and regulation. This is the first known analysis that assessed PID regulatory filings, finding substantial inaccuracies in provider network information submitted by carriers as well as discrepancies compared with consumer-facing provider directories. Broadly, this work supports existing concerns that network adequacy assessment efforts, using existing data sources, do not accurately capture consumer access challenges. Taken together, findings suggest that substantial policy action to improve the regulation of provider networks is needed to enhance the consumer experience and ensure adequate access to care. 

Author Affiliations: Division of Health Services Management and Policy, College of Public Health, The Ohio State University (SFH), Columbus, OH; Division of General Internal Medicine and Geriatrics, School of Medicine, Oregon Health & Science University (JMZ), Portland, OR.

Source of Funding: CMS/HHS as part of a financial assistance award totaling $1,446,775. The contents are those of the authors and do not necessarily represent the official views of, nor an endorsement by, CMS/HHS or the US government. This work was also funded by the Insurance Department of the Commonwealth of Pennsylvania and the Robert Wood Johnson Foundation.

Author Disclosures: Dr Haeder has consulted for the Washington state attorney general, Watts Guerra, and Cera LLP and has received grants from the Robert Wood Johnson Foundation. Dr Zhu is a member of the Cambia Health Solutions Physician Advisory Board (reviewing clinical coverage policies) and has received grants from the Agency for Healthcare Research and Quality, American Psychiatric Association, National Institutes of Health, and National Institute for Health Care Management on unrelated topics.

Authorship Information: Concept and design (SFH); acquisition of data (SFH); analysis and interpretation of data (SFH, JMZ); drafting of the manuscript (SFH, JMZ); critical revision of the manuscript for important intellectual content (SFH, JMZ); statistical analysis (SFH); provision of patients or study materials (SFH); obtaining funding (SFH); administrative, technical, or logistic support (SFH, JMZ); and supervision (SFH).

Address Correspondence to: Simon F. Haeder, PhD, MPA, College of Public Health, The Ohio State University, 250 Cunz Hall, 1841 Neil Ave, Columbus, OH 43210. Email: haeder.1@osu.edu.

REFERENCES

1. Ahrnsbrak R, Bose J, Hedden SL, Lipari RN, Park-Lee E. Key Substance Use and Mental Health Indicators in the United States: Results From the 2016 National Survey on Drug Use and Health. Substance Abuse and Mental Health Services Administration; 2017. Accessed September 1, 2024. https://www.samhsa.gov/data/sites/default/files/NSDUH-FFR1-2016/NSDUH-FFR1-2016.htm

2. Burman A, Haeder SF, Xu WY. Provider directory inaccuracy and timely access for mental health care. Am J Manag Care. 2023;29(2):96-102. doi:10.37765/ajmc.2023.89318

3. Zhu JM, Zhang Y, Polsky D. Networks in ACA marketplaces are narrower for mental health care than for primary care. Health Aff (Millwood). 2017;36(9):1624-1631. doi:10.1377/hlthaff.2017.0325

4. Melek S, Davenport S, Gray TJ. Addiction and Mental Health vs. Physical Health: Widening Disparities in Network Use and Provider Reimbursement. Milliman Inc; 2019. Accessed September 1, 2024. https://edge.sitecorecloud.io/millimaninc5660-milliman6442-prod27d5-0001/media/Milliman/importedfiles/ektron/AddictionandmentalhealthvsphysicalhealthWideningdisparitiesinnetworkuseandproviderreimbursement.pdf

5. Busch SH, Kyanko KA. Incorrect provider directories associated with out-of-network mental health care and outpatient surprise bills. Health Aff (Millwood). 2020;39(6):975-983. doi:10.1377/hlthaff.2019.01501

6. Zhu JM, Huntington A, Haeder S, Wolk C, McConnell KJ. Insurance acceptance and cash pay rates for psychotherapy in the US. Health Aff Sch. 2024;2(9):qxae110. doi:10.1093/haschl/qxae110

7. Haeder SF, Weimer DL, Mukamel DB. A knotty problem: consumer access and the regulation of provider networks. J Health Polit Policy Law. 2019;44(6):937-954. doi:10.1215/03616878-7785835

8. Haeder SF, Xu WY, Elton T IV, Pitcher A. State efforts to regulate provider networks and directories: lessons for the future. J Health Polit Policy Law. 2023;48(6):951-968. doi:10.1215/03616878-10852610

9. Zhu JM, Polsky D, Johnstone C, McConnell KJ. Variation in network adequacy standards in Medicaid managed care. Am J Manag Care. 2022;28(6):288-292. doi:10.37765/ajmc.2022.89156

10. Giovannelli J, Lucia KW, Corlette S. Implementing the Affordable Care Act: state regulation of marketplace plan provider networks. Issue Brief (Commonw Fund). 2015:10:1-11.

11. Hall MA, Ginsburg PB. A Better Approach to Regulating Provider Adequacy. USC-Brookings Schaeffer Initiative for Health Policy; September 2017. Accessed September 2, 2024. https://www.brookings.edu/wp-content/uploads/2017/09/regulatory-options-for-provider-network-adequacy.pdf

12. PY2023 ACA-compliant health insurance form filing guidance. Pennsylvania Insurance Department. March 18, 2022. Accessed September 2, 2024. https://www.pa.gov/content/dam/copapwp-pagov/en/insurance/documents/consumers/documents/2023%20aca%20compliant%20health%20insurance%20form%20filing%20guidance%20final.pdf

13. Burman A, Haeder SF. Directory accuracy and timely access to in Maryland’s Medicaid managed care program. J Health Care Poor Underserved. 2022;33(2):597-611. doi:10.1353/hpu.2022.0050

14. Elton T IV, Xu WY, Haeder SF. Provider directory inaccuracy and timely access to physical therapy. World Med Health Policy. 2024;16(3):447-459. doi:10.1002/wmh3.607

15. Zhu JM, Charlesworth CJ, Polsky D, McConnell KJ. Phantom networks: discrepancies between reported and realized mental health care access in Oregon Medicaid. Health Aff (Millwood). 2022;41(7):1013-1022. doi:10.1377/hlthaff.2022.00052

16. Xu WY, Raver EW, Elton T IV, Davis M, Haeder SF. Disconnects between provider network directories and patient preferences. Am J Manag Care. 2024;30(12):660-666. doi:10.37765/ajmc.2024.89638

17. Top 5 reasons why provider directories are often inaccurate. Quest Analytics. July 24, 2023. Accessed September 11, 2024. https://questanalytics.com/news/why-provider-directories-are-inaccurate/

18. Burman A, Haeder SF. Without a dedicated enforcement mechanism, new federal protections are unlikely to improve provider directory accuracy. Health Affairs Forefront. November 5, 2021. Accessed September 1, 2024. https://www.healthaffairs.org/content/forefront/without-dedicated-enforcement-mechanism-new-federal-protections-unlikely-improve

19. Haeder SF, Weimer DL, Mukamel DB. A consumer-centric approach to network adequacy: access to four specialties in California’s marketplace. Health Aff (Millwood). 2019;38(11):1918-1926. doi:10.1377/hlthaff.2019.00116

20. Haeder SF, Weimer DL, Mukamel DB. Mixed signals: the inadequacy of provider-per-enrollee ratios for assessing network adequacy in California (and elsewhere). World Med Health Policy. 2023;15(3):258-272. doi:10.1002/wmh3.466

21. Ijaz S, Davies P, Williams CJ, Kessler D, Lewis G, Wiles N. Psychological therapies for treatment-resistant depression in adults. Cochrane Database Syst Rev. 2018;5(5):CD010558. doi:10.1002/14651858.CD010558.pub2

22. Burman A, Haeder SF. Potemkin protections: assessing provider directory accuracy and timely access for four specialties in California. J Health Polit Policy Law. 2022;47(3):319-349. doi:10.1215/03616878-9626866

23. Haeder SF, Zhu JM. Inaccuracies in provider directories persist for long periods of time. Health Aff Sch. 2024;2(6):qxae079. doi:10.1093/haschl/qxae079

24. Haeder SF, Zhu JM. Persistence of provider directory inaccuracies after the No Surprises Act. Am J Manag Care. 2024;30(11):584-588. doi:10.37765/ajmc.2024.89627

25. Brahmbhatt D, Schpero WL. Access to psychiatric appointments for Medicaid enrollees in 4 large US cities. JAMA. 2024;332(8):668-669. doi:10.1001/jama.2024.13074

26. Online provider directory review report. CMS. 2018. Accessed September 1, 2024. https://www.cms.gov/Medicare/Health-Plans/ManagedCareMarketing/Downloads/Provider_Directory_Review_Industry_Report_Year2_Final_1-19-18.pdf

27. Haeder SF, Xu WY. When is a network adequate? consumer perspectives on network adequacy definitions. Am J Manag Care. 2024;30(9):424-432. doi:10.37765/ajmc.2024.89601

28. Pollitz K. Network adequacy standards and enforcement. KFF. February 4, 2022. Accessed September 1, 2024. https://www.kff.org/affordable-care-act/issue-brief/network-adequacy-standards-and-enforcement/

29. Vadivel R, Shoib S, El Halabi S, et al. Mental health in the post-COVID-19 era: challenges and the way forward. Gen Psychiatr. 2021;34(1):e100424. doi:10.1136/gpsych-2020-100424