The American Journal of Managed Care April 2007
Managed Care Organizations' Performance in Delivery of Childhood Immunizations (HEDIS, 1999-2002)
Objectives: To examine recent trends in childhood immunizations recommended by the Advisory Committee for Immunization Practices measured by the Health Plan Employer Data and Information Set (HEDIS) and to describe the factors associated with higher rates over time.
Design: The HEDIS performance measures from 1999 to 2002 and plan characteristics include approximately 400 enrollees per plan each year.
Methods: Longitudinal regression analysis of commercial managed care organizations' HEDIS measures. The outcome measure was the proportion of children aged 24 to 35 months in the plan who received 4 doses of diphtheriatetanus-pertussis vaccine, 3 doses of polio vaccine, 1 dose of measles-mumps-rubella vaccine, 3 doses of Haemophilus influenzae type b vaccine, and 3 doses of hepatitis B vaccine.
Results: The mean immunization rate for health insurance plans increased from 65.7% in 1999 to 67.9% to 2002. Plans that reported publicly had higher childhood immunization rates than plans that did not report publicly (P < .001). Plans with higher proportions of Hispanics or African Americans had lower childhood immunization rates (P < .001). Immunization rates varied significantly by type of visit; plans with higher proportions of children making visits to their primary care physician had higher rates of immunization (P < .001).
Conclusions: Managed care organizations' performance measured by childhood immunization rates varies by organizational and demographic factors. Our findings suggest that plans should ensure efficient and accurate data collection systems and should encourage their providers to assess for immunizations at sick-child and well-child care visits.
(Am J Manag Care. 2007;13:193-200)
Health plan decision makers should be aware of the following regarding childhood immunization rates:
- Plans that reported publicly had higher childhood immunization rates than plans that did not report publicly (P < .001).
- Plans with higher proportions of Hispanics or African Americans had lower childhood immunization rates (P < .001).
- Immunization rates varied significantly by the type of visit. Plans with higher proportions of children making visits to their primary care physician had higher rates of immunization, whereas plans with higher proportions of infants making sick-child care visits (outpatient or emergency department) had lower rates of immunization (P < .001).
Measures of quality give purchasers of healthcare and consumers of this care information about how insurance plans perform. The Health Plan Employer Data and Information Set (HEDIS) from the National Committee for Quality Assurance (NCQA) provides comparative information across health plans to measure the quality of care and preventive services for health plan beneficiaries. More than 66% of commercial health plans report to NCQA, representing 85% of the commercially enrolled managed care organization (MCO) population, or approximately 70 million to 73 million enrollees.1
Results have shown that differences in childhood immunization rates vary depending on the collection methods (eg, National Immunization Survey, National Health Interview Survey, or HEDIS).2 Investigations using HEDIS data have described factors associated with immunization rates (eg, physician credentials)3; however, little is known about factors associated with immunization rates accounting for time. For HEDIS, reporting these associations can help administrators of health insurance plans in identifying characteristics that are more predictive of delivery of immunization services. This knowledge can assist in the development of strategies to maintain high immunization rates, as well as inform potential purchasers of insurance plans about what plan characteristics are associated with sustained high immunization rates over time. The method of longitudinal regression analysis has been rarely used when examining HEDIS data, although its necessity has been acknowledged.3
Because 6 or more encounters with a healthcare provider are needed in the first 2 years of life to fully immunize a child, timely receipt of childhood immunizations, captured in the immunization rates, are an indication of how well the healthcare delivery system is working. Influential employers such as General Motors, a company that has given employees significant financial incentives to choose health plans with better HEDIS scores, emphasize childhood immunization performance among important measures to assess.4 In the present study, longitudinal regression analysis was used to examine the factors associated with higher childhood immunizations rates reported by public reporting and nonpublic reporting commercial health plans to the NCQA.
MATERIALS AND METHODS
The analyses included the HEDIS data sets from 1999 to 2002 for public reporting and nonpublic reporting commercial health insurance plans (for care received from 1998 to 2001). Plans included in the analyses were operating in all US states (except Montana), in Guam, and in Washington, DC. The sample sizes were 423 plans in 1999, 383 plans in 2000, 371 plans in 2001, and 332 plans in 2002. Variables examined from the Consumer Assessment of Health Plans survey included the proportions of respondents by race/ethnicity.5 Variables were examined from 8 HEDIS measurement categories.5 The NCQA accreditation status changed during the years, so plans were considered accredited if their status was excellent, commendable, full accreditation, or 1-year accreditation; otherwise, the plan was not considered accredited by the NCQA. Because this study constituted analysis of secondary data without identifiers, it did not require a review by the Centers for Disease Control and Prevention Institutional Review Board, and consent was not required.
For HEDIS clinical measures (eg, immunizations) that require medical record review, a sample of 411 enrollees is required to obtain a valid rate. If the plan has fewer than 411 eligible persons for a particular measure, all eligible enrollees are sampled. For measures that use administrative data only (eg, child access to a primary care provider), the health plan samples the entire eligible population. For the Consumer Assessment of Health Plans survey, a minimum of 100 responses is required for reporting particular rates.
For immunization measures, children were systematically selected from the enrollment cohort who reached their second birthday in the reporting year and who met the continuous enrollment criterion of no more than 1 gap in enrollment of up to 45 days during the 12 months before their second birthday. Children identified as having contraindications (eg, anaphylactic reaction to a vaccine or its components) were excluded from all immunization statistics.5 Childhood immunization statistics from plans were calculated and reported (1) by using only administrative records specific to the plan (administrative method) or (2) by using administrative data supplemented with medical record data (hybrid method), which may contain information about immunizations received outside of the plan. To avoid ascertainment bias and to maintain data comparability, only plans using the hybrid method were included for immunization measures. Few plans that used the administrative method were excluded (16 plans in 1999, 5 plans in 2000, 11 plans in 2001, and 8 plans in 2002).
The plan characteristics examined were those groups of measures, defined by NCQA, that have been shown previously in the literature to be associated with receipt of immunizations6 or those that could theoretically be associated with preventive services, including receipt of immunizations. These domains include access and availability of care and health plan stability. Health plan descriptive characteristics included NCQA accreditation status, region of operation, public reporting status, vaccine policy of states of operation, enrollment size (below or above the median), race/ethnicity proportions (based on the sample of Consumer Assessment of Health Plans survey respondents), and MCO type (eg, health maintenance organization, point of service, or combined health maintenance organization and point of service). The Vaccines for Children program is a federal entitlement program that supplies public and private healthcare providers with federally purchased vaccines at no cost for administration to eligible children. Vaccine policies include (1) Vaccines for Children program only, (2) Vaccines for Children program and underinsured, (3) universal,7 and (4) multiple states, in which the insurance plan operates in several states with various vaccine policies. Because of policy changes, the vaccine policy variable was coded for the immunization policy in effect in the state of operation for each year that a plan reported to the NCQA. Variables assessing access and availability of care were calculated and reported using the administrative method and included the proportion of board-certified providers per number of enrollees, proportion of total providers per total commercial enrollees, proportion of children aged 25 months to 6 years who made visits to their primary care physician during the reporting year (visits are for well-child or sick-child care), and proportion of outpatient sick-child care visits and emergency department visits made by children younger than 1 year annually. Variables used to assess health plan stability included the number of years in business by MCO type and the practitioner turnover rate for the reporting year. Plans reporting to be in business for more than 25 years were considered as being in business for 25 years.
Main Outcome Measures
Because the NCQA releases data aggregated by managed care plans rather than by enrolled individuals, the unit of analysis was the plan. Childhood immunization statistics were calculated as the proportion of children sampled in the plan who received 4 doses of diphtheria-tetanus-pertussis vaccine, 3 doses of polio vaccine (inactivated from 1999 to 2002 and oral vaccine in 1999), 1 dose of measles-mumps-rubella vaccine, 3 doses of Haemophilus influenzae type b vaccine, and 3 doses of hepatitis B vaccine as recommended by the Advisory Committee for Immunization Practices. Immunizations were only included if they were determined to be valid according to HEDIS criteria (Table 1). Because of the vaccine shortages in 2001 and 2002, plans had the option to report the previous year's immunization statistics if the plan or entity did not change (ie, did not merge or split into different plans) and they had reported in the previous year.5 To include all available data, all variables of interest reported by these plans were included except for the redundant immunization statistics reported during 2 consecutive years.
Data were examined using SAS software version 9.1.3 (SAS Institute, Cary, NC). Data were modeled longitudinally using PROC MIXED to fit linear mixed models that included a random intercept, an unstructured covariance, and a robust variance estimator, the sandwich estimator, which was used to calculate valid standard errors while accounting for misspecification of the covariance and a normal distribution. Plan characteristics were fixed effects. An advantage of fitting linear mixed models is that PROC MIXED uses all available data in the analysis.8 Therefore, the plans that opted to report the previous year's immunization data during the years of shortage were included in the model using their independent variables, although the dependent variable data were missing. In the bivariate analyses, variables of interest were modeled with the year of report and an interaction term (eg, MCO type) to determine if they were significantly associated with the change in proportion immunized over time at P < .05. Testing for interactions among variables was performed (NCQA accreditation status with region, public reporting status, and proportion of visits with primary care physicians), and region was tested with race/ethnicity proportions. A significant interaction term consisting of an independent variable with time (year or report) indicates that one group's rate of change is faster than another group's rate of change. Backward elimination was used to determine the multivariate model; fit statistics were used to determine the best fit of the model, and variables significant at P < .05 remained in the model.
The rates for full immunization remained below 70% during the study period. The mean + standard deviation (SD) rates were as follows: 65.7% + 13.5% (range, 0.0%-93.6%) for 1999, 63.3% + 14.9% (range, 5.5%-92.9%) for 2000, 65.6% + 13.7% (range, 11.9%-90.5%) for 2001, and 67.9% + 12.2% (range, 17.6%-88.6%) for 2002 (Table 2).
Two hundred forty-seven plans reported for 1 year, 132 plans reported for 2 years, 110 plans reported for 3 years, and 147 plans reported for 4 years. Health plan characteristics are given in Table 2 and Table 3 for continuous and categorical variables, respectively. The proportion of publicly reporting plans increased from 67.7% in 1999 to 83.7% in 2002 (Table 3). Similarly, the proportion of NCQA-accredited plans increased from 48.3% in 1999 to 65.2% in 2002. The proportions of enrollees by race/ethnicity remained steady during the 4 years (Table 2).
Factors Associated With Higher Immunization Rates Over Time
1. Scholle SH, Mardon R, Shih SC, Pawlson LG. The relationship between quality and utilization in managed care. Am J Manag Care. 2005;11:521-527.
2. Fairbrother G, Freed GL,Thompson JW. Measuring immunization coverage. Am J Prev Med. 2000;19(suppl):78-88.
3.Wan TT, Pai CW,Wan GJ. Organizational and market determinants of HMOs' performance of preventive practices. J Healthc Qual. 1998;20:14-19, 52.
4. Berman HS. Performance measures: the destination or the journey? Eff Clin Pract. 1999;2:284-288.
5. National Committee for Quality Assurance. HEDIS 2002 Technical Specifications.Washington, DC: National Committee for Quality Assurance; 2002.
6. Rickert D, Deladisma A,Yusuf H, Averhoff F, Brink E, Shih S. Adolescent immunizations: are we ready for a new wave? Am J Prev Med. 2004;26:22-28.
7. Centers for Disease Control and Prevention. Childhood vaccine supply policy. 2005. Available at: http://www.cdc.gov/nip/vfc/st_immz_proj/data/vacc_supply_public_2005.htm. Accessed April 25, 2006.
8.Verbeke G, Molenberghs G. Linear Mixed Models in Practice: An SAS-Oriented Approach. New York, NY: Springer Publishing Co Inc;1997.
9. Centers for Disease Control and Prevention (CDC). Update on the supply of tetanus and diphtheria toxoids and of diphtheria and tetanus toxoids and acellular pertussis vaccine. MMWR Morb Mortal Wkly Rep. 2001;50:189-190.
10. Centers for Disease Control and Prevention (CDC). Resumption of routine schedule for diphtheria and tetanus toxoids and acellular pertussis vaccine and for measles, mumps, and rubella vaccine [published correction appears in MMWR Morb Mortal Wkly Rep. 2002;51:622]. MMWR Morb Mortal Wkly Rep. 2002;51:598-599.
11. Centers for Disease Control and Prevention (CDC). Impact of the 1999 AAP/USPHS joint statement on thimerosal in vaccines on infant hepatitis B vaccination practices. MMWR Morb Mortal Wkly Rep. 2001;50:94-97.
12. Santibanez TA, Santoli JM, Barker LE. Differential effects of the DTaP and MMR vaccine shortages on timeliness of childhood vaccination coverage. Am J Public Health. 2006;96:691-696.
13. National Commission for Quality Assurance. Childhood immunization status. 2006. Available at: http://www.ncqa.org/somc2001/CHILD_IMM/SOMC_2001_CIS.html#performing. Accessed April 3, 2006.
14. Diamond F. Believe it: HEDIS scores matter to employers. Manag Care. 2003;12:34, 37.
15. Lo Sasso AT, Perloff L, Schield J, Murphy JJ, Mortimer JD, Budetti PP. Beyond cost: 'responsible purchasing' of managed care by employers. Health Aff (Millwood). 1999;18:212-223.
16. Rollins G. Sizing up HEDIS: experts take system's measure. J AHIMA. 2002;73:30-36.
17. Irigoyen M, See D, Findley SE. Annotation: children's disengagement from medical homes: a neglected public health imperative. Am J Public Health. 1999;89:157-159.
18. Santoli JM, Rodewald LE, Maes EF, Battaglia MP, Coronado VG. Vaccines for Children program, United States, 1997. Pediatrics. 1999;104:e15.
19. Smith PJ, Santoli JM, Chu SY, Ochoa DQ, Rodewald LE. The association between having a medical home and vaccination coverage among children eligible for the Vaccines for Children program. Pediatrics. 2005;116:130-139.
20. Holt E, Guyer B, Hughart N, et al. The contribution of missed opportunities to childhood underimmunization in Baltimore. Pediatrics. 1996;97:474-480.
21. National Vaccine Advisory Committee. Standards for child and adolescent immunization practices. Pediatrics. 2003;112:958-963.
22. US Department of Health and Human Services. Healthy People 2000. March 1994. Available at: http://www.crisny.org/health/us/ health7.html. Accessed October 19, 2004.
23. US Department of Health and Human Services. Healthy People 2010, 2nd ed: With Understanding and Improving Health and Objectives for Improving Health.Washington, DC: US Dept of Health and Human Services; 2004.
24. McCormick D, Himmelstein DU,Woolhandler S,Wolfe SM, Bor DH. Relationship between low quality-of-care scores and HMOs' subsequent public disclosure of quality-of-care scores. JAMA. 2002;288:1484-1490.