The American Journal of Managed Care
November 2017
Volume 23
Issue 11

Electronic Reminder's Role in Promoting Human Papillomavirus Vaccine Use

The use of the electronic health record’s clinical reminder functionality is systematically related to higher human papillomavirus vaccine administration rates.


Objectives: To study the association of using an electronic health record (EHR)’s clinical reminder functionality with increased human papillomavirus (HPV) vaccine administration among US office-based physicians.

Study Design: National Ambulatory Medical Care Survey data from 2007-2012 were pooled together to determine if EHR systems that employ clinical reminders are associated with increased immunization rates in different populations.

Methods: The administration of HPV vaccine served as the dependent variable, with the EHR reminder being the primary independent variable of interest. Logit regression was used to assess the relationship between using EHR reminders and HPV vaccine administration.

Results: Analyses indicated that compared with physicians without clinical reminder functions, physicians with clinical reminder functions were more likely to order HPV vaccines. Clinical reminder functions were particularly effective at increasing HPV vaccine use among adolescent males.

Conclusions: Overall, physicians who report using clinical reminders were more likely to order HPV immunizations. However, the association of clinical reminders with HPV immunizations was not significant in the younger adolescent subpopulation. Further, given that HPV incidence decreases significantly even with small gains in vaccination rates, the increase in HPV immunizations found in the male population aged 11 to 21 years is promising. Therefore, targeting males to receive HPV vaccination immunizations through clinical reminders provides a positively disproportionate return on vaccination rates and disease burden.

Am J Manag Care. 2017;23(11):e353-e359Takeaway Points

A significant amount of time, effort, and money has gone into increasing the meaningful use of electronic health records (EHRs). This study’s findings demonstrate that:

  • Using clinical reminders is strongly associated with higher vaccination rates for human papillomavirus (HPV).
  • The adolescent male population has an especially higher HPV vaccination rate in the presence of clinical reminders.
  • EHR-driven clinical reminders have the potential to increase preventive care among at-risk, but often neglected, subpopulations.

Human papillomavirus (HPV) is the most common sexually transmitted infection in the United States, causing genital warts and cervical and other cancers. It is estimated that 79 million people are currently infected and 14 million people acquire new HPV infections each year in the United States.1 Furthermore, 17,500 female and 9300 male Americans are affected by HPV-associated cancers annually.2 The widespread availability of an HPV vaccine has the promise to reduce the spread of many infectious diseases, including several that lead to cancers.3,4 Despite the efficacy of HPV vaccines, their use continues to lag behind that of other immunizations (eg, the tetanus, diphtheria, and pertussis vaccine and the first of 2 doses of quadrivalent conjugate meningococcal vaccine) among the targeted age groups.5,6 Moreover, differences in vaccination rates may be systematically linked to sociodemographic characteristics and represent an inequality in care delivery.7,8 Therefore, increasing HPV vaccine use to the levels of other commonly administered immunizations and across at-risk populations is desirable.9

One tool for improving primary care quality, particularly for routine tasks such as immunization administration, is the electronic health record (EHR). EHRs have the ability to provide clinical reminders that support evidence-based care; as such, there is an expectation that such systems will improve care processes and outcomes. Prior study results have shown that uses of EHR and clinical decision support are associated with improvements in preventive care services, such as health behavioral counseling, screening tests, prevention and management of chronic diseases (eg, cancer and cardiovascular diseases), preventive medications prescribed, and vaccination.10-21 The promise of improved care was foundational to the US government’s Meaningful Use program to increase physicians’ adoption and implementation of EHR systems.22 Within that context, one can empirically test the question of whether EHRs with clinical reminders have improved adherence to HPV immunization guidelines.

The purpose of this article is to explore the relationships among EHR adoption, clinical reminder use, and HPV immunization rates. Data from the 2007-2012 National Ambulatory Medical Care Survey (NAMCS) are analyzed using logit model regressions, with HPV immunization rates as the dependent variable. The results include analyses of HPV vaccination rates across sociodemographic groups; a discussion of the findings’ implications and areas of future research are promulgated.

The study’s findings address an important interest area for public health officials, policy makers charged with promoting EHR Meaningful Use criteria, those seeking equity in healthcare provision, and clinicians. For public health advocates, any tool that increases effective immunization adherence in support of disease prevention is an important contribution to the well-being of communities. Additionally, policy makers responsible for the Meaningful Use program need meaningful measurements to assess whether increased EHR adoption leads to better treatment modalities. Mitigating care delivery disparities is an important aim for many government programs, including Meaningful Use. Lastly, for clinicians, evidence that EHRs can help them deliver better care is needed to more effectively assess the return on investment associated with owning and operating such a technology.


Data Source and Sample

This study used the 2007-2012 iterations of the NAMCS to assess the use of clinical reminders in relation to HPV immunization rates in visits to office-based physicians. The NAMCS is a national probability sample survey administered by the National Center for Health Statistics on behalf of the CDC that collects data on patient visits to non—federally employed office-based physicians. For each visit, physicians or staff members complete a 1-page survey containing patient demographics, reasons for the visit, physician’s diagnoses, and medications ordered, supplied, administered, or continued during patient encounters. The NAMCS uses a multistage probability sampling design, which allows for the generation of nationally representative estimates.

This study analyzed visits by adolescents recommended to receive HPV vaccines to all office-based physicians, including primary care physicians, pediatricians, internists, obstetricians/gynecologists, and other specialists. Detailed descriptions of physician specialties are available from the authors and on the NAMCS website. HPV vaccination is recommended for both girls and boys at age 11 or 12 years. Females aged 13 to 26 years and males aged 13 to 21 years who have not been vaccinated previously are also recommended to receive HPV vaccines.2 To estimate the influence of clinical reminders on HPV vaccines ordered or administered in the ambulatory setting during adolescent visits, the study analyzed 3 adolescent visit samples during 2007-2012: 1) visits with patients aged 11 to 12 years (3388 visits), 2) visits with patients aged 11 to 18 years (14,354 visits), and 3) visits with female patients aged 11 to 26 years and male patients aged 11 to 21 years (25,573 visits).


HPV vaccine immunization. The NAMCS contains data on medications ordered, supplied, administered, or continued during patient encounters. The outcome of interest in this study was a dichotomous measure that identifies whether the physician ordered, supplied, administered, or continued HPV vaccine immunizations during adolescent patient visits.

Clinical reminder use. In the NAMCS survey, physicians reported whether their practices used an EHR system and, if so, which functionalities their EHR included. Based on physician-reported information regarding specific EHR functionalities used in their practices, the key independent variable of this study was related to physicians’ use of the clinical reminder function of the EHR. The clinical reminder was coded by examining physicians’ response to the question, “Does your practice have computerized capabilities of providing reminders for guideline-based interventions or screening tests and how often is the capability used?” “Clinical reminder” was coded as "1" if the response was “Yes” and coded as "0" for those who responded “No,” “Unknown,” or “Yes, but turned off or not used.” In 2012, the NAMCS changed questions on specific EHR functionalities to differentiate between “routine use” and “nonroutine use” of specific functions such as clinical reminders.

Statistical Analysis

A cross-sectional analysis of pooled survey data was conducted. We used weighted multivariable logit regression to estimate the association of clinical reminder use with HPV vaccine immunization rates at adolescent visits. In each regression, we included interaction terms between a dichotomous indicator for patients’ sex (female) and clinical reminder use to examine how the association of clinical reminder use with HPV vaccine immunization rates varies between male and female adolescent patients. All of the regressions controlled for potential confounders, including patient demographic characteristics (race/ethnicity), visit characteristics (ie, preventive care visits, patient’s own primary care physician), and other covariates, such as metropolitan statistical area status and survey year. Instead of adjusted odds ratios, marginal effects of coefficients in logit regressions were estimated and presented in the subsequent result section because odds ratio interpretation of interaction terms in nonlinear models, such as logit models, are problematic.23,24


HPV Vaccine Immunization Rates by Clinical Reminder Use (unadjusted)

Survey-weighted descriptive statistics for HPV vaccine immunizations, clinical reminder use, and other covariates are reported in Table 1. From 2007 to 2012 in the United States, there were 89.1 million office-based physician visits by adolescents aged 11 to 12 years, 381.6 million ambulatory clinic visits by those aged 11 to 18 years, and 672.7 million visits by female adolescents aged 11 to 26 years and male adolescents aged 11 to 21 years. In ambulatory clinic visits by adolescents aged 11 to 18 years, approximately 40% were with clinical reminders.

Table 2 presents survey-weighted rates of HPV vaccine immunizations by clinical reminder use in the different samples of adolescent visits. The rate of HPV vaccine immunizations varied by the use of clinical reminder function. Compared with physicians without clinical reminder functions, physicians with clinical reminder functions were more likely to order HPV vaccines. The pattern holds consistently across patients’ sex and age-group samples. For example, in female patients aged 11 to 12 years, HPV vaccines were ordered in 6.44% of encounters where a clinical reminder function was present and 5.24% when it was not. In visits by male patients aged 11 to 12 years, HPV vaccines were ordered in 1.50% of encounters with clinical reminders and 0.61% without. The subsequent section will detail further analysis on the statistical significance of these patterns, controlling for patient and visit characteristics.

Estimated Association Between Clinical Reminder Use and HPV Vaccine Immunization

Table 3 presents the differential estimated association of clinical reminder use with HPV vaccine immunizations by patient sex. Logit coefficients are presented in odd columns and marginal effects are presented in even columns. No statistically significant associations were found between clinical reminder use and HPV vaccine immunization in the samples of: 1) visits with patients aged 11 to 12 years and 2) visits with patients aged 11 to 18 years. In visits with female patients aged 11 to 26 years and male patients aged 11 to 21 years, use of clinical reminders was associated with a higher rate of HPV vaccine immunizations for male and female patients. Furthermore, the coefficients for the interaction term (Clinical reminder use × Female) were negative (marginal effect, —0.0136; P <.1), and the magnitude of the coefficients on the interaction is less than the magnitude of coefficients on clinical reminder use (marginal effect, 0.0164; P <.01). This indicates that the association of clinical reminder use with HPV vaccines ordered is greater in male encounters than in female encounters.

Compared with male patients aged 11 to 21 years who visited physicians who did not adopt clinical reminders, male patients in the same age group who visited physicians who did use clinical reminders received more HPV vaccines, by 1.64 percentage points. This represents a relative increase of 146% from the mean probability of HPV vaccine immunizations (1.12%). Compared with female patients aged 11 to 26 years who visited physicians who did not adopt clinical reminders, female patients in the same age group who visited physicians who did use clinical reminders received more HPV vaccines, by 0.28 percentage points. This represents a relative increase of 10.7% from the mean probability of HPV vaccine immunizations (2.61%).

We performed sensitivity tests to address concerns that the actual use of the reminder function, not merely the simple adoption of clinical reminders, guarantees the improvement in care. Using 2012 data that are able to differentiate “routine use” and “nonroutine use” of clinical reminders, we compared rates of HPV vaccine immunizations by the level of clinical reminder use. Compared with physicians who did not adopt clinical reminders, physicians routinely using clinical reminders were more likely to order HPV vaccines (eAppendix Table [eAppendix available at]). On the other hand, physicians nonroutinely using clinical reminders were not consistently more likely to order HPV vaccines compared with physicians not adopting clinical reminders. Thus, these results indicate that we may have underestimated the association of clinical reminder use with HPV vaccine immunizations, which supports the robustness of our primary findings.


Among physicians reporting that they used clinical reminders, the measure was positively correlated with HPV vaccine immunizations. The associations are greater in magnitude for males aged 11 to 21 years than females aged 11 to 26 years. One explanation for this phenomenon is that physicians may consider HPV to be primarily a gynecological issue. Therefore, the reminder would have a greater impact in populations where the disease risk is less well understood.

No such associations of clinical reminder use and HPV vaccine immunizations were seen in the subpopulation of younger adolescents. The difference between age groups suggests that certain behavioral characteristics associated with recommending the HPV vaccine may be difficult to overcome in the younger population.25 Given that HPV vaccine immunization is designed to have greater impact earlier in life, this result suggests that clinical reminders may not be adequate in increasing vaccination rates in this population. It may be necessary to couple clinical reminders with other behavioral interventions for physicians. Tobacco use, in particular, merits attention, as those who smoke are at elevated risk for oral cancers that need HPV exposure to develop.26

The significant increase in vaccination rates associated with clinical reminder use that we saw in the male population aged 11 to 21 years is a promising avenue for intervention. For HPV, the disease burden for the individual and the population decreases significantly even with small increases in vaccination rates.27 Given that males have a higher prevalence of HPV infections, they merit additional attention.28 Thus, targeting males to receive HPV vaccination immunizations through clinical reminders provides a positively disproportionate return on vaccination rates and disease burden. These results also suggest that barriers to HPV vaccination recommendations may be less for males than females. Many parents are resistant to vaccination of their daughters due to the concern that the HPV vaccine may encourage sexual debut of their daughters.29,30

Reviewing the control variables, several key observations can be made. As expected, preventive care visits were associated with significantly higher rates of HPV vaccination, along with visits to patients’ own primary care physicians. We saw no significant associations between race and HPV vaccination. This suggests that disparities in HPV preventive care may be minimal in this sample population. However, it is important to carefully monitor the traditionally underserved population, often with higher rates of cervical cancer, to ensure that access to preventive care visits is available in order to obtain HPV vaccines.25


There are limitations to this study. We used a cross-sectional snapshot to determine the correlation between clinical reminders and HPV vaccination rates; thus, we cannot establish a causal relationship. It could be that physicians adopting new technology, such as EHRs, and clinical decision support systems tend to follow recently released recommendations on HPV vaccination. It is also possible that physicians who adhere to clinical guidelines on vaccination are more interested in and likely to adopt clinical support systems as tools to improve their quality of care. Furthermore, the lack of longitudinal data means that patients are not being tracked over time. Given that the HPV vaccine requires 3 doses to be administered over an extended period, it is not possible to assess whether reminders promote better completion of the recommended regime.31

Additionally, the NAMCS cannot identify whether the vaccine order is for the first, second, or third dose of HPV vaccine. Besides, the NAMCS data set is a self-report survey by physicians and staff members, who may be prone to over- or under-report preventive services and EHR functionalities. Furthermore, the NAMCS does not sufficiently report on both patient and physician practice characteristics, and these unobservable characteristics, such as patient socioeconomic characteristics or physician age and years of practice, could confound the association between clinical reminder use and our outcome of interest. Finally, the NAMCS information on the diversity and complexity of EHR systems and specific functionalities used in clinical practices was limited. Thus, we cannot fully identify whether and how different EHR systems and their clinical reminder functions help HPV vaccine orders in visits made by adolescents who are recommended to take HPV vaccines. In addition, this study could not address the impact of type of vendor, data architecture, and end-user interface.

Minority access to practices with high-functioning EHRs and the concomitant impact on preventive care is an area that merits more research. If EHRs produce a significant improvement in care quality, as most expect, then this trend represents an emerging disparity. Another area of research that needs further work is related to the herd immunity effect of HPV vaccines. If relatively small increases in vaccination rates lead to significantly large reductions in HPV infection rates, this needs to be better understood.


Clinical reminders are positively correlated with better care processes related to HPV vaccination. Promoting the routine use of clinical reminders for vaccination will yield significant benefits for the general population. Clinician-initiated recommendation is a key ingredient in starting successful HPV vaccinations.25,32 Thus, clinical reminders serve as an important gateway for physicians to initiate a conversation on HPV vaccination with families. This is especially important in addressing coverage in traditionally underserved populations. Especially for HPV vaccinations, where strong social barriers may exist in its disease etiology and cause, it is important to decrease other barriers for clinical opportunity.33Author Affiliations: School of Health Studies, Northern Illinois University (JB), Dekalb, IL; Health Policy and Management, Johns Hopkins Bloomberg School of Public Health (EWF, SW), Baltimore, MD; Department of Family Medicine, College of Medicine, The Ohio State University (TH), Columbus, OH.

Source of Funding: None.

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (JB, EWF); acquisition of data (JB); analysis and interpretation of data (JB, SW, TH); drafting of the manuscript (JB, EWF, SW, TH); critical revision of the manuscript for important intellectual content (JB, EWF, TH); statistical analysis (TH); administrative, technical, or logistic support (EWF, SW); and supervision (TH).

Address Correspondence to: Eric W. Ford, PhD, MPH, Johns Hopkins Bloomberg School of Public Health, 624 N Broadway, Hampton House 533, Baltimore, MD 21205. E-mail:

1. CDC. Incidence, prevalence, and cost of sexually transmitted infections in the United States. Published February 2013. Accessed October 8, 2016.

2. Markowitz LE, Dunne EF, Saraiya M, et al; CDC. Human papillomavirus vaccination: recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Recomm Rep. 2014;63(RR-05):1-30.

3. Trottier H, Franco EL. Human papillomavirus and cervical cancer: burden of illness and basis for prevention. Am J Manag Care. 2006;12(suppl 17):S462-S472.

4. Zhai L, Tumban E. Gardasil-9: a global survey of projected efficacy. Antiviral Res. 2016;130:101-109. doi: 10.1016/j.antiviral.2016.03.016.

5. Suryadevara M, Paton L, Domachowske JB. Adolescent immunization: 2015 and beyond. Pediatr Ann. 2015;44(4):e82-e88. doi: 10.3928/00904481-20150410-09.

6. Hymel PA. Decreasing risk: impact of HPV vaccination on outcomes. Am J Manag Care. 2006;12(suppl 17):S473-S483.

7. Drolet M, Deeks SL, Kliewer E, Musto G, Lambert P, Brisson M. Can high overall human papillomavirus vaccination coverage hide sociodemographic inequalities? an ecological analysis in Canada. Vaccine. 2016;34(16):1874-1880. doi: 10.1016/j.vaccine.2016.02.069.

8. Holman DM, Benard V, Roland KB, Watson M, Liddon N, Stokley S. Barriers to human papillomavirus vaccination among US adolescents: a systematic review of the literature. JAMA Pediatr. 2014;168(1):76-82. doi: 10.1001/jamapediatrics.2013.2752.

9. Dempsey AF, Davis MM. Overcoming barriers to adherence to HPV vaccination recommendations. Am J Manag Care. 2006;12(suppl 17):S484-S491.

10. Bae J, Ford EW, Huerta TR. The electronic medical record’s role in support of smoking cessation activities. Nicotine Tob Res. 2016;18(5):1019-1024. doi: 10.1093/ntr/ntv270.

11. Bae J, Hockenberry JM, Rask KJ, Becker ER. Evidence that electronic health records can promote physician counseling for healthy behaviors. Health Care Manage Rev. 2017;42(3):258-268. doi: 10.1097/HMR.0000000000000108.

12. Bae J, Huerta TR, Ford EW. Cardiovascular disease documentation and counseling in electronic medical records. Prev Med. 2016;88:27-32. doi: 10.1016/j.ypmed.2016.03.017.

13. De Leon SF, Shih SC. Tracking the delivery of prevention-oriented care among primary care providers who have adopted electronic health records. J Am Med Inform Assoc. 2011;18(suppl 1):i91-i95. doi: 10.1136/amiajnl-2011-000219.

14. Federman DG, Kravetz JD, Lerz KA, et al. Implementation of an electronic clinical reminder to improve rates of lung cancer screening. Am J Med. 2014;127(9):813-816. doi: 10.1016/j.amjmed.2014.04.010.

15. Furukawa MF. Electronic medical records and efficiency and productivity during office visits. Am J Manag Care. 2011;17(4):296-303.

16. Loo TS, Davis RB, Lipsitz LA, et al. Electronic medical record reminders and panel management to improve primary care of elderly patients. Arch Intern Med. 2011;171(17):1552-1558. doi: 10.1001/archinternmed.2011.394.

17. Mishuris RG, Linder JA, Bates DW, Bitton A. Using electronic health record clinical decision support is associated with improved quality of care. Am J Manag Care. 2014;20(10):e445-e452.

18. Rand CM, Blumkin A, Szilagyi PG. Electronic health record use and preventive counseling for US children and adolescents. J Am Med Inform Assoc. 2014;21(e1):e152-e156. doi: 10.1136/amiajnl-2013-002260.

19. Schwartz AR, Levin FL, O’Neil JA Jr, Braithwaite RS. Pilot of decision support to individualize colorectal cancer screening recommendations. Am J Manag Care. 2015;21(7):e439-e446.

20. Wagholikar KB, MacLaughlin KL, Henry MR, et al. Clinical decision support with automated text processing for cervical cancer screening. J Am Med Inform Assoc. 2012;19(5):833-839. doi: 10.1136/amiajnl-2012-000820.

21. Tarver WL, Menachemi N. The impact of health information technology on cancer care across the continuum: a systematic review and meta-analysis. J Am Med Inform Assoc. 2016;23(2):420-427. doi: 10.1093/jamia/ocv064.

22. Mennemeyer ST, Menachemi N, Rahurkar S, Ford EW. Impact of the HITECH Act on physicians’ adoption of electronic health records. J Am Med Inform Assoc. 2016;23(2):375-379. doi: 10.1093/jamia/ocv103.

23. Ai C, Norton EC. Interaction terms in logit and probit models. Econ Letters. 2003;80(1):123-129. doi: 10.1016/S0165-1765(03)00032-6.

24. Karaca-Mandic P, Norton EC, Dowd B. Interaction terms in nonlinear models. Health Serv Res. 2012;47(1 pt 1):255-274. doi: 10.1111/j.1475-6773.2011.01314.x.

25. Brewer NT, Fazekas KI. Predictors of HPV vaccine acceptability: a theory-informed, systematic review. Prev Med. 2007;45(2-3):107-114. doi: 10.1016/j.ypmed.2007.05.013.

26. Fakhry C, Gillison ML, D’Souza G. Tobacco use and oral HPV-16 infection. JAMA. 2014;312(14):1465-1467. doi: 10.1001/jama.2014.13183.

27. Jemal A, Simard EP, Dorell C, et al. Annual report to the nation on the status of cancer, 1975-2009, featuring the burden and trends in human papillomavirus (HPV)-associated cancers and HPV vaccination coverage levels. J Natl Cancer Inst. 2013;105(3):175-201. doi: 10.1093/jnci/djs491.

28. Gillison ML, Broutian T, Pickard RK, et al. Prevalence of oral HPV infection in the United States, 2009-2010. JAMA. 2012;307(7):693-703. doi: 10.1001/jama.2012.101.

29. Ferrer HB, Trotter C, Hickman M, Audrey S. Barriers and facilitators to HPV vaccination of young women in high-income countries: a qualitative systematic review and evidence synthesis. BMC Public Health. 2014;14:700. doi: 10.1186/1471-2458-14-700.

30. Quinn GP, Murphy D, Malo TL, Christie J, Vadaparampil ST. A national survey about human papillomavirus vaccination: what we didn’t ask, but physicians wanted us to know. J Pediatr Adolesc Gynecol. 2012;25(4):254-258. doi: 10.1016/j.jpag.2012.02.007.

31. Dobson SR, McNeil S, Dionne M, et al. Immunogenicity of 2 doses of HPV vaccine in younger adolescents vs 3 doses in young women: a randomized clinical trial. JAMA. 2013;309(17):1793-1802. doi: 10.1001/jama.2013.1625.

32. Fiks AG, Grundmeier RW, Mayne S, et al. Effectiveness of decision support for families, clinicians, or both on HPV vaccine receipt. Pediatrics. 2013;131(6):1114-1124. doi: 10.1542/peds.2012-3122.

33. Vadaparampil ST, Kahn JA, Salmon D, et al. Missed clinical opportunities: provider recommendations for HPV vaccination for 11-12 year old girls are limited. Vaccine. 2011;29(47):8634-8641. doi: 10.1016/j.vaccine.2011.09.006.

Related Videos
Related Content
CH LogoCenter for Biosimilars Logo