Currently Viewing:
The American Journal of Managed Care June 2016
Currently Reading
Development of a Tethered Personal Health Record Framework for Early End-of-Life Discussions
Seuli Bose-Brill, MD; Matthew Kretovics, MPH; Taylor Ballenger, BS; Gabriella Modan, PhD; Albert Lai, PhD; Lindsay Belanger, MPH; Stephen Koesters, MD; Taylor Pressler-Vydra, MS; and Celia Wills, PhD, RN
Variations in Patient Response to Tiered Physician Networks
Anna D. Sinaiko, PhD
Primary Care Appointment Availability and Nonphysician Providers One Year After Medicaid Expansion
Renuka Tipirneni, MD, MSc; Karin V. Rhodes, MD, MS; Rodney A. Hayward, MD; Richard L. Lichtenstein, PhD; HwaJung Choi, PhD; Elyse N. Reamer, BS; and Matthew M. Davis, MD, MAPP
Impact of Type 2 Diabetes Medication Cost Sharing on Patient Outcomes and Health Plan Costs
Julia Thornton Snider, PhD; Seth Seabury, PhD; Janice Lopez, PharmD, MPH; Scott McKenzie, MD; Yanyu Wu, PhD; and Dana P. Goldman, PhD
Risk Contracting and Operational Capabilities in Large Medical Groups During National Healthcare Reform
Robert. E. Mechanic, MBA, and Darren Zinner, PhD
The Evolving Role of Subspecialties in Population Health Management and New Healthcare Delivery Models
Dhruv Khullar, MD, MPP; Sandhya K. Rao, MD; Sreekanth K. Chaguturu, MD; and Rahul Rajkumar, MD, JD
When Doctors Go to Business School: Career Choices of Physician-MBAs
Damir Ljuboja, BS, BA; Brian W. Powers, AB; Benjamin Robbins, MD, MBA; Robert Huckman, PhD; Krishna Yeshwant, MD, MBA; and Sachin H. Jain, MD, MBA
Review of Outcomes Associated With Restricted Access to Atypical Antipsychotics
Krithika Rajagopalan, PhD; Mariam Hassan, PhD; Kimberly Boswell, MD; Evelyn Sarnes, PharmD, MPH; Kellie Meyer, PharmD, MPH; and Fred Grossman, MD, PhD
Value of Improved Lipid Control in Patients at High Risk for Adverse Cardiac Events
Anupam B. Jena, MD, PhD; Daniel M. Blumenthal, MD, MBA; Warren Stevens, PhD; Jacquelyn W. Chou, MPP, MPL; Thanh G.N. Ton, PhD; and Dana P. Goldman, PhD
Effects of Physician Payment Reform on Provision of Home Dialysis
Kevin F. Erickson, MD, MS; Wolfgang C. Winkelmayer, MD, ScD; Glenn M. Chertow, MD, MPH; and Jay Bhattacharya, MD, PhD
Adoption of New Agents and Changes in Treatment Patterns for Hepatitis C: 2010-2014
Xiaoxi Yao, PhD; Lindsey R. Sangaralingham, MPH; Joseph S. Ross, MD; Nilay D. Shah, PhD; and Jayant A. Talwalkar, MD

Development of a Tethered Personal Health Record Framework for Early End-of-Life Discussions

Seuli Bose-Brill, MD; Matthew Kretovics, MPH; Taylor Ballenger, BS; Gabriella Modan, PhD; Albert Lai, PhD; Lindsay Belanger, MPH; Stephen Koesters, MD; Taylor Pressler-Vydra, MS; and Celia Wills, PhD, RN
By using a novel, pre-defined advance care planning (ACP) framework, the personal health record can be used to elicit meaningful ACP documentation that is effective for both patients and providers.
 Participants were instructed to “think aloud” in order to provide immediate feedback on the content, structure, and layout of the framework as they read through it for the first time. The research assistants asked probing questions only about comments made by the participants. If a participant specifically requested help in navigating the PHR or the framework, the research assistant: a) provided targeted assistance, and b) documented participant difficulty with the domain for which help was requested. Participant comments were scribed by the research assistant immediately after the interview. Interviews were conducted until data saturation was reached.29 Observations were then compiled and used to revise the initial framework.30

Focus Groups (phase 1)

Patient focus groups were analyzed using the scissor-and-sort method of transcripts and scribe notes.31,32 Only elements that were identified by all 3 analysts were considered in assembling the framework. A summary of these elements was compiled and reviewed during framework development. Patient focus group analysis revealed several common preferences present in each of the respective groups: a preference for clear language in communication tools; endorsement of MyChart as a helpful communication tool; a need to qualify and disqualify preferred decision makers; and a desire to personalize content, often based on previous experiences (Table 4). These preferences were used to tailor an initial ACP framework for MyChart.

Cognitive Interviewing (phase 2)

Notes from the interviews were reviewed by 2 independent reviewers; a count of major themes was taken (Table 5) and categorized into “strengths” and “weaknesses.” Summary lists were compared and discrepancies were noted. Repeat review was conducted until analysts reached agreement in summaries.

Two of the most frequently mentioned “weaknesses” of the framework were that it was difficult to access MyChart—meaning they had difficulty logging in or registering for MyChart on their own (10 of 22)—and to locate the message within the account (14 of 22). These comments did not directly reflect the content of the framework, but they were still important considerations and were subsequently communicated to institutional information technology (IT) developers. Thirteen of the 22 participants mentioned that they would re-word questions. Other common themes were discomfort in answering the questions (5 of 22), need for an introduction to the framework (5 of 22), and desire for follow-up and discussion (5 of 22).

The physician focus group was also analyzed using a scissor-and-sort method.32 Physician comments were used to prepare a specific office workflow for the framework distribution and inform a streamlined ACP framework seeking specific ACP elements. Broader concerns about PHR use and billing that were beyond the scope of ACP framework development and workflow were communicated with participant permission to IT and medical center management.

The ACP process holds several advantages for patients, yet documentation of ACP and ADs remain low. The use of an EHR to deliver and support the ACP process could be advantageous to both care providers and patients, offering a more efficient use of time and resources. Although stand-alone tools aid in the process, these tools do not interface with medical records. Our framework allows for ACP documentation to be accessible by the individual and their medical team when it is most needed. This newly developed framework serves as a clinical tool, yet retains benefits of patient-initiated electronic ACP documentation.

Through focus group testing, we determined that patients desired a clear, concise, and accessible communication tool that would allow them to voice wishes and desires during the ACP process. Provider feedback indicated that the framework should help patients reflect and give a starting point for the conversations, but emphasized that they wanted to know only “vital” information. Providers also voiced their desire for the framework to fit within the EHR and the clinical workflow. While cognitive interviewing using the framework confirmed that patients approved of the content and delivery method, it also highlighted the need for small edits to language and workflow.


The focus groups had a high rate of “no-shows.” In addition, the study results would have been more generalizable had the study taken place across multiple clinics, not just a single clinical population. While minority participants were purposively sampled in focus groups, they were underrepresented during cognitive interviews. The use of only 1 physician focus group should also be considered, as resulting data may be incomplete. Although data from physician focus groups were consistent with those of similar studies, inclusion of a larger number of physicians may have yielded more reliable, complete results. That was not possible in this study, due to the small number of physicians at the study site. Future studies involving multiple practices will allow more robust exploration of perspectives about the framework. Ideally, additional focus groups would have been conducted in order to ensure saturation of opinion within the study population.

This project set out to develop a usable, patient-centered ACP framework to improve ACP documentation. Patient impressions reported during cognitive interviews suggested that patients found the framework accessible. The use of questions and content vetted by patients in the target population was an essential component of the development.

Future investigation should focus on larger, more diverse populations in order to improve the generalizability of this study and the framework. Investigators and providers will also need to consider how to make the electronic framework more accessible to patients who face some barrier to navigating or accessing their EHR.

Qualitative evidence would suggest that the developed framework would meet the needs of both patients and providers as a tool for documentation in the PHR, particularly in primary care. Incorporating the framework should be done with careful discussions with clinic providers in order to tailor workflows to individual practices (eAppendix B).


We would like to thank the following people for their support over the course of the study: Barbara Longo, Patricia Strickland, and Ann Henry of Ohio State University (OSU) Internal Medicine and Pediatrics Grandview for their administrative support of the project; Rose Hallarn, OSU Center for Clinical and Translational Science, for her assistance with participant recruitment strategies; Peter Embi,  MD, MS, FACP, FACMI, Vice Chair of the Department of Biomedical Informatics at the Ohio State University  for his advice on managing an Epic-based interdisciplinary study; and Lori Blum, Grants Administration for the Department of General Internal Medicine at the Ohio State University, for her assistance in budgetary matters. The project described was supported by Award Number Grant 8UL1TR000090-05 from the National Center for Advancing Translational Sciences. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Center for Advancing Translational Sciences or the National Institutes of Health.

Author Affiliations: College of Nursing (CW), and Division of General Internal Medicine, Department of Internal Medicine, College of Medicine (SB-B, MK, TB, SK), and Department of English (GM), Department of Biomedical Informatics (AL), Ohio State University, Columbus, OH; Boston University School of Public Health (LB), Boston, MA; Aver Informatics (TP-V), Columbus, OH.

Source of Funding: The project described was supported by Award Number Grant 8UL1TR000090-05 from the National Center for Advancing Translational Sciences.

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (SB-B, SK, TP-V, CW); acquisition of data (LB, SB-B, MK, SK); analysis and interpretation of data (LB, SB-B, TB, MK, AL, GM, TP-V); drafting of the manuscript (LB, SB-B, TB, MK, AL, CW); critical revision of the manuscript for important intellectual content (SB-B, TB, MK, SK, AL, GM); statistical analysis (TP-V); provision of patients or study materials (SB-B, MK); obtaining funding (SB-B); administrative, technical, or logistic support (LB, SB-B, SK, GM); and supervision (SB-B).

Address correspondence to: Seuli Bose-Brill, MD, Division of General Internal Medicine, Department of Internal Medicine, College of Medicine, Ohio State University, 895 Yard St, Columbus, OH 43212. E-mail:

1. Curd PR. Advance care planning reconsidered: toward an operational definition of outpatient advance care planning. J Palliat Med. 1999;2(2):157-159.

2. Levi BH, Green MJ. Too soon to give up: re-examining the value of advance directives. Am J Bioeth. 2010;10(4):3-22. doi: 10.1080/15265161003599691.

3. Detering KM, Hancock AD, Reade MC, Silvester W. The impact of advance care planning on end of life care in elderly patients: randomised controlled trial. BMJ. 2010;340:c1345. doi: 10.1136/bmj.c1345.

4. Heyland DK, Allan DE, Rocker G, Dodek P, Pichora D, Gafni A; Canadian Researchers at the End-of-Life Network (CARENET). Discussing prognosis with patients and their families near the end of life: impact on satisfaction with end-of-life care. Open Med. 2009;3(2):e101-e110.

5. Zhang B, Wright AA, Huskamp HA, et al. Health care costs in the last week of life: associations with end-of-life conversations. Arch Intern Med. 2009;169(5):480-488. doi: 10.1001/archinternmed.2008.587.

6. Temel JS, Greer JA, Admane S, et al. Code status documentation in the outpatient electronic medical records of patients with metastatic cancer. J Gen Intern Med. 2010;25(2):150-153. doi: 10.1007/s11606-009-1161-z.

7. Bradley NM, Sinclair E, Danjoux C, et al. The do-not-resuscitate order: incidence of documentation in the medical records of cancer patients referred for palliative radiotherapy. Curr Oncol. 2006;13(2):47-54.

8. Teno JM, Licks S, Lynn J, et al. Do advance directives provide instructions that direct care? SUPPORT Investigators. Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatment. J Am Geriatr Soc. 1997;45(4):508-512.

9. Mercer SW, Watt GC. The inverse care law: clinical primary care encounters in deprived and affluent areas of Scotland. Ann Fam Med. 2007;5(6):503-510.

10. Minto F, Strickland K. Anticipating emotion: a qualitative study of advance care planning in the community setting. Int J Palliat Nurs. 2011;17(6):278-284.

11. Munday D, Petrova M, Dale J. Exploring preferences for place of death with terminally ill patients: qualitative study of experiences of general practitioners and community nurses in England. BMJ. 2009;339:b2391. doi: 10.1136/bmj.b2391.

12. Aronsky D, Kasworm E, Jacobson JA, Haug PJ, Dean NC. Electronic screening of dictated reports to identify patients with do-not-resuscitate status. J Am Med Inform Assoc. 2004;11(5):403-409.

13. Jeong SY, Higgins I, McMillan M. Experiences with advance care planning: nurses’ perspective. Int J Older People Nurs. 2011;6(3):165-175.

14. Green MJ, Levi BH. Development of an interactive computer program for advance care planning. Health Expect. 2009;12(1):60-69. doi: 10.1111/j.1369-7625.2008.00517.x.

15. Sudore RL, Knight SJ, McMahan RD, et al. A novel website to prepare diverse older adults for decision making and advance care planning: a pilot study. J Pain Symptom Manage. 2014;47(4):674-686. doi: 10.1016/j.jpainsymman.2013.05.023.

16. Health Information Technology for Economic and Clinical Health (HITECH) Act. website. Published 2009. Accessed June 1, 2012.

17. Tang KC, Davis A. Critical factors in the determination of focus group size. Fam Pract. 1995;12(4):474-475.

18. Onwuegbuzie AJ, Collins KMT. A typology of mixed methods sampling designs in social science research. Qualitative Report. 2007;12(2):281-316.

19. Kitzinger J. The methodology of focus groups: the importance of interaction between research participants. Sociology of Health and Illness. 1994;16(1):103-121.

20. Karel MJ, Powell J, Cantor MD. Using a values discussion guide to facilitate communication in advance care planning. Patient Educ Couns. 2004;55(1):22-31.

21. Hammes BJ, Rooney BL, Gundrum JD. A comparative, retrospective, observational study of the prevalence, availability, and specificity of advance care plans in a county that implemented an advance care planning microsystem. J Am Geriatr Soc. 2010;58(7):1249-1255. doi: 10.1111/j.1532-5415.2010.02956.x.

22. Advance care planning—selected resources for the public. CDC website. Accessed February 2, 2012. 

23. Onwuegbuzie AJ, Dickinson WB, Leech NL, Zoran AG. A qualitative framework for collecting and analyzing data in focus group research. Int J Qual Methods. 2009;8(3):1-21.

24. Wilkinson S. Focus groups in health research: exploring the meanings of health and illness. J Health Psychol. 1998;3(3):329-348. doi: 10.1177/135910539800300304.

25. Potter J, Wetherell M. Discourse and Social Psychology: Beyond Attitudes and Behaviour. Newbury Park, CA: Sage Publishing; 1987.

26. Drew P, Chatwin J, Collins S. Conversation analysis: a method for research into interactions between patients and health-care professionals. Health Expect. 2001;4(1):58-70.

27. Moretti F, van Vliet L, Bensing J, et al. A standardized approach to qualitative content analysis of focus group discussions from different countries. Patient Educ Couns. 2011;82(3):420-428.

28. Beatty PC, Willis GB. Research synthesis: the practice of cognitive interviewing. Public Opin Q. 2007;71(2):287-311. doi: 10.1093/poq/nfm006.

29. Francis JJ, Johnston M, Robertson C, et al. What is an adequate sample size? operationalising data saturation for theory-based interview studies. Psychol Health. 2010;25(10):1229-1245.

30. Drennan J. Cognitive interviewing: verbal data in the design and pretesting of questionnaires. J Adv Nurs. 2003;42(1):57-63.

31. Krueger RA, Casey MA, Donner J, Kirsch S, Maack JN. Social development papers: social analysis—selected tools and techniques. World Bank website. Published June 2001. Accessed June 1, 2012.

32. Stewart D, Shamdasani P, Rook D. Focus Groups: Theory and Practice. 2nd edition. Thousand Oaks, CA: Sage Publications; 2007. 
Copyright AJMC 2006-2020 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up