Samuel W. Woolford, PhD

To demonstrate how the analysis of clinical process, cost, and outcomes can identify healthcare improvements that reduce cost without sacrificing quality, using the example of the initial visit associated with oral contraceptive pill use.

Cross-sectional study using data collected by HealthMETRICS between 1996 and 2009.

Using data collected from 106 sites in 24 states, the unintended pregnancy (UIP) rate, effectiveness of patient education, and unit visit cost were calculated. Staff type providing education and placement of education were recorded. Two-way analysis of variance models were created and tested for significance to identify differences between groups.

Sites using nonclinical staff to provide education outside the exam were associated with lower cost, higher education scores, and a UIP rate no different from that of sites using clinical staff. Sites also providing patient education during the physical examination were associated with higher cost, lower education scores, and a UIP rate no lower than that of sites providing education outside of the exam.

 Through analyzing process, cost, and quality, lower-cost processes that did not reduce clinical quality were identified. This methodology is applicable to other clinical services for identifying low-cost processes that do not result in lower clinical quality. By using nonclinical staff educators to provide education outside of the physical examination, sites could save an average of 32% of the total cost of the visit.

This study compares the cost and clinical quality associated with 2 specific processes used to provide the initial visit for oral contraceptive pills.

		Using nonclinical staff to provide patient education reduces staff cost and may be associated with better education scores.

		Providing patient education exclusively outside of the exam reduces staff cost and is not associated with decreased clinical quality.

		Utilizing both processes described above could save sites 32% of the total visit cost.

Because reducing healthcare costs and improving access to highquality care are pressing issues, there is a constant need to improve the processes used to deliver care to reduce cost without negatively impacting clinical quality. Unfortunately, few studies explore the relationship between clinical process and outcome. Clinicians and administrative staff are left to make cost-saving process changes without access to process-specific data showing how those changes will impact clinical quality. There are tremendous variations in both clinical process and unit cost for the delivery of a given clinical service across the country.

 While it has also been shown that variations in clinical decision making exist,

 clinical process variations (eg, the sequencing of the steps during a visit, staff mix providing those steps, scheduling structure) have a significant impact on cost.

This study used the initial visit for obtaining oral contraceptive pills (OCPs) as a model to demonstrate how clinical processes and outcomes can be linked to help reduce costs while maintaining clinical quality. The initial visit for obtaining OCPs was chosen as the clinical model for study because it is an easily defined outpatient visit with a high volume and a clearly defined patient population. With an estimated 66 million US women of reproductive age

and 10.7 million US women currently using OCPs,

 slight reductions in visit cost could result in substantial savings to the healthcare system. This study focuses on the process used to deliver patient education regarding OCP use during the initial OCP visit. Specifically, variation in the mix of staff utilized and the placement of the patient education step within the initial OCP visit process were considered. The goal was to determine whether the staff used to provide the education or the placement of the education in the initial OCP visit had any impact on cost or clinical quality.

This cross-sectional study utilizes data that were collected by Health- METRICS staff from 106 participating women’s healthcare sites in 24 states between 1996 and 2009. Settings included county health departments, private healthcare providers, community health centers, and others. These data were collected under contractual arrangement with HealthMETRICS and represent a broad spectrum of processes associated with the initial OCP process. HealthMETRICS is a consulting company providing the information healthcare managers need to ensure that they are providing care as effectively and efficiently as possible. All of the participating sites voluntarily asked to participate in the HealthMETRICS Family Planning Optimal Performance Project. HealthMETRICS did not choose sites to participate. From our experience analyzing the process for delivering family planning services over the last 15 years, the analysis includes most if not all of the different processes for providing this service.

Two or more HealthMETRICS staff members traveled to each site to interview clinic staff, document the initial OCP process, and distribute data collection tools. Subjects were all women at least 16 years of age without any significant medical complaints at the time of their visit. We focused on the data related to the patient education portion of the initial OCP visit, including the staff utilized, the placement of the education in the initial OCP process, the effectiveness of the patient education, the unintended pregnancy (UIP) rate, and unit visit costs.

Effectiveness of patient education and observed UIP rate were used as markers of clinical quality. Effectiveness of patient education was assessed through 4 survey questions that asked subjects to rate how clearly they understood 4 key components of the counseling portion of the visit: when to start the pill, what to do if a dose is missed, what to do if there is spotting or bleeding while on the pill, and possible side effects (

). For each site, the percentage of patients who gave the highest rating for all 4 questions was calculated. Sites collecting fewer than 30 surveys (n = 57) were excluded.

Chart reviews were conducted by clinic staff at each site. The number of patients who had a UIP within 6 months of their initial OCP visit, as determined at the follow-up visit, was tallied for each site and reported as the number of UIPs per 50 patients. Sites that reviewed fewer than 40 charts (n = 1) were excluded.

HealthMETRICS staff visited sites and identified the level of the staff members used to provide patient education. While clinical staff performed the physical exam for all patients at all sites, the educational component of the initial OCP visit was conducted in multiple ways by a variety of clinical and nonclinical staff. Clinical staff was considered to be registered nurses (RNs) and nurse practitioners. Nonclinical staff included all other staff types that participated in the visit (

Sites were categorized based on the level of staff used to provide the education. Sites exclusively utilizing clinical staff were categorized as having a higher-cost process. Sites utilizing nonclinical staff were categorized as having a lower-cost process. These designations were based on the higher hourly cost associated with RNs and nurse practitioners.

The staff level used to provide patient education varied among sites, with 13 distinct models being used (Table 1). The most common staff level used was the RN (52% of sites). A higher-cost model was used by 57% of sites; 43% used a lower-cost model.

Position of Education Step Within OCP Initial Visit Process

The initial OCP visit process largely consists of 3 steps: intake, exam, and checkout. The point in time at which education was provided varied. Some sites provided at least some education during the physical exam step, while others provided education only outside of the exam. At least some education was provided during the physical exam by 79% of sites and only outside of the exam by 21% of sites.

Unit cost was measured through a combination of time logs, financial data provided by the sites, and appointment schedules. The annual visit volume was recorded. Direct costs were adjusted for inflation using data from the Consumer Pricing Index. Indirect costs such as administrative overhead were not considered. Salaries were adjusted for regional differences and inflation using data from the employee cost index. These data were used to calculate an average hourly cost for each staff level. Time and staffing data gathered by the time logs were used to allocate the labor cost to each step in the visit. The annual cost of staff directly involved in the visit but not interacting directly with the patient was allocated over the annual visit volume. Unit staff cost was defined as the sum of all direct labor unit costs. Unit total cost was defined as the sum of the unit staff cost and all other nonlabor direct costs.

Our objective was to determine whether the factors representing the different staff mixes and the placement of the education step had any impact on either the education score, the UIP rate, or unit staff cost. We hypothesized the 2-way analysis of variance (ANOVA) model

y(i,j,k) = μ + S(i) + P(j) + S(i)P(j) + ε(i,j,k)

 the education score/UIP/unit staff cost for site k utilizing staff mix i and education step placement j; S(i)P(j) the interaction between factors; and ε the associated random error. The null hypothesis is that there was no effect on clinical quality or cost due to the main effects of who educates the patient and when education occurs, or due to the interaction of these factors.

The mean education score was 88% with a coefficient of variation of 7%. The UIP rate varied more, with a mean of 0.60 and coefficient of variation of 161% (

). A general linear model was used to test the 2-way ANOVA for education score and UIP using SPSS version 18 (SPSS Inc, Chicago, Illinois).

The results for the education score indicate that there was a significant effect of staff mix (

= .026) and placement of education step (

= .037) on education score. The interaction between these 2 variables was significant (

 = .030). Sites that utilized nonclinical staff to provide education outside of the exam had the highest education score. When education was provided during the physical exam, clinical and nonclinical staff education scores were not statistically different. A total of 57 sites were excluded from this analysis because they collected fewer than 30 patient education surveys, resulting in 6 observations of nonclinical staff providing education outside the exam, 26 of clinical staff educating patients during the exam, and 8 in each of the remaining 2 groups. The 57 cases that were excluded were fairly evenly distributed (44% higher cost, 56% lower cost).

The results for UIP rate indicated that there was no staff mix effect (

 = .548), no placement of education effect (

 = .643), and no interaction effect (P = .195). The validity of the 2-way ANOVA analyses was predicated on demonstrating that the data were normally distributed and that the variances of the data were the same across all combinations. Levene’s test was used to confirm the equality-of-variance assumption for education score (

 = .338). Normality was tested using the Shapiro-Wilk test and was violated for 1 subgroup in education score data. The violation was identified with 1 outlier. After removing the outlier, the conclusions did not change. All subgroups of the UIP data failed the Shapiro-Wilk test for normality. A bootstrap analysis was performed to analyze the 2-way ANOVA model for UIP rate. The bootstrap analysis does not require the normality assumption necessitated by our ANOVA analysis. The bootstrap results confirmed the results of the ANOVA model.

Articles

Linking variation in process with cost and quality provides the opportunity for identifying low-cost, high-quality processes.

Process, Cost, and Clinical Quality: The Initial Oral Contraceptive Visit

 To identify optimal transferable practice processes for provision of a clinical service by studying the relationships among unit cost, clinical outcome, patient satisfaction, and staff satisfaction observed for a discreet service performed at multiple sites in a well-defined patient population.

Cross-sectional study using data collected by HealthMETRICS from 1996 to 2007.

Data from 165 US clinics in 29 states, totaling 8835 patients and 1583 clinic staff, were reviewed. Four parameters of the initial visit for oral contraceptives (OCs) were measured: unit visit cost, patient satisfaction, staff satisfaction, and clinical indicators, including patient education effectiveness and occurrence of unintended pregnancies within 6 months of the initial OC visit. Patient population and visit type were narrowly defined to ensure intersite comparability. Data collection tools included surveys, time logs, financial work sheets, and on-site visits to document process.

Clinical process variation was widespread. The unit cost of an initial OC visit varied from $42 to $206 (mean: $90, coefficient of variation: 38%). Staff satisfaction varied more than patient satisfaction. Clinical indicators, including unintended pregnancies, varied little. The sites achieving lower unit costs demonstrated no apparent decrease in clinical quality.

Clinical processes used to provide initial visits for OC services varied demonstrably across study sites, generating variation in cost with little impact on clinical quality or patient satisfaction. By adopting appropriate components of the optimal practice process, clinical sites could lower the care cost by more than 20% while maintaining or increasing care quality.

This study measures variation in unit costs, clinical processes, staff and patient satisfaction,

and clinical outcomes in the initial visit for oral contraception at 165 clinics in 29 states.

		Observed process variation generated more variation in cost than in clinical quality.

		 Efficient processes are transferable and can lower cost without negatively impacting outcomes.

		The information collected in this study could be used to provide benchmarking data for those responsible for implementing clinical processes.

Wide variation in the delivery of clinical services, resulting in differences in the cost and quality of care, can be observed repeatedly among similar providers with similar patients across the United States. Although variation in clinical practice by geographic region and other demographic characteristics has been documented,

 healthcare managers and providers rarely have processspecific benchmarking data for daily outpatient clinical process that control for the patient population being treated and narrow the clinical service sufficiently to allow for head-to-head comparisons. Some clinical process and cost benchmarking data do currently exist, like those provided by the Joint Commission (formerly The Joint Commission on Accreditation of Healthcare Organizations). However, the Joint Commission does not include data regarding the cost or efficiency of the hospitals’ clinical processes. Local efforts, such as the Massachusetts Health Quality Partners,

 compare Massachusetts physician networks, medical groups, and practice sites with respect to patient satisfaction and the percentage of patients receiving the correct treatment. Massachusetts Health Quality Partners provides a snapshot of results of the clinical process without specifically examining what causes variation or whether there exists a correlation between process, provider satisfaction, and cost.

This study establishes a baseline framework for examining the contribution of variability among clinical processes along 4 critical interdependent parameters: unit visit cost, patient satisfaction, staff satisfaction, and clinical indicators including effectiveness of patient education. We demonstrate that variability in the process of care delivery from 1 healthcare provider to another for similar patients for the same clinical service results in differences in cost of care, and to a lesser extent, differences in clinical outcomes and the satisfaction of both patients and care providers at all levels. We show that although no single process provides the optimal clinical outcome with the lowest unit cost and highest patient and staff satisfaction in all settings, the range of processes that can provide optimal results is much narrower than that observed in daily clinical practice. Finally, we provide evidence that identifying and transferring the processes that optimize clinical care along these dimensions could lower costs with the same or better clinical quality.

We focused on the provision of oral contraceptives (OCs), one of the most common and easily defined outpatient visit types in women’s healthcare, to demonstrate this clinical process variability. We examined the relationships among cost, clinical outcome, patient satisfaction, and staff satisfaction for this single clinical service performed at multiple sites in a welldefined patient population.

The goal of this study was to document variations in the care delivery process, identify how clinical outcome and cost are impacted by process variation, and quantify the opportunity to improve clinical outcomes at lower costs by optimizing the clinical process.

This cross-sectional study quantifies variation in the provision of the initial OC visit across 4 measures: unit cost, patient satisfaction, staff satisfaction, and clinical outcomes. Data collection tools were developed to quantify each of these 4 parameters. The initial OC visit process was chosen because it tends to be the highest volume service of the participating sites, and a uniform patient population can be readily identified. Participating sites included 165 US women’s healthcare facilities in 29 states. Settings included county health departments, private healthcare providers, community health centers, and others. Data collection occurred between 1996 and 2007. HealthMETRICS staff collected scheduling data and floor plans, and 2 or more HealthMETRICS staff members traveled to each site to interview clinic staff, document the initial OC visit process, and distribute data collection tools.

Subjects were women at least 16 years of age, without any significant medical complaints, who presented for an initial OC visit.

Outcomes were assessed by retrospective review of 50 charts, selected at random by site staff from patients who had an initial OC visit at least 6 months prior to the site’s study start date. The number of patients who had an unintended pregnancy (UIP) within 6 months following their initial OC visit was tallied for each site. Sites that reviewed fewer than 40 patient charts (n = 44) were excluded.

Unit costs were measured by utilizing several data collection tools. Only direct costs were considered; indirect costs such as administrative overhead were not included. Labor costs were calculated using a combination of direct financial data, actual clinical time logs, and a review of actual appointment schedules. Staff salaries were adjusted to control for the effect of regional economic differences. Using the data described above, hourly costs were calculated for all staff positions that involved direct interaction with the patient. Clinic time logs were used to allocate this labor cost to each step in the visit. The annual cost of support staff not directly interacting with the patient was allocated over the annual visit volume to obtain the unit visit cost. Sites were asked to collect 20 time logs, which tracked the labor involved in all stages of the initial OC visit. Those collecting fewer than 10 were included only if this number was enough to establish a clear pattern of how the initial OC visit was provided (24 sites excluded). Material and space costs as well as staff costs were adjusted for inflation to ensure that unit visit cost across all clinics was comparable. We used an activity-based costing software package that was developed and validated in-house. Similaractivity-based software packages that can use time as a basis for calculating costs should provide similar results. Costing programs that use resource-based relative value scale units instead of time and salary data do not provide the granularity required for this approach.

Patient satisfaction was assessed by an anonymous survey distributed to, and completed by, patients at the end of their visit. Patients were asked to rate how satisfied they were with the visit (

). Sites collecting less than 30 surveys (n = 101) were excluded. Surveys were returned by patients to clinic staff in sealed envelopes and mailed to HealthMETRICS, where they were opened for analysis.

Staff satisfaction surveys were completed by all clinic staff involved in the initial OC visit process. Staff members were asked to rate how satisfied they were with the initial OC visit process at their clinic (eAppendix A, available at www. ajmc.com). Surveys were anonymous and returned to Health METRICS in sealed envelopes, where they were opened for analysis. The staff satisfaction surveys from 2 sites were unavailable at the time of this study and were excluded.

A fifth element, the effectiveness of education regarding use of OC that was provided to the patients during their initial OC visit, was assessed by an anonymous survey distributed to, and completed by, patients at the end of their visits. This element was included as part of the clinical quality measure. Patients rated 4 questions regarding how clearly the use of OCs was explained to them (eAppendix A, available at www. ajmc.com). Surveys were returned by patients to clinic staff in sealed envelopes and mailed to HealthMETRICS, where they were opened for analysis. Sites collecting fewer then 30 surveys (n = 101) were excluded.

Variation was observed in cost, clinical quality, and patient and staff satisfaction outcomes as a result of differences in the process for providing family planning services. Results are summarized in the Table. Box plots of each parameter are shown in 

The number of UIPs was low, with an average of 0.7 UIPs per 50 patients per site. The OC failure rate was 1.4% in the first 6 months following OC prescription, with a coefficient of variation of 144%. Of 122 sites included in this analysis, 5 sites had at least 3 UIPs per 50 patients per site. The rate at the remaining 117 clinics was less than 2.2 UIPs per 50 patients per site. The OC failure rate seen in the United States for typical OC use is 8.7% UIPs, though measured over a 12-month period.3

Far less variation was seen among patient education scores, with an average of 89% of patients rating the explanation of the use of OC pills as “very clear” (

The average total adjusted direct unit cost for a visit was $90, with a coefficient of variation of 38%. Unit costs ranged from $42 to $206 per visit. Adjusted direct staffing costs, excluding space, utilities, and other nonsalary expenses, varied from $22 to $164 (

) and mirrored the distribution of total unit costs. Two sites operating with high clinical and patient satisfaction scores had lower costs: 38% and 34% below the average.

Wide variation in the way clinical services are provided to similar patients for similar visit types results in highly variable costs but similar clinical outcomes.

Samuel W. Woolford, PhD

Articles

Process, Cost, and Clinical Quality: The Initial Oral Contraceptive Visit

Clinical Process Variation: Effect on Quality and Cost of Care