Process Reengineering and Patient-Centered Approach Strengthen Efficiency in Specialized Care

Improving efficiency is complex and requires a multimodal approach. Health information systems, patient feedback, and multidisciplinary teams are components that can improve clinical processes.

ABSTRACT

Objectives: To improve multiple levels of utilization and efficiency in specialized outpatient consults using information technology—based systems, process reengineering, and patient-centeredness.

Study Design: Prospective research from 2008 to 2014 conducted in a hospital in Madrid, Spain. Quantitative analysis of 1,162,477 consecutive consultation requests and qualitative techniques of 72,368 surveys using a structured questionnaire.

Methods: Key performance indicators were evaluated: operational outcomes (productivity, time gap between requested consultations and performance, staffing accuracy, wait time, and underlying variability), administrative burden (downtime losses; no-show, drop-in, cancelled, and rescheduled visits), perceived quality scores, and income. Data mining, modeling seasonality in demand, process reengineering, and a patient-centered approach were incorporated as strategies to drive changes.

Results: Productivity increased 34% for the entire period, closing the gap between consultation request and performance from 43.7% to 8.7%. Wait time decreased from 82.7 to 7.9 days, with an 82.9% reduction in interservice variability. Staffing adjustments prevented understaffing situations; more accurate modeling reduced understaffing from 282 to 4 full-time equivalent staff. A seasonal autoregressive integrated moving average (1,0,0)(0,1,0) model explained 90.3% of forecasted data variability with an absolute percentage error between 2.4% and 8.3%. The project reduced administrative burden, inefficiency, and downtime losses by 47.3%, 53.7%, and 54.5%, respectively. Perceived quality indices improved by 19.6%, and complaints were reduced from 63 to 10 per 10,000 consultation-years. Hospital incomes rose by 49.4%.

Conclusions: Improving efficiency is complex and requires a multimodal approach. Health information systems, patient feedback, and multidisciplinary teams are components that can improve clinical processes.

Am J Manag Care. 2019;25(2):e50-e57Takeaway Points

Our study shows that process reengineering and a patient-centered approach are demonstrable ways to achieve tangible and positive administrative and health service—related outcomes. Predictive modeling reliably forecasts the demand for consultation for specialist care.

  • Patient-centered approaches allow hospitals to prioritize action plans and to engage professionals in workforce planning and modeling future needs.
  • Strategies including process reengineering and a patient-centered approach achieved successful targets in efficiency and perceived quality.
  • Developing a multifaceted concept of a global change management system puts hospitals in a better position to deal with exceptional situations.

Innovation is increasingly necessary in the costly and highly demanding national healthcare systems worldwide. In particular, innovative medical services are essential to drive and stimulate the efficiency and sustainability of public services.1 However, discrepancies in opinion exist on where to focus efforts to improve healthcare while raising awareness about the importance of monitoring the use of resources and reducing ever-growing healthcare spending.2 Many hospitals are focusing on enhancing efficiency through a process reengineering (PR) strategy.3 PR has been implemented worldwide, although the long-term effects on organizational performance, costs, and competitiveness have not yet been proven. Some authors have called for the integration of reengineering and quality management systems to achieve more robust outcomes.4-6 Nevertheless, experience in process improvement has demonstrated that the solution often does not improve the outcome of individual activities, but, instead, leads to an entirely new work approach.7

Recently, electronic health records (EHRs) and health information systems (HIS) have become primary sources of data to potentially support improvements in the efficiency of healthcare processes. However, traditional analysis tools, such as relational databases and statistical tools, have failed to prove their adequacy in evaluating the massive influx of data collected in EHRs. New methodology for big data analysis has broadened the opportunities to discover patterns in complex and heterogeneous data.8-10 However, the knowledge gained from high volumes of data is not enough to help hospitals develop new ways to provide their services, and so the impacts of EHRs and HIS on quality, efficiency, and outcomes of care are currently questionable. Furthermore, although patient-centered approaches in every step of the process are increasingly advocated, limited data exist to assess the effects of interventions on patients’ health status; even fewer data exist on the effects on healthcare service utilization.11-13

Prior studies using EHR data have examined process improvement tools, both concurrent with and independent from quality improvement initiatives.14-20 Most studies have attempted a single-intervention approach (eg, changing referral forms), but these procedural changes almost invariably fall short of expectations to improve quality of healthcare.21,22

As the use of specialized outpatient consult (SOC) divisions is outpacing other types of healthcare provision, wait times in terms of days and the expenses associated with specialized services have increased.23-26 Our study reports the results of a multiyear project to improve utilization and efficiency in an SOC as part of the overall management of a general hospital. In our SOC, the following issues were explored: long wait times for appointments after referrals, low patient satisfaction, seasonal variability in referral volume, fluctuations in provider availability, and income losses. The change strategy described here incorporates PR and patient-centered considerations through multiple approaches, including modeling low- and high-volume periods and staffing accordingly, incorporation of patients’ preferences (as expressed in structured surveys), collaboration with referring primary care physicians (PCPs), and expanding the use of HIS. In doing so, we provide tactical and operational strategies, which, in turn, may boost organizational and process efficiency and effectiveness.

METHODS

The goal of this research is to evaluate the efficiency of SOC interventions after the implementation of a global change management strategy to address PR using information technology—based systems and patient surveys. Table 1 describes the activities and objectives of the program, which had the overall goals to (1) optimize the current systems to increase their efficiency and productivity by enhancing cross-functional performance and (2) build a clinical management model to guide long-term staffing and maximize productivity, ensuring competitiveness.

Further, the secondary objectives of this research are ensuring appropriate referrals from primary care, identifying and solving bottlenecks and non—value-added activities from a patient’s perspective, and shortening the time between the date an appointment is requested and the date for which the appointment is scheduled (eAppendix Figure 1 [eAppendix available at ajmc.com]).

This study used data from 2008 to 2014, with 2008 being a 1-year preintervention period prior to implementation. The study site was the Hospital Universitario del Tajo, a public general hospital in Madrid, Spain, which serves 100,000 people. The SOC provides specialist services for allergies, anesthesiology and perioperative medicine, cardiology, dermatology, endocrinology, gastroenterology and hepatology, internal medicine, general surgery, hematology, laboratory medicine and pathology, nephrology, neurology, obstetrics and gynecology, medical oncology, ophthalmology, orthopedic surgery, otorhinolaryngology, physical medicine and rehabilitation, pulmonary medicine, radiology, rheumatology, specialized pediatric and adolescent medicine, and urology. Patient services at the SOC are funded by the Spanish National Health Service (NHS). Like many other European NHS programs, the Spanish NHS has a service portfolio defined by the healthcare authorities. There is no patient co-payment except for a partial amount required for the purchase of medications. Similar to other NHS programs, patients need their PCP’s referral to have access to specialized care. Hospitals are required to respond to specialist consultation requests within a maximum period of 30 days; otherwise, they are penalized. Furthermore, patients can freely visit any other medical center if the referral process takes longer than 30 days.

Data Collection and Descriptive Analysis

Data were collected from 2 primary sources: the EHR and patient satisfaction surveys. Quantitative analysis consisted of prospective analysis of the EHR information of 1,162,477 consecutive requests for the SOC. Both the hospital and primary care facilities use EHRs and HIS. Therefore, all administrative and clinical activity of the hospital was gathered in the Oracle-based electronic data warehouse (Oracle Corporation; Redwood City, California). Qualitative techniques were used to analyze the surveys of 72,368 SOC patients. Patients were recruited consecutively at the time of an administrative procedure consultation. Structured interviews to assess patients’ perceptions were conducted by trained staff at the Patient Care Department. These surveys included 30 questions related to the appointment management process, managed care, complaints, and suggestions for improvement. The information collected in these surveys constituted a key element of the reengineering process. Additionally, hospital satisfaction surveys were undertaken annually by an external consulting firm using a structured questionnaire for computer-assisted telephone interviews targeting 161 people who used SOC (sampling error ± 1.5%). All statistical analyses were performed using IBM SPSS Statistics for Windows, version 20.0 (IBM Corporation; Armonk, New York).

Outcomes Assessment and Definitions

We selected the following as key performance indicators: operational outcomes, administrative workload management, perceived quality, and hospital income. Operational outcomes included productivity, gap between the number of consultations requested and the number performed (gap), staffing accuracy, wait time, and underlying variability. Productivity was defined as the number of consultations performed by every medical unit, considering human resources in full-time equivalents (FTEs). Staffing accuracy refers to the excess or shortage of scheduled staff. Wait time refers to the time from the consultation request until the time when the consultation was performed. Clinical variability refers to the difference in wait time among medical departments. Effectiveness in administrative workload management refers to the reduction of inefficient work (nonrecorded and no-show visits), overload (nonscheduled, cancelled, and rescheduled visits), and downtime losses. Nonrecorded activity refers to nonbillable visits because of lack of clinical records related to the consultation in EHR. Overload indices refer to additional work performed to accommodate consultations for patients without an appointment, cancellations, and rescheduling of appointments. Downtime losses refer to unused time scheduled for consultation. All of these indices were measured daily, although descriptive and time-series analyses were performed on monthly aggregated data. Income indicates the hospital’s revenues based on the number of valid consultations. Revenues correspond to a fixed price per consultation. The costs and cost savings are calculated based on these prices.

Forecasting and Model Validation

Staffing needs may vary depending on seasonal workload fluctuations. Drawing on the knowledge gathered in the preimplementation period, successive simulated postsample forecasting scenarios of requests and staffing were used. Accordingly, every monthly prediction was translated into scheduling of staff in each medical unit. The discrepancies between the forecast value and observed workload were analyzed. Knowledge gained from these projections fed back to a fast-cycle iteration for additional forecasting analysis that progressively built into a demand-variability model. The final time series pattern was described by trend, seasonality, and random error components. Autoregressive integrated moving average (ARIMA) analysis was used to develop forecast modeling. The model reflected the trend and seasonality in the series so that the data in earlier periods and the errors made in the estimate were compared against the actual values to explain the variability. Once the pattern was established, it was extrapolated to predict future values. The simplest lowest-order model with the lowest error component was used for the final ARIMA model parameters. The stationary R2, mean absolute percentage error (MAPE), and maximum absolute percentage error (MaxAPE) were used to check the quality of the fitted model. MaxAPE measures the variation of a dependent series versus its model-predicted level. The final model was validated by plotting the autocorrelation function residuals and Ljung-Box test. The model calculates the 50th and 95th percentile CIs of the predictions.

Process Reengineering

A process improvement team analyzed the preintervention framework and prioritized the proposed system changes. This steering group was composed of the medical director, nursing director, consultants, and nursing supervisors. They worked in cross-functional teams on a weekly basis and performed a monthly analysis of the balanced scorecard and SOC surveys. Quarterly meetings with referring PCPs provided a review of outcomes and efficacy of the PR program. Finally, high-level annual meetings were held with the hospital general director and primary care manager.

RESULTS

Preintervention

As shown in Table 2, during the preintervention period in 2008, 89,587 consultations were conducted (51,853 first medical visits and 37,734 follow-up visits). On average, the SOC was unable to serve 44.1% and 57.6% of new and follow-up visits, respectively, within the 30-day response period, with significant variability between services (interquartile range [IQR], 22.5%-71.2%). The average wait time was 82.7 days (IQR, 23.8-116.1) prior to service. The waiting list was the main source of patient complaints (48% in total), and 33% of respondents indicated that they were less than satisfied. There were significant fluctuations in demand, with frequent understaffing and overstaffing during high-peak and low-peak periods, respectively, which translated into a staffing gap of more than 6400 hours during the year. Furthermore, more than 3% of patient interactions were not recorded. As such, these visits were not billed in the preintervention period, which led to revenue losses of more than $500,000.

Operational Outcomes

As shown in Table 2, during the PR period, we observed that the number of consultations increased by 88.4%. Overall, per-provider productivity increased most significantly during the first and second years (19% and 10%, respectively), with a 5% annual increase in each of the following years. The total productivity increase for the entire study period was 34% (P = .001). A slight upward increase of 4.9% in demand was observed for the entire study period. The projected increase in demand for the entire period was expected to be 11.2%. The gap between requests and performance decreased by 80% in the second year and remained steady below 9% afterward. In terms of the staffing gap, scores significantly improved, showing a switch from a starting imbalance of 43.7% to 9.2% (282 and 4 understaffing FTEs, respectively).

We further found a significant reduction in administrative management indices (P = .001). In the PR project, there was a 47.3% reduction in administrative overload, 53.7% reduction in work inefficiency, and 54.5% in downtime losses. Additional workload due to rescheduled appointments, unscheduled visits, and patient cancellations dropped by 46%, 48%, and 57%, respectively. Nonrecorded activity decreased by 96%. There was no significant impact on no-show visits (6.8% vs 5.8%; P = .20). Hospital income rose by 49.4% between 2008 and 2014. The savings due to staffing mismatch improvements and due to increased numbers of patients being seen in a 30-day window were $45,833 and $78,238, respectively.

As shown in Figure 1, the average wait time decreased from 82.7 days to 7.9 days (IQR, 7.6-17.3), with a reduction of 82.9% in the average interservice wait time (P = .0001). There was a significant decrease in the total number of complaints, particularly those related to delays in their first visit to the SOC (P = .0001). This improvement was more marked in the first year (77.6%) and it remained steady during the following years.

Forecasting and Staffing Schedule

The scatterplot diagram (Figure 2) of the time series data on appointments for SOC services shows an increase in overall demand during the study period with an underlying annual periodicity, of which seasonality was a prominent feature. Seasonal peaks were observed in October, November, and March, and low demand was observed in July and August. A seasonal ARIMA (1,0,0)(0,1,0) model was progressively developed (eAppendix Figure 2). The modeling predictions performed well and explained 90.3% of the variability of forecasted data. The MAPE across all models was 2.4%. The MaxAPE in the worst-case scenario was 8.3%. Reliability and accuracy of the predictive model steadily increased (Figure 2). Using this model, we were able to reduce mismatch between predicted and actual staffing requirements and narrow the gap over time (Figure 3).

DISCUSSION

Current hospital reengineering substantially differs from early practices.5 PR has been strongly linked to cost-cutting strategies and, to a lesser extent, streamlining delivery systems and improving patient care outcomes. Nevertheless, the benefits of reengineering may go beyond economics if properly implemented with a clear and consistent definition of the goals for systemic change. Change requires a reordering of activities, responsibilities, and the allocation of resources. A high-quality PR program may create an efficient process using HIS information and patient feedback on service quality. An understanding of all the elements of the process—backlogs, demographic characteristics, proportion of acute conditions and chronic disease dysfunctions, profile use of resources, bottlenecks, and coordination problems, among other issues—is critical.21 Concrete action plans may address each of these problems in responding to patients’ demand.

Consequently, workforce planning should be a crucial component of this strategy.27 In this research, techniques for noncausal prediction to modeling backlogs were used. A dynamic adjustment of resources provided useful information on how the resources required for each activity are interconnected.28-30 Results are not immediate because they rely on the length of the historical series, although repeated modeling achieves greater forecasting accuracy. The likelihood of understaffing is lower with more precise staffing allocation, which in turn reduces the need for hiring short-term staff. In our research, improved forecasting led to 282 FTEs per year in savings.

However, hospital process standardization is a challenge. The identification of problems and implementation of improvements continues to be demanding. It is difficult to synchronize care in a traditionally departmentalized organization with poor coordination among units and health-related professionals working semiautonomously. Successful PR requires a steering group to engage others in enhancing organizational performance and cross-functional teams that share objectives and values for a more functional approach to processes.31,32 Traditionally, delayed access to hospital services has been related to a model based on a fixed number of slots per healthcare provider. The improvements we observed will have an impact on healthcare providers’ daily work due to a decrease in waiting lists. This will allow hospitals to focus on managed care for the patient’s benefit.

Finally, optimal PR places patients’ needs at the core of decision-making processes to foster patient engagement in healthcare policy.33-35 To illustrate this, respondents who participated in the structured interviews mentioned the impact of online consultations in dermatology and the usefulness of specialist consultations via telemedicine for chronic disorders. Thus, a patient-centered approach allows hospitals to prioritize action plans and is also a source of innovation.

The collected data strongly suggest that productivity may increase with a more integrated and coordinated system. This, in turn, may lead to an improvement in efficiency, shared responsibilities, and optimal use of financial resources available.36 Another important factor is the need to share and disseminate the project achievements among patients, PCPs, specialists, and hospital leaders to enhance its perceived success and foster a stronger sense of community belonging.

Limitations

Some limitations pertain to this approach. Although PR can be successfully used in an SOC, additional barriers still need to be addressed. First, this approach cannot be applied universally, as each hospital system faces unique challenges. The use of this approach in private institutions may be even more demanding. Thus, the program described here cannot be used as a simple template for other hospitals, but rather as a model that should be customized based on the needs of each institution. Second, the intervention does not address the burden of healthcare overutilization. Although the improved framework has enhanced the perceived quality of the service and reduced complaints, additional efforts are required to monitor overuse of the public health system.37 Finally, despite improved satisfaction and revenue results, the improvements in wait time that we observed do not necessarily correlate with improvements in patient health outcomes.2

CONCLUSIONS

Improving efficiency in healthcare is complex and requires a multimodal approach with HIS, patient feedback, and multidisciplinary teams as integral tools in identifying opportunities to improve clinical processes. 

Acknowledgments

The authors would like to acknowledge the contribution of Elena Flores, interpreter and translator, as well as the anonymous reviewers for their valuable comments and suggestions to improve the quality of the paper.Author Affiliations: Hospital Universitario del Tajo (JAÁ, RFF, JM), Madrid, Spain; Hospital Universitario Puerta de Hierro Majadahonda (JAÁ), Madrid, Spain; University Francisco de Vitoria Medical School (JÁG), Madrid, Spain.

Source of Funding: None.

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (JAÁ); acquisition of data (JAÁ, JM); analysis and interpretation of data (JAÁ, JÁG, JM); drafting of the manuscript (JAÁ, JÁG); critical revision of the manuscript for important intellectual content (JAÁ, RFF); statistical analysis (JAÁ, JÁG, JM); provision of patients or study materials (JAÁ, RFF); administrative, technical, or logistic support (JAÁ, RFF); and supervision (JAÁ).

Address Correspondence to: Jesús Antonio Álvarez, PhD, MD, Hospital Universitario Puerta de Hierro Majadahonda, Calle Manuel de Falla 1, 28222 Majadahonda, Madrid, Spain. Email: jesus.alvarezs@salud.madrid.org.REFERENCES

1. Semple A. Guidance for public authorities on public procurement of innovation. Procurement of Innovation Platform website. innovation-procurement.org/fileadmin/editor-content/Guides/PPI-Platform_Guide_new-final_download.pdf. Accessed June 19, 2018.

2. Segal JB, Nassery N, Chang HY, Chang E, Chan K, Bridges JF. An index for measuring overuse of health care resources with Medicare claims. Med Care. 2015;53(3):230-236. doi: 10.1097/MLR.0000000000000304.

3. EFQM Excellence Model. European Foundation for Quality Management website. www.efqm.org/index.php/efqm-model-2013. Published 2013. Accessed June 24, 2014.

4. Walston SL, Bogue RJ. The effects of reengineering: fad or competitive factor? J Healthcare Manag. 1999;44(6):456-474; discussion 474-476.

5. Walston SL, Urden LD, Sullivan P. Hospital reengineering: an evolving management innovation: history, current status and future direction. J Health Hum Serv Adm. 2001;23(4):388-415.

6. Urden LD, Walston SL. Outcomes of hospital restructuring and reengineering: how is success or failure being measured? J Nurs Adm. 2001;31(4):203-209.

7. Wright CM, Mechling J. The importance of operations management problems in service organizations. Omega. 2002;30(2):77-87.

8. Colla CH, Schpero WL, Gottlieb DJ, et al. Tracking spending among commercially insured beneficiaries using a distributed data model. Am J Manag Care. 2014;20(8):650-657.

9. Graetz I, Reed M, Shortell SM, Rundall TG, Bellows J, Hsu J. The next step towards making use meaningful: electronic information exchange and care coordination across clinicians and delivery sites. Med Care. 2014;52(12):1037-1041. doi: 10.1097/MLR.0000000000000245.

10. Yoo S, Cho M, Kim E, et al. Assessment of hospital processes using a process mining technique: outpatient process analysis at a tertiary hospital. Int J Med Inform. 2016;88:34-43. doi: 10.1016/j.ijmedinf.2015.12.018.

11. Dwamena F, Holmes-Rovner M, Gaulden CM, et al. Interventions for providers to promote a patient-centered approach in clinical consultations. Cochrane Database Syst Rev. 2012;12:CD003267. doi: 10.1002/14651858.CD003267.pub2.

12. Yasaitis LC, Bynum JP, Skinner JS. Association between physician supply, local practice norms, and outpatient visit rates. Med Care. 2013;51(6):524-531. doi: 10.1097/MLR.0b013e3182928f67.

13. Hernandez SE, Conrad DA, Marcus-Smith MS, Reed P, Watts C. Patient-centered innovation in health care organizations: a conceptual framework and case study application. Health Care Manag Rev. 2013;38(2):166-175. doi: 10.1097/HMR.0b013e31825e718a.

14. Shea CM, Malone R, Weinberger M, et al. Assessing organizational capacity for achieving meaningful use of electronic health records. Health Care Manag Rev. 2014;39(2):124-133. doi: 10.1097/HMR.0b013e3182860937.

15. 2016 Committee on Performance Improvement. The imperative for strategic workforce planning and development: challenges and opportunities. American Hospital Association website. aha.org/system/files/2018-02/imperative-strategic-planning-developement-challenges-opportunities-2017.pdf. Published 2017. Accessed March 21, 2017.

16. Peiró S, Artells JJ, Meneu R. Identificación y priorización de actuaciones de mejora de la eficiencia en el Sistema Nacional de Salud. Gac Sanit. 2011;25(2):95-105. doi: 10.1016/j.gaceta.2010.12.005.

17. Borgonovi E, Compagni A. Sustaining universal health coverage: the interaction of social, political, and economic sustainability. Value Health. 2013;16(suppl 1):S34-S38. doi: 10.1016/j.jval.2012.10.006.

18. Isken MK, Rajagopalan B. Data mining to support simulation modeling of patient flow in hospitals. J Med Syst. 2002;26(2):179-197.

19. Rivero Corte P, Alfaro Latorre M; Agencia de Calidad del Sistema Nacional de Salud, Instituto de Información Sanitaria. Estadística de establecimientos sanitarios con régimen de internado en España. Evolución 2000-2008. Madrid, 2009. Gobierno de España Ministerio de Sanidad website. www.mscbs.gob.es/en/estadEstudios/estadisticas/docs/Evolutivo_2000-2008.pdf. Accessed June 25, 2012.

20. Annual Report on the National Health System of Spain 2012. Madrid, Spain: Ministry of Health, Social Services and Equality; 2011. Gobierno de España Ministerio de Sanidad website. www.mscbs.gob.es/estadEstudios/estadisticas/sisInfSanSNS/tablasEstadisticas/EnglishSNSReport2012.pdf. Accessed February 25, 2013.

21. Dettmer HW. The Logical Thinking Process: A Systems Approach to Complex Problem Solving. Milwaukee, WI: ASQ Quality Press; 2007.

22. Wagner EH, Sandhu N, Coleman K, Phillips KE, Sugarman JR. Improving care coordination in primary care. Med Care. 2014;52(11, suppl 4):S33-S38. doi: 10.1097/MLR.0000000000000197.

23. Blumenthal D. Performance improvement in health care—seizing the moment. N Engl J Med. 2012;366(21):1953-1955. doi: 10.1056/NEJMp1203427.

24. Schappert SM, Burt CW. Ambulatory care visits to physician offices, hospital outpatient departments, and emergency departments: United States, 2001-02. Vital Health Stat 13. 2006;13(159):1-66.

25. OECD Health Statistics [database]. Organization for Economic Co-operation and Development website. doi: 10.1787/data-00542-en.

26. Narayan PK. Examining structural breaks and growth rates in international health expenditures. J Health Econ. 2006;25(5):877-890. doi: 10.1016/j.jhealeco.2005.12.001.

27. Cronin PR, Kimball AB. Success of automated algorithmic scheduling in an outpatient setting. Am J Manag Care. 2014;20(7):570-576.

28. Kumar A, Shim SJ. Eliminating emergency department wait by BPR implementation. Presented at: 2007 International Conference on Industrial Engineering and Engineering Management; December 2-4, 2017; Singapore. doi: 10.1109/IEEM.2007.4419478.

29. Sinreich D, Jabali O. Staggered work shifts: a way to downsize and restructure an emergency department workforce yet maintain current operational performance. Health Care Manag Sci. 2007;10(3):293-308.

30. De Gooijer JG, Hyndman RJ. 25 years of time series forecasting. Int J Forecasting. 2006;22(3):443-473. doi: 10.1016/j.ijforecast.2006.01.001.

31. Jonk Y, Lawson K, O’Connor H, et al. How effective is health coaching in reducing health services expenditures? Med Care. 2015;53(2):133-140. doi: 10.1097/MLR.0000000000000287.

32. Büchner VA, Hinz V, Schreyögg J. Cooperation for a competitive position: the impact of hospital cooperation behavior on organizational performance. Health Care Manage Rev. 2015;40(3):214-224. doi: 10.1097/HMR.0000000000000027.

33. Fleurence R, Whicher D, Dunham K, Gerson J, Newhouse R, Luce B. The Patient-Centered Outcomes Research Institute’s role in advancing methods for patient-centered outcomes research. Med Care. 2015;53(1):2-8. doi: 10.1097/MLR.0000000000000244.

34. Aboumatar HJ, Chang BH, Al Danaf J, et al. Promising practices for achieving patient-centered hospital care: a national study of high-performing US hospitals. Med Care. 2015;53(9):758-767. doi: 10.1097/MLR.0000000000000396.

35. Huerta TR, Harle CA, Ford EW, Diana ML, Menachemi N. Measuring patient satisfaction’s relationship to hospital cost efficiency: can administrators make a difference? Health Care Manage Rev. 2016;41(1):56-63. doi: 10.1097/HMR.0000000000000045.

36. Litvak E, Long MC. Cost and quality under managed care: irreconcilable differences? Am J Manag Care. 2000;6(3):305-312.

37. Guy R, Hocking J, Wand H, Stott S, Ali H, Kaldor J. How effective are short message service reminders at increasing clinic attendance? a meta-analysis and systematic review. Health Serv Res. 2012;47(2):614-632. doi: 10.1111/j.1475-6773.2011.01342.x.