User login
Using Clinical Decision Support to Reduce Inappropriate Imaging: A Health Care Improvement Case Study
From the Office of Science Policy and Communications, National Institute on Drug Abuse, National Institutes of Health, Rockville, MD, and George Washington University, Washington, DC (Dr. Jones), Office of the National Coordinator for Health Information Technology, US Department of Health and Human Services, Washington, DC (Mr. Swain), and Banner Health, Phoenix, AZ (Ms. Burdick).
Abstract
- Objective: Clinical decision support (CDS) can be a useful tool to decrease inappropriate imaging by providing evidence-based information to clinicians at the point of care. The objective of this case study is to highlight lessons from a health care improvement initiative using CDS to encourage use of ultrasound rather than computed tomography (CT) scans as an initial diagnostic tool for suspected appendicitis in pediatric patients.
- Methods: The percentage of suspected pediatric appendicitis cases receiving ultrasounds and CT scans was calculated using electronic health record data. Four steps for implementing health information technology were identified in a literature scan that guided data collection and analysis: planning, software customization and workflow design, training and user support, and optimization.
- Results: During the fourth quarter of 2010, 1 in 7 pediatric patients with suspected appendicitis received an ultrasound and almost half received a CT scan. By the first quarter of 2012, ultrasounds were performed in 40.8% of these cases and the use of CT scans declined to 39.9% of suspected pediatric appendicitis cases.
- Conclusion: Four lessons emerged. First, all levels of staff should be involved in the planning process to make organizational priorities actionable and build buy-in for each healthcare improvement initiative. Second, it takes time to design and test the alert to ensure that clinical guidelines are being properly applied. Third, re-engineering the workflow is critical for usability; in this case, ensuring the availability of ultrasound staff was particularly important. Finally, the effectiveness of CDS depends on applying relevant evidence-based practice guidelines to real-time patient data.
Diagnostic imaging is a useful tool for identifying and guiding the treatment of many health conditions, but evidence indicates that health care providers do not always use imaging appropriately. In fact, a substantial proportion of diagnostic imaging procedures performed in hospital and ambulatory settings are not supported by clinical guideline recommendations [1,2]. Spending on diagnostic imaging is rapidly increasing, and some patients receive unnecessary radiation exposure that can lead to adverse health impacts [3]. Inappropriate imaging falls into 3 broad categories: imaging that does not conform to clinical guidelines, imaging that is contraindicated due to an allergy or implantable medical device, and imaging that might be clinically indicated but is duplicative of prior imaging services.
Clinical decision support (CDS) functionality supports health care improvement initiatives to narrow the gap between evidence-based practices and routine care [4]. CDS merges patient-specific clinical information with relevant information about evidence-based practices, providing health care providers with timely information to guide decisions at the point of care [5]. Decision support is most commonly delivered in the form of alerts and reminders [6]. CDS can be effective in reducing adverse drug events [7], sepsis [8,9], and other conditions in hospital [10–12] and ambulatory settings [13,14].
For the evaluation of suspected appendicitis in children, ultrasound is the preferred initial consideration for imaging examination [15]. Evidence suggests that CDS can increase the use of ultrasound for suspected pediatric appendicitis [16,17] and has affirmed the utility of ultrasound as a first-line diagnostic tool for suspected appendicitis [18,19]. In the Choosing Wisely campaign, the American College of Surgeons and the American College of Radiology have both endorsed ultrasound as an option to consider prior to conducting a CT scan to evaluate suspected appendicitis in children [15].
Banner Health, a large health system headquartered in Phoenix, Arizona, implemented a health care improvement initiative using CDS functionality to encourage providers to use ultrasound instead of CT as a first-line diagnostic tool for suspected pediatric appendicitis. We conducted a site visit to Banner Health, an organization who had had attained a high score on the EMR Adoption Model [20] to examine their implementation process. We sought to build on previous research examining the use of health information technology to improve performance in large health systems [21–23].
Methods
Setting
Banner Health is a large not-for-profit health system that is comprised of 24 acute care hospitals across several states, as well as ambulatory medical practices, behavioral health, home care, and ambulatory surgery centers [24,25]. The health system is the largest employer in Arizona and one of the largest in the United States with over 50,000 employees. Banner Health has been nationally recognized for clinical quality [26], an innovative leadership team [27], and using health IT to improve quality [20]. The health system was also selected as one of the Centers for Medicare & Medicaid Services (CMS) Pioneer Accountable Care Organizations.
Site Visit
The first 2 authors conducted a 2-day site visit to the Banner Health headquarters in Phoenix, Arizona in November 2013. The team conducted discussions with over 20 individuals, including health system leadership, frontline clinicians in several units of an acute care hospital, staff members in 2 telehealth hubs—including a tele-ICU hub—and trainers in a simulation facility that is used for staff training. The discussions were conducted with groups of staff or on an individual basis, as appropriate. At the outset of the project, an environmental scan of relevant grey and peer-reviewed literature was conducted under contract on behalf of the authors to guide data collection and analysis [28]. An interview protocol was created to guide the discussions. The protocol contained modules that were used during each discussion, if relevant. The modules addressed topics such as technical issues with designing and deploying health information technology functionalities such as clinical decision support systems, the organizational processes and structures needed to launch health care improvement initiatives, and using health information technology care coordination. Within each module, questions probed about the challenges that arose and the solutions to these challenges, with a focus on the four phases of implementing a health information technology intervention: functionality planning, software customization and workflow design, training and user support, and optimization. To assist with interpreting the qualitative findings, an evolving outline of the findings was maintained. Salient themes and conceptual categories were tracked, which helped the researchers organize, synthesize, and interpret the information collected during the site visit. Once the authors chose to focus on clinical decision support, summary notes from the discussions were reviewed for relevant information, and this information was compiled and organized under the rubric of the four implementation phases. The findings and key themes from the discussion notes were distilled into key lessons for the field.
Data obtained included the percentage of pediatric patients with suspected appendicitis who received ultrasounds and CT scans each month from 1 October 2010 through 31 March 2012. Banner Health staff originally collected the data to support the implementation of health care improvement initiative; the use of these data in this paper is a secondary use [29].
This manuscript was prepared using the SQUIRE 2.0 guidelines [30]. No patient-identifiable data were used, so institutional review board approval was not sought.
Results
The 4 steps of implementing CDS can be described as functionality planning, software customization and workflow design, training and user support, and optimization [31].
Pre-Implementation
The use of computerized provider order entry (CPOE) is a precursor to using clinical decision support, since orders must be entered electronically to be subject to CDS review. Banner Health deployed CPOE to its various facilities starting in 2008. The deployment was staged in a rolling fashion with one or two facilities going live every few months so that the deployment team was available at each facility.
Phase 1: Planning
In contrast to many large health systems, the organization has a single board of directors that oversees the entire system of over 37,000 employees. Activities and relationships to promote the use of evidence-based practices are built into the organizational structure. For example, Banner Health maintains a Care Management Council, a group comprised of clinical and administrative leadership to provide executive oversight of health care improvement projects. The Council convenes on a quarterly basis to review and approve the adoption of new clinical practice guidelines, policies, and standardized standing orders that have been developed by multidisciplinary groups of physicians and other clinicians. A key focus of the Council is ensuring consistent application of evidence-based guidelines to clinical care and disseminating knowledge of clinical best practices across a large and complex enterprise.
Interdisciplinary clinical consensus groups support the Council’s work. These groups are comprised of administrative and program management staff, physicians and other clinicians, and engineers. Each clinical consensus group focuses on emerging issues and improvement opportunities within a specific clinical domain and leads the implementation of health care improvement initiatives in that domain. Providers and staff at all levels of the organization were involved in planning and implementing the health care improvement initiative in inappropriate imaging. This increased buy-in and staff support, which are associated with successful health care improvement initiatives [32]. Banner Health staff rallied around the idea of addressing inappropriate imaging as a key priority initiative. The teams that implement each initiative include an engineer that focuses on redesigning clinical workflows for each initiative. There is also an organizational unit responsible for project management that provides teams with logistical and operational support.
Phase 2: Software Customization and Workflow Redesign
Once the clinical consensus group selected inappropriate imaging as a priority, the next step was to examine the process flow for imaging ordering. In 2011 Banner Health integrated CDS functionality with CPOE into the electronic health record. Before the use of CDS, inpatient and emergency department imaging orders were simply transmitted to imaging staff after the order was entered. After CDS implementation, the process flow begins with an inpatient imaging order and entailed checking the order against clinical guidelines on the proper use of imaging. If the image order did not conform to guidelines, which in this case indicate that ultrasound should be used before CT scans as a diagnostic tool for suspected pediatric appendicitis, the CDS system triggered an alert [15].
Bringing the perspective and skill sets of engineers to the process of redesigning clinical workflows was particularly valuable [33]. While CDS has the potential to reduce inpatient inappropriate imaging, effectiveness depends on adjusting workflows to ensure that the information provided by CDS alerts and reminders is actionable. To reduce alert fatigue among the clinical staff, the team identified the appropriate level of workflow interruption for each alert and reminder (hard stop, workflow interruption, or informational) [5,6].
The design principles that were used to design the alert include intuitive system development to promote ease of use, one set of screen formats and data definitions, and a set of consistent core reports and standard system output formats across facilities. The alert’s appearance was tailored for maximal impact and covered most of the screen. Color contrast was used, but since some people are color-blind, the meaning of the alert did not depend on the color contrast. The alerts included recommendations for changing the treatment plan to encourage using ultrasound as a first-line diagnostic tool. Minimizing the number of clicks to accept the proposed treatment plan change in the alert is desirable.
Phase 3: Training and User Support
Training and support structures and tools were critical to the rollout of the inappropriate imaging alerts. Providers were reminded about clinical best practices and informed during staff meetings about the new CDS rules. In addition, various types of training and support were available to clinicians and staff during the rollout process. Dedicated time for end-user training provided an opportunity to identify and cultivate super-users. These super-users not only helped provide technical support to their colleagues, but also helped create excitement for the initiative. A centralized support desk provided telephone support for providers in facilities throughout the Banner Health system. Site managers were provided toolkits to support providers and staff throughout the implementation process. The toolkits included frequently asked questions and answers, and were maintained as ‘living documents’ that were updated based on emerging issues and questions.
To keep things on track, project managers from the central project management department were involved in the initiative to provide direct project management services to the initiative. They also worked to instill project management competencies throughout the organization, applying a train-the-trainer approach to disseminate best practices for enhancing communication among team members, implementing workflow changes, and monitoring the results.
Phase 4: Optimization
The optimization phase is continuous and continues to the present day. Notably, the success of the CDS rules depends on the availability of current clinical information for each patient, in addition to information about the treatment plan. For this initiative, Banner Health maintained aggregated clinical patient data in the data warehouse that aggregated data from disparate sources, including billing and EHR data from different care settings such as ambulatory offices, inpatient units, the emergency department, home care, and ambulatory surgery centers. The data warehouse is housed in a strategically chosen physical location to minimize the threat of natural disasters, and cloud-based backup is also used. A master patient index and provider credentialing system are integrated with the data warehouse. Query-based health information exchange is used, when possible, to collect information on care received by patients outside of the Banner Health system.
It is important to note that many CDS alerts are over-ridden without changes to clinical care [34]. Previous research indicates that alert fatigue from “false positives” can impede the effectiveness of alerts [35]. Banner Health monitors the rate at which CDS alerts are over-ridden. Figure 1 shows the percentage of all alerts for radiation exposure—including the alert related to using ultrasound as a diagnostic tool for pediatric appendicitis—that led to order cancellations. The percentage of CT orders that generated the alert and were cancelled fell from 18.9% in March 2011 to 13.6% in February 2012. The rate of order cancellations might have declined over time due to a change in provider behavior from the alert. That is, if inappropriate CT scan orders declined over time, then providers would cancel a decreasing percentage of the CT scan orders that prompted an alert. 
Imaging Use
In Figure 2, data on the use of the 2 imaging procedures for the diagnosis of pediatric appendicitis is presented. During the fourth quarter of 2010, almost half of pediatric patients with suspected appendicitis received a CT scan and only about 1 in 7 received an ultrasound. After the clinical decision support alert was put in place to remind providers to perform an ultrasound as a first-line diagnostic tool, the use of ultrasounds increased sharply. By the first quarter of 2012, ultrasounds were performed in 40.8% of these cases and the use of CT scans declined to 39.9% of suspected pediatric appendicitis cases. 
Discussion
This case study discusses the application of CDS functionality in a health care improvement initiative to address inappropriate imaging in a large health system. Four main implementation lessons emerge for the field. First, it is important to involve all levels of staff in the planning process to ensure that health care improvement activities are prioritized correctly and to build buy-in for the priorities addressed with health care improvement activities. Second, it is necessary to allow time to design the alert or reminder, as well as testing it during the implementation process to ensure that clinical guidelines are being properly applied. Third, re-engineering the workflow and ensuring usability of the alert or reminder are important, and using the skills of trained engineers helps in this process. Ensuring the availability of trained ultrasound staff was particularly important to this initiative. Finally, the effectiveness of CDS depends on having complete data for each patient, as well as up-to-date information on the relevant evidence-based practice guidelines.
These results can help guide the implementation of health care improvement initiatives that use CDS functionality to address inappropriate imaging. The adoption of electronic health records with CDS functionality was incentivized and supported by the Medicare and Medicaid Electronic Health Record Incentive Programs; the Medicare program now exists as part of MACRA. Using CDS to reduce inappropriate imaging is required for Medicare fee-for-service patients in the 2014 Protecting Access to Medicare Act (PAMA), highlighting the critical nature of these results, which can guide implementation of CDS to reduce inappropriate imaging [41].
As noted above, the optimization phase is continuous. Banner Health still encourages use of ultrasounds as a first-line diagnostic tool for pediatric appendicitis. Identifying which patients should immediately receive CT scans is difficult, and sometimes the decision depends on the availability of staff to conduct the ultrasound scans. Ways to maximize the productivity of ultrasound technicians have been explored. Another focus area since the original implementation of this health care improvement initiative has been health information exchange, to ensure that complete, up-to-date information is available for each patient.
Banner Health often implements CDS in conjunction with other health IT functionalities. For example, CDS and telehealth are used together to improve care in the intensive care unit (ICU) for patients with sepsis and delirium. An offsite hub of experienced ICU physicians and nurses remotely monitors ICU patients in facilities across Banner Health, using cameras with zoom capability. The intensive care specialists in the tele-hub act as part of the care team; in addition to receiving video feed, they communicate verbally with patients and ICU staff members. Predictive analytics are used to generate clinical decision support alerts and reminders, with a special focus on early intervention if a patient’s clinical indicators are trending downward. The 4 lessons described in this study were also used in the ICU sepsis and delirium initiative; staff were involved in the planning process, alerts and reminders were thoroughly tested, the workflow was adjusted to accommodate the physicians in the tele-ICU hub, and up-to-date and complete clinical information for each patient is maintained. In addition, the design principles for alerts described in this study, such as covering most of the screen and providing recommendations for changing the treatment plan within the alert itself, were also used in the ICU sepsis and delirium initiative.
One limitation of this study is that it was conducted at a single health system. Thus, the findings might not be generalizable to other health systems, particularly if a robust health IT infrastructure is not in place. The culture of Banner Health values quality and involved providers and staff at all levels in selecting and implementing health care improvement initiatives. In addition, engineers assisted with implementation. Finally, the study design does not permit conclusions about the causality of the decline in CT scans and the increase in ultrasounds for suspected pediatric appendicitis cases; unobserved factors might have contributed to the changes in CT and ultrasound use.
Future research should focus on ways to improve the implementation and organization learning process, particularly through engagement of frontline staff by leadership [36] and explore how to operationalize previous findings indicating that innovations in hospital settings are more likely to be sustained when intrinsically rewarding to staff, either by making clinician and staff jobs easier to perform or more gratifying [37]. Future research should focus on facilitating health information exchange between providers in different health systems.
Disclaimer: The views expressed in the article are solely the views of the authors and do not represent those of the National Institutes of Health or the U.S. Government.
Acknowledgments: The authors wish to thank the Banner Health team for taking time to share their insights on how health information technology can be used for health care improvement initiatives, especially John Hensing. We also thank Michael Furukawa of the Agency for Healthcare Research and Quality, formerly of the Office of the National Coordinator for Health Information Technology, who played a key role in the conceptualization of this study and data collection.
Corresponding author: Emily Jones, PhD, MPP, National Institutes of Health, 6001 Executive Blvd., #5232 Rockville, MD 20852, emilybjones@gmail.com.
Financial disclosures: None
1. Lehnert B, Bree R. Analysis of appropriateness of outpatient CT and MRI referred from primary care clinics at an academic medical center: how critical is the need for improved decision support? J Am Coll Radiol 2010;7:192–7.
2. Ip I, Schneider L, Hanson R, et al. Adoption and meaningful use of computed physician order entry with an integrated clinical decision support system for radiology: ten-year analysis in an urban teaching hospital. J Am Coll Radiol 2012;9:129–36.
3. Bernardy M, Ullrich C, Rawson J, et al. Strategies for managing imaging utilization. J Am Coll Radiol 2009;6:844–50.
4. Amland R, Dean B, Yu HT et al. Computed clinical decision support to prevent venous thromboembolism among hospitalized patients: proximal outcomes from a multiyear quality improvement project. J Healthcare Qual 2015;37:221–31.
5. Kahn C. Improving outcomes in radiology: bringing computer-based decision support and education to the point of care. Acad Radiology 2005;12:409–14.
6. Phansalkar S, Desai A, Bell D et al. High-priority drug-drug interactions for use in electronic health records. J Am Med Inform Assoc 2012;19:735–43.
7. Wolfstadt J, Gurwitz J, Field T, et al. The effect of computed physician order entry with clinical decision support on the rates of adverse drug events: a systematic review. J Gen Intern Med 2008;23:451–8.
8. Amland R, Hahn-Cover K. Clinical decision support for early recognition of sepsis. Am J Med Qual 2014;1–8.
9. Amland R, Haley J, Lyons J. A multidisciplinary sepsis program enabled by a two-stage clinical decision support system: factors that influence patient outcomes. Am J Med Qual 2015;1–8.
10. Umscheid C, Hanish A, Chittams J, et al. Effectiveness of a novel and scalable clinical decision support intervention to improve venous thromboembolism prophylaxis: a quasi-experimental study. BMC Med Inform Dec Making 2012;12:92–104.
11. Mack EH, Wheeler DS, Embi PJ. Clinical decision support systems in the pediatric intensive care unit. Pediatric Crit Care Med 2009;10:23–8.
12. Kollef M, Heard K, Chen Y, et al. Mortality and length of stay trends following implementation of a rapid response system and real-time automated clinical deterioration alerts. Am J Med Qual 2015; online first.
13. Ali S, Giordano R, Lakhani S, Walker D. A review of randomized controlled trials of medical record powered clinical decision support system to improve quality of diabetes care. Int J Med Informatics 2016;87:91–100.
14. Gill J, Mainous A, Koopman R et al. Impact of EHR-based clinical decision support on adherence to guidelines for patients on NSAIDs: a randomized controlled trial. Ann Fam Med 2011;9:22–30.
15. Choosing Wisely. Accessed 1 May 2017 at http://www.choosingwisely.org/clinician-lists/#keyword=appendicitis.
16. Hendrickson M, Wey A, Gaillard P, Kharbanda A. Implementation of an electronic clinical decision support tool for pediatric appendicitis within a hospital network. Pediatric Emerg Care 2017 (online first).
17. Kharbanda A, Madhok M, Krause E, et al. Implementation of electronic clinical decision support for pediatric appendicitis. Pediatrics 2016;137:e20151745.
18. Schuh S, Chan K, Langer J, et al. Properties of serial ultrasound clinical diagnostic pathway in suspected appendicitis and related computed tomography use. Acad Emerg Med 2015;22:406–14.
19. Ramarajan N, Krishnamoorthi R, Barth R, et al. An interdisciplinary initiative to reduce radiation exposure: evaluation of appendicitis in a pediatric emergency department with clinical assessment supported by a staged ultrasound and computed tomography pathway. Acad Emerg Med 2009;16:1258–65.
20. HIMSS Analytics. Stage 7 Hospitals. Accessed at www.himssanalytics.org/emram/stage7Hospitals.aspx.
21. Rizer M, et al. Top 10 lessons learned from electronic health record implementation in a large academic medical center. Perspectives in Health Information Management. Summer 2015.
22. Cosgrove DM, Fisher M, Gabow P, et al. Ten strategies to lower costs, improve quality, and engage patients: the view from leading health system CEOs. Health Aff (Millwood) 2013;32:321–7.
23. Cresswell KM, Bates DW, Sheikh A. Ten key considerations for the successful implementation and adoption of large-scale health information technology. J Am Med Inform Assoc 2013 Apr 18.
24. Hensing JA. The quest for upper-quartile performance at Banner Health. J Healthc Qual 2008;30:18–24
25. Hensing J, Dahlen D, Warden M, et al. Measuring the benefits of IT-enabled care transformation. Healthc Financ Manage 2008;62:74–80.
26. Truven Health Analytics. 15 Top Health Systems Study. 6th ed. April 2014. Accessed at http://100tophospitals.com/portals/2/assets/15-Top-Health-Systems-Study.pdf.
27. Aiello M. 2011 Top leadership team awards recognize big moves. Health Leaders Media. August 2011. Accessed at www.healthleadersmedia.com/page-2/LED-269808/2011-Top-Leadership-Team-Awards-Recognize-Big-Moves.
28. Rosenthal D, Stout M. Radiology order entry: features and performance requirements. J Am Coll Radiol 2006;3:554–7.
29. Kirsh S, Wu WC, Edelman D, Aron D. Research versus quality improvement: distinct or a distinction without a difference? A case study comparison of two studies. Jt Comm J Qual Patient Safety 2014;40:365–75.
30. Ogrinc G, Davies L, Goodman D, et al. SQUIRE 2.0 (Standards for Quality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process. BMJ Qual Saf 2015;0:1–7.
31. Blavin F, Ramos C, Shah A, Devers K. Lessons from the literature on electronic health record implementation.1 Aug 2013. The Urban Institute. Prepared for the Office of the National Coordinator for Health Information Technology. Available at www.urban.org/research/publication/lessons-literature-electronic-health-record-implementation.
32. Needleman J, Pearson ML, Upenieks VV, et al. Engaging frontline staff in performance improvement: the American Organization of Nurse Executives implementation of transforming care at the beside collaborative. Jt Comm J Qual Patient Safety 2016;42:61–9.
33. Jones E, Swain M, Patel V, Furukawa M. Supporting HITECH implementation and assessing lessons for the future: the role of program evaluation. Healthcare: The Journal of Delivery Science and Innovation 2014;2:4–8.
34. Phansalkar S, Zachariah M, Seidling H, et al. Evaluation of medication alerts in electronic health records for compliance with human factors principles. J Am Med Inform Assoc 2014;21:e332–e340.
35. Handler S, Altman R, Perera S, et al. A systematic review of the performance characteristics of clinical event monitor signals to detect adverse drug events in the hospital setting. J Am Med Inform Assoc 2007;14:451–8.
36. Singer S, Rivard P, Hayes J, et al. Improving patient care through leadership engagement with frontline staff: a Department of Veteran Affairs study. Jt Comm J Qual Patient Safety 2013;39):349–60.
37. Brewster A, Curry L, Cherlin E, et al. Integrating new practices: a qualitative study of how hospital innovations become routine. Implement Sci 2015;5(10):168.
From the Office of Science Policy and Communications, National Institute on Drug Abuse, National Institutes of Health, Rockville, MD, and George Washington University, Washington, DC (Dr. Jones), Office of the National Coordinator for Health Information Technology, US Department of Health and Human Services, Washington, DC (Mr. Swain), and Banner Health, Phoenix, AZ (Ms. Burdick).
Abstract
- Objective: Clinical decision support (CDS) can be a useful tool to decrease inappropriate imaging by providing evidence-based information to clinicians at the point of care. The objective of this case study is to highlight lessons from a health care improvement initiative using CDS to encourage use of ultrasound rather than computed tomography (CT) scans as an initial diagnostic tool for suspected appendicitis in pediatric patients.
- Methods: The percentage of suspected pediatric appendicitis cases receiving ultrasounds and CT scans was calculated using electronic health record data. Four steps for implementing health information technology were identified in a literature scan that guided data collection and analysis: planning, software customization and workflow design, training and user support, and optimization.
- Results: During the fourth quarter of 2010, 1 in 7 pediatric patients with suspected appendicitis received an ultrasound and almost half received a CT scan. By the first quarter of 2012, ultrasounds were performed in 40.8% of these cases and the use of CT scans declined to 39.9% of suspected pediatric appendicitis cases.
- Conclusion: Four lessons emerged. First, all levels of staff should be involved in the planning process to make organizational priorities actionable and build buy-in for each healthcare improvement initiative. Second, it takes time to design and test the alert to ensure that clinical guidelines are being properly applied. Third, re-engineering the workflow is critical for usability; in this case, ensuring the availability of ultrasound staff was particularly important. Finally, the effectiveness of CDS depends on applying relevant evidence-based practice guidelines to real-time patient data.
Diagnostic imaging is a useful tool for identifying and guiding the treatment of many health conditions, but evidence indicates that health care providers do not always use imaging appropriately. In fact, a substantial proportion of diagnostic imaging procedures performed in hospital and ambulatory settings are not supported by clinical guideline recommendations [1,2]. Spending on diagnostic imaging is rapidly increasing, and some patients receive unnecessary radiation exposure that can lead to adverse health impacts [3]. Inappropriate imaging falls into 3 broad categories: imaging that does not conform to clinical guidelines, imaging that is contraindicated due to an allergy or implantable medical device, and imaging that might be clinically indicated but is duplicative of prior imaging services.
Clinical decision support (CDS) functionality supports health care improvement initiatives to narrow the gap between evidence-based practices and routine care [4]. CDS merges patient-specific clinical information with relevant information about evidence-based practices, providing health care providers with timely information to guide decisions at the point of care [5]. Decision support is most commonly delivered in the form of alerts and reminders [6]. CDS can be effective in reducing adverse drug events [7], sepsis [8,9], and other conditions in hospital [10–12] and ambulatory settings [13,14].
For the evaluation of suspected appendicitis in children, ultrasound is the preferred initial consideration for imaging examination [15]. Evidence suggests that CDS can increase the use of ultrasound for suspected pediatric appendicitis [16,17] and has affirmed the utility of ultrasound as a first-line diagnostic tool for suspected appendicitis [18,19]. In the Choosing Wisely campaign, the American College of Surgeons and the American College of Radiology have both endorsed ultrasound as an option to consider prior to conducting a CT scan to evaluate suspected appendicitis in children [15].
Banner Health, a large health system headquartered in Phoenix, Arizona, implemented a health care improvement initiative using CDS functionality to encourage providers to use ultrasound instead of CT as a first-line diagnostic tool for suspected pediatric appendicitis. We conducted a site visit to Banner Health, an organization who had had attained a high score on the EMR Adoption Model [20] to examine their implementation process. We sought to build on previous research examining the use of health information technology to improve performance in large health systems [21–23].
Methods
Setting
Banner Health is a large not-for-profit health system that is comprised of 24 acute care hospitals across several states, as well as ambulatory medical practices, behavioral health, home care, and ambulatory surgery centers [24,25]. The health system is the largest employer in Arizona and one of the largest in the United States with over 50,000 employees. Banner Health has been nationally recognized for clinical quality [26], an innovative leadership team [27], and using health IT to improve quality [20]. The health system was also selected as one of the Centers for Medicare & Medicaid Services (CMS) Pioneer Accountable Care Organizations.
Site Visit
The first 2 authors conducted a 2-day site visit to the Banner Health headquarters in Phoenix, Arizona in November 2013. The team conducted discussions with over 20 individuals, including health system leadership, frontline clinicians in several units of an acute care hospital, staff members in 2 telehealth hubs—including a tele-ICU hub—and trainers in a simulation facility that is used for staff training. The discussions were conducted with groups of staff or on an individual basis, as appropriate. At the outset of the project, an environmental scan of relevant grey and peer-reviewed literature was conducted under contract on behalf of the authors to guide data collection and analysis [28]. An interview protocol was created to guide the discussions. The protocol contained modules that were used during each discussion, if relevant. The modules addressed topics such as technical issues with designing and deploying health information technology functionalities such as clinical decision support systems, the organizational processes and structures needed to launch health care improvement initiatives, and using health information technology care coordination. Within each module, questions probed about the challenges that arose and the solutions to these challenges, with a focus on the four phases of implementing a health information technology intervention: functionality planning, software customization and workflow design, training and user support, and optimization. To assist with interpreting the qualitative findings, an evolving outline of the findings was maintained. Salient themes and conceptual categories were tracked, which helped the researchers organize, synthesize, and interpret the information collected during the site visit. Once the authors chose to focus on clinical decision support, summary notes from the discussions were reviewed for relevant information, and this information was compiled and organized under the rubric of the four implementation phases. The findings and key themes from the discussion notes were distilled into key lessons for the field.
Data obtained included the percentage of pediatric patients with suspected appendicitis who received ultrasounds and CT scans each month from 1 October 2010 through 31 March 2012. Banner Health staff originally collected the data to support the implementation of health care improvement initiative; the use of these data in this paper is a secondary use [29].
This manuscript was prepared using the SQUIRE 2.0 guidelines [30]. No patient-identifiable data were used, so institutional review board approval was not sought.
Results
The 4 steps of implementing CDS can be described as functionality planning, software customization and workflow design, training and user support, and optimization [31].
Pre-Implementation
The use of computerized provider order entry (CPOE) is a precursor to using clinical decision support, since orders must be entered electronically to be subject to CDS review. Banner Health deployed CPOE to its various facilities starting in 2008. The deployment was staged in a rolling fashion with one or two facilities going live every few months so that the deployment team was available at each facility.
Phase 1: Planning
In contrast to many large health systems, the organization has a single board of directors that oversees the entire system of over 37,000 employees. Activities and relationships to promote the use of evidence-based practices are built into the organizational structure. For example, Banner Health maintains a Care Management Council, a group comprised of clinical and administrative leadership to provide executive oversight of health care improvement projects. The Council convenes on a quarterly basis to review and approve the adoption of new clinical practice guidelines, policies, and standardized standing orders that have been developed by multidisciplinary groups of physicians and other clinicians. A key focus of the Council is ensuring consistent application of evidence-based guidelines to clinical care and disseminating knowledge of clinical best practices across a large and complex enterprise.
Interdisciplinary clinical consensus groups support the Council’s work. These groups are comprised of administrative and program management staff, physicians and other clinicians, and engineers. Each clinical consensus group focuses on emerging issues and improvement opportunities within a specific clinical domain and leads the implementation of health care improvement initiatives in that domain. Providers and staff at all levels of the organization were involved in planning and implementing the health care improvement initiative in inappropriate imaging. This increased buy-in and staff support, which are associated with successful health care improvement initiatives [32]. Banner Health staff rallied around the idea of addressing inappropriate imaging as a key priority initiative. The teams that implement each initiative include an engineer that focuses on redesigning clinical workflows for each initiative. There is also an organizational unit responsible for project management that provides teams with logistical and operational support.
Phase 2: Software Customization and Workflow Redesign
Once the clinical consensus group selected inappropriate imaging as a priority, the next step was to examine the process flow for imaging ordering. In 2011 Banner Health integrated CDS functionality with CPOE into the electronic health record. Before the use of CDS, inpatient and emergency department imaging orders were simply transmitted to imaging staff after the order was entered. After CDS implementation, the process flow begins with an inpatient imaging order and entailed checking the order against clinical guidelines on the proper use of imaging. If the image order did not conform to guidelines, which in this case indicate that ultrasound should be used before CT scans as a diagnostic tool for suspected pediatric appendicitis, the CDS system triggered an alert [15].
Bringing the perspective and skill sets of engineers to the process of redesigning clinical workflows was particularly valuable [33]. While CDS has the potential to reduce inpatient inappropriate imaging, effectiveness depends on adjusting workflows to ensure that the information provided by CDS alerts and reminders is actionable. To reduce alert fatigue among the clinical staff, the team identified the appropriate level of workflow interruption for each alert and reminder (hard stop, workflow interruption, or informational) [5,6].
The design principles that were used to design the alert include intuitive system development to promote ease of use, one set of screen formats and data definitions, and a set of consistent core reports and standard system output formats across facilities. The alert’s appearance was tailored for maximal impact and covered most of the screen. Color contrast was used, but since some people are color-blind, the meaning of the alert did not depend on the color contrast. The alerts included recommendations for changing the treatment plan to encourage using ultrasound as a first-line diagnostic tool. Minimizing the number of clicks to accept the proposed treatment plan change in the alert is desirable.
Phase 3: Training and User Support
Training and support structures and tools were critical to the rollout of the inappropriate imaging alerts. Providers were reminded about clinical best practices and informed during staff meetings about the new CDS rules. In addition, various types of training and support were available to clinicians and staff during the rollout process. Dedicated time for end-user training provided an opportunity to identify and cultivate super-users. These super-users not only helped provide technical support to their colleagues, but also helped create excitement for the initiative. A centralized support desk provided telephone support for providers in facilities throughout the Banner Health system. Site managers were provided toolkits to support providers and staff throughout the implementation process. The toolkits included frequently asked questions and answers, and were maintained as ‘living documents’ that were updated based on emerging issues and questions.
To keep things on track, project managers from the central project management department were involved in the initiative to provide direct project management services to the initiative. They also worked to instill project management competencies throughout the organization, applying a train-the-trainer approach to disseminate best practices for enhancing communication among team members, implementing workflow changes, and monitoring the results.
Phase 4: Optimization
The optimization phase is continuous and continues to the present day. Notably, the success of the CDS rules depends on the availability of current clinical information for each patient, in addition to information about the treatment plan. For this initiative, Banner Health maintained aggregated clinical patient data in the data warehouse that aggregated data from disparate sources, including billing and EHR data from different care settings such as ambulatory offices, inpatient units, the emergency department, home care, and ambulatory surgery centers. The data warehouse is housed in a strategically chosen physical location to minimize the threat of natural disasters, and cloud-based backup is also used. A master patient index and provider credentialing system are integrated with the data warehouse. Query-based health information exchange is used, when possible, to collect information on care received by patients outside of the Banner Health system.
It is important to note that many CDS alerts are over-ridden without changes to clinical care [34]. Previous research indicates that alert fatigue from “false positives” can impede the effectiveness of alerts [35]. Banner Health monitors the rate at which CDS alerts are over-ridden. Figure 1 shows the percentage of all alerts for radiation exposure—including the alert related to using ultrasound as a diagnostic tool for pediatric appendicitis—that led to order cancellations. The percentage of CT orders that generated the alert and were cancelled fell from 18.9% in March 2011 to 13.6% in February 2012. The rate of order cancellations might have declined over time due to a change in provider behavior from the alert. That is, if inappropriate CT scan orders declined over time, then providers would cancel a decreasing percentage of the CT scan orders that prompted an alert.
Imaging Use
In Figure 2, data on the use of the 2 imaging procedures for the diagnosis of pediatric appendicitis is presented. During the fourth quarter of 2010, almost half of pediatric patients with suspected appendicitis received a CT scan and only about 1 in 7 received an ultrasound. After the clinical decision support alert was put in place to remind providers to perform an ultrasound as a first-line diagnostic tool, the use of ultrasounds increased sharply. By the first quarter of 2012, ultrasounds were performed in 40.8% of these cases and the use of CT scans declined to 39.9% of suspected pediatric appendicitis cases.
Discussion
This case study discusses the application of CDS functionality in a health care improvement initiative to address inappropriate imaging in a large health system. Four main implementation lessons emerge for the field. First, it is important to involve all levels of staff in the planning process to ensure that health care improvement activities are prioritized correctly and to build buy-in for the priorities addressed with health care improvement activities. Second, it is necessary to allow time to design the alert or reminder, as well as testing it during the implementation process to ensure that clinical guidelines are being properly applied. Third, re-engineering the workflow and ensuring usability of the alert or reminder are important, and using the skills of trained engineers helps in this process. Ensuring the availability of trained ultrasound staff was particularly important to this initiative. Finally, the effectiveness of CDS depends on having complete data for each patient, as well as up-to-date information on the relevant evidence-based practice guidelines.
These results can help guide the implementation of health care improvement initiatives that use CDS functionality to address inappropriate imaging. The adoption of electronic health records with CDS functionality was incentivized and supported by the Medicare and Medicaid Electronic Health Record Incentive Programs; the Medicare program now exists as part of MACRA. Using CDS to reduce inappropriate imaging is required for Medicare fee-for-service patients in the 2014 Protecting Access to Medicare Act (PAMA), highlighting the critical nature of these results, which can guide implementation of CDS to reduce inappropriate imaging [41].
As noted above, the optimization phase is continuous. Banner Health still encourages use of ultrasounds as a first-line diagnostic tool for pediatric appendicitis. Identifying which patients should immediately receive CT scans is difficult, and sometimes the decision depends on the availability of staff to conduct the ultrasound scans. Ways to maximize the productivity of ultrasound technicians have been explored. Another focus area since the original implementation of this health care improvement initiative has been health information exchange, to ensure that complete, up-to-date information is available for each patient.
Banner Health often implements CDS in conjunction with other health IT functionalities. For example, CDS and telehealth are used together to improve care in the intensive care unit (ICU) for patients with sepsis and delirium. An offsite hub of experienced ICU physicians and nurses remotely monitors ICU patients in facilities across Banner Health, using cameras with zoom capability. The intensive care specialists in the tele-hub act as part of the care team; in addition to receiving video feed, they communicate verbally with patients and ICU staff members. Predictive analytics are used to generate clinical decision support alerts and reminders, with a special focus on early intervention if a patient’s clinical indicators are trending downward. The 4 lessons described in this study were also used in the ICU sepsis and delirium initiative; staff were involved in the planning process, alerts and reminders were thoroughly tested, the workflow was adjusted to accommodate the physicians in the tele-ICU hub, and up-to-date and complete clinical information for each patient is maintained. In addition, the design principles for alerts described in this study, such as covering most of the screen and providing recommendations for changing the treatment plan within the alert itself, were also used in the ICU sepsis and delirium initiative.
One limitation of this study is that it was conducted at a single health system. Thus, the findings might not be generalizable to other health systems, particularly if a robust health IT infrastructure is not in place. The culture of Banner Health values quality and involved providers and staff at all levels in selecting and implementing health care improvement initiatives. In addition, engineers assisted with implementation. Finally, the study design does not permit conclusions about the causality of the decline in CT scans and the increase in ultrasounds for suspected pediatric appendicitis cases; unobserved factors might have contributed to the changes in CT and ultrasound use.
Future research should focus on ways to improve the implementation and organization learning process, particularly through engagement of frontline staff by leadership [36] and explore how to operationalize previous findings indicating that innovations in hospital settings are more likely to be sustained when intrinsically rewarding to staff, either by making clinician and staff jobs easier to perform or more gratifying [37]. Future research should focus on facilitating health information exchange between providers in different health systems.
Disclaimer: The views expressed in the article are solely the views of the authors and do not represent those of the National Institutes of Health or the U.S. Government.
Acknowledgments: The authors wish to thank the Banner Health team for taking time to share their insights on how health information technology can be used for health care improvement initiatives, especially John Hensing. We also thank Michael Furukawa of the Agency for Healthcare Research and Quality, formerly of the Office of the National Coordinator for Health Information Technology, who played a key role in the conceptualization of this study and data collection.
Corresponding author: Emily Jones, PhD, MPP, National Institutes of Health, 6001 Executive Blvd., #5232 Rockville, MD 20852, emilybjones@gmail.com.
Financial disclosures: None
From the Office of Science Policy and Communications, National Institute on Drug Abuse, National Institutes of Health, Rockville, MD, and George Washington University, Washington, DC (Dr. Jones), Office of the National Coordinator for Health Information Technology, US Department of Health and Human Services, Washington, DC (Mr. Swain), and Banner Health, Phoenix, AZ (Ms. Burdick).
Abstract
- Objective: Clinical decision support (CDS) can be a useful tool to decrease inappropriate imaging by providing evidence-based information to clinicians at the point of care. The objective of this case study is to highlight lessons from a health care improvement initiative using CDS to encourage use of ultrasound rather than computed tomography (CT) scans as an initial diagnostic tool for suspected appendicitis in pediatric patients.
- Methods: The percentage of suspected pediatric appendicitis cases receiving ultrasounds and CT scans was calculated using electronic health record data. Four steps for implementing health information technology were identified in a literature scan that guided data collection and analysis: planning, software customization and workflow design, training and user support, and optimization.
- Results: During the fourth quarter of 2010, 1 in 7 pediatric patients with suspected appendicitis received an ultrasound and almost half received a CT scan. By the first quarter of 2012, ultrasounds were performed in 40.8% of these cases and the use of CT scans declined to 39.9% of suspected pediatric appendicitis cases.
- Conclusion: Four lessons emerged. First, all levels of staff should be involved in the planning process to make organizational priorities actionable and build buy-in for each healthcare improvement initiative. Second, it takes time to design and test the alert to ensure that clinical guidelines are being properly applied. Third, re-engineering the workflow is critical for usability; in this case, ensuring the availability of ultrasound staff was particularly important. Finally, the effectiveness of CDS depends on applying relevant evidence-based practice guidelines to real-time patient data.
Diagnostic imaging is a useful tool for identifying and guiding the treatment of many health conditions, but evidence indicates that health care providers do not always use imaging appropriately. In fact, a substantial proportion of diagnostic imaging procedures performed in hospital and ambulatory settings are not supported by clinical guideline recommendations [1,2]. Spending on diagnostic imaging is rapidly increasing, and some patients receive unnecessary radiation exposure that can lead to adverse health impacts [3]. Inappropriate imaging falls into 3 broad categories: imaging that does not conform to clinical guidelines, imaging that is contraindicated due to an allergy or implantable medical device, and imaging that might be clinically indicated but is duplicative of prior imaging services.
Clinical decision support (CDS) functionality supports health care improvement initiatives to narrow the gap between evidence-based practices and routine care [4]. CDS merges patient-specific clinical information with relevant information about evidence-based practices, providing health care providers with timely information to guide decisions at the point of care [5]. Decision support is most commonly delivered in the form of alerts and reminders [6]. CDS can be effective in reducing adverse drug events [7], sepsis [8,9], and other conditions in hospital [10–12] and ambulatory settings [13,14].
For the evaluation of suspected appendicitis in children, ultrasound is the preferred initial consideration for imaging examination [15]. Evidence suggests that CDS can increase the use of ultrasound for suspected pediatric appendicitis [16,17] and has affirmed the utility of ultrasound as a first-line diagnostic tool for suspected appendicitis [18,19]. In the Choosing Wisely campaign, the American College of Surgeons and the American College of Radiology have both endorsed ultrasound as an option to consider prior to conducting a CT scan to evaluate suspected appendicitis in children [15].
Banner Health, a large health system headquartered in Phoenix, Arizona, implemented a health care improvement initiative using CDS functionality to encourage providers to use ultrasound instead of CT as a first-line diagnostic tool for suspected pediatric appendicitis. We conducted a site visit to Banner Health, an organization who had had attained a high score on the EMR Adoption Model [20] to examine their implementation process. We sought to build on previous research examining the use of health information technology to improve performance in large health systems [21–23].
Methods
Setting
Banner Health is a large not-for-profit health system that is comprised of 24 acute care hospitals across several states, as well as ambulatory medical practices, behavioral health, home care, and ambulatory surgery centers [24,25]. The health system is the largest employer in Arizona and one of the largest in the United States with over 50,000 employees. Banner Health has been nationally recognized for clinical quality [26], an innovative leadership team [27], and using health IT to improve quality [20]. The health system was also selected as one of the Centers for Medicare & Medicaid Services (CMS) Pioneer Accountable Care Organizations.
Site Visit
The first 2 authors conducted a 2-day site visit to the Banner Health headquarters in Phoenix, Arizona in November 2013. The team conducted discussions with over 20 individuals, including health system leadership, frontline clinicians in several units of an acute care hospital, staff members in 2 telehealth hubs—including a tele-ICU hub—and trainers in a simulation facility that is used for staff training. The discussions were conducted with groups of staff or on an individual basis, as appropriate. At the outset of the project, an environmental scan of relevant grey and peer-reviewed literature was conducted under contract on behalf of the authors to guide data collection and analysis [28]. An interview protocol was created to guide the discussions. The protocol contained modules that were used during each discussion, if relevant. The modules addressed topics such as technical issues with designing and deploying health information technology functionalities such as clinical decision support systems, the organizational processes and structures needed to launch health care improvement initiatives, and using health information technology care coordination. Within each module, questions probed about the challenges that arose and the solutions to these challenges, with a focus on the four phases of implementing a health information technology intervention: functionality planning, software customization and workflow design, training and user support, and optimization. To assist with interpreting the qualitative findings, an evolving outline of the findings was maintained. Salient themes and conceptual categories were tracked, which helped the researchers organize, synthesize, and interpret the information collected during the site visit. Once the authors chose to focus on clinical decision support, summary notes from the discussions were reviewed for relevant information, and this information was compiled and organized under the rubric of the four implementation phases. The findings and key themes from the discussion notes were distilled into key lessons for the field.
Data obtained included the percentage of pediatric patients with suspected appendicitis who received ultrasounds and CT scans each month from 1 October 2010 through 31 March 2012. Banner Health staff originally collected the data to support the implementation of health care improvement initiative; the use of these data in this paper is a secondary use [29].
This manuscript was prepared using the SQUIRE 2.0 guidelines [30]. No patient-identifiable data were used, so institutional review board approval was not sought.
Results
The 4 steps of implementing CDS can be described as functionality planning, software customization and workflow design, training and user support, and optimization [31].
Pre-Implementation
The use of computerized provider order entry (CPOE) is a precursor to using clinical decision support, since orders must be entered electronically to be subject to CDS review. Banner Health deployed CPOE to its various facilities starting in 2008. The deployment was staged in a rolling fashion with one or two facilities going live every few months so that the deployment team was available at each facility.
Phase 1: Planning
In contrast to many large health systems, the organization has a single board of directors that oversees the entire system of over 37,000 employees. Activities and relationships to promote the use of evidence-based practices are built into the organizational structure. For example, Banner Health maintains a Care Management Council, a group comprised of clinical and administrative leadership to provide executive oversight of health care improvement projects. The Council convenes on a quarterly basis to review and approve the adoption of new clinical practice guidelines, policies, and standardized standing orders that have been developed by multidisciplinary groups of physicians and other clinicians. A key focus of the Council is ensuring consistent application of evidence-based guidelines to clinical care and disseminating knowledge of clinical best practices across a large and complex enterprise.
Interdisciplinary clinical consensus groups support the Council’s work. These groups are comprised of administrative and program management staff, physicians and other clinicians, and engineers. Each clinical consensus group focuses on emerging issues and improvement opportunities within a specific clinical domain and leads the implementation of health care improvement initiatives in that domain. Providers and staff at all levels of the organization were involved in planning and implementing the health care improvement initiative in inappropriate imaging. This increased buy-in and staff support, which are associated with successful health care improvement initiatives [32]. Banner Health staff rallied around the idea of addressing inappropriate imaging as a key priority initiative. The teams that implement each initiative include an engineer that focuses on redesigning clinical workflows for each initiative. There is also an organizational unit responsible for project management that provides teams with logistical and operational support.
Phase 2: Software Customization and Workflow Redesign
Once the clinical consensus group selected inappropriate imaging as a priority, the next step was to examine the process flow for imaging ordering. In 2011 Banner Health integrated CDS functionality with CPOE into the electronic health record. Before the use of CDS, inpatient and emergency department imaging orders were simply transmitted to imaging staff after the order was entered. After CDS implementation, the process flow begins with an inpatient imaging order and entailed checking the order against clinical guidelines on the proper use of imaging. If the image order did not conform to guidelines, which in this case indicate that ultrasound should be used before CT scans as a diagnostic tool for suspected pediatric appendicitis, the CDS system triggered an alert [15].
Bringing the perspective and skill sets of engineers to the process of redesigning clinical workflows was particularly valuable [33]. While CDS has the potential to reduce inpatient inappropriate imaging, effectiveness depends on adjusting workflows to ensure that the information provided by CDS alerts and reminders is actionable. To reduce alert fatigue among the clinical staff, the team identified the appropriate level of workflow interruption for each alert and reminder (hard stop, workflow interruption, or informational) [5,6].
The design principles that were used to design the alert include intuitive system development to promote ease of use, one set of screen formats and data definitions, and a set of consistent core reports and standard system output formats across facilities. The alert’s appearance was tailored for maximal impact and covered most of the screen. Color contrast was used, but since some people are color-blind, the meaning of the alert did not depend on the color contrast. The alerts included recommendations for changing the treatment plan to encourage using ultrasound as a first-line diagnostic tool. Minimizing the number of clicks to accept the proposed treatment plan change in the alert is desirable.
Phase 3: Training and User Support
Training and support structures and tools were critical to the rollout of the inappropriate imaging alerts. Providers were reminded about clinical best practices and informed during staff meetings about the new CDS rules. In addition, various types of training and support were available to clinicians and staff during the rollout process. Dedicated time for end-user training provided an opportunity to identify and cultivate super-users. These super-users not only helped provide technical support to their colleagues, but also helped create excitement for the initiative. A centralized support desk provided telephone support for providers in facilities throughout the Banner Health system. Site managers were provided toolkits to support providers and staff throughout the implementation process. The toolkits included frequently asked questions and answers, and were maintained as ‘living documents’ that were updated based on emerging issues and questions.
To keep things on track, project managers from the central project management department were involved in the initiative to provide direct project management services to the initiative. They also worked to instill project management competencies throughout the organization, applying a train-the-trainer approach to disseminate best practices for enhancing communication among team members, implementing workflow changes, and monitoring the results.
Phase 4: Optimization
The optimization phase is continuous and continues to the present day. Notably, the success of the CDS rules depends on the availability of current clinical information for each patient, in addition to information about the treatment plan. For this initiative, Banner Health maintained aggregated clinical patient data in the data warehouse that aggregated data from disparate sources, including billing and EHR data from different care settings such as ambulatory offices, inpatient units, the emergency department, home care, and ambulatory surgery centers. The data warehouse is housed in a strategically chosen physical location to minimize the threat of natural disasters, and cloud-based backup is also used. A master patient index and provider credentialing system are integrated with the data warehouse. Query-based health information exchange is used, when possible, to collect information on care received by patients outside of the Banner Health system.
It is important to note that many CDS alerts are over-ridden without changes to clinical care [34]. Previous research indicates that alert fatigue from “false positives” can impede the effectiveness of alerts [35]. Banner Health monitors the rate at which CDS alerts are over-ridden. Figure 1 shows the percentage of all alerts for radiation exposure—including the alert related to using ultrasound as a diagnostic tool for pediatric appendicitis—that led to order cancellations. The percentage of CT orders that generated the alert and were cancelled fell from 18.9% in March 2011 to 13.6% in February 2012. The rate of order cancellations might have declined over time due to a change in provider behavior from the alert. That is, if inappropriate CT scan orders declined over time, then providers would cancel a decreasing percentage of the CT scan orders that prompted an alert.
Imaging Use
In Figure 2, data on the use of the 2 imaging procedures for the diagnosis of pediatric appendicitis is presented. During the fourth quarter of 2010, almost half of pediatric patients with suspected appendicitis received a CT scan and only about 1 in 7 received an ultrasound. After the clinical decision support alert was put in place to remind providers to perform an ultrasound as a first-line diagnostic tool, the use of ultrasounds increased sharply. By the first quarter of 2012, ultrasounds were performed in 40.8% of these cases and the use of CT scans declined to 39.9% of suspected pediatric appendicitis cases.
Discussion
This case study discusses the application of CDS functionality in a health care improvement initiative to address inappropriate imaging in a large health system. Four main implementation lessons emerge for the field. First, it is important to involve all levels of staff in the planning process to ensure that health care improvement activities are prioritized correctly and to build buy-in for the priorities addressed with health care improvement activities. Second, it is necessary to allow time to design the alert or reminder, as well as testing it during the implementation process to ensure that clinical guidelines are being properly applied. Third, re-engineering the workflow and ensuring usability of the alert or reminder are important, and using the skills of trained engineers helps in this process. Ensuring the availability of trained ultrasound staff was particularly important to this initiative. Finally, the effectiveness of CDS depends on having complete data for each patient, as well as up-to-date information on the relevant evidence-based practice guidelines.
These results can help guide the implementation of health care improvement initiatives that use CDS functionality to address inappropriate imaging. The adoption of electronic health records with CDS functionality was incentivized and supported by the Medicare and Medicaid Electronic Health Record Incentive Programs; the Medicare program now exists as part of MACRA. Using CDS to reduce inappropriate imaging is required for Medicare fee-for-service patients in the 2014 Protecting Access to Medicare Act (PAMA), highlighting the critical nature of these results, which can guide implementation of CDS to reduce inappropriate imaging [41].
As noted above, the optimization phase is continuous. Banner Health still encourages use of ultrasounds as a first-line diagnostic tool for pediatric appendicitis. Identifying which patients should immediately receive CT scans is difficult, and sometimes the decision depends on the availability of staff to conduct the ultrasound scans. Ways to maximize the productivity of ultrasound technicians have been explored. Another focus area since the original implementation of this health care improvement initiative has been health information exchange, to ensure that complete, up-to-date information is available for each patient.
Banner Health often implements CDS in conjunction with other health IT functionalities. For example, CDS and telehealth are used together to improve care in the intensive care unit (ICU) for patients with sepsis and delirium. An offsite hub of experienced ICU physicians and nurses remotely monitors ICU patients in facilities across Banner Health, using cameras with zoom capability. The intensive care specialists in the tele-hub act as part of the care team; in addition to receiving video feed, they communicate verbally with patients and ICU staff members. Predictive analytics are used to generate clinical decision support alerts and reminders, with a special focus on early intervention if a patient’s clinical indicators are trending downward. The 4 lessons described in this study were also used in the ICU sepsis and delirium initiative; staff were involved in the planning process, alerts and reminders were thoroughly tested, the workflow was adjusted to accommodate the physicians in the tele-ICU hub, and up-to-date and complete clinical information for each patient is maintained. In addition, the design principles for alerts described in this study, such as covering most of the screen and providing recommendations for changing the treatment plan within the alert itself, were also used in the ICU sepsis and delirium initiative.
One limitation of this study is that it was conducted at a single health system. Thus, the findings might not be generalizable to other health systems, particularly if a robust health IT infrastructure is not in place. The culture of Banner Health values quality and involved providers and staff at all levels in selecting and implementing health care improvement initiatives. In addition, engineers assisted with implementation. Finally, the study design does not permit conclusions about the causality of the decline in CT scans and the increase in ultrasounds for suspected pediatric appendicitis cases; unobserved factors might have contributed to the changes in CT and ultrasound use.
Future research should focus on ways to improve the implementation and organization learning process, particularly through engagement of frontline staff by leadership [36] and explore how to operationalize previous findings indicating that innovations in hospital settings are more likely to be sustained when intrinsically rewarding to staff, either by making clinician and staff jobs easier to perform or more gratifying [37]. Future research should focus on facilitating health information exchange between providers in different health systems.
Disclaimer: The views expressed in the article are solely the views of the authors and do not represent those of the National Institutes of Health or the U.S. Government.
Acknowledgments: The authors wish to thank the Banner Health team for taking time to share their insights on how health information technology can be used for health care improvement initiatives, especially John Hensing. We also thank Michael Furukawa of the Agency for Healthcare Research and Quality, formerly of the Office of the National Coordinator for Health Information Technology, who played a key role in the conceptualization of this study and data collection.
Corresponding author: Emily Jones, PhD, MPP, National Institutes of Health, 6001 Executive Blvd., #5232 Rockville, MD 20852, emilybjones@gmail.com.
Financial disclosures: None
1. Lehnert B, Bree R. Analysis of appropriateness of outpatient CT and MRI referred from primary care clinics at an academic medical center: how critical is the need for improved decision support? J Am Coll Radiol 2010;7:192–7.
2. Ip I, Schneider L, Hanson R, et al. Adoption and meaningful use of computed physician order entry with an integrated clinical decision support system for radiology: ten-year analysis in an urban teaching hospital. J Am Coll Radiol 2012;9:129–36.
3. Bernardy M, Ullrich C, Rawson J, et al. Strategies for managing imaging utilization. J Am Coll Radiol 2009;6:844–50.
4. Amland R, Dean B, Yu HT et al. Computed clinical decision support to prevent venous thromboembolism among hospitalized patients: proximal outcomes from a multiyear quality improvement project. J Healthcare Qual 2015;37:221–31.
5. Kahn C. Improving outcomes in radiology: bringing computer-based decision support and education to the point of care. Acad Radiology 2005;12:409–14.
6. Phansalkar S, Desai A, Bell D et al. High-priority drug-drug interactions for use in electronic health records. J Am Med Inform Assoc 2012;19:735–43.
7. Wolfstadt J, Gurwitz J, Field T, et al. The effect of computed physician order entry with clinical decision support on the rates of adverse drug events: a systematic review. J Gen Intern Med 2008;23:451–8.
8. Amland R, Hahn-Cover K. Clinical decision support for early recognition of sepsis. Am J Med Qual 2014;1–8.
9. Amland R, Haley J, Lyons J. A multidisciplinary sepsis program enabled by a two-stage clinical decision support system: factors that influence patient outcomes. Am J Med Qual 2015;1–8.
10. Umscheid C, Hanish A, Chittams J, et al. Effectiveness of a novel and scalable clinical decision support intervention to improve venous thromboembolism prophylaxis: a quasi-experimental study. BMC Med Inform Dec Making 2012;12:92–104.
11. Mack EH, Wheeler DS, Embi PJ. Clinical decision support systems in the pediatric intensive care unit. Pediatric Crit Care Med 2009;10:23–8.
12. Kollef M, Heard K, Chen Y, et al. Mortality and length of stay trends following implementation of a rapid response system and real-time automated clinical deterioration alerts. Am J Med Qual 2015; online first.
13. Ali S, Giordano R, Lakhani S, Walker D. A review of randomized controlled trials of medical record powered clinical decision support system to improve quality of diabetes care. Int J Med Informatics 2016;87:91–100.
14. Gill J, Mainous A, Koopman R et al. Impact of EHR-based clinical decision support on adherence to guidelines for patients on NSAIDs: a randomized controlled trial. Ann Fam Med 2011;9:22–30.
15. Choosing Wisely. Accessed 1 May 2017 at http://www.choosingwisely.org/clinician-lists/#keyword=appendicitis.
16. Hendrickson M, Wey A, Gaillard P, Kharbanda A. Implementation of an electronic clinical decision support tool for pediatric appendicitis within a hospital network. Pediatric Emerg Care 2017 (online first).
17. Kharbanda A, Madhok M, Krause E, et al. Implementation of electronic clinical decision support for pediatric appendicitis. Pediatrics 2016;137:e20151745.
18. Schuh S, Chan K, Langer J, et al. Properties of serial ultrasound clinical diagnostic pathway in suspected appendicitis and related computed tomography use. Acad Emerg Med 2015;22:406–14.
19. Ramarajan N, Krishnamoorthi R, Barth R, et al. An interdisciplinary initiative to reduce radiation exposure: evaluation of appendicitis in a pediatric emergency department with clinical assessment supported by a staged ultrasound and computed tomography pathway. Acad Emerg Med 2009;16:1258–65.
20. HIMSS Analytics. Stage 7 Hospitals. Accessed at www.himssanalytics.org/emram/stage7Hospitals.aspx.
21. Rizer M, et al. Top 10 lessons learned from electronic health record implementation in a large academic medical center. Perspectives in Health Information Management. Summer 2015.
22. Cosgrove DM, Fisher M, Gabow P, et al. Ten strategies to lower costs, improve quality, and engage patients: the view from leading health system CEOs. Health Aff (Millwood) 2013;32:321–7.
23. Cresswell KM, Bates DW, Sheikh A. Ten key considerations for the successful implementation and adoption of large-scale health information technology. J Am Med Inform Assoc 2013 Apr 18.
24. Hensing JA. The quest for upper-quartile performance at Banner Health. J Healthc Qual 2008;30:18–24
25. Hensing J, Dahlen D, Warden M, et al. Measuring the benefits of IT-enabled care transformation. Healthc Financ Manage 2008;62:74–80.
26. Truven Health Analytics. 15 Top Health Systems Study. 6th ed. April 2014. Accessed at http://100tophospitals.com/portals/2/assets/15-Top-Health-Systems-Study.pdf.
27. Aiello M. 2011 Top leadership team awards recognize big moves. Health Leaders Media. August 2011. Accessed at www.healthleadersmedia.com/page-2/LED-269808/2011-Top-Leadership-Team-Awards-Recognize-Big-Moves.
28. Rosenthal D, Stout M. Radiology order entry: features and performance requirements. J Am Coll Radiol 2006;3:554–7.
29. Kirsh S, Wu WC, Edelman D, Aron D. Research versus quality improvement: distinct or a distinction without a difference? A case study comparison of two studies. Jt Comm J Qual Patient Safety 2014;40:365–75.
30. Ogrinc G, Davies L, Goodman D, et al. SQUIRE 2.0 (Standards for Quality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process. BMJ Qual Saf 2015;0:1–7.
31. Blavin F, Ramos C, Shah A, Devers K. Lessons from the literature on electronic health record implementation.1 Aug 2013. The Urban Institute. Prepared for the Office of the National Coordinator for Health Information Technology. Available at www.urban.org/research/publication/lessons-literature-electronic-health-record-implementation.
32. Needleman J, Pearson ML, Upenieks VV, et al. Engaging frontline staff in performance improvement: the American Organization of Nurse Executives implementation of transforming care at the beside collaborative. Jt Comm J Qual Patient Safety 2016;42:61–9.
33. Jones E, Swain M, Patel V, Furukawa M. Supporting HITECH implementation and assessing lessons for the future: the role of program evaluation. Healthcare: The Journal of Delivery Science and Innovation 2014;2:4–8.
34. Phansalkar S, Zachariah M, Seidling H, et al. Evaluation of medication alerts in electronic health records for compliance with human factors principles. J Am Med Inform Assoc 2014;21:e332–e340.
35. Handler S, Altman R, Perera S, et al. A systematic review of the performance characteristics of clinical event monitor signals to detect adverse drug events in the hospital setting. J Am Med Inform Assoc 2007;14:451–8.
36. Singer S, Rivard P, Hayes J, et al. Improving patient care through leadership engagement with frontline staff: a Department of Veteran Affairs study. Jt Comm J Qual Patient Safety 2013;39):349–60.
37. Brewster A, Curry L, Cherlin E, et al. Integrating new practices: a qualitative study of how hospital innovations become routine. Implement Sci 2015;5(10):168.
1. Lehnert B, Bree R. Analysis of appropriateness of outpatient CT and MRI referred from primary care clinics at an academic medical center: how critical is the need for improved decision support? J Am Coll Radiol 2010;7:192–7.
2. Ip I, Schneider L, Hanson R, et al. Adoption and meaningful use of computed physician order entry with an integrated clinical decision support system for radiology: ten-year analysis in an urban teaching hospital. J Am Coll Radiol 2012;9:129–36.
3. Bernardy M, Ullrich C, Rawson J, et al. Strategies for managing imaging utilization. J Am Coll Radiol 2009;6:844–50.
4. Amland R, Dean B, Yu HT et al. Computed clinical decision support to prevent venous thromboembolism among hospitalized patients: proximal outcomes from a multiyear quality improvement project. J Healthcare Qual 2015;37:221–31.
5. Kahn C. Improving outcomes in radiology: bringing computer-based decision support and education to the point of care. Acad Radiology 2005;12:409–14.
6. Phansalkar S, Desai A, Bell D et al. High-priority drug-drug interactions for use in electronic health records. J Am Med Inform Assoc 2012;19:735–43.
7. Wolfstadt J, Gurwitz J, Field T, et al. The effect of computed physician order entry with clinical decision support on the rates of adverse drug events: a systematic review. J Gen Intern Med 2008;23:451–8.
8. Amland R, Hahn-Cover K. Clinical decision support for early recognition of sepsis. Am J Med Qual 2014;1–8.
9. Amland R, Haley J, Lyons J. A multidisciplinary sepsis program enabled by a two-stage clinical decision support system: factors that influence patient outcomes. Am J Med Qual 2015;1–8.
10. Umscheid C, Hanish A, Chittams J, et al. Effectiveness of a novel and scalable clinical decision support intervention to improve venous thromboembolism prophylaxis: a quasi-experimental study. BMC Med Inform Dec Making 2012;12:92–104.
11. Mack EH, Wheeler DS, Embi PJ. Clinical decision support systems in the pediatric intensive care unit. Pediatric Crit Care Med 2009;10:23–8.
12. Kollef M, Heard K, Chen Y, et al. Mortality and length of stay trends following implementation of a rapid response system and real-time automated clinical deterioration alerts. Am J Med Qual 2015; online first.
13. Ali S, Giordano R, Lakhani S, Walker D. A review of randomized controlled trials of medical record powered clinical decision support system to improve quality of diabetes care. Int J Med Informatics 2016;87:91–100.
14. Gill J, Mainous A, Koopman R et al. Impact of EHR-based clinical decision support on adherence to guidelines for patients on NSAIDs: a randomized controlled trial. Ann Fam Med 2011;9:22–30.
15. Choosing Wisely. Accessed 1 May 2017 at http://www.choosingwisely.org/clinician-lists/#keyword=appendicitis.
16. Hendrickson M, Wey A, Gaillard P, Kharbanda A. Implementation of an electronic clinical decision support tool for pediatric appendicitis within a hospital network. Pediatric Emerg Care 2017 (online first).
17. Kharbanda A, Madhok M, Krause E, et al. Implementation of electronic clinical decision support for pediatric appendicitis. Pediatrics 2016;137:e20151745.
18. Schuh S, Chan K, Langer J, et al. Properties of serial ultrasound clinical diagnostic pathway in suspected appendicitis and related computed tomography use. Acad Emerg Med 2015;22:406–14.
19. Ramarajan N, Krishnamoorthi R, Barth R, et al. An interdisciplinary initiative to reduce radiation exposure: evaluation of appendicitis in a pediatric emergency department with clinical assessment supported by a staged ultrasound and computed tomography pathway. Acad Emerg Med 2009;16:1258–65.
20. HIMSS Analytics. Stage 7 Hospitals. Accessed at www.himssanalytics.org/emram/stage7Hospitals.aspx.
21. Rizer M, et al. Top 10 lessons learned from electronic health record implementation in a large academic medical center. Perspectives in Health Information Management. Summer 2015.
22. Cosgrove DM, Fisher M, Gabow P, et al. Ten strategies to lower costs, improve quality, and engage patients: the view from leading health system CEOs. Health Aff (Millwood) 2013;32:321–7.
23. Cresswell KM, Bates DW, Sheikh A. Ten key considerations for the successful implementation and adoption of large-scale health information technology. J Am Med Inform Assoc 2013 Apr 18.
24. Hensing JA. The quest for upper-quartile performance at Banner Health. J Healthc Qual 2008;30:18–24
25. Hensing J, Dahlen D, Warden M, et al. Measuring the benefits of IT-enabled care transformation. Healthc Financ Manage 2008;62:74–80.
26. Truven Health Analytics. 15 Top Health Systems Study. 6th ed. April 2014. Accessed at http://100tophospitals.com/portals/2/assets/15-Top-Health-Systems-Study.pdf.
27. Aiello M. 2011 Top leadership team awards recognize big moves. Health Leaders Media. August 2011. Accessed at www.healthleadersmedia.com/page-2/LED-269808/2011-Top-Leadership-Team-Awards-Recognize-Big-Moves.
28. Rosenthal D, Stout M. Radiology order entry: features and performance requirements. J Am Coll Radiol 2006;3:554–7.
29. Kirsh S, Wu WC, Edelman D, Aron D. Research versus quality improvement: distinct or a distinction without a difference? A case study comparison of two studies. Jt Comm J Qual Patient Safety 2014;40:365–75.
30. Ogrinc G, Davies L, Goodman D, et al. SQUIRE 2.0 (Standards for Quality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process. BMJ Qual Saf 2015;0:1–7.
31. Blavin F, Ramos C, Shah A, Devers K. Lessons from the literature on electronic health record implementation.1 Aug 2013. The Urban Institute. Prepared for the Office of the National Coordinator for Health Information Technology. Available at www.urban.org/research/publication/lessons-literature-electronic-health-record-implementation.
32. Needleman J, Pearson ML, Upenieks VV, et al. Engaging frontline staff in performance improvement: the American Organization of Nurse Executives implementation of transforming care at the beside collaborative. Jt Comm J Qual Patient Safety 2016;42:61–9.
33. Jones E, Swain M, Patel V, Furukawa M. Supporting HITECH implementation and assessing lessons for the future: the role of program evaluation. Healthcare: The Journal of Delivery Science and Innovation 2014;2:4–8.
34. Phansalkar S, Zachariah M, Seidling H, et al. Evaluation of medication alerts in electronic health records for compliance with human factors principles. J Am Med Inform Assoc 2014;21:e332–e340.
35. Handler S, Altman R, Perera S, et al. A systematic review of the performance characteristics of clinical event monitor signals to detect adverse drug events in the hospital setting. J Am Med Inform Assoc 2007;14:451–8.
36. Singer S, Rivard P, Hayes J, et al. Improving patient care through leadership engagement with frontline staff: a Department of Veteran Affairs study. Jt Comm J Qual Patient Safety 2013;39):349–60.
37. Brewster A, Curry L, Cherlin E, et al. Integrating new practices: a qualitative study of how hospital innovations become routine. Implement Sci 2015;5(10):168.