Affiliations
Division of Hospital Medicine, University of California, San Francisco, San Francisco, California
Given name(s)
James D.
Family name
Harrison
Degrees
PhD

Hospitalist and Internal Medicine Leaders’ Perspectives of Early Discharge Challenges at Academic Medical Centers

Article Type
Changed
Wed, 07/11/2018 - 06:53

The discharge process is a critical bottleneck for efficient patient flow through the hospital. Delayed discharges translate into delays in admissions and other patient transitions, often leading to excess costs, patient dissatisfaction, and even patient harm.1-3 The emergency department is particularly impacted by these delays; bottlenecks there lead to overcrowding, increased overall hospital length of stay, and increased risks for bad outcomes during hospitalization.2

Academic medical centers in particular may struggle with delayed discharges. In a typical teaching hospital, a team composed of an attending physician and housestaff share responsibility for determining the discharge plan. Additionally, clinical teaching activities may affect the process and quality of discharge.4-6

The prevalence and causes of delayed discharges vary greatly.7-9 To improve efficiency around discharge, many hospitals have launched initiatives designed to discharge patients earlier in the day, including goal setting (“discharge by noon”), scheduling discharge appointments, and using quality-improvement methods, such as Lean Methodology (LEAN), to remove inefficiencies within discharge processes.10-12 However, there are few data on the prevalence and effectiveness of different strategies.

The aim of this study was to survey academic hospitalist and general internal medicine physician leaders to elicit their perspectives on the factors contributing to discharge timing and the relative importance and effectiveness of early-discharge initiatives.

METHODS

Study Design, Participants, and Oversight

We obtained a list of 115 university-affiliated hospitals associated with a residency program and, in most cases, a medical school from Vizient Inc. (formerly University HealthSystem Consortium), an alliance of academic medical centers and affiliated hospitals. Each member institution submits clinical data to allow for the benchmarking of outcomes to drive transparency and quality improvement.13 More than 95% of the nation’s academic medical centers and affiliated hospitals participate in this collaborative. Vizient works with members but does not set nor promote quality metrics, such as discharge timeliness. E-mail addresses for hospital medicine physician leaders (eg, division chief) of major academic medical centers were obtained from each institution via publicly available data (eg, the institution’s website). When an institution did not have a hospital medicine section, we identified the division chief of general internal medicine. The University of California, San Francisco Institutional Review Board approved this study.

Survey Development and Domains

We developed a 30-item survey to evaluate 5 main domains of interest: current discharge practices, degree of prioritization of early discharge on the inpatient service, barriers to timely discharge, prevalence and perceived effectiveness of implemented early-discharge initiatives, and barriers to implementation of early-discharge initiatives.

Respondents were first asked to identify their institutions’ goals for discharge time. They were then asked to compare the priority of early-discharge initiatives to other departmental quality-improvement initiatives, such as reducing 30-day readmissions, improving interpreter use, and improving patient satisfaction. Next, respondents were asked to estimate the degree to which clinical or patient factors contributed to delays in discharge. Respondents were then asked whether specific early-discharge initiatives, such as changes to rounding practices or communication interventions, were implemented at their institutions and, if so, the perceived effectiveness of these initiatives at meeting discharge targets. We piloted the questions locally with physicians and researchers prior to finalizing the survey.

Data Collection

We sent surveys via an online platform (Research Electronic Data Capture).14 Nonresponders were sent 2 e-mail reminders and then a follow-up telephone call asking them to complete the survey. Only 1 survey per academic medical center was collected. Any respondent who completed the survey within 2 weeks of receiving it was entered to win a Kindle Fire.

Data Analysis

We summarized survey responses using descriptive statistics. Analysis was completed in IBM SPSS version 22 (Armonk, NY).

RESULTS

Survey Respondent and Institutional Characteristics

Of the 115 institutions surveyed, we received 61 responses (response rate of 53%), with 39 (64%) respondents from divisions of hospital medicine and 22 (36%) from divisions of general internal medicine. A majority (n = 53; 87%) stated their medicine services have a combination of teaching (with residents) and nonteaching (without residents) teams. Thirty-nine (64%) reported having daily multidisciplinary rounds.

 

 

Early Discharge as a Priority

Forty-seven (77%) institutional representatives strongly agreed or agreed that early discharge was a priority, with discharge by noon being the most common target time (n = 23; 38%). Thirty (50%) respondents rated early discharge as more important than improving interpreter use for non-English-speaking patients and equally important as reducing 30-day readmissions (n = 29; 48%) and improving patient satisfaction (n = 27; 44%).

Factors Delaying Discharge

The most common factors perceived as delaying discharge were considered external to the hospital, such as postacute care bed availability or scheduled (eg, ambulance) transport delays (n = 48; 79%), followed by patient factors such as patient transport issues (n = 44; 72%). Less commonly reported were workflow issues, such as competing primary team priorities or case manager bandwidth (n = 38; 62%; Table 1).

Initiatives to Improve Discharge

The most commonly implemented initiatives perceived as effective at improving discharge times were the preemptive identification of early discharges to plan discharge paperwork (n = 34; 56%), communication with patients about anticipated discharge time on the day prior to discharge (n = 29; 48%), and the implementation of additional rounds between physician teams and case managers specifically around discharge planning (n = 28; 46%). Initiatives not commonly implemented included regular audit of and feedback on discharge times to providers and teams (n = 21; 34%), the use of a discharge readiness checklist (n = 26; 43%), incentives such as bonuses or penalties (n = 37; 61%), the use of a whiteboard to indicate discharge times (n = 23; 38%), and dedicated quality-improvement approaches such as LEAN (n = 37; 61%; Table 2).

DISCUSSION

Our study suggests early discharge for medicine patients is a priority among academic institutions. Hospitalist and general internal medicine physician leaders in our study generally attributed delayed discharges to external factors, particularly unavailability of postacute care facilities and transportation delays. Having issues with finding postacute care placements is consistent with previous findings by Selker et al.15 and Carey et al.8 This is despite the 20-year difference between Selker et al.’s study and the current study, reflecting a continued opportunity for improvement, including stronger partnerships with local and regional postacute care facilities to expedite care transition and stronger discharge-planning efforts early in the admission process. Efforts in postacute care placement may be particularly important for Medicaid-insured and uninsured patients.

Our responders, hospitalist and internal medicine physician leaders, did not perceive the additional responsibilities of teaching and supervising trainees to be factors that significantly delayed patient discharge. This is in contrast to previous studies, which attributed delays in discharge to prolonged clinical decision-making related to teaching and supervision.4-6,8 This discrepancy may be due to the fact that we only surveyed single physician leaders at each institution and not residents. Our finding warrants further investigation to understand the degree to which resident skills may impact discharge planning and processes.

Institutions represented in our study have attempted a variety of initiatives promoting earlier discharge, with varying levels of perceived success. Initiatives perceived to be the most effective by hospital leaders centered on 2 main areas: (1) changing individual provider practice and (2) anticipatory discharge preparation. Interestingly, this is in discordance with the main factors labeled as causing delays in discharges, such as obtaining postacute care beds, busy case managers, and competing demands on primary teams. We hypothesize this may be because such changes require organization- or system-level changes and are perceived as more arduous than changes at the individual level. In addition, changes to individual provider behavior may be more cost- and time-effective than more systemic initiatives.

Our findings are consistent with the work published by Wertheimer and colleagues,11 who show that additional afternoon interdisciplinary rounds can help identify patients who may be discharged before noon the next day. In their study, identifying such patients in advance improved the overall early-discharge rate the following day.

Our findings should be interpreted in light of several limitations. Our survey only considers the perspectives of hospitalist and general internal medicine physician leaders at academic medical centers that are part of the Vizient Inc. collaborative. They do not represent all academic or community-based medical centers. Although the perceived effectiveness of some initiatives was high, we did not collect empirical data to support these claims or to determine which initiative had the greatest relative impact on discharge timeliness. Lastly, we did not obtain resident, nursing, or case manager perspectives on discharge practices. Given their roles as frontline providers, we may have missed these alternative perspectives.

Our study shows there is a strong interest in increasing early discharges in an effort to improve hospital throughput and patient flow.

 

 

Acknowledgments

The authors thank all participants who completed the survey and Danielle Carrier at Vizient Inc. (formally University HealthSystem Consortium) for her assistance in obtaining data.

Disclosures

Hemali Patel, Margaret Fang, Michelle Mourad, Adrienne Green, Ryan Murphy, and James Harrison report no conflicts of interest. At the time the research was conducted, Robert Wachter reported that he is a member of the Lucian Leape Institute at the National Patient Safety Foundation (no compensation except travel expenses); recently chaired an advisory board to England’s National Health Service (NHS) reviewing the NHS’s digital health strategy (no compensation except travel expenses); has a contract with UCSF from the Agency for Healthcare Research and Quality to edit a patient-safety website; receives compensation from John Wiley & Sons for writing a blog; receives royalties from Lippincott Williams & Wilkins and McGraw-Hill Education for writing and/or editing several books; receives stock options for serving on the board of Acuity Medical Management Systems; receives a yearly stipend for serving on the board of The Doctors Company; serves on the scientific advisory boards for amino.com, PatientSafe Solutions Inc., Twine, and EarlySense (for which he receives stock options); has a small royalty stake in CareWeb, a hospital communication tool developed at UCSF; and holds the Marc and Lynne Benioff Endowed Chair in Hospital Medicine and the Holly Smith Distinguished Professorship in Science and Medicine at UCSF.

 

Files
References

1. Khanna S, Boyle J, Good N, Lind J. Impact of admission and discharge peak times on hospital overcrowding. Stud Health Technol Inform. 2011;168:82-88. PubMed
2. White BA, Biddinger PD, Chang Y, Grabowski B, Carignan S, Brown DFM. Boarding Inpatients in the Emergency Department Increases Discharged Patient Length of Stay. J Emerg Med. 2013;44(1):230-235. doi:10.1016/j.jemermed.2012.05.007. PubMed
3. Derlet RW, Richards JR. Overcrowding in the nation’s emergency departments: complex causes and disturbing effects. Ann Emerg Med. 2000;35(1):63-68. PubMed
4. da Silva SA, Valácio RA, Botelho FC, Amaral CFS. Reasons for discharge delays in teaching hospitals. Rev Saúde Pública. 2014;48(2):314-321. doi:10.1590/S0034-8910.2014048004971. PubMed
5. Greysen SR, Schiliro D, Horwitz LI, Curry L, Bradley EH. “Out of Sight, Out of Mind”: Housestaff Perceptions of Quality-Limiting Factors in Discharge Care at Teaching Hospitals. J Hosp Med Off Publ Soc Hosp Med. 2012;7(5):376-381. doi:10.1002/jhm.1928. PubMed
6. Goldman J, Reeves S, Wu R, Silver I, MacMillan K, Kitto S. Medical Residents and Interprofessional Interactions in Discharge: An Ethnographic Exploration of Factors That Affect Negotiation. J Gen Intern Med. 2015;30(10):1454-1460. doi:10.1007/s11606-015-3306-6. PubMed
7. Okoniewska B, Santana MJ, Groshaus H, et al. Barriers to discharge in an acute care medical teaching unit: a qualitative analysis of health providers’ perceptions. J Multidiscip Healthc. 2015;8:83-89. doi:10.2147/JMDH.S72633. PubMed
8. Carey MR, Sheth H, Scott Braithwaite R. A Prospective Study of Reasons for Prolonged Hospitalizations on a General Medicine Teaching Service. J Gen Intern Med. 2005;20(2):108-115. doi:10.1111/j.1525-1497.2005.40269.x. PubMed
9. Kim CS, Hart AL, Paretti RF, et al. Excess Hospitalization Days in an Academic Medical Center: Perceptions of Hospitalists and Discharge Planners. Am J Manag Care. 2011;17(2):e34-e42. http://www.ajmc.com/journals/issue/2011/2011-2-vol17-n2/AJMC_11feb_Kim_WebX_e34to42/. Accessed on October 26, 2016.
10. Gershengorn HB, Kocher R, Factor P. Management Strategies to Effect Change in Intensive Care Units: Lessons from the World of Business. Part II. Quality-Improvement Strategies. Ann Am Thorac Soc. 2014;11(3):444-453. doi:10.1513/AnnalsATS.201311-392AS. PubMed
11. Wertheimer B, Jacobs REA, Bailey M, et al. Discharge before noon: An achievable hospital goal. J Hosp Med. 2014;9(4):210-214. doi:10.1002/jhm.2154. PubMed
12. Manning DM, Tammel KJ, Blegen RN, et al. In-room display of day and time patient is anticipated to leave hospital: a “discharge appointment.” J Hosp Med. 2007;2(1):13-16. doi:10.1002/jhm.146. PubMed
13. Networks for academic medical centers. https://www.vizientinc.com/Our-networks/Networks-for-academic-medical-centers. Accessed on July 13, 2017.
14. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research Electronic Data Capture (REDCap) - A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381. doi:10.1016/j.jbi.2008.08.010. PubMed
15. Selker HP, Beshansky JR, Pauker SG, Kassirer JP. The epidemiology of delays in a teaching hospital. The development and use of a tool that detects unnecessary hospital days. Med Care. 1989;27(2):112-129. PubMed

Article PDF
Issue
Journal of Hospital Medicine 13(6)
Publications
Topics
Page Number
388-391. Published online first December 6, 2017.
Sections
Files
Files
Article PDF
Article PDF

The discharge process is a critical bottleneck for efficient patient flow through the hospital. Delayed discharges translate into delays in admissions and other patient transitions, often leading to excess costs, patient dissatisfaction, and even patient harm.1-3 The emergency department is particularly impacted by these delays; bottlenecks there lead to overcrowding, increased overall hospital length of stay, and increased risks for bad outcomes during hospitalization.2

Academic medical centers in particular may struggle with delayed discharges. In a typical teaching hospital, a team composed of an attending physician and housestaff share responsibility for determining the discharge plan. Additionally, clinical teaching activities may affect the process and quality of discharge.4-6

The prevalence and causes of delayed discharges vary greatly.7-9 To improve efficiency around discharge, many hospitals have launched initiatives designed to discharge patients earlier in the day, including goal setting (“discharge by noon”), scheduling discharge appointments, and using quality-improvement methods, such as Lean Methodology (LEAN), to remove inefficiencies within discharge processes.10-12 However, there are few data on the prevalence and effectiveness of different strategies.

The aim of this study was to survey academic hospitalist and general internal medicine physician leaders to elicit their perspectives on the factors contributing to discharge timing and the relative importance and effectiveness of early-discharge initiatives.

METHODS

Study Design, Participants, and Oversight

We obtained a list of 115 university-affiliated hospitals associated with a residency program and, in most cases, a medical school from Vizient Inc. (formerly University HealthSystem Consortium), an alliance of academic medical centers and affiliated hospitals. Each member institution submits clinical data to allow for the benchmarking of outcomes to drive transparency and quality improvement.13 More than 95% of the nation’s academic medical centers and affiliated hospitals participate in this collaborative. Vizient works with members but does not set nor promote quality metrics, such as discharge timeliness. E-mail addresses for hospital medicine physician leaders (eg, division chief) of major academic medical centers were obtained from each institution via publicly available data (eg, the institution’s website). When an institution did not have a hospital medicine section, we identified the division chief of general internal medicine. The University of California, San Francisco Institutional Review Board approved this study.

Survey Development and Domains

We developed a 30-item survey to evaluate 5 main domains of interest: current discharge practices, degree of prioritization of early discharge on the inpatient service, barriers to timely discharge, prevalence and perceived effectiveness of implemented early-discharge initiatives, and barriers to implementation of early-discharge initiatives.

Respondents were first asked to identify their institutions’ goals for discharge time. They were then asked to compare the priority of early-discharge initiatives to other departmental quality-improvement initiatives, such as reducing 30-day readmissions, improving interpreter use, and improving patient satisfaction. Next, respondents were asked to estimate the degree to which clinical or patient factors contributed to delays in discharge. Respondents were then asked whether specific early-discharge initiatives, such as changes to rounding practices or communication interventions, were implemented at their institutions and, if so, the perceived effectiveness of these initiatives at meeting discharge targets. We piloted the questions locally with physicians and researchers prior to finalizing the survey.

Data Collection

We sent surveys via an online platform (Research Electronic Data Capture).14 Nonresponders were sent 2 e-mail reminders and then a follow-up telephone call asking them to complete the survey. Only 1 survey per academic medical center was collected. Any respondent who completed the survey within 2 weeks of receiving it was entered to win a Kindle Fire.

Data Analysis

We summarized survey responses using descriptive statistics. Analysis was completed in IBM SPSS version 22 (Armonk, NY).

RESULTS

Survey Respondent and Institutional Characteristics

Of the 115 institutions surveyed, we received 61 responses (response rate of 53%), with 39 (64%) respondents from divisions of hospital medicine and 22 (36%) from divisions of general internal medicine. A majority (n = 53; 87%) stated their medicine services have a combination of teaching (with residents) and nonteaching (without residents) teams. Thirty-nine (64%) reported having daily multidisciplinary rounds.

 

 

Early Discharge as a Priority

Forty-seven (77%) institutional representatives strongly agreed or agreed that early discharge was a priority, with discharge by noon being the most common target time (n = 23; 38%). Thirty (50%) respondents rated early discharge as more important than improving interpreter use for non-English-speaking patients and equally important as reducing 30-day readmissions (n = 29; 48%) and improving patient satisfaction (n = 27; 44%).

Factors Delaying Discharge

The most common factors perceived as delaying discharge were considered external to the hospital, such as postacute care bed availability or scheduled (eg, ambulance) transport delays (n = 48; 79%), followed by patient factors such as patient transport issues (n = 44; 72%). Less commonly reported were workflow issues, such as competing primary team priorities or case manager bandwidth (n = 38; 62%; Table 1).

Initiatives to Improve Discharge

The most commonly implemented initiatives perceived as effective at improving discharge times were the preemptive identification of early discharges to plan discharge paperwork (n = 34; 56%), communication with patients about anticipated discharge time on the day prior to discharge (n = 29; 48%), and the implementation of additional rounds between physician teams and case managers specifically around discharge planning (n = 28; 46%). Initiatives not commonly implemented included regular audit of and feedback on discharge times to providers and teams (n = 21; 34%), the use of a discharge readiness checklist (n = 26; 43%), incentives such as bonuses or penalties (n = 37; 61%), the use of a whiteboard to indicate discharge times (n = 23; 38%), and dedicated quality-improvement approaches such as LEAN (n = 37; 61%; Table 2).

DISCUSSION

Our study suggests early discharge for medicine patients is a priority among academic institutions. Hospitalist and general internal medicine physician leaders in our study generally attributed delayed discharges to external factors, particularly unavailability of postacute care facilities and transportation delays. Having issues with finding postacute care placements is consistent with previous findings by Selker et al.15 and Carey et al.8 This is despite the 20-year difference between Selker et al.’s study and the current study, reflecting a continued opportunity for improvement, including stronger partnerships with local and regional postacute care facilities to expedite care transition and stronger discharge-planning efforts early in the admission process. Efforts in postacute care placement may be particularly important for Medicaid-insured and uninsured patients.

Our responders, hospitalist and internal medicine physician leaders, did not perceive the additional responsibilities of teaching and supervising trainees to be factors that significantly delayed patient discharge. This is in contrast to previous studies, which attributed delays in discharge to prolonged clinical decision-making related to teaching and supervision.4-6,8 This discrepancy may be due to the fact that we only surveyed single physician leaders at each institution and not residents. Our finding warrants further investigation to understand the degree to which resident skills may impact discharge planning and processes.

Institutions represented in our study have attempted a variety of initiatives promoting earlier discharge, with varying levels of perceived success. Initiatives perceived to be the most effective by hospital leaders centered on 2 main areas: (1) changing individual provider practice and (2) anticipatory discharge preparation. Interestingly, this is in discordance with the main factors labeled as causing delays in discharges, such as obtaining postacute care beds, busy case managers, and competing demands on primary teams. We hypothesize this may be because such changes require organization- or system-level changes and are perceived as more arduous than changes at the individual level. In addition, changes to individual provider behavior may be more cost- and time-effective than more systemic initiatives.

Our findings are consistent with the work published by Wertheimer and colleagues,11 who show that additional afternoon interdisciplinary rounds can help identify patients who may be discharged before noon the next day. In their study, identifying such patients in advance improved the overall early-discharge rate the following day.

Our findings should be interpreted in light of several limitations. Our survey only considers the perspectives of hospitalist and general internal medicine physician leaders at academic medical centers that are part of the Vizient Inc. collaborative. They do not represent all academic or community-based medical centers. Although the perceived effectiveness of some initiatives was high, we did not collect empirical data to support these claims or to determine which initiative had the greatest relative impact on discharge timeliness. Lastly, we did not obtain resident, nursing, or case manager perspectives on discharge practices. Given their roles as frontline providers, we may have missed these alternative perspectives.

Our study shows there is a strong interest in increasing early discharges in an effort to improve hospital throughput and patient flow.

 

 

Acknowledgments

The authors thank all participants who completed the survey and Danielle Carrier at Vizient Inc. (formally University HealthSystem Consortium) for her assistance in obtaining data.

Disclosures

Hemali Patel, Margaret Fang, Michelle Mourad, Adrienne Green, Ryan Murphy, and James Harrison report no conflicts of interest. At the time the research was conducted, Robert Wachter reported that he is a member of the Lucian Leape Institute at the National Patient Safety Foundation (no compensation except travel expenses); recently chaired an advisory board to England’s National Health Service (NHS) reviewing the NHS’s digital health strategy (no compensation except travel expenses); has a contract with UCSF from the Agency for Healthcare Research and Quality to edit a patient-safety website; receives compensation from John Wiley & Sons for writing a blog; receives royalties from Lippincott Williams & Wilkins and McGraw-Hill Education for writing and/or editing several books; receives stock options for serving on the board of Acuity Medical Management Systems; receives a yearly stipend for serving on the board of The Doctors Company; serves on the scientific advisory boards for amino.com, PatientSafe Solutions Inc., Twine, and EarlySense (for which he receives stock options); has a small royalty stake in CareWeb, a hospital communication tool developed at UCSF; and holds the Marc and Lynne Benioff Endowed Chair in Hospital Medicine and the Holly Smith Distinguished Professorship in Science and Medicine at UCSF.

 

The discharge process is a critical bottleneck for efficient patient flow through the hospital. Delayed discharges translate into delays in admissions and other patient transitions, often leading to excess costs, patient dissatisfaction, and even patient harm.1-3 The emergency department is particularly impacted by these delays; bottlenecks there lead to overcrowding, increased overall hospital length of stay, and increased risks for bad outcomes during hospitalization.2

Academic medical centers in particular may struggle with delayed discharges. In a typical teaching hospital, a team composed of an attending physician and housestaff share responsibility for determining the discharge plan. Additionally, clinical teaching activities may affect the process and quality of discharge.4-6

The prevalence and causes of delayed discharges vary greatly.7-9 To improve efficiency around discharge, many hospitals have launched initiatives designed to discharge patients earlier in the day, including goal setting (“discharge by noon”), scheduling discharge appointments, and using quality-improvement methods, such as Lean Methodology (LEAN), to remove inefficiencies within discharge processes.10-12 However, there are few data on the prevalence and effectiveness of different strategies.

The aim of this study was to survey academic hospitalist and general internal medicine physician leaders to elicit their perspectives on the factors contributing to discharge timing and the relative importance and effectiveness of early-discharge initiatives.

METHODS

Study Design, Participants, and Oversight

We obtained a list of 115 university-affiliated hospitals associated with a residency program and, in most cases, a medical school from Vizient Inc. (formerly University HealthSystem Consortium), an alliance of academic medical centers and affiliated hospitals. Each member institution submits clinical data to allow for the benchmarking of outcomes to drive transparency and quality improvement.13 More than 95% of the nation’s academic medical centers and affiliated hospitals participate in this collaborative. Vizient works with members but does not set nor promote quality metrics, such as discharge timeliness. E-mail addresses for hospital medicine physician leaders (eg, division chief) of major academic medical centers were obtained from each institution via publicly available data (eg, the institution’s website). When an institution did not have a hospital medicine section, we identified the division chief of general internal medicine. The University of California, San Francisco Institutional Review Board approved this study.

Survey Development and Domains

We developed a 30-item survey to evaluate 5 main domains of interest: current discharge practices, degree of prioritization of early discharge on the inpatient service, barriers to timely discharge, prevalence and perceived effectiveness of implemented early-discharge initiatives, and barriers to implementation of early-discharge initiatives.

Respondents were first asked to identify their institutions’ goals for discharge time. They were then asked to compare the priority of early-discharge initiatives to other departmental quality-improvement initiatives, such as reducing 30-day readmissions, improving interpreter use, and improving patient satisfaction. Next, respondents were asked to estimate the degree to which clinical or patient factors contributed to delays in discharge. Respondents were then asked whether specific early-discharge initiatives, such as changes to rounding practices or communication interventions, were implemented at their institutions and, if so, the perceived effectiveness of these initiatives at meeting discharge targets. We piloted the questions locally with physicians and researchers prior to finalizing the survey.

Data Collection

We sent surveys via an online platform (Research Electronic Data Capture).14 Nonresponders were sent 2 e-mail reminders and then a follow-up telephone call asking them to complete the survey. Only 1 survey per academic medical center was collected. Any respondent who completed the survey within 2 weeks of receiving it was entered to win a Kindle Fire.

Data Analysis

We summarized survey responses using descriptive statistics. Analysis was completed in IBM SPSS version 22 (Armonk, NY).

RESULTS

Survey Respondent and Institutional Characteristics

Of the 115 institutions surveyed, we received 61 responses (response rate of 53%), with 39 (64%) respondents from divisions of hospital medicine and 22 (36%) from divisions of general internal medicine. A majority (n = 53; 87%) stated their medicine services have a combination of teaching (with residents) and nonteaching (without residents) teams. Thirty-nine (64%) reported having daily multidisciplinary rounds.

 

 

Early Discharge as a Priority

Forty-seven (77%) institutional representatives strongly agreed or agreed that early discharge was a priority, with discharge by noon being the most common target time (n = 23; 38%). Thirty (50%) respondents rated early discharge as more important than improving interpreter use for non-English-speaking patients and equally important as reducing 30-day readmissions (n = 29; 48%) and improving patient satisfaction (n = 27; 44%).

Factors Delaying Discharge

The most common factors perceived as delaying discharge were considered external to the hospital, such as postacute care bed availability or scheduled (eg, ambulance) transport delays (n = 48; 79%), followed by patient factors such as patient transport issues (n = 44; 72%). Less commonly reported were workflow issues, such as competing primary team priorities or case manager bandwidth (n = 38; 62%; Table 1).

Initiatives to Improve Discharge

The most commonly implemented initiatives perceived as effective at improving discharge times were the preemptive identification of early discharges to plan discharge paperwork (n = 34; 56%), communication with patients about anticipated discharge time on the day prior to discharge (n = 29; 48%), and the implementation of additional rounds between physician teams and case managers specifically around discharge planning (n = 28; 46%). Initiatives not commonly implemented included regular audit of and feedback on discharge times to providers and teams (n = 21; 34%), the use of a discharge readiness checklist (n = 26; 43%), incentives such as bonuses or penalties (n = 37; 61%), the use of a whiteboard to indicate discharge times (n = 23; 38%), and dedicated quality-improvement approaches such as LEAN (n = 37; 61%; Table 2).

DISCUSSION

Our study suggests early discharge for medicine patients is a priority among academic institutions. Hospitalist and general internal medicine physician leaders in our study generally attributed delayed discharges to external factors, particularly unavailability of postacute care facilities and transportation delays. Having issues with finding postacute care placements is consistent with previous findings by Selker et al.15 and Carey et al.8 This is despite the 20-year difference between Selker et al.’s study and the current study, reflecting a continued opportunity for improvement, including stronger partnerships with local and regional postacute care facilities to expedite care transition and stronger discharge-planning efforts early in the admission process. Efforts in postacute care placement may be particularly important for Medicaid-insured and uninsured patients.

Our responders, hospitalist and internal medicine physician leaders, did not perceive the additional responsibilities of teaching and supervising trainees to be factors that significantly delayed patient discharge. This is in contrast to previous studies, which attributed delays in discharge to prolonged clinical decision-making related to teaching and supervision.4-6,8 This discrepancy may be due to the fact that we only surveyed single physician leaders at each institution and not residents. Our finding warrants further investigation to understand the degree to which resident skills may impact discharge planning and processes.

Institutions represented in our study have attempted a variety of initiatives promoting earlier discharge, with varying levels of perceived success. Initiatives perceived to be the most effective by hospital leaders centered on 2 main areas: (1) changing individual provider practice and (2) anticipatory discharge preparation. Interestingly, this is in discordance with the main factors labeled as causing delays in discharges, such as obtaining postacute care beds, busy case managers, and competing demands on primary teams. We hypothesize this may be because such changes require organization- or system-level changes and are perceived as more arduous than changes at the individual level. In addition, changes to individual provider behavior may be more cost- and time-effective than more systemic initiatives.

Our findings are consistent with the work published by Wertheimer and colleagues,11 who show that additional afternoon interdisciplinary rounds can help identify patients who may be discharged before noon the next day. In their study, identifying such patients in advance improved the overall early-discharge rate the following day.

Our findings should be interpreted in light of several limitations. Our survey only considers the perspectives of hospitalist and general internal medicine physician leaders at academic medical centers that are part of the Vizient Inc. collaborative. They do not represent all academic or community-based medical centers. Although the perceived effectiveness of some initiatives was high, we did not collect empirical data to support these claims or to determine which initiative had the greatest relative impact on discharge timeliness. Lastly, we did not obtain resident, nursing, or case manager perspectives on discharge practices. Given their roles as frontline providers, we may have missed these alternative perspectives.

Our study shows there is a strong interest in increasing early discharges in an effort to improve hospital throughput and patient flow.

 

 

Acknowledgments

The authors thank all participants who completed the survey and Danielle Carrier at Vizient Inc. (formally University HealthSystem Consortium) for her assistance in obtaining data.

Disclosures

Hemali Patel, Margaret Fang, Michelle Mourad, Adrienne Green, Ryan Murphy, and James Harrison report no conflicts of interest. At the time the research was conducted, Robert Wachter reported that he is a member of the Lucian Leape Institute at the National Patient Safety Foundation (no compensation except travel expenses); recently chaired an advisory board to England’s National Health Service (NHS) reviewing the NHS’s digital health strategy (no compensation except travel expenses); has a contract with UCSF from the Agency for Healthcare Research and Quality to edit a patient-safety website; receives compensation from John Wiley & Sons for writing a blog; receives royalties from Lippincott Williams & Wilkins and McGraw-Hill Education for writing and/or editing several books; receives stock options for serving on the board of Acuity Medical Management Systems; receives a yearly stipend for serving on the board of The Doctors Company; serves on the scientific advisory boards for amino.com, PatientSafe Solutions Inc., Twine, and EarlySense (for which he receives stock options); has a small royalty stake in CareWeb, a hospital communication tool developed at UCSF; and holds the Marc and Lynne Benioff Endowed Chair in Hospital Medicine and the Holly Smith Distinguished Professorship in Science and Medicine at UCSF.

 

References

1. Khanna S, Boyle J, Good N, Lind J. Impact of admission and discharge peak times on hospital overcrowding. Stud Health Technol Inform. 2011;168:82-88. PubMed
2. White BA, Biddinger PD, Chang Y, Grabowski B, Carignan S, Brown DFM. Boarding Inpatients in the Emergency Department Increases Discharged Patient Length of Stay. J Emerg Med. 2013;44(1):230-235. doi:10.1016/j.jemermed.2012.05.007. PubMed
3. Derlet RW, Richards JR. Overcrowding in the nation’s emergency departments: complex causes and disturbing effects. Ann Emerg Med. 2000;35(1):63-68. PubMed
4. da Silva SA, Valácio RA, Botelho FC, Amaral CFS. Reasons for discharge delays in teaching hospitals. Rev Saúde Pública. 2014;48(2):314-321. doi:10.1590/S0034-8910.2014048004971. PubMed
5. Greysen SR, Schiliro D, Horwitz LI, Curry L, Bradley EH. “Out of Sight, Out of Mind”: Housestaff Perceptions of Quality-Limiting Factors in Discharge Care at Teaching Hospitals. J Hosp Med Off Publ Soc Hosp Med. 2012;7(5):376-381. doi:10.1002/jhm.1928. PubMed
6. Goldman J, Reeves S, Wu R, Silver I, MacMillan K, Kitto S. Medical Residents and Interprofessional Interactions in Discharge: An Ethnographic Exploration of Factors That Affect Negotiation. J Gen Intern Med. 2015;30(10):1454-1460. doi:10.1007/s11606-015-3306-6. PubMed
7. Okoniewska B, Santana MJ, Groshaus H, et al. Barriers to discharge in an acute care medical teaching unit: a qualitative analysis of health providers’ perceptions. J Multidiscip Healthc. 2015;8:83-89. doi:10.2147/JMDH.S72633. PubMed
8. Carey MR, Sheth H, Scott Braithwaite R. A Prospective Study of Reasons for Prolonged Hospitalizations on a General Medicine Teaching Service. J Gen Intern Med. 2005;20(2):108-115. doi:10.1111/j.1525-1497.2005.40269.x. PubMed
9. Kim CS, Hart AL, Paretti RF, et al. Excess Hospitalization Days in an Academic Medical Center: Perceptions of Hospitalists and Discharge Planners. Am J Manag Care. 2011;17(2):e34-e42. http://www.ajmc.com/journals/issue/2011/2011-2-vol17-n2/AJMC_11feb_Kim_WebX_e34to42/. Accessed on October 26, 2016.
10. Gershengorn HB, Kocher R, Factor P. Management Strategies to Effect Change in Intensive Care Units: Lessons from the World of Business. Part II. Quality-Improvement Strategies. Ann Am Thorac Soc. 2014;11(3):444-453. doi:10.1513/AnnalsATS.201311-392AS. PubMed
11. Wertheimer B, Jacobs REA, Bailey M, et al. Discharge before noon: An achievable hospital goal. J Hosp Med. 2014;9(4):210-214. doi:10.1002/jhm.2154. PubMed
12. Manning DM, Tammel KJ, Blegen RN, et al. In-room display of day and time patient is anticipated to leave hospital: a “discharge appointment.” J Hosp Med. 2007;2(1):13-16. doi:10.1002/jhm.146. PubMed
13. Networks for academic medical centers. https://www.vizientinc.com/Our-networks/Networks-for-academic-medical-centers. Accessed on July 13, 2017.
14. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research Electronic Data Capture (REDCap) - A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381. doi:10.1016/j.jbi.2008.08.010. PubMed
15. Selker HP, Beshansky JR, Pauker SG, Kassirer JP. The epidemiology of delays in a teaching hospital. The development and use of a tool that detects unnecessary hospital days. Med Care. 1989;27(2):112-129. PubMed

References

1. Khanna S, Boyle J, Good N, Lind J. Impact of admission and discharge peak times on hospital overcrowding. Stud Health Technol Inform. 2011;168:82-88. PubMed
2. White BA, Biddinger PD, Chang Y, Grabowski B, Carignan S, Brown DFM. Boarding Inpatients in the Emergency Department Increases Discharged Patient Length of Stay. J Emerg Med. 2013;44(1):230-235. doi:10.1016/j.jemermed.2012.05.007. PubMed
3. Derlet RW, Richards JR. Overcrowding in the nation’s emergency departments: complex causes and disturbing effects. Ann Emerg Med. 2000;35(1):63-68. PubMed
4. da Silva SA, Valácio RA, Botelho FC, Amaral CFS. Reasons for discharge delays in teaching hospitals. Rev Saúde Pública. 2014;48(2):314-321. doi:10.1590/S0034-8910.2014048004971. PubMed
5. Greysen SR, Schiliro D, Horwitz LI, Curry L, Bradley EH. “Out of Sight, Out of Mind”: Housestaff Perceptions of Quality-Limiting Factors in Discharge Care at Teaching Hospitals. J Hosp Med Off Publ Soc Hosp Med. 2012;7(5):376-381. doi:10.1002/jhm.1928. PubMed
6. Goldman J, Reeves S, Wu R, Silver I, MacMillan K, Kitto S. Medical Residents and Interprofessional Interactions in Discharge: An Ethnographic Exploration of Factors That Affect Negotiation. J Gen Intern Med. 2015;30(10):1454-1460. doi:10.1007/s11606-015-3306-6. PubMed
7. Okoniewska B, Santana MJ, Groshaus H, et al. Barriers to discharge in an acute care medical teaching unit: a qualitative analysis of health providers’ perceptions. J Multidiscip Healthc. 2015;8:83-89. doi:10.2147/JMDH.S72633. PubMed
8. Carey MR, Sheth H, Scott Braithwaite R. A Prospective Study of Reasons for Prolonged Hospitalizations on a General Medicine Teaching Service. J Gen Intern Med. 2005;20(2):108-115. doi:10.1111/j.1525-1497.2005.40269.x. PubMed
9. Kim CS, Hart AL, Paretti RF, et al. Excess Hospitalization Days in an Academic Medical Center: Perceptions of Hospitalists and Discharge Planners. Am J Manag Care. 2011;17(2):e34-e42. http://www.ajmc.com/journals/issue/2011/2011-2-vol17-n2/AJMC_11feb_Kim_WebX_e34to42/. Accessed on October 26, 2016.
10. Gershengorn HB, Kocher R, Factor P. Management Strategies to Effect Change in Intensive Care Units: Lessons from the World of Business. Part II. Quality-Improvement Strategies. Ann Am Thorac Soc. 2014;11(3):444-453. doi:10.1513/AnnalsATS.201311-392AS. PubMed
11. Wertheimer B, Jacobs REA, Bailey M, et al. Discharge before noon: An achievable hospital goal. J Hosp Med. 2014;9(4):210-214. doi:10.1002/jhm.2154. PubMed
12. Manning DM, Tammel KJ, Blegen RN, et al. In-room display of day and time patient is anticipated to leave hospital: a “discharge appointment.” J Hosp Med. 2007;2(1):13-16. doi:10.1002/jhm.146. PubMed
13. Networks for academic medical centers. https://www.vizientinc.com/Our-networks/Networks-for-academic-medical-centers. Accessed on July 13, 2017.
14. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research Electronic Data Capture (REDCap) - A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381. doi:10.1016/j.jbi.2008.08.010. PubMed
15. Selker HP, Beshansky JR, Pauker SG, Kassirer JP. The epidemiology of delays in a teaching hospital. The development and use of a tool that detects unnecessary hospital days. Med Care. 1989;27(2):112-129. PubMed

Issue
Journal of Hospital Medicine 13(6)
Issue
Journal of Hospital Medicine 13(6)
Page Number
388-391. Published online first December 6, 2017.
Page Number
388-391. Published online first December 6, 2017.
Publications
Publications
Topics
Article Type
Sections
Article Source

© 2017 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Hemali Patel, MD, 12401 E. 17th Ave, Suite 450B, Mail Stop F-782, Aurora, CO 80045; Telephone: 720-848-4289; Fax: 720-848-4293; E-mail: Hemali.Patel@ucdenver.edu
Content Gating
Gated (full article locked unless allowed per User)
Alternative CME
Disqus Comments
Default
Gate On Date
Wed, 07/11/2018 - 05:00
Un-Gate On Date
Wed, 06/13/2018 - 05:00
Use ProPublica
Gating Strategy
First Peek Free
Article PDF Media
Media Files

It all just clicks: Development of an inpatient e-consult program

Article Type
Changed
Sat, 05/13/2017 - 13:47
Display Headline
It all just clicks: Development of an inpatient e-consult program

Electronic consultation (e-consult) in the outpatient setting allows subspecialists to provide assessment and recommendations for patients without in-person visits.1 An e-consult is an asynchronous communication that uses the electronic medical record (EMR) and typically involves an electronic order from a requesting provider and an electronic note from a consulting provider. The initial motivation for developing this consultation modality was to improve access to subspecialty care for patients in the primary care setting, and findings of studies at several sites support this claim.1-4 In addition, e-consult may also reduce cost because converting unnecessary face-to-face encounters into e-consults reduces patients’ travel costs and healthcare organizations’ expensive subspecialty clinic time.3,5 Moreover, instead of addressing less complex clinical questions in informal, undocumented face-to-face or telephone “curbside” consultations with specialists, providers can instead ask for e-consults and thereby ensure thorough chart review and proper documentation.6

Use of e-consults in the inpatient setting is relatively novel.7 In addition to having the advantages already mentioned, e-consults are faster than in-person bedside consultations and may be beneficial in the fast-moving inpatient care setting. Finally, healthcare systems with multiple hospital sites may not have the capacity to physically locate subspecialists at each site, which makes e-consults attractive for avoiding unnecessary travel time.

In this article, we describe how we developed an inpatient e-consult protocol for a new, remote hospital within our healthcare system and explore data on safety and physician attitudes after e-consult implementation.

METHODS

The Institutional Review Board of the University of California San Francisco (UCSF) approved this study.

Setting

In February 2015, UCSF opened a new hospital in the Mission Bay neighborhood of San Francisco, 4 miles from the existing hospital. The new hospital is home to several adult inpatient services: urology, otolaryngology, colorectal surgery, obstetrics, and gynecologic surgery. A hospitalist is on-site 24 hours a day to provide consultation for these services around issues that relate to internal medicine. A hospitalist who requires subspecialty expertise to answer a clinical question can request a consultation by in-person visit, video telemedicine, or e-consult, each of which is available 24/7. Almost all of the medicine subspecialists work on the existing campus, not in Mission Bay.

Protocol Development and Implementation

The protocol for the e-consult program was developed over several months by an interdisciplinary group that included 3 hospitalists, 1 obstetrician, 1 project manager, and 1 informaticist. The group outlined the process for requesting and completing an e-consult (Figure), designed a note template for consultants to use for EMR documentation, conducted outreach with subspecialty groups to discuss the protocol, and developed an EMR report to track e-consult use and content over time. As our medical center does not bill payers for inpatient e-consults, e-consult note tracking is used to provide reimbursement internally, from the medical center to the respective departments of the consultants. Reimbursement is made at a set rate per e-consult note, with the rate set to approximate the reimbursement of a low-acuity in-person consult on the main campus.

Process for requesting and completing an e-consult. Order not mandatory.
Figure

The workflow of an e-consult is as follows: (1) When a primary team requires a consultation on an issue that falls within the purview of internal medicine, it pages the on-site hospitalist. (2) The hospitalist accepts the consultation by phone, reviews the chart, and examines the patient. (3) If the hospitalist requires subspecialty assistance to answer a clinical question, he or she contacts the appropriate subspecialty service by pager. (4) The subspecialist speaks with the hospitalist about the consultation question, and together they decide if an e-consult is appropriate, based on the complexity of the clinical scenario. (5) The subspecialist reviews the patient’s chart and documents their care plan recommendations in an e-consult note. Consultants can use e-consult for both initial and follow-up assessment, and there is no strict requirement that they also contact the hospitalist or the primary team by phone in addition to consultation note. Given their novelty, almost all e-consults are performed by subspecialist attendings, not residents or fellows.

 

 

Evaluation

Each month, we tracked e-consult use using an EMR report built as part of the implementation of the program. For the first four months of implementation, every patient who received an e-consult also had a manual chart review of the period around the e-consult, performed by a hospitalist, in order to audit for any potential safety issues. These issues included, for example, an e-consult performed for a patient whose complexity or severity of illness was felt to be too great to defer an in-person visit, or a patient who received e-consult recommendations that were significantly retracted in a follow-up in-person note.

Eight months after the program started, we assessed experience by electronically surveying the 9 hospitalists and 11 consultants who had requested or performed at least 2 e-consults.8 Survey items were measured on a 5-point Likert scale: strongly disagree to strongly agree. The items, which related to ease of calling for a consultation, quality of e-consults, impact on clinical care, safety concerns, and satisfaction, were inspired by themes identified in a systematic review of the literature on e-consults in the outpatient setting.2 We sent 2 reminders to responders. Data were summarized using descriptive statistics. Analysis was performed in SPSS version 22.0 (IBM).

RESULTS

There were 143 initial subspecialty consultations by e-consult between program launch in February 2015 and manuscript preparation in February 2016, an average of 11 e-consults per month. There were 313 total e-consult notes (these included both initial and follow-up e-consult notes). By comparison, 240 initial in-person consultations occurred during the same period, and there were 435 total in-person consultation notes (46% new or initial notes, 54% follow-up notes). The top 5 subspecialties by volume of e-consults were infectious disease (35%), hematology (20%), endocrinology (14%), nephrology (13%), and cardiology (8%). For reference, e-consults are also available from psychiatry, neurology, oncology, gastroenterology, pulmonology, and rheumatology. Percentage of consultations performed during daytime hours (defined as 8 a.m. to 5 p.m.) was 92% for e-consults and 96% for in-person consultations.

There were no e-consult–related patient safety issues reported through the medical center’s incident reporting system during the study period. There were also no patient safety issues identified in the manual audits of 80 charts during the first 4 months of the program.

Seven (78%) of 9 hospitalists and 7 (64%) of 11 consultants completed the survey. Both groups agreed that e-consults were easy to use and efficient (Table). All hospitalists were satisfied with the quality of e-consult recommendations, but only 3 (43%) of the 7 consultants agreed they could provide high-quality consultation by e-consult. In their comments, 2 consultants expressed concerns. One concern was about missing crucial information by performing only a chart review, and the other was about being tempted to perform an e-consult simply because it is expedient.

Results of E-Consult Survey Given to Hospitalists and Consultants
Table

DISCUSSION

Although use of e-consults in the outpatient setting is relatively commonplace, our program represents a novel use of e-consults in safely and efficiently providing subspecialty consultation to inpatients at a remote hospital.

For hospitalists, an e-consult system offers numerous benefits. Clinical questions beyond an internists’ scope of practice come up often, and simple questions might traditionally result in an informal curbside consult. While a curbside consult provides answers faster than an in-person visit, it creates risks for the requesting hospitalists: the consultants only know what they are told, whether the information is incomplete or erroneous; their opinions are given without documentation or compensation, which reduces a sense of accountability; and the lack of documentation does not allow their advice to persist in the chart as a reference for future providers.9 Our e-consult program solves these problems by requiring that consultants perform chart review and provide documentation as well as obligating the medical center to pay a small compensation to consultants for their time. We hope this lowers the bar to requesting consultation for remote sites, where the alternative would be burdensome travel time to do an in-person visit.

In our study, hospitalists were universally pleased with the quality of e-consult recommendations, but only 43% of consultants agreed. These findings correlate with the literature on e-consults in the outpatient setting.2 Unfortunately, our survey comments did not shed further light on this sentiment. In the outpatient literature, consultants were most concerned with having a clear clinical question, facing the liability of providing recommendations without performing an examination, and receiving appropriate compensation for answering e-consults.

The generalizability of our program findings is limited most significantly by the particular arrangement of our clinical services: Our remote site is home to a select group of adult inpatient services, a hospitalist is available on-site for these services 24 hours a day, and the distance to the remote site can be overcome with modest effort should a patient require an in-person visit in the initial or follow-up period. The generalizability of our safety findings is limited by the use of a single reviewer for chart auditing.

Given the rise of accountable care organizations and the prevalence of hospital mergers in the healthcare landscape, we believe that healthcare systems that operate remote sites under constrained budgets could look to e-consults to more cost-effectively extend subspecialty expertise across the inpatient enterprise. With improvements in health information exchange, it may also become feasible for consultants to offer e-consults to hospitals outside a medical center’s network. Our study showed that inpatient e-consult programs can be developed and implemented, that they appear not to pose any significant safety issues, and that they can facilitate delivery of timely clinical care.

 

 

Acknowledgment

The authors thank Raphaela Levy-Moore for creating and implementing the e-consult note template for our electronic medical record.

Disclosure

Nothing to report.

 

References

1. Chen AH, Murphy EJ, Yee HF Jr. eReferral—a new model for integrated care. N Engl J Med. 2013;368(26):2450-2453. PubMed
2. Vimalananda VG, Gupte G, Seraj SM, et al. Electronic consultations (e-consults) to improve access to specialty care: a systematic review and narrative synthesis. J Telemed Telecare. 2015;21(6):323-330. PubMed
3. Kirsh S, Carey E, Aron DC, et al. Impact of a national specialty e-consultation implementation project on access. Am J Manag Care. 2015;21(12):e648-e654. PubMed
4. Bergman J, Neuhausen K, Chamie K, et al. Building a medical neighborhood in the safety net: an innovative technology improves hematuria workups. Urology. 2013;82(6):1277-1282. PubMed
5. Wasfy JH, Rao SK, Chittle MD, Gallen KM, Isselbacher EM, Ferris TG. Initial results of a cardiac e-consult pilot program. J Am Coll Cardiol. 2014;64(24):2706-2707. PubMed
6. Perley CM. Physician use of the curbside consultation to address information needs: report on a collective case study. J Med Libr Assoc. 2006;94(2):137-144. PubMed
7. Gupte G, Vimalananda V, Simon SR, DeVito K, Clark J, Orlander JD. Disruptive innovation: implementation of electronic consultations in a Veterans Affairs health care system. JMIR Med Inform. 2016;4(1):e6. PubMed
8. REDCap. Vanderbilt University website. http://www.project-redcap.org. 2015. Accessed March 3, 2016.
9. Burden M, Sarcone E, Keniston A, et al. Prospective comparison of curbside versus formal consultations. J Hosp Med. 2013;8(1):31-35. PubMed

Article PDF
Issue
Journal of Hospital Medicine 12(5)
Publications
Topics
Page Number
332-334
Sections
Article PDF
Article PDF

Electronic consultation (e-consult) in the outpatient setting allows subspecialists to provide assessment and recommendations for patients without in-person visits.1 An e-consult is an asynchronous communication that uses the electronic medical record (EMR) and typically involves an electronic order from a requesting provider and an electronic note from a consulting provider. The initial motivation for developing this consultation modality was to improve access to subspecialty care for patients in the primary care setting, and findings of studies at several sites support this claim.1-4 In addition, e-consult may also reduce cost because converting unnecessary face-to-face encounters into e-consults reduces patients’ travel costs and healthcare organizations’ expensive subspecialty clinic time.3,5 Moreover, instead of addressing less complex clinical questions in informal, undocumented face-to-face or telephone “curbside” consultations with specialists, providers can instead ask for e-consults and thereby ensure thorough chart review and proper documentation.6

Use of e-consults in the inpatient setting is relatively novel.7 In addition to having the advantages already mentioned, e-consults are faster than in-person bedside consultations and may be beneficial in the fast-moving inpatient care setting. Finally, healthcare systems with multiple hospital sites may not have the capacity to physically locate subspecialists at each site, which makes e-consults attractive for avoiding unnecessary travel time.

In this article, we describe how we developed an inpatient e-consult protocol for a new, remote hospital within our healthcare system and explore data on safety and physician attitudes after e-consult implementation.

METHODS

The Institutional Review Board of the University of California San Francisco (UCSF) approved this study.

Setting

In February 2015, UCSF opened a new hospital in the Mission Bay neighborhood of San Francisco, 4 miles from the existing hospital. The new hospital is home to several adult inpatient services: urology, otolaryngology, colorectal surgery, obstetrics, and gynecologic surgery. A hospitalist is on-site 24 hours a day to provide consultation for these services around issues that relate to internal medicine. A hospitalist who requires subspecialty expertise to answer a clinical question can request a consultation by in-person visit, video telemedicine, or e-consult, each of which is available 24/7. Almost all of the medicine subspecialists work on the existing campus, not in Mission Bay.

Protocol Development and Implementation

The protocol for the e-consult program was developed over several months by an interdisciplinary group that included 3 hospitalists, 1 obstetrician, 1 project manager, and 1 informaticist. The group outlined the process for requesting and completing an e-consult (Figure), designed a note template for consultants to use for EMR documentation, conducted outreach with subspecialty groups to discuss the protocol, and developed an EMR report to track e-consult use and content over time. As our medical center does not bill payers for inpatient e-consults, e-consult note tracking is used to provide reimbursement internally, from the medical center to the respective departments of the consultants. Reimbursement is made at a set rate per e-consult note, with the rate set to approximate the reimbursement of a low-acuity in-person consult on the main campus.

Process for requesting and completing an e-consult. Order not mandatory.
Figure

The workflow of an e-consult is as follows: (1) When a primary team requires a consultation on an issue that falls within the purview of internal medicine, it pages the on-site hospitalist. (2) The hospitalist accepts the consultation by phone, reviews the chart, and examines the patient. (3) If the hospitalist requires subspecialty assistance to answer a clinical question, he or she contacts the appropriate subspecialty service by pager. (4) The subspecialist speaks with the hospitalist about the consultation question, and together they decide if an e-consult is appropriate, based on the complexity of the clinical scenario. (5) The subspecialist reviews the patient’s chart and documents their care plan recommendations in an e-consult note. Consultants can use e-consult for both initial and follow-up assessment, and there is no strict requirement that they also contact the hospitalist or the primary team by phone in addition to consultation note. Given their novelty, almost all e-consults are performed by subspecialist attendings, not residents or fellows.

 

 

Evaluation

Each month, we tracked e-consult use using an EMR report built as part of the implementation of the program. For the first four months of implementation, every patient who received an e-consult also had a manual chart review of the period around the e-consult, performed by a hospitalist, in order to audit for any potential safety issues. These issues included, for example, an e-consult performed for a patient whose complexity or severity of illness was felt to be too great to defer an in-person visit, or a patient who received e-consult recommendations that were significantly retracted in a follow-up in-person note.

Eight months after the program started, we assessed experience by electronically surveying the 9 hospitalists and 11 consultants who had requested or performed at least 2 e-consults.8 Survey items were measured on a 5-point Likert scale: strongly disagree to strongly agree. The items, which related to ease of calling for a consultation, quality of e-consults, impact on clinical care, safety concerns, and satisfaction, were inspired by themes identified in a systematic review of the literature on e-consults in the outpatient setting.2 We sent 2 reminders to responders. Data were summarized using descriptive statistics. Analysis was performed in SPSS version 22.0 (IBM).

RESULTS

There were 143 initial subspecialty consultations by e-consult between program launch in February 2015 and manuscript preparation in February 2016, an average of 11 e-consults per month. There were 313 total e-consult notes (these included both initial and follow-up e-consult notes). By comparison, 240 initial in-person consultations occurred during the same period, and there were 435 total in-person consultation notes (46% new or initial notes, 54% follow-up notes). The top 5 subspecialties by volume of e-consults were infectious disease (35%), hematology (20%), endocrinology (14%), nephrology (13%), and cardiology (8%). For reference, e-consults are also available from psychiatry, neurology, oncology, gastroenterology, pulmonology, and rheumatology. Percentage of consultations performed during daytime hours (defined as 8 a.m. to 5 p.m.) was 92% for e-consults and 96% for in-person consultations.

There were no e-consult–related patient safety issues reported through the medical center’s incident reporting system during the study period. There were also no patient safety issues identified in the manual audits of 80 charts during the first 4 months of the program.

Seven (78%) of 9 hospitalists and 7 (64%) of 11 consultants completed the survey. Both groups agreed that e-consults were easy to use and efficient (Table). All hospitalists were satisfied with the quality of e-consult recommendations, but only 3 (43%) of the 7 consultants agreed they could provide high-quality consultation by e-consult. In their comments, 2 consultants expressed concerns. One concern was about missing crucial information by performing only a chart review, and the other was about being tempted to perform an e-consult simply because it is expedient.

Results of E-Consult Survey Given to Hospitalists and Consultants
Table

DISCUSSION

Although use of e-consults in the outpatient setting is relatively commonplace, our program represents a novel use of e-consults in safely and efficiently providing subspecialty consultation to inpatients at a remote hospital.

For hospitalists, an e-consult system offers numerous benefits. Clinical questions beyond an internists’ scope of practice come up often, and simple questions might traditionally result in an informal curbside consult. While a curbside consult provides answers faster than an in-person visit, it creates risks for the requesting hospitalists: the consultants only know what they are told, whether the information is incomplete or erroneous; their opinions are given without documentation or compensation, which reduces a sense of accountability; and the lack of documentation does not allow their advice to persist in the chart as a reference for future providers.9 Our e-consult program solves these problems by requiring that consultants perform chart review and provide documentation as well as obligating the medical center to pay a small compensation to consultants for their time. We hope this lowers the bar to requesting consultation for remote sites, where the alternative would be burdensome travel time to do an in-person visit.

In our study, hospitalists were universally pleased with the quality of e-consult recommendations, but only 43% of consultants agreed. These findings correlate with the literature on e-consults in the outpatient setting.2 Unfortunately, our survey comments did not shed further light on this sentiment. In the outpatient literature, consultants were most concerned with having a clear clinical question, facing the liability of providing recommendations without performing an examination, and receiving appropriate compensation for answering e-consults.

The generalizability of our program findings is limited most significantly by the particular arrangement of our clinical services: Our remote site is home to a select group of adult inpatient services, a hospitalist is available on-site for these services 24 hours a day, and the distance to the remote site can be overcome with modest effort should a patient require an in-person visit in the initial or follow-up period. The generalizability of our safety findings is limited by the use of a single reviewer for chart auditing.

Given the rise of accountable care organizations and the prevalence of hospital mergers in the healthcare landscape, we believe that healthcare systems that operate remote sites under constrained budgets could look to e-consults to more cost-effectively extend subspecialty expertise across the inpatient enterprise. With improvements in health information exchange, it may also become feasible for consultants to offer e-consults to hospitals outside a medical center’s network. Our study showed that inpatient e-consult programs can be developed and implemented, that they appear not to pose any significant safety issues, and that they can facilitate delivery of timely clinical care.

 

 

Acknowledgment

The authors thank Raphaela Levy-Moore for creating and implementing the e-consult note template for our electronic medical record.

Disclosure

Nothing to report.

 

Electronic consultation (e-consult) in the outpatient setting allows subspecialists to provide assessment and recommendations for patients without in-person visits.1 An e-consult is an asynchronous communication that uses the electronic medical record (EMR) and typically involves an electronic order from a requesting provider and an electronic note from a consulting provider. The initial motivation for developing this consultation modality was to improve access to subspecialty care for patients in the primary care setting, and findings of studies at several sites support this claim.1-4 In addition, e-consult may also reduce cost because converting unnecessary face-to-face encounters into e-consults reduces patients’ travel costs and healthcare organizations’ expensive subspecialty clinic time.3,5 Moreover, instead of addressing less complex clinical questions in informal, undocumented face-to-face or telephone “curbside” consultations with specialists, providers can instead ask for e-consults and thereby ensure thorough chart review and proper documentation.6

Use of e-consults in the inpatient setting is relatively novel.7 In addition to having the advantages already mentioned, e-consults are faster than in-person bedside consultations and may be beneficial in the fast-moving inpatient care setting. Finally, healthcare systems with multiple hospital sites may not have the capacity to physically locate subspecialists at each site, which makes e-consults attractive for avoiding unnecessary travel time.

In this article, we describe how we developed an inpatient e-consult protocol for a new, remote hospital within our healthcare system and explore data on safety and physician attitudes after e-consult implementation.

METHODS

The Institutional Review Board of the University of California San Francisco (UCSF) approved this study.

Setting

In February 2015, UCSF opened a new hospital in the Mission Bay neighborhood of San Francisco, 4 miles from the existing hospital. The new hospital is home to several adult inpatient services: urology, otolaryngology, colorectal surgery, obstetrics, and gynecologic surgery. A hospitalist is on-site 24 hours a day to provide consultation for these services around issues that relate to internal medicine. A hospitalist who requires subspecialty expertise to answer a clinical question can request a consultation by in-person visit, video telemedicine, or e-consult, each of which is available 24/7. Almost all of the medicine subspecialists work on the existing campus, not in Mission Bay.

Protocol Development and Implementation

The protocol for the e-consult program was developed over several months by an interdisciplinary group that included 3 hospitalists, 1 obstetrician, 1 project manager, and 1 informaticist. The group outlined the process for requesting and completing an e-consult (Figure), designed a note template for consultants to use for EMR documentation, conducted outreach with subspecialty groups to discuss the protocol, and developed an EMR report to track e-consult use and content over time. As our medical center does not bill payers for inpatient e-consults, e-consult note tracking is used to provide reimbursement internally, from the medical center to the respective departments of the consultants. Reimbursement is made at a set rate per e-consult note, with the rate set to approximate the reimbursement of a low-acuity in-person consult on the main campus.

Process for requesting and completing an e-consult. Order not mandatory.
Figure

The workflow of an e-consult is as follows: (1) When a primary team requires a consultation on an issue that falls within the purview of internal medicine, it pages the on-site hospitalist. (2) The hospitalist accepts the consultation by phone, reviews the chart, and examines the patient. (3) If the hospitalist requires subspecialty assistance to answer a clinical question, he or she contacts the appropriate subspecialty service by pager. (4) The subspecialist speaks with the hospitalist about the consultation question, and together they decide if an e-consult is appropriate, based on the complexity of the clinical scenario. (5) The subspecialist reviews the patient’s chart and documents their care plan recommendations in an e-consult note. Consultants can use e-consult for both initial and follow-up assessment, and there is no strict requirement that they also contact the hospitalist or the primary team by phone in addition to consultation note. Given their novelty, almost all e-consults are performed by subspecialist attendings, not residents or fellows.

 

 

Evaluation

Each month, we tracked e-consult use using an EMR report built as part of the implementation of the program. For the first four months of implementation, every patient who received an e-consult also had a manual chart review of the period around the e-consult, performed by a hospitalist, in order to audit for any potential safety issues. These issues included, for example, an e-consult performed for a patient whose complexity or severity of illness was felt to be too great to defer an in-person visit, or a patient who received e-consult recommendations that were significantly retracted in a follow-up in-person note.

Eight months after the program started, we assessed experience by electronically surveying the 9 hospitalists and 11 consultants who had requested or performed at least 2 e-consults.8 Survey items were measured on a 5-point Likert scale: strongly disagree to strongly agree. The items, which related to ease of calling for a consultation, quality of e-consults, impact on clinical care, safety concerns, and satisfaction, were inspired by themes identified in a systematic review of the literature on e-consults in the outpatient setting.2 We sent 2 reminders to responders. Data were summarized using descriptive statistics. Analysis was performed in SPSS version 22.0 (IBM).

RESULTS

There were 143 initial subspecialty consultations by e-consult between program launch in February 2015 and manuscript preparation in February 2016, an average of 11 e-consults per month. There were 313 total e-consult notes (these included both initial and follow-up e-consult notes). By comparison, 240 initial in-person consultations occurred during the same period, and there were 435 total in-person consultation notes (46% new or initial notes, 54% follow-up notes). The top 5 subspecialties by volume of e-consults were infectious disease (35%), hematology (20%), endocrinology (14%), nephrology (13%), and cardiology (8%). For reference, e-consults are also available from psychiatry, neurology, oncology, gastroenterology, pulmonology, and rheumatology. Percentage of consultations performed during daytime hours (defined as 8 a.m. to 5 p.m.) was 92% for e-consults and 96% for in-person consultations.

There were no e-consult–related patient safety issues reported through the medical center’s incident reporting system during the study period. There were also no patient safety issues identified in the manual audits of 80 charts during the first 4 months of the program.

Seven (78%) of 9 hospitalists and 7 (64%) of 11 consultants completed the survey. Both groups agreed that e-consults were easy to use and efficient (Table). All hospitalists were satisfied with the quality of e-consult recommendations, but only 3 (43%) of the 7 consultants agreed they could provide high-quality consultation by e-consult. In their comments, 2 consultants expressed concerns. One concern was about missing crucial information by performing only a chart review, and the other was about being tempted to perform an e-consult simply because it is expedient.

Results of E-Consult Survey Given to Hospitalists and Consultants
Table

DISCUSSION

Although use of e-consults in the outpatient setting is relatively commonplace, our program represents a novel use of e-consults in safely and efficiently providing subspecialty consultation to inpatients at a remote hospital.

For hospitalists, an e-consult system offers numerous benefits. Clinical questions beyond an internists’ scope of practice come up often, and simple questions might traditionally result in an informal curbside consult. While a curbside consult provides answers faster than an in-person visit, it creates risks for the requesting hospitalists: the consultants only know what they are told, whether the information is incomplete or erroneous; their opinions are given without documentation or compensation, which reduces a sense of accountability; and the lack of documentation does not allow their advice to persist in the chart as a reference for future providers.9 Our e-consult program solves these problems by requiring that consultants perform chart review and provide documentation as well as obligating the medical center to pay a small compensation to consultants for their time. We hope this lowers the bar to requesting consultation for remote sites, where the alternative would be burdensome travel time to do an in-person visit.

In our study, hospitalists were universally pleased with the quality of e-consult recommendations, but only 43% of consultants agreed. These findings correlate with the literature on e-consults in the outpatient setting.2 Unfortunately, our survey comments did not shed further light on this sentiment. In the outpatient literature, consultants were most concerned with having a clear clinical question, facing the liability of providing recommendations without performing an examination, and receiving appropriate compensation for answering e-consults.

The generalizability of our program findings is limited most significantly by the particular arrangement of our clinical services: Our remote site is home to a select group of adult inpatient services, a hospitalist is available on-site for these services 24 hours a day, and the distance to the remote site can be overcome with modest effort should a patient require an in-person visit in the initial or follow-up period. The generalizability of our safety findings is limited by the use of a single reviewer for chart auditing.

Given the rise of accountable care organizations and the prevalence of hospital mergers in the healthcare landscape, we believe that healthcare systems that operate remote sites under constrained budgets could look to e-consults to more cost-effectively extend subspecialty expertise across the inpatient enterprise. With improvements in health information exchange, it may also become feasible for consultants to offer e-consults to hospitals outside a medical center’s network. Our study showed that inpatient e-consult programs can be developed and implemented, that they appear not to pose any significant safety issues, and that they can facilitate delivery of timely clinical care.

 

 

Acknowledgment

The authors thank Raphaela Levy-Moore for creating and implementing the e-consult note template for our electronic medical record.

Disclosure

Nothing to report.

 

References

1. Chen AH, Murphy EJ, Yee HF Jr. eReferral—a new model for integrated care. N Engl J Med. 2013;368(26):2450-2453. PubMed
2. Vimalananda VG, Gupte G, Seraj SM, et al. Electronic consultations (e-consults) to improve access to specialty care: a systematic review and narrative synthesis. J Telemed Telecare. 2015;21(6):323-330. PubMed
3. Kirsh S, Carey E, Aron DC, et al. Impact of a national specialty e-consultation implementation project on access. Am J Manag Care. 2015;21(12):e648-e654. PubMed
4. Bergman J, Neuhausen K, Chamie K, et al. Building a medical neighborhood in the safety net: an innovative technology improves hematuria workups. Urology. 2013;82(6):1277-1282. PubMed
5. Wasfy JH, Rao SK, Chittle MD, Gallen KM, Isselbacher EM, Ferris TG. Initial results of a cardiac e-consult pilot program. J Am Coll Cardiol. 2014;64(24):2706-2707. PubMed
6. Perley CM. Physician use of the curbside consultation to address information needs: report on a collective case study. J Med Libr Assoc. 2006;94(2):137-144. PubMed
7. Gupte G, Vimalananda V, Simon SR, DeVito K, Clark J, Orlander JD. Disruptive innovation: implementation of electronic consultations in a Veterans Affairs health care system. JMIR Med Inform. 2016;4(1):e6. PubMed
8. REDCap. Vanderbilt University website. http://www.project-redcap.org. 2015. Accessed March 3, 2016.
9. Burden M, Sarcone E, Keniston A, et al. Prospective comparison of curbside versus formal consultations. J Hosp Med. 2013;8(1):31-35. PubMed

References

1. Chen AH, Murphy EJ, Yee HF Jr. eReferral—a new model for integrated care. N Engl J Med. 2013;368(26):2450-2453. PubMed
2. Vimalananda VG, Gupte G, Seraj SM, et al. Electronic consultations (e-consults) to improve access to specialty care: a systematic review and narrative synthesis. J Telemed Telecare. 2015;21(6):323-330. PubMed
3. Kirsh S, Carey E, Aron DC, et al. Impact of a national specialty e-consultation implementation project on access. Am J Manag Care. 2015;21(12):e648-e654. PubMed
4. Bergman J, Neuhausen K, Chamie K, et al. Building a medical neighborhood in the safety net: an innovative technology improves hematuria workups. Urology. 2013;82(6):1277-1282. PubMed
5. Wasfy JH, Rao SK, Chittle MD, Gallen KM, Isselbacher EM, Ferris TG. Initial results of a cardiac e-consult pilot program. J Am Coll Cardiol. 2014;64(24):2706-2707. PubMed
6. Perley CM. Physician use of the curbside consultation to address information needs: report on a collective case study. J Med Libr Assoc. 2006;94(2):137-144. PubMed
7. Gupte G, Vimalananda V, Simon SR, DeVito K, Clark J, Orlander JD. Disruptive innovation: implementation of electronic consultations in a Veterans Affairs health care system. JMIR Med Inform. 2016;4(1):e6. PubMed
8. REDCap. Vanderbilt University website. http://www.project-redcap.org. 2015. Accessed March 3, 2016.
9. Burden M, Sarcone E, Keniston A, et al. Prospective comparison of curbside versus formal consultations. J Hosp Med. 2013;8(1):31-35. PubMed

Issue
Journal of Hospital Medicine 12(5)
Issue
Journal of Hospital Medicine 12(5)
Page Number
332-334
Page Number
332-334
Publications
Publications
Topics
Article Type
Display Headline
It all just clicks: Development of an inpatient e-consult program
Display Headline
It all just clicks: Development of an inpatient e-consult program
Sections
Article Source

© 2017 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Nader Najafi, MD, Division of Hospital Medicine, University of California San Francisco, 505 Parnassus Ave, M1283, San Francisco, CA 94143-0131; Telephone: 415-476-2223; Fax: 415-502-1963; E-mail: nader.najafi@ucsf.edu

Content Gating
Gated (full article locked unless allowed per User)
Alternative CME
Gating Strategy
First Peek Free
Article PDF Media

Promoting Scholarship for Hospitalists

Article Type
Changed
Wed, 04/05/2017 - 08:03
Display Headline
Implementation and evaluation of a “works‐in‐progress” session to promote scholarship in an academic hospitalist group

Academic hospital medicine is a fast‐growing specialty and has a strong emphasis on high‐value care, efficiency, and quality improvement (QI).[1] Developing scholarly work in these areas and describing findings in peer‐reviewed publications can help disseminate ideas and innovations more widely. In addition, success in academic medicine, at least in part, continues to be measured by traditional academic benchmarks, including the production of scholarly publications, conference presentations, and abstracts.[2]

Hospital medicine, however, faces challenges in providing an academic environment conducive to fostering scholarly work. As a relatively young specialty, there may be a dearth of senior mentors and experienced researchers; lack of structured mentorship can be associated with failure to produce publications or lead national teaching sessions.[3] Relatively few hospitalists undergo fellowships or other specialized training that provides a clinical research background, and internal medicine residency programs rarely provide the comprehensive research skill set required to design, implement, or disseminate academic work.[4, 5, 6] Finally, heavy clinical responsibilities may hinder efforts to conduct and sustain research.

A works‐in‐progress (WIP) session, commonly employed in clinical research groups, can provide a forum to discuss and receive feedback on evolving projects and can foster mentorship, motivation, and training.[7] Although a WIP session may stimulate discussion and advance project ideas, academic hospitalist groups do not commonly employ this model, and it is not known if a regularly scheduled WIP session can provide the mentorship, training, and motivation necessary to assist junior faculty in advancing scholarly project to completion.[8] In this article, we describe how we developed a regular WIP series to promote scholarship activities within our rapidly growing, primarily clinically focused Division of Hospital Medicine (DHM) at the University of California, San Francisco (UCSF), and the results of a survey of WIP participants. We hope that our experience can help illustrate key features of such a model, as well as describe inherent challenges and lessons learned to help promote successful academic efforts at other institutions.

METHODS

Program Setting

During years 2010 to 2013, the time period captured by our survey, the DHM at UCSF grew from 37 to 46 full‐time hospitalists, with 76% primarily clinical faculty (nonresearchers) and 24% primarily clinician‐investigators (researchers), defined as individuals having completed a 2‐year clinical research fellowship and/or dedicating 70% time in their faculty position to clinical research. In addition, there were between 1 and 3 hospitalist fellows per year. In 2012, a PhD researcher joined the division to support research and academic activities within the division as well as to pursue an independent research career.

Program Description

The DHM WIP, named the Incubator, was initially developed in 2007 when researchers recognized the need and desire for a forum where scholarly projects could be reviewed and evaluated. In the first year, the Incubator was primarily utilized by junior research‐trained mentees applying for National Institutes of Health career development awards. However, it soon became clear that nonresearch trained junior fellow and faculty members were pursuing scholarly projects needing additional guidance and input. In particular, the Incubator became frequently utilized by academic hospital medicine fellows and resident trainees pursuing QI and education projects. Over time, more DHM faculty, and junior faculty in particular, began to present their projects and receive structured feedback from researchers as well as other senior members of the group.

Incubator is structured as a 50‐minute session held from 1:10 to 2:00 pm on Thursdays in a DHM conference room. The time was selected because it did not conflict with other divisional conferences and to reserve mornings for clinical responsibilities. Incubator is held on most weeks of the year except for holidays or when there is no scheduled presenter. Presenting at Incubator is voluntary, and presenters sign up for open spots in advance with the upcoming presenter schedule sent out to the division in advance of the conference. Incubator is also used as a forum to provide feedback on anticipated abstract submissions for professional society meetings. For the purposes of the survey described in this article, we did not include Incubator sessions on reviewing abstracts/posters. Trainees and hospitalists present a broad range of projects at any stage of preparation. These include project ideas, grant applications, manuscripts, abstracts, and oral presentations at any stage of completion for feedback. Our mission was to create a forum where researchers, clinicians, and educators meet to provide the tools and guidance necessary to promote scholarly projects across the range of the division's activities by connecting individuals with complementary skills and interests and providing necessary mentorship and peer support. We have defined scholarship broadly, including evaluation of QI, global health, or other health system innovations, as well as advancements in medical education and traditional clinical research.

All faculty are invited to Incubator, and attendees include senior and junior faculty, researchers in the division, fellows, and occasionally residents and medical students. One week prior to the session, an administrative assistant solicits project information, including any related materials and questions the presenter may have for the group using a prespecified template, and emails this information to division members for review. In addition, the same materials are also printed prior to Incubator for any attendees who may not have reviewed the material in advance. Also, prior to the session, a physician is specified to serve as moderator of the discussion, and another physician is assigned the role of primary reviewer to provide the initial specific feedback and recommendations. The role of the moderator is to manage the discussion and keep the focus on time, and is assigned to a researcher or senior clinical faculty member. The role of primary reviewer is assigned to provide more junior faculty (both researchers and nonresearchers) the opportunity to practice their editing and critiquing skills by providing the initial feedback. Presenters and moderators receive worksheets outlining the structure of Incubator and their respective roles (see Supporting Information, Appendix 1, in the online version of this article).

Incubator begins with the presenter providing a brief synopsis of their project and their specific goals and objectives for the session. The moderator then leads the discussion and guides the format, often starting with any questions the group may have for the presenter followed by the specific feedback from the primary reviewer. The primary reviewer, having reviewed the materials in advance of the session, answers the prespecified questions as listed by the presenter, occasionally providing additional targeted feedback. The session is then opened to the rest of the group for feedback and suggestions. Meanwhile, the presenter is encouraged to wait until the end of the hour to summarize their take on the feedback and what their initial thoughts on the next to do items would be (Table 1).

Summary of Incubator Roles
PresenterAdministrative assistant
2‐ to 3‐sentence summary of career focusSchedule session and conference room
Distribute short set of materials in advanceCollect presenters' materials in advance
Summarize feedback at end of sessionPrepare materials for Incubator
Brainstorm on next steps at end of sessionMonitor attendance and topics of presentation
Primary reviewerModerator
Junior faculty (24 years)Senior or research faculty
Provide brief overview of projectKeep session on time
Reiterate key questionsGive additional input
Provide 2 major, 3 minor suggestionsSummarize comments from group at the end
Constructive, outside the box feedbackAllow last 10 minutes for presenter to discuss plans

Program Evaluation

Survey Respondents and Process

We retrospectively surveyed the lead presenter for each Incubator session held between May 2010 through November 2013. Surveys were administered through the Research Electronic Data Capture application (REDCap).[9] Participants who were lead presenters at Incubator for more than 1 Incubator session completed a survey for each individual presentation. Therefore, some presenters completed more than 1 survey. The presenters included resident physicians, hospital medicine fellows, junior faculty, and researchers. We defined researchers as hospitalists who had completed a 2‐year research fellowship and/or devoted at least 70% time in their faculty position to research.

Survey Development and Domains

We developed a survey questionnaire using the Kirkpatrick 4‐level model to evaluate the educational experience of the primary presenters and to determine how the session impacted their progress on the project, with each model component graded according to a Likert scale.[10] The 4 major components of the model are: (1) Reaction: participants' estimates of satisfaction with Incubator; (2) Learning: extent of knowledge acquisition achieved at Incubator; (3) Behavior: extent to which learning has been applied or transfer of skills through participation in Incubator; and (4) Results: results of the project, wider changes in organizational scholarship as impacted by Incubator.

We also collected information on the presenter's status at time of presentation including career paths (researcher or nonresearcher), their job description (faculty, fellow, resident), and the total number of years on faculty (if applicable). Hospitalists in their first 2 years on the faculty were considered junior physicians. We also collected information on the number of times they had presented at the Incubator sessions and stage of progress of the project, whether in the early, mid, or late phase at the time of presentation. Early phase was defined as presenting an initial project idea or brainstorming possible project options and/or directions. Mid phase was defined as presenting initial results, data, and initial drafts prior to completion of analysis. Late phase was defined as presenting a project nearing completion such as a written abstract, oral presentation, paper, or grant application. Respondents were also asked to identify the main focus of their projects, selecting the categories based on the interests of the division, including medical education, clinical research, QI, high‐value care, and global health.

Survey Data Analysis

We converted Likert scale data into dichotomous variables, with paring of positive responses versus the negative options. We summarized survey responses using descriptive statistics and determined if there were any differences in responses between career researchers and nonresearchers using 2 tests. All analysis was performed using StataSE version 13.1 (StataCorp, College Station, TX).

RESULTS

Survey Respondent Characteristics

We received 51 completed surveys from presenters at an Incubator session, for a total survey response rate of 70%. Of the 51 presentations, 26 (51%) of the projects were led by physicians in training or junior faculty, and 35 (69%) of the presenters were nonresearchers.

Project Characteristics

The most frequently presented topic areas were QI (N = 20), clinical research (N = 14), medical education (N = 6), and global health (N = 6). Whereas researchers were more likely to present clinical research topics and grant applications, nonresearchers more often presented on QI or medical education projects (Table 2). Projects were presented at all stages of development, with the middle stage, where presenters presented initial results, being the most common phase.

Characteristics of Work‐in‐Progress Session Presentations Among 51 Nonresearchers and Researchers
 AllNonresearcher, No. (%)Researcher, No. (%)P Value
  • NOTE: *Early stage was defined as presenting an initial project idea or brainstorming possible project options and/or directions. Mid stage was defined as presenting initial results, data, and initial drafts prior to completion of analysis. Late stage was defined as presenting a project nearing completion such as a written abstract, oral presentation, paper, or grant application.

Total513516 
Trainee or junior faculty 19 (54%)7 (44%)0.49
Topic of project   0.02
Quality improvement20 (39%)15 (43%)5 (31%) 
Clinical research14 (27%)8 (23%)6 (38%) 
Medical education6 (12%)5 (14%)1 (6%) 
Health technology4 (8%)0 (0%)4 (25%) 
High‐value care1 (2%)1 (3%)0 (0%) 
Global health6 (12%)6 (12%)0 (0%) 
Stage of project   0.31
Early*12 (23%)7 (20%)5 (31%) 
Middle24 (47%)19 (54%)5 (31%) 
Late15 (29%)9 (26%)6 (38%) 

Impact of Incubator

The reaction to the session was very positive, with 100% of respondents recommending Incubator to others (Table 3), and 35% reported learning as a result of the session. Twenty‐three (45%) of respondents reported that the session helped reframe the project idea and changed the study design, and 20 (39%) reported improved written or oral presentation style. A majority (45, 88%) reported that Incubator was valuable in advancing the project to completion.

Survey Responses of 51 WIP Presenters According to the Kirkpatrick Evaluation Hierarchy
 AllNonresearcher, No. (%)Researcher, No. (%)P Value
  • NOTE: Abbreviations: WIP, works‐in‐progress.

Trainee or junior faculty5135 (69%)16 (31%)0.49
Reaction    
Satisfied with their WIP session50 (98%)35 (100%)15 (94%)0.25
Would recommend WIP to others51 (100%)35 (100%)16 (100%)1.00
Any of the above 35 (100%)16 (100%)1.00
Learning    
Advanced research methodology18 (35%)12 (34%)6 (38%)0.82
Advanced knowledge in the area9 (18%)5 (14%)4 (25%)0.35
Any of the above 14 (40%)9 (56%)0.28
Behavior    
Current project    
Reframed project idea23 (45%)15 (43%)8 (50%)0.63
Changed study design or methodology23 (45%)16 (46%)7 (44%)0.9
Improved written or oral presentation style20 (39%)15 (43%)5 (31%)0.43
Future projects    
Changed approach to future projects19 (37%)17 (49%)2 (13%)0.01
Any of the above 34 (97%)14 (88%)0.17
Results    
Valuable in advancing project to completion45 (88%)31 (89%)14 (88%)0.18
Provided mentoring and peer support29 (57%)24 (69%)5 (31%)0.01
Connected individuals with similar results13 (13%)9 (26%)4 (25%)0.96
Any of the above 34 (97%)14 (88%)0.17

Survey results of researchers compared to nonresearchers were similar overall, although nonresearchers were more likely to report changes in behavior and in improved mentoring as a result of presenting at Incubator. Notably, 17 (49%) of nonresearchers reported that Incubator changed their approach to future projects as opposed to only 2 (13%) researchers (P = 0.01). In addition, 24 (69%) nonresearchers reported value in mentorship and peer support compared to 5 (31%) researchers (P = 0.01). A reasonably large proportion of projects originally presented during the Incubator sessions became published articles at the time of survey completion (N = 19, 37%) or were publications in progress (N = 14, 27%). For all remaining items, there were no statistically significant differences in the survey responses among junior faculty/trainees (N = 26) compared to nonjunior faculty (N = 25) presenters (P > 0.05).

Attendance at Incubator During the Study Period

Attendance at Incubator was open and voluntary for all DHM faculty, fellows, and collaborating UCSF trainees. From July 2012, when we began tracking attendance, through the end of the survey period in November 2013, the average number of attendees for each session was 10.7 (standard deviation [SD] 3.8). On average, 50% (SD 16%) of attendees at Incubator were career researchers.

DISCUSSION

The results of this program evaluation suggest that a WIP session employed by an academic division of hospital medicine, consisting of a weekly moderated session, can help advance scholarly work. Our evaluation found that presenters, both researchers and nonresearchers, favorably viewed the regular WIP sessions and reported that feedback in the Incubator helped them advance their project to completion. Importantly, nonresearch‐focused faculty and fellows reported the biggest gains in learning from presenting at Incubator. Whereas half the Incubator attendees were career researchers, consistent with the observation that researchers within the division were most committed to attending Incubator regularly, 69% of the presenters were nonresearchers, demonstrating strong participation among both researchers and nonresearchers within the division.

WIP sessions, though informal, are interactive, inspire critical self‐reflection, and encourage physicians to act on generated ideas, as evidenced by the change in behavior of the participants after the session. These sessions allow for transformative learning by encouraging physicians to be open to alternative viewpoints and engage in discourse, boosting learning beyond just content knowledge. Prior assessments of WIP seminars similarly found high satisfaction with these formats.[11]

Although we cannot identify specifically which features made Incubator effective, we believe that our WIP had some characteristics that may have contributed to its success and may aid in implementation at other institutions: holding the session regularly, voluntary participation, distributing the materials and questions for the group in advance, and designating a moderator for the session in advance to facilitate discussion.

A potential strength of the Incubator is that both researchers and nonresearchers attend. We hypothesize that combining these groups provides improved mentorship and learning for nonresearchers, in particular. In addition, it creates a mutually beneficial environment where each group is able to witness the diversity of projects within the division and learn to provide focused, constructive feedback on the presented work. Not only did this create a transparent environment with better understanding of divisional activities, but also fostered collaboration among hospitalists with similar interests and complementary skills.

Challenges, Setbacks, Updated Approaches

The creation of a successful Incubator session, however, was not without its challenges. At initial inception, the WIP was attended primarily by researchers and had low overall attendance. Members of the division who were primarily clinicians initially perceived the conferences as largely inapplicable to their career objectives and had competing demands from patient care, educational, or administrative responsibilities. However, over time and with encouragement from divisional leaders and service line directors, increasing numbers of hospitalists began to participate in Incubator. The timing of Incubator during afternoons after the Department of Medicine Grand Rounds was chosen specifically to allow clinicians to complete their responsibilities, including morning rounds and teaching, to allow better attendance.

In addition, the results of our survey informed changes to the structure of Incubator. The efficacy of assigning a primary reviewer for each session was not clear, so this component was eventually dropped. The finding that nonresearchers in particular reported a benefit from mentoring and peer‐support at Incubator led to the implementation of querying the presenter for a wish list of faculty attendees at their Incubator session. We then sent a special invitation to those faculty members thought to have special insights on the project. This gave junior faculty the opportunity to present their projects to more senior faculty members within their areas of research, as well as to receive focused expert feedback.

Finally, we have initiated special Incubator sessions focused more on didactics to teach the process of writing manuscripts and brainstorming workshop ideas for national meetings.

Limitations

Our study has limitations. It is a single‐center study based on a small overall sample size, and it is not certain whether a similar innovation would have comparable effects at another institution. In addition, generalizability of our results may be limited for hospital medicine groups without a robust research program. We did not have a control group nor do we know whether participants would have been equally successful without Incubator. We also were unable to assess how Incubator affected long‐term outcomes such as promotion and overall publication record, as we do not have detailed data on productivity prior to the survey period. Finally, we are unable to quantify the effect of Incubator on scholarly success in the division. Although the numbers of published articles and grant funding has increased since the Incubator began (data not shown), the division also grew both in number of research‐focused and nonresearch‐focused faculty, and this study does not account for other temporal changes that may have contributed to improvements in the scholarly output of the division.

CONCLUSIONS

In summary, the Incubator has been a successful program that fostered progress on scholarly projects within a largely clinically focused DHM. Given the importance of scholarship in academic hospital medicine, a WIP session such as the one we describe is a valuable way to support and mentor junior hospitalists and nonresearchers.

Acknowledgements

The authors extend special thanks to Oralia Schatzman, divisional administrative assistant, who organized and arranged the Incubator sessions and recorded attendance, and to Katherine Li, who collected data on numbers of faculty within the division over the duration of the study.

Disclosures: Dr. Hemali Patel, Dr. Margaret Fang, and Mr. James Harrison report no conflicts of interest. At the time the research was conducted, Dr. Kangelaris was supported by the National Heart, Lung, and Blood Institute (1K23HL116800‐01). Dr. Auerbach was supported by the National Heart, Lung, and Blood Institute (K24 K24HL098372).

Files
References
  1. Sehgal NL, Sharpe BA, Auerbach AA, Wachter RM. Investing in the future: building an academic hospitalist faculty development program. J Hosp Med. 2011;6(3):161166.
  2. Leykum LK, Parekh VI, Sharpe B, Boonyasai RT, Centor RM. Tried and true: a survey of successfully promoted academic hospitalists. J Hosp Med. 2011;6(7):411415.
  3. Reid MB, Misky GJ, Harrison RA, Sharpe B, Auerbach A, Glasheen JJ. Mentorship, productivity, and promotion among academic hospitalists. J Gen Intern Med. 2012;27(1):2327.
  4. Kohlwes R, Shunk R, Avins A, Garber J, Bent S, Shlipak M. The PRIME curriculum. J Gen Intern Med. 2006;21(5):506509.
  5. Ranji SR, Rosenman DJ, Amin AN, Kripalani S. Hospital medicine fellowships: works in progress. Am J Med. 2006;119(1):72.e1–7.
  6. Wilper AP, Smith CS, Weppner W. Instituting systems‐based practice and practice‐based learning and improvement: a curriculum of inquiry. Med Educ Online. 2013;18:21612.
  7. Grzybowski SCW, Bates J, Calam B, et al. A physician peer support writing group. Fam Med. 2003;35(3):195201.
  8. Abougergi MS, Wright SM, Landis R, Howell EE. Research in progress conference for hospitalists provides valuable peer mentoring. J Hosp Med. 2011;6(1):4346.
  9. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata‐driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377381.
  10. Kirkpatrick DL, Kirkpatrick JD. Evaluating Training Programs: The Four Levels. 3rd ed. San Francisco, CA: Berrett‐Koehler; 2006.
  11. Chang S, Hughes DC, Chamberlain RM. Works‐in‐progress: guiding junior scientists through career development applications. J Cancer Educ. 2008;23(3):142148.
Article PDF
Issue
Journal of Hospital Medicine - 11(10)
Publications
Page Number
719-723
Sections
Files
Files
Article PDF
Article PDF

Academic hospital medicine is a fast‐growing specialty and has a strong emphasis on high‐value care, efficiency, and quality improvement (QI).[1] Developing scholarly work in these areas and describing findings in peer‐reviewed publications can help disseminate ideas and innovations more widely. In addition, success in academic medicine, at least in part, continues to be measured by traditional academic benchmarks, including the production of scholarly publications, conference presentations, and abstracts.[2]

Hospital medicine, however, faces challenges in providing an academic environment conducive to fostering scholarly work. As a relatively young specialty, there may be a dearth of senior mentors and experienced researchers; lack of structured mentorship can be associated with failure to produce publications or lead national teaching sessions.[3] Relatively few hospitalists undergo fellowships or other specialized training that provides a clinical research background, and internal medicine residency programs rarely provide the comprehensive research skill set required to design, implement, or disseminate academic work.[4, 5, 6] Finally, heavy clinical responsibilities may hinder efforts to conduct and sustain research.

A works‐in‐progress (WIP) session, commonly employed in clinical research groups, can provide a forum to discuss and receive feedback on evolving projects and can foster mentorship, motivation, and training.[7] Although a WIP session may stimulate discussion and advance project ideas, academic hospitalist groups do not commonly employ this model, and it is not known if a regularly scheduled WIP session can provide the mentorship, training, and motivation necessary to assist junior faculty in advancing scholarly project to completion.[8] In this article, we describe how we developed a regular WIP series to promote scholarship activities within our rapidly growing, primarily clinically focused Division of Hospital Medicine (DHM) at the University of California, San Francisco (UCSF), and the results of a survey of WIP participants. We hope that our experience can help illustrate key features of such a model, as well as describe inherent challenges and lessons learned to help promote successful academic efforts at other institutions.

METHODS

Program Setting

During years 2010 to 2013, the time period captured by our survey, the DHM at UCSF grew from 37 to 46 full‐time hospitalists, with 76% primarily clinical faculty (nonresearchers) and 24% primarily clinician‐investigators (researchers), defined as individuals having completed a 2‐year clinical research fellowship and/or dedicating 70% time in their faculty position to clinical research. In addition, there were between 1 and 3 hospitalist fellows per year. In 2012, a PhD researcher joined the division to support research and academic activities within the division as well as to pursue an independent research career.

Program Description

The DHM WIP, named the Incubator, was initially developed in 2007 when researchers recognized the need and desire for a forum where scholarly projects could be reviewed and evaluated. In the first year, the Incubator was primarily utilized by junior research‐trained mentees applying for National Institutes of Health career development awards. However, it soon became clear that nonresearch trained junior fellow and faculty members were pursuing scholarly projects needing additional guidance and input. In particular, the Incubator became frequently utilized by academic hospital medicine fellows and resident trainees pursuing QI and education projects. Over time, more DHM faculty, and junior faculty in particular, began to present their projects and receive structured feedback from researchers as well as other senior members of the group.

Incubator is structured as a 50‐minute session held from 1:10 to 2:00 pm on Thursdays in a DHM conference room. The time was selected because it did not conflict with other divisional conferences and to reserve mornings for clinical responsibilities. Incubator is held on most weeks of the year except for holidays or when there is no scheduled presenter. Presenting at Incubator is voluntary, and presenters sign up for open spots in advance with the upcoming presenter schedule sent out to the division in advance of the conference. Incubator is also used as a forum to provide feedback on anticipated abstract submissions for professional society meetings. For the purposes of the survey described in this article, we did not include Incubator sessions on reviewing abstracts/posters. Trainees and hospitalists present a broad range of projects at any stage of preparation. These include project ideas, grant applications, manuscripts, abstracts, and oral presentations at any stage of completion for feedback. Our mission was to create a forum where researchers, clinicians, and educators meet to provide the tools and guidance necessary to promote scholarly projects across the range of the division's activities by connecting individuals with complementary skills and interests and providing necessary mentorship and peer support. We have defined scholarship broadly, including evaluation of QI, global health, or other health system innovations, as well as advancements in medical education and traditional clinical research.

All faculty are invited to Incubator, and attendees include senior and junior faculty, researchers in the division, fellows, and occasionally residents and medical students. One week prior to the session, an administrative assistant solicits project information, including any related materials and questions the presenter may have for the group using a prespecified template, and emails this information to division members for review. In addition, the same materials are also printed prior to Incubator for any attendees who may not have reviewed the material in advance. Also, prior to the session, a physician is specified to serve as moderator of the discussion, and another physician is assigned the role of primary reviewer to provide the initial specific feedback and recommendations. The role of the moderator is to manage the discussion and keep the focus on time, and is assigned to a researcher or senior clinical faculty member. The role of primary reviewer is assigned to provide more junior faculty (both researchers and nonresearchers) the opportunity to practice their editing and critiquing skills by providing the initial feedback. Presenters and moderators receive worksheets outlining the structure of Incubator and their respective roles (see Supporting Information, Appendix 1, in the online version of this article).

Incubator begins with the presenter providing a brief synopsis of their project and their specific goals and objectives for the session. The moderator then leads the discussion and guides the format, often starting with any questions the group may have for the presenter followed by the specific feedback from the primary reviewer. The primary reviewer, having reviewed the materials in advance of the session, answers the prespecified questions as listed by the presenter, occasionally providing additional targeted feedback. The session is then opened to the rest of the group for feedback and suggestions. Meanwhile, the presenter is encouraged to wait until the end of the hour to summarize their take on the feedback and what their initial thoughts on the next to do items would be (Table 1).

Summary of Incubator Roles
PresenterAdministrative assistant
2‐ to 3‐sentence summary of career focusSchedule session and conference room
Distribute short set of materials in advanceCollect presenters' materials in advance
Summarize feedback at end of sessionPrepare materials for Incubator
Brainstorm on next steps at end of sessionMonitor attendance and topics of presentation
Primary reviewerModerator
Junior faculty (24 years)Senior or research faculty
Provide brief overview of projectKeep session on time
Reiterate key questionsGive additional input
Provide 2 major, 3 minor suggestionsSummarize comments from group at the end
Constructive, outside the box feedbackAllow last 10 minutes for presenter to discuss plans

Program Evaluation

Survey Respondents and Process

We retrospectively surveyed the lead presenter for each Incubator session held between May 2010 through November 2013. Surveys were administered through the Research Electronic Data Capture application (REDCap).[9] Participants who were lead presenters at Incubator for more than 1 Incubator session completed a survey for each individual presentation. Therefore, some presenters completed more than 1 survey. The presenters included resident physicians, hospital medicine fellows, junior faculty, and researchers. We defined researchers as hospitalists who had completed a 2‐year research fellowship and/or devoted at least 70% time in their faculty position to research.

Survey Development and Domains

We developed a survey questionnaire using the Kirkpatrick 4‐level model to evaluate the educational experience of the primary presenters and to determine how the session impacted their progress on the project, with each model component graded according to a Likert scale.[10] The 4 major components of the model are: (1) Reaction: participants' estimates of satisfaction with Incubator; (2) Learning: extent of knowledge acquisition achieved at Incubator; (3) Behavior: extent to which learning has been applied or transfer of skills through participation in Incubator; and (4) Results: results of the project, wider changes in organizational scholarship as impacted by Incubator.

We also collected information on the presenter's status at time of presentation including career paths (researcher or nonresearcher), their job description (faculty, fellow, resident), and the total number of years on faculty (if applicable). Hospitalists in their first 2 years on the faculty were considered junior physicians. We also collected information on the number of times they had presented at the Incubator sessions and stage of progress of the project, whether in the early, mid, or late phase at the time of presentation. Early phase was defined as presenting an initial project idea or brainstorming possible project options and/or directions. Mid phase was defined as presenting initial results, data, and initial drafts prior to completion of analysis. Late phase was defined as presenting a project nearing completion such as a written abstract, oral presentation, paper, or grant application. Respondents were also asked to identify the main focus of their projects, selecting the categories based on the interests of the division, including medical education, clinical research, QI, high‐value care, and global health.

Survey Data Analysis

We converted Likert scale data into dichotomous variables, with paring of positive responses versus the negative options. We summarized survey responses using descriptive statistics and determined if there were any differences in responses between career researchers and nonresearchers using 2 tests. All analysis was performed using StataSE version 13.1 (StataCorp, College Station, TX).

RESULTS

Survey Respondent Characteristics

We received 51 completed surveys from presenters at an Incubator session, for a total survey response rate of 70%. Of the 51 presentations, 26 (51%) of the projects were led by physicians in training or junior faculty, and 35 (69%) of the presenters were nonresearchers.

Project Characteristics

The most frequently presented topic areas were QI (N = 20), clinical research (N = 14), medical education (N = 6), and global health (N = 6). Whereas researchers were more likely to present clinical research topics and grant applications, nonresearchers more often presented on QI or medical education projects (Table 2). Projects were presented at all stages of development, with the middle stage, where presenters presented initial results, being the most common phase.

Characteristics of Work‐in‐Progress Session Presentations Among 51 Nonresearchers and Researchers
 AllNonresearcher, No. (%)Researcher, No. (%)P Value
  • NOTE: *Early stage was defined as presenting an initial project idea or brainstorming possible project options and/or directions. Mid stage was defined as presenting initial results, data, and initial drafts prior to completion of analysis. Late stage was defined as presenting a project nearing completion such as a written abstract, oral presentation, paper, or grant application.

Total513516 
Trainee or junior faculty 19 (54%)7 (44%)0.49
Topic of project   0.02
Quality improvement20 (39%)15 (43%)5 (31%) 
Clinical research14 (27%)8 (23%)6 (38%) 
Medical education6 (12%)5 (14%)1 (6%) 
Health technology4 (8%)0 (0%)4 (25%) 
High‐value care1 (2%)1 (3%)0 (0%) 
Global health6 (12%)6 (12%)0 (0%) 
Stage of project   0.31
Early*12 (23%)7 (20%)5 (31%) 
Middle24 (47%)19 (54%)5 (31%) 
Late15 (29%)9 (26%)6 (38%) 

Impact of Incubator

The reaction to the session was very positive, with 100% of respondents recommending Incubator to others (Table 3), and 35% reported learning as a result of the session. Twenty‐three (45%) of respondents reported that the session helped reframe the project idea and changed the study design, and 20 (39%) reported improved written or oral presentation style. A majority (45, 88%) reported that Incubator was valuable in advancing the project to completion.

Survey Responses of 51 WIP Presenters According to the Kirkpatrick Evaluation Hierarchy
 AllNonresearcher, No. (%)Researcher, No. (%)P Value
  • NOTE: Abbreviations: WIP, works‐in‐progress.

Trainee or junior faculty5135 (69%)16 (31%)0.49
Reaction    
Satisfied with their WIP session50 (98%)35 (100%)15 (94%)0.25
Would recommend WIP to others51 (100%)35 (100%)16 (100%)1.00
Any of the above 35 (100%)16 (100%)1.00
Learning    
Advanced research methodology18 (35%)12 (34%)6 (38%)0.82
Advanced knowledge in the area9 (18%)5 (14%)4 (25%)0.35
Any of the above 14 (40%)9 (56%)0.28
Behavior    
Current project    
Reframed project idea23 (45%)15 (43%)8 (50%)0.63
Changed study design or methodology23 (45%)16 (46%)7 (44%)0.9
Improved written or oral presentation style20 (39%)15 (43%)5 (31%)0.43
Future projects    
Changed approach to future projects19 (37%)17 (49%)2 (13%)0.01
Any of the above 34 (97%)14 (88%)0.17
Results    
Valuable in advancing project to completion45 (88%)31 (89%)14 (88%)0.18
Provided mentoring and peer support29 (57%)24 (69%)5 (31%)0.01
Connected individuals with similar results13 (13%)9 (26%)4 (25%)0.96
Any of the above 34 (97%)14 (88%)0.17

Survey results of researchers compared to nonresearchers were similar overall, although nonresearchers were more likely to report changes in behavior and in improved mentoring as a result of presenting at Incubator. Notably, 17 (49%) of nonresearchers reported that Incubator changed their approach to future projects as opposed to only 2 (13%) researchers (P = 0.01). In addition, 24 (69%) nonresearchers reported value in mentorship and peer support compared to 5 (31%) researchers (P = 0.01). A reasonably large proportion of projects originally presented during the Incubator sessions became published articles at the time of survey completion (N = 19, 37%) or were publications in progress (N = 14, 27%). For all remaining items, there were no statistically significant differences in the survey responses among junior faculty/trainees (N = 26) compared to nonjunior faculty (N = 25) presenters (P > 0.05).

Attendance at Incubator During the Study Period

Attendance at Incubator was open and voluntary for all DHM faculty, fellows, and collaborating UCSF trainees. From July 2012, when we began tracking attendance, through the end of the survey period in November 2013, the average number of attendees for each session was 10.7 (standard deviation [SD] 3.8). On average, 50% (SD 16%) of attendees at Incubator were career researchers.

DISCUSSION

The results of this program evaluation suggest that a WIP session employed by an academic division of hospital medicine, consisting of a weekly moderated session, can help advance scholarly work. Our evaluation found that presenters, both researchers and nonresearchers, favorably viewed the regular WIP sessions and reported that feedback in the Incubator helped them advance their project to completion. Importantly, nonresearch‐focused faculty and fellows reported the biggest gains in learning from presenting at Incubator. Whereas half the Incubator attendees were career researchers, consistent with the observation that researchers within the division were most committed to attending Incubator regularly, 69% of the presenters were nonresearchers, demonstrating strong participation among both researchers and nonresearchers within the division.

WIP sessions, though informal, are interactive, inspire critical self‐reflection, and encourage physicians to act on generated ideas, as evidenced by the change in behavior of the participants after the session. These sessions allow for transformative learning by encouraging physicians to be open to alternative viewpoints and engage in discourse, boosting learning beyond just content knowledge. Prior assessments of WIP seminars similarly found high satisfaction with these formats.[11]

Although we cannot identify specifically which features made Incubator effective, we believe that our WIP had some characteristics that may have contributed to its success and may aid in implementation at other institutions: holding the session regularly, voluntary participation, distributing the materials and questions for the group in advance, and designating a moderator for the session in advance to facilitate discussion.

A potential strength of the Incubator is that both researchers and nonresearchers attend. We hypothesize that combining these groups provides improved mentorship and learning for nonresearchers, in particular. In addition, it creates a mutually beneficial environment where each group is able to witness the diversity of projects within the division and learn to provide focused, constructive feedback on the presented work. Not only did this create a transparent environment with better understanding of divisional activities, but also fostered collaboration among hospitalists with similar interests and complementary skills.

Challenges, Setbacks, Updated Approaches

The creation of a successful Incubator session, however, was not without its challenges. At initial inception, the WIP was attended primarily by researchers and had low overall attendance. Members of the division who were primarily clinicians initially perceived the conferences as largely inapplicable to their career objectives and had competing demands from patient care, educational, or administrative responsibilities. However, over time and with encouragement from divisional leaders and service line directors, increasing numbers of hospitalists began to participate in Incubator. The timing of Incubator during afternoons after the Department of Medicine Grand Rounds was chosen specifically to allow clinicians to complete their responsibilities, including morning rounds and teaching, to allow better attendance.

In addition, the results of our survey informed changes to the structure of Incubator. The efficacy of assigning a primary reviewer for each session was not clear, so this component was eventually dropped. The finding that nonresearchers in particular reported a benefit from mentoring and peer‐support at Incubator led to the implementation of querying the presenter for a wish list of faculty attendees at their Incubator session. We then sent a special invitation to those faculty members thought to have special insights on the project. This gave junior faculty the opportunity to present their projects to more senior faculty members within their areas of research, as well as to receive focused expert feedback.

Finally, we have initiated special Incubator sessions focused more on didactics to teach the process of writing manuscripts and brainstorming workshop ideas for national meetings.

Limitations

Our study has limitations. It is a single‐center study based on a small overall sample size, and it is not certain whether a similar innovation would have comparable effects at another institution. In addition, generalizability of our results may be limited for hospital medicine groups without a robust research program. We did not have a control group nor do we know whether participants would have been equally successful without Incubator. We also were unable to assess how Incubator affected long‐term outcomes such as promotion and overall publication record, as we do not have detailed data on productivity prior to the survey period. Finally, we are unable to quantify the effect of Incubator on scholarly success in the division. Although the numbers of published articles and grant funding has increased since the Incubator began (data not shown), the division also grew both in number of research‐focused and nonresearch‐focused faculty, and this study does not account for other temporal changes that may have contributed to improvements in the scholarly output of the division.

CONCLUSIONS

In summary, the Incubator has been a successful program that fostered progress on scholarly projects within a largely clinically focused DHM. Given the importance of scholarship in academic hospital medicine, a WIP session such as the one we describe is a valuable way to support and mentor junior hospitalists and nonresearchers.

Acknowledgements

The authors extend special thanks to Oralia Schatzman, divisional administrative assistant, who organized and arranged the Incubator sessions and recorded attendance, and to Katherine Li, who collected data on numbers of faculty within the division over the duration of the study.

Disclosures: Dr. Hemali Patel, Dr. Margaret Fang, and Mr. James Harrison report no conflicts of interest. At the time the research was conducted, Dr. Kangelaris was supported by the National Heart, Lung, and Blood Institute (1K23HL116800‐01). Dr. Auerbach was supported by the National Heart, Lung, and Blood Institute (K24 K24HL098372).

Academic hospital medicine is a fast‐growing specialty and has a strong emphasis on high‐value care, efficiency, and quality improvement (QI).[1] Developing scholarly work in these areas and describing findings in peer‐reviewed publications can help disseminate ideas and innovations more widely. In addition, success in academic medicine, at least in part, continues to be measured by traditional academic benchmarks, including the production of scholarly publications, conference presentations, and abstracts.[2]

Hospital medicine, however, faces challenges in providing an academic environment conducive to fostering scholarly work. As a relatively young specialty, there may be a dearth of senior mentors and experienced researchers; lack of structured mentorship can be associated with failure to produce publications or lead national teaching sessions.[3] Relatively few hospitalists undergo fellowships or other specialized training that provides a clinical research background, and internal medicine residency programs rarely provide the comprehensive research skill set required to design, implement, or disseminate academic work.[4, 5, 6] Finally, heavy clinical responsibilities may hinder efforts to conduct and sustain research.

A works‐in‐progress (WIP) session, commonly employed in clinical research groups, can provide a forum to discuss and receive feedback on evolving projects and can foster mentorship, motivation, and training.[7] Although a WIP session may stimulate discussion and advance project ideas, academic hospitalist groups do not commonly employ this model, and it is not known if a regularly scheduled WIP session can provide the mentorship, training, and motivation necessary to assist junior faculty in advancing scholarly project to completion.[8] In this article, we describe how we developed a regular WIP series to promote scholarship activities within our rapidly growing, primarily clinically focused Division of Hospital Medicine (DHM) at the University of California, San Francisco (UCSF), and the results of a survey of WIP participants. We hope that our experience can help illustrate key features of such a model, as well as describe inherent challenges and lessons learned to help promote successful academic efforts at other institutions.

METHODS

Program Setting

During years 2010 to 2013, the time period captured by our survey, the DHM at UCSF grew from 37 to 46 full‐time hospitalists, with 76% primarily clinical faculty (nonresearchers) and 24% primarily clinician‐investigators (researchers), defined as individuals having completed a 2‐year clinical research fellowship and/or dedicating 70% time in their faculty position to clinical research. In addition, there were between 1 and 3 hospitalist fellows per year. In 2012, a PhD researcher joined the division to support research and academic activities within the division as well as to pursue an independent research career.

Program Description

The DHM WIP, named the Incubator, was initially developed in 2007 when researchers recognized the need and desire for a forum where scholarly projects could be reviewed and evaluated. In the first year, the Incubator was primarily utilized by junior research‐trained mentees applying for National Institutes of Health career development awards. However, it soon became clear that nonresearch trained junior fellow and faculty members were pursuing scholarly projects needing additional guidance and input. In particular, the Incubator became frequently utilized by academic hospital medicine fellows and resident trainees pursuing QI and education projects. Over time, more DHM faculty, and junior faculty in particular, began to present their projects and receive structured feedback from researchers as well as other senior members of the group.

Incubator is structured as a 50‐minute session held from 1:10 to 2:00 pm on Thursdays in a DHM conference room. The time was selected because it did not conflict with other divisional conferences and to reserve mornings for clinical responsibilities. Incubator is held on most weeks of the year except for holidays or when there is no scheduled presenter. Presenting at Incubator is voluntary, and presenters sign up for open spots in advance with the upcoming presenter schedule sent out to the division in advance of the conference. Incubator is also used as a forum to provide feedback on anticipated abstract submissions for professional society meetings. For the purposes of the survey described in this article, we did not include Incubator sessions on reviewing abstracts/posters. Trainees and hospitalists present a broad range of projects at any stage of preparation. These include project ideas, grant applications, manuscripts, abstracts, and oral presentations at any stage of completion for feedback. Our mission was to create a forum where researchers, clinicians, and educators meet to provide the tools and guidance necessary to promote scholarly projects across the range of the division's activities by connecting individuals with complementary skills and interests and providing necessary mentorship and peer support. We have defined scholarship broadly, including evaluation of QI, global health, or other health system innovations, as well as advancements in medical education and traditional clinical research.

All faculty are invited to Incubator, and attendees include senior and junior faculty, researchers in the division, fellows, and occasionally residents and medical students. One week prior to the session, an administrative assistant solicits project information, including any related materials and questions the presenter may have for the group using a prespecified template, and emails this information to division members for review. In addition, the same materials are also printed prior to Incubator for any attendees who may not have reviewed the material in advance. Also, prior to the session, a physician is specified to serve as moderator of the discussion, and another physician is assigned the role of primary reviewer to provide the initial specific feedback and recommendations. The role of the moderator is to manage the discussion and keep the focus on time, and is assigned to a researcher or senior clinical faculty member. The role of primary reviewer is assigned to provide more junior faculty (both researchers and nonresearchers) the opportunity to practice their editing and critiquing skills by providing the initial feedback. Presenters and moderators receive worksheets outlining the structure of Incubator and their respective roles (see Supporting Information, Appendix 1, in the online version of this article).

Incubator begins with the presenter providing a brief synopsis of their project and their specific goals and objectives for the session. The moderator then leads the discussion and guides the format, often starting with any questions the group may have for the presenter followed by the specific feedback from the primary reviewer. The primary reviewer, having reviewed the materials in advance of the session, answers the prespecified questions as listed by the presenter, occasionally providing additional targeted feedback. The session is then opened to the rest of the group for feedback and suggestions. Meanwhile, the presenter is encouraged to wait until the end of the hour to summarize their take on the feedback and what their initial thoughts on the next to do items would be (Table 1).

Summary of Incubator Roles
PresenterAdministrative assistant
2‐ to 3‐sentence summary of career focusSchedule session and conference room
Distribute short set of materials in advanceCollect presenters' materials in advance
Summarize feedback at end of sessionPrepare materials for Incubator
Brainstorm on next steps at end of sessionMonitor attendance and topics of presentation
Primary reviewerModerator
Junior faculty (24 years)Senior or research faculty
Provide brief overview of projectKeep session on time
Reiterate key questionsGive additional input
Provide 2 major, 3 minor suggestionsSummarize comments from group at the end
Constructive, outside the box feedbackAllow last 10 minutes for presenter to discuss plans

Program Evaluation

Survey Respondents and Process

We retrospectively surveyed the lead presenter for each Incubator session held between May 2010 through November 2013. Surveys were administered through the Research Electronic Data Capture application (REDCap).[9] Participants who were lead presenters at Incubator for more than 1 Incubator session completed a survey for each individual presentation. Therefore, some presenters completed more than 1 survey. The presenters included resident physicians, hospital medicine fellows, junior faculty, and researchers. We defined researchers as hospitalists who had completed a 2‐year research fellowship and/or devoted at least 70% time in their faculty position to research.

Survey Development and Domains

We developed a survey questionnaire using the Kirkpatrick 4‐level model to evaluate the educational experience of the primary presenters and to determine how the session impacted their progress on the project, with each model component graded according to a Likert scale.[10] The 4 major components of the model are: (1) Reaction: participants' estimates of satisfaction with Incubator; (2) Learning: extent of knowledge acquisition achieved at Incubator; (3) Behavior: extent to which learning has been applied or transfer of skills through participation in Incubator; and (4) Results: results of the project, wider changes in organizational scholarship as impacted by Incubator.

We also collected information on the presenter's status at time of presentation including career paths (researcher or nonresearcher), their job description (faculty, fellow, resident), and the total number of years on faculty (if applicable). Hospitalists in their first 2 years on the faculty were considered junior physicians. We also collected information on the number of times they had presented at the Incubator sessions and stage of progress of the project, whether in the early, mid, or late phase at the time of presentation. Early phase was defined as presenting an initial project idea or brainstorming possible project options and/or directions. Mid phase was defined as presenting initial results, data, and initial drafts prior to completion of analysis. Late phase was defined as presenting a project nearing completion such as a written abstract, oral presentation, paper, or grant application. Respondents were also asked to identify the main focus of their projects, selecting the categories based on the interests of the division, including medical education, clinical research, QI, high‐value care, and global health.

Survey Data Analysis

We converted Likert scale data into dichotomous variables, with paring of positive responses versus the negative options. We summarized survey responses using descriptive statistics and determined if there were any differences in responses between career researchers and nonresearchers using 2 tests. All analysis was performed using StataSE version 13.1 (StataCorp, College Station, TX).

RESULTS

Survey Respondent Characteristics

We received 51 completed surveys from presenters at an Incubator session, for a total survey response rate of 70%. Of the 51 presentations, 26 (51%) of the projects were led by physicians in training or junior faculty, and 35 (69%) of the presenters were nonresearchers.

Project Characteristics

The most frequently presented topic areas were QI (N = 20), clinical research (N = 14), medical education (N = 6), and global health (N = 6). Whereas researchers were more likely to present clinical research topics and grant applications, nonresearchers more often presented on QI or medical education projects (Table 2). Projects were presented at all stages of development, with the middle stage, where presenters presented initial results, being the most common phase.

Characteristics of Work‐in‐Progress Session Presentations Among 51 Nonresearchers and Researchers
 AllNonresearcher, No. (%)Researcher, No. (%)P Value
  • NOTE: *Early stage was defined as presenting an initial project idea or brainstorming possible project options and/or directions. Mid stage was defined as presenting initial results, data, and initial drafts prior to completion of analysis. Late stage was defined as presenting a project nearing completion such as a written abstract, oral presentation, paper, or grant application.

Total513516 
Trainee or junior faculty 19 (54%)7 (44%)0.49
Topic of project   0.02
Quality improvement20 (39%)15 (43%)5 (31%) 
Clinical research14 (27%)8 (23%)6 (38%) 
Medical education6 (12%)5 (14%)1 (6%) 
Health technology4 (8%)0 (0%)4 (25%) 
High‐value care1 (2%)1 (3%)0 (0%) 
Global health6 (12%)6 (12%)0 (0%) 
Stage of project   0.31
Early*12 (23%)7 (20%)5 (31%) 
Middle24 (47%)19 (54%)5 (31%) 
Late15 (29%)9 (26%)6 (38%) 

Impact of Incubator

The reaction to the session was very positive, with 100% of respondents recommending Incubator to others (Table 3), and 35% reported learning as a result of the session. Twenty‐three (45%) of respondents reported that the session helped reframe the project idea and changed the study design, and 20 (39%) reported improved written or oral presentation style. A majority (45, 88%) reported that Incubator was valuable in advancing the project to completion.

Survey Responses of 51 WIP Presenters According to the Kirkpatrick Evaluation Hierarchy
 AllNonresearcher, No. (%)Researcher, No. (%)P Value
  • NOTE: Abbreviations: WIP, works‐in‐progress.

Trainee or junior faculty5135 (69%)16 (31%)0.49
Reaction    
Satisfied with their WIP session50 (98%)35 (100%)15 (94%)0.25
Would recommend WIP to others51 (100%)35 (100%)16 (100%)1.00
Any of the above 35 (100%)16 (100%)1.00
Learning    
Advanced research methodology18 (35%)12 (34%)6 (38%)0.82
Advanced knowledge in the area9 (18%)5 (14%)4 (25%)0.35
Any of the above 14 (40%)9 (56%)0.28
Behavior    
Current project    
Reframed project idea23 (45%)15 (43%)8 (50%)0.63
Changed study design or methodology23 (45%)16 (46%)7 (44%)0.9
Improved written or oral presentation style20 (39%)15 (43%)5 (31%)0.43
Future projects    
Changed approach to future projects19 (37%)17 (49%)2 (13%)0.01
Any of the above 34 (97%)14 (88%)0.17
Results    
Valuable in advancing project to completion45 (88%)31 (89%)14 (88%)0.18
Provided mentoring and peer support29 (57%)24 (69%)5 (31%)0.01
Connected individuals with similar results13 (13%)9 (26%)4 (25%)0.96
Any of the above 34 (97%)14 (88%)0.17

Survey results of researchers compared to nonresearchers were similar overall, although nonresearchers were more likely to report changes in behavior and in improved mentoring as a result of presenting at Incubator. Notably, 17 (49%) of nonresearchers reported that Incubator changed their approach to future projects as opposed to only 2 (13%) researchers (P = 0.01). In addition, 24 (69%) nonresearchers reported value in mentorship and peer support compared to 5 (31%) researchers (P = 0.01). A reasonably large proportion of projects originally presented during the Incubator sessions became published articles at the time of survey completion (N = 19, 37%) or were publications in progress (N = 14, 27%). For all remaining items, there were no statistically significant differences in the survey responses among junior faculty/trainees (N = 26) compared to nonjunior faculty (N = 25) presenters (P > 0.05).

Attendance at Incubator During the Study Period

Attendance at Incubator was open and voluntary for all DHM faculty, fellows, and collaborating UCSF trainees. From July 2012, when we began tracking attendance, through the end of the survey period in November 2013, the average number of attendees for each session was 10.7 (standard deviation [SD] 3.8). On average, 50% (SD 16%) of attendees at Incubator were career researchers.

DISCUSSION

The results of this program evaluation suggest that a WIP session employed by an academic division of hospital medicine, consisting of a weekly moderated session, can help advance scholarly work. Our evaluation found that presenters, both researchers and nonresearchers, favorably viewed the regular WIP sessions and reported that feedback in the Incubator helped them advance their project to completion. Importantly, nonresearch‐focused faculty and fellows reported the biggest gains in learning from presenting at Incubator. Whereas half the Incubator attendees were career researchers, consistent with the observation that researchers within the division were most committed to attending Incubator regularly, 69% of the presenters were nonresearchers, demonstrating strong participation among both researchers and nonresearchers within the division.

WIP sessions, though informal, are interactive, inspire critical self‐reflection, and encourage physicians to act on generated ideas, as evidenced by the change in behavior of the participants after the session. These sessions allow for transformative learning by encouraging physicians to be open to alternative viewpoints and engage in discourse, boosting learning beyond just content knowledge. Prior assessments of WIP seminars similarly found high satisfaction with these formats.[11]

Although we cannot identify specifically which features made Incubator effective, we believe that our WIP had some characteristics that may have contributed to its success and may aid in implementation at other institutions: holding the session regularly, voluntary participation, distributing the materials and questions for the group in advance, and designating a moderator for the session in advance to facilitate discussion.

A potential strength of the Incubator is that both researchers and nonresearchers attend. We hypothesize that combining these groups provides improved mentorship and learning for nonresearchers, in particular. In addition, it creates a mutually beneficial environment where each group is able to witness the diversity of projects within the division and learn to provide focused, constructive feedback on the presented work. Not only did this create a transparent environment with better understanding of divisional activities, but also fostered collaboration among hospitalists with similar interests and complementary skills.

Challenges, Setbacks, Updated Approaches

The creation of a successful Incubator session, however, was not without its challenges. At initial inception, the WIP was attended primarily by researchers and had low overall attendance. Members of the division who were primarily clinicians initially perceived the conferences as largely inapplicable to their career objectives and had competing demands from patient care, educational, or administrative responsibilities. However, over time and with encouragement from divisional leaders and service line directors, increasing numbers of hospitalists began to participate in Incubator. The timing of Incubator during afternoons after the Department of Medicine Grand Rounds was chosen specifically to allow clinicians to complete their responsibilities, including morning rounds and teaching, to allow better attendance.

In addition, the results of our survey informed changes to the structure of Incubator. The efficacy of assigning a primary reviewer for each session was not clear, so this component was eventually dropped. The finding that nonresearchers in particular reported a benefit from mentoring and peer‐support at Incubator led to the implementation of querying the presenter for a wish list of faculty attendees at their Incubator session. We then sent a special invitation to those faculty members thought to have special insights on the project. This gave junior faculty the opportunity to present their projects to more senior faculty members within their areas of research, as well as to receive focused expert feedback.

Finally, we have initiated special Incubator sessions focused more on didactics to teach the process of writing manuscripts and brainstorming workshop ideas for national meetings.

Limitations

Our study has limitations. It is a single‐center study based on a small overall sample size, and it is not certain whether a similar innovation would have comparable effects at another institution. In addition, generalizability of our results may be limited for hospital medicine groups without a robust research program. We did not have a control group nor do we know whether participants would have been equally successful without Incubator. We also were unable to assess how Incubator affected long‐term outcomes such as promotion and overall publication record, as we do not have detailed data on productivity prior to the survey period. Finally, we are unable to quantify the effect of Incubator on scholarly success in the division. Although the numbers of published articles and grant funding has increased since the Incubator began (data not shown), the division also grew both in number of research‐focused and nonresearch‐focused faculty, and this study does not account for other temporal changes that may have contributed to improvements in the scholarly output of the division.

CONCLUSIONS

In summary, the Incubator has been a successful program that fostered progress on scholarly projects within a largely clinically focused DHM. Given the importance of scholarship in academic hospital medicine, a WIP session such as the one we describe is a valuable way to support and mentor junior hospitalists and nonresearchers.

Acknowledgements

The authors extend special thanks to Oralia Schatzman, divisional administrative assistant, who organized and arranged the Incubator sessions and recorded attendance, and to Katherine Li, who collected data on numbers of faculty within the division over the duration of the study.

Disclosures: Dr. Hemali Patel, Dr. Margaret Fang, and Mr. James Harrison report no conflicts of interest. At the time the research was conducted, Dr. Kangelaris was supported by the National Heart, Lung, and Blood Institute (1K23HL116800‐01). Dr. Auerbach was supported by the National Heart, Lung, and Blood Institute (K24 K24HL098372).

References
  1. Sehgal NL, Sharpe BA, Auerbach AA, Wachter RM. Investing in the future: building an academic hospitalist faculty development program. J Hosp Med. 2011;6(3):161166.
  2. Leykum LK, Parekh VI, Sharpe B, Boonyasai RT, Centor RM. Tried and true: a survey of successfully promoted academic hospitalists. J Hosp Med. 2011;6(7):411415.
  3. Reid MB, Misky GJ, Harrison RA, Sharpe B, Auerbach A, Glasheen JJ. Mentorship, productivity, and promotion among academic hospitalists. J Gen Intern Med. 2012;27(1):2327.
  4. Kohlwes R, Shunk R, Avins A, Garber J, Bent S, Shlipak M. The PRIME curriculum. J Gen Intern Med. 2006;21(5):506509.
  5. Ranji SR, Rosenman DJ, Amin AN, Kripalani S. Hospital medicine fellowships: works in progress. Am J Med. 2006;119(1):72.e1–7.
  6. Wilper AP, Smith CS, Weppner W. Instituting systems‐based practice and practice‐based learning and improvement: a curriculum of inquiry. Med Educ Online. 2013;18:21612.
  7. Grzybowski SCW, Bates J, Calam B, et al. A physician peer support writing group. Fam Med. 2003;35(3):195201.
  8. Abougergi MS, Wright SM, Landis R, Howell EE. Research in progress conference for hospitalists provides valuable peer mentoring. J Hosp Med. 2011;6(1):4346.
  9. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata‐driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377381.
  10. Kirkpatrick DL, Kirkpatrick JD. Evaluating Training Programs: The Four Levels. 3rd ed. San Francisco, CA: Berrett‐Koehler; 2006.
  11. Chang S, Hughes DC, Chamberlain RM. Works‐in‐progress: guiding junior scientists through career development applications. J Cancer Educ. 2008;23(3):142148.
References
  1. Sehgal NL, Sharpe BA, Auerbach AA, Wachter RM. Investing in the future: building an academic hospitalist faculty development program. J Hosp Med. 2011;6(3):161166.
  2. Leykum LK, Parekh VI, Sharpe B, Boonyasai RT, Centor RM. Tried and true: a survey of successfully promoted academic hospitalists. J Hosp Med. 2011;6(7):411415.
  3. Reid MB, Misky GJ, Harrison RA, Sharpe B, Auerbach A, Glasheen JJ. Mentorship, productivity, and promotion among academic hospitalists. J Gen Intern Med. 2012;27(1):2327.
  4. Kohlwes R, Shunk R, Avins A, Garber J, Bent S, Shlipak M. The PRIME curriculum. J Gen Intern Med. 2006;21(5):506509.
  5. Ranji SR, Rosenman DJ, Amin AN, Kripalani S. Hospital medicine fellowships: works in progress. Am J Med. 2006;119(1):72.e1–7.
  6. Wilper AP, Smith CS, Weppner W. Instituting systems‐based practice and practice‐based learning and improvement: a curriculum of inquiry. Med Educ Online. 2013;18:21612.
  7. Grzybowski SCW, Bates J, Calam B, et al. A physician peer support writing group. Fam Med. 2003;35(3):195201.
  8. Abougergi MS, Wright SM, Landis R, Howell EE. Research in progress conference for hospitalists provides valuable peer mentoring. J Hosp Med. 2011;6(1):4346.
  9. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata‐driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377381.
  10. Kirkpatrick DL, Kirkpatrick JD. Evaluating Training Programs: The Four Levels. 3rd ed. San Francisco, CA: Berrett‐Koehler; 2006.
  11. Chang S, Hughes DC, Chamberlain RM. Works‐in‐progress: guiding junior scientists through career development applications. J Cancer Educ. 2008;23(3):142148.
Issue
Journal of Hospital Medicine - 11(10)
Issue
Journal of Hospital Medicine - 11(10)
Page Number
719-723
Page Number
719-723
Publications
Publications
Article Type
Display Headline
Implementation and evaluation of a “works‐in‐progress” session to promote scholarship in an academic hospitalist group
Display Headline
Implementation and evaluation of a “works‐in‐progress” session to promote scholarship in an academic hospitalist group
Sections
Article Source

© 2016 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Hemali Patel, MD, 12401 E 17th Avenue, Suite 450B, Mail Stop F‐782, Aurora, CO 80045; Telephone: 720‐848‐4289; Fax: 720‐848‐4293; E‐mail: hemali.patel@ucdenver.edu
Content Gating
Gated (full article locked unless allowed per User)
Alternative CME
Gating Strategy
First Peek Free
Article PDF Media
Media Files

Effect of an RRT on Resident Perceptions

Article Type
Changed
Sun, 05/21/2017 - 13:32
Display Headline
The effect of a rapid response team on resident perceptions of education and autonomy

Rapid response teams (RRTs) have been promoted by patient safety and quality‐improvement organizations as a strategy to reduce preventable in‐hospital deaths.[1] To date, critical analysis of RRTs has focused primarily on their impact on quality‐of‐care metrics.[2, 3, 4] Comparatively few studies have examined the cultural and educational impact of RRTs, particularly at academic medical centers, and those that do exist have focused almost exclusively on perceptions of nurses rather than resident physicians.[5, 6, 7, 8, 9, 10]

Although a prior study found that internal medicine and general surgery residents believed that RRTs improved patient safety, they were largely ambivalent about the RRT's impact on education and training.[11] To date, there has been no focused assessment of resident physician impressions of an RRT across years of training and medical specialty to inform the use of this multidisciplinary team as a component of their residency education.

We sought to determine whether resident physicians at a tertiary care academic medical center perceive educational benefit from collaboration with the RRT and whether they feel that the RRT adversely affects clinical autonomy.

METHODS

The Hospital

Moffitt‐Long Hospital, the tertiary academic medical center of the University of California, San Francisco (UCSF), is a 600‐bed acute care hospital that provides comprehensive critical care services and serves as a major referral center in northern California. There are roughly 5000 admissions to the hospital annually. At the time the study was conducted, there were approximately 200 RRT calls per 1000 adult hospital discharges.

The Rapid Response Team

The RRT is called to assess, triage, and treat patients who have experienced a decline in their clinical status short of a cardiopulmonary arrest. The RRT has been operational at UCSF since June 1, 2007, and is composed of a dedicated critical care nurse and respiratory therapist available 24 hours a day, 7 days a week. The RRT can be activated by any concerned staff member based on vital sign abnormalities, decreased urine output, changes in mental status, or any significant concern about the trajectory of the patient's clinical course.

When the RRT is called on a given patient, the patient's primary physician (at our institution, a resident) is also called to the bedside and works alongside the RRT to address the patient's acute clinical needs. The primary physician, bedside nurse, and RRT discuss the plan of care for the patient, including clinical evaluation, management, and the need for additional monitoring or a transition to a higher level of care. Residents at our institution receive no formal instruction regarding the role of the RRT or curriculum on interfacing with the RRT, and they do not serve as members of the RRT as part of a clinical rotation.

The Survey Process

Study subjects were asked via e‐mail to participate in a brief online survey. Subjects were offered the opportunity to win a $100 gift certificate in return for their participation. Weekly e‐mail reminders were sent for a period of 3 months or until a given subject had completed the survey. The survey was administered over a 3‐month period, from March through May, to allow time for residents to work with the RRT during the academic year. The Committee on Human Research at the University of California San Francisco Medical Center approved the study.

Target Population

All residents in specialties that involved direct patient care and the potential to use the adult RRT were included in the study. This included residents in the fields of internal medicine, neurology, general surgery, orthopedic surgery, neurosurgery, plastic surgery, urology, and otolaryngology (Table 1). Residents in pediatrics and obstetrics and gynecology were excluded, as emergencies in their patients are addressed by a pediatric RRT and an obstetric anesthesiologist, respectively. Residents in anesthesiology were excluded as they do not care for nonintensive care unit (ICU) patients as part of the primary team and are not involved in RRT encounters.

Demographics of Survey Respondents (N=236)
DemographicNo. (%)
  • NOTE: Abbreviations: RRT, rapid response team; SD, standard deviation.

  • Where data do not equal 100%, this is due to missing data or rounding. Table does not include 10 respondents who had never cared for a patient for whom the RRT was activated.

Medical specialty 
Internal medicine145 (61.4)
Neurology18 (7.6)
General surgery31 (13.1)
Orthopedic surgery17 (7.2)
Neurosurgery4 (1.7)
Plastic surgery2 (0.8)
Urology9 (3.8)
Otolaryngology10 (4.2)
Years of postgraduate trainingAverage 2.34 (SD 1.41)
183 (35.2)
260 (25.4)
355 (23.3)
420 (8.5)
58 (3.4)
65 (2.1)
75 (2.1)
Gender 
Male133 (56.4)
Female102 (43.2)
Had exposure to RRT during training 
Yes106 (44.9)
No127 (53.8)
Had previously initiated a call to the RRT 
Yes106 (44.9)
No128 (54.2)

Survey Design

The resident survey contained 20 RRT‐related items and 7 demographic and practice items. Responses for RRT‐related questions utilized a 5‐point Likert scale ranging from strongly disagree to strongly agree. The survey was piloted prior to administration to check comprehension and interpretation by physicians with experience in survey writing (for the full survey, see Supporting Information, Appendix, in the online version of this article).

Survey Objectives

The survey was designed to capture the experiences of residents who had cared for a patient for whom the RRT had been activated. Data collected included residents' perceptions of the impact of the RRT on their residency education and clinical autonomy, the quality of care provided, patient safety, and hospital‐wide culture. Potential barriers to use of the RRT were also examined.

Outcomes

The study's primary outcomes included the perceived educational benefit of the RRT and its perceived impact on clinical autonomy. Secondary outcomes included the effect of years of training and resident specialty on both the perceived educational benefit and impact on clinical autonomy among our study group.

Statistical Analysis

Responses to each survey item were described for each specialty, and subgroup analysis was conducted. For years of training, that item was dichotomized into either 1 year (henceforth referred to as interns) or greater than 1 year (henceforth referred to as upper‐level residents). Resident specialty was dichotomized into medical fields (internal medicine and neurology) or surgical fields. For statistical analysis, agreement statements were collapsed to either disagree (strongly disagree/disagree), neutral, or agree (strongly agree/agree). The influence of years of resident training and resident specialty was assessed for all items in the survey using 2 or Fisher exact tests as appropriate for the 3 agreement categories. Analysis was conducted using SPSS 21.0 (IBM Corp., Armonk, NY).

RESULTS

There were 246 responses to the survey of a possible 342, yielding a response rate of 72% (Table 2). Ten respondents stated that they had never cared for a patient where the RRT had been activated. Given their lack of exposure to the RRT, these respondents were excluded from the analysis, yielding a final sample size of 236. The demographic and clinical practice characteristics of respondents are shown in Table 1.

Resident Perceptions of the RRT (N=236)
The residentStrongly Disagree/Disagree, n (%)Neutral, n (%)Agree/ Strongly Agree, n (%)
  • NOTE: Abbreviations: RRT, rapid response team.

  • Where data do not equal 100%, this is due to missing data or rounding. Includes only data for respondents who had cared for a patient that required RRT activation.

Is comfortable managing the unstable patient without the RRT104 (44.1)64 (27.1)66 (28.0)
And RRT work together to make treatment decisions10 (4.2)13 (5.5)208 (88.1)
Believes there are fewer opportunities to care for unstable floor patients due to the RRT188 (79.7)26 (11.0)17 (7.2)
Feels less prepared to care for unstable patients due to the RRT201 (85.2)22 (9.3)13 (5.5)
Feels that working with the RRT creates a valuable educational experience9 (3.8)39 (16.5)184 (78.0)
Feels that nurses caring for the unstable patient should always contact them prior to contacting the RRT123 (52.1)33 (14.0)76 (32.2)
Would be unhappy with nurses calling RRT prior to contacting them141 (59.7)44 (18.6)51 (21.6)
Perceives that the presence of RRT decreases residents' autonomy179 (75.8)25 (10.6)28 (11.9)

Demographics and Primary Outcomes

Interns comprised 83 (35%) of the respondents; the average time in postgraduate training was 2.34 years (standard deviation=1.41). Of respondents, 163 (69%) were in medical fields, and 73 (31%) were in surgical fields. Overall responses to the survey are shown in Table 2, and subgroup analysis is shown in Table 3.

Perceptions of the RRT Based on Years of Training and Specialty
The resident1 Year, n=83, n (%)>1 Year, n=153, n (%)P ValueMedical, n=163, n (%)Surgical, n=73, n (%)P Value
  • NOTE: Abbreviations: RRT, rapid response team.

  • Where data do not equal 100%, this is due to missing data or rounding.

Is comfortable managing the unstable patient without the RRT  0.01  <0.01
Strongly disagree/disagree39 (47.6)65 (42.8) 67 (41.6)37 (50.7) 
Neutral29 (35.4)35 (23.0) 56 (34.8)8 (11.0) 
Agree/strongly agree14 (17.1)52 (34.2) 38 (23.6)28 (38.4) 
And RRT work together to make treatment decisions  0.61  0.04
Strongly disagree/disagree2 (2.4)8 (5.4) 4 (2.5)6 (8.7) 
Neutral5 (6.1)8 (5.4) 7 (4.3)6 (8.7) 
Agree/strongly agree75 (91.5)137 (89.3) 151 (93.2)57 (82.6) 
Believes there are fewer opportunities to care for unstable floor patients due to the RRT  0.05  0.04
Strongly disagree/disagree59 (72.8)129 (86.0) 136 (85.5)52 (72.2) 
Neutral13 (16.0)13 (8.7) 15 (9.4)11 (15.3) 
Agree/strongly agree9 (11.1)8 (5.3) 8 (5.0)9 (12.5) 
Feels less prepared to care for unstable patients due to the RRT  <0.01  0.79
Strongly disagree/disagree62 (74.7)139 (90.8) 140 (85.9)61 (83.6) 
Neutral14 (16.9)8 (5.2) 15 (9.2)7 (9.6) 
Agree/Strongly agree7 (8.4)6 (3.9) 8 (4.9)5 (6.8) 
Feels working with the RRT is a valuable educational experience  0.61  0.01
Strongly disagree/disagree2 (2.4)7 (4.7) 2 (1.2)7 (9.9) 
Neutral12 (14.6)27 (18.0) 25 (15.5)14 (19.7) 
Agree/strongly agree68 (82.9)116 (77.3) 134 (83.2)50 (70.4) 
Feels nurses caring for unstable patients should always contact the resident prior to contacting the RRT  0.49  <0.01
Strongly disagree/disagree47 (57.3)76 (50.7) 97 (60.2)26 (36.6) 
Neutral9 (11.0)24 (16.0) 26 (16.1)7 (9.9) 
Agree/strongly agree26 (31.7)50 (33.3) 38 (23.6)38 (53.5) 
Would be unhappy with nurses calling RRT prior to contacting them  0.81  <0.01
Strongly disagree/disagree51 (61.4)90 (58.8) 109 (66.9)32 (43.8) 
Neutral16 (19.3)28 (18.3) 30 (18.4)14 (19.2) 
Agree/strongly agree16 (19.3)35 (22.9) 24 (14.7)27 (37.0) 
Perceives that the presence of the RRT decreases autonomy as a physician  0.95  0.18
Strongly disagree/disagree63 (77.8)116 (76.8) 127 (79.9)52 (71.2) 
Neutral9 (11.1)16 (10.6) 17 (10.7)8 (11.0) 
Agree/strongly agree9 (11.1)19 (12.6) 15 (9.4)13 (17.8) 

Effect of the RRT on Resident Education

Of all residents, 66 (28%) agreed that they felt comfortable managing an unstable patient without the assistance of the RRT. Surgical residents felt more comfortable managing an unstable patient alone (38%) compared medical residents (24%) (P<0.01). Interns felt less comfortable caring for unstable patients without the RRT's assistance (17%) compared with upper‐level residents (34%) (P=0.01).

Residents overall disagreed with the statement that the RRT left them feeling less prepared to care for unstable patients (n=201; 85%). More upper‐level residents disagreed with this assertion (91%) compared with interns (75%) (P<0.01). Responses to this question did not differ significantly between medical and surgical residents.

Upper‐level residents were more likely to disagree with the statement that the RRT resulted in fewer opportunities to care for unstable patients (n=129; 86%) compared with interns (n=59; 73%) (P=0.05). Medical residents were also more likely to disagree with this statement (n=136; 86%) compared with surgical residents (n=52; 72%) (P=0.04).

With respect to residents' overall impressions of the educational value of the RRT, 68 (83%) interns and 116 (77%) upper‐level residents agreed that it provided a valuable educational experience (P=0.61). Medical and surgical residents differed in this regard, with 134 (83%) medical residents and 50 (70%) surgical residents agreeing that the RRT provided a valuable educational experience (P=0.01).

Effect of the RRT on Clinical Autonomy

Of all residents, 123 (52%) disagreed that the bedside nurse should always contact the primary resident prior to calling the RRT; 76 (32%) agreed with this statement. Medicine residents were more likely to disagree with this approach (n=97; 60%) than were surgical residents (n=26; 36%) (P<0.01). There was no difference between interns and upper‐level residents in response to this question. Most of those who disagreed with this statement were medical residents, whereas most surgical residents (n=38; 54%) agreed that they should be contacted first (P<0.01).

There were no differences between interns and upper‐level residents with respect to perceptions of the RRT's impact on clinical autonomy: 11% of interns and 13% of residents agreed that the RRT decreased their clinical autonomy as a physician. There was no significant difference between medical and surgical residents' responses to this question.

The majority of residents (n=208; 88%) agreed that they and the RRT work together to make treatment decisions for patients. This was true regardless of year of training (P=0.61), but it was expressed more often among medical residents than surgical residents (n=151, 93% vs n=57, 83%; P=0.04).

DISCUSSION

Most studies examining the educational and cultural impact of RRTs exist in the nursing literature. These studies demonstrate that medical and surgical nurses are often reluctant to call the RRT for fear of criticism by the patient's physician.[5, 8, 9, 10, 11, 12, 13] In contrast, our data demonstrate that resident physicians across all levels of training and specialties have a positive view of the RRT and its role in patient care. The data support our hypothesis that although most residents perceive educational benefit from their interactions with the RRT, this perception is greater for less‐experienced residents and for those residents who routinely provide care for critically ill patients and serve as code team leaders. In addition, a minority of residents, irrespective of years of training or medical specialty, felt that the RRT negatively impacted their clinical autonomy.

Our data have several important implications. First, although over half of the residents surveyed had not been exposed to RRTs during medical school, and despite having no formal training on the role of the RRT during residency, most residents identified their interactions with the RRT as potential learning opportunities. This finding differs from that of Benin and colleagues, who suggested that RRTs might negatively impact residents' educational development and decrease opportunities for high‐stakes clinical reasoning by allowing the clinical decision‐making process to be driven by the RRT staff rather than the resident.[5] One possible explanation for this discrepancy is the variable makeup of the RRT at different institutions. At our medical center, the RRT is comprised of a critical care nurse and respiratory therapist, whereas at other institutions, the RRT may be led by a resident, fellow, attending hospitalist, or intensivist, any of whom might supersede the primary resident once the RRT is engaged.

In our study, the perceived educational benefit of the RRT was most pronounced with interns. Interns likely derive incrementally greater benefit from each encounter with an acutely decompensating patient than do senior residents, whether the RRT is present or not. Observing the actions of seasoned nurses and respiratory therapists may demonstrate new tools for interns to use in their management of such situations; for example, the RRT may suggest different modes of oxygen delivery or new diagnostic tests. The RRT also likely helps interns navigate the hospital system by assisting with decisions around escalation of care and serving as a liaison to ICU staff.

Our data also have implications for resident perceptions of clinical autonomy. Interns, far less experienced caring for unstable patients than upper‐level residents, expressed more concern about the RRT stripping them of opportunities to do so and about feeling less prepared to handle clinically deteriorating patients. Part of this perception may be due to interns feeling less comfortable taking charge of a patient's care in the presence of an experienced critical care nurse and respiratory therapist, both for reasons related to clinical experience and to a cultural hierarchy that often places the intern at the bottom of the authority spectrum. In addition, when the RRT is called on an intern's patient, the senior resident may accompany the intern to the bedside and guide the intern on his or her approach to the situation; in some cases, the senior resident may take charge, leaving the intern feeling less autonomous.

If training sessions could be developed to address not only clinical decision making, but also multidisciplinary team interactions and roles in the acute care setting, this may mitigate interns' concerns. Such curricula could also enhance residents' experience in interprofessional care, an aspect of clinical training that has become increasingly important in the age of limited duty hours and higher volume, and higher acuity inpatient censuses. An RRT model, like a code blue model, could be used in simulation‐based training to increase both comfort with use of the RRT and efficiency of the RRTresidentnurse team. Although our study did not address specifically residents' perceptions of multidisciplinary teams, this could be a promising area for further study.

For surgical residents, additional factors are likely at play. Surgical residents spend significant time in the operating room, reducing time present at the bedside and hindering the ability to respond swiftly when an RRT is called on their patient. This could cause surgical residents to feel less involved in the care of that patientsupported by our finding that fewer surgical residents felt able to collaborate with the RRTand also to derive less educational benefit and clinical satisfaction from the experience. Differences between medical and surgical postgraduate training also likely play a role, manifest by varying clinical roles and duration of training, and as such it may not be appropriate to draw direct comparisons between respective postgraduate year levels. In addition, differences in patients' medical complexity, varying allegiance to the traditional hierarchy of medical providers, and degree of familiarity with the RRT itself may impact surgical residents' comfort with the RRT.

Limitations of our study include that it was conducted at a single site and addressed a specific population of residents at our tertiary academic center. Though we achieved an excellent response rate, our subspecialty sample sizes were too small to allow for individual comparisons among those groups. Conducting a larger study at multiple institutions where the makeup of the RRT differs could provide further insight into how different clinical environments and different RRT models impact resident perceptions. Finally, we allowed each respondent to interpret both educational benefit and clinical autonomy in the context of their own level of training and clinical practice rather than providing strict definitions of these terms. There is no standardized definition of autonomy in the context of resident clinical practice, and we did not measure direct educational outcomes. Our study design therefore allowed only for measurement of perceptions of these concepts. Measurement of actual educational value of the RRTfor example, through direct clinical observation or by incorporating the RRT experience into an entrustable professional activitywould provide more quantitative evidence of the RRT's utility for our resident population. Future study in this area would help to support the development and ongoing assessment of RRT‐based curricula moving forward.

CONCLUSION

Our data show that resident physicians have a strongly favorable opinion of the RRT at our institution. Future studies should aim to quantify the educational benefit of RRTs for residents and identify areas for curricular development to enhance resident education as RRTs become more pervasive.

Files
References
  1. Institute for Healthcare Improvement. Rapid response teams. Available at: http://www.ihi.org/topics/rapidresponseteams. Accessed May 5, 2014.
  2. Chan PS, Jain R, Nallmothu BK, Berg RA, Sasson C. Rapid response teams: a systematic review and meta‐analysis. Arch Intern Med. 2010;170(1):1826.
  3. Devita MA, Bellomo R, Hillman K, et al. Findings of the first consensus conference on medical emergency teams. Crit Care Med. 2006;34(9):24632478.
  4. Winters BD, Pham JC, Hunt EA, Guallar E, Berenholtz S, Pronovost PJ. Rapid response systems: a systematic review. Crit Care Med. 2007;35(5):12381243.
  5. Benin AL, Borgstrom CP, Jenq GY, Roumanis SA, Horwitz LI. Defining impact of a rapid response team: qualitative study with nurses, physicians and hospital administrators. BMJ Qual Saf. 2012;21(5):391398.
  6. Leach LS, Mayo A, O'Rourke M. How RNs rescue patients: a qualitative study of RNs' perceived involvement in rapid response teams. Qual Saf Health Care. 2010;19(5):e13.
  7. Metcalf R, Scott S, Ridgway M, Gibson D. Rapid response team approach to staff satisfaction. Orthop Nurs. 2008;27(5):266271; quiz 272–273.
  8. Salamonson Y, Heere B, Everett B, Davidson P. Voices from the floor: nurses' perceptions of the medical emergency team. Intensive Crit Care Nurs. 2006;22(3):138143.
  9. Shapiro SE, Donaldson NE, Scott MB. Rapid response teams seen through the eyes of the nurse. Am J Nurs. 2010;110(6):2834; quiz 35–36.
  10. Shearer B, Marshall S, Buist MD, et al. What stops hospital clinical staff from following protocols? An analysis of the incidence and factors behind the failure of bedside clinical staff to activate the rapid response system in a multi‐campus Australian metropolitan healthcare service. BMJ Qual Saf. 2012;21(7):569575.
  11. Sarani B, Sonnad S, Bergey MR, et al. Resident and RN perceptions of the impact of a medical emergency team on education and patient safety in an academic medical center. Crit Care Med. 2009;37(12):30913096.
  12. Marshall SD, Kitto S, Shearer W, et al. Why don't hospital staff activate the rapid response system (RRS)? How frequently is it needed and can the process be improved? Implement Sci. 2011;6:39.
  13. Peebles E, Subbe CP, Hughes P, Gemmell L. Timing and teamwork–an observational pilot study of patients referred to a rapid response team with the aim of identifying factors amenable to re‐design of a rapid response system. Resuscitation. 2012;83(6):782787.
Article PDF
Issue
Journal of Hospital Medicine - 10(1)
Publications
Page Number
8-12
Sections
Files
Files
Article PDF
Article PDF

Rapid response teams (RRTs) have been promoted by patient safety and quality‐improvement organizations as a strategy to reduce preventable in‐hospital deaths.[1] To date, critical analysis of RRTs has focused primarily on their impact on quality‐of‐care metrics.[2, 3, 4] Comparatively few studies have examined the cultural and educational impact of RRTs, particularly at academic medical centers, and those that do exist have focused almost exclusively on perceptions of nurses rather than resident physicians.[5, 6, 7, 8, 9, 10]

Although a prior study found that internal medicine and general surgery residents believed that RRTs improved patient safety, they were largely ambivalent about the RRT's impact on education and training.[11] To date, there has been no focused assessment of resident physician impressions of an RRT across years of training and medical specialty to inform the use of this multidisciplinary team as a component of their residency education.

We sought to determine whether resident physicians at a tertiary care academic medical center perceive educational benefit from collaboration with the RRT and whether they feel that the RRT adversely affects clinical autonomy.

METHODS

The Hospital

Moffitt‐Long Hospital, the tertiary academic medical center of the University of California, San Francisco (UCSF), is a 600‐bed acute care hospital that provides comprehensive critical care services and serves as a major referral center in northern California. There are roughly 5000 admissions to the hospital annually. At the time the study was conducted, there were approximately 200 RRT calls per 1000 adult hospital discharges.

The Rapid Response Team

The RRT is called to assess, triage, and treat patients who have experienced a decline in their clinical status short of a cardiopulmonary arrest. The RRT has been operational at UCSF since June 1, 2007, and is composed of a dedicated critical care nurse and respiratory therapist available 24 hours a day, 7 days a week. The RRT can be activated by any concerned staff member based on vital sign abnormalities, decreased urine output, changes in mental status, or any significant concern about the trajectory of the patient's clinical course.

When the RRT is called on a given patient, the patient's primary physician (at our institution, a resident) is also called to the bedside and works alongside the RRT to address the patient's acute clinical needs. The primary physician, bedside nurse, and RRT discuss the plan of care for the patient, including clinical evaluation, management, and the need for additional monitoring or a transition to a higher level of care. Residents at our institution receive no formal instruction regarding the role of the RRT or curriculum on interfacing with the RRT, and they do not serve as members of the RRT as part of a clinical rotation.

The Survey Process

Study subjects were asked via e‐mail to participate in a brief online survey. Subjects were offered the opportunity to win a $100 gift certificate in return for their participation. Weekly e‐mail reminders were sent for a period of 3 months or until a given subject had completed the survey. The survey was administered over a 3‐month period, from March through May, to allow time for residents to work with the RRT during the academic year. The Committee on Human Research at the University of California San Francisco Medical Center approved the study.

Target Population

All residents in specialties that involved direct patient care and the potential to use the adult RRT were included in the study. This included residents in the fields of internal medicine, neurology, general surgery, orthopedic surgery, neurosurgery, plastic surgery, urology, and otolaryngology (Table 1). Residents in pediatrics and obstetrics and gynecology were excluded, as emergencies in their patients are addressed by a pediatric RRT and an obstetric anesthesiologist, respectively. Residents in anesthesiology were excluded as they do not care for nonintensive care unit (ICU) patients as part of the primary team and are not involved in RRT encounters.

Demographics of Survey Respondents (N=236)
DemographicNo. (%)
  • NOTE: Abbreviations: RRT, rapid response team; SD, standard deviation.

  • Where data do not equal 100%, this is due to missing data or rounding. Table does not include 10 respondents who had never cared for a patient for whom the RRT was activated.

Medical specialty 
Internal medicine145 (61.4)
Neurology18 (7.6)
General surgery31 (13.1)
Orthopedic surgery17 (7.2)
Neurosurgery4 (1.7)
Plastic surgery2 (0.8)
Urology9 (3.8)
Otolaryngology10 (4.2)
Years of postgraduate trainingAverage 2.34 (SD 1.41)
183 (35.2)
260 (25.4)
355 (23.3)
420 (8.5)
58 (3.4)
65 (2.1)
75 (2.1)
Gender 
Male133 (56.4)
Female102 (43.2)
Had exposure to RRT during training 
Yes106 (44.9)
No127 (53.8)
Had previously initiated a call to the RRT 
Yes106 (44.9)
No128 (54.2)

Survey Design

The resident survey contained 20 RRT‐related items and 7 demographic and practice items. Responses for RRT‐related questions utilized a 5‐point Likert scale ranging from strongly disagree to strongly agree. The survey was piloted prior to administration to check comprehension and interpretation by physicians with experience in survey writing (for the full survey, see Supporting Information, Appendix, in the online version of this article).

Survey Objectives

The survey was designed to capture the experiences of residents who had cared for a patient for whom the RRT had been activated. Data collected included residents' perceptions of the impact of the RRT on their residency education and clinical autonomy, the quality of care provided, patient safety, and hospital‐wide culture. Potential barriers to use of the RRT were also examined.

Outcomes

The study's primary outcomes included the perceived educational benefit of the RRT and its perceived impact on clinical autonomy. Secondary outcomes included the effect of years of training and resident specialty on both the perceived educational benefit and impact on clinical autonomy among our study group.

Statistical Analysis

Responses to each survey item were described for each specialty, and subgroup analysis was conducted. For years of training, that item was dichotomized into either 1 year (henceforth referred to as interns) or greater than 1 year (henceforth referred to as upper‐level residents). Resident specialty was dichotomized into medical fields (internal medicine and neurology) or surgical fields. For statistical analysis, agreement statements were collapsed to either disagree (strongly disagree/disagree), neutral, or agree (strongly agree/agree). The influence of years of resident training and resident specialty was assessed for all items in the survey using 2 or Fisher exact tests as appropriate for the 3 agreement categories. Analysis was conducted using SPSS 21.0 (IBM Corp., Armonk, NY).

RESULTS

There were 246 responses to the survey of a possible 342, yielding a response rate of 72% (Table 2). Ten respondents stated that they had never cared for a patient where the RRT had been activated. Given their lack of exposure to the RRT, these respondents were excluded from the analysis, yielding a final sample size of 236. The demographic and clinical practice characteristics of respondents are shown in Table 1.

Resident Perceptions of the RRT (N=236)
The residentStrongly Disagree/Disagree, n (%)Neutral, n (%)Agree/ Strongly Agree, n (%)
  • NOTE: Abbreviations: RRT, rapid response team.

  • Where data do not equal 100%, this is due to missing data or rounding. Includes only data for respondents who had cared for a patient that required RRT activation.

Is comfortable managing the unstable patient without the RRT104 (44.1)64 (27.1)66 (28.0)
And RRT work together to make treatment decisions10 (4.2)13 (5.5)208 (88.1)
Believes there are fewer opportunities to care for unstable floor patients due to the RRT188 (79.7)26 (11.0)17 (7.2)
Feels less prepared to care for unstable patients due to the RRT201 (85.2)22 (9.3)13 (5.5)
Feels that working with the RRT creates a valuable educational experience9 (3.8)39 (16.5)184 (78.0)
Feels that nurses caring for the unstable patient should always contact them prior to contacting the RRT123 (52.1)33 (14.0)76 (32.2)
Would be unhappy with nurses calling RRT prior to contacting them141 (59.7)44 (18.6)51 (21.6)
Perceives that the presence of RRT decreases residents' autonomy179 (75.8)25 (10.6)28 (11.9)

Demographics and Primary Outcomes

Interns comprised 83 (35%) of the respondents; the average time in postgraduate training was 2.34 years (standard deviation=1.41). Of respondents, 163 (69%) were in medical fields, and 73 (31%) were in surgical fields. Overall responses to the survey are shown in Table 2, and subgroup analysis is shown in Table 3.

Perceptions of the RRT Based on Years of Training and Specialty
The resident1 Year, n=83, n (%)>1 Year, n=153, n (%)P ValueMedical, n=163, n (%)Surgical, n=73, n (%)P Value
  • NOTE: Abbreviations: RRT, rapid response team.

  • Where data do not equal 100%, this is due to missing data or rounding.

Is comfortable managing the unstable patient without the RRT  0.01  <0.01
Strongly disagree/disagree39 (47.6)65 (42.8) 67 (41.6)37 (50.7) 
Neutral29 (35.4)35 (23.0) 56 (34.8)8 (11.0) 
Agree/strongly agree14 (17.1)52 (34.2) 38 (23.6)28 (38.4) 
And RRT work together to make treatment decisions  0.61  0.04
Strongly disagree/disagree2 (2.4)8 (5.4) 4 (2.5)6 (8.7) 
Neutral5 (6.1)8 (5.4) 7 (4.3)6 (8.7) 
Agree/strongly agree75 (91.5)137 (89.3) 151 (93.2)57 (82.6) 
Believes there are fewer opportunities to care for unstable floor patients due to the RRT  0.05  0.04
Strongly disagree/disagree59 (72.8)129 (86.0) 136 (85.5)52 (72.2) 
Neutral13 (16.0)13 (8.7) 15 (9.4)11 (15.3) 
Agree/strongly agree9 (11.1)8 (5.3) 8 (5.0)9 (12.5) 
Feels less prepared to care for unstable patients due to the RRT  <0.01  0.79
Strongly disagree/disagree62 (74.7)139 (90.8) 140 (85.9)61 (83.6) 
Neutral14 (16.9)8 (5.2) 15 (9.2)7 (9.6) 
Agree/Strongly agree7 (8.4)6 (3.9) 8 (4.9)5 (6.8) 
Feels working with the RRT is a valuable educational experience  0.61  0.01
Strongly disagree/disagree2 (2.4)7 (4.7) 2 (1.2)7 (9.9) 
Neutral12 (14.6)27 (18.0) 25 (15.5)14 (19.7) 
Agree/strongly agree68 (82.9)116 (77.3) 134 (83.2)50 (70.4) 
Feels nurses caring for unstable patients should always contact the resident prior to contacting the RRT  0.49  <0.01
Strongly disagree/disagree47 (57.3)76 (50.7) 97 (60.2)26 (36.6) 
Neutral9 (11.0)24 (16.0) 26 (16.1)7 (9.9) 
Agree/strongly agree26 (31.7)50 (33.3) 38 (23.6)38 (53.5) 
Would be unhappy with nurses calling RRT prior to contacting them  0.81  <0.01
Strongly disagree/disagree51 (61.4)90 (58.8) 109 (66.9)32 (43.8) 
Neutral16 (19.3)28 (18.3) 30 (18.4)14 (19.2) 
Agree/strongly agree16 (19.3)35 (22.9) 24 (14.7)27 (37.0) 
Perceives that the presence of the RRT decreases autonomy as a physician  0.95  0.18
Strongly disagree/disagree63 (77.8)116 (76.8) 127 (79.9)52 (71.2) 
Neutral9 (11.1)16 (10.6) 17 (10.7)8 (11.0) 
Agree/strongly agree9 (11.1)19 (12.6) 15 (9.4)13 (17.8) 

Effect of the RRT on Resident Education

Of all residents, 66 (28%) agreed that they felt comfortable managing an unstable patient without the assistance of the RRT. Surgical residents felt more comfortable managing an unstable patient alone (38%) compared medical residents (24%) (P<0.01). Interns felt less comfortable caring for unstable patients without the RRT's assistance (17%) compared with upper‐level residents (34%) (P=0.01).

Residents overall disagreed with the statement that the RRT left them feeling less prepared to care for unstable patients (n=201; 85%). More upper‐level residents disagreed with this assertion (91%) compared with interns (75%) (P<0.01). Responses to this question did not differ significantly between medical and surgical residents.

Upper‐level residents were more likely to disagree with the statement that the RRT resulted in fewer opportunities to care for unstable patients (n=129; 86%) compared with interns (n=59; 73%) (P=0.05). Medical residents were also more likely to disagree with this statement (n=136; 86%) compared with surgical residents (n=52; 72%) (P=0.04).

With respect to residents' overall impressions of the educational value of the RRT, 68 (83%) interns and 116 (77%) upper‐level residents agreed that it provided a valuable educational experience (P=0.61). Medical and surgical residents differed in this regard, with 134 (83%) medical residents and 50 (70%) surgical residents agreeing that the RRT provided a valuable educational experience (P=0.01).

Effect of the RRT on Clinical Autonomy

Of all residents, 123 (52%) disagreed that the bedside nurse should always contact the primary resident prior to calling the RRT; 76 (32%) agreed with this statement. Medicine residents were more likely to disagree with this approach (n=97; 60%) than were surgical residents (n=26; 36%) (P<0.01). There was no difference between interns and upper‐level residents in response to this question. Most of those who disagreed with this statement were medical residents, whereas most surgical residents (n=38; 54%) agreed that they should be contacted first (P<0.01).

There were no differences between interns and upper‐level residents with respect to perceptions of the RRT's impact on clinical autonomy: 11% of interns and 13% of residents agreed that the RRT decreased their clinical autonomy as a physician. There was no significant difference between medical and surgical residents' responses to this question.

The majority of residents (n=208; 88%) agreed that they and the RRT work together to make treatment decisions for patients. This was true regardless of year of training (P=0.61), but it was expressed more often among medical residents than surgical residents (n=151, 93% vs n=57, 83%; P=0.04).

DISCUSSION

Most studies examining the educational and cultural impact of RRTs exist in the nursing literature. These studies demonstrate that medical and surgical nurses are often reluctant to call the RRT for fear of criticism by the patient's physician.[5, 8, 9, 10, 11, 12, 13] In contrast, our data demonstrate that resident physicians across all levels of training and specialties have a positive view of the RRT and its role in patient care. The data support our hypothesis that although most residents perceive educational benefit from their interactions with the RRT, this perception is greater for less‐experienced residents and for those residents who routinely provide care for critically ill patients and serve as code team leaders. In addition, a minority of residents, irrespective of years of training or medical specialty, felt that the RRT negatively impacted their clinical autonomy.

Our data have several important implications. First, although over half of the residents surveyed had not been exposed to RRTs during medical school, and despite having no formal training on the role of the RRT during residency, most residents identified their interactions with the RRT as potential learning opportunities. This finding differs from that of Benin and colleagues, who suggested that RRTs might negatively impact residents' educational development and decrease opportunities for high‐stakes clinical reasoning by allowing the clinical decision‐making process to be driven by the RRT staff rather than the resident.[5] One possible explanation for this discrepancy is the variable makeup of the RRT at different institutions. At our medical center, the RRT is comprised of a critical care nurse and respiratory therapist, whereas at other institutions, the RRT may be led by a resident, fellow, attending hospitalist, or intensivist, any of whom might supersede the primary resident once the RRT is engaged.

In our study, the perceived educational benefit of the RRT was most pronounced with interns. Interns likely derive incrementally greater benefit from each encounter with an acutely decompensating patient than do senior residents, whether the RRT is present or not. Observing the actions of seasoned nurses and respiratory therapists may demonstrate new tools for interns to use in their management of such situations; for example, the RRT may suggest different modes of oxygen delivery or new diagnostic tests. The RRT also likely helps interns navigate the hospital system by assisting with decisions around escalation of care and serving as a liaison to ICU staff.

Our data also have implications for resident perceptions of clinical autonomy. Interns, far less experienced caring for unstable patients than upper‐level residents, expressed more concern about the RRT stripping them of opportunities to do so and about feeling less prepared to handle clinically deteriorating patients. Part of this perception may be due to interns feeling less comfortable taking charge of a patient's care in the presence of an experienced critical care nurse and respiratory therapist, both for reasons related to clinical experience and to a cultural hierarchy that often places the intern at the bottom of the authority spectrum. In addition, when the RRT is called on an intern's patient, the senior resident may accompany the intern to the bedside and guide the intern on his or her approach to the situation; in some cases, the senior resident may take charge, leaving the intern feeling less autonomous.

If training sessions could be developed to address not only clinical decision making, but also multidisciplinary team interactions and roles in the acute care setting, this may mitigate interns' concerns. Such curricula could also enhance residents' experience in interprofessional care, an aspect of clinical training that has become increasingly important in the age of limited duty hours and higher volume, and higher acuity inpatient censuses. An RRT model, like a code blue model, could be used in simulation‐based training to increase both comfort with use of the RRT and efficiency of the RRTresidentnurse team. Although our study did not address specifically residents' perceptions of multidisciplinary teams, this could be a promising area for further study.

For surgical residents, additional factors are likely at play. Surgical residents spend significant time in the operating room, reducing time present at the bedside and hindering the ability to respond swiftly when an RRT is called on their patient. This could cause surgical residents to feel less involved in the care of that patientsupported by our finding that fewer surgical residents felt able to collaborate with the RRTand also to derive less educational benefit and clinical satisfaction from the experience. Differences between medical and surgical postgraduate training also likely play a role, manifest by varying clinical roles and duration of training, and as such it may not be appropriate to draw direct comparisons between respective postgraduate year levels. In addition, differences in patients' medical complexity, varying allegiance to the traditional hierarchy of medical providers, and degree of familiarity with the RRT itself may impact surgical residents' comfort with the RRT.

Limitations of our study include that it was conducted at a single site and addressed a specific population of residents at our tertiary academic center. Though we achieved an excellent response rate, our subspecialty sample sizes were too small to allow for individual comparisons among those groups. Conducting a larger study at multiple institutions where the makeup of the RRT differs could provide further insight into how different clinical environments and different RRT models impact resident perceptions. Finally, we allowed each respondent to interpret both educational benefit and clinical autonomy in the context of their own level of training and clinical practice rather than providing strict definitions of these terms. There is no standardized definition of autonomy in the context of resident clinical practice, and we did not measure direct educational outcomes. Our study design therefore allowed only for measurement of perceptions of these concepts. Measurement of actual educational value of the RRTfor example, through direct clinical observation or by incorporating the RRT experience into an entrustable professional activitywould provide more quantitative evidence of the RRT's utility for our resident population. Future study in this area would help to support the development and ongoing assessment of RRT‐based curricula moving forward.

CONCLUSION

Our data show that resident physicians have a strongly favorable opinion of the RRT at our institution. Future studies should aim to quantify the educational benefit of RRTs for residents and identify areas for curricular development to enhance resident education as RRTs become more pervasive.

Rapid response teams (RRTs) have been promoted by patient safety and quality‐improvement organizations as a strategy to reduce preventable in‐hospital deaths.[1] To date, critical analysis of RRTs has focused primarily on their impact on quality‐of‐care metrics.[2, 3, 4] Comparatively few studies have examined the cultural and educational impact of RRTs, particularly at academic medical centers, and those that do exist have focused almost exclusively on perceptions of nurses rather than resident physicians.[5, 6, 7, 8, 9, 10]

Although a prior study found that internal medicine and general surgery residents believed that RRTs improved patient safety, they were largely ambivalent about the RRT's impact on education and training.[11] To date, there has been no focused assessment of resident physician impressions of an RRT across years of training and medical specialty to inform the use of this multidisciplinary team as a component of their residency education.

We sought to determine whether resident physicians at a tertiary care academic medical center perceive educational benefit from collaboration with the RRT and whether they feel that the RRT adversely affects clinical autonomy.

METHODS

The Hospital

Moffitt‐Long Hospital, the tertiary academic medical center of the University of California, San Francisco (UCSF), is a 600‐bed acute care hospital that provides comprehensive critical care services and serves as a major referral center in northern California. There are roughly 5000 admissions to the hospital annually. At the time the study was conducted, there were approximately 200 RRT calls per 1000 adult hospital discharges.

The Rapid Response Team

The RRT is called to assess, triage, and treat patients who have experienced a decline in their clinical status short of a cardiopulmonary arrest. The RRT has been operational at UCSF since June 1, 2007, and is composed of a dedicated critical care nurse and respiratory therapist available 24 hours a day, 7 days a week. The RRT can be activated by any concerned staff member based on vital sign abnormalities, decreased urine output, changes in mental status, or any significant concern about the trajectory of the patient's clinical course.

When the RRT is called on a given patient, the patient's primary physician (at our institution, a resident) is also called to the bedside and works alongside the RRT to address the patient's acute clinical needs. The primary physician, bedside nurse, and RRT discuss the plan of care for the patient, including clinical evaluation, management, and the need for additional monitoring or a transition to a higher level of care. Residents at our institution receive no formal instruction regarding the role of the RRT or curriculum on interfacing with the RRT, and they do not serve as members of the RRT as part of a clinical rotation.

The Survey Process

Study subjects were asked via e‐mail to participate in a brief online survey. Subjects were offered the opportunity to win a $100 gift certificate in return for their participation. Weekly e‐mail reminders were sent for a period of 3 months or until a given subject had completed the survey. The survey was administered over a 3‐month period, from March through May, to allow time for residents to work with the RRT during the academic year. The Committee on Human Research at the University of California San Francisco Medical Center approved the study.

Target Population

All residents in specialties that involved direct patient care and the potential to use the adult RRT were included in the study. This included residents in the fields of internal medicine, neurology, general surgery, orthopedic surgery, neurosurgery, plastic surgery, urology, and otolaryngology (Table 1). Residents in pediatrics and obstetrics and gynecology were excluded, as emergencies in their patients are addressed by a pediatric RRT and an obstetric anesthesiologist, respectively. Residents in anesthesiology were excluded as they do not care for nonintensive care unit (ICU) patients as part of the primary team and are not involved in RRT encounters.

Demographics of Survey Respondents (N=236)
DemographicNo. (%)
  • NOTE: Abbreviations: RRT, rapid response team; SD, standard deviation.

  • Where data do not equal 100%, this is due to missing data or rounding. Table does not include 10 respondents who had never cared for a patient for whom the RRT was activated.

Medical specialty 
Internal medicine145 (61.4)
Neurology18 (7.6)
General surgery31 (13.1)
Orthopedic surgery17 (7.2)
Neurosurgery4 (1.7)
Plastic surgery2 (0.8)
Urology9 (3.8)
Otolaryngology10 (4.2)
Years of postgraduate trainingAverage 2.34 (SD 1.41)
183 (35.2)
260 (25.4)
355 (23.3)
420 (8.5)
58 (3.4)
65 (2.1)
75 (2.1)
Gender 
Male133 (56.4)
Female102 (43.2)
Had exposure to RRT during training 
Yes106 (44.9)
No127 (53.8)
Had previously initiated a call to the RRT 
Yes106 (44.9)
No128 (54.2)

Survey Design

The resident survey contained 20 RRT‐related items and 7 demographic and practice items. Responses for RRT‐related questions utilized a 5‐point Likert scale ranging from strongly disagree to strongly agree. The survey was piloted prior to administration to check comprehension and interpretation by physicians with experience in survey writing (for the full survey, see Supporting Information, Appendix, in the online version of this article).

Survey Objectives

The survey was designed to capture the experiences of residents who had cared for a patient for whom the RRT had been activated. Data collected included residents' perceptions of the impact of the RRT on their residency education and clinical autonomy, the quality of care provided, patient safety, and hospital‐wide culture. Potential barriers to use of the RRT were also examined.

Outcomes

The study's primary outcomes included the perceived educational benefit of the RRT and its perceived impact on clinical autonomy. Secondary outcomes included the effect of years of training and resident specialty on both the perceived educational benefit and impact on clinical autonomy among our study group.

Statistical Analysis

Responses to each survey item were described for each specialty, and subgroup analysis was conducted. For years of training, that item was dichotomized into either 1 year (henceforth referred to as interns) or greater than 1 year (henceforth referred to as upper‐level residents). Resident specialty was dichotomized into medical fields (internal medicine and neurology) or surgical fields. For statistical analysis, agreement statements were collapsed to either disagree (strongly disagree/disagree), neutral, or agree (strongly agree/agree). The influence of years of resident training and resident specialty was assessed for all items in the survey using 2 or Fisher exact tests as appropriate for the 3 agreement categories. Analysis was conducted using SPSS 21.0 (IBM Corp., Armonk, NY).

RESULTS

There were 246 responses to the survey of a possible 342, yielding a response rate of 72% (Table 2). Ten respondents stated that they had never cared for a patient where the RRT had been activated. Given their lack of exposure to the RRT, these respondents were excluded from the analysis, yielding a final sample size of 236. The demographic and clinical practice characteristics of respondents are shown in Table 1.

Resident Perceptions of the RRT (N=236)
The residentStrongly Disagree/Disagree, n (%)Neutral, n (%)Agree/ Strongly Agree, n (%)
  • NOTE: Abbreviations: RRT, rapid response team.

  • Where data do not equal 100%, this is due to missing data or rounding. Includes only data for respondents who had cared for a patient that required RRT activation.

Is comfortable managing the unstable patient without the RRT104 (44.1)64 (27.1)66 (28.0)
And RRT work together to make treatment decisions10 (4.2)13 (5.5)208 (88.1)
Believes there are fewer opportunities to care for unstable floor patients due to the RRT188 (79.7)26 (11.0)17 (7.2)
Feels less prepared to care for unstable patients due to the RRT201 (85.2)22 (9.3)13 (5.5)
Feels that working with the RRT creates a valuable educational experience9 (3.8)39 (16.5)184 (78.0)
Feels that nurses caring for the unstable patient should always contact them prior to contacting the RRT123 (52.1)33 (14.0)76 (32.2)
Would be unhappy with nurses calling RRT prior to contacting them141 (59.7)44 (18.6)51 (21.6)
Perceives that the presence of RRT decreases residents' autonomy179 (75.8)25 (10.6)28 (11.9)

Demographics and Primary Outcomes

Interns comprised 83 (35%) of the respondents; the average time in postgraduate training was 2.34 years (standard deviation=1.41). Of respondents, 163 (69%) were in medical fields, and 73 (31%) were in surgical fields. Overall responses to the survey are shown in Table 2, and subgroup analysis is shown in Table 3.

Perceptions of the RRT Based on Years of Training and Specialty
The resident1 Year, n=83, n (%)>1 Year, n=153, n (%)P ValueMedical, n=163, n (%)Surgical, n=73, n (%)P Value
  • NOTE: Abbreviations: RRT, rapid response team.

  • Where data do not equal 100%, this is due to missing data or rounding.

Is comfortable managing the unstable patient without the RRT  0.01  <0.01
Strongly disagree/disagree39 (47.6)65 (42.8) 67 (41.6)37 (50.7) 
Neutral29 (35.4)35 (23.0) 56 (34.8)8 (11.0) 
Agree/strongly agree14 (17.1)52 (34.2) 38 (23.6)28 (38.4) 
And RRT work together to make treatment decisions  0.61  0.04
Strongly disagree/disagree2 (2.4)8 (5.4) 4 (2.5)6 (8.7) 
Neutral5 (6.1)8 (5.4) 7 (4.3)6 (8.7) 
Agree/strongly agree75 (91.5)137 (89.3) 151 (93.2)57 (82.6) 
Believes there are fewer opportunities to care for unstable floor patients due to the RRT  0.05  0.04
Strongly disagree/disagree59 (72.8)129 (86.0) 136 (85.5)52 (72.2) 
Neutral13 (16.0)13 (8.7) 15 (9.4)11 (15.3) 
Agree/strongly agree9 (11.1)8 (5.3) 8 (5.0)9 (12.5) 
Feels less prepared to care for unstable patients due to the RRT  <0.01  0.79
Strongly disagree/disagree62 (74.7)139 (90.8) 140 (85.9)61 (83.6) 
Neutral14 (16.9)8 (5.2) 15 (9.2)7 (9.6) 
Agree/Strongly agree7 (8.4)6 (3.9) 8 (4.9)5 (6.8) 
Feels working with the RRT is a valuable educational experience  0.61  0.01
Strongly disagree/disagree2 (2.4)7 (4.7) 2 (1.2)7 (9.9) 
Neutral12 (14.6)27 (18.0) 25 (15.5)14 (19.7) 
Agree/strongly agree68 (82.9)116 (77.3) 134 (83.2)50 (70.4) 
Feels nurses caring for unstable patients should always contact the resident prior to contacting the RRT  0.49  <0.01
Strongly disagree/disagree47 (57.3)76 (50.7) 97 (60.2)26 (36.6) 
Neutral9 (11.0)24 (16.0) 26 (16.1)7 (9.9) 
Agree/strongly agree26 (31.7)50 (33.3) 38 (23.6)38 (53.5) 
Would be unhappy with nurses calling RRT prior to contacting them  0.81  <0.01
Strongly disagree/disagree51 (61.4)90 (58.8) 109 (66.9)32 (43.8) 
Neutral16 (19.3)28 (18.3) 30 (18.4)14 (19.2) 
Agree/strongly agree16 (19.3)35 (22.9) 24 (14.7)27 (37.0) 
Perceives that the presence of the RRT decreases autonomy as a physician  0.95  0.18
Strongly disagree/disagree63 (77.8)116 (76.8) 127 (79.9)52 (71.2) 
Neutral9 (11.1)16 (10.6) 17 (10.7)8 (11.0) 
Agree/strongly agree9 (11.1)19 (12.6) 15 (9.4)13 (17.8) 

Effect of the RRT on Resident Education

Of all residents, 66 (28%) agreed that they felt comfortable managing an unstable patient without the assistance of the RRT. Surgical residents felt more comfortable managing an unstable patient alone (38%) compared medical residents (24%) (P<0.01). Interns felt less comfortable caring for unstable patients without the RRT's assistance (17%) compared with upper‐level residents (34%) (P=0.01).

Residents overall disagreed with the statement that the RRT left them feeling less prepared to care for unstable patients (n=201; 85%). More upper‐level residents disagreed with this assertion (91%) compared with interns (75%) (P<0.01). Responses to this question did not differ significantly between medical and surgical residents.

Upper‐level residents were more likely to disagree with the statement that the RRT resulted in fewer opportunities to care for unstable patients (n=129; 86%) compared with interns (n=59; 73%) (P=0.05). Medical residents were also more likely to disagree with this statement (n=136; 86%) compared with surgical residents (n=52; 72%) (P=0.04).

With respect to residents' overall impressions of the educational value of the RRT, 68 (83%) interns and 116 (77%) upper‐level residents agreed that it provided a valuable educational experience (P=0.61). Medical and surgical residents differed in this regard, with 134 (83%) medical residents and 50 (70%) surgical residents agreeing that the RRT provided a valuable educational experience (P=0.01).

Effect of the RRT on Clinical Autonomy

Of all residents, 123 (52%) disagreed that the bedside nurse should always contact the primary resident prior to calling the RRT; 76 (32%) agreed with this statement. Medicine residents were more likely to disagree with this approach (n=97; 60%) than were surgical residents (n=26; 36%) (P<0.01). There was no difference between interns and upper‐level residents in response to this question. Most of those who disagreed with this statement were medical residents, whereas most surgical residents (n=38; 54%) agreed that they should be contacted first (P<0.01).

There were no differences between interns and upper‐level residents with respect to perceptions of the RRT's impact on clinical autonomy: 11% of interns and 13% of residents agreed that the RRT decreased their clinical autonomy as a physician. There was no significant difference between medical and surgical residents' responses to this question.

The majority of residents (n=208; 88%) agreed that they and the RRT work together to make treatment decisions for patients. This was true regardless of year of training (P=0.61), but it was expressed more often among medical residents than surgical residents (n=151, 93% vs n=57, 83%; P=0.04).

DISCUSSION

Most studies examining the educational and cultural impact of RRTs exist in the nursing literature. These studies demonstrate that medical and surgical nurses are often reluctant to call the RRT for fear of criticism by the patient's physician.[5, 8, 9, 10, 11, 12, 13] In contrast, our data demonstrate that resident physicians across all levels of training and specialties have a positive view of the RRT and its role in patient care. The data support our hypothesis that although most residents perceive educational benefit from their interactions with the RRT, this perception is greater for less‐experienced residents and for those residents who routinely provide care for critically ill patients and serve as code team leaders. In addition, a minority of residents, irrespective of years of training or medical specialty, felt that the RRT negatively impacted their clinical autonomy.

Our data have several important implications. First, although over half of the residents surveyed had not been exposed to RRTs during medical school, and despite having no formal training on the role of the RRT during residency, most residents identified their interactions with the RRT as potential learning opportunities. This finding differs from that of Benin and colleagues, who suggested that RRTs might negatively impact residents' educational development and decrease opportunities for high‐stakes clinical reasoning by allowing the clinical decision‐making process to be driven by the RRT staff rather than the resident.[5] One possible explanation for this discrepancy is the variable makeup of the RRT at different institutions. At our medical center, the RRT is comprised of a critical care nurse and respiratory therapist, whereas at other institutions, the RRT may be led by a resident, fellow, attending hospitalist, or intensivist, any of whom might supersede the primary resident once the RRT is engaged.

In our study, the perceived educational benefit of the RRT was most pronounced with interns. Interns likely derive incrementally greater benefit from each encounter with an acutely decompensating patient than do senior residents, whether the RRT is present or not. Observing the actions of seasoned nurses and respiratory therapists may demonstrate new tools for interns to use in their management of such situations; for example, the RRT may suggest different modes of oxygen delivery or new diagnostic tests. The RRT also likely helps interns navigate the hospital system by assisting with decisions around escalation of care and serving as a liaison to ICU staff.

Our data also have implications for resident perceptions of clinical autonomy. Interns, far less experienced caring for unstable patients than upper‐level residents, expressed more concern about the RRT stripping them of opportunities to do so and about feeling less prepared to handle clinically deteriorating patients. Part of this perception may be due to interns feeling less comfortable taking charge of a patient's care in the presence of an experienced critical care nurse and respiratory therapist, both for reasons related to clinical experience and to a cultural hierarchy that often places the intern at the bottom of the authority spectrum. In addition, when the RRT is called on an intern's patient, the senior resident may accompany the intern to the bedside and guide the intern on his or her approach to the situation; in some cases, the senior resident may take charge, leaving the intern feeling less autonomous.

If training sessions could be developed to address not only clinical decision making, but also multidisciplinary team interactions and roles in the acute care setting, this may mitigate interns' concerns. Such curricula could also enhance residents' experience in interprofessional care, an aspect of clinical training that has become increasingly important in the age of limited duty hours and higher volume, and higher acuity inpatient censuses. An RRT model, like a code blue model, could be used in simulation‐based training to increase both comfort with use of the RRT and efficiency of the RRTresidentnurse team. Although our study did not address specifically residents' perceptions of multidisciplinary teams, this could be a promising area for further study.

For surgical residents, additional factors are likely at play. Surgical residents spend significant time in the operating room, reducing time present at the bedside and hindering the ability to respond swiftly when an RRT is called on their patient. This could cause surgical residents to feel less involved in the care of that patientsupported by our finding that fewer surgical residents felt able to collaborate with the RRTand also to derive less educational benefit and clinical satisfaction from the experience. Differences between medical and surgical postgraduate training also likely play a role, manifest by varying clinical roles and duration of training, and as such it may not be appropriate to draw direct comparisons between respective postgraduate year levels. In addition, differences in patients' medical complexity, varying allegiance to the traditional hierarchy of medical providers, and degree of familiarity with the RRT itself may impact surgical residents' comfort with the RRT.

Limitations of our study include that it was conducted at a single site and addressed a specific population of residents at our tertiary academic center. Though we achieved an excellent response rate, our subspecialty sample sizes were too small to allow for individual comparisons among those groups. Conducting a larger study at multiple institutions where the makeup of the RRT differs could provide further insight into how different clinical environments and different RRT models impact resident perceptions. Finally, we allowed each respondent to interpret both educational benefit and clinical autonomy in the context of their own level of training and clinical practice rather than providing strict definitions of these terms. There is no standardized definition of autonomy in the context of resident clinical practice, and we did not measure direct educational outcomes. Our study design therefore allowed only for measurement of perceptions of these concepts. Measurement of actual educational value of the RRTfor example, through direct clinical observation or by incorporating the RRT experience into an entrustable professional activitywould provide more quantitative evidence of the RRT's utility for our resident population. Future study in this area would help to support the development and ongoing assessment of RRT‐based curricula moving forward.

CONCLUSION

Our data show that resident physicians have a strongly favorable opinion of the RRT at our institution. Future studies should aim to quantify the educational benefit of RRTs for residents and identify areas for curricular development to enhance resident education as RRTs become more pervasive.

References
  1. Institute for Healthcare Improvement. Rapid response teams. Available at: http://www.ihi.org/topics/rapidresponseteams. Accessed May 5, 2014.
  2. Chan PS, Jain R, Nallmothu BK, Berg RA, Sasson C. Rapid response teams: a systematic review and meta‐analysis. Arch Intern Med. 2010;170(1):1826.
  3. Devita MA, Bellomo R, Hillman K, et al. Findings of the first consensus conference on medical emergency teams. Crit Care Med. 2006;34(9):24632478.
  4. Winters BD, Pham JC, Hunt EA, Guallar E, Berenholtz S, Pronovost PJ. Rapid response systems: a systematic review. Crit Care Med. 2007;35(5):12381243.
  5. Benin AL, Borgstrom CP, Jenq GY, Roumanis SA, Horwitz LI. Defining impact of a rapid response team: qualitative study with nurses, physicians and hospital administrators. BMJ Qual Saf. 2012;21(5):391398.
  6. Leach LS, Mayo A, O'Rourke M. How RNs rescue patients: a qualitative study of RNs' perceived involvement in rapid response teams. Qual Saf Health Care. 2010;19(5):e13.
  7. Metcalf R, Scott S, Ridgway M, Gibson D. Rapid response team approach to staff satisfaction. Orthop Nurs. 2008;27(5):266271; quiz 272–273.
  8. Salamonson Y, Heere B, Everett B, Davidson P. Voices from the floor: nurses' perceptions of the medical emergency team. Intensive Crit Care Nurs. 2006;22(3):138143.
  9. Shapiro SE, Donaldson NE, Scott MB. Rapid response teams seen through the eyes of the nurse. Am J Nurs. 2010;110(6):2834; quiz 35–36.
  10. Shearer B, Marshall S, Buist MD, et al. What stops hospital clinical staff from following protocols? An analysis of the incidence and factors behind the failure of bedside clinical staff to activate the rapid response system in a multi‐campus Australian metropolitan healthcare service. BMJ Qual Saf. 2012;21(7):569575.
  11. Sarani B, Sonnad S, Bergey MR, et al. Resident and RN perceptions of the impact of a medical emergency team on education and patient safety in an academic medical center. Crit Care Med. 2009;37(12):30913096.
  12. Marshall SD, Kitto S, Shearer W, et al. Why don't hospital staff activate the rapid response system (RRS)? How frequently is it needed and can the process be improved? Implement Sci. 2011;6:39.
  13. Peebles E, Subbe CP, Hughes P, Gemmell L. Timing and teamwork–an observational pilot study of patients referred to a rapid response team with the aim of identifying factors amenable to re‐design of a rapid response system. Resuscitation. 2012;83(6):782787.
References
  1. Institute for Healthcare Improvement. Rapid response teams. Available at: http://www.ihi.org/topics/rapidresponseteams. Accessed May 5, 2014.
  2. Chan PS, Jain R, Nallmothu BK, Berg RA, Sasson C. Rapid response teams: a systematic review and meta‐analysis. Arch Intern Med. 2010;170(1):1826.
  3. Devita MA, Bellomo R, Hillman K, et al. Findings of the first consensus conference on medical emergency teams. Crit Care Med. 2006;34(9):24632478.
  4. Winters BD, Pham JC, Hunt EA, Guallar E, Berenholtz S, Pronovost PJ. Rapid response systems: a systematic review. Crit Care Med. 2007;35(5):12381243.
  5. Benin AL, Borgstrom CP, Jenq GY, Roumanis SA, Horwitz LI. Defining impact of a rapid response team: qualitative study with nurses, physicians and hospital administrators. BMJ Qual Saf. 2012;21(5):391398.
  6. Leach LS, Mayo A, O'Rourke M. How RNs rescue patients: a qualitative study of RNs' perceived involvement in rapid response teams. Qual Saf Health Care. 2010;19(5):e13.
  7. Metcalf R, Scott S, Ridgway M, Gibson D. Rapid response team approach to staff satisfaction. Orthop Nurs. 2008;27(5):266271; quiz 272–273.
  8. Salamonson Y, Heere B, Everett B, Davidson P. Voices from the floor: nurses' perceptions of the medical emergency team. Intensive Crit Care Nurs. 2006;22(3):138143.
  9. Shapiro SE, Donaldson NE, Scott MB. Rapid response teams seen through the eyes of the nurse. Am J Nurs. 2010;110(6):2834; quiz 35–36.
  10. Shearer B, Marshall S, Buist MD, et al. What stops hospital clinical staff from following protocols? An analysis of the incidence and factors behind the failure of bedside clinical staff to activate the rapid response system in a multi‐campus Australian metropolitan healthcare service. BMJ Qual Saf. 2012;21(7):569575.
  11. Sarani B, Sonnad S, Bergey MR, et al. Resident and RN perceptions of the impact of a medical emergency team on education and patient safety in an academic medical center. Crit Care Med. 2009;37(12):30913096.
  12. Marshall SD, Kitto S, Shearer W, et al. Why don't hospital staff activate the rapid response system (RRS)? How frequently is it needed and can the process be improved? Implement Sci. 2011;6:39.
  13. Peebles E, Subbe CP, Hughes P, Gemmell L. Timing and teamwork–an observational pilot study of patients referred to a rapid response team with the aim of identifying factors amenable to re‐design of a rapid response system. Resuscitation. 2012;83(6):782787.
Issue
Journal of Hospital Medicine - 10(1)
Issue
Journal of Hospital Medicine - 10(1)
Page Number
8-12
Page Number
8-12
Publications
Publications
Article Type
Display Headline
The effect of a rapid response team on resident perceptions of education and autonomy
Display Headline
The effect of a rapid response team on resident perceptions of education and autonomy
Sections
Article Source

© 2015 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Sumant R. Ranji, MD, UCSF Division of Hospital Medicine, 533 Parnassus Avenue, Box 0131, San Francisco, CA 94143‐0131; Telephone: 415‐514‐9256; Fax: 415‐514‐2094; E‐mail: sumantr@medicine.ucsf.edu
Content Gating
No Gating (article Unlocked/Free)
Alternative CME
Article PDF Media
Media Files