Neera Ahuja, MD

Point-of-care ultrasonography (POCUS) can transform healthcare delivery through its diagnostic and therapeutic expediency.¹ POCUS has been shown to bolster diagnostic accuracy, reduce procedural complications, decrease inpatient length of stay, and improve patient satisfaction by encouraging the physician to be present at the bedside.^2-8

POCUS has become widespread across a variety of clinical settings as more investigations have demonstrated its positive impact on patient care.^1,9-12 This includes the use of POCUS by trainees, who are now utilizing this technology as part of their assessments of patients.^13,14 However, trainees may be performing these examinations with minimal oversight, and outside of emergency medicine, there are few guidelines on how to effectively teach POCUS or measure competency.^13,14 While POCUS is rapidly becoming a part of inpatient care, teaching physicians may have little experience in ultrasound or the expertise to adequately supervise trainees.¹⁴ There is a growing need to study what trainees can learn and how this knowledge is acquired.

Previous investigations have demonstrated that inexperienced users can be taught to use POCUS to identify a variety of pathological states.^2,3,15-23 Most of these curricula used a single lecture series as their pedagogical vehicle, and they variably included junior medical trainees. More importantly, the investigations did not explore whether personal access to handheld ultrasound devices (HUDs) improved learning. In theory, improved access to POCUS devices increases opportunities for authentic and deliberate practice, which may be needed to improve trainee skill with POCUS beyond the classroom setting.¹⁴

This study aimed to address several ongoing gaps in knowledge related to learning POCUS. First, we hypothesized that personal HUD access would improve trainees’ POCUS-­related knowledge and interpretive ability as a result of increased practice opportunities. Second, we hypothesized that trainees who receive personal access to HUDs would be more likely to perform POCUS examinations and feel more confident in their interpretations. Finally, we hypothesized that repeated exposure to POCUS-related lectures would result in greater improvements in knowledge as compared with a single lecture series.

Participants and Setting

The 2017 intern class (n = 47) at an academic internal medicine residency program participated in the study. Control data were obtained from the 2016 intern class (historical control; n = 50) and the 2018 intern class (contemporaneous control; n = 52). The Stanford University Institutional Review Board approved this study.

Study Design

The 2017 intern class (n = 47) received POCUS didactics from June 2017 to June 2018. To evaluate if increased access to HUDs improved learning outcomes, the 2017 interns were randomized 1:1 to receive their own personal HUD that could be used for patient care and/or self-directed learning (n = 24) vs no-HUD (n = 23; Figure). Learning outcomes were assessed over the course of 1 year (see “Outcomes” below) and were compared with the 2016 and 2018 controls. The 2016 intern class had completed a year of training but had not received formalized POCUS didactics (historical control), whereas the 2018 intern class was assessed at the beginning of their year (contemporaneous control; Figure). In order to make comparisons based on intern experience, baseline data for the 2017 intern class were compared with the 2018 intern class, whereas end-of-study data for 2017 interns were compared with 2016 interns.

Outcomes

The primary outcome was the difference in assessment scores at the end of the study period between interns randomized to receive a HUD and those who were not. Secondary outcomes included differences in HUD usage rates, lecture attendance, and assessment scores. To assess whether repeated lecture exposure resulted in greater amounts of learning, this study evaluated for assessment score improvements after each lecture block. Finally, trainee attitudes toward POCUS and their confidence in their interpretative ability were measured at the beginning and end of the study period.

Curriculum Implementation

The lectures were administered as once-weekly didactics of 1-hour duration to interns rotating on the inpatient wards rotation. This rotation is 4 weeks long, and each intern will experience the rotation two to four times per year. Each lecture contained two parts: (1) 20-30 minutes of didactics via Microsoft PowerPoint^TM and (2) 30-40 minutes of supervised practice using HUDs on standardized patients. Four lectures were given each month: (1) introduction to POCUS and ultrasound physics, (2) thoracic/lung ultrasound, (3) echocardiography, and (4) abdominal POCUS. The lectures consisted of contrasting cases of normal/abnormal videos and clinical vignettes. These four lectures were repeated each month as new interns rotated on service. Some interns experienced the same content multiple times, which was intentional in order to assess their rates of learning over time. Lecture contents were based on previously published guidelines and expert consensus for teaching POCUS in internal medicine.^{13, 24-26} Content from the Accreditation Council for Graduate Medical Education (ACGME) and the American College of Emergency Physicians (ACEP) was also incorporated because these organizations had published relevant guidelines for teaching POCUS.^13,26 Further development of the lectures occurred through review of previously described POCUS-relevant curricula.^27-32

Handheld Ultrasound Devices

This study used the Philips Lumify^TM, a United States Food and Drug Administration–approved device. Interns randomized to HUDs received their own device at the start of the rotation. It was at their discretion to use the device outside of the course. All devices were approved for patient use and were encrypted in compliance with our information security office. For privacy reasons, any saved patient images were not reviewed by the researchers. Interns were encouraged to share their findings with supervising physicians during rounds, but actual oversight was not measured. Interns not randomized to HUDs could access a single community device that was shared among all residents and fellows in the hospital. Interns reported the average number of POCUS examinations performed each week via a survey sent during the last week of the rotation.

Assessment Design and Implementation

Assessments evaluating trainee knowledge were administered before, during, and after the study period (Figure). For the 2017 cohort, assessments were also administered at the start and end of the ward month to track knowledge acquisition. Assessment contents were selected from POCUS guidelines for internal medicine and adaptation of the ACGME and ACEP guidelines.^13,24,26 Additional content was obtained from major society POCUS tutorials and deidentified images collected by the study authors.^13,24,33 In keeping with previously described methodology, the images were shown for approximately 12 seconds, followed by five additional seconds to allow the learner to answer the question.³² Final assessment contents were determined by the authors using the Delphi method.³⁴ A sample assessment can be found in the Appendix Material.

Surveys

Surveys were administered alongside the assessments to the 2016-2018 intern classes. These surveys assessed trainee attitudes toward POCUS and were based on previously validated assessments.^27,28,30 Attitudes were measured using 5-point Likert scales.

Statistical Analysis

For the primary outcome, we performed generalized binomial mixed-effect regressions using the survey periods, randomization group, and the interaction of the two as independent variables after adjusting for attendance and controlling of intra-intern correlations. The bivariate unadjusted analysis was performed to display the distribution of overall correctness on the assessments. Wilcoxon signed rank test was used to determine score significance for dependent score variables (R-­Statistical Programming Language, Vienna, Austria).

Baseline Characteristics

There were 149 interns who participated in this study (Figure). Assessment/survey completion rates were as follows: 2016 control: 68.0%; 2017 preintervention: 97.9%; 2017 postintervention: 89.4%; and 2018 control: 100%. The 2017 interns reported similar amounts of prior POCUS exposure in medical school (Table 1).

Primary Outcome: Assessment Scores (HUD vs no HUD)

There were no significant differences in assessment scores at the end of the study between interns randomized to personal HUD access vs those to no-HUD access (Table 1). HUD interns reported performing POCUS assessments on patients a mean 6.8 (standard deviation [SD] 2.2) times per week vs 6.4 (SD 2.9) times per week in the no-HUD arm (P = .66). The mean lecture attendance was 75.0% and did not significantly differ between the HUD arms (Table 1).

Secondary Outcomes

Impact of Repeating Lectures

The 2017 interns demonstrated significant increases in preblock vs postblock assessment scores after first-time exposure to the lectures (median preblock score 0.61 [interquartile range (IQR), 0.53-0.70] vs postblock score 0.81 [IQR, 0.72-0.86]; P < .001; Table 2). However, intern performance on the preblock vs postblock assessments after second-time exposure to the curriculum failed to improve (median second preblock score 0.78 [IQR, 0.69-0.83] vs postblock score 0.81 [IQR, 0.64-0.89]; P = .94). Intern performance on individual domains of knowledge for each block is listed in Appendix Table 1.

Intervention Performance vs Controls

The 2016 historical control had significantly higher scores compared with the 2017 preintervention group (P < .001; Appendix Table 2). The year-long lecture series resulted in significant increases in median scores for the 2017 group (median preintervention score 0.55 [0.41-0.61] vs median postintervention score 0.84 [0.71-0.90]; P = .006; Appendix Table 1). At the end of the study, the 2017 postintervention scores were significantly higher across multiple knowledge domains compared with the 2016 historical control (Appendix Table 2).

Survey Results

Notably, the 2017 intern class at the end of the intervention did not have significantly different assessment scores for several disease-specific domains, compared with the 2016 control (Appendix Table 2). Nonetheless, the 2017 intern class reported higher levels of confidence in these same domains despite similar scores (Supplementary Figure). The HUD group seldomly cited a lack of confidence in their abilities as a barrier to performing POCUS examinations (17.6%), compared with the no-HUD group (50.0%), despite nearly identical assessment scores between the two groups (Table 1).

Previous guidelines have recommended increased HUD access for learners,^13,24,35,36 but there have been few investigations that have evaluated the impact of such access on learning POCUS. One previous investigation found that hospitalists who carried HUDs were more likely to identify heart failure on bedside examination.³⁷ In contrast, our study found no improvement in interpretative ability when randomizing interns to carry HUDs for patient care. Notably, interns did not perform more POCUS examinations when given HUDs. We offer several explanations for this finding. First, time-motion studies have demonstrated that internal medicine interns spend less than 15% of their time toward direct patient care.³⁸ It is possible that the demands of being an intern impeded their ability to perform more POCUS examinations on their patients, regardless of HUD access. Alternatively, the interns randomized to no personal access may have used the community device more frequently as a result of the lecture series. Given the cost of HUDs, further studies are needed to assess the degree to which HUD access will improve trainee interpretive ability, especially as more training programs consider the creation of ultrasound curricula.^{10,11,24,39,40}

This study was unique because it followed interns over a year-long course that repeated the same material to assess rates of learning with repeated exposure. Learners improved their scores after the first, but not second, block. Furthermore, the median scores were nearly identical between the first postblock assessment and second preblock assessment (0.81 vs 0.78), suggesting that knowledge was retained between blocks. Together, these findings suggest there may be limitations of traditional lectures that use standardized patient models for practice. Supplementary pedagogies, such as in-the-moment feedback with actual patients, may be needed to promote mastery.^14,35

Despite no formal curriculum, the 2016 intern class (historical control) had learned POCUS to some degree based on their higher assessment scores compared with the 2017 intern class during the preintervention period. Such learning may be informal, and yet, trainees may feel confident in making clinical decisions without formalized training, accreditation, or oversight. As suggested by this study, adding regular didactics or giving trainees HUDs may not immediately solve this issue. For assessment items in which the 2017 interns did not significantly differ from the controls, they nonetheless reported higher confidence in their abilities. Similarly, interns randomized to HUDs less frequently cited a lack of confidence in their abilities, despite similar scores to the no-HUD group. Such confidence may be incongruent with their actual knowledge or ability to safely use POCUS. This phenomenon of misplaced confidence is known as the Dunning–Kruger effect, and it may be common with ultrasound learning.⁴¹ While confidence can be part of a holistic definition of competency,¹⁴ these results raise the concern that trainees may have difficulty assessing their own competency level with POCUS.³⁵

There are several limitations to this study. It was performed at a single institution with limited sample size. It examined only intern physicians because of funding constraints, which limits the generalizability of these findings among medical trainees. Technical ability assessments (including obtaining and interpreting images) were not included. We were unable to track the timing or location of the devices’ usage, and the interns’ self-reported usage rates may be subject to recall bias. To our knowledge, there were no significant lapses in device availability/functionality. Intern physicians in the HUD arm did not receive formal feedback on personally acquired patient images, which may have limited the intervention’s impact.

In conclusion, internal medicine interns who received personal HUDs were not better at recognizing normal/abnormal findings on image assessments, and they did not report performing more POCUS examinations. Since the minority of a trainee’s time is spent toward direct patient care, offering trainees HUDs without substantial guidance may not be enough to promote mastery. Notably, trainees who received HUDs felt more confident in their abilities, despite no objective increase in their actual skill. Finally, interns who received POCUS-related lectures experienced significant benefit upon first exposure to the material, while repeated exposures did not improve performance. Future investigations should stringently track trainee POCUS usage rates with HUDs and assess whether image acquisition ability improves as a result of personal access.

Point-of-care ultrasonography (POCUS) can transform healthcare delivery through its diagnostic and therapeutic expediency.¹ POCUS has been shown to bolster diagnostic accuracy, reduce procedural complications, decrease inpatient length of stay, and improve patient satisfaction by encouraging the physician to be present at the bedside.^2-8

POCUS has become widespread across a variety of clinical settings as more investigations have demonstrated its positive impact on patient care.^1,9-12 This includes the use of POCUS by trainees, who are now utilizing this technology as part of their assessments of patients.^13,14 However, trainees may be performing these examinations with minimal oversight, and outside of emergency medicine, there are few guidelines on how to effectively teach POCUS or measure competency.^13,14 While POCUS is rapidly becoming a part of inpatient care, teaching physicians may have little experience in ultrasound or the expertise to adequately supervise trainees.¹⁴ There is a growing need to study what trainees can learn and how this knowledge is acquired.

Previous investigations have demonstrated that inexperienced users can be taught to use POCUS to identify a variety of pathological states.^2,3,15-23 Most of these curricula used a single lecture series as their pedagogical vehicle, and they variably included junior medical trainees. More importantly, the investigations did not explore whether personal access to handheld ultrasound devices (HUDs) improved learning. In theory, improved access to POCUS devices increases opportunities for authentic and deliberate practice, which may be needed to improve trainee skill with POCUS beyond the classroom setting.¹⁴

This study aimed to address several ongoing gaps in knowledge related to learning POCUS. First, we hypothesized that personal HUD access would improve trainees’ POCUS-­related knowledge and interpretive ability as a result of increased practice opportunities. Second, we hypothesized that trainees who receive personal access to HUDs would be more likely to perform POCUS examinations and feel more confident in their interpretations. Finally, we hypothesized that repeated exposure to POCUS-related lectures would result in greater improvements in knowledge as compared with a single lecture series.

Participants and Setting

The 2017 intern class (n = 47) at an academic internal medicine residency program participated in the study. Control data were obtained from the 2016 intern class (historical control; n = 50) and the 2018 intern class (contemporaneous control; n = 52). The Stanford University Institutional Review Board approved this study.

Study Design

The 2017 intern class (n = 47) received POCUS didactics from June 2017 to June 2018. To evaluate if increased access to HUDs improved learning outcomes, the 2017 interns were randomized 1:1 to receive their own personal HUD that could be used for patient care and/or self-directed learning (n = 24) vs no-HUD (n = 23; Figure). Learning outcomes were assessed over the course of 1 year (see “Outcomes” below) and were compared with the 2016 and 2018 controls. The 2016 intern class had completed a year of training but had not received formalized POCUS didactics (historical control), whereas the 2018 intern class was assessed at the beginning of their year (contemporaneous control; Figure). In order to make comparisons based on intern experience, baseline data for the 2017 intern class were compared with the 2018 intern class, whereas end-of-study data for 2017 interns were compared with 2016 interns.

Outcomes

The primary outcome was the difference in assessment scores at the end of the study period between interns randomized to receive a HUD and those who were not. Secondary outcomes included differences in HUD usage rates, lecture attendance, and assessment scores. To assess whether repeated lecture exposure resulted in greater amounts of learning, this study evaluated for assessment score improvements after each lecture block. Finally, trainee attitudes toward POCUS and their confidence in their interpretative ability were measured at the beginning and end of the study period.

Curriculum Implementation

The lectures were administered as once-weekly didactics of 1-hour duration to interns rotating on the inpatient wards rotation. This rotation is 4 weeks long, and each intern will experience the rotation two to four times per year. Each lecture contained two parts: (1) 20-30 minutes of didactics via Microsoft PowerPoint^TM and (2) 30-40 minutes of supervised practice using HUDs on standardized patients. Four lectures were given each month: (1) introduction to POCUS and ultrasound physics, (2) thoracic/lung ultrasound, (3) echocardiography, and (4) abdominal POCUS. The lectures consisted of contrasting cases of normal/abnormal videos and clinical vignettes. These four lectures were repeated each month as new interns rotated on service. Some interns experienced the same content multiple times, which was intentional in order to assess their rates of learning over time. Lecture contents were based on previously published guidelines and expert consensus for teaching POCUS in internal medicine.^{13, 24-26} Content from the Accreditation Council for Graduate Medical Education (ACGME) and the American College of Emergency Physicians (ACEP) was also incorporated because these organizations had published relevant guidelines for teaching POCUS.^13,26 Further development of the lectures occurred through review of previously described POCUS-relevant curricula.^27-32

Handheld Ultrasound Devices

This study used the Philips Lumify^TM, a United States Food and Drug Administration–approved device. Interns randomized to HUDs received their own device at the start of the rotation. It was at their discretion to use the device outside of the course. All devices were approved for patient use and were encrypted in compliance with our information security office. For privacy reasons, any saved patient images were not reviewed by the researchers. Interns were encouraged to share their findings with supervising physicians during rounds, but actual oversight was not measured. Interns not randomized to HUDs could access a single community device that was shared among all residents and fellows in the hospital. Interns reported the average number of POCUS examinations performed each week via a survey sent during the last week of the rotation.

Assessment Design and Implementation

Assessments evaluating trainee knowledge were administered before, during, and after the study period (Figure). For the 2017 cohort, assessments were also administered at the start and end of the ward month to track knowledge acquisition. Assessment contents were selected from POCUS guidelines for internal medicine and adaptation of the ACGME and ACEP guidelines.^13,24,26 Additional content was obtained from major society POCUS tutorials and deidentified images collected by the study authors.^13,24,33 In keeping with previously described methodology, the images were shown for approximately 12 seconds, followed by five additional seconds to allow the learner to answer the question.³² Final assessment contents were determined by the authors using the Delphi method.³⁴ A sample assessment can be found in the Appendix Material.

Surveys

Surveys were administered alongside the assessments to the 2016-2018 intern classes. These surveys assessed trainee attitudes toward POCUS and were based on previously validated assessments.^27,28,30 Attitudes were measured using 5-point Likert scales.

Statistical Analysis

For the primary outcome, we performed generalized binomial mixed-effect regressions using the survey periods, randomization group, and the interaction of the two as independent variables after adjusting for attendance and controlling of intra-intern correlations. The bivariate unadjusted analysis was performed to display the distribution of overall correctness on the assessments. Wilcoxon signed rank test was used to determine score significance for dependent score variables (R-­Statistical Programming Language, Vienna, Austria).

Baseline Characteristics

There were 149 interns who participated in this study (Figure). Assessment/survey completion rates were as follows: 2016 control: 68.0%; 2017 preintervention: 97.9%; 2017 postintervention: 89.4%; and 2018 control: 100%. The 2017 interns reported similar amounts of prior POCUS exposure in medical school (Table 1).

Primary Outcome: Assessment Scores (HUD vs no HUD)

There were no significant differences in assessment scores at the end of the study between interns randomized to personal HUD access vs those to no-HUD access (Table 1). HUD interns reported performing POCUS assessments on patients a mean 6.8 (standard deviation [SD] 2.2) times per week vs 6.4 (SD 2.9) times per week in the no-HUD arm (P = .66). The mean lecture attendance was 75.0% and did not significantly differ between the HUD arms (Table 1).

Secondary Outcomes

Impact of Repeating Lectures

The 2017 interns demonstrated significant increases in preblock vs postblock assessment scores after first-time exposure to the lectures (median preblock score 0.61 [interquartile range (IQR), 0.53-0.70] vs postblock score 0.81 [IQR, 0.72-0.86]; P < .001; Table 2). However, intern performance on the preblock vs postblock assessments after second-time exposure to the curriculum failed to improve (median second preblock score 0.78 [IQR, 0.69-0.83] vs postblock score 0.81 [IQR, 0.64-0.89]; P = .94). Intern performance on individual domains of knowledge for each block is listed in Appendix Table 1.

Intervention Performance vs Controls

The 2016 historical control had significantly higher scores compared with the 2017 preintervention group (P < .001; Appendix Table 2). The year-long lecture series resulted in significant increases in median scores for the 2017 group (median preintervention score 0.55 [0.41-0.61] vs median postintervention score 0.84 [0.71-0.90]; P = .006; Appendix Table 1). At the end of the study, the 2017 postintervention scores were significantly higher across multiple knowledge domains compared with the 2016 historical control (Appendix Table 2).

Survey Results

Notably, the 2017 intern class at the end of the intervention did not have significantly different assessment scores for several disease-specific domains, compared with the 2016 control (Appendix Table 2). Nonetheless, the 2017 intern class reported higher levels of confidence in these same domains despite similar scores (Supplementary Figure). The HUD group seldomly cited a lack of confidence in their abilities as a barrier to performing POCUS examinations (17.6%), compared with the no-HUD group (50.0%), despite nearly identical assessment scores between the two groups (Table 1).

Previous guidelines have recommended increased HUD access for learners,^13,24,35,36 but there have been few investigations that have evaluated the impact of such access on learning POCUS. One previous investigation found that hospitalists who carried HUDs were more likely to identify heart failure on bedside examination.³⁷ In contrast, our study found no improvement in interpretative ability when randomizing interns to carry HUDs for patient care. Notably, interns did not perform more POCUS examinations when given HUDs. We offer several explanations for this finding. First, time-motion studies have demonstrated that internal medicine interns spend less than 15% of their time toward direct patient care.³⁸ It is possible that the demands of being an intern impeded their ability to perform more POCUS examinations on their patients, regardless of HUD access. Alternatively, the interns randomized to no personal access may have used the community device more frequently as a result of the lecture series. Given the cost of HUDs, further studies are needed to assess the degree to which HUD access will improve trainee interpretive ability, especially as more training programs consider the creation of ultrasound curricula.^{10,11,24,39,40}

This study was unique because it followed interns over a year-long course that repeated the same material to assess rates of learning with repeated exposure. Learners improved their scores after the first, but not second, block. Furthermore, the median scores were nearly identical between the first postblock assessment and second preblock assessment (0.81 vs 0.78), suggesting that knowledge was retained between blocks. Together, these findings suggest there may be limitations of traditional lectures that use standardized patient models for practice. Supplementary pedagogies, such as in-the-moment feedback with actual patients, may be needed to promote mastery.^14,35

Despite no formal curriculum, the 2016 intern class (historical control) had learned POCUS to some degree based on their higher assessment scores compared with the 2017 intern class during the preintervention period. Such learning may be informal, and yet, trainees may feel confident in making clinical decisions without formalized training, accreditation, or oversight. As suggested by this study, adding regular didactics or giving trainees HUDs may not immediately solve this issue. For assessment items in which the 2017 interns did not significantly differ from the controls, they nonetheless reported higher confidence in their abilities. Similarly, interns randomized to HUDs less frequently cited a lack of confidence in their abilities, despite similar scores to the no-HUD group. Such confidence may be incongruent with their actual knowledge or ability to safely use POCUS. This phenomenon of misplaced confidence is known as the Dunning–Kruger effect, and it may be common with ultrasound learning.⁴¹ While confidence can be part of a holistic definition of competency,¹⁴ these results raise the concern that trainees may have difficulty assessing their own competency level with POCUS.³⁵

There are several limitations to this study. It was performed at a single institution with limited sample size. It examined only intern physicians because of funding constraints, which limits the generalizability of these findings among medical trainees. Technical ability assessments (including obtaining and interpreting images) were not included. We were unable to track the timing or location of the devices’ usage, and the interns’ self-reported usage rates may be subject to recall bias. To our knowledge, there were no significant lapses in device availability/functionality. Intern physicians in the HUD arm did not receive formal feedback on personally acquired patient images, which may have limited the intervention’s impact.

In conclusion, internal medicine interns who received personal HUDs were not better at recognizing normal/abnormal findings on image assessments, and they did not report performing more POCUS examinations. Since the minority of a trainee’s time is spent toward direct patient care, offering trainees HUDs without substantial guidance may not be enough to promote mastery. Notably, trainees who received HUDs felt more confident in their abilities, despite no objective increase in their actual skill. Finally, interns who received POCUS-related lectures experienced significant benefit upon first exposure to the material, while repeated exposures did not improve performance. Future investigations should stringently track trainee POCUS usage rates with HUDs and assess whether image acquisition ability improves as a result of personal access.

Point-of-care ultrasonography (POCUS) can transform healthcare delivery through its diagnostic and therapeutic expediency.¹ POCUS has been shown to bolster diagnostic accuracy, reduce procedural complications, decrease inpatient length of stay, and improve patient satisfaction by encouraging the physician to be present at the bedside.^2-8

POCUS has become widespread across a variety of clinical settings as more investigations have demonstrated its positive impact on patient care.^1,9-12 This includes the use of POCUS by trainees, who are now utilizing this technology as part of their assessments of patients.^13,14 However, trainees may be performing these examinations with minimal oversight, and outside of emergency medicine, there are few guidelines on how to effectively teach POCUS or measure competency.^13,14 While POCUS is rapidly becoming a part of inpatient care, teaching physicians may have little experience in ultrasound or the expertise to adequately supervise trainees.¹⁴ There is a growing need to study what trainees can learn and how this knowledge is acquired.

Previous investigations have demonstrated that inexperienced users can be taught to use POCUS to identify a variety of pathological states.^2,3,15-23 Most of these curricula used a single lecture series as their pedagogical vehicle, and they variably included junior medical trainees. More importantly, the investigations did not explore whether personal access to handheld ultrasound devices (HUDs) improved learning. In theory, improved access to POCUS devices increases opportunities for authentic and deliberate practice, which may be needed to improve trainee skill with POCUS beyond the classroom setting.¹⁴

This study aimed to address several ongoing gaps in knowledge related to learning POCUS. First, we hypothesized that personal HUD access would improve trainees’ POCUS-­related knowledge and interpretive ability as a result of increased practice opportunities. Second, we hypothesized that trainees who receive personal access to HUDs would be more likely to perform POCUS examinations and feel more confident in their interpretations. Finally, we hypothesized that repeated exposure to POCUS-related lectures would result in greater improvements in knowledge as compared with a single lecture series.

Participants and Setting

The 2017 intern class (n = 47) at an academic internal medicine residency program participated in the study. Control data were obtained from the 2016 intern class (historical control; n = 50) and the 2018 intern class (contemporaneous control; n = 52). The Stanford University Institutional Review Board approved this study.

Study Design

The 2017 intern class (n = 47) received POCUS didactics from June 2017 to June 2018. To evaluate if increased access to HUDs improved learning outcomes, the 2017 interns were randomized 1:1 to receive their own personal HUD that could be used for patient care and/or self-directed learning (n = 24) vs no-HUD (n = 23; Figure). Learning outcomes were assessed over the course of 1 year (see “Outcomes” below) and were compared with the 2016 and 2018 controls. The 2016 intern class had completed a year of training but had not received formalized POCUS didactics (historical control), whereas the 2018 intern class was assessed at the beginning of their year (contemporaneous control; Figure). In order to make comparisons based on intern experience, baseline data for the 2017 intern class were compared with the 2018 intern class, whereas end-of-study data for 2017 interns were compared with 2016 interns.

Outcomes

The primary outcome was the difference in assessment scores at the end of the study period between interns randomized to receive a HUD and those who were not. Secondary outcomes included differences in HUD usage rates, lecture attendance, and assessment scores. To assess whether repeated lecture exposure resulted in greater amounts of learning, this study evaluated for assessment score improvements after each lecture block. Finally, trainee attitudes toward POCUS and their confidence in their interpretative ability were measured at the beginning and end of the study period.

Curriculum Implementation

The lectures were administered as once-weekly didactics of 1-hour duration to interns rotating on the inpatient wards rotation. This rotation is 4 weeks long, and each intern will experience the rotation two to four times per year. Each lecture contained two parts: (1) 20-30 minutes of didactics via Microsoft PowerPoint^TM and (2) 30-40 minutes of supervised practice using HUDs on standardized patients. Four lectures were given each month: (1) introduction to POCUS and ultrasound physics, (2) thoracic/lung ultrasound, (3) echocardiography, and (4) abdominal POCUS. The lectures consisted of contrasting cases of normal/abnormal videos and clinical vignettes. These four lectures were repeated each month as new interns rotated on service. Some interns experienced the same content multiple times, which was intentional in order to assess their rates of learning over time. Lecture contents were based on previously published guidelines and expert consensus for teaching POCUS in internal medicine.^{13, 24-26} Content from the Accreditation Council for Graduate Medical Education (ACGME) and the American College of Emergency Physicians (ACEP) was also incorporated because these organizations had published relevant guidelines for teaching POCUS.^13,26 Further development of the lectures occurred through review of previously described POCUS-relevant curricula.^27-32

Handheld Ultrasound Devices

This study used the Philips Lumify^TM, a United States Food and Drug Administration–approved device. Interns randomized to HUDs received their own device at the start of the rotation. It was at their discretion to use the device outside of the course. All devices were approved for patient use and were encrypted in compliance with our information security office. For privacy reasons, any saved patient images were not reviewed by the researchers. Interns were encouraged to share their findings with supervising physicians during rounds, but actual oversight was not measured. Interns not randomized to HUDs could access a single community device that was shared among all residents and fellows in the hospital. Interns reported the average number of POCUS examinations performed each week via a survey sent during the last week of the rotation.

Assessment Design and Implementation

Assessments evaluating trainee knowledge were administered before, during, and after the study period (Figure). For the 2017 cohort, assessments were also administered at the start and end of the ward month to track knowledge acquisition. Assessment contents were selected from POCUS guidelines for internal medicine and adaptation of the ACGME and ACEP guidelines.^13,24,26 Additional content was obtained from major society POCUS tutorials and deidentified images collected by the study authors.^13,24,33 In keeping with previously described methodology, the images were shown for approximately 12 seconds, followed by five additional seconds to allow the learner to answer the question.³² Final assessment contents were determined by the authors using the Delphi method.³⁴ A sample assessment can be found in the Appendix Material.

Surveys

Surveys were administered alongside the assessments to the 2016-2018 intern classes. These surveys assessed trainee attitudes toward POCUS and were based on previously validated assessments.^27,28,30 Attitudes were measured using 5-point Likert scales.

Statistical Analysis

For the primary outcome, we performed generalized binomial mixed-effect regressions using the survey periods, randomization group, and the interaction of the two as independent variables after adjusting for attendance and controlling of intra-intern correlations. The bivariate unadjusted analysis was performed to display the distribution of overall correctness on the assessments. Wilcoxon signed rank test was used to determine score significance for dependent score variables (R-­Statistical Programming Language, Vienna, Austria).

Baseline Characteristics

There were 149 interns who participated in this study (Figure). Assessment/survey completion rates were as follows: 2016 control: 68.0%; 2017 preintervention: 97.9%; 2017 postintervention: 89.4%; and 2018 control: 100%. The 2017 interns reported similar amounts of prior POCUS exposure in medical school (Table 1).

Primary Outcome: Assessment Scores (HUD vs no HUD)

There were no significant differences in assessment scores at the end of the study between interns randomized to personal HUD access vs those to no-HUD access (Table 1). HUD interns reported performing POCUS assessments on patients a mean 6.8 (standard deviation [SD] 2.2) times per week vs 6.4 (SD 2.9) times per week in the no-HUD arm (P = .66). The mean lecture attendance was 75.0% and did not significantly differ between the HUD arms (Table 1).

Secondary Outcomes

Impact of Repeating Lectures

The 2017 interns demonstrated significant increases in preblock vs postblock assessment scores after first-time exposure to the lectures (median preblock score 0.61 [interquartile range (IQR), 0.53-0.70] vs postblock score 0.81 [IQR, 0.72-0.86]; P < .001; Table 2). However, intern performance on the preblock vs postblock assessments after second-time exposure to the curriculum failed to improve (median second preblock score 0.78 [IQR, 0.69-0.83] vs postblock score 0.81 [IQR, 0.64-0.89]; P = .94). Intern performance on individual domains of knowledge for each block is listed in Appendix Table 1.

Intervention Performance vs Controls

The 2016 historical control had significantly higher scores compared with the 2017 preintervention group (P < .001; Appendix Table 2). The year-long lecture series resulted in significant increases in median scores for the 2017 group (median preintervention score 0.55 [0.41-0.61] vs median postintervention score 0.84 [0.71-0.90]; P = .006; Appendix Table 1). At the end of the study, the 2017 postintervention scores were significantly higher across multiple knowledge domains compared with the 2016 historical control (Appendix Table 2).

Survey Results

Notably, the 2017 intern class at the end of the intervention did not have significantly different assessment scores for several disease-specific domains, compared with the 2016 control (Appendix Table 2). Nonetheless, the 2017 intern class reported higher levels of confidence in these same domains despite similar scores (Supplementary Figure). The HUD group seldomly cited a lack of confidence in their abilities as a barrier to performing POCUS examinations (17.6%), compared with the no-HUD group (50.0%), despite nearly identical assessment scores between the two groups (Table 1).

Previous guidelines have recommended increased HUD access for learners,^13,24,35,36 but there have been few investigations that have evaluated the impact of such access on learning POCUS. One previous investigation found that hospitalists who carried HUDs were more likely to identify heart failure on bedside examination.³⁷ In contrast, our study found no improvement in interpretative ability when randomizing interns to carry HUDs for patient care. Notably, interns did not perform more POCUS examinations when given HUDs. We offer several explanations for this finding. First, time-motion studies have demonstrated that internal medicine interns spend less than 15% of their time toward direct patient care.³⁸ It is possible that the demands of being an intern impeded their ability to perform more POCUS examinations on their patients, regardless of HUD access. Alternatively, the interns randomized to no personal access may have used the community device more frequently as a result of the lecture series. Given the cost of HUDs, further studies are needed to assess the degree to which HUD access will improve trainee interpretive ability, especially as more training programs consider the creation of ultrasound curricula.^{10,11,24,39,40}

This study was unique because it followed interns over a year-long course that repeated the same material to assess rates of learning with repeated exposure. Learners improved their scores after the first, but not second, block. Furthermore, the median scores were nearly identical between the first postblock assessment and second preblock assessment (0.81 vs 0.78), suggesting that knowledge was retained between blocks. Together, these findings suggest there may be limitations of traditional lectures that use standardized patient models for practice. Supplementary pedagogies, such as in-the-moment feedback with actual patients, may be needed to promote mastery.^14,35

Despite no formal curriculum, the 2016 intern class (historical control) had learned POCUS to some degree based on their higher assessment scores compared with the 2017 intern class during the preintervention period. Such learning may be informal, and yet, trainees may feel confident in making clinical decisions without formalized training, accreditation, or oversight. As suggested by this study, adding regular didactics or giving trainees HUDs may not immediately solve this issue. For assessment items in which the 2017 interns did not significantly differ from the controls, they nonetheless reported higher confidence in their abilities. Similarly, interns randomized to HUDs less frequently cited a lack of confidence in their abilities, despite similar scores to the no-HUD group. Such confidence may be incongruent with their actual knowledge or ability to safely use POCUS. This phenomenon of misplaced confidence is known as the Dunning–Kruger effect, and it may be common with ultrasound learning.⁴¹ While confidence can be part of a holistic definition of competency,¹⁴ these results raise the concern that trainees may have difficulty assessing their own competency level with POCUS.³⁵

There are several limitations to this study. It was performed at a single institution with limited sample size. It examined only intern physicians because of funding constraints, which limits the generalizability of these findings among medical trainees. Technical ability assessments (including obtaining and interpreting images) were not included. We were unable to track the timing or location of the devices’ usage, and the interns’ self-reported usage rates may be subject to recall bias. To our knowledge, there were no significant lapses in device availability/functionality. Intern physicians in the HUD arm did not receive formal feedback on personally acquired patient images, which may have limited the intervention’s impact.

In conclusion, internal medicine interns who received personal HUDs were not better at recognizing normal/abnormal findings on image assessments, and they did not report performing more POCUS examinations. Since the minority of a trainee’s time is spent toward direct patient care, offering trainees HUDs without substantial guidance may not be enough to promote mastery. Notably, trainees who received HUDs felt more confident in their abilities, despite no objective increase in their actual skill. Finally, interns who received POCUS-related lectures experienced significant benefit upon first exposure to the material, while repeated exposures did not improve performance. Future investigations should stringently track trainee POCUS usage rates with HUDs and assess whether image acquisition ability improves as a result of personal access.

In surgical comanagement (SCM), surgeons and hospitalists share responsibility of care for surgical patients. While SCM has been increasingly utilized, many of the reported models are a modification of the consultation model, in which a group of rotating hospitalists, internists, or geriatricians care for the surgical patients, often after medical complications may have occured.^1-4

In August 2012, we implemented SCM in Orthopedic and Neurosurgery services at our institution.⁵ This model is unique because the same Internal Medicine hospitalists are dedicated year round to the same surgical service. SCM hospitalists see patients on their assigned surgical service only; they do not see patients on the Internal Medicine service. After the first year of implementing SCM, we conducted a propensity score–weighted study with 17,057 discharges in the pre-SCM group (January 2009 to July 2012) and 5,533 discharges in the post-SCM group (September 2012 to September 2013).⁵ In this study, SCM was associated with a decrease in medical complications, length of stay (LOS), medical consultations, 30-day readmissions, and cost.⁵

Since SCM requires ongoing investment by institutions, we now report a follow-up study to explore if there were continued improvements in patient outcomes with SCM. In this study, we evaluate if there was a decrease in medical complications, LOS, number of medical consultations, rapid response team calls, and code blues and an increase in patient satisfaction with SCM in Orthopedic and Neurosurgery services between 2012 and 2018.

We included 26,380 discharges from Orthopedic and Neurosurgery services between September 1, 2012, and June 30, 2018, at our academic medical center. We excluded patients discharged in August 2012 as we transitioned to the SCM model. Our Institutional Review Board exempted this study from further review.

SCM Structure

SCM structure was detailed in a prior article.⁵ We have 3.0 clinical full-time equivalents on the Orthopedic surgery SCM service and 1.2 on the Neurosurgery SCM service. On weekdays, during the day (8 am to 5 pm), there are two SCM hospitalists on Orthopedic surgery service and one on Neurosurgery service. One SCM hospitalist is on call every week and takes after-hours calls from both surgical services and sees patients on both services on the weekend.

During the day, SCM hospitalists receive the first call for medical issues. After 5 pm and on weekends and holidays, surgical services take all calls first and reach out to the on-call SCM hospitalist for any medical issues for which they need assistance. Surgery service is the primary team and does the discharge summaries. SCM hospitalists write any medical orders as needed. Medical students, physician assistant students, medicine housestaff, and geriatric medicine fellows rotate through SCM. SCM hospitalists directly communicate with the surgical service and not through the learners. There are no advanced practice providers on SCM service. Surgery housestaff attend the multidisciplinary team care rounds with the case manager, social worker, rehabilitation services, and pharmacy with ad hoc presence of SCM hospitalists for selected patients. SCM hospitalists often see sick patients with the surgery service at the bedside, and they work together with the surgery service on order sets, quality improvement projects, and scholarly work.

SCM hospitalists screen the entire patient list on their assigned surgery service each day. After screening the patient list, SCM hospitalists formally see select patients with preventable or active medical conditions and write notes on the patient’s chart. There are no set criteria to determine which patients would be seen by SCM. This is because surgeries can decompensate stable medical conditions or new unexpected medical complications may occur. Additionally, in our prior study, we reported that SCM reduced medical complications and LOS regardless of age or patient acuity.⁵

Outcomes

Our primary outcome was proportion of patients with ≥1 medical complication (sepsis, pneumonia, urinary tract infection, delirium, acute kidney injury, atrial fibrillation, or ileus). Our secondary outcomes included mean LOS, proportion of patients with ≥2 medical consultations, rapid response team calls, code blues, and top-box patient satisfaction score. Though cost is an important consideration in implementing SCM, limited financial data were available. However, since LOS is a key component in calculating direct costs,⁶ we estimated the cost savings per discharge using mean direct cost per day and the difference in mean LOS between pre- and post-SCM groups.⁵

We defined medical complications using International Classification of Disease (ICD) Codes 9 or 10 that were coded as “not present on admission” (Appendix 1). We used Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey for three questions for patient satisfaction: Did doctors treat with courtesy and respect, listen carefully, and explain things in a way you could understand?

Statistical Analysis

We used regression analysis to assess trends in patient characteristics by year (Appendix 2). Logistic regression with logit link was used to assess the yearly change in our binary outcomes (proportion of patients with ≥1 medical complication, those with ≥2 medical consultations, rapid response team calls, code blue, and top-box patient satisfaction score) and reported odds ratios. Gamma regression with identity link was performed for our continuous outcome (LOS). Beta coefficient was reported to estimate the yearly change in LOS under their original scales. Age, primary insurance, race, Charlson comorbidity score, general or regional anesthesia, surgical service, and duration of surgery were adjusted in the regression analyses for outcomes. SAS 9.4 was used for analysis.

Patient characteristics are shown in Table 1. Overall, 62.8% patients were discharged from Orthopedic surgery service, 72.5% patients underwent elective surgery, and 88.8% received general anesthesia. Between 2012 and 2018, there was a significant increase in the median age of patients (from 60 years to 63 years), mean Charlson comorbidity score increased from 1.07 to 1.46, and median case mix index, a measure of patient acuity, increased from 2.10 to 2.36 (Appendix 2).

Comparing pre-SCM unadjusted rates reported in our prior study (January 2009 to July 2012) to post-SCM (September 2012 to June 2018; Appendix 3), patients with ≥1 medical complication decreased from 10.1% to 6.1%, LOS (mean ± standard deviation) changed from 5.4 ± 2.2 days to 4.6 ± 5.8 days, patients with ≥2 medical consultations decreased from 19.4% to 9.2%, rapid response team calls changed from 1% to 0.9%, code blues changed from 0.3% to 0.2%, and patients with top-box patient satisfaction score increased from 86.4% to 94.2%.⁵

In the adjusted analysis from 2012 to 2018, the odds of patients with ≥1 medical complication decreased by 3.8% per year (P = .01), estimated LOS decreased by 0.3 days per year (P < .0001), and the odds of rapid response team calls decreased by 12.2% per year (P = .001; Table 2). Changes over time in the odds of patients with ≥2 medical consultations, code blues, or top-box patient satisfaction score were not statistically significant (Table 2). Based on the LOS reduction pre- to post-SCM, there were estimated average direct cost savings of $3,424 per discharge between 2012 and 2018.

Since the implementation of SCM on Orthopedic and Neurosurgery services at our institution, there was a decrease in medical complications, LOS, and rapid response team calls. To our knowledge, this is one of the largest studies evaluating the benefits of SCM over 5.8 years. Similar to our prior studies on this SCM model of care,^5,7 other studies have reported a decrease in medical complications,^8-10 LOS,^11-13 and cost of care¹⁴ with SCM.

While the changes in the unadjusted rates of outcomes over the years appeared to be small, while our patient population became older and sicker, there were significant changes in several of our outcomes in the adjusted analysis. We believe that SCM hospitalists have developed a skill set and understanding of these surgical patients over time and can manage more medically complex patients without an increase in medical complications or LOS. We attribute this to our unique SCM model in which the same hospitalists stay year round on the same surgical service. SCM hospitalists have built trusting relationships with the surgical team with greater involvement in decision making, care planning, and patient selection. With minimal turnover in the SCM group and with ongoing learning, SCM hospitalists can anticipate fluid or pain medication requirements after specific surgeries and the surgery-specific medical complications. SCM hospitalists are available on the patient units to provide timely intervention in case of medical deterioration; answer any questions from patients, families, or nursing while the surgical teams may be in the operating room; and coordinate with other medical consultants or outpatient providers as needed.

This study has several limitations. This is a single-center study at an academic institution, limited to two surgical services. We did not have a control group and multiple hospital-­wide interventions may have affected these outcomes. This is an observational study in which unobserved variables may bias the results. We used ICD codes to identify medical complications, which relies on the quality of physician documentation. While our response rate of 21.1% for HCAHPS was comparable to the national average of 26.7%, it may not reliably represent our patient population.¹⁵ Lastly, we had limited financial data.

With the move toward value-based payment and increasing medical complexity of surgical patients, SCM by hospitalists may deliver high-quality care.

In surgical comanagement (SCM), surgeons and hospitalists share responsibility of care for surgical patients. While SCM has been increasingly utilized, many of the reported models are a modification of the consultation model, in which a group of rotating hospitalists, internists, or geriatricians care for the surgical patients, often after medical complications may have occured.^1-4

In August 2012, we implemented SCM in Orthopedic and Neurosurgery services at our institution.⁵ This model is unique because the same Internal Medicine hospitalists are dedicated year round to the same surgical service. SCM hospitalists see patients on their assigned surgical service only; they do not see patients on the Internal Medicine service. After the first year of implementing SCM, we conducted a propensity score–weighted study with 17,057 discharges in the pre-SCM group (January 2009 to July 2012) and 5,533 discharges in the post-SCM group (September 2012 to September 2013).⁵ In this study, SCM was associated with a decrease in medical complications, length of stay (LOS), medical consultations, 30-day readmissions, and cost.⁵

Since SCM requires ongoing investment by institutions, we now report a follow-up study to explore if there were continued improvements in patient outcomes with SCM. In this study, we evaluate if there was a decrease in medical complications, LOS, number of medical consultations, rapid response team calls, and code blues and an increase in patient satisfaction with SCM in Orthopedic and Neurosurgery services between 2012 and 2018.

We included 26,380 discharges from Orthopedic and Neurosurgery services between September 1, 2012, and June 30, 2018, at our academic medical center. We excluded patients discharged in August 2012 as we transitioned to the SCM model. Our Institutional Review Board exempted this study from further review.

SCM Structure

SCM structure was detailed in a prior article.⁵ We have 3.0 clinical full-time equivalents on the Orthopedic surgery SCM service and 1.2 on the Neurosurgery SCM service. On weekdays, during the day (8 am to 5 pm), there are two SCM hospitalists on Orthopedic surgery service and one on Neurosurgery service. One SCM hospitalist is on call every week and takes after-hours calls from both surgical services and sees patients on both services on the weekend.

During the day, SCM hospitalists receive the first call for medical issues. After 5 pm and on weekends and holidays, surgical services take all calls first and reach out to the on-call SCM hospitalist for any medical issues for which they need assistance. Surgery service is the primary team and does the discharge summaries. SCM hospitalists write any medical orders as needed. Medical students, physician assistant students, medicine housestaff, and geriatric medicine fellows rotate through SCM. SCM hospitalists directly communicate with the surgical service and not through the learners. There are no advanced practice providers on SCM service. Surgery housestaff attend the multidisciplinary team care rounds with the case manager, social worker, rehabilitation services, and pharmacy with ad hoc presence of SCM hospitalists for selected patients. SCM hospitalists often see sick patients with the surgery service at the bedside, and they work together with the surgery service on order sets, quality improvement projects, and scholarly work.

SCM hospitalists screen the entire patient list on their assigned surgery service each day. After screening the patient list, SCM hospitalists formally see select patients with preventable or active medical conditions and write notes on the patient’s chart. There are no set criteria to determine which patients would be seen by SCM. This is because surgeries can decompensate stable medical conditions or new unexpected medical complications may occur. Additionally, in our prior study, we reported that SCM reduced medical complications and LOS regardless of age or patient acuity.⁵

Outcomes

Our primary outcome was proportion of patients with ≥1 medical complication (sepsis, pneumonia, urinary tract infection, delirium, acute kidney injury, atrial fibrillation, or ileus). Our secondary outcomes included mean LOS, proportion of patients with ≥2 medical consultations, rapid response team calls, code blues, and top-box patient satisfaction score. Though cost is an important consideration in implementing SCM, limited financial data were available. However, since LOS is a key component in calculating direct costs,⁶ we estimated the cost savings per discharge using mean direct cost per day and the difference in mean LOS between pre- and post-SCM groups.⁵

We defined medical complications using International Classification of Disease (ICD) Codes 9 or 10 that were coded as “not present on admission” (Appendix 1). We used Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey for three questions for patient satisfaction: Did doctors treat with courtesy and respect, listen carefully, and explain things in a way you could understand?

Statistical Analysis

We used regression analysis to assess trends in patient characteristics by year (Appendix 2). Logistic regression with logit link was used to assess the yearly change in our binary outcomes (proportion of patients with ≥1 medical complication, those with ≥2 medical consultations, rapid response team calls, code blue, and top-box patient satisfaction score) and reported odds ratios. Gamma regression with identity link was performed for our continuous outcome (LOS). Beta coefficient was reported to estimate the yearly change in LOS under their original scales. Age, primary insurance, race, Charlson comorbidity score, general or regional anesthesia, surgical service, and duration of surgery were adjusted in the regression analyses for outcomes. SAS 9.4 was used for analysis.

Patient characteristics are shown in Table 1. Overall, 62.8% patients were discharged from Orthopedic surgery service, 72.5% patients underwent elective surgery, and 88.8% received general anesthesia. Between 2012 and 2018, there was a significant increase in the median age of patients (from 60 years to 63 years), mean Charlson comorbidity score increased from 1.07 to 1.46, and median case mix index, a measure of patient acuity, increased from 2.10 to 2.36 (Appendix 2).

Comparing pre-SCM unadjusted rates reported in our prior study (January 2009 to July 2012) to post-SCM (September 2012 to June 2018; Appendix 3), patients with ≥1 medical complication decreased from 10.1% to 6.1%, LOS (mean ± standard deviation) changed from 5.4 ± 2.2 days to 4.6 ± 5.8 days, patients with ≥2 medical consultations decreased from 19.4% to 9.2%, rapid response team calls changed from 1% to 0.9%, code blues changed from 0.3% to 0.2%, and patients with top-box patient satisfaction score increased from 86.4% to 94.2%.⁵

In the adjusted analysis from 2012 to 2018, the odds of patients with ≥1 medical complication decreased by 3.8% per year (P = .01), estimated LOS decreased by 0.3 days per year (P < .0001), and the odds of rapid response team calls decreased by 12.2% per year (P = .001; Table 2). Changes over time in the odds of patients with ≥2 medical consultations, code blues, or top-box patient satisfaction score were not statistically significant (Table 2). Based on the LOS reduction pre- to post-SCM, there were estimated average direct cost savings of $3,424 per discharge between 2012 and 2018.

Since the implementation of SCM on Orthopedic and Neurosurgery services at our institution, there was a decrease in medical complications, LOS, and rapid response team calls. To our knowledge, this is one of the largest studies evaluating the benefits of SCM over 5.8 years. Similar to our prior studies on this SCM model of care,^5,7 other studies have reported a decrease in medical complications,^8-10 LOS,^11-13 and cost of care¹⁴ with SCM.

While the changes in the unadjusted rates of outcomes over the years appeared to be small, while our patient population became older and sicker, there were significant changes in several of our outcomes in the adjusted analysis. We believe that SCM hospitalists have developed a skill set and understanding of these surgical patients over time and can manage more medically complex patients without an increase in medical complications or LOS. We attribute this to our unique SCM model in which the same hospitalists stay year round on the same surgical service. SCM hospitalists have built trusting relationships with the surgical team with greater involvement in decision making, care planning, and patient selection. With minimal turnover in the SCM group and with ongoing learning, SCM hospitalists can anticipate fluid or pain medication requirements after specific surgeries and the surgery-specific medical complications. SCM hospitalists are available on the patient units to provide timely intervention in case of medical deterioration; answer any questions from patients, families, or nursing while the surgical teams may be in the operating room; and coordinate with other medical consultants or outpatient providers as needed.

This study has several limitations. This is a single-center study at an academic institution, limited to two surgical services. We did not have a control group and multiple hospital-­wide interventions may have affected these outcomes. This is an observational study in which unobserved variables may bias the results. We used ICD codes to identify medical complications, which relies on the quality of physician documentation. While our response rate of 21.1% for HCAHPS was comparable to the national average of 26.7%, it may not reliably represent our patient population.¹⁵ Lastly, we had limited financial data.

With the move toward value-based payment and increasing medical complexity of surgical patients, SCM by hospitalists may deliver high-quality care.

In surgical comanagement (SCM), surgeons and hospitalists share responsibility of care for surgical patients. While SCM has been increasingly utilized, many of the reported models are a modification of the consultation model, in which a group of rotating hospitalists, internists, or geriatricians care for the surgical patients, often after medical complications may have occured.^1-4

In August 2012, we implemented SCM in Orthopedic and Neurosurgery services at our institution.⁵ This model is unique because the same Internal Medicine hospitalists are dedicated year round to the same surgical service. SCM hospitalists see patients on their assigned surgical service only; they do not see patients on the Internal Medicine service. After the first year of implementing SCM, we conducted a propensity score–weighted study with 17,057 discharges in the pre-SCM group (January 2009 to July 2012) and 5,533 discharges in the post-SCM group (September 2012 to September 2013).⁵ In this study, SCM was associated with a decrease in medical complications, length of stay (LOS), medical consultations, 30-day readmissions, and cost.⁵

Since SCM requires ongoing investment by institutions, we now report a follow-up study to explore if there were continued improvements in patient outcomes with SCM. In this study, we evaluate if there was a decrease in medical complications, LOS, number of medical consultations, rapid response team calls, and code blues and an increase in patient satisfaction with SCM in Orthopedic and Neurosurgery services between 2012 and 2018.

We included 26,380 discharges from Orthopedic and Neurosurgery services between September 1, 2012, and June 30, 2018, at our academic medical center. We excluded patients discharged in August 2012 as we transitioned to the SCM model. Our Institutional Review Board exempted this study from further review.

SCM Structure

SCM structure was detailed in a prior article.⁵ We have 3.0 clinical full-time equivalents on the Orthopedic surgery SCM service and 1.2 on the Neurosurgery SCM service. On weekdays, during the day (8 am to 5 pm), there are two SCM hospitalists on Orthopedic surgery service and one on Neurosurgery service. One SCM hospitalist is on call every week and takes after-hours calls from both surgical services and sees patients on both services on the weekend.

During the day, SCM hospitalists receive the first call for medical issues. After 5 pm and on weekends and holidays, surgical services take all calls first and reach out to the on-call SCM hospitalist for any medical issues for which they need assistance. Surgery service is the primary team and does the discharge summaries. SCM hospitalists write any medical orders as needed. Medical students, physician assistant students, medicine housestaff, and geriatric medicine fellows rotate through SCM. SCM hospitalists directly communicate with the surgical service and not through the learners. There are no advanced practice providers on SCM service. Surgery housestaff attend the multidisciplinary team care rounds with the case manager, social worker, rehabilitation services, and pharmacy with ad hoc presence of SCM hospitalists for selected patients. SCM hospitalists often see sick patients with the surgery service at the bedside, and they work together with the surgery service on order sets, quality improvement projects, and scholarly work.

SCM hospitalists screen the entire patient list on their assigned surgery service each day. After screening the patient list, SCM hospitalists formally see select patients with preventable or active medical conditions and write notes on the patient’s chart. There are no set criteria to determine which patients would be seen by SCM. This is because surgeries can decompensate stable medical conditions or new unexpected medical complications may occur. Additionally, in our prior study, we reported that SCM reduced medical complications and LOS regardless of age or patient acuity.⁵

Outcomes

Our primary outcome was proportion of patients with ≥1 medical complication (sepsis, pneumonia, urinary tract infection, delirium, acute kidney injury, atrial fibrillation, or ileus). Our secondary outcomes included mean LOS, proportion of patients with ≥2 medical consultations, rapid response team calls, code blues, and top-box patient satisfaction score. Though cost is an important consideration in implementing SCM, limited financial data were available. However, since LOS is a key component in calculating direct costs,⁶ we estimated the cost savings per discharge using mean direct cost per day and the difference in mean LOS between pre- and post-SCM groups.⁵

We defined medical complications using International Classification of Disease (ICD) Codes 9 or 10 that were coded as “not present on admission” (Appendix 1). We used Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey for three questions for patient satisfaction: Did doctors treat with courtesy and respect, listen carefully, and explain things in a way you could understand?

Statistical Analysis

We used regression analysis to assess trends in patient characteristics by year (Appendix 2). Logistic regression with logit link was used to assess the yearly change in our binary outcomes (proportion of patients with ≥1 medical complication, those with ≥2 medical consultations, rapid response team calls, code blue, and top-box patient satisfaction score) and reported odds ratios. Gamma regression with identity link was performed for our continuous outcome (LOS). Beta coefficient was reported to estimate the yearly change in LOS under their original scales. Age, primary insurance, race, Charlson comorbidity score, general or regional anesthesia, surgical service, and duration of surgery were adjusted in the regression analyses for outcomes. SAS 9.4 was used for analysis.

Patient characteristics are shown in Table 1. Overall, 62.8% patients were discharged from Orthopedic surgery service, 72.5% patients underwent elective surgery, and 88.8% received general anesthesia. Between 2012 and 2018, there was a significant increase in the median age of patients (from 60 years to 63 years), mean Charlson comorbidity score increased from 1.07 to 1.46, and median case mix index, a measure of patient acuity, increased from 2.10 to 2.36 (Appendix 2).

Comparing pre-SCM unadjusted rates reported in our prior study (January 2009 to July 2012) to post-SCM (September 2012 to June 2018; Appendix 3), patients with ≥1 medical complication decreased from 10.1% to 6.1%, LOS (mean ± standard deviation) changed from 5.4 ± 2.2 days to 4.6 ± 5.8 days, patients with ≥2 medical consultations decreased from 19.4% to 9.2%, rapid response team calls changed from 1% to 0.9%, code blues changed from 0.3% to 0.2%, and patients with top-box patient satisfaction score increased from 86.4% to 94.2%.⁵

In the adjusted analysis from 2012 to 2018, the odds of patients with ≥1 medical complication decreased by 3.8% per year (P = .01), estimated LOS decreased by 0.3 days per year (P < .0001), and the odds of rapid response team calls decreased by 12.2% per year (P = .001; Table 2). Changes over time in the odds of patients with ≥2 medical consultations, code blues, or top-box patient satisfaction score were not statistically significant (Table 2). Based on the LOS reduction pre- to post-SCM, there were estimated average direct cost savings of $3,424 per discharge between 2012 and 2018.

Since the implementation of SCM on Orthopedic and Neurosurgery services at our institution, there was a decrease in medical complications, LOS, and rapid response team calls. To our knowledge, this is one of the largest studies evaluating the benefits of SCM over 5.8 years. Similar to our prior studies on this SCM model of care,^5,7 other studies have reported a decrease in medical complications,^8-10 LOS,^11-13 and cost of care¹⁴ with SCM.

While the changes in the unadjusted rates of outcomes over the years appeared to be small, while our patient population became older and sicker, there were significant changes in several of our outcomes in the adjusted analysis. We believe that SCM hospitalists have developed a skill set and understanding of these surgical patients over time and can manage more medically complex patients without an increase in medical complications or LOS. We attribute this to our unique SCM model in which the same hospitalists stay year round on the same surgical service. SCM hospitalists have built trusting relationships with the surgical team with greater involvement in decision making, care planning, and patient selection. With minimal turnover in the SCM group and with ongoing learning, SCM hospitalists can anticipate fluid or pain medication requirements after specific surgeries and the surgery-specific medical complications. SCM hospitalists are available on the patient units to provide timely intervention in case of medical deterioration; answer any questions from patients, families, or nursing while the surgical teams may be in the operating room; and coordinate with other medical consultants or outpatient providers as needed.

This study has several limitations. This is a single-center study at an academic institution, limited to two surgical services. We did not have a control group and multiple hospital-­wide interventions may have affected these outcomes. This is an observational study in which unobserved variables may bias the results. We used ICD codes to identify medical complications, which relies on the quality of physician documentation. While our response rate of 21.1% for HCAHPS was comparable to the national average of 26.7%, it may not reliably represent our patient population.¹⁵ Lastly, we had limited financial data.

With the move toward value-based payment and increasing medical complexity of surgical patients, SCM by hospitalists may deliver high-quality care.

Almost all specialties in internal medicine have a sound scientific research base through which clinical practice is informed.¹ For the field of Hospital Medicine (HM), this evidence has largely comprised research generated from fields outside of the specialty. The need to develop, invest, and grow investigators in hospital-based medicine remains unmet as HM and its footprint in hospital systems continue to grow.^2,3

Despite this fact, little is known about the current state of research in HM. A 2014 survey of the members of the Society of Hospital Medicine (SHM) found that research output across the field of HM, as measured on the basis of peer-reviewed publications, was growing.⁴ Since then, however, the numbers of individuals engaged in research activities, their background and training, publication output, or funding sources have not been quantified. Similarly, little is known about which institutions support the development of junior investigators (ie, HM research fellowships), how these programs are funded, and whether or not matriculants enter the field as investigators. These gaps must be measured, evaluated, and ideally addressed through strategic policy and funding initiatives to advance the state of science within HM.

Members of the SHM Research Committee developed, designed, and deployed a survey to improve the understanding of the state of research in HM. In this study, we aimed to establish the baseline of research in HM to enable the measurement of progress through periodic waves of data collection. Specifically, we sought to quantify and describe the characteristics of existing research programs, the sources and types of funding, the number and background of faculty, and the availability of resources for training researchers in HM.

Study Setting and Participants

Given that no defined list, database, or external resource that identifies research programs and contacts in HM exists, we began by creating a strategy to identify and sample adult HM programs and their leaders engaged in research activity. We iteratively developed a two-step approach to maximize inclusivity. First, we partnered with SHM to identify programs and leaders actively engaging in research activities. SHM is the largest professional organization within HM and maintains an extensive membership database that includes the titles, e-mail addresses, and affiliations of hospitalists in the United States, including academic and nonacademic sites. This list was manually scanned, and the leaders of academic and research programs in adult HM were identified by examining their titles (eg, Division Chief, Research Lead, etc.) and academic affiliations. During this step, members of the committee noticed that certain key individuals were either missing, no longer occupying their role/title, or had been replaced by others. Therefore, we performed a second step and asked the members of the SHM Research Committee to identify academic and research leaders by using current personal contacts, publication history, and social networks. We asked members to identify individuals and programs that had received grant funding, were actively presenting research at SHM (or other major national venues), and/or were producing peer-reviewed publications related to HM. These programs were purposefully chosen (ie, over HM programs known for clinical activities) to create an enriched sample of those engaged in research in HM. The research committee performed the “second pass” to ensure that established investigators who may not be accurately captured within the SHM database were included to maximize yield for the survey. Finally, these two sources were merged to ensure the absence of duplicate contacts and the identification of a primary respondent for each affiliate. As a result, a convenience sample of 100 programs and corresponding individuals was compiled for the purposes of this survey.

Survey Development

A workgroup within the SHM Research Committee was tasked to create a survey that would achieve four distinct goals: (1) identify institutions currently engaging in hospital-based research; (2) define the characteristics, including sources of research funding, training opportunities, criteria for promotion, and grant support, of research programs within institutions; (3) understand the prevalence of research fellowship programs, including size, training curricula, and funding sources; and (4) evaluate the productivity and funding sources of HM investigators at each site.

Survey questions that target each of these domains were drafted by the workgroup. Questions were pretested with colleagues outside the workgroup focused on this project (ie, from the main research committee). The instrument was refined and edited to improve the readability and clarity of questions on the basis of the feedback obtained through the iterative process. The revised instrument was then programmed into an online survey administration tool (SurveyMonkey^®) to facilitate electronic dissemination. Finally, the members of the workgroup tested the online survey to ensure functionality. No identifiable information was collected from respondents, and no monetary incentive was offered for the completion of the survey. An invitation to participate in the survey was sent via e-mail to each of the program contacts identified.

Statistical Analysis

Descriptive statistics, including proportions, means, and percentages, were used to tabulate results. All analyses were conducted using Stata 13 MP/SE (StataCorp, College Station, Texas).

Ethical and Regulatory Considerations

The study was reviewed and deemed exempt from regulation by the University of Michigan Institutional Review Board (HUM000138628).

General Characteristics of Research Programs and Faculty

Out of 100 program contacts, 28 (representing 1,586 faculty members) responded and were included in the survey (program response rate = 28%). When comparing programs that did respond with those that did not, a greater proportion of programs in university settings were noted among respondents (79% vs 21%). Respondents represented programs from all regions of the United States, with most representing university-based (79%), university-affiliated (14%) or Veterans Health Administration (VHA; 11%) programs. Most respondents were in leadership roles, including division chiefs (32%), research directors/leads (21%), section chiefs (18%), and related titles, such as program director. Respondents indicated that the total number of faculty members in their programs (including nonclinicians and advance practice providers) varied from eight to 152 (mean [SD] = 57 [36]) members, with physicians representing the majority of faculty members (Table 1).

Among the 1,586 faculty members within the 28 programs, respondents identified 192 faculty members (12%) as currently receiving extra- or intramural support for research activities. Of these faculty, over half (58%) received <25% of effort from intra or extramural sources, and 28 (15%) and 52 (27%) faculty members received 25%-50% or >50% of support for their effort, respectively. The number of investigators who received funding across programs ranged from 0 to 28 faculty members. Compared with the 192 funded investigators, respondents indicated that a larger number of faculty in their programs (n = 656 or 41%) were involved in local quality improvement (QI) efforts. Of the 656 faculty members involved in QI efforts, 241 individuals (37%) were internally funded and received protected time/effort for their work.

Key Attributes of Research Programs

In the evaluation of the amount of total grant funding, respondents from 17 programs indicated that they received $500,000 in annual extra and intramural funding, and those from three programs stated that they received $500,000 to $999,999 in funding. Five respondents indicated that their programs currently received $1 million to $5 million in grant funding, and three reported >$5 million in research support. The sources of research funding included several divisions within the National Institute of Health (NIH, 12 programs), Agency for Healthcare Research and Quality (AHRQ, four programs), foundations (four programs), and internal grants (six programs). Additionally, six programs indicated “other” sources of funding that included the VHA, Patient-Centered Outcomes Research Institute (PCORI), Centers for Medicare and Medicaid Services, Centers for Disease Control (CDC), and industry sources.

A range of grants, including career development awards (11 programs); small grants, such as R21 and R03s (eight programs); R-level grants, including VA merit awards (five programs); program series grants, such as P and U grants (five programs), and foundation grants (eight programs), were reported as types of awards. Respondents from 16 programs indicated that they provided internal pilot grants. Amounts for such grants ranged from <$50,000 (14 programs) to $50,000-$100,000 (two programs).

Research Fellowship Programs/Training Programs

Only five of the 28 surveyed programs indicated that they currently had a research training or fellowship program for developing hospitalist investigators. The age of these programs varied from <1 year to 10 years. Three of the five programs stated that they had two fellows per year, and two stated they had spots for one trainee annually. All respondents indicated that fellows received training on study design, research methods, quantitative (eg, large database and secondary analyses) and qualitative data analysis. In addition, two programs included training in systematic review and meta-analyses, and three included focused courses on healthcare policy. Four of the five programs included training in QI tools, such as LEAN and Six Sigma. Funding for four of the five fellowship programs came from internal sources (eg, department and CTSA). However, two programs added they received some support from extramural funding and philanthropy. Following training, respondents from programs indicated that the majority of their graduates (60%) went on to hybrid research/QI roles (50/50 research/clinical effort), whereas 40% obtained dedicated research investigator (80/20) positions (Table 2).

The 23 institutions without research training programs cited that the most important barrier for establishing such programs was lack of funding (12 programs) and the lack of a pipeline of hospitalists seeking such training (six programs). However, 15 programs indicated that opportunities for hospitalists to gain research training in the form of courses were available internally (eg, courses in the department or medical school) or externally (eg, School of Public Health). Seven programs indicated that they were planning to start a HM research fellowship within the next five years.

Research Faculty

Among the 28 respondents, 15 stated that they have faculty members who conduct research as their main professional activity (ie, >50% effort). The number of faculty members in each program in such roles varied from one to 10. Respondents indicated that faculty members in this category were most often midcareer assistant or associate professors with few full professors. All programs indicated that scholarship in the form of peer-reviewed publications was required for the promotion of faculty. Faculty members who performed research as their main activity had all received formal fellowship training and consequently had dual degrees (MD with MPH or MD, with MSc being the two most common combinations). With respect to clinical activities, most respondents indicated that research faculty spent 10% to 49% of their effort on clinical work. However, five respondents indicated that research faculty had <10% effort on clinical duties (Table 3).

Eleven respondents (39%) identified the main focus of faculty as health service research, where four (14%) identified their main focus as clinical trials. Regardless of funding status, all respondents stated that their faculty were interested in studying quality and process improvement efforts (eg, transitions or readmissions, n = 19), patient safety initiatives (eg, hospital-acquired complications, n = 17), and disease-specific areas (eg, thrombosis, n = 15).

In terms of research output, 12 respondents stated that their research/QI faculty collectively published 11-50 peer-reviewed papers during the academic year, and 10 programs indicated that their faculty published 0-10 papers per year. Only three programs reported that their faculty collectively published 50-99 peer-reviewed papers per year. With respect to abstract presentations at national conferences, 13 programs indicated that they presented 0-10 abstracts, and 12 indicated that they presented 11-50.

In this first survey quantifying research activities in HM, respondents from 28 programs shared important insights into research activities at their institutions. Although our sample size was small, substantial variation in the size, composition, and structure of research programs in HM among respondents was observed. For example, few respondents indicated the availability of training programs for research in HM at their institutions. Similarly, among faculty who focused mainly on research, variation in funding streams and effort protection was observed. A preponderance of midcareer faculty with a range of funding sources, including NIH, AHRQ, VHA, CMS, and CDC was reported. Collectively, these data not only provide a unique glimpse into the state of research in HM but also help establish a baseline of the status of the field at large.

Some findings of our study are intuitive given our sampling strategy and the types of programs that responded. For example, the fact that most respondents for research programs represented university-based or affiliated institutions is expected given the tripartite academic mission. However, even within our sample of highly motivated programs, some findings are surprising and merit further exploration. For example, the observation that some respondents identified HM investigators within their program with <25% in intra- or extramural funding was unexpected. On the other extreme, we were surprised to find that three programs reported >$5 million in research funding. Understanding whether specific factors, such as the availability of experienced mentors within and outside departments or assistance from support staff (eg, statisticians and project managers), are associated with success and funding within these programs are important questions to answer. By focusing on these issues, we will be well poised as a field to understand what works, what does not work, and why.

Likewise, the finding that few programs within our sample offer formal training in the form of fellowships to research investigators represents an improvement opportunity. A pipeline for growing investigators is critical for the specialty that is HM. Notably, this call is not new; rather, previous investigators have highlighted the importance of developing academically oriented hospitalists for the future of the field.⁵ The implementation of faculty scholarship development programs has improved the scholarly output, mentoring activities, and succession planning of academics within HM.^6,7 Conversely, lack of adequate mentorship and support for academic activities remains a challenge and as a factor associated with the failure to produce academic work.⁸ Without a cadre of investigators asking critical questions related to care delivery, the legitimacy of our field may be threatened.

While extrapolating to the field is difficult given the small number of our respondents, highlighting the progress that has been made is important. For example, while misalignment between funding and clinical and research mission persists, our survey found that several programs have been successful in securing extramural funding for their investigators. Additionally, internal funding for QI work appears to be increasing, with hospitalists receiving dedicated effort for much of this work. Innovation in how best to support and develop these types of efforts have also emerged. For example, the University of Michigan Specialist Hospitalist Allied Research Program offers dedicated effort and funding for hospitalists tackling projects germane to HM (eg, ordering of blood cultures for febrile inpatients) that overlap with subspecialists (eg, infectious diseases).⁹ Thus, hospitalists are linked with other specialties in the development of research agendas and academic products. Similarly, the launch of the HOMERUN network, a coalition of investigators who bridge health systems to study problems central to HM, has helped usher in a new era of research opportunities in the specialty.¹⁰ Fundamentally, the culture of HM has begun to place an emphasis on academic and scholarly productivity in addition to clinical prowess.^11-13 Increased support and funding for training programs geared toward innovation and research in HM is needed to continue this mission. The Society for General Internal Medicine, American College of Physicians, and SHM have important roles to play as the largest professional organizations for generalists in this respect. Support for research, QI, and investigators in HM remains an urgent and largely unmet need.

Our study has limitations. First, our response rate was low at 28% but is consistent with the response rates of other surveys of physician groups.¹⁴ Caution in making inferences to the field at large is necessary given the potential for selection and nonresponse bias. However, we expect that respondents are likely biased toward programs actively conducting research and engaged in QI, thus better reflecting the state of these activities in HM. Second, given that we did not ask for any identifying information, we have no way of establishing the accuracy of the data provided by respondents. However, we have no reason to believe that responses would be altered in a systematic fashion. Future studies that link our findings to publicly available data (eg, databases of active grants and funding) might be useful. Third, while our survey instrument was created and internally validated by hospitalist researchers, its lack of external validation could limit findings. Finally, our results vary on the basis of how respondents answered questions related to effort and time allocation given that these measures differ across programs.

In summary, the findings from this study highlight substantial variations in the number, training, and funding of research faculty across HM programs. Understanding the factors behind the success of some programs and the failures of others appears important in informing and growing the research in the field. Future studies that aim to expand survey participation, raise the awareness of the state of research in HM, and identify barriers and facilitators to academic success in HM are needed.

Disclosures

Dr. Chopra discloses grant funding from the Agency for Healthcare Research and Quality (AHRQ), VA Health Services and Research Department, and Centers for Disease Control. Dr. Jones discloses grant funding from AHRQ. All other authors disclose no conflicts of interest.

Almost all specialties in internal medicine have a sound scientific research base through which clinical practice is informed.¹ For the field of Hospital Medicine (HM), this evidence has largely comprised research generated from fields outside of the specialty. The need to develop, invest, and grow investigators in hospital-based medicine remains unmet as HM and its footprint in hospital systems continue to grow.^2,3

Despite this fact, little is known about the current state of research in HM. A 2014 survey of the members of the Society of Hospital Medicine (SHM) found that research output across the field of HM, as measured on the basis of peer-reviewed publications, was growing.⁴ Since then, however, the numbers of individuals engaged in research activities, their background and training, publication output, or funding sources have not been quantified. Similarly, little is known about which institutions support the development of junior investigators (ie, HM research fellowships), how these programs are funded, and whether or not matriculants enter the field as investigators. These gaps must be measured, evaluated, and ideally addressed through strategic policy and funding initiatives to advance the state of science within HM.

Members of the SHM Research Committee developed, designed, and deployed a survey to improve the understanding of the state of research in HM. In this study, we aimed to establish the baseline of research in HM to enable the measurement of progress through periodic waves of data collection. Specifically, we sought to quantify and describe the characteristics of existing research programs, the sources and types of funding, the number and background of faculty, and the availability of resources for training researchers in HM.

Study Setting and Participants

Given that no defined list, database, or external resource that identifies research programs and contacts in HM exists, we began by creating a strategy to identify and sample adult HM programs and their leaders engaged in research activity. We iteratively developed a two-step approach to maximize inclusivity. First, we partnered with SHM to identify programs and leaders actively engaging in research activities. SHM is the largest professional organization within HM and maintains an extensive membership database that includes the titles, e-mail addresses, and affiliations of hospitalists in the United States, including academic and nonacademic sites. This list was manually scanned, and the leaders of academic and research programs in adult HM were identified by examining their titles (eg, Division Chief, Research Lead, etc.) and academic affiliations. During this step, members of the committee noticed that certain key individuals were either missing, no longer occupying their role/title, or had been replaced by others. Therefore, we performed a second step and asked the members of the SHM Research Committee to identify academic and research leaders by using current personal contacts, publication history, and social networks. We asked members to identify individuals and programs that had received grant funding, were actively presenting research at SHM (or other major national venues), and/or were producing peer-reviewed publications related to HM. These programs were purposefully chosen (ie, over HM programs known for clinical activities) to create an enriched sample of those engaged in research in HM. The research committee performed the “second pass” to ensure that established investigators who may not be accurately captured within the SHM database were included to maximize yield for the survey. Finally, these two sources were merged to ensure the absence of duplicate contacts and the identification of a primary respondent for each affiliate. As a result, a convenience sample of 100 programs and corresponding individuals was compiled for the purposes of this survey.

Survey Development

A workgroup within the SHM Research Committee was tasked to create a survey that would achieve four distinct goals: (1) identify institutions currently engaging in hospital-based research; (2) define the characteristics, including sources of research funding, training opportunities, criteria for promotion, and grant support, of research programs within institutions; (3) understand the prevalence of research fellowship programs, including size, training curricula, and funding sources; and (4) evaluate the productivity and funding sources of HM investigators at each site.

Survey questions that target each of these domains were drafted by the workgroup. Questions were pretested with colleagues outside the workgroup focused on this project (ie, from the main research committee). The instrument was refined and edited to improve the readability and clarity of questions on the basis of the feedback obtained through the iterative process. The revised instrument was then programmed into an online survey administration tool (SurveyMonkey^®) to facilitate electronic dissemination. Finally, the members of the workgroup tested the online survey to ensure functionality. No identifiable information was collected from respondents, and no monetary incentive was offered for the completion of the survey. An invitation to participate in the survey was sent via e-mail to each of the program contacts identified.

Statistical Analysis

Descriptive statistics, including proportions, means, and percentages, were used to tabulate results. All analyses were conducted using Stata 13 MP/SE (StataCorp, College Station, Texas).

Ethical and Regulatory Considerations

The study was reviewed and deemed exempt from regulation by the University of Michigan Institutional Review Board (HUM000138628).

General Characteristics of Research Programs and Faculty

Out of 100 program contacts, 28 (representing 1,586 faculty members) responded and were included in the survey (program response rate = 28%). When comparing programs that did respond with those that did not, a greater proportion of programs in university settings were noted among respondents (79% vs 21%). Respondents represented programs from all regions of the United States, with most representing university-based (79%), university-affiliated (14%) or Veterans Health Administration (VHA; 11%) programs. Most respondents were in leadership roles, including division chiefs (32%), research directors/leads (21%), section chiefs (18%), and related titles, such as program director. Respondents indicated that the total number of faculty members in their programs (including nonclinicians and advance practice providers) varied from eight to 152 (mean [SD] = 57 [36]) members, with physicians representing the majority of faculty members (Table 1).

Among the 1,586 faculty members within the 28 programs, respondents identified 192 faculty members (12%) as currently receiving extra- or intramural support for research activities. Of these faculty, over half (58%) received <25% of effort from intra or extramural sources, and 28 (15%) and 52 (27%) faculty members received 25%-50% or >50% of support for their effort, respectively. The number of investigators who received funding across programs ranged from 0 to 28 faculty members. Compared with the 192 funded investigators, respondents indicated that a larger number of faculty in their programs (n = 656 or 41%) were involved in local quality improvement (QI) efforts. Of the 656 faculty members involved in QI efforts, 241 individuals (37%) were internally funded and received protected time/effort for their work.

Key Attributes of Research Programs

In the evaluation of the amount of total grant funding, respondents from 17 programs indicated that they received $500,000 in annual extra and intramural funding, and those from three programs stated that they received $500,000 to $999,999 in funding. Five respondents indicated that their programs currently received $1 million to $5 million in grant funding, and three reported >$5 million in research support. The sources of research funding included several divisions within the National Institute of Health (NIH, 12 programs), Agency for Healthcare Research and Quality (AHRQ, four programs), foundations (four programs), and internal grants (six programs). Additionally, six programs indicated “other” sources of funding that included the VHA, Patient-Centered Outcomes Research Institute (PCORI), Centers for Medicare and Medicaid Services, Centers for Disease Control (CDC), and industry sources.

A range of grants, including career development awards (11 programs); small grants, such as R21 and R03s (eight programs); R-level grants, including VA merit awards (five programs); program series grants, such as P and U grants (five programs), and foundation grants (eight programs), were reported as types of awards. Respondents from 16 programs indicated that they provided internal pilot grants. Amounts for such grants ranged from <$50,000 (14 programs) to $50,000-$100,000 (two programs).

Research Fellowship Programs/Training Programs

Only five of the 28 surveyed programs indicated that they currently had a research training or fellowship program for developing hospitalist investigators. The age of these programs varied from <1 year to 10 years. Three of the five programs stated that they had two fellows per year, and two stated they had spots for one trainee annually. All respondents indicated that fellows received training on study design, research methods, quantitative (eg, large database and secondary analyses) and qualitative data analysis. In addition, two programs included training in systematic review and meta-analyses, and three included focused courses on healthcare policy. Four of the five programs included training in QI tools, such as LEAN and Six Sigma. Funding for four of the five fellowship programs came from internal sources (eg, department and CTSA). However, two programs added they received some support from extramural funding and philanthropy. Following training, respondents from programs indicated that the majority of their graduates (60%) went on to hybrid research/QI roles (50/50 research/clinical effort), whereas 40% obtained dedicated research investigator (80/20) positions (Table 2).

The 23 institutions without research training programs cited that the most important barrier for establishing such programs was lack of funding (12 programs) and the lack of a pipeline of hospitalists seeking such training (six programs). However, 15 programs indicated that opportunities for hospitalists to gain research training in the form of courses were available internally (eg, courses in the department or medical school) or externally (eg, School of Public Health). Seven programs indicated that they were planning to start a HM research fellowship within the next five years.

Research Faculty

Among the 28 respondents, 15 stated that they have faculty members who conduct research as their main professional activity (ie, >50% effort). The number of faculty members in each program in such roles varied from one to 10. Respondents indicated that faculty members in this category were most often midcareer assistant or associate professors with few full professors. All programs indicated that scholarship in the form of peer-reviewed publications was required for the promotion of faculty. Faculty members who performed research as their main activity had all received formal fellowship training and consequently had dual degrees (MD with MPH or MD, with MSc being the two most common combinations). With respect to clinical activities, most respondents indicated that research faculty spent 10% to 49% of their effort on clinical work. However, five respondents indicated that research faculty had <10% effort on clinical duties (Table 3).

Eleven respondents (39%) identified the main focus of faculty as health service research, where four (14%) identified their main focus as clinical trials. Regardless of funding status, all respondents stated that their faculty were interested in studying quality and process improvement efforts (eg, transitions or readmissions, n = 19), patient safety initiatives (eg, hospital-acquired complications, n = 17), and disease-specific areas (eg, thrombosis, n = 15).

In terms of research output, 12 respondents stated that their research/QI faculty collectively published 11-50 peer-reviewed papers during the academic year, and 10 programs indicated that their faculty published 0-10 papers per year. Only three programs reported that their faculty collectively published 50-99 peer-reviewed papers per year. With respect to abstract presentations at national conferences, 13 programs indicated that they presented 0-10 abstracts, and 12 indicated that they presented 11-50.

In this first survey quantifying research activities in HM, respondents from 28 programs shared important insights into research activities at their institutions. Although our sample size was small, substantial variation in the size, composition, and structure of research programs in HM among respondents was observed. For example, few respondents indicated the availability of training programs for research in HM at their institutions. Similarly, among faculty who focused mainly on research, variation in funding streams and effort protection was observed. A preponderance of midcareer faculty with a range of funding sources, including NIH, AHRQ, VHA, CMS, and CDC was reported. Collectively, these data not only provide a unique glimpse into the state of research in HM but also help establish a baseline of the status of the field at large.

Some findings of our study are intuitive given our sampling strategy and the types of programs that responded. For example, the fact that most respondents for research programs represented university-based or affiliated institutions is expected given the tripartite academic mission. However, even within our sample of highly motivated programs, some findings are surprising and merit further exploration. For example, the observation that some respondents identified HM investigators within their program with <25% in intra- or extramural funding was unexpected. On the other extreme, we were surprised to find that three programs reported >$5 million in research funding. Understanding whether specific factors, such as the availability of experienced mentors within and outside departments or assistance from support staff (eg, statisticians and project managers), are associated with success and funding within these programs are important questions to answer. By focusing on these issues, we will be well poised as a field to understand what works, what does not work, and why.

Likewise, the finding that few programs within our sample offer formal training in the form of fellowships to research investigators represents an improvement opportunity. A pipeline for growing investigators is critical for the specialty that is HM. Notably, this call is not new; rather, previous investigators have highlighted the importance of developing academically oriented hospitalists for the future of the field.⁵ The implementation of faculty scholarship development programs has improved the scholarly output, mentoring activities, and succession planning of academics within HM.^6,7 Conversely, lack of adequate mentorship and support for academic activities remains a challenge and as a factor associated with the failure to produce academic work.⁸ Without a cadre of investigators asking critical questions related to care delivery, the legitimacy of our field may be threatened.

While extrapolating to the field is difficult given the small number of our respondents, highlighting the progress that has been made is important. For example, while misalignment between funding and clinical and research mission persists, our survey found that several programs have been successful in securing extramural funding for their investigators. Additionally, internal funding for QI work appears to be increasing, with hospitalists receiving dedicated effort for much of this work. Innovation in how best to support and develop these types of efforts have also emerged. For example, the University of Michigan Specialist Hospitalist Allied Research Program offers dedicated effort and funding for hospitalists tackling projects germane to HM (eg, ordering of blood cultures for febrile inpatients) that overlap with subspecialists (eg, infectious diseases).⁹ Thus, hospitalists are linked with other specialties in the development of research agendas and academic products. Similarly, the launch of the HOMERUN network, a coalition of investigators who bridge health systems to study problems central to HM, has helped usher in a new era of research opportunities in the specialty.¹⁰ Fundamentally, the culture of HM has begun to place an emphasis on academic and scholarly productivity in addition to clinical prowess.^11-13 Increased support and funding for training programs geared toward innovation and research in HM is needed to continue this mission. The Society for General Internal Medicine, American College of Physicians, and SHM have important roles to play as the largest professional organizations for generalists in this respect. Support for research, QI, and investigators in HM remains an urgent and largely unmet need.

Our study has limitations. First, our response rate was low at 28% but is consistent with the response rates of other surveys of physician groups.¹⁴ Caution in making inferences to the field at large is necessary given the potential for selection and nonresponse bias. However, we expect that respondents are likely biased toward programs actively conducting research and engaged in QI, thus better reflecting the state of these activities in HM. Second, given that we did not ask for any identifying information, we have no way of establishing the accuracy of the data provided by respondents. However, we have no reason to believe that responses would be altered in a systematic fashion. Future studies that link our findings to publicly available data (eg, databases of active grants and funding) might be useful. Third, while our survey instrument was created and internally validated by hospitalist researchers, its lack of external validation could limit findings. Finally, our results vary on the basis of how respondents answered questions related to effort and time allocation given that these measures differ across programs.

In summary, the findings from this study highlight substantial variations in the number, training, and funding of research faculty across HM programs. Understanding the factors behind the success of some programs and the failures of others appears important in informing and growing the research in the field. Future studies that aim to expand survey participation, raise the awareness of the state of research in HM, and identify barriers and facilitators to academic success in HM are needed.

Disclosures

Dr. Chopra discloses grant funding from the Agency for Healthcare Research and Quality (AHRQ), VA Health Services and Research Department, and Centers for Disease Control. Dr. Jones discloses grant funding from AHRQ. All other authors disclose no conflicts of interest.

Almost all specialties in internal medicine have a sound scientific research base through which clinical practice is informed.¹ For the field of Hospital Medicine (HM), this evidence has largely comprised research generated from fields outside of the specialty. The need to develop, invest, and grow investigators in hospital-based medicine remains unmet as HM and its footprint in hospital systems continue to grow.^2,3

Despite this fact, little is known about the current state of research in HM. A 2014 survey of the members of the Society of Hospital Medicine (SHM) found that research output across the field of HM, as measured on the basis of peer-reviewed publications, was growing.⁴ Since then, however, the numbers of individuals engaged in research activities, their background and training, publication output, or funding sources have not been quantified. Similarly, little is known about which institutions support the development of junior investigators (ie, HM research fellowships), how these programs are funded, and whether or not matriculants enter the field as investigators. These gaps must be measured, evaluated, and ideally addressed through strategic policy and funding initiatives to advance the state of science within HM.

Members of the SHM Research Committee developed, designed, and deployed a survey to improve the understanding of the state of research in HM. In this study, we aimed to establish the baseline of research in HM to enable the measurement of progress through periodic waves of data collection. Specifically, we sought to quantify and describe the characteristics of existing research programs, the sources and types of funding, the number and background of faculty, and the availability of resources for training researchers in HM.

Study Setting and Participants

Given that no defined list, database, or external resource that identifies research programs and contacts in HM exists, we began by creating a strategy to identify and sample adult HM programs and their leaders engaged in research activity. We iteratively developed a two-step approach to maximize inclusivity. First, we partnered with SHM to identify programs and leaders actively engaging in research activities. SHM is the largest professional organization within HM and maintains an extensive membership database that includes the titles, e-mail addresses, and affiliations of hospitalists in the United States, including academic and nonacademic sites. This list was manually scanned, and the leaders of academic and research programs in adult HM were identified by examining their titles (eg, Division Chief, Research Lead, etc.) and academic affiliations. During this step, members of the committee noticed that certain key individuals were either missing, no longer occupying their role/title, or had been replaced by others. Therefore, we performed a second step and asked the members of the SHM Research Committee to identify academic and research leaders by using current personal contacts, publication history, and social networks. We asked members to identify individuals and programs that had received grant funding, were actively presenting research at SHM (or other major national venues), and/or were producing peer-reviewed publications related to HM. These programs were purposefully chosen (ie, over HM programs known for clinical activities) to create an enriched sample of those engaged in research in HM. The research committee performed the “second pass” to ensure that established investigators who may not be accurately captured within the SHM database were included to maximize yield for the survey. Finally, these two sources were merged to ensure the absence of duplicate contacts and the identification of a primary respondent for each affiliate. As a result, a convenience sample of 100 programs and corresponding individuals was compiled for the purposes of this survey.

Survey Development

A workgroup within the SHM Research Committee was tasked to create a survey that would achieve four distinct goals: (1) identify institutions currently engaging in hospital-based research; (2) define the characteristics, including sources of research funding, training opportunities, criteria for promotion, and grant support, of research programs within institutions; (3) understand the prevalence of research fellowship programs, including size, training curricula, and funding sources; and (4) evaluate the productivity and funding sources of HM investigators at each site.

Survey questions that target each of these domains were drafted by the workgroup. Questions were pretested with colleagues outside the workgroup focused on this project (ie, from the main research committee). The instrument was refined and edited to improve the readability and clarity of questions on the basis of the feedback obtained through the iterative process. The revised instrument was then programmed into an online survey administration tool (SurveyMonkey^®) to facilitate electronic dissemination. Finally, the members of the workgroup tested the online survey to ensure functionality. No identifiable information was collected from respondents, and no monetary incentive was offered for the completion of the survey. An invitation to participate in the survey was sent via e-mail to each of the program contacts identified.

Statistical Analysis

Descriptive statistics, including proportions, means, and percentages, were used to tabulate results. All analyses were conducted using Stata 13 MP/SE (StataCorp, College Station, Texas).

Ethical and Regulatory Considerations

The study was reviewed and deemed exempt from regulation by the University of Michigan Institutional Review Board (HUM000138628).

General Characteristics of Research Programs and Faculty

Out of 100 program contacts, 28 (representing 1,586 faculty members) responded and were included in the survey (program response rate = 28%). When comparing programs that did respond with those that did not, a greater proportion of programs in university settings were noted among respondents (79% vs 21%). Respondents represented programs from all regions of the United States, with most representing university-based (79%), university-affiliated (14%) or Veterans Health Administration (VHA; 11%) programs. Most respondents were in leadership roles, including division chiefs (32%), research directors/leads (21%), section chiefs (18%), and related titles, such as program director. Respondents indicated that the total number of faculty members in their programs (including nonclinicians and advance practice providers) varied from eight to 152 (mean [SD] = 57 [36]) members, with physicians representing the majority of faculty members (Table 1).

Among the 1,586 faculty members within the 28 programs, respondents identified 192 faculty members (12%) as currently receiving extra- or intramural support for research activities. Of these faculty, over half (58%) received <25% of effort from intra or extramural sources, and 28 (15%) and 52 (27%) faculty members received 25%-50% or >50% of support for their effort, respectively. The number of investigators who received funding across programs ranged from 0 to 28 faculty members. Compared with the 192 funded investigators, respondents indicated that a larger number of faculty in their programs (n = 656 or 41%) were involved in local quality improvement (QI) efforts. Of the 656 faculty members involved in QI efforts, 241 individuals (37%) were internally funded and received protected time/effort for their work.

Key Attributes of Research Programs

In the evaluation of the amount of total grant funding, respondents from 17 programs indicated that they received $500,000 in annual extra and intramural funding, and those from three programs stated that they received $500,000 to $999,999 in funding. Five respondents indicated that their programs currently received $1 million to $5 million in grant funding, and three reported >$5 million in research support. The sources of research funding included several divisions within the National Institute of Health (NIH, 12 programs), Agency for Healthcare Research and Quality (AHRQ, four programs), foundations (four programs), and internal grants (six programs). Additionally, six programs indicated “other” sources of funding that included the VHA, Patient-Centered Outcomes Research Institute (PCORI), Centers for Medicare and Medicaid Services, Centers for Disease Control (CDC), and industry sources.

A range of grants, including career development awards (11 programs); small grants, such as R21 and R03s (eight programs); R-level grants, including VA merit awards (five programs); program series grants, such as P and U grants (five programs), and foundation grants (eight programs), were reported as types of awards. Respondents from 16 programs indicated that they provided internal pilot grants. Amounts for such grants ranged from <$50,000 (14 programs) to $50,000-$100,000 (two programs).

Research Fellowship Programs/Training Programs

Only five of the 28 surveyed programs indicated that they currently had a research training or fellowship program for developing hospitalist investigators. The age of these programs varied from <1 year to 10 years. Three of the five programs stated that they had two fellows per year, and two stated they had spots for one trainee annually. All respondents indicated that fellows received training on study design, research methods, quantitative (eg, large database and secondary analyses) and qualitative data analysis. In addition, two programs included training in systematic review and meta-analyses, and three included focused courses on healthcare policy. Four of the five programs included training in QI tools, such as LEAN and Six Sigma. Funding for four of the five fellowship programs came from internal sources (eg, department and CTSA). However, two programs added they received some support from extramural funding and philanthropy. Following training, respondents from programs indicated that the majority of their graduates (60%) went on to hybrid research/QI roles (50/50 research/clinical effort), whereas 40% obtained dedicated research investigator (80/20) positions (Table 2).

The 23 institutions without research training programs cited that the most important barrier for establishing such programs was lack of funding (12 programs) and the lack of a pipeline of hospitalists seeking such training (six programs). However, 15 programs indicated that opportunities for hospitalists to gain research training in the form of courses were available internally (eg, courses in the department or medical school) or externally (eg, School of Public Health). Seven programs indicated that they were planning to start a HM research fellowship within the next five years.

Research Faculty

Among the 28 respondents, 15 stated that they have faculty members who conduct research as their main professional activity (ie, >50% effort). The number of faculty members in each program in such roles varied from one to 10. Respondents indicated that faculty members in this category were most often midcareer assistant or associate professors with few full professors. All programs indicated that scholarship in the form of peer-reviewed publications was required for the promotion of faculty. Faculty members who performed research as their main activity had all received formal fellowship training and consequently had dual degrees (MD with MPH or MD, with MSc being the two most common combinations). With respect to clinical activities, most respondents indicated that research faculty spent 10% to 49% of their effort on clinical work. However, five respondents indicated that research faculty had <10% effort on clinical duties (Table 3).

Eleven respondents (39%) identified the main focus of faculty as health service research, where four (14%) identified their main focus as clinical trials. Regardless of funding status, all respondents stated that their faculty were interested in studying quality and process improvement efforts (eg, transitions or readmissions, n = 19), patient safety initiatives (eg, hospital-acquired complications, n = 17), and disease-specific areas (eg, thrombosis, n = 15).

In terms of research output, 12 respondents stated that their research/QI faculty collectively published 11-50 peer-reviewed papers during the academic year, and 10 programs indicated that their faculty published 0-10 papers per year. Only three programs reported that their faculty collectively published 50-99 peer-reviewed papers per year. With respect to abstract presentations at national conferences, 13 programs indicated that they presented 0-10 abstracts, and 12 indicated that they presented 11-50.

In this first survey quantifying research activities in HM, respondents from 28 programs shared important insights into research activities at their institutions. Although our sample size was small, substantial variation in the size, composition, and structure of research programs in HM among respondents was observed. For example, few respondents indicated the availability of training programs for research in HM at their institutions. Similarly, among faculty who focused mainly on research, variation in funding streams and effort protection was observed. A preponderance of midcareer faculty with a range of funding sources, including NIH, AHRQ, VHA, CMS, and CDC was reported. Collectively, these data not only provide a unique glimpse into the state of research in HM but also help establish a baseline of the status of the field at large.

Some findings of our study are intuitive given our sampling strategy and the types of programs that responded. For example, the fact that most respondents for research programs represented university-based or affiliated institutions is expected given the tripartite academic mission. However, even within our sample of highly motivated programs, some findings are surprising and merit further exploration. For example, the observation that some respondents identified HM investigators within their program with <25% in intra- or extramural funding was unexpected. On the other extreme, we were surprised to find that three programs reported >$5 million in research funding. Understanding whether specific factors, such as the availability of experienced mentors within and outside departments or assistance from support staff (eg, statisticians and project managers), are associated with success and funding within these programs are important questions to answer. By focusing on these issues, we will be well poised as a field to understand what works, what does not work, and why.

Likewise, the finding that few programs within our sample offer formal training in the form of fellowships to research investigators represents an improvement opportunity. A pipeline for growing investigators is critical for the specialty that is HM. Notably, this call is not new; rather, previous investigators have highlighted the importance of developing academically oriented hospitalists for the future of the field.⁵ The implementation of faculty scholarship development programs has improved the scholarly output, mentoring activities, and succession planning of academics within HM.^6,7 Conversely, lack of adequate mentorship and support for academic activities remains a challenge and as a factor associated with the failure to produce academic work.⁸ Without a cadre of investigators asking critical questions related to care delivery, the legitimacy of our field may be threatened.

While extrapolating to the field is difficult given the small number of our respondents, highlighting the progress that has been made is important. For example, while misalignment between funding and clinical and research mission persists, our survey found that several programs have been successful in securing extramural funding for their investigators. Additionally, internal funding for QI work appears to be increasing, with hospitalists receiving dedicated effort for much of this work. Innovation in how best to support and develop these types of efforts have also emerged. For example, the University of Michigan Specialist Hospitalist Allied Research Program offers dedicated effort and funding for hospitalists tackling projects germane to HM (eg, ordering of blood cultures for febrile inpatients) that overlap with subspecialists (eg, infectious diseases).⁹ Thus, hospitalists are linked with other specialties in the development of research agendas and academic products. Similarly, the launch of the HOMERUN network, a coalition of investigators who bridge health systems to study problems central to HM, has helped usher in a new era of research opportunities in the specialty.¹⁰ Fundamentally, the culture of HM has begun to place an emphasis on academic and scholarly productivity in addition to clinical prowess.^11-13 Increased support and funding for training programs geared toward innovation and research in HM is needed to continue this mission. The Society for General Internal Medicine, American College of Physicians, and SHM have important roles to play as the largest professional organizations for generalists in this respect. Support for research, QI, and investigators in HM remains an urgent and largely unmet need.

Our study has limitations. First, our response rate was low at 28% but is consistent with the response rates of other surveys of physician groups.¹⁴ Caution in making inferences to the field at large is necessary given the potential for selection and nonresponse bias. However, we expect that respondents are likely biased toward programs actively conducting research and engaged in QI, thus better reflecting the state of these activities in HM. Second, given that we did not ask for any identifying information, we have no way of establishing the accuracy of the data provided by respondents. However, we have no reason to believe that responses would be altered in a systematic fashion. Future studies that link our findings to publicly available data (eg, databases of active grants and funding) might be useful. Third, while our survey instrument was created and internally validated by hospitalist researchers, its lack of external validation could limit findings. Finally, our results vary on the basis of how respondents answered questions related to effort and time allocation given that these measures differ across programs.

In summary, the findings from this study highlight substantial variations in the number, training, and funding of research faculty across HM programs. Understanding the factors behind the success of some programs and the failures of others appears important in informing and growing the research in the field. Future studies that aim to expand survey participation, raise the awareness of the state of research in HM, and identify barriers and facilitators to academic success in HM are needed.

Disclosures

Dr. Chopra discloses grant funding from the Agency for Healthcare Research and Quality (AHRQ), VA Health Services and Research Department, and Centers for Disease Control. Dr. Jones discloses grant funding from AHRQ. All other authors disclose no conflicts of interest.

Given that multiple disciplines are often involved in caring for patients admitted to the hospital, timely communication, collaboration, and coordination amongst various disciplines is necessary for safe and effective patient care.¹ With the focus on improving patient satisfaction and throughput in hospitals, it is also important to make more accurate predictions of the discharge date and allow time for patients and their families to prepare for discharge.^2-4

Multidisciplinary rounds (MDR) are defined as structured daily communication amongst key members of the patient’s care team (eg, nurses, physicians, case managers, social workers, pharmacists, and rehabilitation services). MDR have shown to be a useful strategy for ensuring that all members of the care team are updated on the plan of care for the patient.⁵ During MDR, a brief “check-in” discussing the patient’s plan of care, pending needs, and barriers to discharge allows all team members, patients, and families to effectively coordinate care and plan and prepare for discharge.

Multiple studies have reported increased collaboration and improved communication between disciplines with the use of such multidisciplinary rounding.^2,5-7 Additionally, MDR have been shown to improve patient outcomes⁸ and reduce adverse events,⁹ length of stay (LOS),^6,8 cost of care,⁸ and readmissions.¹

We redesigned MDR on the general medicine wards at our institution in October 2014 by using Lean management techniques. Lean is defined as a set of philosophies and methods that aim to create transformation in thinking, behavior, and culture in each process, with the goal of maximizing the value for the patients and providers, adding efficiency, and reducing waste and waits.¹⁰

In this study, we evaluate whether this new model of MDR was associated with a decrease in the LOS. We also evaluate whether this new model of MDR was associated with an increase in discharges before noon, documentation of estimated discharge date (EDD) in our electronic health record (EHR), and patient satisfaction.

Setting, Design, and Patients

The study was conducted on the teaching general medicine service at our institution, an urban, 484-bed academic hospital. The general medicine service has patients on 4 inpatient units (total of 95 beds) and is managed by 5 teaching service teams.

We performed a pre-post study. The preperiod (in which the old model of MDR was followed) included 4000 patients discharged between September 1, 2013, and October 22, 2014. The postperiod (in which the new model of MDR was followed) included 2085 patients discharged between October 23, 2014, and April 30, 2015. We excluded 139 patients that died in the hospital prior to discharge and patients on the nonteaching and/or private practice service.

All data were provided by our institution’s Digital Solutions Department. Our institutional review board issued a letter of determination exempting this study from further review because it was deemed to be a quality improvement initiative.

Use of Lean Management to Redesign our MDR

Our institution has incorporated the Lean management system to continually add value to services through the elimination of waste, thus simultaneously optimizing the quality of patient care, cost, and patient satisfaction.¹¹ Lean, derived from the Toyota Production System, has long been used in manufacturing and in recent decades has spread to healthcare.¹² We leveraged the following 3 key Lean techniques to redesign our MDR: (1) value stream management (VSM), (2) rapid process improvement workshops (RPIW), and (3) active daily management (ADM), as detailed in supplementary Appendix 1.

Interventions

Outcomes

The primary outcome was mean LOS. The secondary outcomes were (1) discharges before noon, (2) recording of the EDD in our EHR within 24 hours of admission (as time stamped on our EHR), and (3) patient satisfaction.

Data for patient satisfaction were obtained using the Press Ganey survey. We used data on patient satisfaction scores for the following 2 relevant questions on this survey: (1) extent to which the patient felt ready to be discharged and (2) how well staff worked together to care for the patient. Proportions of the “top-box” (“very good”) were used for the analysis. These survey data were available on 467 patients (11.7%) in the preperiod and 188 patients (9.0%) in the postperiod.

Data Analysis

A sensitivity analysis was conducted on a second cohort that included a subset of patients from the preperiod between November 1, 2013, and April 30, 2014, and a subset of patients from the postperiod between November 1, 2014, and April 1, 2015, to control for the calendar period (supplementary Appendix 2).

All analyses were conducted in R version 3.3.0, with the linear mixed-effects model lme4 statistical package.^13,14

Table 3 shows the differences in the outcomes between the pre- and postperiods. There was no change in the LOS or LOS adjusted for CMI. There was a 3.9% increase in discharges before noon in the postperiod compared with the preperiod (95% CI, 2.4% to 5.3%; P < .001). There was a 9.9% increase in the percentage of patients for whom the EDD was recorded in our EHR within 24 hours of admission (95% CI, 7.4% to 12.4%; P < .001). There was no change in the “top-box” patient satisfaction scores.

There were only marginal differences in the results between the entire cohort and a second subset cohort used for sensitivity analysis (supplementary Appendix 2).

In our study, there was no change in the mean LOS with the new model of MDR. There was an increase in discharges before noon and in recording of the EDD in our EHR within 24 hours of admission in the postperiod when the Lean-based new model of MDR was utilized. There was no change in patient satisfaction. With no change in staffing, we were able to accommodate the increase in the discharge volume in the postperiod.

We believe our results are attributable to several factors, including clearly defined roles and responsibilities for all participants of MDR, the inclusion of more experienced general medicine attending physician (compared with housestaff), Lean management techniques to identify gaps in the patient’s journey from emergency department to discharge using VSM, the development of appropriate workflows and standard work on how the multidisciplinary teams would work together at RPIWs, and ADM to ensure sustainability and engagement among frontline members and institutional leaders. In order to sustain this, we planned to continue monitoring data in daily, weekly, and monthly forums with senior physician and administrative leaders. Planning for additional interventions is underway, including moving MDR to the bedside, instituting an afternoon “check-in” that would enable more detailed action planning, and addressing barriers in a timely manner for patients ready to discharge the following day.

Our study has a few limitations. First, this is an observational study that cannot determine causation. Second, this is a single-center study conducted on patients only on the general medicine teaching service. Third, there were several concurrent interventions implemented at our institution to improve LOS, throughput, and patient satisfaction in addition to MDR, thus making it difficult to isolate the impact of our intervention. Fourth, in the new model of MDR, rounds took place only 5 days per week, thereby possibly limiting the potential impact on our outcomes. Fifth, while we showed improvements in the discharges before noon and recording of EDD in the post period, we were not able to achieve our target of 25% discharges before noon or 100% recording of EDD in this time period. We believe the limited amount of time between the pre- and postperiods to allow for adoption and learning of the processes might have contributed to the underestimation of the impact of the new model of MDR, thereby limiting our ability to achieve our targets. Sixth, the response rate on the Press Ganey survey was low, and we did not directly survey patients or families for their satisfaction with MDR.

Our study has several strengths. To our knowledge, this is the first study to embed Lean management techniques in the design of MDR in the inpatient setting. While several studies have demonstrated improvements in discharges before noon through the implementation of MDR, they have not incorporated Lean management techniques, which we believe are critical to ensure the sustainability of results.^1,3,5,6,8,15 Second, while it was not measured, there was a high level of provider engagement in the process in the new model of MDR. Third, because the MDR were conducted at the nurse’s station on each inpatient unit in the new model instead of in a conference room, it was well attended by all members of the multidisciplinary team. Fourth, the presence of a visibility board allowed for all team members to have easy access to visual feedback throughout the day, even if they were not present at the MDR. Fifth, we believe that there was also more accurate estimation of the date and time of discharge in the new model of MDR because the discussion was facilitated by the case manager, who is experienced in identifying barriers to discharge (compared with the housestaff in the old model of MDR), and included the more experienced attending physician. Finally, the consistent presence of a multidisciplinary team at MDR allowed for the incorporation of everyone’s concerns at one time, thereby limiting the need for paging multiple disciplines throughout the day, which led to quicker resolution of issues and assignment of pending tasks.

In conclusion, our study shows no change in the mean LOS when the Lean-based model of MDR was utilized. Our study demonstrates an increase in discharges before noon and in recording of EDD on our EHR within 24 hours of admission in the post period when the Lean-based model of MDR was utilized. There was no change in patient satisfaction. While this study was conducted at an academic medical center on the general medicine wards, we believe our new model of MDR, which leveraged Lean management techniques, may successfully impact patient flow in all inpatient clinical services and nonteaching hospitals.

Disclosure

The authors report no financial conflicts of interest and have nothing to disclose.

Given that multiple disciplines are often involved in caring for patients admitted to the hospital, timely communication, collaboration, and coordination amongst various disciplines is necessary for safe and effective patient care.¹ With the focus on improving patient satisfaction and throughput in hospitals, it is also important to make more accurate predictions of the discharge date and allow time for patients and their families to prepare for discharge.^2-4

Multidisciplinary rounds (MDR) are defined as structured daily communication amongst key members of the patient’s care team (eg, nurses, physicians, case managers, social workers, pharmacists, and rehabilitation services). MDR have shown to be a useful strategy for ensuring that all members of the care team are updated on the plan of care for the patient.⁵ During MDR, a brief “check-in” discussing the patient’s plan of care, pending needs, and barriers to discharge allows all team members, patients, and families to effectively coordinate care and plan and prepare for discharge.

Multiple studies have reported increased collaboration and improved communication between disciplines with the use of such multidisciplinary rounding.^2,5-7 Additionally, MDR have been shown to improve patient outcomes⁸ and reduce adverse events,⁹ length of stay (LOS),^6,8 cost of care,⁸ and readmissions.¹

We redesigned MDR on the general medicine wards at our institution in October 2014 by using Lean management techniques. Lean is defined as a set of philosophies and methods that aim to create transformation in thinking, behavior, and culture in each process, with the goal of maximizing the value for the patients and providers, adding efficiency, and reducing waste and waits.¹⁰

In this study, we evaluate whether this new model of MDR was associated with a decrease in the LOS. We also evaluate whether this new model of MDR was associated with an increase in discharges before noon, documentation of estimated discharge date (EDD) in our electronic health record (EHR), and patient satisfaction.

Setting, Design, and Patients

The study was conducted on the teaching general medicine service at our institution, an urban, 484-bed academic hospital. The general medicine service has patients on 4 inpatient units (total of 95 beds) and is managed by 5 teaching service teams.

We performed a pre-post study. The preperiod (in which the old model of MDR was followed) included 4000 patients discharged between September 1, 2013, and October 22, 2014. The postperiod (in which the new model of MDR was followed) included 2085 patients discharged between October 23, 2014, and April 30, 2015. We excluded 139 patients that died in the hospital prior to discharge and patients on the nonteaching and/or private practice service.

All data were provided by our institution’s Digital Solutions Department. Our institutional review board issued a letter of determination exempting this study from further review because it was deemed to be a quality improvement initiative.

Use of Lean Management to Redesign our MDR

Our institution has incorporated the Lean management system to continually add value to services through the elimination of waste, thus simultaneously optimizing the quality of patient care, cost, and patient satisfaction.¹¹ Lean, derived from the Toyota Production System, has long been used in manufacturing and in recent decades has spread to healthcare.¹² We leveraged the following 3 key Lean techniques to redesign our MDR: (1) value stream management (VSM), (2) rapid process improvement workshops (RPIW), and (3) active daily management (ADM), as detailed in supplementary Appendix 1.

Interventions

Outcomes

The primary outcome was mean LOS. The secondary outcomes were (1) discharges before noon, (2) recording of the EDD in our EHR within 24 hours of admission (as time stamped on our EHR), and (3) patient satisfaction.

Data for patient satisfaction were obtained using the Press Ganey survey. We used data on patient satisfaction scores for the following 2 relevant questions on this survey: (1) extent to which the patient felt ready to be discharged and (2) how well staff worked together to care for the patient. Proportions of the “top-box” (“very good”) were used for the analysis. These survey data were available on 467 patients (11.7%) in the preperiod and 188 patients (9.0%) in the postperiod.

Data Analysis

A sensitivity analysis was conducted on a second cohort that included a subset of patients from the preperiod between November 1, 2013, and April 30, 2014, and a subset of patients from the postperiod between November 1, 2014, and April 1, 2015, to control for the calendar period (supplementary Appendix 2).

All analyses were conducted in R version 3.3.0, with the linear mixed-effects model lme4 statistical package.^13,14

Table 3 shows the differences in the outcomes between the pre- and postperiods. There was no change in the LOS or LOS adjusted for CMI. There was a 3.9% increase in discharges before noon in the postperiod compared with the preperiod (95% CI, 2.4% to 5.3%; P < .001). There was a 9.9% increase in the percentage of patients for whom the EDD was recorded in our EHR within 24 hours of admission (95% CI, 7.4% to 12.4%; P < .001). There was no change in the “top-box” patient satisfaction scores.

There were only marginal differences in the results between the entire cohort and a second subset cohort used for sensitivity analysis (supplementary Appendix 2).

In our study, there was no change in the mean LOS with the new model of MDR. There was an increase in discharges before noon and in recording of the EDD in our EHR within 24 hours of admission in the postperiod when the Lean-based new model of MDR was utilized. There was no change in patient satisfaction. With no change in staffing, we were able to accommodate the increase in the discharge volume in the postperiod.

We believe our results are attributable to several factors, including clearly defined roles and responsibilities for all participants of MDR, the inclusion of more experienced general medicine attending physician (compared with housestaff), Lean management techniques to identify gaps in the patient’s journey from emergency department to discharge using VSM, the development of appropriate workflows and standard work on how the multidisciplinary teams would work together at RPIWs, and ADM to ensure sustainability and engagement among frontline members and institutional leaders. In order to sustain this, we planned to continue monitoring data in daily, weekly, and monthly forums with senior physician and administrative leaders. Planning for additional interventions is underway, including moving MDR to the bedside, instituting an afternoon “check-in” that would enable more detailed action planning, and addressing barriers in a timely manner for patients ready to discharge the following day.

Our study has a few limitations. First, this is an observational study that cannot determine causation. Second, this is a single-center study conducted on patients only on the general medicine teaching service. Third, there were several concurrent interventions implemented at our institution to improve LOS, throughput, and patient satisfaction in addition to MDR, thus making it difficult to isolate the impact of our intervention. Fourth, in the new model of MDR, rounds took place only 5 days per week, thereby possibly limiting the potential impact on our outcomes. Fifth, while we showed improvements in the discharges before noon and recording of EDD in the post period, we were not able to achieve our target of 25% discharges before noon or 100% recording of EDD in this time period. We believe the limited amount of time between the pre- and postperiods to allow for adoption and learning of the processes might have contributed to the underestimation of the impact of the new model of MDR, thereby limiting our ability to achieve our targets. Sixth, the response rate on the Press Ganey survey was low, and we did not directly survey patients or families for their satisfaction with MDR.

Our study has several strengths. To our knowledge, this is the first study to embed Lean management techniques in the design of MDR in the inpatient setting. While several studies have demonstrated improvements in discharges before noon through the implementation of MDR, they have not incorporated Lean management techniques, which we believe are critical to ensure the sustainability of results.^1,3,5,6,8,15 Second, while it was not measured, there was a high level of provider engagement in the process in the new model of MDR. Third, because the MDR were conducted at the nurse’s station on each inpatient unit in the new model instead of in a conference room, it was well attended by all members of the multidisciplinary team. Fourth, the presence of a visibility board allowed for all team members to have easy access to visual feedback throughout the day, even if they were not present at the MDR. Fifth, we believe that there was also more accurate estimation of the date and time of discharge in the new model of MDR because the discussion was facilitated by the case manager, who is experienced in identifying barriers to discharge (compared with the housestaff in the old model of MDR), and included the more experienced attending physician. Finally, the consistent presence of a multidisciplinary team at MDR allowed for the incorporation of everyone’s concerns at one time, thereby limiting the need for paging multiple disciplines throughout the day, which led to quicker resolution of issues and assignment of pending tasks.

In conclusion, our study shows no change in the mean LOS when the Lean-based model of MDR was utilized. Our study demonstrates an increase in discharges before noon and in recording of EDD on our EHR within 24 hours of admission in the post period when the Lean-based model of MDR was utilized. There was no change in patient satisfaction. While this study was conducted at an academic medical center on the general medicine wards, we believe our new model of MDR, which leveraged Lean management techniques, may successfully impact patient flow in all inpatient clinical services and nonteaching hospitals.

Disclosure

The authors report no financial conflicts of interest and have nothing to disclose.

Given that multiple disciplines are often involved in caring for patients admitted to the hospital, timely communication, collaboration, and coordination amongst various disciplines is necessary for safe and effective patient care.¹ With the focus on improving patient satisfaction and throughput in hospitals, it is also important to make more accurate predictions of the discharge date and allow time for patients and their families to prepare for discharge.^2-4

Multidisciplinary rounds (MDR) are defined as structured daily communication amongst key members of the patient’s care team (eg, nurses, physicians, case managers, social workers, pharmacists, and rehabilitation services). MDR have shown to be a useful strategy for ensuring that all members of the care team are updated on the plan of care for the patient.⁵ During MDR, a brief “check-in” discussing the patient’s plan of care, pending needs, and barriers to discharge allows all team members, patients, and families to effectively coordinate care and plan and prepare for discharge.

Multiple studies have reported increased collaboration and improved communication between disciplines with the use of such multidisciplinary rounding.^2,5-7 Additionally, MDR have been shown to improve patient outcomes⁸ and reduce adverse events,⁹ length of stay (LOS),^6,8 cost of care,⁸ and readmissions.¹

We redesigned MDR on the general medicine wards at our institution in October 2014 by using Lean management techniques. Lean is defined as a set of philosophies and methods that aim to create transformation in thinking, behavior, and culture in each process, with the goal of maximizing the value for the patients and providers, adding efficiency, and reducing waste and waits.¹⁰

In this study, we evaluate whether this new model of MDR was associated with a decrease in the LOS. We also evaluate whether this new model of MDR was associated with an increase in discharges before noon, documentation of estimated discharge date (EDD) in our electronic health record (EHR), and patient satisfaction.

Setting, Design, and Patients

The study was conducted on the teaching general medicine service at our institution, an urban, 484-bed academic hospital. The general medicine service has patients on 4 inpatient units (total of 95 beds) and is managed by 5 teaching service teams.

We performed a pre-post study. The preperiod (in which the old model of MDR was followed) included 4000 patients discharged between September 1, 2013, and October 22, 2014. The postperiod (in which the new model of MDR was followed) included 2085 patients discharged between October 23, 2014, and April 30, 2015. We excluded 139 patients that died in the hospital prior to discharge and patients on the nonteaching and/or private practice service.

All data were provided by our institution’s Digital Solutions Department. Our institutional review board issued a letter of determination exempting this study from further review because it was deemed to be a quality improvement initiative.

Use of Lean Management to Redesign our MDR

Our institution has incorporated the Lean management system to continually add value to services through the elimination of waste, thus simultaneously optimizing the quality of patient care, cost, and patient satisfaction.¹¹ Lean, derived from the Toyota Production System, has long been used in manufacturing and in recent decades has spread to healthcare.¹² We leveraged the following 3 key Lean techniques to redesign our MDR: (1) value stream management (VSM), (2) rapid process improvement workshops (RPIW), and (3) active daily management (ADM), as detailed in supplementary Appendix 1.

Interventions

Outcomes

The primary outcome was mean LOS. The secondary outcomes were (1) discharges before noon, (2) recording of the EDD in our EHR within 24 hours of admission (as time stamped on our EHR), and (3) patient satisfaction.

Data for patient satisfaction were obtained using the Press Ganey survey. We used data on patient satisfaction scores for the following 2 relevant questions on this survey: (1) extent to which the patient felt ready to be discharged and (2) how well staff worked together to care for the patient. Proportions of the “top-box” (“very good”) were used for the analysis. These survey data were available on 467 patients (11.7%) in the preperiod and 188 patients (9.0%) in the postperiod.

Data Analysis

A sensitivity analysis was conducted on a second cohort that included a subset of patients from the preperiod between November 1, 2013, and April 30, 2014, and a subset of patients from the postperiod between November 1, 2014, and April 1, 2015, to control for the calendar period (supplementary Appendix 2).

All analyses were conducted in R version 3.3.0, with the linear mixed-effects model lme4 statistical package.^13,14

Table 3 shows the differences in the outcomes between the pre- and postperiods. There was no change in the LOS or LOS adjusted for CMI. There was a 3.9% increase in discharges before noon in the postperiod compared with the preperiod (95% CI, 2.4% to 5.3%; P < .001). There was a 9.9% increase in the percentage of patients for whom the EDD was recorded in our EHR within 24 hours of admission (95% CI, 7.4% to 12.4%; P < .001). There was no change in the “top-box” patient satisfaction scores.

There were only marginal differences in the results between the entire cohort and a second subset cohort used for sensitivity analysis (supplementary Appendix 2).

In our study, there was no change in the mean LOS with the new model of MDR. There was an increase in discharges before noon and in recording of the EDD in our EHR within 24 hours of admission in the postperiod when the Lean-based new model of MDR was utilized. There was no change in patient satisfaction. With no change in staffing, we were able to accommodate the increase in the discharge volume in the postperiod.

We believe our results are attributable to several factors, including clearly defined roles and responsibilities for all participants of MDR, the inclusion of more experienced general medicine attending physician (compared with housestaff), Lean management techniques to identify gaps in the patient’s journey from emergency department to discharge using VSM, the development of appropriate workflows and standard work on how the multidisciplinary teams would work together at RPIWs, and ADM to ensure sustainability and engagement among frontline members and institutional leaders. In order to sustain this, we planned to continue monitoring data in daily, weekly, and monthly forums with senior physician and administrative leaders. Planning for additional interventions is underway, including moving MDR to the bedside, instituting an afternoon “check-in” that would enable more detailed action planning, and addressing barriers in a timely manner for patients ready to discharge the following day.

Our study has a few limitations. First, this is an observational study that cannot determine causation. Second, this is a single-center study conducted on patients only on the general medicine teaching service. Third, there were several concurrent interventions implemented at our institution to improve LOS, throughput, and patient satisfaction in addition to MDR, thus making it difficult to isolate the impact of our intervention. Fourth, in the new model of MDR, rounds took place only 5 days per week, thereby possibly limiting the potential impact on our outcomes. Fifth, while we showed improvements in the discharges before noon and recording of EDD in the post period, we were not able to achieve our target of 25% discharges before noon or 100% recording of EDD in this time period. We believe the limited amount of time between the pre- and postperiods to allow for adoption and learning of the processes might have contributed to the underestimation of the impact of the new model of MDR, thereby limiting our ability to achieve our targets. Sixth, the response rate on the Press Ganey survey was low, and we did not directly survey patients or families for their satisfaction with MDR.

Our study has several strengths. To our knowledge, this is the first study to embed Lean management techniques in the design of MDR in the inpatient setting. While several studies have demonstrated improvements in discharges before noon through the implementation of MDR, they have not incorporated Lean management techniques, which we believe are critical to ensure the sustainability of results.^1,3,5,6,8,15 Second, while it was not measured, there was a high level of provider engagement in the process in the new model of MDR. Third, because the MDR were conducted at the nurse’s station on each inpatient unit in the new model instead of in a conference room, it was well attended by all members of the multidisciplinary team. Fourth, the presence of a visibility board allowed for all team members to have easy access to visual feedback throughout the day, even if they were not present at the MDR. Fifth, we believe that there was also more accurate estimation of the date and time of discharge in the new model of MDR because the discussion was facilitated by the case manager, who is experienced in identifying barriers to discharge (compared with the housestaff in the old model of MDR), and included the more experienced attending physician. Finally, the consistent presence of a multidisciplinary team at MDR allowed for the incorporation of everyone’s concerns at one time, thereby limiting the need for paging multiple disciplines throughout the day, which led to quicker resolution of issues and assignment of pending tasks.

In conclusion, our study shows no change in the mean LOS when the Lean-based model of MDR was utilized. Our study demonstrates an increase in discharges before noon and in recording of EDD on our EHR within 24 hours of admission in the post period when the Lean-based model of MDR was utilized. There was no change in patient satisfaction. While this study was conducted at an academic medical center on the general medicine wards, we believe our new model of MDR, which leveraged Lean management techniques, may successfully impact patient flow in all inpatient clinical services and nonteaching hospitals.

Disclosure

The authors report no financial conflicts of interest and have nothing to disclose.