Vineet M. Arora, MD, MAPP

Vision impairment is an under‐recognized risk factor for adverse events among hospitalized patients.[1, 2, 3] Inpatients with poor vision are at increased risk for falls and delirium[1, 3] and have more difficulty taking medications.[4, 5] They may also be at risk for being unable to read critical health information, including consent forms and discharge instructions, or decreased quality of life such as simply ordering food from menus. However, vision is neither routinely tested nor documented for inpatients. Low‐cost ($8 and up) nonprescription reading glasses, known as readers may be a simple, high‐value intervention to improve inpatients' vision. We aimed to study initial feasibility and efficacy of screening and correcting inpatients' vision.

METHODS

From June 2012 through January 2014, research assistants (RAs) identified eligible (adults [18 years], English speaking) participants daily from electronic medical records as part of an ongoing study of general medicine inpatients measuring quality‐of‐care at the University of Chicago Medicine.[6] RAs tested visual acuity using Snellen pocket charts (participants wore corrective lenses if available). For eligible participants, readers were tested with sequential fitting (+2/+2.25/+2.75/+3.25) until vision was corrected (sufficient vision: at least 20/50 acuity in at least 1 eye).[7] Eligible participants included those with insufficient vision who were not already wearing corrective lenses and had no documented blindness or medically severe vision loss, for whom nonprescription readers would be unlikely to correct vision deficiencies such as cataracts or glaucoma. The study was approved by the University of Chicago Institutional Review Board (IRB #9967).

Of note, although readers are typically used in populations over 40 years of age, readers were fitted for all participants to assess their utility for any hospitalized adult patient. Upon completing the vision screening and readers interventions, participants received instruction on how to access vision care and how to obtain readers (if they corrected vision) after hospital discharge.

Descriptive statistics and tests of comparison, including t tests and [2] tests, were used when appropriate. All analyses were performed using Stata version 12 (StataCorp, College Station, TX).

RESULTS

Over 800 participants' vision was screened (n=853); the majority were female (56%, 480/853), African American (76%, 650/853), with a mean age of 53.4 years (standard deviation 18.7), consistent with our study site's demographics. Over one‐third (36%, 304/853) of participants had insufficient vision. Older (65 years) participants (56%, 136/244) were more likely to have insufficient vision than younger participants (28%, 168/608; P<0.001).

Participants with insufficient vision were wearing their own corrective lenses during the testing (150/304, 49%), did not use corrective lenses (53/304, 17%), or were without available corrective lenses (99/304, 33%) (Figure 1A).

Figure 1

One‐hundred sixteen of 304 participants approached for the readers intervention were eligible (112 reported medical eye disease, 65 were wearing lenses, and 11 refused or were discharged before intervention implementation).

Nonprescription readers corrected the majority of eligible participants' vision (82%, 95/116). Most participants' (81/116, 70%) vision was corrected using the 2 lowest calibration readers (+2/+2.25); another 14 participants' (12%) vision was corrected with higher‐strength lenses (+2.75/+3.25) (Figure 1B)

DISCUSSION

We found that over one‐third of the inpatients we examined have poor vision. Furthermore, among an easily identified subgroup of inpatients with poor vision, low‐cost readers successfully corrected most participants' vision. Although preventive health is not commonly considered an inpatient issue, hospitalists and other clinicians working in the inpatient setting can play an important role in identifying opportunities to provide high‐value care related to patients' vision.

Several important ethical, safety, and cost considerations related to these findings exist. Hospitalized patients commonly sign written informed consent; therefore, due diligence to ensure patients' ability to read and understand the forms is imperative. Further, inpatient delirium is common, particularly among older patients.[8] Existing or new onset delirium occurs in up to 24% to 35% of elderly inpatients.[8] Vision is an important risk factor for multifactorial inpatient delirium, and early vision correction has been shown to improve delirium rates, as part of a multicomponent intervention.[9] Hospital‐related patient costs per delirium episode have been estimated at $16,303 to $64,421.[10] The cost of a multicomponent intervention was $6341 per case of delirium prevented,[9] whereas only 1 potentially critical component, the cost of readers ($8+), would pale in comparison.[1] Vision screening takes approximately 2.25 minutes plus 2 to 6 minutes for the readers' assessment, with little training and high fidelity. Therefore, this easily implemented, potentially cost saving, intervention targeting inpatients with poor vision may improve patient safety and quality of life in the hospital and even after discharge.

Limitations of the study include considerations of generalizability, as participants were from a single, urban, academic medical center. Additionally, long‐term benefits of the readers intervention were not assessed in this study. Finally, RAs provided the assessments; therefore, further work is required to determine costs of efficient large‐scale clinical implementation through nurse‐led programs.

Despite these study limitations, the surprisingly high prevalence of poor vision among inpatients is a call to action for hospitalists. Future work should investigate the impact and cost of vision correction on hospital outcomes such as patient satisfaction, reduced rehospitalizations, and decreased delirium.[11]

Vision impairment is an under‐recognized risk factor for adverse events among hospitalized patients.[1, 2, 3] Inpatients with poor vision are at increased risk for falls and delirium[1, 3] and have more difficulty taking medications.[4, 5] They may also be at risk for being unable to read critical health information, including consent forms and discharge instructions, or decreased quality of life such as simply ordering food from menus. However, vision is neither routinely tested nor documented for inpatients. Low‐cost ($8 and up) nonprescription reading glasses, known as readers may be a simple, high‐value intervention to improve inpatients' vision. We aimed to study initial feasibility and efficacy of screening and correcting inpatients' vision.

METHODS

From June 2012 through January 2014, research assistants (RAs) identified eligible (adults [18 years], English speaking) participants daily from electronic medical records as part of an ongoing study of general medicine inpatients measuring quality‐of‐care at the University of Chicago Medicine.[6] RAs tested visual acuity using Snellen pocket charts (participants wore corrective lenses if available). For eligible participants, readers were tested with sequential fitting (+2/+2.25/+2.75/+3.25) until vision was corrected (sufficient vision: at least 20/50 acuity in at least 1 eye).[7] Eligible participants included those with insufficient vision who were not already wearing corrective lenses and had no documented blindness or medically severe vision loss, for whom nonprescription readers would be unlikely to correct vision deficiencies such as cataracts or glaucoma. The study was approved by the University of Chicago Institutional Review Board (IRB #9967).

Of note, although readers are typically used in populations over 40 years of age, readers were fitted for all participants to assess their utility for any hospitalized adult patient. Upon completing the vision screening and readers interventions, participants received instruction on how to access vision care and how to obtain readers (if they corrected vision) after hospital discharge.

Descriptive statistics and tests of comparison, including t tests and [2] tests, were used when appropriate. All analyses were performed using Stata version 12 (StataCorp, College Station, TX).

RESULTS

Over 800 participants' vision was screened (n=853); the majority were female (56%, 480/853), African American (76%, 650/853), with a mean age of 53.4 years (standard deviation 18.7), consistent with our study site's demographics. Over one‐third (36%, 304/853) of participants had insufficient vision. Older (65 years) participants (56%, 136/244) were more likely to have insufficient vision than younger participants (28%, 168/608; P<0.001).

Participants with insufficient vision were wearing their own corrective lenses during the testing (150/304, 49%), did not use corrective lenses (53/304, 17%), or were without available corrective lenses (99/304, 33%) (Figure 1A).

Figure 1

One‐hundred sixteen of 304 participants approached for the readers intervention were eligible (112 reported medical eye disease, 65 were wearing lenses, and 11 refused or were discharged before intervention implementation).

Nonprescription readers corrected the majority of eligible participants' vision (82%, 95/116). Most participants' (81/116, 70%) vision was corrected using the 2 lowest calibration readers (+2/+2.25); another 14 participants' (12%) vision was corrected with higher‐strength lenses (+2.75/+3.25) (Figure 1B)

DISCUSSION

We found that over one‐third of the inpatients we examined have poor vision. Furthermore, among an easily identified subgroup of inpatients with poor vision, low‐cost readers successfully corrected most participants' vision. Although preventive health is not commonly considered an inpatient issue, hospitalists and other clinicians working in the inpatient setting can play an important role in identifying opportunities to provide high‐value care related to patients' vision.

Several important ethical, safety, and cost considerations related to these findings exist. Hospitalized patients commonly sign written informed consent; therefore, due diligence to ensure patients' ability to read and understand the forms is imperative. Further, inpatient delirium is common, particularly among older patients.[8] Existing or new onset delirium occurs in up to 24% to 35% of elderly inpatients.[8] Vision is an important risk factor for multifactorial inpatient delirium, and early vision correction has been shown to improve delirium rates, as part of a multicomponent intervention.[9] Hospital‐related patient costs per delirium episode have been estimated at $16,303 to $64,421.[10] The cost of a multicomponent intervention was $6341 per case of delirium prevented,[9] whereas only 1 potentially critical component, the cost of readers ($8+), would pale in comparison.[1] Vision screening takes approximately 2.25 minutes plus 2 to 6 minutes for the readers' assessment, with little training and high fidelity. Therefore, this easily implemented, potentially cost saving, intervention targeting inpatients with poor vision may improve patient safety and quality of life in the hospital and even after discharge.

Limitations of the study include considerations of generalizability, as participants were from a single, urban, academic medical center. Additionally, long‐term benefits of the readers intervention were not assessed in this study. Finally, RAs provided the assessments; therefore, further work is required to determine costs of efficient large‐scale clinical implementation through nurse‐led programs.

Despite these study limitations, the surprisingly high prevalence of poor vision among inpatients is a call to action for hospitalists. Future work should investigate the impact and cost of vision correction on hospital outcomes such as patient satisfaction, reduced rehospitalizations, and decreased delirium.[11]

Vision impairment is an under‐recognized risk factor for adverse events among hospitalized patients.[1, 2, 3] Inpatients with poor vision are at increased risk for falls and delirium[1, 3] and have more difficulty taking medications.[4, 5] They may also be at risk for being unable to read critical health information, including consent forms and discharge instructions, or decreased quality of life such as simply ordering food from menus. However, vision is neither routinely tested nor documented for inpatients. Low‐cost ($8 and up) nonprescription reading glasses, known as readers may be a simple, high‐value intervention to improve inpatients' vision. We aimed to study initial feasibility and efficacy of screening and correcting inpatients' vision.

METHODS

From June 2012 through January 2014, research assistants (RAs) identified eligible (adults [18 years], English speaking) participants daily from electronic medical records as part of an ongoing study of general medicine inpatients measuring quality‐of‐care at the University of Chicago Medicine.[6] RAs tested visual acuity using Snellen pocket charts (participants wore corrective lenses if available). For eligible participants, readers were tested with sequential fitting (+2/+2.25/+2.75/+3.25) until vision was corrected (sufficient vision: at least 20/50 acuity in at least 1 eye).[7] Eligible participants included those with insufficient vision who were not already wearing corrective lenses and had no documented blindness or medically severe vision loss, for whom nonprescription readers would be unlikely to correct vision deficiencies such as cataracts or glaucoma. The study was approved by the University of Chicago Institutional Review Board (IRB #9967).

Of note, although readers are typically used in populations over 40 years of age, readers were fitted for all participants to assess their utility for any hospitalized adult patient. Upon completing the vision screening and readers interventions, participants received instruction on how to access vision care and how to obtain readers (if they corrected vision) after hospital discharge.

Descriptive statistics and tests of comparison, including t tests and [2] tests, were used when appropriate. All analyses were performed using Stata version 12 (StataCorp, College Station, TX).

RESULTS

Over 800 participants' vision was screened (n=853); the majority were female (56%, 480/853), African American (76%, 650/853), with a mean age of 53.4 years (standard deviation 18.7), consistent with our study site's demographics. Over one‐third (36%, 304/853) of participants had insufficient vision. Older (65 years) participants (56%, 136/244) were more likely to have insufficient vision than younger participants (28%, 168/608; P<0.001).

Participants with insufficient vision were wearing their own corrective lenses during the testing (150/304, 49%), did not use corrective lenses (53/304, 17%), or were without available corrective lenses (99/304, 33%) (Figure 1A).

Figure 1

One‐hundred sixteen of 304 participants approached for the readers intervention were eligible (112 reported medical eye disease, 65 were wearing lenses, and 11 refused or were discharged before intervention implementation).

Nonprescription readers corrected the majority of eligible participants' vision (82%, 95/116). Most participants' (81/116, 70%) vision was corrected using the 2 lowest calibration readers (+2/+2.25); another 14 participants' (12%) vision was corrected with higher‐strength lenses (+2.75/+3.25) (Figure 1B)

DISCUSSION

We found that over one‐third of the inpatients we examined have poor vision. Furthermore, among an easily identified subgroup of inpatients with poor vision, low‐cost readers successfully corrected most participants' vision. Although preventive health is not commonly considered an inpatient issue, hospitalists and other clinicians working in the inpatient setting can play an important role in identifying opportunities to provide high‐value care related to patients' vision.

Several important ethical, safety, and cost considerations related to these findings exist. Hospitalized patients commonly sign written informed consent; therefore, due diligence to ensure patients' ability to read and understand the forms is imperative. Further, inpatient delirium is common, particularly among older patients.[8] Existing or new onset delirium occurs in up to 24% to 35% of elderly inpatients.[8] Vision is an important risk factor for multifactorial inpatient delirium, and early vision correction has been shown to improve delirium rates, as part of a multicomponent intervention.[9] Hospital‐related patient costs per delirium episode have been estimated at $16,303 to $64,421.[10] The cost of a multicomponent intervention was $6341 per case of delirium prevented,[9] whereas only 1 potentially critical component, the cost of readers ($8+), would pale in comparison.[1] Vision screening takes approximately 2.25 minutes plus 2 to 6 minutes for the readers' assessment, with little training and high fidelity. Therefore, this easily implemented, potentially cost saving, intervention targeting inpatients with poor vision may improve patient safety and quality of life in the hospital and even after discharge.

Limitations of the study include considerations of generalizability, as participants were from a single, urban, academic medical center. Additionally, long‐term benefits of the readers intervention were not assessed in this study. Finally, RAs provided the assessments; therefore, further work is required to determine costs of efficient large‐scale clinical implementation through nurse‐led programs.

Despite these study limitations, the surprisingly high prevalence of poor vision among inpatients is a call to action for hospitalists. Future work should investigate the impact and cost of vision correction on hospital outcomes such as patient satisfaction, reduced rehospitalizations, and decreased delirium.[11]

Over the last decade, there has been an unprecedented focus on physician handoffs in US hospitals. One major reason for this are the reductions in residency duty hours that have been mandated by the American Council for Graduate Medical Education (ACGME), first in 2003 and subsequently revised in 2011.[1, 2] As residents work fewer hours, experts believe that potential safety gains from reduced fatigue are countered by an increase in the number of handoffs, which represent a risk due to the potential miscommunication. Prior studies show that critical patient information is often lost or altered during this transfer of clinical information and professional responsibility, which can result in patient harm.[3, 4] As a result of these concerns, the ACGME now requires residency programs to ensure and monitor effective, structured hand‐over processes to facilitate both continuity of care and patient safety. Programs must ensure that residents are competent in communicating with team members in the hand‐over process.[2] Moreover, handoffs have also been a major improvement focus for organizations with broader scope than teaching hospitals, including the World Health Organization, Joint Commission, and the Society for Hospital Medicine (SHM).[5, 6, 7]

Despite this focus on handoffs, monitoring quality of handoffs has proven challenging due to lack of a reliable and validated tool to measure handoff quality. More recently, the Accreditation Council of Graduate Medical Education's introduction of the Next Accreditation System, with its focus on direct observation of clinical skills to achieve milestones, makes it crucial for residency educators to have valid tools to measure competence in handoffs. As a result, it is critical that instruments to measure handoff performance are not only created but also validated.[8]

To help fill this gap, we previously reported on the development of a 9‐item Handoff Clinical Examination Exercise (CEX) assessment tool. The Handoff CEX, designed for use by those participating in the handoff or by a third‐party observer, can be used to rate the quality of patient handoffs in domains such as professionalism and communication skills between the receiver and sender of patient information.[9, 10] Despite prior demonstration of feasibility of use, the initial tool was perceived as lengthy and redundant. In addition, although the tool has been shown to discriminate between performance of novice and expert nurses, the construct validity of this tool has not been established.[11] Establishing construct validity is important to ensuring that the tool can measure the construct in question, namely whether it detects those who are actually competent to perform handoffs safely and effectively. We present here the results of the development of a shorter Handoff Mini‐CEX, along with the formal establishment of its construct validity, namely its ability to distinguish between levels of performance in 3 domains of handoff quality.

METHODS

RESULTS

Forty‐seven faculty members (14=site 1; 33=site 2) participated in the validation workshops (2 at the University of Chicago, and 2 at Yale University), which were held in August 2011 and September 2011, providing a total of 172 observations of a possible 191 (90%).

The overall handoff quality ratings for the superior, gold standard video (superior communication, professionalism, and communication) ranged from 7 to 9 with a mean of 8.5 (standard deviation [SD] 0.7). The overall ratings for the video depicting satisfactory communication (satisfactory communication, superior professionalism and setting) ranged from 5 to 9 with a mean of 7.3 (SD 1.1). The overall ratings for the unsatisfactory communication (unsatisfactory communication, superior professionalism and setting) video ranged from 1 to 7 with a mean of 2.6 (SD 1.2). The overall ratings for the satisfactory professionalism video (satisfactory professionalism, superior communication and setting) ranged from 4 to 8 with a mean of 5.7 (SD 1.3). The overall ratings for the unsatisfactory professionalism (unsatisfactory professionalism, superior communication and setting) video ranged from 2 to 5 with a mean of 2.4 (SD 1.03). Finally, the overall ratings for the unsatisfactory setting (unsatisfactory setting, superior communication and professionalism) video ranged from 1 to 8 with a mean of 3.1 (SD 1.7).

Figure 1 demonstrates that for the domain of communication, the raters were able to discern the unsatisfactory performance but had difficulty reliably distinguishing between superior and satisfactory performance. Figure 2 illustrates that for the domain of professionalism, raters were able to detect the videos' changing levels of performance at the extremes of behavior, with unsatisfactory and superior displays more readily identified. Figure 3 shows that for the domain of setting, the raters were able to discern the unsatisfactory versus superior level of the changing setting. Of note, we also found a moderate significant correlation between ratings of professionalism and communication (r=0.47, P<0.001).

Figure 1

Figure 2

Figure 3

The Cronbach , or measurement of internal reliability and consistency, for the Handoff Mini‐CEX (3 items plus overall) was 0.77, indicating high internal reliability and consistency. Using data from University of Chicago, where raters were labeled with a unique identifier, the Kendall coefficient of concordance was calculated to be 0.79, demonstrating high inter‐rater reliability of the faculty raters. High inter‐rater reliability was also seen using intraclass coefficients for each domain: communication (0.84), professionalism (0.68), setting (0.83), and overall (0.89). Using generalizability analysis, the average reliability was determined to be above 0.9 for all domains (0.99 for overall).

Last, the 2‐way ANOVA (n=75 observations from 13 raters) revealed no evidence of rater bias when examining the coefficient for attending rater (P=0.55 for professionalism, P=0.45 for communication, P=0.92 for setting). The range of scores for each video, however, was broad (Table 2).

DISCUSSION

This study demonstrates that valid conclusions on handoff performance can be drawn using the Handoff CEX as the instrument to rate handoff quality. Utilizing standardized videos depicting varying levels of performance communication, professionalism, and setting, the Handoff Mini‐CEX has demonstrated potential to discern between increasing levels of performance, providing evidence for the construct validity of the instrument.

We observed that faculty could reliably detect unsatisfactory professionalism with ease, and that there was a distinct correlation between faculty ratings and the internally set levels of performance displayed in the videos. This trend demonstrated that faculty were able to discern different levels of professionalism using the Handoff Mini‐CEX. It became more difficult, however, for faculty to detect superior professionalism when the domain of communication was permuted. If the sender of the handoff was professional but the information delivered was disorganized, inaccurate, and missing crucial pieces of information, the faculty perceived this ineffective communication as unprofessional. Prior literature on professionalism has found that communication is a necessary component of professional behavior, and consequently, being a competent communicator is necessary to fulfill ones duty as a professional physician.[15, 16]

This is of note because we did find a moderate significant correlation between ratings of professionalism and communication. It is possible that this distinction would be made clearer with formal rater training in the future prior to any evaluations. However, it is also possible that professionalism and communication, due to a synergistic role between the 2 domains, cannot be separated. If this is the case, it would be important to educate clinicians to present patients in a concise, clear, and accurate way with a professional demeanor. Acknowledging professional responsibility as an integral piece of patient care is also critical in effectively communicating patient information.[5]

We also noted that faculty could detect unsatisfactory communication consistently; however, they were unable to differentiate between satisfactory and superior communication reliably or consistently. Because the unsatisfactory professionalism, unsatisfactory setting, and satisfactory professionalism videos all demonstrated superior communication, we believe that the faculty penalized communication when distractions, in the form of interruptions and rude behavior by the resident giving the handoff, interrupted the flow of the handoff. Thus, the wide ranges in scores observed by some raters may be attributed to this interaction between the Handoff Mini‐CEX domains. In the future, definitions of the anchors, including at the middle spectrum of performance, and rater training may improve the ability of raters to distinguish performance between each domain.

The overall value of the Handoff Mini‐CEX is in its ease of use, in part due to its brevity, as well as evidence for its validity in distinguishing between varying levels of performance. Given the emphasis on monitoring handoff quality and performance, the Handoff Mini‐CEX provides a standard foundation from which baseline handoff performance can be easily measured and improved. Moreover, it can also be used to give individual feedback to a specific practicing clinician on their practices and an opportunity to improve. This is particularly important given current recommendations by the Joint Commission, that handoffs are standardized, and by the ACGME, that residents are competent in handoff skills. Moreover, given the creation of the SHM's handoff recommendations and handoffs as a core competency for hospitalists, the tool provides the ability for hospitalist programs to actually assess their handoff practices as baseline measurements for any quality improvement activities that may take place.

Faculty were able to discern the superior and unsatisfactory levels of setting with ease. After watching and rating the videos, participants said that the chaotic scene of the unsatisfactory setting video had significant authenticity, and that they were constantly interrupted during their own handoffs by pages, phone calls, and people entering the handoff space. System‐level fixes, such as protected time and dedicated space for handoffs, and discouraging pages to be sent during the designated handoff time, could mitigate the reality of unsatisfactory settings.[17, 18]

Our study has several limitations. First, although this study was held at 2 sites, it included a small number of faculty, which can impact the generalizability of our findings. Implementation varied at Yale University and the University of Chicago, preventing use of all data for all analyses. Furthermore, institutional culture may also impact faculty raters' perceptions, so future work aims at repeating our protocol at partner institutions, increasing both the number and diversity of participants. We were also unable to compare the new shorter Handoff Mini‐CEX to the larger 9‐item Handoff CEX in this study.

Despite these limitations, we believe that the Handoff Mini‐CEX, has future potential as an instrument with which to make valid and reliable conclusions about handoff quality, and could be used to both evaluate handoff quality and as an educational tool for trainees and faculty on effective handoff communication.

Over the last decade, there has been an unprecedented focus on physician handoffs in US hospitals. One major reason for this are the reductions in residency duty hours that have been mandated by the American Council for Graduate Medical Education (ACGME), first in 2003 and subsequently revised in 2011.[1, 2] As residents work fewer hours, experts believe that potential safety gains from reduced fatigue are countered by an increase in the number of handoffs, which represent a risk due to the potential miscommunication. Prior studies show that critical patient information is often lost or altered during this transfer of clinical information and professional responsibility, which can result in patient harm.[3, 4] As a result of these concerns, the ACGME now requires residency programs to ensure and monitor effective, structured hand‐over processes to facilitate both continuity of care and patient safety. Programs must ensure that residents are competent in communicating with team members in the hand‐over process.[2] Moreover, handoffs have also been a major improvement focus for organizations with broader scope than teaching hospitals, including the World Health Organization, Joint Commission, and the Society for Hospital Medicine (SHM).[5, 6, 7]

Despite this focus on handoffs, monitoring quality of handoffs has proven challenging due to lack of a reliable and validated tool to measure handoff quality. More recently, the Accreditation Council of Graduate Medical Education's introduction of the Next Accreditation System, with its focus on direct observation of clinical skills to achieve milestones, makes it crucial for residency educators to have valid tools to measure competence in handoffs. As a result, it is critical that instruments to measure handoff performance are not only created but also validated.[8]

To help fill this gap, we previously reported on the development of a 9‐item Handoff Clinical Examination Exercise (CEX) assessment tool. The Handoff CEX, designed for use by those participating in the handoff or by a third‐party observer, can be used to rate the quality of patient handoffs in domains such as professionalism and communication skills between the receiver and sender of patient information.[9, 10] Despite prior demonstration of feasibility of use, the initial tool was perceived as lengthy and redundant. In addition, although the tool has been shown to discriminate between performance of novice and expert nurses, the construct validity of this tool has not been established.[11] Establishing construct validity is important to ensuring that the tool can measure the construct in question, namely whether it detects those who are actually competent to perform handoffs safely and effectively. We present here the results of the development of a shorter Handoff Mini‐CEX, along with the formal establishment of its construct validity, namely its ability to distinguish between levels of performance in 3 domains of handoff quality.

METHODS

RESULTS

Forty‐seven faculty members (14=site 1; 33=site 2) participated in the validation workshops (2 at the University of Chicago, and 2 at Yale University), which were held in August 2011 and September 2011, providing a total of 172 observations of a possible 191 (90%).

The overall handoff quality ratings for the superior, gold standard video (superior communication, professionalism, and communication) ranged from 7 to 9 with a mean of 8.5 (standard deviation [SD] 0.7). The overall ratings for the video depicting satisfactory communication (satisfactory communication, superior professionalism and setting) ranged from 5 to 9 with a mean of 7.3 (SD 1.1). The overall ratings for the unsatisfactory communication (unsatisfactory communication, superior professionalism and setting) video ranged from 1 to 7 with a mean of 2.6 (SD 1.2). The overall ratings for the satisfactory professionalism video (satisfactory professionalism, superior communication and setting) ranged from 4 to 8 with a mean of 5.7 (SD 1.3). The overall ratings for the unsatisfactory professionalism (unsatisfactory professionalism, superior communication and setting) video ranged from 2 to 5 with a mean of 2.4 (SD 1.03). Finally, the overall ratings for the unsatisfactory setting (unsatisfactory setting, superior communication and professionalism) video ranged from 1 to 8 with a mean of 3.1 (SD 1.7).

Figure 1 demonstrates that for the domain of communication, the raters were able to discern the unsatisfactory performance but had difficulty reliably distinguishing between superior and satisfactory performance. Figure 2 illustrates that for the domain of professionalism, raters were able to detect the videos' changing levels of performance at the extremes of behavior, with unsatisfactory and superior displays more readily identified. Figure 3 shows that for the domain of setting, the raters were able to discern the unsatisfactory versus superior level of the changing setting. Of note, we also found a moderate significant correlation between ratings of professionalism and communication (r=0.47, P<0.001).

Figure 1

Figure 2

Figure 3

The Cronbach , or measurement of internal reliability and consistency, for the Handoff Mini‐CEX (3 items plus overall) was 0.77, indicating high internal reliability and consistency. Using data from University of Chicago, where raters were labeled with a unique identifier, the Kendall coefficient of concordance was calculated to be 0.79, demonstrating high inter‐rater reliability of the faculty raters. High inter‐rater reliability was also seen using intraclass coefficients for each domain: communication (0.84), professionalism (0.68), setting (0.83), and overall (0.89). Using generalizability analysis, the average reliability was determined to be above 0.9 for all domains (0.99 for overall).

Last, the 2‐way ANOVA (n=75 observations from 13 raters) revealed no evidence of rater bias when examining the coefficient for attending rater (P=0.55 for professionalism, P=0.45 for communication, P=0.92 for setting). The range of scores for each video, however, was broad (Table 2).

DISCUSSION

This study demonstrates that valid conclusions on handoff performance can be drawn using the Handoff CEX as the instrument to rate handoff quality. Utilizing standardized videos depicting varying levels of performance communication, professionalism, and setting, the Handoff Mini‐CEX has demonstrated potential to discern between increasing levels of performance, providing evidence for the construct validity of the instrument.

We observed that faculty could reliably detect unsatisfactory professionalism with ease, and that there was a distinct correlation between faculty ratings and the internally set levels of performance displayed in the videos. This trend demonstrated that faculty were able to discern different levels of professionalism using the Handoff Mini‐CEX. It became more difficult, however, for faculty to detect superior professionalism when the domain of communication was permuted. If the sender of the handoff was professional but the information delivered was disorganized, inaccurate, and missing crucial pieces of information, the faculty perceived this ineffective communication as unprofessional. Prior literature on professionalism has found that communication is a necessary component of professional behavior, and consequently, being a competent communicator is necessary to fulfill ones duty as a professional physician.[15, 16]

This is of note because we did find a moderate significant correlation between ratings of professionalism and communication. It is possible that this distinction would be made clearer with formal rater training in the future prior to any evaluations. However, it is also possible that professionalism and communication, due to a synergistic role between the 2 domains, cannot be separated. If this is the case, it would be important to educate clinicians to present patients in a concise, clear, and accurate way with a professional demeanor. Acknowledging professional responsibility as an integral piece of patient care is also critical in effectively communicating patient information.[5]

We also noted that faculty could detect unsatisfactory communication consistently; however, they were unable to differentiate between satisfactory and superior communication reliably or consistently. Because the unsatisfactory professionalism, unsatisfactory setting, and satisfactory professionalism videos all demonstrated superior communication, we believe that the faculty penalized communication when distractions, in the form of interruptions and rude behavior by the resident giving the handoff, interrupted the flow of the handoff. Thus, the wide ranges in scores observed by some raters may be attributed to this interaction between the Handoff Mini‐CEX domains. In the future, definitions of the anchors, including at the middle spectrum of performance, and rater training may improve the ability of raters to distinguish performance between each domain.

The overall value of the Handoff Mini‐CEX is in its ease of use, in part due to its brevity, as well as evidence for its validity in distinguishing between varying levels of performance. Given the emphasis on monitoring handoff quality and performance, the Handoff Mini‐CEX provides a standard foundation from which baseline handoff performance can be easily measured and improved. Moreover, it can also be used to give individual feedback to a specific practicing clinician on their practices and an opportunity to improve. This is particularly important given current recommendations by the Joint Commission, that handoffs are standardized, and by the ACGME, that residents are competent in handoff skills. Moreover, given the creation of the SHM's handoff recommendations and handoffs as a core competency for hospitalists, the tool provides the ability for hospitalist programs to actually assess their handoff practices as baseline measurements for any quality improvement activities that may take place.

Faculty were able to discern the superior and unsatisfactory levels of setting with ease. After watching and rating the videos, participants said that the chaotic scene of the unsatisfactory setting video had significant authenticity, and that they were constantly interrupted during their own handoffs by pages, phone calls, and people entering the handoff space. System‐level fixes, such as protected time and dedicated space for handoffs, and discouraging pages to be sent during the designated handoff time, could mitigate the reality of unsatisfactory settings.[17, 18]

Our study has several limitations. First, although this study was held at 2 sites, it included a small number of faculty, which can impact the generalizability of our findings. Implementation varied at Yale University and the University of Chicago, preventing use of all data for all analyses. Furthermore, institutional culture may also impact faculty raters' perceptions, so future work aims at repeating our protocol at partner institutions, increasing both the number and diversity of participants. We were also unable to compare the new shorter Handoff Mini‐CEX to the larger 9‐item Handoff CEX in this study.

Despite these limitations, we believe that the Handoff Mini‐CEX, has future potential as an instrument with which to make valid and reliable conclusions about handoff quality, and could be used to both evaluate handoff quality and as an educational tool for trainees and faculty on effective handoff communication.

Over the last decade, there has been an unprecedented focus on physician handoffs in US hospitals. One major reason for this are the reductions in residency duty hours that have been mandated by the American Council for Graduate Medical Education (ACGME), first in 2003 and subsequently revised in 2011.[1, 2] As residents work fewer hours, experts believe that potential safety gains from reduced fatigue are countered by an increase in the number of handoffs, which represent a risk due to the potential miscommunication. Prior studies show that critical patient information is often lost or altered during this transfer of clinical information and professional responsibility, which can result in patient harm.[3, 4] As a result of these concerns, the ACGME now requires residency programs to ensure and monitor effective, structured hand‐over processes to facilitate both continuity of care and patient safety. Programs must ensure that residents are competent in communicating with team members in the hand‐over process.[2] Moreover, handoffs have also been a major improvement focus for organizations with broader scope than teaching hospitals, including the World Health Organization, Joint Commission, and the Society for Hospital Medicine (SHM).[5, 6, 7]

Despite this focus on handoffs, monitoring quality of handoffs has proven challenging due to lack of a reliable and validated tool to measure handoff quality. More recently, the Accreditation Council of Graduate Medical Education's introduction of the Next Accreditation System, with its focus on direct observation of clinical skills to achieve milestones, makes it crucial for residency educators to have valid tools to measure competence in handoffs. As a result, it is critical that instruments to measure handoff performance are not only created but also validated.[8]

To help fill this gap, we previously reported on the development of a 9‐item Handoff Clinical Examination Exercise (CEX) assessment tool. The Handoff CEX, designed for use by those participating in the handoff or by a third‐party observer, can be used to rate the quality of patient handoffs in domains such as professionalism and communication skills between the receiver and sender of patient information.[9, 10] Despite prior demonstration of feasibility of use, the initial tool was perceived as lengthy and redundant. In addition, although the tool has been shown to discriminate between performance of novice and expert nurses, the construct validity of this tool has not been established.[11] Establishing construct validity is important to ensuring that the tool can measure the construct in question, namely whether it detects those who are actually competent to perform handoffs safely and effectively. We present here the results of the development of a shorter Handoff Mini‐CEX, along with the formal establishment of its construct validity, namely its ability to distinguish between levels of performance in 3 domains of handoff quality.

METHODS

RESULTS

Forty‐seven faculty members (14=site 1; 33=site 2) participated in the validation workshops (2 at the University of Chicago, and 2 at Yale University), which were held in August 2011 and September 2011, providing a total of 172 observations of a possible 191 (90%).

The overall handoff quality ratings for the superior, gold standard video (superior communication, professionalism, and communication) ranged from 7 to 9 with a mean of 8.5 (standard deviation [SD] 0.7). The overall ratings for the video depicting satisfactory communication (satisfactory communication, superior professionalism and setting) ranged from 5 to 9 with a mean of 7.3 (SD 1.1). The overall ratings for the unsatisfactory communication (unsatisfactory communication, superior professionalism and setting) video ranged from 1 to 7 with a mean of 2.6 (SD 1.2). The overall ratings for the satisfactory professionalism video (satisfactory professionalism, superior communication and setting) ranged from 4 to 8 with a mean of 5.7 (SD 1.3). The overall ratings for the unsatisfactory professionalism (unsatisfactory professionalism, superior communication and setting) video ranged from 2 to 5 with a mean of 2.4 (SD 1.03). Finally, the overall ratings for the unsatisfactory setting (unsatisfactory setting, superior communication and professionalism) video ranged from 1 to 8 with a mean of 3.1 (SD 1.7).

Figure 1 demonstrates that for the domain of communication, the raters were able to discern the unsatisfactory performance but had difficulty reliably distinguishing between superior and satisfactory performance. Figure 2 illustrates that for the domain of professionalism, raters were able to detect the videos' changing levels of performance at the extremes of behavior, with unsatisfactory and superior displays more readily identified. Figure 3 shows that for the domain of setting, the raters were able to discern the unsatisfactory versus superior level of the changing setting. Of note, we also found a moderate significant correlation between ratings of professionalism and communication (r=0.47, P<0.001).

Figure 1

Figure 2

Figure 3

The Cronbach , or measurement of internal reliability and consistency, for the Handoff Mini‐CEX (3 items plus overall) was 0.77, indicating high internal reliability and consistency. Using data from University of Chicago, where raters were labeled with a unique identifier, the Kendall coefficient of concordance was calculated to be 0.79, demonstrating high inter‐rater reliability of the faculty raters. High inter‐rater reliability was also seen using intraclass coefficients for each domain: communication (0.84), professionalism (0.68), setting (0.83), and overall (0.89). Using generalizability analysis, the average reliability was determined to be above 0.9 for all domains (0.99 for overall).

Last, the 2‐way ANOVA (n=75 observations from 13 raters) revealed no evidence of rater bias when examining the coefficient for attending rater (P=0.55 for professionalism, P=0.45 for communication, P=0.92 for setting). The range of scores for each video, however, was broad (Table 2).

DISCUSSION

This study demonstrates that valid conclusions on handoff performance can be drawn using the Handoff CEX as the instrument to rate handoff quality. Utilizing standardized videos depicting varying levels of performance communication, professionalism, and setting, the Handoff Mini‐CEX has demonstrated potential to discern between increasing levels of performance, providing evidence for the construct validity of the instrument.

We observed that faculty could reliably detect unsatisfactory professionalism with ease, and that there was a distinct correlation between faculty ratings and the internally set levels of performance displayed in the videos. This trend demonstrated that faculty were able to discern different levels of professionalism using the Handoff Mini‐CEX. It became more difficult, however, for faculty to detect superior professionalism when the domain of communication was permuted. If the sender of the handoff was professional but the information delivered was disorganized, inaccurate, and missing crucial pieces of information, the faculty perceived this ineffective communication as unprofessional. Prior literature on professionalism has found that communication is a necessary component of professional behavior, and consequently, being a competent communicator is necessary to fulfill ones duty as a professional physician.[15, 16]

This is of note because we did find a moderate significant correlation between ratings of professionalism and communication. It is possible that this distinction would be made clearer with formal rater training in the future prior to any evaluations. However, it is also possible that professionalism and communication, due to a synergistic role between the 2 domains, cannot be separated. If this is the case, it would be important to educate clinicians to present patients in a concise, clear, and accurate way with a professional demeanor. Acknowledging professional responsibility as an integral piece of patient care is also critical in effectively communicating patient information.[5]

We also noted that faculty could detect unsatisfactory communication consistently; however, they were unable to differentiate between satisfactory and superior communication reliably or consistently. Because the unsatisfactory professionalism, unsatisfactory setting, and satisfactory professionalism videos all demonstrated superior communication, we believe that the faculty penalized communication when distractions, in the form of interruptions and rude behavior by the resident giving the handoff, interrupted the flow of the handoff. Thus, the wide ranges in scores observed by some raters may be attributed to this interaction between the Handoff Mini‐CEX domains. In the future, definitions of the anchors, including at the middle spectrum of performance, and rater training may improve the ability of raters to distinguish performance between each domain.

The overall value of the Handoff Mini‐CEX is in its ease of use, in part due to its brevity, as well as evidence for its validity in distinguishing between varying levels of performance. Given the emphasis on monitoring handoff quality and performance, the Handoff Mini‐CEX provides a standard foundation from which baseline handoff performance can be easily measured and improved. Moreover, it can also be used to give individual feedback to a specific practicing clinician on their practices and an opportunity to improve. This is particularly important given current recommendations by the Joint Commission, that handoffs are standardized, and by the ACGME, that residents are competent in handoff skills. Moreover, given the creation of the SHM's handoff recommendations and handoffs as a core competency for hospitalists, the tool provides the ability for hospitalist programs to actually assess their handoff practices as baseline measurements for any quality improvement activities that may take place.

Faculty were able to discern the superior and unsatisfactory levels of setting with ease. After watching and rating the videos, participants said that the chaotic scene of the unsatisfactory setting video had significant authenticity, and that they were constantly interrupted during their own handoffs by pages, phone calls, and people entering the handoff space. System‐level fixes, such as protected time and dedicated space for handoffs, and discouraging pages to be sent during the designated handoff time, could mitigate the reality of unsatisfactory settings.[17, 18]

Our study has several limitations. First, although this study was held at 2 sites, it included a small number of faculty, which can impact the generalizability of our findings. Implementation varied at Yale University and the University of Chicago, preventing use of all data for all analyses. Furthermore, institutional culture may also impact faculty raters' perceptions, so future work aims at repeating our protocol at partner institutions, increasing both the number and diversity of participants. We were also unable to compare the new shorter Handoff Mini‐CEX to the larger 9‐item Handoff CEX in this study.

Despite these limitations, we believe that the Handoff Mini‐CEX, has future potential as an instrument with which to make valid and reliable conclusions about handoff quality, and could be used to both evaluate handoff quality and as an educational tool for trainees and faculty on effective handoff communication.

Only 20 years ago, science from peer‐reviewed journals was still distributed and consumed in the same fashion that evolved from the earliest days of medical science: in print at monthly or weekly intervals. The Internet radically accelerated this paradigm but left the essential processes intact; journals could publish the information and readers could read it more easily, but the basic forums for interaction and discussion over the content remained the same. Enter Web 2.0 and the era of social media. Authors, editors, and readers can now interact easily with each other over the content in real time and across great distances.

Social media may not have changed the way science is produced and reviewed, but it is certainly changing how people consume and use the science. Some have suggested that social media activity around particular articles or journals may be a more important measure of impact than traditional measures of citation,[1] and others have suggested that Twitter activity in particular has changed both the speed and quality of discussion about new studies within the scientific community.[2] In the face of these trends, the Journal of Hospital Medicine (JHM) has decided to develop a bold strategy for leadership in this emerging area, with an initial focus on increasing JHM's activity and visibility on Twitter.

As part of this initial focus, JHM has successfully developed and implemented a protocol for use by authors to compose 2 Tweets describing their publications: the first announces the article's publication (e.g., New evidence on white coats and risk for hospital‐acquired infections), and the second promotes a key point from the article (e.g., Does the doctor's white coat spread hospital infection?). These Tweets are encouraged (but not required) from the corresponding author for every article in every edition, and JHM's editorial staff works with individual authors to refine their message and maximize their impact. To help authors, we have developed several tips for effective tweeting (Table 1).

Even after just 1 year of this Twitter‐focused strategy, we are already seeing noteworthy impact and have learned several lessons.

AUTHORS CAN AND WILL GENERATE TWEETS FOR THEIR ARTICLES

When we started asking authors to generate tweets for their articles, Twitter was relatively new, and we were unsure if authors would be willing and able to participate. Since we started, we have noticed a steady increase in the number of author‐generated tweets. Today, more than three‐quarters of tweets per issue are author generated. Anecdotal feedback has been very positive, and authors have expressed interest in the plan for tweeting as well as feedback on how well their tweets were written. If authors or institutions are on Twitter, we also encourage using the Twitter name or handle in the tweet, which serves as a way for others on Twitter to identify directly with the author or institution and often results in greater interest in a particular tweet. Of note, authors have no obligation to become regular users of Twitter or engage with followers of JHM's Twitter feed, but many find themselves following the journal's feed more closely (and responding to posts by other authors) once they have joined Twitter and tweeted about their own work via JHM.

#HASHTAGS MAKE IT HAPPEN

Because Twitter users are a very large crowd of people with diverse interests, it is important to target tweets to the groups that would be most interested in studies. The use of hashtags makes it easy to index tweets. One of the major edits of author‐generated tweets that we provide is to index the articles to the most popular hashtags. For example, medical education studies can be indexed under #meded, which is a popular hashtag for clinician educators. Other important hashtags for hospitalists include #ptsafety, #readmissions, #healthpolicy, #healthcosts, or #infectiousdisease. To select hashtags, we have found the healthcare hashtag directory maintained by Symplur (Upland, CA; http://www.symplur.com/healthcare‐hashtags) to be a helpful resource for figuring out what the most popular ways to index tweets are and also for identifying hashtags for areas that are less well known to hospitalists, such as #histmedicine, which is for the history of medicine.

HIGH IMPACT STUDIES MAKE A BIGGER IMPACT ON TWITTER

We observed a high number of retweets and comments about articles that were the most viewed studies on JHM online, referring to Project BOOST (Better Outcomes for Older Adults Through Safe Transitions) and the Society of Hospital Medicine's Choosing Wisely campaign. This is not surprising given the national focus on readmissions as well as cost‐conscious care. Moreover, our experience is in line with observations that Twitter provides an additional source of page views and article downloads for medical journals[3] and research, which demonstrates that studies that are tweeted will eventually be cited more.[4, 5]

TECHNOLOGY STUDIES ARE ADORED BY TWITTER

Studies and research examining the use of smartphones, apps, or social media in healthcare draw a lot of attention on Twitter, particularly from other technophiles in healthcare who often use the #hscm healthcare social media hashtag. Such studies often resonate with Twitter users, who tend to be engaged in technology at a high level and are interested in how to advance the use of technology in the healthcare workplace.

JHM's social media strategy has already been very successful in its early implementation; the JHM twitter feed has >600 followers. Although most authors submit their own tweets (71/117 or 61% of articles over the last year), JHM has also created social media roles for editors to fill in tweets when missing and ensure timely and consistent output from the JHM feed. We have also started a Facebook page, with a rapidly growing number of followers, and we continue to see our social media influence scores rise. In the next year we hope to develop a JHM blog, with invited commentary as well as a process for unsolicited submissions from our readership.

Increasingly, a journal's impact (small i) is measured not only in the traditional metric of impact factor (a representation of the number of papers cited in a given journal publication year), but also by the journal's ability to disseminate knowledge and awareness of issues key to the field. Social media is a major element of the next phase of evidence dissemination, and JHM is pleased to be developing and growing its footprint in the digital world.

Only 20 years ago, science from peer‐reviewed journals was still distributed and consumed in the same fashion that evolved from the earliest days of medical science: in print at monthly or weekly intervals. The Internet radically accelerated this paradigm but left the essential processes intact; journals could publish the information and readers could read it more easily, but the basic forums for interaction and discussion over the content remained the same. Enter Web 2.0 and the era of social media. Authors, editors, and readers can now interact easily with each other over the content in real time and across great distances.

Social media may not have changed the way science is produced and reviewed, but it is certainly changing how people consume and use the science. Some have suggested that social media activity around particular articles or journals may be a more important measure of impact than traditional measures of citation,[1] and others have suggested that Twitter activity in particular has changed both the speed and quality of discussion about new studies within the scientific community.[2] In the face of these trends, the Journal of Hospital Medicine (JHM) has decided to develop a bold strategy for leadership in this emerging area, with an initial focus on increasing JHM's activity and visibility on Twitter.

As part of this initial focus, JHM has successfully developed and implemented a protocol for use by authors to compose 2 Tweets describing their publications: the first announces the article's publication (e.g., New evidence on white coats and risk for hospital‐acquired infections), and the second promotes a key point from the article (e.g., Does the doctor's white coat spread hospital infection?). These Tweets are encouraged (but not required) from the corresponding author for every article in every edition, and JHM's editorial staff works with individual authors to refine their message and maximize their impact. To help authors, we have developed several tips for effective tweeting (Table 1).

Even after just 1 year of this Twitter‐focused strategy, we are already seeing noteworthy impact and have learned several lessons.

AUTHORS CAN AND WILL GENERATE TWEETS FOR THEIR ARTICLES

When we started asking authors to generate tweets for their articles, Twitter was relatively new, and we were unsure if authors would be willing and able to participate. Since we started, we have noticed a steady increase in the number of author‐generated tweets. Today, more than three‐quarters of tweets per issue are author generated. Anecdotal feedback has been very positive, and authors have expressed interest in the plan for tweeting as well as feedback on how well their tweets were written. If authors or institutions are on Twitter, we also encourage using the Twitter name or handle in the tweet, which serves as a way for others on Twitter to identify directly with the author or institution and often results in greater interest in a particular tweet. Of note, authors have no obligation to become regular users of Twitter or engage with followers of JHM's Twitter feed, but many find themselves following the journal's feed more closely (and responding to posts by other authors) once they have joined Twitter and tweeted about their own work via JHM.

#HASHTAGS MAKE IT HAPPEN

Because Twitter users are a very large crowd of people with diverse interests, it is important to target tweets to the groups that would be most interested in studies. The use of hashtags makes it easy to index tweets. One of the major edits of author‐generated tweets that we provide is to index the articles to the most popular hashtags. For example, medical education studies can be indexed under #meded, which is a popular hashtag for clinician educators. Other important hashtags for hospitalists include #ptsafety, #readmissions, #healthpolicy, #healthcosts, or #infectiousdisease. To select hashtags, we have found the healthcare hashtag directory maintained by Symplur (Upland, CA; http://www.symplur.com/healthcare‐hashtags) to be a helpful resource for figuring out what the most popular ways to index tweets are and also for identifying hashtags for areas that are less well known to hospitalists, such as #histmedicine, which is for the history of medicine.

HIGH IMPACT STUDIES MAKE A BIGGER IMPACT ON TWITTER

We observed a high number of retweets and comments about articles that were the most viewed studies on JHM online, referring to Project BOOST (Better Outcomes for Older Adults Through Safe Transitions) and the Society of Hospital Medicine's Choosing Wisely campaign. This is not surprising given the national focus on readmissions as well as cost‐conscious care. Moreover, our experience is in line with observations that Twitter provides an additional source of page views and article downloads for medical journals[3] and research, which demonstrates that studies that are tweeted will eventually be cited more.[4, 5]

TECHNOLOGY STUDIES ARE ADORED BY TWITTER

Studies and research examining the use of smartphones, apps, or social media in healthcare draw a lot of attention on Twitter, particularly from other technophiles in healthcare who often use the #hscm healthcare social media hashtag. Such studies often resonate with Twitter users, who tend to be engaged in technology at a high level and are interested in how to advance the use of technology in the healthcare workplace.

JHM's social media strategy has already been very successful in its early implementation; the JHM twitter feed has >600 followers. Although most authors submit their own tweets (71/117 or 61% of articles over the last year), JHM has also created social media roles for editors to fill in tweets when missing and ensure timely and consistent output from the JHM feed. We have also started a Facebook page, with a rapidly growing number of followers, and we continue to see our social media influence scores rise. In the next year we hope to develop a JHM blog, with invited commentary as well as a process for unsolicited submissions from our readership.

Increasingly, a journal's impact (small i) is measured not only in the traditional metric of impact factor (a representation of the number of papers cited in a given journal publication year), but also by the journal's ability to disseminate knowledge and awareness of issues key to the field. Social media is a major element of the next phase of evidence dissemination, and JHM is pleased to be developing and growing its footprint in the digital world.

Only 20 years ago, science from peer‐reviewed journals was still distributed and consumed in the same fashion that evolved from the earliest days of medical science: in print at monthly or weekly intervals. The Internet radically accelerated this paradigm but left the essential processes intact; journals could publish the information and readers could read it more easily, but the basic forums for interaction and discussion over the content remained the same. Enter Web 2.0 and the era of social media. Authors, editors, and readers can now interact easily with each other over the content in real time and across great distances.

Social media may not have changed the way science is produced and reviewed, but it is certainly changing how people consume and use the science. Some have suggested that social media activity around particular articles or journals may be a more important measure of impact than traditional measures of citation,[1] and others have suggested that Twitter activity in particular has changed both the speed and quality of discussion about new studies within the scientific community.[2] In the face of these trends, the Journal of Hospital Medicine (JHM) has decided to develop a bold strategy for leadership in this emerging area, with an initial focus on increasing JHM's activity and visibility on Twitter.

As part of this initial focus, JHM has successfully developed and implemented a protocol for use by authors to compose 2 Tweets describing their publications: the first announces the article's publication (e.g., New evidence on white coats and risk for hospital‐acquired infections), and the second promotes a key point from the article (e.g., Does the doctor's white coat spread hospital infection?). These Tweets are encouraged (but not required) from the corresponding author for every article in every edition, and JHM's editorial staff works with individual authors to refine their message and maximize their impact. To help authors, we have developed several tips for effective tweeting (Table 1).

Even after just 1 year of this Twitter‐focused strategy, we are already seeing noteworthy impact and have learned several lessons.

AUTHORS CAN AND WILL GENERATE TWEETS FOR THEIR ARTICLES

When we started asking authors to generate tweets for their articles, Twitter was relatively new, and we were unsure if authors would be willing and able to participate. Since we started, we have noticed a steady increase in the number of author‐generated tweets. Today, more than three‐quarters of tweets per issue are author generated. Anecdotal feedback has been very positive, and authors have expressed interest in the plan for tweeting as well as feedback on how well their tweets were written. If authors or institutions are on Twitter, we also encourage using the Twitter name or handle in the tweet, which serves as a way for others on Twitter to identify directly with the author or institution and often results in greater interest in a particular tweet. Of note, authors have no obligation to become regular users of Twitter or engage with followers of JHM's Twitter feed, but many find themselves following the journal's feed more closely (and responding to posts by other authors) once they have joined Twitter and tweeted about their own work via JHM.

#HASHTAGS MAKE IT HAPPEN

Because Twitter users are a very large crowd of people with diverse interests, it is important to target tweets to the groups that would be most interested in studies. The use of hashtags makes it easy to index tweets. One of the major edits of author‐generated tweets that we provide is to index the articles to the most popular hashtags. For example, medical education studies can be indexed under #meded, which is a popular hashtag for clinician educators. Other important hashtags for hospitalists include #ptsafety, #readmissions, #healthpolicy, #healthcosts, or #infectiousdisease. To select hashtags, we have found the healthcare hashtag directory maintained by Symplur (Upland, CA; http://www.symplur.com/healthcare‐hashtags) to be a helpful resource for figuring out what the most popular ways to index tweets are and also for identifying hashtags for areas that are less well known to hospitalists, such as #histmedicine, which is for the history of medicine.

HIGH IMPACT STUDIES MAKE A BIGGER IMPACT ON TWITTER

We observed a high number of retweets and comments about articles that were the most viewed studies on JHM online, referring to Project BOOST (Better Outcomes for Older Adults Through Safe Transitions) and the Society of Hospital Medicine's Choosing Wisely campaign. This is not surprising given the national focus on readmissions as well as cost‐conscious care. Moreover, our experience is in line with observations that Twitter provides an additional source of page views and article downloads for medical journals[3] and research, which demonstrates that studies that are tweeted will eventually be cited more.[4, 5]

TECHNOLOGY STUDIES ARE ADORED BY TWITTER

Studies and research examining the use of smartphones, apps, or social media in healthcare draw a lot of attention on Twitter, particularly from other technophiles in healthcare who often use the #hscm healthcare social media hashtag. Such studies often resonate with Twitter users, who tend to be engaged in technology at a high level and are interested in how to advance the use of technology in the healthcare workplace.

JHM's social media strategy has already been very successful in its early implementation; the JHM twitter feed has >600 followers. Although most authors submit their own tweets (71/117 or 61% of articles over the last year), JHM has also created social media roles for editors to fill in tweets when missing and ensure timely and consistent output from the JHM feed. We have also started a Facebook page, with a rapidly growing number of followers, and we continue to see our social media influence scores rise. In the next year we hope to develop a JHM blog, with invited commentary as well as a process for unsolicited submissions from our readership.

Increasingly, a journal's impact (small i) is measured not only in the traditional metric of impact factor (a representation of the number of papers cited in a given journal publication year), but also by the journal's ability to disseminate knowledge and awareness of issues key to the field. Social media is a major element of the next phase of evidence dissemination, and JHM is pleased to be developing and growing its footprint in the digital world.

At the core of a good physician is mastery of critical communication skills. Good communication establishes rapport and can also heal patients. As communication is an essential ingredient of good physicianship, the recipe starts with a fundamental staplethe physician introduction. The physician introduction is step 2 of Kahn's etiquette‐based medicine checklist to promote good doctoring.[1] Although such rudimentary communication skills are cemented in kindergarten, sadly, more training is needed for doctors. In a recent Journal of Hospital Medicine study, interns failed to introduce themselves in 3 out of 5 inpatient encounters.[2]

Despite waning introductions, increasing importance is being placed on hospitalized patient's knowledge of their treating physician's name and role for patient safety. The Transitions of Care Consensus Policy Statement endorsed by 6 medical societies, including the Society of Hospital Medicine, recommend patients know who their treating physician is while caring for them at every step across the continuum, including hospitalization.[3] The Accreditation Council for Graduate Medical Education requires that patients be informed of who the supervising physician is and understand the roles of any trainees in their care.[4] Last, the death of young Lewis Blackman in South Carolina resulted in state legislation requiring clear identification of physicians and their roles for patients.[5] Given these recommendations, tools to remind physicians to introduce themselves and explain their role to patients are worth consideration. In this issue of the Journal of Hospital Medicine, the effectiveness of 2 interventions using physician photo tools is described.[6, 7]

Even though both studies advance our knowledge on the effectiveness of such interventions, nonrandom variable uptake by physicians represents a major common hurdle. Physician workload, competing priorities, and time pressures prevent physicians from distributing such tools. Consistent adopters of the cards likely already introduce themselves regularly. Interestingly, physicians likely withhold the cards from patients they perceive as unsatisfied, who ironically have the most to gain. System changes, such as increasing handoffs and transient coverage with resident duty hours, can also hamper tool effectiveness through the introduction of more physicians to remember, inherently decreasing the ability of patients to identify their treating physicians.[8]

Patient factors also affect the success of such interventions. Interestingly, patients' baseline ability to identify their physician ranged from 11% to 51% in these studies. Such differences can be readily attributed to previous disparities noted by age, race, gender, and education level in patient recall of their physician.[8] Future work should target interventions for these subgroups, while also accounting for the high prevalence of low health literacy, memory impairment, sleep loss, and poor vision among inpatients, all of which can hamper such interventions.[9, 10]

Although neither intervention improved overall patient satisfaction, patient satisfaction is influenced by a variety of factors unrelated to physician care, such as nursing or the environment. Given the inherent ceiling effect in patient satisfaction metrics, both studies were underpowered to show minor differences. It is also worth noting that complex social interventions depend on their context. Although some patients may enjoy receiving the cards, others may feel that it is not critical to their patient satisfaction. Using a realist evaluation would ask patients what they thought of the cards and why.[11] Like one of the authors, we noted that patients do like the cards, suggesting the problem is not the cards but the metrics of evaluation.[12]

In addition to robust evaluation metrics, future interventions should incorporate patient‐centered approaches to empower patients to ask their doctors about their name and role. With the request coming from patients, doctors are much more likely to comply. Using lessons from marketing and advertising, the hospital is full of artifacts, such as white boards, wristbands, remote controls, and monitors, that can be repurposed to advertise the doctor's name to the patient. Future advances can exploit new mobile technologies and repurpose old ones, such as the hospital television, to remind patients of their care team and other critical information. Regardless of what the future may bring, let's face itintroducing yourself properly to your patients is always good medicine.

At the core of a good physician is mastery of critical communication skills. Good communication establishes rapport and can also heal patients. As communication is an essential ingredient of good physicianship, the recipe starts with a fundamental staplethe physician introduction. The physician introduction is step 2 of Kahn's etiquette‐based medicine checklist to promote good doctoring.[1] Although such rudimentary communication skills are cemented in kindergarten, sadly, more training is needed for doctors. In a recent Journal of Hospital Medicine study, interns failed to introduce themselves in 3 out of 5 inpatient encounters.[2]

Despite waning introductions, increasing importance is being placed on hospitalized patient's knowledge of their treating physician's name and role for patient safety. The Transitions of Care Consensus Policy Statement endorsed by 6 medical societies, including the Society of Hospital Medicine, recommend patients know who their treating physician is while caring for them at every step across the continuum, including hospitalization.[3] The Accreditation Council for Graduate Medical Education requires that patients be informed of who the supervising physician is and understand the roles of any trainees in their care.[4] Last, the death of young Lewis Blackman in South Carolina resulted in state legislation requiring clear identification of physicians and their roles for patients.[5] Given these recommendations, tools to remind physicians to introduce themselves and explain their role to patients are worth consideration. In this issue of the Journal of Hospital Medicine, the effectiveness of 2 interventions using physician photo tools is described.[6, 7]

Even though both studies advance our knowledge on the effectiveness of such interventions, nonrandom variable uptake by physicians represents a major common hurdle. Physician workload, competing priorities, and time pressures prevent physicians from distributing such tools. Consistent adopters of the cards likely already introduce themselves regularly. Interestingly, physicians likely withhold the cards from patients they perceive as unsatisfied, who ironically have the most to gain. System changes, such as increasing handoffs and transient coverage with resident duty hours, can also hamper tool effectiveness through the introduction of more physicians to remember, inherently decreasing the ability of patients to identify their treating physicians.[8]

Patient factors also affect the success of such interventions. Interestingly, patients' baseline ability to identify their physician ranged from 11% to 51% in these studies. Such differences can be readily attributed to previous disparities noted by age, race, gender, and education level in patient recall of their physician.[8] Future work should target interventions for these subgroups, while also accounting for the high prevalence of low health literacy, memory impairment, sleep loss, and poor vision among inpatients, all of which can hamper such interventions.[9, 10]

Although neither intervention improved overall patient satisfaction, patient satisfaction is influenced by a variety of factors unrelated to physician care, such as nursing or the environment. Given the inherent ceiling effect in patient satisfaction metrics, both studies were underpowered to show minor differences. It is also worth noting that complex social interventions depend on their context. Although some patients may enjoy receiving the cards, others may feel that it is not critical to their patient satisfaction. Using a realist evaluation would ask patients what they thought of the cards and why.[11] Like one of the authors, we noted that patients do like the cards, suggesting the problem is not the cards but the metrics of evaluation.[12]

In addition to robust evaluation metrics, future interventions should incorporate patient‐centered approaches to empower patients to ask their doctors about their name and role. With the request coming from patients, doctors are much more likely to comply. Using lessons from marketing and advertising, the hospital is full of artifacts, such as white boards, wristbands, remote controls, and monitors, that can be repurposed to advertise the doctor's name to the patient. Future advances can exploit new mobile technologies and repurpose old ones, such as the hospital television, to remind patients of their care team and other critical information. Regardless of what the future may bring, let's face itintroducing yourself properly to your patients is always good medicine.

At the core of a good physician is mastery of critical communication skills. Good communication establishes rapport and can also heal patients. As communication is an essential ingredient of good physicianship, the recipe starts with a fundamental staplethe physician introduction. The physician introduction is step 2 of Kahn's etiquette‐based medicine checklist to promote good doctoring.[1] Although such rudimentary communication skills are cemented in kindergarten, sadly, more training is needed for doctors. In a recent Journal of Hospital Medicine study, interns failed to introduce themselves in 3 out of 5 inpatient encounters.[2]

Despite waning introductions, increasing importance is being placed on hospitalized patient's knowledge of their treating physician's name and role for patient safety. The Transitions of Care Consensus Policy Statement endorsed by 6 medical societies, including the Society of Hospital Medicine, recommend patients know who their treating physician is while caring for them at every step across the continuum, including hospitalization.[3] The Accreditation Council for Graduate Medical Education requires that patients be informed of who the supervising physician is and understand the roles of any trainees in their care.[4] Last, the death of young Lewis Blackman in South Carolina resulted in state legislation requiring clear identification of physicians and their roles for patients.[5] Given these recommendations, tools to remind physicians to introduce themselves and explain their role to patients are worth consideration. In this issue of the Journal of Hospital Medicine, the effectiveness of 2 interventions using physician photo tools is described.[6, 7]

Even though both studies advance our knowledge on the effectiveness of such interventions, nonrandom variable uptake by physicians represents a major common hurdle. Physician workload, competing priorities, and time pressures prevent physicians from distributing such tools. Consistent adopters of the cards likely already introduce themselves regularly. Interestingly, physicians likely withhold the cards from patients they perceive as unsatisfied, who ironically have the most to gain. System changes, such as increasing handoffs and transient coverage with resident duty hours, can also hamper tool effectiveness through the introduction of more physicians to remember, inherently decreasing the ability of patients to identify their treating physicians.[8]

Patient factors also affect the success of such interventions. Interestingly, patients' baseline ability to identify their physician ranged from 11% to 51% in these studies. Such differences can be readily attributed to previous disparities noted by age, race, gender, and education level in patient recall of their physician.[8] Future work should target interventions for these subgroups, while also accounting for the high prevalence of low health literacy, memory impairment, sleep loss, and poor vision among inpatients, all of which can hamper such interventions.[9, 10]

Although neither intervention improved overall patient satisfaction, patient satisfaction is influenced by a variety of factors unrelated to physician care, such as nursing or the environment. Given the inherent ceiling effect in patient satisfaction metrics, both studies were underpowered to show minor differences. It is also worth noting that complex social interventions depend on their context. Although some patients may enjoy receiving the cards, others may feel that it is not critical to their patient satisfaction. Using a realist evaluation would ask patients what they thought of the cards and why.[11] Like one of the authors, we noted that patients do like the cards, suggesting the problem is not the cards but the metrics of evaluation.[12]

In addition to robust evaluation metrics, future interventions should incorporate patient‐centered approaches to empower patients to ask their doctors about their name and role. With the request coming from patients, doctors are much more likely to comply. Using lessons from marketing and advertising, the hospital is full of artifacts, such as white boards, wristbands, remote controls, and monitors, that can be repurposed to advertise the doctor's name to the patient. Future advances can exploit new mobile technologies and repurpose old ones, such as the hospital television, to remind patients of their care team and other critical information. Regardless of what the future may bring, let's face itintroducing yourself properly to your patients is always good medicine.

Unprofessional behavior in the inpatient setting has the potential to impact care delivery and the quality of trainee's educational experience. These behaviors, from disparaging colleagues to blocking admissions, can negatively impact the learning environment. The learning environment or conditions created by the patient care team's actions play a critical role in the development of trainees.[1, 2] The rising presence of hospitalists in the inpatient setting raises the question of how their actions impact the learning environment. Professional behavior has been defined as a core competency for hospitalists by the Society of Hospital Medicine.[3] Professional behavior of all team members, from faculty to trainee, can impact the learning environment and patient safety.[4, 5] However, few educational materials exist to train faculty and housestaff on recognizing and ameliorating unprofessional behaviors.

A prior assessment regarding hospitalists' lapses in professionalism identified scenarios that demonstrated increased participation by hospitalists at 3 institutions.[6] Participants reported observation or participation in specific unprofessional behaviors and rated their perception of these behaviors. Additional work within those residency environments demonstrated that residents' perceptions of and participation in these behaviors increased throughout training, with environmental characteristics, specifically faculty behavior, influencing trainee professional development and acclimation of these behaviors.[7, 8]

Although overall participation in egregious behavior was low, resident participation in 3 categories of unprofessional behavior increased during internship. Those scenarios included disparaging the emergency room or primary care physician for missed findings or management decisions, blocking or not taking admissions appropriate for the service in question, and misrepresenting a test as urgent to expedite obtaining the test. We developed our intervention focused on these areas to address professionalism lapses that occur during internship. Our earlier work showed faculty role models influenced trainee behavior. For this reason, we provided education to both residents and hospitalists to maximize the impact of the intervention.

We present here a novel, interactive, video‐based workshop curriculum for faculty and trainees that aims to illustrate unprofessional behaviors and outlines the role faculty may play in promoting such behaviors. In addition, we review the result of postworkshop evaluation on intent to change behavior and satisfaction.

METHODS

A grant from the American Board of Internal Medicine Foundation supported this project. The working group that resulted, the Chicago Professional Practice Project and Outcomes, included faculty representation from 3 Chicago‐area hospitals: the University of Chicago, Northwestern University, and NorthShore University HealthSystem. Academic hospitalists at these sites were invited to participate. Each site also has an internal medicine residency program in which hospitalists were expected to attend the teaching service. Given this, resident trainees at all participating sites, and 1 community teaching affiliate program (Mercy Hospital and Medical Center) where academic hospitalists at the University of Chicago rotate, were recruited for participation. Faculty champions were identified for each site, and 1 internal and external faculty representative from the working group served to debrief and facilitate. Trainee workshops were administered by 1 internal and external collaborator, and for the community site, 2 external faculty members. Workshops were held during established educational conference times, and lunch was provided.

Scripts highlighting each of the behaviors identified in the prior survey were developed and peer reviewed for clarity and face validity across the 3 sites. Medical student and resident actors were trained utilizing the finalized scripts, and a performance artist affiliated with the Screen Actors Guild assisted in their preparation for filming. All videos were filmed at the University of Chicago Pritzker School of Medicine Clinical Performance Center. The final videos ranged in length from 4 to 7 minutes and included title, cast, and funding source. As an example, 1 video highlighted the unprofessional behavior of misrepresenting a test as urgent to prioritize one's patient in the queue. This video included a resident, intern, and attending on inpatient rounds during which the resident encouraged the intern to misrepresent the patient's status to expedite obtaining the study and facilitate the patient's discharge. The resident stressed that he would be in the clinic and had many patients to see, highlighting the impact of workload on unprofessional behavior, and aggressively persuaded the intern to sell her test to have it performed the same day. When this occurred, the attending applauded the intern for her strong work.

A moderator guide and debriefing tools were developed to facilitate discussion. The duration of each of the workshops was approximately 60 minutes. After welcoming remarks, participants were provided tools to utilize during the viewing of each video. These checklists noted the roles of those depicted in the video, asked to identify positive or negative behaviors displayed, and included questions regarding how behaviors could be detrimental and how the situation could have been prevented. After viewing the videos, participants divided into small groups to discuss the individual exhibiting the unprofessional behavior, their perceived motivation for said behavior, and its impact on the team culture and patient care. Following a small‐group discussion, large‐group debriefing was performed, addressing the barriers and facilitators to professional behavior. Two videos were shown at each workshop, and participants completed a postworkshop evaluation. Videos chosen for viewing were based upon preworkshop survey results that highlighted areas of concern at that specific site.

Postworkshop paper‐based evaluations assessed participants' perception of displayed behaviors on a Likert‐type scale (1=unprofessional to 5=professional) utilizing items validated in prior work,[6, 7, 8] their level of agreement regarding the impact of video‐based exercises, and intent to change behavior using a Likert‐type scale (1=strongly disagree to 5=strongly agree). A constructed‐response section for comments regarding their experience was included. Descriptive statistics and Wilcoxon rank sum analyses were performed.

RESULTS

Forty‐four academic hospitalist faculty members (44/83; 53%) and 244 resident trainees (244/356; 68%) participated. When queried regarding their perception of the displayed behaviors in the videos, nearly 100% of faculty and trainees felt disparaging the emergency department or primary care physician for missed findings or clinical decisions was somewhat unprofessional or unprofessional. Ninety percent of hospitalists and 93% of trainees rated celebrating a blocked admission as somewhat unprofessional or unprofessional (Table 1).

The scenarios portrayed were well received, with more than 85% of faculty and trainees agreeing that the behaviors displayed were realistic. Those who perceived videos as very realistic were more likely to report intent to change behavior (93% vs 53%, P=0.01). Nearly two‐thirds of faculty and 67% of housestaff expressed agreement that they intended to change behavior based upon the experience (Table 2).

Qualitative comments in the constructed‐response portion of the evaluation noted the effectiveness of the interactive materials. In addition, the need for focused faculty development was identified by 1 respondent who stated: If unprofessional behavior is the unwritten curriculum, there needs to be an explicit, written curriculum to address it. Finally, the aim of facilitating self‐reflection is echoed in this faculty respondent's comment: Always good to be reminded of our behaviors and the influence they have on others and from this resident physician It helps to re‐evaluate how you talk to people.

CONCLUSIONS

Faculty can be a large determinant of the learning environment and impact trainees' professional development.[9] Hospitalists should be encouraged to embrace faculty role‐modeling of effective professional behaviors, especially given their increased presence in the inpatient learning environment. In addition, resident trainees and their behaviors contribute to the learning environment and influence the further professional development of more junior trainees.[10] Targeting professionalism education toward previously identified and prevalent unprofessional behaviors in the inpatient care of patients may serve to affect the most change among providers who practice in this setting. Individualized assessment of the learning environment may aid in identifying common scenarios that may plague a specific learning culture, allowing for relevant and targeted discussion of factors that promote and perpetuate such behaviors.[11]

Interactive, video‐based modules provided an effective way to promote interactive reflection and robust discussion. This model of experiential learning is an effective form of professional development as it engages the learner and stimulates ongoing incorporation of the topics addressed.[12, 13] Creating a shared concrete experience among targeted learners, using the video‐based scenarios, stimulates reflective observation, and ultimately experimentation, or incorporation into practice.[14]

There are several limitations to our evaluation including that we focused solely on academic hospitalist programs, and our sample size for faculty and residents was small. Also, we only addressed a small, though representative, sample of unprofessional behaviors and have not yet linked intervention to actual behavior change. Finally, the script scenarios that we used in this study were not previously published as they were created specifically for this intervention. Validity evidence for these scenarios include that they were based upon the results of earlier work from our institutions and underwent thorough peer review for content and clarity. Further studies will be required to do this. However, we do believe that these are positive findings for utilizing this type of interactive curriculum for professionalism education to promote self‐reflection and behavior change.

Video‐based professionalism education is a feasible, interactive mechanism to encourage self‐reflection and intent to change behavior among faculty and resident physicians. Future study is underway to conduct longitudinal assessments of the learning environments at the participating institutions to assess culture change, perceptions of behaviors, and sustainability of this type of intervention.

Disclosures: The authors acknowledge funding from the American Board of Internal Medicine. The funders had no role in the design of the study; the collection, analysis, and interpretation of the data; or the decision to approve publication of the finished manuscript. Results from this work have been presented at the Midwest Society of General Internal Medicine Regional Meeting, Chicago, Illinois, September 2011; Midwest Society of Hospital Medicine Regional Meeting, Chicago, Illinois, October 2011, and Society of Hospital Medicine Annual Meeting, San Diego, California, April 2012. The authors declare that they do not have any conflicts of interest to disclose.

Unprofessional behavior in the inpatient setting has the potential to impact care delivery and the quality of trainee's educational experience. These behaviors, from disparaging colleagues to blocking admissions, can negatively impact the learning environment. The learning environment or conditions created by the patient care team's actions play a critical role in the development of trainees.[1, 2] The rising presence of hospitalists in the inpatient setting raises the question of how their actions impact the learning environment. Professional behavior has been defined as a core competency for hospitalists by the Society of Hospital Medicine.[3] Professional behavior of all team members, from faculty to trainee, can impact the learning environment and patient safety.[4, 5] However, few educational materials exist to train faculty and housestaff on recognizing and ameliorating unprofessional behaviors.

A prior assessment regarding hospitalists' lapses in professionalism identified scenarios that demonstrated increased participation by hospitalists at 3 institutions.[6] Participants reported observation or participation in specific unprofessional behaviors and rated their perception of these behaviors. Additional work within those residency environments demonstrated that residents' perceptions of and participation in these behaviors increased throughout training, with environmental characteristics, specifically faculty behavior, influencing trainee professional development and acclimation of these behaviors.[7, 8]

Although overall participation in egregious behavior was low, resident participation in 3 categories of unprofessional behavior increased during internship. Those scenarios included disparaging the emergency room or primary care physician for missed findings or management decisions, blocking or not taking admissions appropriate for the service in question, and misrepresenting a test as urgent to expedite obtaining the test. We developed our intervention focused on these areas to address professionalism lapses that occur during internship. Our earlier work showed faculty role models influenced trainee behavior. For this reason, we provided education to both residents and hospitalists to maximize the impact of the intervention.

We present here a novel, interactive, video‐based workshop curriculum for faculty and trainees that aims to illustrate unprofessional behaviors and outlines the role faculty may play in promoting such behaviors. In addition, we review the result of postworkshop evaluation on intent to change behavior and satisfaction.

METHODS

A grant from the American Board of Internal Medicine Foundation supported this project. The working group that resulted, the Chicago Professional Practice Project and Outcomes, included faculty representation from 3 Chicago‐area hospitals: the University of Chicago, Northwestern University, and NorthShore University HealthSystem. Academic hospitalists at these sites were invited to participate. Each site also has an internal medicine residency program in which hospitalists were expected to attend the teaching service. Given this, resident trainees at all participating sites, and 1 community teaching affiliate program (Mercy Hospital and Medical Center) where academic hospitalists at the University of Chicago rotate, were recruited for participation. Faculty champions were identified for each site, and 1 internal and external faculty representative from the working group served to debrief and facilitate. Trainee workshops were administered by 1 internal and external collaborator, and for the community site, 2 external faculty members. Workshops were held during established educational conference times, and lunch was provided.

Scripts highlighting each of the behaviors identified in the prior survey were developed and peer reviewed for clarity and face validity across the 3 sites. Medical student and resident actors were trained utilizing the finalized scripts, and a performance artist affiliated with the Screen Actors Guild assisted in their preparation for filming. All videos were filmed at the University of Chicago Pritzker School of Medicine Clinical Performance Center. The final videos ranged in length from 4 to 7 minutes and included title, cast, and funding source. As an example, 1 video highlighted the unprofessional behavior of misrepresenting a test as urgent to prioritize one's patient in the queue. This video included a resident, intern, and attending on inpatient rounds during which the resident encouraged the intern to misrepresent the patient's status to expedite obtaining the study and facilitate the patient's discharge. The resident stressed that he would be in the clinic and had many patients to see, highlighting the impact of workload on unprofessional behavior, and aggressively persuaded the intern to sell her test to have it performed the same day. When this occurred, the attending applauded the intern for her strong work.

A moderator guide and debriefing tools were developed to facilitate discussion. The duration of each of the workshops was approximately 60 minutes. After welcoming remarks, participants were provided tools to utilize during the viewing of each video. These checklists noted the roles of those depicted in the video, asked to identify positive or negative behaviors displayed, and included questions regarding how behaviors could be detrimental and how the situation could have been prevented. After viewing the videos, participants divided into small groups to discuss the individual exhibiting the unprofessional behavior, their perceived motivation for said behavior, and its impact on the team culture and patient care. Following a small‐group discussion, large‐group debriefing was performed, addressing the barriers and facilitators to professional behavior. Two videos were shown at each workshop, and participants completed a postworkshop evaluation. Videos chosen for viewing were based upon preworkshop survey results that highlighted areas of concern at that specific site.

Postworkshop paper‐based evaluations assessed participants' perception of displayed behaviors on a Likert‐type scale (1=unprofessional to 5=professional) utilizing items validated in prior work,[6, 7, 8] their level of agreement regarding the impact of video‐based exercises, and intent to change behavior using a Likert‐type scale (1=strongly disagree to 5=strongly agree). A constructed‐response section for comments regarding their experience was included. Descriptive statistics and Wilcoxon rank sum analyses were performed.

RESULTS

Forty‐four academic hospitalist faculty members (44/83; 53%) and 244 resident trainees (244/356; 68%) participated. When queried regarding their perception of the displayed behaviors in the videos, nearly 100% of faculty and trainees felt disparaging the emergency department or primary care physician for missed findings or clinical decisions was somewhat unprofessional or unprofessional. Ninety percent of hospitalists and 93% of trainees rated celebrating a blocked admission as somewhat unprofessional or unprofessional (Table 1).

The scenarios portrayed were well received, with more than 85% of faculty and trainees agreeing that the behaviors displayed were realistic. Those who perceived videos as very realistic were more likely to report intent to change behavior (93% vs 53%, P=0.01). Nearly two‐thirds of faculty and 67% of housestaff expressed agreement that they intended to change behavior based upon the experience (Table 2).

Qualitative comments in the constructed‐response portion of the evaluation noted the effectiveness of the interactive materials. In addition, the need for focused faculty development was identified by 1 respondent who stated: If unprofessional behavior is the unwritten curriculum, there needs to be an explicit, written curriculum to address it. Finally, the aim of facilitating self‐reflection is echoed in this faculty respondent's comment: Always good to be reminded of our behaviors and the influence they have on others and from this resident physician It helps to re‐evaluate how you talk to people.

CONCLUSIONS

Faculty can be a large determinant of the learning environment and impact trainees' professional development.[9] Hospitalists should be encouraged to embrace faculty role‐modeling of effective professional behaviors, especially given their increased presence in the inpatient learning environment. In addition, resident trainees and their behaviors contribute to the learning environment and influence the further professional development of more junior trainees.[10] Targeting professionalism education toward previously identified and prevalent unprofessional behaviors in the inpatient care of patients may serve to affect the most change among providers who practice in this setting. Individualized assessment of the learning environment may aid in identifying common scenarios that may plague a specific learning culture, allowing for relevant and targeted discussion of factors that promote and perpetuate such behaviors.[11]

Interactive, video‐based modules provided an effective way to promote interactive reflection and robust discussion. This model of experiential learning is an effective form of professional development as it engages the learner and stimulates ongoing incorporation of the topics addressed.[12, 13] Creating a shared concrete experience among targeted learners, using the video‐based scenarios, stimulates reflective observation, and ultimately experimentation, or incorporation into practice.[14]

There are several limitations to our evaluation including that we focused solely on academic hospitalist programs, and our sample size for faculty and residents was small. Also, we only addressed a small, though representative, sample of unprofessional behaviors and have not yet linked intervention to actual behavior change. Finally, the script scenarios that we used in this study were not previously published as they were created specifically for this intervention. Validity evidence for these scenarios include that they were based upon the results of earlier work from our institutions and underwent thorough peer review for content and clarity. Further studies will be required to do this. However, we do believe that these are positive findings for utilizing this type of interactive curriculum for professionalism education to promote self‐reflection and behavior change.

Video‐based professionalism education is a feasible, interactive mechanism to encourage self‐reflection and intent to change behavior among faculty and resident physicians. Future study is underway to conduct longitudinal assessments of the learning environments at the participating institutions to assess culture change, perceptions of behaviors, and sustainability of this type of intervention.

Disclosures: The authors acknowledge funding from the American Board of Internal Medicine. The funders had no role in the design of the study; the collection, analysis, and interpretation of the data; or the decision to approve publication of the finished manuscript. Results from this work have been presented at the Midwest Society of General Internal Medicine Regional Meeting, Chicago, Illinois, September 2011; Midwest Society of Hospital Medicine Regional Meeting, Chicago, Illinois, October 2011, and Society of Hospital Medicine Annual Meeting, San Diego, California, April 2012. The authors declare that they do not have any conflicts of interest to disclose.

Unprofessional behavior in the inpatient setting has the potential to impact care delivery and the quality of trainee's educational experience. These behaviors, from disparaging colleagues to blocking admissions, can negatively impact the learning environment. The learning environment or conditions created by the patient care team's actions play a critical role in the development of trainees.[1, 2] The rising presence of hospitalists in the inpatient setting raises the question of how their actions impact the learning environment. Professional behavior has been defined as a core competency for hospitalists by the Society of Hospital Medicine.[3] Professional behavior of all team members, from faculty to trainee, can impact the learning environment and patient safety.[4, 5] However, few educational materials exist to train faculty and housestaff on recognizing and ameliorating unprofessional behaviors.

A prior assessment regarding hospitalists' lapses in professionalism identified scenarios that demonstrated increased participation by hospitalists at 3 institutions.[6] Participants reported observation or participation in specific unprofessional behaviors and rated their perception of these behaviors. Additional work within those residency environments demonstrated that residents' perceptions of and participation in these behaviors increased throughout training, with environmental characteristics, specifically faculty behavior, influencing trainee professional development and acclimation of these behaviors.[7, 8]

Although overall participation in egregious behavior was low, resident participation in 3 categories of unprofessional behavior increased during internship. Those scenarios included disparaging the emergency room or primary care physician for missed findings or management decisions, blocking or not taking admissions appropriate for the service in question, and misrepresenting a test as urgent to expedite obtaining the test. We developed our intervention focused on these areas to address professionalism lapses that occur during internship. Our earlier work showed faculty role models influenced trainee behavior. For this reason, we provided education to both residents and hospitalists to maximize the impact of the intervention.

We present here a novel, interactive, video‐based workshop curriculum for faculty and trainees that aims to illustrate unprofessional behaviors and outlines the role faculty may play in promoting such behaviors. In addition, we review the result of postworkshop evaluation on intent to change behavior and satisfaction.

METHODS

A grant from the American Board of Internal Medicine Foundation supported this project. The working group that resulted, the Chicago Professional Practice Project and Outcomes, included faculty representation from 3 Chicago‐area hospitals: the University of Chicago, Northwestern University, and NorthShore University HealthSystem. Academic hospitalists at these sites were invited to participate. Each site also has an internal medicine residency program in which hospitalists were expected to attend the teaching service. Given this, resident trainees at all participating sites, and 1 community teaching affiliate program (Mercy Hospital and Medical Center) where academic hospitalists at the University of Chicago rotate, were recruited for participation. Faculty champions were identified for each site, and 1 internal and external faculty representative from the working group served to debrief and facilitate. Trainee workshops were administered by 1 internal and external collaborator, and for the community site, 2 external faculty members. Workshops were held during established educational conference times, and lunch was provided.

Scripts highlighting each of the behaviors identified in the prior survey were developed and peer reviewed for clarity and face validity across the 3 sites. Medical student and resident actors were trained utilizing the finalized scripts, and a performance artist affiliated with the Screen Actors Guild assisted in their preparation for filming. All videos were filmed at the University of Chicago Pritzker School of Medicine Clinical Performance Center. The final videos ranged in length from 4 to 7 minutes and included title, cast, and funding source. As an example, 1 video highlighted the unprofessional behavior of misrepresenting a test as urgent to prioritize one's patient in the queue. This video included a resident, intern, and attending on inpatient rounds during which the resident encouraged the intern to misrepresent the patient's status to expedite obtaining the study and facilitate the patient's discharge. The resident stressed that he would be in the clinic and had many patients to see, highlighting the impact of workload on unprofessional behavior, and aggressively persuaded the intern to sell her test to have it performed the same day. When this occurred, the attending applauded the intern for her strong work.

A moderator guide and debriefing tools were developed to facilitate discussion. The duration of each of the workshops was approximately 60 minutes. After welcoming remarks, participants were provided tools to utilize during the viewing of each video. These checklists noted the roles of those depicted in the video, asked to identify positive or negative behaviors displayed, and included questions regarding how behaviors could be detrimental and how the situation could have been prevented. After viewing the videos, participants divided into small groups to discuss the individual exhibiting the unprofessional behavior, their perceived motivation for said behavior, and its impact on the team culture and patient care. Following a small‐group discussion, large‐group debriefing was performed, addressing the barriers and facilitators to professional behavior. Two videos were shown at each workshop, and participants completed a postworkshop evaluation. Videos chosen for viewing were based upon preworkshop survey results that highlighted areas of concern at that specific site.

Postworkshop paper‐based evaluations assessed participants' perception of displayed behaviors on a Likert‐type scale (1=unprofessional to 5=professional) utilizing items validated in prior work,[6, 7, 8] their level of agreement regarding the impact of video‐based exercises, and intent to change behavior using a Likert‐type scale (1=strongly disagree to 5=strongly agree). A constructed‐response section for comments regarding their experience was included. Descriptive statistics and Wilcoxon rank sum analyses were performed.

RESULTS

Forty‐four academic hospitalist faculty members (44/83; 53%) and 244 resident trainees (244/356; 68%) participated. When queried regarding their perception of the displayed behaviors in the videos, nearly 100% of faculty and trainees felt disparaging the emergency department or primary care physician for missed findings or clinical decisions was somewhat unprofessional or unprofessional. Ninety percent of hospitalists and 93% of trainees rated celebrating a blocked admission as somewhat unprofessional or unprofessional (Table 1).

The scenarios portrayed were well received, with more than 85% of faculty and trainees agreeing that the behaviors displayed were realistic. Those who perceived videos as very realistic were more likely to report intent to change behavior (93% vs 53%, P=0.01). Nearly two‐thirds of faculty and 67% of housestaff expressed agreement that they intended to change behavior based upon the experience (Table 2).

Qualitative comments in the constructed‐response portion of the evaluation noted the effectiveness of the interactive materials. In addition, the need for focused faculty development was identified by 1 respondent who stated: If unprofessional behavior is the unwritten curriculum, there needs to be an explicit, written curriculum to address it. Finally, the aim of facilitating self‐reflection is echoed in this faculty respondent's comment: Always good to be reminded of our behaviors and the influence they have on others and from this resident physician It helps to re‐evaluate how you talk to people.

CONCLUSIONS

Faculty can be a large determinant of the learning environment and impact trainees' professional development.[9] Hospitalists should be encouraged to embrace faculty role‐modeling of effective professional behaviors, especially given their increased presence in the inpatient learning environment. In addition, resident trainees and their behaviors contribute to the learning environment and influence the further professional development of more junior trainees.[10] Targeting professionalism education toward previously identified and prevalent unprofessional behaviors in the inpatient care of patients may serve to affect the most change among providers who practice in this setting. Individualized assessment of the learning environment may aid in identifying common scenarios that may plague a specific learning culture, allowing for relevant and targeted discussion of factors that promote and perpetuate such behaviors.[11]

Interactive, video‐based modules provided an effective way to promote interactive reflection and robust discussion. This model of experiential learning is an effective form of professional development as it engages the learner and stimulates ongoing incorporation of the topics addressed.[12, 13] Creating a shared concrete experience among targeted learners, using the video‐based scenarios, stimulates reflective observation, and ultimately experimentation, or incorporation into practice.[14]

There are several limitations to our evaluation including that we focused solely on academic hospitalist programs, and our sample size for faculty and residents was small. Also, we only addressed a small, though representative, sample of unprofessional behaviors and have not yet linked intervention to actual behavior change. Finally, the script scenarios that we used in this study were not previously published as they were created specifically for this intervention. Validity evidence for these scenarios include that they were based upon the results of earlier work from our institutions and underwent thorough peer review for content and clarity. Further studies will be required to do this. However, we do believe that these are positive findings for utilizing this type of interactive curriculum for professionalism education to promote self‐reflection and behavior change.

Video‐based professionalism education is a feasible, interactive mechanism to encourage self‐reflection and intent to change behavior among faculty and resident physicians. Future study is underway to conduct longitudinal assessments of the learning environments at the participating institutions to assess culture change, perceptions of behaviors, and sustainability of this type of intervention.

Disclosures: The authors acknowledge funding from the American Board of Internal Medicine. The funders had no role in the design of the study; the collection, analysis, and interpretation of the data; or the decision to approve publication of the finished manuscript. Results from this work have been presented at the Midwest Society of General Internal Medicine Regional Meeting, Chicago, Illinois, September 2011; Midwest Society of Hospital Medicine Regional Meeting, Chicago, Illinois, October 2011, and Society of Hospital Medicine Annual Meeting, San Diego, California, April 2012. The authors declare that they do not have any conflicts of interest to disclose.

The implementation of resident duty hour restrictions has created a clinical learning environment on the wards quite different from any previous era. The Accreditation Council for Graduate Medical Education issued its first set of regulations limiting consecutive hours worked for residents in 2003, and further restricted hours in 2011.[1] These restrictions have had many implications across several aspects of patient care, education, and clinical training, particularly for hospitalists who spend the majority of their time in this setting and are heavily involved in undergraduate and graduate clinical education in academic medical centers.[2, 3]

As learning environments have been shifting, so has the composition of learners. The Millennial Generation (or Generation Y), defined as those born approximately between 1980 and 2000, represents those young clinicians currently filling the halls of medical schools and ranks of residency and fellowship programs.[4] Interestingly, the current system of restricted work hours is the only system under which the Millennial Generation has ever trained.

As this new generation represents the bulk of current trainees, hospitalist faculty must consider how their teaching styles can be adapted to accommodate these learners. For teaching hospitalists, an approach that considers the learning environment as affected by duty hours, as well as the preferences of Millennial learners, is necessary to educate the next generation of trainees. This article aimed to introduce potential strategies for hospitalists to better align teaching on the wards with the preferences of Millennial learners under the constraints of residency duty hours.

THE NEWEST GENERATION OF LEARNERS

The Millennial Generation has been well described.[4, 5, 6, 7, 8, 9, 10] Broadly speaking, this generation is thought to have been raised by attentive and involved parents, influencing relationships with educators and mentors; they respect authority but do not hesitate to question the relevance of assignments or decisions. Millennials prefer structured learning environments that focus heavily on interaction and experiential learning, and they value design and appearance in how material is presented.[7] Millennials also seek clear expectations and immediate feedback on their performance, and though they have sometimes been criticized for a strong sense of entitlement, they have a strong desire for collaboration and group‐based activity.[5, 6]

One of the most notable and defining characteristics of the Millennial Generation is an affinity for technology and innovation.[7, 8, 9] Web‐based learning tools that are interactive and engaging, such as blogs, podcasts, or streaming videos are familiar and favored methods of learning. Millennials are skilled at finding information and providing answers and data, but may need help with synthesis and application.[5] They take pride in their ability to multitask, but can be prone to doing so inappropriately, particularly with technology that is readily available.[11]

Few studies have explored characteristics of the Millennial Generation specific to medical trainees. One study examined personality characteristics of Millennial medical students compared to Generation X students (those born from 19651980) at a single institution. Millennial students scored higher on warmth, reasoning, emotional stability, rule consciousness, social boldness, sensitivity, apprehension, openness to change, and perfectionism compared to Generation X students. They scored lower on measures for self‐reliance.[12] Additionally, when motives for behavior were studied, Millennial medical students scored higher on needs for affiliation and achievement, and lower on needs for power.[13]

DUTY HOURS: A GENERATION APART

As noted previously, the Millennial Generation is the first to train exclusively in the era of duty hours restrictions. The oldest members of this generation, those born in 1981, were entering medical school at the time of the first duty hours restrictions in 2003, and thus have always been educated, trained, and practiced in an environment in which work hours were an essential part of residency training.

Though duty hours have been an omnipresent part of training for the Millennial Generation, the clinical learning environment that they have known continues to evolve and change. Time for teaching, in particular, has been especially strained by work hour limits, and this has been noted by both attending physicians and trainees with each iteration of work hours limits. Attendings in one study estimated that time spent teaching on general medicine wards was reduced by about 20% following the 2003 limits, and over 40% of residents in a national survey reported that the 2011 limits had worsened the quality of education.[14, 15]

GENERATIONAL STRATEGIES FOR SUCCESS FOR HOSPITALIST TEACHING ATTENDINGS

The time limitations imposed by duty hours restrictions have compelled teaching rounds to become more patient‐care centered and often less learner‐centered, as providing patient care becomes the prime obligation for this limited time period. Millennial learners are accustomed to being the center of attention in educational environments, and changing the focus from education to patient care in the wards setting may be an abrupt transition for some learners.[6] However, hospitalists can help restructure teaching opportunities on the clinical wards by using teaching methods of the highest value to Millennial learners to promote learning under the conditions of duty hours limitations.

An approach using these methods was developed by reviewing recent literature as well as educational innovations that have been presented at scholarly meetings (eg, Sal Khan's presentation at the 2012 Association of American Medical Colleges meeting).[16] The authors discussed potential teaching techniques that were thought to be feasible to implement in the context of the current learning environment, with consideration of learning theories that would be most effective for the target group of learners (eg, adult learning theory).[17] A mnemonic was created to consolidate strategies thought to best represent these techniques. FUTURE is a group of teaching strategies that can be used by hospitalists to improve teaching rounds by Flipping the Wards, Using Documentation to Teach, Technology‐Enabled Teaching, Using Guerilla Teaching Tactics, Rainy Day Teaching, and Embedding Teaching Moments into Rounds.

ACCEPTING THE CHALLENGE

The landscape of clinical teaching has shifted considerably in recent years, in both the makeup of learners for whom educators are responsible for teaching as well as the challenges in teaching under the duty hours restrictions. Though rounds are more focused on patient care than in the past, it is possible to work within the current structure to promote successful learning with an approach that considers the preferences of today's learners.

A hospitalist's natural habitat, the busy inpatient wards, is a clinical learning environment with rich potential for innovation and excellence in teaching. The challenges in practicing hospital medicine closely parallel the challenges in teaching under the constraints of duty hours restrictions; both require a creative approach to problem solving and an affinity for teamwork. The hospitalist community is well suited to not only meet these challenges but become leaders in embracing how to teach effectively on today's wards. Maximizing interaction, embracing technology, and encouraging group‐based learning may represent the keys to a successful approach to teaching the Millennial Generation in a post‐duty hours world.

The implementation of resident duty hour restrictions has created a clinical learning environment on the wards quite different from any previous era. The Accreditation Council for Graduate Medical Education issued its first set of regulations limiting consecutive hours worked for residents in 2003, and further restricted hours in 2011.[1] These restrictions have had many implications across several aspects of patient care, education, and clinical training, particularly for hospitalists who spend the majority of their time in this setting and are heavily involved in undergraduate and graduate clinical education in academic medical centers.[2, 3]

As learning environments have been shifting, so has the composition of learners. The Millennial Generation (or Generation Y), defined as those born approximately between 1980 and 2000, represents those young clinicians currently filling the halls of medical schools and ranks of residency and fellowship programs.[4] Interestingly, the current system of restricted work hours is the only system under which the Millennial Generation has ever trained.

As this new generation represents the bulk of current trainees, hospitalist faculty must consider how their teaching styles can be adapted to accommodate these learners. For teaching hospitalists, an approach that considers the learning environment as affected by duty hours, as well as the preferences of Millennial learners, is necessary to educate the next generation of trainees. This article aimed to introduce potential strategies for hospitalists to better align teaching on the wards with the preferences of Millennial learners under the constraints of residency duty hours.

THE NEWEST GENERATION OF LEARNERS

The Millennial Generation has been well described.[4, 5, 6, 7, 8, 9, 10] Broadly speaking, this generation is thought to have been raised by attentive and involved parents, influencing relationships with educators and mentors; they respect authority but do not hesitate to question the relevance of assignments or decisions. Millennials prefer structured learning environments that focus heavily on interaction and experiential learning, and they value design and appearance in how material is presented.[7] Millennials also seek clear expectations and immediate feedback on their performance, and though they have sometimes been criticized for a strong sense of entitlement, they have a strong desire for collaboration and group‐based activity.[5, 6]

One of the most notable and defining characteristics of the Millennial Generation is an affinity for technology and innovation.[7, 8, 9] Web‐based learning tools that are interactive and engaging, such as blogs, podcasts, or streaming videos are familiar and favored methods of learning. Millennials are skilled at finding information and providing answers and data, but may need help with synthesis and application.[5] They take pride in their ability to multitask, but can be prone to doing so inappropriately, particularly with technology that is readily available.[11]

Few studies have explored characteristics of the Millennial Generation specific to medical trainees. One study examined personality characteristics of Millennial medical students compared to Generation X students (those born from 19651980) at a single institution. Millennial students scored higher on warmth, reasoning, emotional stability, rule consciousness, social boldness, sensitivity, apprehension, openness to change, and perfectionism compared to Generation X students. They scored lower on measures for self‐reliance.[12] Additionally, when motives for behavior were studied, Millennial medical students scored higher on needs for affiliation and achievement, and lower on needs for power.[13]

DUTY HOURS: A GENERATION APART

As noted previously, the Millennial Generation is the first to train exclusively in the era of duty hours restrictions. The oldest members of this generation, those born in 1981, were entering medical school at the time of the first duty hours restrictions in 2003, and thus have always been educated, trained, and practiced in an environment in which work hours were an essential part of residency training.

Though duty hours have been an omnipresent part of training for the Millennial Generation, the clinical learning environment that they have known continues to evolve and change. Time for teaching, in particular, has been especially strained by work hour limits, and this has been noted by both attending physicians and trainees with each iteration of work hours limits. Attendings in one study estimated that time spent teaching on general medicine wards was reduced by about 20% following the 2003 limits, and over 40% of residents in a national survey reported that the 2011 limits had worsened the quality of education.[14, 15]

GENERATIONAL STRATEGIES FOR SUCCESS FOR HOSPITALIST TEACHING ATTENDINGS

The time limitations imposed by duty hours restrictions have compelled teaching rounds to become more patient‐care centered and often less learner‐centered, as providing patient care becomes the prime obligation for this limited time period. Millennial learners are accustomed to being the center of attention in educational environments, and changing the focus from education to patient care in the wards setting may be an abrupt transition for some learners.[6] However, hospitalists can help restructure teaching opportunities on the clinical wards by using teaching methods of the highest value to Millennial learners to promote learning under the conditions of duty hours limitations.

An approach using these methods was developed by reviewing recent literature as well as educational innovations that have been presented at scholarly meetings (eg, Sal Khan's presentation at the 2012 Association of American Medical Colleges meeting).[16] The authors discussed potential teaching techniques that were thought to be feasible to implement in the context of the current learning environment, with consideration of learning theories that would be most effective for the target group of learners (eg, adult learning theory).[17] A mnemonic was created to consolidate strategies thought to best represent these techniques. FUTURE is a group of teaching strategies that can be used by hospitalists to improve teaching rounds by Flipping the Wards, Using Documentation to Teach, Technology‐Enabled Teaching, Using Guerilla Teaching Tactics, Rainy Day Teaching, and Embedding Teaching Moments into Rounds.

ACCEPTING THE CHALLENGE

The landscape of clinical teaching has shifted considerably in recent years, in both the makeup of learners for whom educators are responsible for teaching as well as the challenges in teaching under the duty hours restrictions. Though rounds are more focused on patient care than in the past, it is possible to work within the current structure to promote successful learning with an approach that considers the preferences of today's learners.

A hospitalist's natural habitat, the busy inpatient wards, is a clinical learning environment with rich potential for innovation and excellence in teaching. The challenges in practicing hospital medicine closely parallel the challenges in teaching under the constraints of duty hours restrictions; both require a creative approach to problem solving and an affinity for teamwork. The hospitalist community is well suited to not only meet these challenges but become leaders in embracing how to teach effectively on today's wards. Maximizing interaction, embracing technology, and encouraging group‐based learning may represent the keys to a successful approach to teaching the Millennial Generation in a post‐duty hours world.

The implementation of resident duty hour restrictions has created a clinical learning environment on the wards quite different from any previous era. The Accreditation Council for Graduate Medical Education issued its first set of regulations limiting consecutive hours worked for residents in 2003, and further restricted hours in 2011.[1] These restrictions have had many implications across several aspects of patient care, education, and clinical training, particularly for hospitalists who spend the majority of their time in this setting and are heavily involved in undergraduate and graduate clinical education in academic medical centers.[2, 3]

As learning environments have been shifting, so has the composition of learners. The Millennial Generation (or Generation Y), defined as those born approximately between 1980 and 2000, represents those young clinicians currently filling the halls of medical schools and ranks of residency and fellowship programs.[4] Interestingly, the current system of restricted work hours is the only system under which the Millennial Generation has ever trained.

As this new generation represents the bulk of current trainees, hospitalist faculty must consider how their teaching styles can be adapted to accommodate these learners. For teaching hospitalists, an approach that considers the learning environment as affected by duty hours, as well as the preferences of Millennial learners, is necessary to educate the next generation of trainees. This article aimed to introduce potential strategies for hospitalists to better align teaching on the wards with the preferences of Millennial learners under the constraints of residency duty hours.

THE NEWEST GENERATION OF LEARNERS

The Millennial Generation has been well described.[4, 5, 6, 7, 8, 9, 10] Broadly speaking, this generation is thought to have been raised by attentive and involved parents, influencing relationships with educators and mentors; they respect authority but do not hesitate to question the relevance of assignments or decisions. Millennials prefer structured learning environments that focus heavily on interaction and experiential learning, and they value design and appearance in how material is presented.[7] Millennials also seek clear expectations and immediate feedback on their performance, and though they have sometimes been criticized for a strong sense of entitlement, they have a strong desire for collaboration and group‐based activity.[5, 6]

One of the most notable and defining characteristics of the Millennial Generation is an affinity for technology and innovation.[7, 8, 9] Web‐based learning tools that are interactive and engaging, such as blogs, podcasts, or streaming videos are familiar and favored methods of learning. Millennials are skilled at finding information and providing answers and data, but may need help with synthesis and application.[5] They take pride in their ability to multitask, but can be prone to doing so inappropriately, particularly with technology that is readily available.[11]

Few studies have explored characteristics of the Millennial Generation specific to medical trainees. One study examined personality characteristics of Millennial medical students compared to Generation X students (those born from 19651980) at a single institution. Millennial students scored higher on warmth, reasoning, emotional stability, rule consciousness, social boldness, sensitivity, apprehension, openness to change, and perfectionism compared to Generation X students. They scored lower on measures for self‐reliance.[12] Additionally, when motives for behavior were studied, Millennial medical students scored higher on needs for affiliation and achievement, and lower on needs for power.[13]

DUTY HOURS: A GENERATION APART

As noted previously, the Millennial Generation is the first to train exclusively in the era of duty hours restrictions. The oldest members of this generation, those born in 1981, were entering medical school at the time of the first duty hours restrictions in 2003, and thus have always been educated, trained, and practiced in an environment in which work hours were an essential part of residency training.

Though duty hours have been an omnipresent part of training for the Millennial Generation, the clinical learning environment that they have known continues to evolve and change. Time for teaching, in particular, has been especially strained by work hour limits, and this has been noted by both attending physicians and trainees with each iteration of work hours limits. Attendings in one study estimated that time spent teaching on general medicine wards was reduced by about 20% following the 2003 limits, and over 40% of residents in a national survey reported that the 2011 limits had worsened the quality of education.[14, 15]

GENERATIONAL STRATEGIES FOR SUCCESS FOR HOSPITALIST TEACHING ATTENDINGS

The time limitations imposed by duty hours restrictions have compelled teaching rounds to become more patient‐care centered and often less learner‐centered, as providing patient care becomes the prime obligation for this limited time period. Millennial learners are accustomed to being the center of attention in educational environments, and changing the focus from education to patient care in the wards setting may be an abrupt transition for some learners.[6] However, hospitalists can help restructure teaching opportunities on the clinical wards by using teaching methods of the highest value to Millennial learners to promote learning under the conditions of duty hours limitations.

An approach using these methods was developed by reviewing recent literature as well as educational innovations that have been presented at scholarly meetings (eg, Sal Khan's presentation at the 2012 Association of American Medical Colleges meeting).[16] The authors discussed potential teaching techniques that were thought to be feasible to implement in the context of the current learning environment, with consideration of learning theories that would be most effective for the target group of learners (eg, adult learning theory).[17] A mnemonic was created to consolidate strategies thought to best represent these techniques. FUTURE is a group of teaching strategies that can be used by hospitalists to improve teaching rounds by Flipping the Wards, Using Documentation to Teach, Technology‐Enabled Teaching, Using Guerilla Teaching Tactics, Rainy Day Teaching, and Embedding Teaching Moments into Rounds.

ACCEPTING THE CHALLENGE

The landscape of clinical teaching has shifted considerably in recent years, in both the makeup of learners for whom educators are responsible for teaching as well as the challenges in teaching under the duty hours restrictions. Though rounds are more focused on patient care than in the past, it is possible to work within the current structure to promote successful learning with an approach that considers the preferences of today's learners.

A hospitalist's natural habitat, the busy inpatient wards, is a clinical learning environment with rich potential for innovation and excellence in teaching. The challenges in practicing hospital medicine closely parallel the challenges in teaching under the constraints of duty hours restrictions; both require a creative approach to problem solving and an affinity for teamwork. The hospitalist community is well suited to not only meet these challenges but become leaders in embracing how to teach effectively on today's wards. Maximizing interaction, embracing technology, and encouraging group‐based learning may represent the keys to a successful approach to teaching the Millennial Generation in a post‐duty hours world.

Transfers among trainee physicians within the hospital typically occur at least twice a day and have been increasing among trainees as work hours have declined.[1] The 2011 Accreditation Council for Graduate Medical Education (ACGME) guidelines,[2] which restrict intern working hours to 16 hours from a previous maximum of 30, have likely increased the frequency of physician trainee handoffs even further. Similarly, transfers among hospitalist attendings occur at least twice a day, given typical shifts of 8 to 12 hours.

Given the frequency of transfers, and the potential for harm generated by failed transitions,[3, 4, 5, 6] the end‐of‐shift written and verbal handoffs have assumed increasingly greater importance in hospital care among both trainees and hospitalist attendings.

The ACGME now requires that programs assess the competency of trainees in handoff communication.[2] Yet, there are few tools for assessing the quality of sign‐out communication. Those that exist primarily focus on the written sign‐out, and are rarely validated.[7, 8, 9, 10, 11, 12] Furthermore, it is uncertain whether such assessments must be done by supervisors or whether peers can participate in the evaluation. In this prospective multi‐institutional study we assess the performance characteristics of a verbal sign‐out evaluation tool for internal medicine housestaff and hospitalist attendings, and examine whether it can be used by peers as well as by external evaluators. This tool has previously been found to effectively discriminate between experienced and inexperienced nurses conducting nursing handoffs.[13]

METHODS

RESULTS

A total of 149 handoff sessions were observed: 89 at UCM and 60 at Yale. Each site conducted a similar total number of evaluations: 336 at UCM, 337 at Yale. These sessions involved 97 unique individuals, 34 at UCM and 63 at Yale. Overall scores were high at both sites, but a wide range of scores was applied (Table 1).

DISCUSSION

In this study we found that an evaluation tool for direct observation of housestaff and hospitalists generated a range of scores and was well validated in the sense of performing similarly across 2 different institutions and among both trainees and attendings, while having high internal consistency. However, external evaluators gave consistently lower marks than peer evaluators at both sites, resulting in low reliability when comparing these 2 groups of raters.

It has traditionally been difficult to conduct direct evaluations of handoffs, because they may occur at haphazard times, in variable locations, and without very much advance notice. For this reason, several attempts have been made to incorporate peers in evaluations of handoff practices.[5, 39, 40] Using peers to conduct evaluations also has the advantage that peers are more likely to be familiar with the patients being handed off and might recognize handoff flaws that external evaluators would miss. Nonetheless, peer evaluations have some important liabilities. Peers may be unwilling or unable to provide honest critiques of their colleagues given that they must work closely together for years. Trainee peers may also lack sufficient clinical expertise or experience to accurately assess competence. In our study, we found that peers gave consistently higher marks to their colleagues than did external evaluators, suggesting they may have found it difficult to criticize their colleagues. We conclude that peer evaluation alone is likely an insufficient means of evaluating handoff quality.

Supervising residents gave very similar marks as intern peers, suggesting that they also are unwilling to criticize, are insufficiently experienced to evaluate, or alternatively, that the peer evaluations were reasonable. We suspect the latter is unlikely given that external evaluator scores were consistently lower than peers. One would expect the external evaluators to be biased toward higher scores given that they are not familiar with the patients and are not able to comment on inaccuracies or omissions in the sign‐out.

The tool appeared to perform less well in most cases for recipients than for providers, with a narrower range of scores and low‐weighted kappa scores. Although recipients play a key role in ensuring a high‐quality sign‐out by paying close attention, ensuring it is a bidirectional conversation, asking appropriate questions, and reading back key information, it may be that evaluators were unable to place these activities within the same domains that were used for the provider evaluation. An altogether different recipient evaluation approach may be necessary.[41]

In general, scores were clustered at the top of the score range, as is typical for evaluations. One strategy to spread out scores further would be to refine the tool by adding anchors for satisfactory performance not just the extremes. A second approach might be to reduce the grading scale to only 3 points (unsatisfactory, satisfactory, superior) to force more scores to the middle. However, this approach might limit the discrimination ability of the tool.

We have previously studied the use of this tool among nurses. In that study, we also found consistently higher scores by peers than by external evaluators. We did, however, find a positive effect of experience, in which more experienced nurses received higher scores on average. We did not observe a similar training effect in this study. There are several possible explanations for the lack of a training effect. It is possible that the types of handoffs assessed played a role. At UCM, some assessed handoffs were night staff to day staff, which might be lower quality than day staff to night staff handoffs, whereas at Yale, all handoffs were day to night teams. Thus, average scores at UCM (primarily hospitalists) might have been lowered by the type of handoff provided. Given that hospitalist evaluations were conducted exclusively at UCM and housestaff evaluations exclusively at Yale, lack of difference between hospitalists and housestaff may also have been related to differences in evaluation practice or handoff practice at the 2 sites, not necessarily related to training level. Third, in our experience, attending physicians provide briefer less‐comprehensive sign‐outs than trainees, particularly when communicating with equally experienced attendings; these sign‐outs may appropriately be scored lower on the tool. Fourth, the great majority of the hospitalists at UCM were within 5 years of residency and therefore not very much more experienced than the trainees. Finally, it is possible that skills do not improve over time given widespread lack of observation and feedback during training years for this important skill.

The high internal consistency of most of the subdomains and the loading of all subdomains except setting onto 1 factor are evidence of convergent construct validity, but also suggest that evaluators have difficulty distinguishing among components of sign‐out quality. Internal consistency may also reflect a halo effect, in which scores on different domains are all influenced by a common overall judgment.[42] We are currently testing a shorter version of the tool including domains only for content, professionalism, and setting in addition to overall score. The fact that setting did not correlate as well with the other domains suggests that sign‐out practitioners may not have or exercise control over their surroundings. Consequently, it may ultimately be reasonable to drop this domain from the tool, or alternatively, to refocus on the need to ensure a quiet setting during sign‐out skills training.

There are several limitations to this study. External evaluations were conducted by personnel who were not familiar with the patients, and they may therefore have overestimated the quality of sign‐out. Studying different types of physicians at different sites might have limited our ability to identify differences by training level. As is commonly seen in evaluation studies, scores were skewed to the high end, although we did observe some use of the full range of the tool. Finally, we were limited in our ability to test inter‐rater reliability because of the multiple sources of variability in the data (numerous different raters, with different backgrounds at different settings, rating different individuals).

In summary, we developed a handoff evaluation tool that was easily completed by housestaff and attendings without training, that performed similarly in a variety of different settings at 2 institutions, and that can in principle be used either for peer evaluations or for external evaluations, although peer evaluations may be positively biased. Further work will be done to refine and simplify the tool.

Handoff CEX scores by site of evaluation

Spearman correlation, recipients (N=330)

All p values <0.0001