Jeanne M. Farnan, MD, MHPE

METHODS

RESULTS

DISCUSSION

METHODS

RESULTS

DISCUSSION

METHODS

RESULTS

DISCUSSION

Studies examining the importance of continuity of care have shown that patients who maintain a continuous relationship with a single physician have improved outcomes.[1, 2] However, most of these studies were performed in the outpatient, rather than the inpatient setting. With over 35 million patients admitted to hospitals in 2013, along with the significant increase in hospital discontinuity over recent years, the impact of inpatient continuity of care on quality outcomes and patient satisfaction is becoming increasingly relevant.[3, 4]

Service handoffs, when a physician hands over treatment responsibility for a panel of patients and is not expected to return, are a type of handoff that contributes to inpatient discontinuity. In particular, service handoffs between hospitalists are an especially common and inherently risky type of transition, as there is a severing of an established relationship during a patient's hospitalization. Unfortunately, due to the lack of evidence on the effects of service handoffs, current guidelines are limited in their recommendations.[5] Whereas several recent studies have begun to explore the effects of these handoffs, no prior study has examined this issue from a patient's perspective.[6, 7, 8]

Patients are uniquely positioned to inform us about their experiences in care transitions. Furthermore, with patient satisfaction now affecting Medicare reimbursement rates, patient experiences while in the hospital are becoming even more significant.[9] Despite this emphasis toward more patient‐centered care, no study has explored the hospitalized patient's experience with hospitalist service handoffs. Our goal was to qualitatively assess the hospitalized patients' experiences with transitions between hospitalists to develop a conceptual model to inform future work on improving inpatient transitions of care.

METHODS

RESULTS

In total, 43 eligible patients were recruited, and 40 (93%) agreed to participate. Interviewed patients were between 51 and 65 (39%) years old, had a mean age of 54.5 (15) years, were predominantly female (65%), African American (58%), had a median length of stay at the time of interview of 6.5 days (interquartile range [IQR]: 48), and had an average of 2.0 (IQR: 13) hospitalists oversee their care at the time of interview (Table 1). Interview times ranged from 10:25 to 25:48 minutes, with an average of 15:32 minutes.

We identified 6 major themes on patient perceptions of hospitalist service handoffs including (1) physician‐patient communication, (2) transparency in the hospitalist transition process, (3) indifference toward the hospitalist transition, (4) hospitalist‐subspecialist communication, (5) recognition of new opportunities due to a transition, and (6) hospitalists' bedside manner (Table 2).

Conceptual Model

Following the collection and analysis of all patient responses, all themes were utilized to construct the ideal patient‐centered service handoff. The ideal transition describes open lines of communication between all involved parties, is facilitated by multiple modalities, such as the EHRs and nursing staff, and recognizes the patient as the primary stakeholder (Figure 1).

Figure 1

DISCUSSION

To our knowledge, this is the first qualitative investigation of the hospitalized patient's experience with service handoffs between hospitalists. The patient perspective adds a personal and first‐hand description of how fragmented care may impact the hospitalized patient experience.

Of the 6 themes, communication was found to be the most pertinent to our respondents. Because much of patient care is an inherently communicative activity, it is not surprising that patients, as well as patient safety experts, have focused on communication as an area in need of improvement in transition processes.[17, 18] Moreover, multiple medical societies have directly called for improvements within this area, and have specifically recommended clear and direct communication of treatment plans between the patient and physician, timely exchange of information, and knowledge of who is primarily in charge of the patients care.[11] Not surprisingly, each of these recommendations appears to be echoed by our participants. This theme is especially important given that good physician‐patient communication has been noted to be a major goal in achieving patient‐centered care, and has been positively correlated to medication adherence, patient satisfaction, and physical health outcomes.[19, 20, 21, 22, 23]

Although not a substitute for face‐to‐face interactions, other communication interventions between physicians and patients should be considered. For example, get to know me posters placed in patient rooms have been shown to encourage communication between patients and physicians.[24] Additionally, physician face cards have been used to improve patients' abilities to identify and clarify physicians' roles in patient care.[25] As a patient put it, If they got a new one [hospitalist], just as if I got a new medicationprint out information on themlike where they went to med school, and stuff(patient 13). These modalities may represent highly implementable, cost‐effective adjuncts to current handoff methods that may improve lines of communication between physicians and patients.

In addition to the importance placed on physician‐patient communication, interprofessional communication between hospitalists and subspecialists was also highly regarded. Studies have shown that practice‐based interprofessional communication, such as daily interdisciplinary rounds and the use of external facilitators, can improve healthcare processes and outcomes.[26] However, these interventions must be weighed with the many conflicting factors that both hospitalists and subspecialists face on daily basis, including high patient volumes, time limitations, patient availability, and scheduling conflicts.[27] None the less, the strong emphasis patients placed on this line of communication highlights this domain as an area in which hospitalist and subspecialist can work together for systematic improvement.

Patients also recognized the complexity of the transfer process between hospitalists and called for improved transparency. For example, patients repeatedly requested to be informed prior to any changes in their hospitalists, a request that remains consistent with current guidelines.[11] There also existed a strong desire for a more formalized process of transitioning between hospitalists, which often described a handoff procedure that would occur at the patient's bedside. This desire seems to be mirrored in the data that show that patients prefer to interact with their care team at the bedside and report higher satisfaction when they are involved with their care.[28, 29] Unfortunately, this desire for more direct interaction with physicians runs counter to the current paradigm of patient care, where most activities on rounds do not take place at the bedside.[30]

In contrast to patient's calls for improved transparency, an equally large portion of patients expressed relative indifference to the transition. Whereas on the surface this may seem salutary, some studies suggest that a lack of patient activation and engagement may have adverse effects toward patients' overall care.[31] Furthermore, others have shown evidence of better healthcare experiences, improved health outcomes, and lower costs in patients who are more active in their care.[30, 31] Altogether, this suggests that despite some patients' indifference, physicians should continue to engage patients in their hospital care.[32]

Although prevailing sentiments among patient safety advocates are that patient handoffs are inherently dangerous and place patients at increased risk of adverse events, patients did not always share this concern. A frequently occurring theme was that the transition is an opportunity for medical reevaluation or the establishment of a new, possibly improved therapeutic alliance. Recognizing this viewpoint offers oncoming hospitalists the opportunity to focus on issues that the patient may have felt were not being properly addressed with their prior physician.

Finally, although our conceptual model is not a strict guideline, we believe that any future studies should consider this framework when constructing interventions to improve service‐level handoffs. Several interventions already exist. For instance, educational interventions, such as patient‐centered interviewing, have been shown to improve patient satisfaction, compliance with medications, lead to fewer lawsuits, and improve health outcomes.[33, 34, 35] Additional methods of keeping the patient more informed include physician face sheets and performance of the handoff at the patient's bedside. Although well known in nursing literature, the idea of physicians performing handoffs at the patient's bedside is a particularly patient‐centric process.[36] This type of intervention may have the ability to transform the handoff from the current state of a 2‐way street, in which information is passed between 2 hospitalists, to a 3‐way stop, in which both hospitalists and the patient are able to communicate at this critical junction of care.

Although our study does offer new insight into the effects of discontinuous care, its exploratory nature does have limitations. First, being performed at a single academic center limits our ability to generalize our findings. Second, perspectives of those who did not wish to participate, patients' family members or caregivers, and those who were not queried, could highly differ from those we interviewed. Additionally, we did not collect data on patients' diagnoses or reason for admission, thus limiting our ability to assess if certain diagnosis or subpopulations predispose patients to experiencing a service handoff. Third, although our study was restricted to English‐speaking patients only, we must consider that non‐English speakers would likely suffer from even greater communication barriers than those who took part in our study. Finally, our interviews and data analysis were conducted by hospitalists, which could have subconsciously influenced the interview process, and the interpretation of patient responses. However, we tried to mitigate these issues by having the same individual interview all participants, by using an interview guide to ensure cross‐cohort consistency, by using open‐ended questions, and by attempting to give patients every opportunity to express themselves.

CONCLUSIONS

From a patients' perspective, inpatient service handoffs are often opaque experiences that are highlighted by poor communication between physicians and patients. Although deficits in communication and transparency acted as barriers to a patient‐centered handoff, physicians should recognize that service handoffs may also represent opportunities for improvement, and should focus on these domains when they start on a new service.

Studies examining the importance of continuity of care have shown that patients who maintain a continuous relationship with a single physician have improved outcomes.[1, 2] However, most of these studies were performed in the outpatient, rather than the inpatient setting. With over 35 million patients admitted to hospitals in 2013, along with the significant increase in hospital discontinuity over recent years, the impact of inpatient continuity of care on quality outcomes and patient satisfaction is becoming increasingly relevant.[3, 4]

Service handoffs, when a physician hands over treatment responsibility for a panel of patients and is not expected to return, are a type of handoff that contributes to inpatient discontinuity. In particular, service handoffs between hospitalists are an especially common and inherently risky type of transition, as there is a severing of an established relationship during a patient's hospitalization. Unfortunately, due to the lack of evidence on the effects of service handoffs, current guidelines are limited in their recommendations.[5] Whereas several recent studies have begun to explore the effects of these handoffs, no prior study has examined this issue from a patient's perspective.[6, 7, 8]

Patients are uniquely positioned to inform us about their experiences in care transitions. Furthermore, with patient satisfaction now affecting Medicare reimbursement rates, patient experiences while in the hospital are becoming even more significant.[9] Despite this emphasis toward more patient‐centered care, no study has explored the hospitalized patient's experience with hospitalist service handoffs. Our goal was to qualitatively assess the hospitalized patients' experiences with transitions between hospitalists to develop a conceptual model to inform future work on improving inpatient transitions of care.

METHODS

RESULTS

In total, 43 eligible patients were recruited, and 40 (93%) agreed to participate. Interviewed patients were between 51 and 65 (39%) years old, had a mean age of 54.5 (15) years, were predominantly female (65%), African American (58%), had a median length of stay at the time of interview of 6.5 days (interquartile range [IQR]: 48), and had an average of 2.0 (IQR: 13) hospitalists oversee their care at the time of interview (Table 1). Interview times ranged from 10:25 to 25:48 minutes, with an average of 15:32 minutes.

We identified 6 major themes on patient perceptions of hospitalist service handoffs including (1) physician‐patient communication, (2) transparency in the hospitalist transition process, (3) indifference toward the hospitalist transition, (4) hospitalist‐subspecialist communication, (5) recognition of new opportunities due to a transition, and (6) hospitalists' bedside manner (Table 2).

Conceptual Model

Following the collection and analysis of all patient responses, all themes were utilized to construct the ideal patient‐centered service handoff. The ideal transition describes open lines of communication between all involved parties, is facilitated by multiple modalities, such as the EHRs and nursing staff, and recognizes the patient as the primary stakeholder (Figure 1).

Figure 1

DISCUSSION

To our knowledge, this is the first qualitative investigation of the hospitalized patient's experience with service handoffs between hospitalists. The patient perspective adds a personal and first‐hand description of how fragmented care may impact the hospitalized patient experience.

Of the 6 themes, communication was found to be the most pertinent to our respondents. Because much of patient care is an inherently communicative activity, it is not surprising that patients, as well as patient safety experts, have focused on communication as an area in need of improvement in transition processes.[17, 18] Moreover, multiple medical societies have directly called for improvements within this area, and have specifically recommended clear and direct communication of treatment plans between the patient and physician, timely exchange of information, and knowledge of who is primarily in charge of the patients care.[11] Not surprisingly, each of these recommendations appears to be echoed by our participants. This theme is especially important given that good physician‐patient communication has been noted to be a major goal in achieving patient‐centered care, and has been positively correlated to medication adherence, patient satisfaction, and physical health outcomes.[19, 20, 21, 22, 23]

Although not a substitute for face‐to‐face interactions, other communication interventions between physicians and patients should be considered. For example, get to know me posters placed in patient rooms have been shown to encourage communication between patients and physicians.[24] Additionally, physician face cards have been used to improve patients' abilities to identify and clarify physicians' roles in patient care.[25] As a patient put it, If they got a new one [hospitalist], just as if I got a new medicationprint out information on themlike where they went to med school, and stuff(patient 13). These modalities may represent highly implementable, cost‐effective adjuncts to current handoff methods that may improve lines of communication between physicians and patients.

In addition to the importance placed on physician‐patient communication, interprofessional communication between hospitalists and subspecialists was also highly regarded. Studies have shown that practice‐based interprofessional communication, such as daily interdisciplinary rounds and the use of external facilitators, can improve healthcare processes and outcomes.[26] However, these interventions must be weighed with the many conflicting factors that both hospitalists and subspecialists face on daily basis, including high patient volumes, time limitations, patient availability, and scheduling conflicts.[27] None the less, the strong emphasis patients placed on this line of communication highlights this domain as an area in which hospitalist and subspecialist can work together for systematic improvement.

Patients also recognized the complexity of the transfer process between hospitalists and called for improved transparency. For example, patients repeatedly requested to be informed prior to any changes in their hospitalists, a request that remains consistent with current guidelines.[11] There also existed a strong desire for a more formalized process of transitioning between hospitalists, which often described a handoff procedure that would occur at the patient's bedside. This desire seems to be mirrored in the data that show that patients prefer to interact with their care team at the bedside and report higher satisfaction when they are involved with their care.[28, 29] Unfortunately, this desire for more direct interaction with physicians runs counter to the current paradigm of patient care, where most activities on rounds do not take place at the bedside.[30]

In contrast to patient's calls for improved transparency, an equally large portion of patients expressed relative indifference to the transition. Whereas on the surface this may seem salutary, some studies suggest that a lack of patient activation and engagement may have adverse effects toward patients' overall care.[31] Furthermore, others have shown evidence of better healthcare experiences, improved health outcomes, and lower costs in patients who are more active in their care.[30, 31] Altogether, this suggests that despite some patients' indifference, physicians should continue to engage patients in their hospital care.[32]

Although prevailing sentiments among patient safety advocates are that patient handoffs are inherently dangerous and place patients at increased risk of adverse events, patients did not always share this concern. A frequently occurring theme was that the transition is an opportunity for medical reevaluation or the establishment of a new, possibly improved therapeutic alliance. Recognizing this viewpoint offers oncoming hospitalists the opportunity to focus on issues that the patient may have felt were not being properly addressed with their prior physician.

Finally, although our conceptual model is not a strict guideline, we believe that any future studies should consider this framework when constructing interventions to improve service‐level handoffs. Several interventions already exist. For instance, educational interventions, such as patient‐centered interviewing, have been shown to improve patient satisfaction, compliance with medications, lead to fewer lawsuits, and improve health outcomes.[33, 34, 35] Additional methods of keeping the patient more informed include physician face sheets and performance of the handoff at the patient's bedside. Although well known in nursing literature, the idea of physicians performing handoffs at the patient's bedside is a particularly patient‐centric process.[36] This type of intervention may have the ability to transform the handoff from the current state of a 2‐way street, in which information is passed between 2 hospitalists, to a 3‐way stop, in which both hospitalists and the patient are able to communicate at this critical junction of care.

Although our study does offer new insight into the effects of discontinuous care, its exploratory nature does have limitations. First, being performed at a single academic center limits our ability to generalize our findings. Second, perspectives of those who did not wish to participate, patients' family members or caregivers, and those who were not queried, could highly differ from those we interviewed. Additionally, we did not collect data on patients' diagnoses or reason for admission, thus limiting our ability to assess if certain diagnosis or subpopulations predispose patients to experiencing a service handoff. Third, although our study was restricted to English‐speaking patients only, we must consider that non‐English speakers would likely suffer from even greater communication barriers than those who took part in our study. Finally, our interviews and data analysis were conducted by hospitalists, which could have subconsciously influenced the interview process, and the interpretation of patient responses. However, we tried to mitigate these issues by having the same individual interview all participants, by using an interview guide to ensure cross‐cohort consistency, by using open‐ended questions, and by attempting to give patients every opportunity to express themselves.

CONCLUSIONS

From a patients' perspective, inpatient service handoffs are often opaque experiences that are highlighted by poor communication between physicians and patients. Although deficits in communication and transparency acted as barriers to a patient‐centered handoff, physicians should recognize that service handoffs may also represent opportunities for improvement, and should focus on these domains when they start on a new service.

Studies examining the importance of continuity of care have shown that patients who maintain a continuous relationship with a single physician have improved outcomes.[1, 2] However, most of these studies were performed in the outpatient, rather than the inpatient setting. With over 35 million patients admitted to hospitals in 2013, along with the significant increase in hospital discontinuity over recent years, the impact of inpatient continuity of care on quality outcomes and patient satisfaction is becoming increasingly relevant.[3, 4]

Service handoffs, when a physician hands over treatment responsibility for a panel of patients and is not expected to return, are a type of handoff that contributes to inpatient discontinuity. In particular, service handoffs between hospitalists are an especially common and inherently risky type of transition, as there is a severing of an established relationship during a patient's hospitalization. Unfortunately, due to the lack of evidence on the effects of service handoffs, current guidelines are limited in their recommendations.[5] Whereas several recent studies have begun to explore the effects of these handoffs, no prior study has examined this issue from a patient's perspective.[6, 7, 8]

Patients are uniquely positioned to inform us about their experiences in care transitions. Furthermore, with patient satisfaction now affecting Medicare reimbursement rates, patient experiences while in the hospital are becoming even more significant.[9] Despite this emphasis toward more patient‐centered care, no study has explored the hospitalized patient's experience with hospitalist service handoffs. Our goal was to qualitatively assess the hospitalized patients' experiences with transitions between hospitalists to develop a conceptual model to inform future work on improving inpatient transitions of care.

METHODS

RESULTS

In total, 43 eligible patients were recruited, and 40 (93%) agreed to participate. Interviewed patients were between 51 and 65 (39%) years old, had a mean age of 54.5 (15) years, were predominantly female (65%), African American (58%), had a median length of stay at the time of interview of 6.5 days (interquartile range [IQR]: 48), and had an average of 2.0 (IQR: 13) hospitalists oversee their care at the time of interview (Table 1). Interview times ranged from 10:25 to 25:48 minutes, with an average of 15:32 minutes.

We identified 6 major themes on patient perceptions of hospitalist service handoffs including (1) physician‐patient communication, (2) transparency in the hospitalist transition process, (3) indifference toward the hospitalist transition, (4) hospitalist‐subspecialist communication, (5) recognition of new opportunities due to a transition, and (6) hospitalists' bedside manner (Table 2).

Conceptual Model

Following the collection and analysis of all patient responses, all themes were utilized to construct the ideal patient‐centered service handoff. The ideal transition describes open lines of communication between all involved parties, is facilitated by multiple modalities, such as the EHRs and nursing staff, and recognizes the patient as the primary stakeholder (Figure 1).

Figure 1

DISCUSSION

To our knowledge, this is the first qualitative investigation of the hospitalized patient's experience with service handoffs between hospitalists. The patient perspective adds a personal and first‐hand description of how fragmented care may impact the hospitalized patient experience.

Of the 6 themes, communication was found to be the most pertinent to our respondents. Because much of patient care is an inherently communicative activity, it is not surprising that patients, as well as patient safety experts, have focused on communication as an area in need of improvement in transition processes.[17, 18] Moreover, multiple medical societies have directly called for improvements within this area, and have specifically recommended clear and direct communication of treatment plans between the patient and physician, timely exchange of information, and knowledge of who is primarily in charge of the patients care.[11] Not surprisingly, each of these recommendations appears to be echoed by our participants. This theme is especially important given that good physician‐patient communication has been noted to be a major goal in achieving patient‐centered care, and has been positively correlated to medication adherence, patient satisfaction, and physical health outcomes.[19, 20, 21, 22, 23]

Although not a substitute for face‐to‐face interactions, other communication interventions between physicians and patients should be considered. For example, get to know me posters placed in patient rooms have been shown to encourage communication between patients and physicians.[24] Additionally, physician face cards have been used to improve patients' abilities to identify and clarify physicians' roles in patient care.[25] As a patient put it, If they got a new one [hospitalist], just as if I got a new medicationprint out information on themlike where they went to med school, and stuff(patient 13). These modalities may represent highly implementable, cost‐effective adjuncts to current handoff methods that may improve lines of communication between physicians and patients.

In addition to the importance placed on physician‐patient communication, interprofessional communication between hospitalists and subspecialists was also highly regarded. Studies have shown that practice‐based interprofessional communication, such as daily interdisciplinary rounds and the use of external facilitators, can improve healthcare processes and outcomes.[26] However, these interventions must be weighed with the many conflicting factors that both hospitalists and subspecialists face on daily basis, including high patient volumes, time limitations, patient availability, and scheduling conflicts.[27] None the less, the strong emphasis patients placed on this line of communication highlights this domain as an area in which hospitalist and subspecialist can work together for systematic improvement.

Patients also recognized the complexity of the transfer process between hospitalists and called for improved transparency. For example, patients repeatedly requested to be informed prior to any changes in their hospitalists, a request that remains consistent with current guidelines.[11] There also existed a strong desire for a more formalized process of transitioning between hospitalists, which often described a handoff procedure that would occur at the patient's bedside. This desire seems to be mirrored in the data that show that patients prefer to interact with their care team at the bedside and report higher satisfaction when they are involved with their care.[28, 29] Unfortunately, this desire for more direct interaction with physicians runs counter to the current paradigm of patient care, where most activities on rounds do not take place at the bedside.[30]

In contrast to patient's calls for improved transparency, an equally large portion of patients expressed relative indifference to the transition. Whereas on the surface this may seem salutary, some studies suggest that a lack of patient activation and engagement may have adverse effects toward patients' overall care.[31] Furthermore, others have shown evidence of better healthcare experiences, improved health outcomes, and lower costs in patients who are more active in their care.[30, 31] Altogether, this suggests that despite some patients' indifference, physicians should continue to engage patients in their hospital care.[32]

Although prevailing sentiments among patient safety advocates are that patient handoffs are inherently dangerous and place patients at increased risk of adverse events, patients did not always share this concern. A frequently occurring theme was that the transition is an opportunity for medical reevaluation or the establishment of a new, possibly improved therapeutic alliance. Recognizing this viewpoint offers oncoming hospitalists the opportunity to focus on issues that the patient may have felt were not being properly addressed with their prior physician.

Finally, although our conceptual model is not a strict guideline, we believe that any future studies should consider this framework when constructing interventions to improve service‐level handoffs. Several interventions already exist. For instance, educational interventions, such as patient‐centered interviewing, have been shown to improve patient satisfaction, compliance with medications, lead to fewer lawsuits, and improve health outcomes.[33, 34, 35] Additional methods of keeping the patient more informed include physician face sheets and performance of the handoff at the patient's bedside. Although well known in nursing literature, the idea of physicians performing handoffs at the patient's bedside is a particularly patient‐centric process.[36] This type of intervention may have the ability to transform the handoff from the current state of a 2‐way street, in which information is passed between 2 hospitalists, to a 3‐way stop, in which both hospitalists and the patient are able to communicate at this critical junction of care.

Although our study does offer new insight into the effects of discontinuous care, its exploratory nature does have limitations. First, being performed at a single academic center limits our ability to generalize our findings. Second, perspectives of those who did not wish to participate, patients' family members or caregivers, and those who were not queried, could highly differ from those we interviewed. Additionally, we did not collect data on patients' diagnoses or reason for admission, thus limiting our ability to assess if certain diagnosis or subpopulations predispose patients to experiencing a service handoff. Third, although our study was restricted to English‐speaking patients only, we must consider that non‐English speakers would likely suffer from even greater communication barriers than those who took part in our study. Finally, our interviews and data analysis were conducted by hospitalists, which could have subconsciously influenced the interview process, and the interpretation of patient responses. However, we tried to mitigate these issues by having the same individual interview all participants, by using an interview guide to ensure cross‐cohort consistency, by using open‐ended questions, and by attempting to give patients every opportunity to express themselves.

CONCLUSIONS

From a patients' perspective, inpatient service handoffs are often opaque experiences that are highlighted by poor communication between physicians and patients. Although deficits in communication and transparency acted as barriers to a patient‐centered handoff, physicians should recognize that service handoffs may also represent opportunities for improvement, and should focus on these domains when they start on a new service.

Over the last decade, there has been an unprecedented focus on physician handoffs in US hospitals. One major reason for this are the reductions in residency duty hours that have been mandated by the American Council for Graduate Medical Education (ACGME), first in 2003 and subsequently revised in 2011.[1, 2] As residents work fewer hours, experts believe that potential safety gains from reduced fatigue are countered by an increase in the number of handoffs, which represent a risk due to the potential miscommunication. Prior studies show that critical patient information is often lost or altered during this transfer of clinical information and professional responsibility, which can result in patient harm.[3, 4] As a result of these concerns, the ACGME now requires residency programs to ensure and monitor effective, structured hand‐over processes to facilitate both continuity of care and patient safety. Programs must ensure that residents are competent in communicating with team members in the hand‐over process.[2] Moreover, handoffs have also been a major improvement focus for organizations with broader scope than teaching hospitals, including the World Health Organization, Joint Commission, and the Society for Hospital Medicine (SHM).[5, 6, 7]

Despite this focus on handoffs, monitoring quality of handoffs has proven challenging due to lack of a reliable and validated tool to measure handoff quality. More recently, the Accreditation Council of Graduate Medical Education's introduction of the Next Accreditation System, with its focus on direct observation of clinical skills to achieve milestones, makes it crucial for residency educators to have valid tools to measure competence in handoffs. As a result, it is critical that instruments to measure handoff performance are not only created but also validated.[8]

To help fill this gap, we previously reported on the development of a 9‐item Handoff Clinical Examination Exercise (CEX) assessment tool. The Handoff CEX, designed for use by those participating in the handoff or by a third‐party observer, can be used to rate the quality of patient handoffs in domains such as professionalism and communication skills between the receiver and sender of patient information.[9, 10] Despite prior demonstration of feasibility of use, the initial tool was perceived as lengthy and redundant. In addition, although the tool has been shown to discriminate between performance of novice and expert nurses, the construct validity of this tool has not been established.[11] Establishing construct validity is important to ensuring that the tool can measure the construct in question, namely whether it detects those who are actually competent to perform handoffs safely and effectively. We present here the results of the development of a shorter Handoff Mini‐CEX, along with the formal establishment of its construct validity, namely its ability to distinguish between levels of performance in 3 domains of handoff quality.

METHODS

RESULTS

Forty‐seven faculty members (14=site 1; 33=site 2) participated in the validation workshops (2 at the University of Chicago, and 2 at Yale University), which were held in August 2011 and September 2011, providing a total of 172 observations of a possible 191 (90%).

The overall handoff quality ratings for the superior, gold standard video (superior communication, professionalism, and communication) ranged from 7 to 9 with a mean of 8.5 (standard deviation [SD] 0.7). The overall ratings for the video depicting satisfactory communication (satisfactory communication, superior professionalism and setting) ranged from 5 to 9 with a mean of 7.3 (SD 1.1). The overall ratings for the unsatisfactory communication (unsatisfactory communication, superior professionalism and setting) video ranged from 1 to 7 with a mean of 2.6 (SD 1.2). The overall ratings for the satisfactory professionalism video (satisfactory professionalism, superior communication and setting) ranged from 4 to 8 with a mean of 5.7 (SD 1.3). The overall ratings for the unsatisfactory professionalism (unsatisfactory professionalism, superior communication and setting) video ranged from 2 to 5 with a mean of 2.4 (SD 1.03). Finally, the overall ratings for the unsatisfactory setting (unsatisfactory setting, superior communication and professionalism) video ranged from 1 to 8 with a mean of 3.1 (SD 1.7).

Figure 1 demonstrates that for the domain of communication, the raters were able to discern the unsatisfactory performance but had difficulty reliably distinguishing between superior and satisfactory performance. Figure 2 illustrates that for the domain of professionalism, raters were able to detect the videos' changing levels of performance at the extremes of behavior, with unsatisfactory and superior displays more readily identified. Figure 3 shows that for the domain of setting, the raters were able to discern the unsatisfactory versus superior level of the changing setting. Of note, we also found a moderate significant correlation between ratings of professionalism and communication (r=0.47, P<0.001).

Figure 1

Figure 2

Figure 3

The Cronbach , or measurement of internal reliability and consistency, for the Handoff Mini‐CEX (3 items plus overall) was 0.77, indicating high internal reliability and consistency. Using data from University of Chicago, where raters were labeled with a unique identifier, the Kendall coefficient of concordance was calculated to be 0.79, demonstrating high inter‐rater reliability of the faculty raters. High inter‐rater reliability was also seen using intraclass coefficients for each domain: communication (0.84), professionalism (0.68), setting (0.83), and overall (0.89). Using generalizability analysis, the average reliability was determined to be above 0.9 for all domains (0.99 for overall).

Last, the 2‐way ANOVA (n=75 observations from 13 raters) revealed no evidence of rater bias when examining the coefficient for attending rater (P=0.55 for professionalism, P=0.45 for communication, P=0.92 for setting). The range of scores for each video, however, was broad (Table 2).

DISCUSSION

This study demonstrates that valid conclusions on handoff performance can be drawn using the Handoff CEX as the instrument to rate handoff quality. Utilizing standardized videos depicting varying levels of performance communication, professionalism, and setting, the Handoff Mini‐CEX has demonstrated potential to discern between increasing levels of performance, providing evidence for the construct validity of the instrument.

We observed that faculty could reliably detect unsatisfactory professionalism with ease, and that there was a distinct correlation between faculty ratings and the internally set levels of performance displayed in the videos. This trend demonstrated that faculty were able to discern different levels of professionalism using the Handoff Mini‐CEX. It became more difficult, however, for faculty to detect superior professionalism when the domain of communication was permuted. If the sender of the handoff was professional but the information delivered was disorganized, inaccurate, and missing crucial pieces of information, the faculty perceived this ineffective communication as unprofessional. Prior literature on professionalism has found that communication is a necessary component of professional behavior, and consequently, being a competent communicator is necessary to fulfill ones duty as a professional physician.[15, 16]

This is of note because we did find a moderate significant correlation between ratings of professionalism and communication. It is possible that this distinction would be made clearer with formal rater training in the future prior to any evaluations. However, it is also possible that professionalism and communication, due to a synergistic role between the 2 domains, cannot be separated. If this is the case, it would be important to educate clinicians to present patients in a concise, clear, and accurate way with a professional demeanor. Acknowledging professional responsibility as an integral piece of patient care is also critical in effectively communicating patient information.[5]

We also noted that faculty could detect unsatisfactory communication consistently; however, they were unable to differentiate between satisfactory and superior communication reliably or consistently. Because the unsatisfactory professionalism, unsatisfactory setting, and satisfactory professionalism videos all demonstrated superior communication, we believe that the faculty penalized communication when distractions, in the form of interruptions and rude behavior by the resident giving the handoff, interrupted the flow of the handoff. Thus, the wide ranges in scores observed by some raters may be attributed to this interaction between the Handoff Mini‐CEX domains. In the future, definitions of the anchors, including at the middle spectrum of performance, and rater training may improve the ability of raters to distinguish performance between each domain.

The overall value of the Handoff Mini‐CEX is in its ease of use, in part due to its brevity, as well as evidence for its validity in distinguishing between varying levels of performance. Given the emphasis on monitoring handoff quality and performance, the Handoff Mini‐CEX provides a standard foundation from which baseline handoff performance can be easily measured and improved. Moreover, it can also be used to give individual feedback to a specific practicing clinician on their practices and an opportunity to improve. This is particularly important given current recommendations by the Joint Commission, that handoffs are standardized, and by the ACGME, that residents are competent in handoff skills. Moreover, given the creation of the SHM's handoff recommendations and handoffs as a core competency for hospitalists, the tool provides the ability for hospitalist programs to actually assess their handoff practices as baseline measurements for any quality improvement activities that may take place.

Faculty were able to discern the superior and unsatisfactory levels of setting with ease. After watching and rating the videos, participants said that the chaotic scene of the unsatisfactory setting video had significant authenticity, and that they were constantly interrupted during their own handoffs by pages, phone calls, and people entering the handoff space. System‐level fixes, such as protected time and dedicated space for handoffs, and discouraging pages to be sent during the designated handoff time, could mitigate the reality of unsatisfactory settings.[17, 18]

Our study has several limitations. First, although this study was held at 2 sites, it included a small number of faculty, which can impact the generalizability of our findings. Implementation varied at Yale University and the University of Chicago, preventing use of all data for all analyses. Furthermore, institutional culture may also impact faculty raters' perceptions, so future work aims at repeating our protocol at partner institutions, increasing both the number and diversity of participants. We were also unable to compare the new shorter Handoff Mini‐CEX to the larger 9‐item Handoff CEX in this study.

Despite these limitations, we believe that the Handoff Mini‐CEX, has future potential as an instrument with which to make valid and reliable conclusions about handoff quality, and could be used to both evaluate handoff quality and as an educational tool for trainees and faculty on effective handoff communication.

Over the last decade, there has been an unprecedented focus on physician handoffs in US hospitals. One major reason for this are the reductions in residency duty hours that have been mandated by the American Council for Graduate Medical Education (ACGME), first in 2003 and subsequently revised in 2011.[1, 2] As residents work fewer hours, experts believe that potential safety gains from reduced fatigue are countered by an increase in the number of handoffs, which represent a risk due to the potential miscommunication. Prior studies show that critical patient information is often lost or altered during this transfer of clinical information and professional responsibility, which can result in patient harm.[3, 4] As a result of these concerns, the ACGME now requires residency programs to ensure and monitor effective, structured hand‐over processes to facilitate both continuity of care and patient safety. Programs must ensure that residents are competent in communicating with team members in the hand‐over process.[2] Moreover, handoffs have also been a major improvement focus for organizations with broader scope than teaching hospitals, including the World Health Organization, Joint Commission, and the Society for Hospital Medicine (SHM).[5, 6, 7]

Despite this focus on handoffs, monitoring quality of handoffs has proven challenging due to lack of a reliable and validated tool to measure handoff quality. More recently, the Accreditation Council of Graduate Medical Education's introduction of the Next Accreditation System, with its focus on direct observation of clinical skills to achieve milestones, makes it crucial for residency educators to have valid tools to measure competence in handoffs. As a result, it is critical that instruments to measure handoff performance are not only created but also validated.[8]

To help fill this gap, we previously reported on the development of a 9‐item Handoff Clinical Examination Exercise (CEX) assessment tool. The Handoff CEX, designed for use by those participating in the handoff or by a third‐party observer, can be used to rate the quality of patient handoffs in domains such as professionalism and communication skills between the receiver and sender of patient information.[9, 10] Despite prior demonstration of feasibility of use, the initial tool was perceived as lengthy and redundant. In addition, although the tool has been shown to discriminate between performance of novice and expert nurses, the construct validity of this tool has not been established.[11] Establishing construct validity is important to ensuring that the tool can measure the construct in question, namely whether it detects those who are actually competent to perform handoffs safely and effectively. We present here the results of the development of a shorter Handoff Mini‐CEX, along with the formal establishment of its construct validity, namely its ability to distinguish between levels of performance in 3 domains of handoff quality.

METHODS

RESULTS

Forty‐seven faculty members (14=site 1; 33=site 2) participated in the validation workshops (2 at the University of Chicago, and 2 at Yale University), which were held in August 2011 and September 2011, providing a total of 172 observations of a possible 191 (90%).

The overall handoff quality ratings for the superior, gold standard video (superior communication, professionalism, and communication) ranged from 7 to 9 with a mean of 8.5 (standard deviation [SD] 0.7). The overall ratings for the video depicting satisfactory communication (satisfactory communication, superior professionalism and setting) ranged from 5 to 9 with a mean of 7.3 (SD 1.1). The overall ratings for the unsatisfactory communication (unsatisfactory communication, superior professionalism and setting) video ranged from 1 to 7 with a mean of 2.6 (SD 1.2). The overall ratings for the satisfactory professionalism video (satisfactory professionalism, superior communication and setting) ranged from 4 to 8 with a mean of 5.7 (SD 1.3). The overall ratings for the unsatisfactory professionalism (unsatisfactory professionalism, superior communication and setting) video ranged from 2 to 5 with a mean of 2.4 (SD 1.03). Finally, the overall ratings for the unsatisfactory setting (unsatisfactory setting, superior communication and professionalism) video ranged from 1 to 8 with a mean of 3.1 (SD 1.7).

Figure 1 demonstrates that for the domain of communication, the raters were able to discern the unsatisfactory performance but had difficulty reliably distinguishing between superior and satisfactory performance. Figure 2 illustrates that for the domain of professionalism, raters were able to detect the videos' changing levels of performance at the extremes of behavior, with unsatisfactory and superior displays more readily identified. Figure 3 shows that for the domain of setting, the raters were able to discern the unsatisfactory versus superior level of the changing setting. Of note, we also found a moderate significant correlation between ratings of professionalism and communication (r=0.47, P<0.001).

Figure 1

Figure 2

Figure 3

The Cronbach , or measurement of internal reliability and consistency, for the Handoff Mini‐CEX (3 items plus overall) was 0.77, indicating high internal reliability and consistency. Using data from University of Chicago, where raters were labeled with a unique identifier, the Kendall coefficient of concordance was calculated to be 0.79, demonstrating high inter‐rater reliability of the faculty raters. High inter‐rater reliability was also seen using intraclass coefficients for each domain: communication (0.84), professionalism (0.68), setting (0.83), and overall (0.89). Using generalizability analysis, the average reliability was determined to be above 0.9 for all domains (0.99 for overall).

Last, the 2‐way ANOVA (n=75 observations from 13 raters) revealed no evidence of rater bias when examining the coefficient for attending rater (P=0.55 for professionalism, P=0.45 for communication, P=0.92 for setting). The range of scores for each video, however, was broad (Table 2).

DISCUSSION

This study demonstrates that valid conclusions on handoff performance can be drawn using the Handoff CEX as the instrument to rate handoff quality. Utilizing standardized videos depicting varying levels of performance communication, professionalism, and setting, the Handoff Mini‐CEX has demonstrated potential to discern between increasing levels of performance, providing evidence for the construct validity of the instrument.

We observed that faculty could reliably detect unsatisfactory professionalism with ease, and that there was a distinct correlation between faculty ratings and the internally set levels of performance displayed in the videos. This trend demonstrated that faculty were able to discern different levels of professionalism using the Handoff Mini‐CEX. It became more difficult, however, for faculty to detect superior professionalism when the domain of communication was permuted. If the sender of the handoff was professional but the information delivered was disorganized, inaccurate, and missing crucial pieces of information, the faculty perceived this ineffective communication as unprofessional. Prior literature on professionalism has found that communication is a necessary component of professional behavior, and consequently, being a competent communicator is necessary to fulfill ones duty as a professional physician.[15, 16]

This is of note because we did find a moderate significant correlation between ratings of professionalism and communication. It is possible that this distinction would be made clearer with formal rater training in the future prior to any evaluations. However, it is also possible that professionalism and communication, due to a synergistic role between the 2 domains, cannot be separated. If this is the case, it would be important to educate clinicians to present patients in a concise, clear, and accurate way with a professional demeanor. Acknowledging professional responsibility as an integral piece of patient care is also critical in effectively communicating patient information.[5]

We also noted that faculty could detect unsatisfactory communication consistently; however, they were unable to differentiate between satisfactory and superior communication reliably or consistently. Because the unsatisfactory professionalism, unsatisfactory setting, and satisfactory professionalism videos all demonstrated superior communication, we believe that the faculty penalized communication when distractions, in the form of interruptions and rude behavior by the resident giving the handoff, interrupted the flow of the handoff. Thus, the wide ranges in scores observed by some raters may be attributed to this interaction between the Handoff Mini‐CEX domains. In the future, definitions of the anchors, including at the middle spectrum of performance, and rater training may improve the ability of raters to distinguish performance between each domain.

The overall value of the Handoff Mini‐CEX is in its ease of use, in part due to its brevity, as well as evidence for its validity in distinguishing between varying levels of performance. Given the emphasis on monitoring handoff quality and performance, the Handoff Mini‐CEX provides a standard foundation from which baseline handoff performance can be easily measured and improved. Moreover, it can also be used to give individual feedback to a specific practicing clinician on their practices and an opportunity to improve. This is particularly important given current recommendations by the Joint Commission, that handoffs are standardized, and by the ACGME, that residents are competent in handoff skills. Moreover, given the creation of the SHM's handoff recommendations and handoffs as a core competency for hospitalists, the tool provides the ability for hospitalist programs to actually assess their handoff practices as baseline measurements for any quality improvement activities that may take place.

Faculty were able to discern the superior and unsatisfactory levels of setting with ease. After watching and rating the videos, participants said that the chaotic scene of the unsatisfactory setting video had significant authenticity, and that they were constantly interrupted during their own handoffs by pages, phone calls, and people entering the handoff space. System‐level fixes, such as protected time and dedicated space for handoffs, and discouraging pages to be sent during the designated handoff time, could mitigate the reality of unsatisfactory settings.[17, 18]

Our study has several limitations. First, although this study was held at 2 sites, it included a small number of faculty, which can impact the generalizability of our findings. Implementation varied at Yale University and the University of Chicago, preventing use of all data for all analyses. Furthermore, institutional culture may also impact faculty raters' perceptions, so future work aims at repeating our protocol at partner institutions, increasing both the number and diversity of participants. We were also unable to compare the new shorter Handoff Mini‐CEX to the larger 9‐item Handoff CEX in this study.

Despite these limitations, we believe that the Handoff Mini‐CEX, has future potential as an instrument with which to make valid and reliable conclusions about handoff quality, and could be used to both evaluate handoff quality and as an educational tool for trainees and faculty on effective handoff communication.

Over the last decade, there has been an unprecedented focus on physician handoffs in US hospitals. One major reason for this are the reductions in residency duty hours that have been mandated by the American Council for Graduate Medical Education (ACGME), first in 2003 and subsequently revised in 2011.[1, 2] As residents work fewer hours, experts believe that potential safety gains from reduced fatigue are countered by an increase in the number of handoffs, which represent a risk due to the potential miscommunication. Prior studies show that critical patient information is often lost or altered during this transfer of clinical information and professional responsibility, which can result in patient harm.[3, 4] As a result of these concerns, the ACGME now requires residency programs to ensure and monitor effective, structured hand‐over processes to facilitate both continuity of care and patient safety. Programs must ensure that residents are competent in communicating with team members in the hand‐over process.[2] Moreover, handoffs have also been a major improvement focus for organizations with broader scope than teaching hospitals, including the World Health Organization, Joint Commission, and the Society for Hospital Medicine (SHM).[5, 6, 7]

Despite this focus on handoffs, monitoring quality of handoffs has proven challenging due to lack of a reliable and validated tool to measure handoff quality. More recently, the Accreditation Council of Graduate Medical Education's introduction of the Next Accreditation System, with its focus on direct observation of clinical skills to achieve milestones, makes it crucial for residency educators to have valid tools to measure competence in handoffs. As a result, it is critical that instruments to measure handoff performance are not only created but also validated.[8]

To help fill this gap, we previously reported on the development of a 9‐item Handoff Clinical Examination Exercise (CEX) assessment tool. The Handoff CEX, designed for use by those participating in the handoff or by a third‐party observer, can be used to rate the quality of patient handoffs in domains such as professionalism and communication skills between the receiver and sender of patient information.[9, 10] Despite prior demonstration of feasibility of use, the initial tool was perceived as lengthy and redundant. In addition, although the tool has been shown to discriminate between performance of novice and expert nurses, the construct validity of this tool has not been established.[11] Establishing construct validity is important to ensuring that the tool can measure the construct in question, namely whether it detects those who are actually competent to perform handoffs safely and effectively. We present here the results of the development of a shorter Handoff Mini‐CEX, along with the formal establishment of its construct validity, namely its ability to distinguish between levels of performance in 3 domains of handoff quality.

METHODS

RESULTS

Forty‐seven faculty members (14=site 1; 33=site 2) participated in the validation workshops (2 at the University of Chicago, and 2 at Yale University), which were held in August 2011 and September 2011, providing a total of 172 observations of a possible 191 (90%).

The overall handoff quality ratings for the superior, gold standard video (superior communication, professionalism, and communication) ranged from 7 to 9 with a mean of 8.5 (standard deviation [SD] 0.7). The overall ratings for the video depicting satisfactory communication (satisfactory communication, superior professionalism and setting) ranged from 5 to 9 with a mean of 7.3 (SD 1.1). The overall ratings for the unsatisfactory communication (unsatisfactory communication, superior professionalism and setting) video ranged from 1 to 7 with a mean of 2.6 (SD 1.2). The overall ratings for the satisfactory professionalism video (satisfactory professionalism, superior communication and setting) ranged from 4 to 8 with a mean of 5.7 (SD 1.3). The overall ratings for the unsatisfactory professionalism (unsatisfactory professionalism, superior communication and setting) video ranged from 2 to 5 with a mean of 2.4 (SD 1.03). Finally, the overall ratings for the unsatisfactory setting (unsatisfactory setting, superior communication and professionalism) video ranged from 1 to 8 with a mean of 3.1 (SD 1.7).

Figure 1 demonstrates that for the domain of communication, the raters were able to discern the unsatisfactory performance but had difficulty reliably distinguishing between superior and satisfactory performance. Figure 2 illustrates that for the domain of professionalism, raters were able to detect the videos' changing levels of performance at the extremes of behavior, with unsatisfactory and superior displays more readily identified. Figure 3 shows that for the domain of setting, the raters were able to discern the unsatisfactory versus superior level of the changing setting. Of note, we also found a moderate significant correlation between ratings of professionalism and communication (r=0.47, P<0.001).

Figure 1

Figure 2

Figure 3

The Cronbach , or measurement of internal reliability and consistency, for the Handoff Mini‐CEX (3 items plus overall) was 0.77, indicating high internal reliability and consistency. Using data from University of Chicago, where raters were labeled with a unique identifier, the Kendall coefficient of concordance was calculated to be 0.79, demonstrating high inter‐rater reliability of the faculty raters. High inter‐rater reliability was also seen using intraclass coefficients for each domain: communication (0.84), professionalism (0.68), setting (0.83), and overall (0.89). Using generalizability analysis, the average reliability was determined to be above 0.9 for all domains (0.99 for overall).

Last, the 2‐way ANOVA (n=75 observations from 13 raters) revealed no evidence of rater bias when examining the coefficient for attending rater (P=0.55 for professionalism, P=0.45 for communication, P=0.92 for setting). The range of scores for each video, however, was broad (Table 2).

DISCUSSION

This study demonstrates that valid conclusions on handoff performance can be drawn using the Handoff CEX as the instrument to rate handoff quality. Utilizing standardized videos depicting varying levels of performance communication, professionalism, and setting, the Handoff Mini‐CEX has demonstrated potential to discern between increasing levels of performance, providing evidence for the construct validity of the instrument.

We observed that faculty could reliably detect unsatisfactory professionalism with ease, and that there was a distinct correlation between faculty ratings and the internally set levels of performance displayed in the videos. This trend demonstrated that faculty were able to discern different levels of professionalism using the Handoff Mini‐CEX. It became more difficult, however, for faculty to detect superior professionalism when the domain of communication was permuted. If the sender of the handoff was professional but the information delivered was disorganized, inaccurate, and missing crucial pieces of information, the faculty perceived this ineffective communication as unprofessional. Prior literature on professionalism has found that communication is a necessary component of professional behavior, and consequently, being a competent communicator is necessary to fulfill ones duty as a professional physician.[15, 16]

This is of note because we did find a moderate significant correlation between ratings of professionalism and communication. It is possible that this distinction would be made clearer with formal rater training in the future prior to any evaluations. However, it is also possible that professionalism and communication, due to a synergistic role between the 2 domains, cannot be separated. If this is the case, it would be important to educate clinicians to present patients in a concise, clear, and accurate way with a professional demeanor. Acknowledging professional responsibility as an integral piece of patient care is also critical in effectively communicating patient information.[5]

We also noted that faculty could detect unsatisfactory communication consistently; however, they were unable to differentiate between satisfactory and superior communication reliably or consistently. Because the unsatisfactory professionalism, unsatisfactory setting, and satisfactory professionalism videos all demonstrated superior communication, we believe that the faculty penalized communication when distractions, in the form of interruptions and rude behavior by the resident giving the handoff, interrupted the flow of the handoff. Thus, the wide ranges in scores observed by some raters may be attributed to this interaction between the Handoff Mini‐CEX domains. In the future, definitions of the anchors, including at the middle spectrum of performance, and rater training may improve the ability of raters to distinguish performance between each domain.

The overall value of the Handoff Mini‐CEX is in its ease of use, in part due to its brevity, as well as evidence for its validity in distinguishing between varying levels of performance. Given the emphasis on monitoring handoff quality and performance, the Handoff Mini‐CEX provides a standard foundation from which baseline handoff performance can be easily measured and improved. Moreover, it can also be used to give individual feedback to a specific practicing clinician on their practices and an opportunity to improve. This is particularly important given current recommendations by the Joint Commission, that handoffs are standardized, and by the ACGME, that residents are competent in handoff skills. Moreover, given the creation of the SHM's handoff recommendations and handoffs as a core competency for hospitalists, the tool provides the ability for hospitalist programs to actually assess their handoff practices as baseline measurements for any quality improvement activities that may take place.

Faculty were able to discern the superior and unsatisfactory levels of setting with ease. After watching and rating the videos, participants said that the chaotic scene of the unsatisfactory setting video had significant authenticity, and that they were constantly interrupted during their own handoffs by pages, phone calls, and people entering the handoff space. System‐level fixes, such as protected time and dedicated space for handoffs, and discouraging pages to be sent during the designated handoff time, could mitigate the reality of unsatisfactory settings.[17, 18]

Our study has several limitations. First, although this study was held at 2 sites, it included a small number of faculty, which can impact the generalizability of our findings. Implementation varied at Yale University and the University of Chicago, preventing use of all data for all analyses. Furthermore, institutional culture may also impact faculty raters' perceptions, so future work aims at repeating our protocol at partner institutions, increasing both the number and diversity of participants. We were also unable to compare the new shorter Handoff Mini‐CEX to the larger 9‐item Handoff CEX in this study.

Despite these limitations, we believe that the Handoff Mini‐CEX, has future potential as an instrument with which to make valid and reliable conclusions about handoff quality, and could be used to both evaluate handoff quality and as an educational tool for trainees and faculty on effective handoff communication.

Determining when residents are independently prepared to perform clinical care tasks safely is not easy or understood. Educators have struggled to identify robust ways to evaluate trainees and their preparedness to treat patients while unsupervised. Trust allows the trainee to experience increasing levels of participation and responsibility in the workplace in a way that builds competence for future practice. The breadth of knowledge and skills required to become a competent and safe physician, coupled with the busy workload confound this challenge. Notably, a technically proficient trainee may not have the clinical judgment to treat patients without supervision.

The Accreditation Council of Graduate Medical Education (ACGME) has previously outlined 6 core competencies for residency training: patient care, medical knowledge, practice‐based learning and improvement, interpersonal and communication skills, professionalism, and systems‐based practice.[1] A systematic literature review suggests that traditional trainee evaluation tools are difficult to use and unreliable in measuring the competencies independently from one another, whereas certain competencies are consistently difficult to quantify in a reliable and valid way.[2] The evaluation of trainees' clinical performance despite efforts to create objective tools remain strongly influenced by subjective measures and continues to be highly variable among different evaluators.[3] Objectively measuring resident autonomy and readiness to supervise junior colleagues remains imprecise.[4]

The ACGME's Next Accreditation System (NAS) incorporates educational milestones as part of the reporting of resident training outcomes.[5] The milestones allow for the translation of the core competencies into integrative and observable abilities. Furthermore, the milestone categories are stratified into tiers to allow progress to be measured longitudinally and by task complexity using a novel assessment strategy.

The development of trust between supervisors and trainees is a critical step in decisions to allow increased responsibility and the provision of autonomous decision making, which is an important aspect of physician training. Identifying the factors that influence the supervisors' evaluation of resident competency and capability is at the crux of trainee maturation as well as patient safety.[4] Trust, defined as believability and discernment by attendings of resident physicians, plays a large role in attending evaluations of residents during their clinical rotations.[3] Trust impacts the decisions of successful performance of entrustable professional activities (EPAs), or those tasks that require mastery prior to completion of training milestones.[6] A study of entrustment decisions made by attending anesthesiologists identified the factors that contribute to the amount of autonomy given to residents, such as trainee trustworthiness, medical knowledge, and level of training.[4] The aim of our study, building on this study, was 2‐fold: (1) use deductive qualitative analysis to apply this framework to existing resident and attending data, and (2) define the categories within this framework and describe how internal medicine attending and resident physician perceptions of trust can impact clinical decision making and patient care.

METHODS

We are reporting on a secondary data analysis of interview transcripts from a study conducted on the inpatient general medicine service at the University of Chicago, an academic tertiary care medical center. The methods for data collection and full consent have been outlined previously.[7, 8, 9] The institutional review board of the University of Chicago approved this study.

Briefly, between January 2006 and November 2006, all eligible internal medicine resident physicians, postgraduate year (PGY)‐2 or PGY‐3, and attending physicians, either generalists or hospitalists, were privately interviewed within 1 week of their final call night on the inpatient general medicine rotation to assess decision making and clinical supervision during the rotation. All interviews were conducted by 1 investigator (J.F.), and discussions were audio taped and transcribed for analysis. Interviews were conducted at the conclusion of the rotation to prevent any influence on resident and attending behavior during the rotation.

The critical incident technique, a procedure used for collecting direct observations of human behavior that have critical significance on the decision‐making process, was used to solicit examples of ineffective supervision, inquiring about 2 to 3 important clinical decisions made on the most recent call night, with probes to identify issues of trust, autonomy, and decision making.[10] A critical incident can be described as one that makes a significant contribution, either positively or negatively, on the process.

Appreciative inquiry, a technique that aims to uncover the best things about the clinical encounter being explored, was used to solicit examples of effective supervision. Probes are used to identify factors, either personal or situational, that influenced the withholding or provision of resident autonomy during periods of clinical care delivery.[11]

All identifiable information was removed from the interview transcripts to protect participant and patient confidentiality. Deductive qualitative analysis was performed using the conceptual EPA framework, which describes several factors that influence the attending physicians' decisions to deem a resident trustworthy to independently fulfill a specific clinical task.[4] These factors include (1) the nature of the task, (2) the qualities of the supervisor, (3) the qualities of the trainee and the quality of the relationship between the supervisor and the trainee, and (4) the circumstances surrounding the clinical task.

The deidentified, anonymous transcripts were reviewed by 2 investigators (K.J.C., J.M.F.) and analyzed using the constant comparative methods to deductively map the content to the existing framework and generate novel sub themes.[12, 13, 14] Novel categories within each of the domains were inductively generated. Two reviewers (K.J.C., J.M.F.) independently applied the themes to a randomly selected 10% portion of the interview transcripts to assess the inter‐rater reliability. The inter‐rater agreement was assessed using the generalized kappa statistic. The discrepancies between reviewers regarding assignment of codes were resolved via discussion and third party adjudication until consensus was achieved on thematic structure. The codes were then applied to the entire dataset.

RESULTS

Between January 2006 and November 2006, 46 of 50 (88%) attending physicians and 44 of 50 (92%) resident physicians were interviewed following the conclusion of their general medicine inpatient rotation. Of attending physicians, 55% were male, 45% were female, and 38% were academic faculty hospitalists. Of the residents who completed interviews, 47% were male, 53% were female, 52% were PGY‐2, and 45% were PGY‐3.

A total of 535 mentions of trust were abstracted from the transcripts. The 4 major domains that influence trusttrainee factors (Table 1), supervisor factors (Table 2), task factors (Table 3), and systems factors (Table 4)were deductively coded with several emerging novel categories and subthemes. The domains were consistent across the postgraduate year of trainee. No differences in themes were noted, other than those explicitly stated, between the postgraduate years.

DISCUSSION

Clinical supervisors must hold the quality of care constant while balancing the amount of supervision and autonomy provided to learners in procedural tasks and clinical decision making. We found that the development of trust is multifactorial and highly contextual. It occurs under the broad constructs of task, supervisor, trainee, and environmental factors, and is well described in prior work. We also demonstrate that often what determines these broader factors is highly subjective, frequently independent of objective measures of trainee performance. Many decisions are based on personal characteristics, such as the perception of honesty, disposition, perceived confidence or perceived overconfidence of the trainee, prior experience, and expressed future field of specialty.

Our findings are consistent with prior research, but go further in describing and demonstrating the existence and innovative use of factors, other than clinical knowledge and skill, in the formation of a multidimensional construct of trust. Kennedy et al. identified 4 dimensions of trust knowledge and skill, discernment, conscientiousness, and truthfulness[15]and demonstrated that supervising physicians rely on specific processes to assess trainee trustworthiness, specifically the use of double checks and language cues. This is consistent with our results, which demonstrate that many attending physicians independently verify information, such as laboratory findings, to inform their perceptions of trainee honesty, attention to detail, and ability to follow orders reliably. Furthermore, our subthemes of communication and the demonstration of logical clinical reasoning correspond to Kennedy's use of language cues.[15] We found that language cues are used as markers of trustworthiness, particularly early on in the rotation, as a litmus test to gauge the trainee's integrity and ability to assess and treat patients unsupervised.

To date, much has been written about the importance of direct observation in the evaluation of trainees.[16, 17, 18, 19] Our results demonstrate that supervising clinicians use a multifactorial, highly nuanced, and subjective process despite validated performance‐based assessment methods, such as the objective structured clinical exam or mini‐clinical evaluation exercise, to assess competence and grant entrustement.[3] Several factors utilized to determine trustworthiness in addition to direct observation are subjective in nature, specifically the trainee's prior experience and expressed career choice.

It is encouraging that attending physicians make use of direct observations to inform decisions of entrustment, albeit in an informal and unstructured way. They also seem to take into account the context and setting in which the observation occurs, and consider both the environmental factors as well as factors that relate to the task itself.[20] For example, attendings and residents reported that team dynamics played a large role in influencing trust decisions. We also found that attending physicians rely on indirect observation and will inquire among their colleagues and other senior residents to gain information about their trainees abilities and integrity. Evaluation tools that facilitate sharing of trainees' level of preparedness, prior feedback, and experience could facilitate the determination of readiness to complete EPAs as well as the reporting of achieved milestones in accordance with the ACGME NAS.

Sharing knowledge about trainees among attendings is common and of increasing importance in the context of attending physicians' shortened exposure to trainees due to the residency work‐hour restrictions and growing productivity pressures. In our study, attending physicians described work‐hour restrictions as detrimental to trainee trustworthiness, either in the context of decreased accountability for patient care or as intrinsic to the nature of forced absences that kept trainees from fully participating in daily ward activities and knowing their patients. Attending physicians felt that trainees did not know their patients well enough to be able to make independent decisions about care. The increased transition to a shift‐based structure of inpatient medicine may result in increasingly less time for direct observation and make it more difficult for attendings to justify their decisions about engendering trust. In addition, the increased fragmentation that is noted in training secondary to the work‐hour regulations may in fact have consequences on the development of clinical skill and decision making, such that increased attention to the need for supervision and longer lead to entrustment may be needed in certain circumstances. Attendings need guidance on how to improve their ability to observe trainees in the context of the new work environment, and how to role model decision making more effectively in the compressed time exposure to housestaff.

Our study has several limitations. The organizational structure and culture of our institution are unique to 1 academic setting. This may undermine our ability to generalize these research findings and analysis to the population at large.[21] In addition, recall bias may have played into the interpretation of the interview content given the timing with which they were performed after the conclusion of the rotation. The study interviews took place in 2006, and it is reasonable to believe that some perceptions concerning duty‐hour restrictions and competency‐based graduate medical education have changed. However, from our ongoing research over the past 5 years[4] and our personal experience with entrustment factors, we believe that the participants' perceptions of trust and competency are valid and have largely remained unchanged, given the similarity in findings to the accepted ten Cate framework. In addition, this work was done following the first iteration of the work‐hour regulations but prior to the implementation of explicit supervisory levels, so it may indeed represent a truer state of the supervisory relationship before external regulations were applied. Finally, this work represents an internal medicine residency training program and may not be generalizable to other specialties that posses different cultural factors that impact the decision for entrustment. However, the congruence of our data with that of the original work of ten Cate, which was done in gynecology,[6] and that of Sterkenberg et al. in anesthesiology,[4] supports our key factors being ubiquitous to all training programs.

In conclusion, we provide new insights into subjective factors that inform the perceptions of trust and entrustment decisions by supervising physicians, specifically subjective trainee characteristics, team dynamics, and informal observation. There was agreement among attendings about which elements of competence are considered most important in their entrustment decisions related to trainee, supervisor, task, and environmental factors. Rather than undervaluing the use of personal factors in the determination of trust, we believe that acknowledgement and appreciation of these factors may be important to give supervisors more confidence and better tools to assess resident physicians, and to understand how their personality traits relate to and impact their professional competence. Our findings are relevant for the development of assessment instruments to evaluate whether medical graduates are ready for safe practice without supervision.

Determining when residents are independently prepared to perform clinical care tasks safely is not easy or understood. Educators have struggled to identify robust ways to evaluate trainees and their preparedness to treat patients while unsupervised. Trust allows the trainee to experience increasing levels of participation and responsibility in the workplace in a way that builds competence for future practice. The breadth of knowledge and skills required to become a competent and safe physician, coupled with the busy workload confound this challenge. Notably, a technically proficient trainee may not have the clinical judgment to treat patients without supervision.

The Accreditation Council of Graduate Medical Education (ACGME) has previously outlined 6 core competencies for residency training: patient care, medical knowledge, practice‐based learning and improvement, interpersonal and communication skills, professionalism, and systems‐based practice.[1] A systematic literature review suggests that traditional trainee evaluation tools are difficult to use and unreliable in measuring the competencies independently from one another, whereas certain competencies are consistently difficult to quantify in a reliable and valid way.[2] The evaluation of trainees' clinical performance despite efforts to create objective tools remain strongly influenced by subjective measures and continues to be highly variable among different evaluators.[3] Objectively measuring resident autonomy and readiness to supervise junior colleagues remains imprecise.[4]

The ACGME's Next Accreditation System (NAS) incorporates educational milestones as part of the reporting of resident training outcomes.[5] The milestones allow for the translation of the core competencies into integrative and observable abilities. Furthermore, the milestone categories are stratified into tiers to allow progress to be measured longitudinally and by task complexity using a novel assessment strategy.

The development of trust between supervisors and trainees is a critical step in decisions to allow increased responsibility and the provision of autonomous decision making, which is an important aspect of physician training. Identifying the factors that influence the supervisors' evaluation of resident competency and capability is at the crux of trainee maturation as well as patient safety.[4] Trust, defined as believability and discernment by attendings of resident physicians, plays a large role in attending evaluations of residents during their clinical rotations.[3] Trust impacts the decisions of successful performance of entrustable professional activities (EPAs), or those tasks that require mastery prior to completion of training milestones.[6] A study of entrustment decisions made by attending anesthesiologists identified the factors that contribute to the amount of autonomy given to residents, such as trainee trustworthiness, medical knowledge, and level of training.[4] The aim of our study, building on this study, was 2‐fold: (1) use deductive qualitative analysis to apply this framework to existing resident and attending data, and (2) define the categories within this framework and describe how internal medicine attending and resident physician perceptions of trust can impact clinical decision making and patient care.

METHODS

We are reporting on a secondary data analysis of interview transcripts from a study conducted on the inpatient general medicine service at the University of Chicago, an academic tertiary care medical center. The methods for data collection and full consent have been outlined previously.[7, 8, 9] The institutional review board of the University of Chicago approved this study.

Briefly, between January 2006 and November 2006, all eligible internal medicine resident physicians, postgraduate year (PGY)‐2 or PGY‐3, and attending physicians, either generalists or hospitalists, were privately interviewed within 1 week of their final call night on the inpatient general medicine rotation to assess decision making and clinical supervision during the rotation. All interviews were conducted by 1 investigator (J.F.), and discussions were audio taped and transcribed for analysis. Interviews were conducted at the conclusion of the rotation to prevent any influence on resident and attending behavior during the rotation.

The critical incident technique, a procedure used for collecting direct observations of human behavior that have critical significance on the decision‐making process, was used to solicit examples of ineffective supervision, inquiring about 2 to 3 important clinical decisions made on the most recent call night, with probes to identify issues of trust, autonomy, and decision making.[10] A critical incident can be described as one that makes a significant contribution, either positively or negatively, on the process.

Appreciative inquiry, a technique that aims to uncover the best things about the clinical encounter being explored, was used to solicit examples of effective supervision. Probes are used to identify factors, either personal or situational, that influenced the withholding or provision of resident autonomy during periods of clinical care delivery.[11]

All identifiable information was removed from the interview transcripts to protect participant and patient confidentiality. Deductive qualitative analysis was performed using the conceptual EPA framework, which describes several factors that influence the attending physicians' decisions to deem a resident trustworthy to independently fulfill a specific clinical task.[4] These factors include (1) the nature of the task, (2) the qualities of the supervisor, (3) the qualities of the trainee and the quality of the relationship between the supervisor and the trainee, and (4) the circumstances surrounding the clinical task.

The deidentified, anonymous transcripts were reviewed by 2 investigators (K.J.C., J.M.F.) and analyzed using the constant comparative methods to deductively map the content to the existing framework and generate novel sub themes.[12, 13, 14] Novel categories within each of the domains were inductively generated. Two reviewers (K.J.C., J.M.F.) independently applied the themes to a randomly selected 10% portion of the interview transcripts to assess the inter‐rater reliability. The inter‐rater agreement was assessed using the generalized kappa statistic. The discrepancies between reviewers regarding assignment of codes were resolved via discussion and third party adjudication until consensus was achieved on thematic structure. The codes were then applied to the entire dataset.

RESULTS

Between January 2006 and November 2006, 46 of 50 (88%) attending physicians and 44 of 50 (92%) resident physicians were interviewed following the conclusion of their general medicine inpatient rotation. Of attending physicians, 55% were male, 45% were female, and 38% were academic faculty hospitalists. Of the residents who completed interviews, 47% were male, 53% were female, 52% were PGY‐2, and 45% were PGY‐3.

A total of 535 mentions of trust were abstracted from the transcripts. The 4 major domains that influence trusttrainee factors (Table 1), supervisor factors (Table 2), task factors (Table 3), and systems factors (Table 4)were deductively coded with several emerging novel categories and subthemes. The domains were consistent across the postgraduate year of trainee. No differences in themes were noted, other than those explicitly stated, between the postgraduate years.

DISCUSSION

Clinical supervisors must hold the quality of care constant while balancing the amount of supervision and autonomy provided to learners in procedural tasks and clinical decision making. We found that the development of trust is multifactorial and highly contextual. It occurs under the broad constructs of task, supervisor, trainee, and environmental factors, and is well described in prior work. We also demonstrate that often what determines these broader factors is highly subjective, frequently independent of objective measures of trainee performance. Many decisions are based on personal characteristics, such as the perception of honesty, disposition, perceived confidence or perceived overconfidence of the trainee, prior experience, and expressed future field of specialty.

Our findings are consistent with prior research, but go further in describing and demonstrating the existence and innovative use of factors, other than clinical knowledge and skill, in the formation of a multidimensional construct of trust. Kennedy et al. identified 4 dimensions of trust knowledge and skill, discernment, conscientiousness, and truthfulness[15]and demonstrated that supervising physicians rely on specific processes to assess trainee trustworthiness, specifically the use of double checks and language cues. This is consistent with our results, which demonstrate that many attending physicians independently verify information, such as laboratory findings, to inform their perceptions of trainee honesty, attention to detail, and ability to follow orders reliably. Furthermore, our subthemes of communication and the demonstration of logical clinical reasoning correspond to Kennedy's use of language cues.[15] We found that language cues are used as markers of trustworthiness, particularly early on in the rotation, as a litmus test to gauge the trainee's integrity and ability to assess and treat patients unsupervised.

To date, much has been written about the importance of direct observation in the evaluation of trainees.[16, 17, 18, 19] Our results demonstrate that supervising clinicians use a multifactorial, highly nuanced, and subjective process despite validated performance‐based assessment methods, such as the objective structured clinical exam or mini‐clinical evaluation exercise, to assess competence and grant entrustement.[3] Several factors utilized to determine trustworthiness in addition to direct observation are subjective in nature, specifically the trainee's prior experience and expressed career choice.

It is encouraging that attending physicians make use of direct observations to inform decisions of entrustment, albeit in an informal and unstructured way. They also seem to take into account the context and setting in which the observation occurs, and consider both the environmental factors as well as factors that relate to the task itself.[20] For example, attendings and residents reported that team dynamics played a large role in influencing trust decisions. We also found that attending physicians rely on indirect observation and will inquire among their colleagues and other senior residents to gain information about their trainees abilities and integrity. Evaluation tools that facilitate sharing of trainees' level of preparedness, prior feedback, and experience could facilitate the determination of readiness to complete EPAs as well as the reporting of achieved milestones in accordance with the ACGME NAS.

Sharing knowledge about trainees among attendings is common and of increasing importance in the context of attending physicians' shortened exposure to trainees due to the residency work‐hour restrictions and growing productivity pressures. In our study, attending physicians described work‐hour restrictions as detrimental to trainee trustworthiness, either in the context of decreased accountability for patient care or as intrinsic to the nature of forced absences that kept trainees from fully participating in daily ward activities and knowing their patients. Attending physicians felt that trainees did not know their patients well enough to be able to make independent decisions about care. The increased transition to a shift‐based structure of inpatient medicine may result in increasingly less time for direct observation and make it more difficult for attendings to justify their decisions about engendering trust. In addition, the increased fragmentation that is noted in training secondary to the work‐hour regulations may in fact have consequences on the development of clinical skill and decision making, such that increased attention to the need for supervision and longer lead to entrustment may be needed in certain circumstances. Attendings need guidance on how to improve their ability to observe trainees in the context of the new work environment, and how to role model decision making more effectively in the compressed time exposure to housestaff.

Our study has several limitations. The organizational structure and culture of our institution are unique to 1 academic setting. This may undermine our ability to generalize these research findings and analysis to the population at large.[21] In addition, recall bias may have played into the interpretation of the interview content given the timing with which they were performed after the conclusion of the rotation. The study interviews took place in 2006, and it is reasonable to believe that some perceptions concerning duty‐hour restrictions and competency‐based graduate medical education have changed. However, from our ongoing research over the past 5 years[4] and our personal experience with entrustment factors, we believe that the participants' perceptions of trust and competency are valid and have largely remained unchanged, given the similarity in findings to the accepted ten Cate framework. In addition, this work was done following the first iteration of the work‐hour regulations but prior to the implementation of explicit supervisory levels, so it may indeed represent a truer state of the supervisory relationship before external regulations were applied. Finally, this work represents an internal medicine residency training program and may not be generalizable to other specialties that posses different cultural factors that impact the decision for entrustment. However, the congruence of our data with that of the original work of ten Cate, which was done in gynecology,[6] and that of Sterkenberg et al. in anesthesiology,[4] supports our key factors being ubiquitous to all training programs.

In conclusion, we provide new insights into subjective factors that inform the perceptions of trust and entrustment decisions by supervising physicians, specifically subjective trainee characteristics, team dynamics, and informal observation. There was agreement among attendings about which elements of competence are considered most important in their entrustment decisions related to trainee, supervisor, task, and environmental factors. Rather than undervaluing the use of personal factors in the determination of trust, we believe that acknowledgement and appreciation of these factors may be important to give supervisors more confidence and better tools to assess resident physicians, and to understand how their personality traits relate to and impact their professional competence. Our findings are relevant for the development of assessment instruments to evaluate whether medical graduates are ready for safe practice without supervision.

Determining when residents are independently prepared to perform clinical care tasks safely is not easy or understood. Educators have struggled to identify robust ways to evaluate trainees and their preparedness to treat patients while unsupervised. Trust allows the trainee to experience increasing levels of participation and responsibility in the workplace in a way that builds competence for future practice. The breadth of knowledge and skills required to become a competent and safe physician, coupled with the busy workload confound this challenge. Notably, a technically proficient trainee may not have the clinical judgment to treat patients without supervision.

The Accreditation Council of Graduate Medical Education (ACGME) has previously outlined 6 core competencies for residency training: patient care, medical knowledge, practice‐based learning and improvement, interpersonal and communication skills, professionalism, and systems‐based practice.[1] A systematic literature review suggests that traditional trainee evaluation tools are difficult to use and unreliable in measuring the competencies independently from one another, whereas certain competencies are consistently difficult to quantify in a reliable and valid way.[2] The evaluation of trainees' clinical performance despite efforts to create objective tools remain strongly influenced by subjective measures and continues to be highly variable among different evaluators.[3] Objectively measuring resident autonomy and readiness to supervise junior colleagues remains imprecise.[4]

The ACGME's Next Accreditation System (NAS) incorporates educational milestones as part of the reporting of resident training outcomes.[5] The milestones allow for the translation of the core competencies into integrative and observable abilities. Furthermore, the milestone categories are stratified into tiers to allow progress to be measured longitudinally and by task complexity using a novel assessment strategy.

The development of trust between supervisors and trainees is a critical step in decisions to allow increased responsibility and the provision of autonomous decision making, which is an important aspect of physician training. Identifying the factors that influence the supervisors' evaluation of resident competency and capability is at the crux of trainee maturation as well as patient safety.[4] Trust, defined as believability and discernment by attendings of resident physicians, plays a large role in attending evaluations of residents during their clinical rotations.[3] Trust impacts the decisions of successful performance of entrustable professional activities (EPAs), or those tasks that require mastery prior to completion of training milestones.[6] A study of entrustment decisions made by attending anesthesiologists identified the factors that contribute to the amount of autonomy given to residents, such as trainee trustworthiness, medical knowledge, and level of training.[4] The aim of our study, building on this study, was 2‐fold: (1) use deductive qualitative analysis to apply this framework to existing resident and attending data, and (2) define the categories within this framework and describe how internal medicine attending and resident physician perceptions of trust can impact clinical decision making and patient care.

METHODS

We are reporting on a secondary data analysis of interview transcripts from a study conducted on the inpatient general medicine service at the University of Chicago, an academic tertiary care medical center. The methods for data collection and full consent have been outlined previously.[7, 8, 9] The institutional review board of the University of Chicago approved this study.

Briefly, between January 2006 and November 2006, all eligible internal medicine resident physicians, postgraduate year (PGY)‐2 or PGY‐3, and attending physicians, either generalists or hospitalists, were privately interviewed within 1 week of their final call night on the inpatient general medicine rotation to assess decision making and clinical supervision during the rotation. All interviews were conducted by 1 investigator (J.F.), and discussions were audio taped and transcribed for analysis. Interviews were conducted at the conclusion of the rotation to prevent any influence on resident and attending behavior during the rotation.

The critical incident technique, a procedure used for collecting direct observations of human behavior that have critical significance on the decision‐making process, was used to solicit examples of ineffective supervision, inquiring about 2 to 3 important clinical decisions made on the most recent call night, with probes to identify issues of trust, autonomy, and decision making.[10] A critical incident can be described as one that makes a significant contribution, either positively or negatively, on the process.

Appreciative inquiry, a technique that aims to uncover the best things about the clinical encounter being explored, was used to solicit examples of effective supervision. Probes are used to identify factors, either personal or situational, that influenced the withholding or provision of resident autonomy during periods of clinical care delivery.[11]

All identifiable information was removed from the interview transcripts to protect participant and patient confidentiality. Deductive qualitative analysis was performed using the conceptual EPA framework, which describes several factors that influence the attending physicians' decisions to deem a resident trustworthy to independently fulfill a specific clinical task.[4] These factors include (1) the nature of the task, (2) the qualities of the supervisor, (3) the qualities of the trainee and the quality of the relationship between the supervisor and the trainee, and (4) the circumstances surrounding the clinical task.

The deidentified, anonymous transcripts were reviewed by 2 investigators (K.J.C., J.M.F.) and analyzed using the constant comparative methods to deductively map the content to the existing framework and generate novel sub themes.[12, 13, 14] Novel categories within each of the domains were inductively generated. Two reviewers (K.J.C., J.M.F.) independently applied the themes to a randomly selected 10% portion of the interview transcripts to assess the inter‐rater reliability. The inter‐rater agreement was assessed using the generalized kappa statistic. The discrepancies between reviewers regarding assignment of codes were resolved via discussion and third party adjudication until consensus was achieved on thematic structure. The codes were then applied to the entire dataset.

RESULTS

Between January 2006 and November 2006, 46 of 50 (88%) attending physicians and 44 of 50 (92%) resident physicians were interviewed following the conclusion of their general medicine inpatient rotation. Of attending physicians, 55% were male, 45% were female, and 38% were academic faculty hospitalists. Of the residents who completed interviews, 47% were male, 53% were female, 52% were PGY‐2, and 45% were PGY‐3.

A total of 535 mentions of trust were abstracted from the transcripts. The 4 major domains that influence trusttrainee factors (Table 1), supervisor factors (Table 2), task factors (Table 3), and systems factors (Table 4)were deductively coded with several emerging novel categories and subthemes. The domains were consistent across the postgraduate year of trainee. No differences in themes were noted, other than those explicitly stated, between the postgraduate years.

DISCUSSION

Clinical supervisors must hold the quality of care constant while balancing the amount of supervision and autonomy provided to learners in procedural tasks and clinical decision making. We found that the development of trust is multifactorial and highly contextual. It occurs under the broad constructs of task, supervisor, trainee, and environmental factors, and is well described in prior work. We also demonstrate that often what determines these broader factors is highly subjective, frequently independent of objective measures of trainee performance. Many decisions are based on personal characteristics, such as the perception of honesty, disposition, perceived confidence or perceived overconfidence of the trainee, prior experience, and expressed future field of specialty.

Our findings are consistent with prior research, but go further in describing and demonstrating the existence and innovative use of factors, other than clinical knowledge and skill, in the formation of a multidimensional construct of trust. Kennedy et al. identified 4 dimensions of trust knowledge and skill, discernment, conscientiousness, and truthfulness[15]and demonstrated that supervising physicians rely on specific processes to assess trainee trustworthiness, specifically the use of double checks and language cues. This is consistent with our results, which demonstrate that many attending physicians independently verify information, such as laboratory findings, to inform their perceptions of trainee honesty, attention to detail, and ability to follow orders reliably. Furthermore, our subthemes of communication and the demonstration of logical clinical reasoning correspond to Kennedy's use of language cues.[15] We found that language cues are used as markers of trustworthiness, particularly early on in the rotation, as a litmus test to gauge the trainee's integrity and ability to assess and treat patients unsupervised.

To date, much has been written about the importance of direct observation in the evaluation of trainees.[16, 17, 18, 19] Our results demonstrate that supervising clinicians use a multifactorial, highly nuanced, and subjective process despite validated performance‐based assessment methods, such as the objective structured clinical exam or mini‐clinical evaluation exercise, to assess competence and grant entrustement.[3] Several factors utilized to determine trustworthiness in addition to direct observation are subjective in nature, specifically the trainee's prior experience and expressed career choice.

It is encouraging that attending physicians make use of direct observations to inform decisions of entrustment, albeit in an informal and unstructured way. They also seem to take into account the context and setting in which the observation occurs, and consider both the environmental factors as well as factors that relate to the task itself.[20] For example, attendings and residents reported that team dynamics played a large role in influencing trust decisions. We also found that attending physicians rely on indirect observation and will inquire among their colleagues and other senior residents to gain information about their trainees abilities and integrity. Evaluation tools that facilitate sharing of trainees' level of preparedness, prior feedback, and experience could facilitate the determination of readiness to complete EPAs as well as the reporting of achieved milestones in accordance with the ACGME NAS.

Sharing knowledge about trainees among attendings is common and of increasing importance in the context of attending physicians' shortened exposure to trainees due to the residency work‐hour restrictions and growing productivity pressures. In our study, attending physicians described work‐hour restrictions as detrimental to trainee trustworthiness, either in the context of decreased accountability for patient care or as intrinsic to the nature of forced absences that kept trainees from fully participating in daily ward activities and knowing their patients. Attending physicians felt that trainees did not know their patients well enough to be able to make independent decisions about care. The increased transition to a shift‐based structure of inpatient medicine may result in increasingly less time for direct observation and make it more difficult for attendings to justify their decisions about engendering trust. In addition, the increased fragmentation that is noted in training secondary to the work‐hour regulations may in fact have consequences on the development of clinical skill and decision making, such that increased attention to the need for supervision and longer lead to entrustment may be needed in certain circumstances. Attendings need guidance on how to improve their ability to observe trainees in the context of the new work environment, and how to role model decision making more effectively in the compressed time exposure to housestaff.

Our study has several limitations. The organizational structure and culture of our institution are unique to 1 academic setting. This may undermine our ability to generalize these research findings and analysis to the population at large.[21] In addition, recall bias may have played into the interpretation of the interview content given the timing with which they were performed after the conclusion of the rotation. The study interviews took place in 2006, and it is reasonable to believe that some perceptions concerning duty‐hour restrictions and competency‐based graduate medical education have changed. However, from our ongoing research over the past 5 years[4] and our personal experience with entrustment factors, we believe that the participants' perceptions of trust and competency are valid and have largely remained unchanged, given the similarity in findings to the accepted ten Cate framework. In addition, this work was done following the first iteration of the work‐hour regulations but prior to the implementation of explicit supervisory levels, so it may indeed represent a truer state of the supervisory relationship before external regulations were applied. Finally, this work represents an internal medicine residency training program and may not be generalizable to other specialties that posses different cultural factors that impact the decision for entrustment. However, the congruence of our data with that of the original work of ten Cate, which was done in gynecology,[6] and that of Sterkenberg et al. in anesthesiology,[4] supports our key factors being ubiquitous to all training programs.

In conclusion, we provide new insights into subjective factors that inform the perceptions of trust and entrustment decisions by supervising physicians, specifically subjective trainee characteristics, team dynamics, and informal observation. There was agreement among attendings about which elements of competence are considered most important in their entrustment decisions related to trainee, supervisor, task, and environmental factors. Rather than undervaluing the use of personal factors in the determination of trust, we believe that acknowledgement and appreciation of these factors may be important to give supervisors more confidence and better tools to assess resident physicians, and to understand how their personality traits relate to and impact their professional competence. Our findings are relevant for the development of assessment instruments to evaluate whether medical graduates are ready for safe practice without supervision.

Unprofessional behavior in the inpatient setting has the potential to impact care delivery and the quality of trainee's educational experience. These behaviors, from disparaging colleagues to blocking admissions, can negatively impact the learning environment. The learning environment or conditions created by the patient care team's actions play a critical role in the development of trainees.[1, 2] The rising presence of hospitalists in the inpatient setting raises the question of how their actions impact the learning environment. Professional behavior has been defined as a core competency for hospitalists by the Society of Hospital Medicine.[3] Professional behavior of all team members, from faculty to trainee, can impact the learning environment and patient safety.[4, 5] However, few educational materials exist to train faculty and housestaff on recognizing and ameliorating unprofessional behaviors.

A prior assessment regarding hospitalists' lapses in professionalism identified scenarios that demonstrated increased participation by hospitalists at 3 institutions.[6] Participants reported observation or participation in specific unprofessional behaviors and rated their perception of these behaviors. Additional work within those residency environments demonstrated that residents' perceptions of and participation in these behaviors increased throughout training, with environmental characteristics, specifically faculty behavior, influencing trainee professional development and acclimation of these behaviors.[7, 8]

Although overall participation in egregious behavior was low, resident participation in 3 categories of unprofessional behavior increased during internship. Those scenarios included disparaging the emergency room or primary care physician for missed findings or management decisions, blocking or not taking admissions appropriate for the service in question, and misrepresenting a test as urgent to expedite obtaining the test. We developed our intervention focused on these areas to address professionalism lapses that occur during internship. Our earlier work showed faculty role models influenced trainee behavior. For this reason, we provided education to both residents and hospitalists to maximize the impact of the intervention.

We present here a novel, interactive, video‐based workshop curriculum for faculty and trainees that aims to illustrate unprofessional behaviors and outlines the role faculty may play in promoting such behaviors. In addition, we review the result of postworkshop evaluation on intent to change behavior and satisfaction.

METHODS

A grant from the American Board of Internal Medicine Foundation supported this project. The working group that resulted, the Chicago Professional Practice Project and Outcomes, included faculty representation from 3 Chicago‐area hospitals: the University of Chicago, Northwestern University, and NorthShore University HealthSystem. Academic hospitalists at these sites were invited to participate. Each site also has an internal medicine residency program in which hospitalists were expected to attend the teaching service. Given this, resident trainees at all participating sites, and 1 community teaching affiliate program (Mercy Hospital and Medical Center) where academic hospitalists at the University of Chicago rotate, were recruited for participation. Faculty champions were identified for each site, and 1 internal and external faculty representative from the working group served to debrief and facilitate. Trainee workshops were administered by 1 internal and external collaborator, and for the community site, 2 external faculty members. Workshops were held during established educational conference times, and lunch was provided.

Scripts highlighting each of the behaviors identified in the prior survey were developed and peer reviewed for clarity and face validity across the 3 sites. Medical student and resident actors were trained utilizing the finalized scripts, and a performance artist affiliated with the Screen Actors Guild assisted in their preparation for filming. All videos were filmed at the University of Chicago Pritzker School of Medicine Clinical Performance Center. The final videos ranged in length from 4 to 7 minutes and included title, cast, and funding source. As an example, 1 video highlighted the unprofessional behavior of misrepresenting a test as urgent to prioritize one's patient in the queue. This video included a resident, intern, and attending on inpatient rounds during which the resident encouraged the intern to misrepresent the patient's status to expedite obtaining the study and facilitate the patient's discharge. The resident stressed that he would be in the clinic and had many patients to see, highlighting the impact of workload on unprofessional behavior, and aggressively persuaded the intern to sell her test to have it performed the same day. When this occurred, the attending applauded the intern for her strong work.

A moderator guide and debriefing tools were developed to facilitate discussion. The duration of each of the workshops was approximately 60 minutes. After welcoming remarks, participants were provided tools to utilize during the viewing of each video. These checklists noted the roles of those depicted in the video, asked to identify positive or negative behaviors displayed, and included questions regarding how behaviors could be detrimental and how the situation could have been prevented. After viewing the videos, participants divided into small groups to discuss the individual exhibiting the unprofessional behavior, their perceived motivation for said behavior, and its impact on the team culture and patient care. Following a small‐group discussion, large‐group debriefing was performed, addressing the barriers and facilitators to professional behavior. Two videos were shown at each workshop, and participants completed a postworkshop evaluation. Videos chosen for viewing were based upon preworkshop survey results that highlighted areas of concern at that specific site.

Postworkshop paper‐based evaluations assessed participants' perception of displayed behaviors on a Likert‐type scale (1=unprofessional to 5=professional) utilizing items validated in prior work,[6, 7, 8] their level of agreement regarding the impact of video‐based exercises, and intent to change behavior using a Likert‐type scale (1=strongly disagree to 5=strongly agree). A constructed‐response section for comments regarding their experience was included. Descriptive statistics and Wilcoxon rank sum analyses were performed.

RESULTS

Forty‐four academic hospitalist faculty members (44/83; 53%) and 244 resident trainees (244/356; 68%) participated. When queried regarding their perception of the displayed behaviors in the videos, nearly 100% of faculty and trainees felt disparaging the emergency department or primary care physician for missed findings or clinical decisions was somewhat unprofessional or unprofessional. Ninety percent of hospitalists and 93% of trainees rated celebrating a blocked admission as somewhat unprofessional or unprofessional (Table 1).

The scenarios portrayed were well received, with more than 85% of faculty and trainees agreeing that the behaviors displayed were realistic. Those who perceived videos as very realistic were more likely to report intent to change behavior (93% vs 53%, P=0.01). Nearly two‐thirds of faculty and 67% of housestaff expressed agreement that they intended to change behavior based upon the experience (Table 2).

Qualitative comments in the constructed‐response portion of the evaluation noted the effectiveness of the interactive materials. In addition, the need for focused faculty development was identified by 1 respondent who stated: If unprofessional behavior is the unwritten curriculum, there needs to be an explicit, written curriculum to address it. Finally, the aim of facilitating self‐reflection is echoed in this faculty respondent's comment: Always good to be reminded of our behaviors and the influence they have on others and from this resident physician It helps to re‐evaluate how you talk to people.

CONCLUSIONS

Faculty can be a large determinant of the learning environment and impact trainees' professional development.[9] Hospitalists should be encouraged to embrace faculty role‐modeling of effective professional behaviors, especially given their increased presence in the inpatient learning environment. In addition, resident trainees and their behaviors contribute to the learning environment and influence the further professional development of more junior trainees.[10] Targeting professionalism education toward previously identified and prevalent unprofessional behaviors in the inpatient care of patients may serve to affect the most change among providers who practice in this setting. Individualized assessment of the learning environment may aid in identifying common scenarios that may plague a specific learning culture, allowing for relevant and targeted discussion of factors that promote and perpetuate such behaviors.[11]

Interactive, video‐based modules provided an effective way to promote interactive reflection and robust discussion. This model of experiential learning is an effective form of professional development as it engages the learner and stimulates ongoing incorporation of the topics addressed.[12, 13] Creating a shared concrete experience among targeted learners, using the video‐based scenarios, stimulates reflective observation, and ultimately experimentation, or incorporation into practice.[14]

There are several limitations to our evaluation including that we focused solely on academic hospitalist programs, and our sample size for faculty and residents was small. Also, we only addressed a small, though representative, sample of unprofessional behaviors and have not yet linked intervention to actual behavior change. Finally, the script scenarios that we used in this study were not previously published as they were created specifically for this intervention. Validity evidence for these scenarios include that they were based upon the results of earlier work from our institutions and underwent thorough peer review for content and clarity. Further studies will be required to do this. However, we do believe that these are positive findings for utilizing this type of interactive curriculum for professionalism education to promote self‐reflection and behavior change.

Video‐based professionalism education is a feasible, interactive mechanism to encourage self‐reflection and intent to change behavior among faculty and resident physicians. Future study is underway to conduct longitudinal assessments of the learning environments at the participating institutions to assess culture change, perceptions of behaviors, and sustainability of this type of intervention.

Disclosures: The authors acknowledge funding from the American Board of Internal Medicine. The funders had no role in the design of the study; the collection, analysis, and interpretation of the data; or the decision to approve publication of the finished manuscript. Results from this work have been presented at the Midwest Society of General Internal Medicine Regional Meeting, Chicago, Illinois, September 2011; Midwest Society of Hospital Medicine Regional Meeting, Chicago, Illinois, October 2011, and Society of Hospital Medicine Annual Meeting, San Diego, California, April 2012. The authors declare that they do not have any conflicts of interest to disclose.

Unprofessional behavior in the inpatient setting has the potential to impact care delivery and the quality of trainee's educational experience. These behaviors, from disparaging colleagues to blocking admissions, can negatively impact the learning environment. The learning environment or conditions created by the patient care team's actions play a critical role in the development of trainees.[1, 2] The rising presence of hospitalists in the inpatient setting raises the question of how their actions impact the learning environment. Professional behavior has been defined as a core competency for hospitalists by the Society of Hospital Medicine.[3] Professional behavior of all team members, from faculty to trainee, can impact the learning environment and patient safety.[4, 5] However, few educational materials exist to train faculty and housestaff on recognizing and ameliorating unprofessional behaviors.

A prior assessment regarding hospitalists' lapses in professionalism identified scenarios that demonstrated increased participation by hospitalists at 3 institutions.[6] Participants reported observation or participation in specific unprofessional behaviors and rated their perception of these behaviors. Additional work within those residency environments demonstrated that residents' perceptions of and participation in these behaviors increased throughout training, with environmental characteristics, specifically faculty behavior, influencing trainee professional development and acclimation of these behaviors.[7, 8]

Although overall participation in egregious behavior was low, resident participation in 3 categories of unprofessional behavior increased during internship. Those scenarios included disparaging the emergency room or primary care physician for missed findings or management decisions, blocking or not taking admissions appropriate for the service in question, and misrepresenting a test as urgent to expedite obtaining the test. We developed our intervention focused on these areas to address professionalism lapses that occur during internship. Our earlier work showed faculty role models influenced trainee behavior. For this reason, we provided education to both residents and hospitalists to maximize the impact of the intervention.

We present here a novel, interactive, video‐based workshop curriculum for faculty and trainees that aims to illustrate unprofessional behaviors and outlines the role faculty may play in promoting such behaviors. In addition, we review the result of postworkshop evaluation on intent to change behavior and satisfaction.

METHODS

A grant from the American Board of Internal Medicine Foundation supported this project. The working group that resulted, the Chicago Professional Practice Project and Outcomes, included faculty representation from 3 Chicago‐area hospitals: the University of Chicago, Northwestern University, and NorthShore University HealthSystem. Academic hospitalists at these sites were invited to participate. Each site also has an internal medicine residency program in which hospitalists were expected to attend the teaching service. Given this, resident trainees at all participating sites, and 1 community teaching affiliate program (Mercy Hospital and Medical Center) where academic hospitalists at the University of Chicago rotate, were recruited for participation. Faculty champions were identified for each site, and 1 internal and external faculty representative from the working group served to debrief and facilitate. Trainee workshops were administered by 1 internal and external collaborator, and for the community site, 2 external faculty members. Workshops were held during established educational conference times, and lunch was provided.

Scripts highlighting each of the behaviors identified in the prior survey were developed and peer reviewed for clarity and face validity across the 3 sites. Medical student and resident actors were trained utilizing the finalized scripts, and a performance artist affiliated with the Screen Actors Guild assisted in their preparation for filming. All videos were filmed at the University of Chicago Pritzker School of Medicine Clinical Performance Center. The final videos ranged in length from 4 to 7 minutes and included title, cast, and funding source. As an example, 1 video highlighted the unprofessional behavior of misrepresenting a test as urgent to prioritize one's patient in the queue. This video included a resident, intern, and attending on inpatient rounds during which the resident encouraged the intern to misrepresent the patient's status to expedite obtaining the study and facilitate the patient's discharge. The resident stressed that he would be in the clinic and had many patients to see, highlighting the impact of workload on unprofessional behavior, and aggressively persuaded the intern to sell her test to have it performed the same day. When this occurred, the attending applauded the intern for her strong work.

A moderator guide and debriefing tools were developed to facilitate discussion. The duration of each of the workshops was approximately 60 minutes. After welcoming remarks, participants were provided tools to utilize during the viewing of each video. These checklists noted the roles of those depicted in the video, asked to identify positive or negative behaviors displayed, and included questions regarding how behaviors could be detrimental and how the situation could have been prevented. After viewing the videos, participants divided into small groups to discuss the individual exhibiting the unprofessional behavior, their perceived motivation for said behavior, and its impact on the team culture and patient care. Following a small‐group discussion, large‐group debriefing was performed, addressing the barriers and facilitators to professional behavior. Two videos were shown at each workshop, and participants completed a postworkshop evaluation. Videos chosen for viewing were based upon preworkshop survey results that highlighted areas of concern at that specific site.

Postworkshop paper‐based evaluations assessed participants' perception of displayed behaviors on a Likert‐type scale (1=unprofessional to 5=professional) utilizing items validated in prior work,[6, 7, 8] their level of agreement regarding the impact of video‐based exercises, and intent to change behavior using a Likert‐type scale (1=strongly disagree to 5=strongly agree). A constructed‐response section for comments regarding their experience was included. Descriptive statistics and Wilcoxon rank sum analyses were performed.

RESULTS

Forty‐four academic hospitalist faculty members (44/83; 53%) and 244 resident trainees (244/356; 68%) participated. When queried regarding their perception of the displayed behaviors in the videos, nearly 100% of faculty and trainees felt disparaging the emergency department or primary care physician for missed findings or clinical decisions was somewhat unprofessional or unprofessional. Ninety percent of hospitalists and 93% of trainees rated celebrating a blocked admission as somewhat unprofessional or unprofessional (Table 1).

The scenarios portrayed were well received, with more than 85% of faculty and trainees agreeing that the behaviors displayed were realistic. Those who perceived videos as very realistic were more likely to report intent to change behavior (93% vs 53%, P=0.01). Nearly two‐thirds of faculty and 67% of housestaff expressed agreement that they intended to change behavior based upon the experience (Table 2).

Qualitative comments in the constructed‐response portion of the evaluation noted the effectiveness of the interactive materials. In addition, the need for focused faculty development was identified by 1 respondent who stated: If unprofessional behavior is the unwritten curriculum, there needs to be an explicit, written curriculum to address it. Finally, the aim of facilitating self‐reflection is echoed in this faculty respondent's comment: Always good to be reminded of our behaviors and the influence they have on others and from this resident physician It helps to re‐evaluate how you talk to people.

CONCLUSIONS

Faculty can be a large determinant of the learning environment and impact trainees' professional development.[9] Hospitalists should be encouraged to embrace faculty role‐modeling of effective professional behaviors, especially given their increased presence in the inpatient learning environment. In addition, resident trainees and their behaviors contribute to the learning environment and influence the further professional development of more junior trainees.[10] Targeting professionalism education toward previously identified and prevalent unprofessional behaviors in the inpatient care of patients may serve to affect the most change among providers who practice in this setting. Individualized assessment of the learning environment may aid in identifying common scenarios that may plague a specific learning culture, allowing for relevant and targeted discussion of factors that promote and perpetuate such behaviors.[11]

Interactive, video‐based modules provided an effective way to promote interactive reflection and robust discussion. This model of experiential learning is an effective form of professional development as it engages the learner and stimulates ongoing incorporation of the topics addressed.[12, 13] Creating a shared concrete experience among targeted learners, using the video‐based scenarios, stimulates reflective observation, and ultimately experimentation, or incorporation into practice.[14]

There are several limitations to our evaluation including that we focused solely on academic hospitalist programs, and our sample size for faculty and residents was small. Also, we only addressed a small, though representative, sample of unprofessional behaviors and have not yet linked intervention to actual behavior change. Finally, the script scenarios that we used in this study were not previously published as they were created specifically for this intervention. Validity evidence for these scenarios include that they were based upon the results of earlier work from our institutions and underwent thorough peer review for content and clarity. Further studies will be required to do this. However, we do believe that these are positive findings for utilizing this type of interactive curriculum for professionalism education to promote self‐reflection and behavior change.

Video‐based professionalism education is a feasible, interactive mechanism to encourage self‐reflection and intent to change behavior among faculty and resident physicians. Future study is underway to conduct longitudinal assessments of the learning environments at the participating institutions to assess culture change, perceptions of behaviors, and sustainability of this type of intervention.

Disclosures: The authors acknowledge funding from the American Board of Internal Medicine. The funders had no role in the design of the study; the collection, analysis, and interpretation of the data; or the decision to approve publication of the finished manuscript. Results from this work have been presented at the Midwest Society of General Internal Medicine Regional Meeting, Chicago, Illinois, September 2011; Midwest Society of Hospital Medicine Regional Meeting, Chicago, Illinois, October 2011, and Society of Hospital Medicine Annual Meeting, San Diego, California, April 2012. The authors declare that they do not have any conflicts of interest to disclose.

Unprofessional behavior in the inpatient setting has the potential to impact care delivery and the quality of trainee's educational experience. These behaviors, from disparaging colleagues to blocking admissions, can negatively impact the learning environment. The learning environment or conditions created by the patient care team's actions play a critical role in the development of trainees.[1, 2] The rising presence of hospitalists in the inpatient setting raises the question of how their actions impact the learning environment. Professional behavior has been defined as a core competency for hospitalists by the Society of Hospital Medicine.[3] Professional behavior of all team members, from faculty to trainee, can impact the learning environment and patient safety.[4, 5] However, few educational materials exist to train faculty and housestaff on recognizing and ameliorating unprofessional behaviors.

A prior assessment regarding hospitalists' lapses in professionalism identified scenarios that demonstrated increased participation by hospitalists at 3 institutions.[6] Participants reported observation or participation in specific unprofessional behaviors and rated their perception of these behaviors. Additional work within those residency environments demonstrated that residents' perceptions of and participation in these behaviors increased throughout training, with environmental characteristics, specifically faculty behavior, influencing trainee professional development and acclimation of these behaviors.[7, 8]

Although overall participation in egregious behavior was low, resident participation in 3 categories of unprofessional behavior increased during internship. Those scenarios included disparaging the emergency room or primary care physician for missed findings or management decisions, blocking or not taking admissions appropriate for the service in question, and misrepresenting a test as urgent to expedite obtaining the test. We developed our intervention focused on these areas to address professionalism lapses that occur during internship. Our earlier work showed faculty role models influenced trainee behavior. For this reason, we provided education to both residents and hospitalists to maximize the impact of the intervention.

We present here a novel, interactive, video‐based workshop curriculum for faculty and trainees that aims to illustrate unprofessional behaviors and outlines the role faculty may play in promoting such behaviors. In addition, we review the result of postworkshop evaluation on intent to change behavior and satisfaction.

METHODS

A grant from the American Board of Internal Medicine Foundation supported this project. The working group that resulted, the Chicago Professional Practice Project and Outcomes, included faculty representation from 3 Chicago‐area hospitals: the University of Chicago, Northwestern University, and NorthShore University HealthSystem. Academic hospitalists at these sites were invited to participate. Each site also has an internal medicine residency program in which hospitalists were expected to attend the teaching service. Given this, resident trainees at all participating sites, and 1 community teaching affiliate program (Mercy Hospital and Medical Center) where academic hospitalists at the University of Chicago rotate, were recruited for participation. Faculty champions were identified for each site, and 1 internal and external faculty representative from the working group served to debrief and facilitate. Trainee workshops were administered by 1 internal and external collaborator, and for the community site, 2 external faculty members. Workshops were held during established educational conference times, and lunch was provided.

Scripts highlighting each of the behaviors identified in the prior survey were developed and peer reviewed for clarity and face validity across the 3 sites. Medical student and resident actors were trained utilizing the finalized scripts, and a performance artist affiliated with the Screen Actors Guild assisted in their preparation for filming. All videos were filmed at the University of Chicago Pritzker School of Medicine Clinical Performance Center. The final videos ranged in length from 4 to 7 minutes and included title, cast, and funding source. As an example, 1 video highlighted the unprofessional behavior of misrepresenting a test as urgent to prioritize one's patient in the queue. This video included a resident, intern, and attending on inpatient rounds during which the resident encouraged the intern to misrepresent the patient's status to expedite obtaining the study and facilitate the patient's discharge. The resident stressed that he would be in the clinic and had many patients to see, highlighting the impact of workload on unprofessional behavior, and aggressively persuaded the intern to sell her test to have it performed the same day. When this occurred, the attending applauded the intern for her strong work.

A moderator guide and debriefing tools were developed to facilitate discussion. The duration of each of the workshops was approximately 60 minutes. After welcoming remarks, participants were provided tools to utilize during the viewing of each video. These checklists noted the roles of those depicted in the video, asked to identify positive or negative behaviors displayed, and included questions regarding how behaviors could be detrimental and how the situation could have been prevented. After viewing the videos, participants divided into small groups to discuss the individual exhibiting the unprofessional behavior, their perceived motivation for said behavior, and its impact on the team culture and patient care. Following a small‐group discussion, large‐group debriefing was performed, addressing the barriers and facilitators to professional behavior. Two videos were shown at each workshop, and participants completed a postworkshop evaluation. Videos chosen for viewing were based upon preworkshop survey results that highlighted areas of concern at that specific site.

Postworkshop paper‐based evaluations assessed participants' perception of displayed behaviors on a Likert‐type scale (1=unprofessional to 5=professional) utilizing items validated in prior work,[6, 7, 8] their level of agreement regarding the impact of video‐based exercises, and intent to change behavior using a Likert‐type scale (1=strongly disagree to 5=strongly agree). A constructed‐response section for comments regarding their experience was included. Descriptive statistics and Wilcoxon rank sum analyses were performed.

RESULTS

Forty‐four academic hospitalist faculty members (44/83; 53%) and 244 resident trainees (244/356; 68%) participated. When queried regarding their perception of the displayed behaviors in the videos, nearly 100% of faculty and trainees felt disparaging the emergency department or primary care physician for missed findings or clinical decisions was somewhat unprofessional or unprofessional. Ninety percent of hospitalists and 93% of trainees rated celebrating a blocked admission as somewhat unprofessional or unprofessional (Table 1).

The scenarios portrayed were well received, with more than 85% of faculty and trainees agreeing that the behaviors displayed were realistic. Those who perceived videos as very realistic were more likely to report intent to change behavior (93% vs 53%, P=0.01). Nearly two‐thirds of faculty and 67% of housestaff expressed agreement that they intended to change behavior based upon the experience (Table 2).

Qualitative comments in the constructed‐response portion of the evaluation noted the effectiveness of the interactive materials. In addition, the need for focused faculty development was identified by 1 respondent who stated: If unprofessional behavior is the unwritten curriculum, there needs to be an explicit, written curriculum to address it. Finally, the aim of facilitating self‐reflection is echoed in this faculty respondent's comment: Always good to be reminded of our behaviors and the influence they have on others and from this resident physician It helps to re‐evaluate how you talk to people.

CONCLUSIONS

Faculty can be a large determinant of the learning environment and impact trainees' professional development.[9] Hospitalists should be encouraged to embrace faculty role‐modeling of effective professional behaviors, especially given their increased presence in the inpatient learning environment. In addition, resident trainees and their behaviors contribute to the learning environment and influence the further professional development of more junior trainees.[10] Targeting professionalism education toward previously identified and prevalent unprofessional behaviors in the inpatient care of patients may serve to affect the most change among providers who practice in this setting. Individualized assessment of the learning environment may aid in identifying common scenarios that may plague a specific learning culture, allowing for relevant and targeted discussion of factors that promote and perpetuate such behaviors.[11]

Interactive, video‐based modules provided an effective way to promote interactive reflection and robust discussion. This model of experiential learning is an effective form of professional development as it engages the learner and stimulates ongoing incorporation of the topics addressed.[12, 13] Creating a shared concrete experience among targeted learners, using the video‐based scenarios, stimulates reflective observation, and ultimately experimentation, or incorporation into practice.[14]

There are several limitations to our evaluation including that we focused solely on academic hospitalist programs, and our sample size for faculty and residents was small. Also, we only addressed a small, though representative, sample of unprofessional behaviors and have not yet linked intervention to actual behavior change. Finally, the script scenarios that we used in this study were not previously published as they were created specifically for this intervention. Validity evidence for these scenarios include that they were based upon the results of earlier work from our institutions and underwent thorough peer review for content and clarity. Further studies will be required to do this. However, we do believe that these are positive findings for utilizing this type of interactive curriculum for professionalism education to promote self‐reflection and behavior change.

Video‐based professionalism education is a feasible, interactive mechanism to encourage self‐reflection and intent to change behavior among faculty and resident physicians. Future study is underway to conduct longitudinal assessments of the learning environments at the participating institutions to assess culture change, perceptions of behaviors, and sustainability of this type of intervention.

Disclosures: The authors acknowledge funding from the American Board of Internal Medicine. The funders had no role in the design of the study; the collection, analysis, and interpretation of the data; or the decision to approve publication of the finished manuscript. Results from this work have been presented at the Midwest Society of General Internal Medicine Regional Meeting, Chicago, Illinois, September 2011; Midwest Society of Hospital Medicine Regional Meeting, Chicago, Illinois, October 2011, and Society of Hospital Medicine Annual Meeting, San Diego, California, April 2012. The authors declare that they do not have any conflicts of interest to disclose.

The implementation of resident duty hour restrictions has created a clinical learning environment on the wards quite different from any previous era. The Accreditation Council for Graduate Medical Education issued its first set of regulations limiting consecutive hours worked for residents in 2003, and further restricted hours in 2011.[1] These restrictions have had many implications across several aspects of patient care, education, and clinical training, particularly for hospitalists who spend the majority of their time in this setting and are heavily involved in undergraduate and graduate clinical education in academic medical centers.[2, 3]

As learning environments have been shifting, so has the composition of learners. The Millennial Generation (or Generation Y), defined as those born approximately between 1980 and 2000, represents those young clinicians currently filling the halls of medical schools and ranks of residency and fellowship programs.[4] Interestingly, the current system of restricted work hours is the only system under which the Millennial Generation has ever trained.

As this new generation represents the bulk of current trainees, hospitalist faculty must consider how their teaching styles can be adapted to accommodate these learners. For teaching hospitalists, an approach that considers the learning environment as affected by duty hours, as well as the preferences of Millennial learners, is necessary to educate the next generation of trainees. This article aimed to introduce potential strategies for hospitalists to better align teaching on the wards with the preferences of Millennial learners under the constraints of residency duty hours.

THE NEWEST GENERATION OF LEARNERS

The Millennial Generation has been well described.[4, 5, 6, 7, 8, 9, 10] Broadly speaking, this generation is thought to have been raised by attentive and involved parents, influencing relationships with educators and mentors; they respect authority but do not hesitate to question the relevance of assignments or decisions. Millennials prefer structured learning environments that focus heavily on interaction and experiential learning, and they value design and appearance in how material is presented.[7] Millennials also seek clear expectations and immediate feedback on their performance, and though they have sometimes been criticized for a strong sense of entitlement, they have a strong desire for collaboration and group‐based activity.[5, 6]

One of the most notable and defining characteristics of the Millennial Generation is an affinity for technology and innovation.[7, 8, 9] Web‐based learning tools that are interactive and engaging, such as blogs, podcasts, or streaming videos are familiar and favored methods of learning. Millennials are skilled at finding information and providing answers and data, but may need help with synthesis and application.[5] They take pride in their ability to multitask, but can be prone to doing so inappropriately, particularly with technology that is readily available.[11]

Few studies have explored characteristics of the Millennial Generation specific to medical trainees. One study examined personality characteristics of Millennial medical students compared to Generation X students (those born from 19651980) at a single institution. Millennial students scored higher on warmth, reasoning, emotional stability, rule consciousness, social boldness, sensitivity, apprehension, openness to change, and perfectionism compared to Generation X students. They scored lower on measures for self‐reliance.[12] Additionally, when motives for behavior were studied, Millennial medical students scored higher on needs for affiliation and achievement, and lower on needs for power.[13]

DUTY HOURS: A GENERATION APART

As noted previously, the Millennial Generation is the first to train exclusively in the era of duty hours restrictions. The oldest members of this generation, those born in 1981, were entering medical school at the time of the first duty hours restrictions in 2003, and thus have always been educated, trained, and practiced in an environment in which work hours were an essential part of residency training.

Though duty hours have been an omnipresent part of training for the Millennial Generation, the clinical learning environment that they have known continues to evolve and change. Time for teaching, in particular, has been especially strained by work hour limits, and this has been noted by both attending physicians and trainees with each iteration of work hours limits. Attendings in one study estimated that time spent teaching on general medicine wards was reduced by about 20% following the 2003 limits, and over 40% of residents in a national survey reported that the 2011 limits had worsened the quality of education.[14, 15]

GENERATIONAL STRATEGIES FOR SUCCESS FOR HOSPITALIST TEACHING ATTENDINGS

The time limitations imposed by duty hours restrictions have compelled teaching rounds to become more patient‐care centered and often less learner‐centered, as providing patient care becomes the prime obligation for this limited time period. Millennial learners are accustomed to being the center of attention in educational environments, and changing the focus from education to patient care in the wards setting may be an abrupt transition for some learners.[6] However, hospitalists can help restructure teaching opportunities on the clinical wards by using teaching methods of the highest value to Millennial learners to promote learning under the conditions of duty hours limitations.

An approach using these methods was developed by reviewing recent literature as well as educational innovations that have been presented at scholarly meetings (eg, Sal Khan's presentation at the 2012 Association of American Medical Colleges meeting).[16] The authors discussed potential teaching techniques that were thought to be feasible to implement in the context of the current learning environment, with consideration of learning theories that would be most effective for the target group of learners (eg, adult learning theory).[17] A mnemonic was created to consolidate strategies thought to best represent these techniques. FUTURE is a group of teaching strategies that can be used by hospitalists to improve teaching rounds by Flipping the Wards, Using Documentation to Teach, Technology‐Enabled Teaching, Using Guerilla Teaching Tactics, Rainy Day Teaching, and Embedding Teaching Moments into Rounds.

ACCEPTING THE CHALLENGE

The landscape of clinical teaching has shifted considerably in recent years, in both the makeup of learners for whom educators are responsible for teaching as well as the challenges in teaching under the duty hours restrictions. Though rounds are more focused on patient care than in the past, it is possible to work within the current structure to promote successful learning with an approach that considers the preferences of today's learners.

A hospitalist's natural habitat, the busy inpatient wards, is a clinical learning environment with rich potential for innovation and excellence in teaching. The challenges in practicing hospital medicine closely parallel the challenges in teaching under the constraints of duty hours restrictions; both require a creative approach to problem solving and an affinity for teamwork. The hospitalist community is well suited to not only meet these challenges but become leaders in embracing how to teach effectively on today's wards. Maximizing interaction, embracing technology, and encouraging group‐based learning may represent the keys to a successful approach to teaching the Millennial Generation in a post‐duty hours world.