Julie K. Johnson, MSPH, PhD

Care of hospitalized patients requires effective teamwork within groups composed of physicians (eg, residents, hospitalists, specialists), advanced practice providers, nurses, patient‐care technicians, pharmacists, social workers, and therapists. Sadly, hospital‐based team members often fail to communicate. For example, 2 studies found that nurses and physicians communicated with one another on only 50% to 60% of their patients' hospital days, resulting in a lack of a mutual understanding of the plan of care.[1, 2]

Failure to communicate effectively may be because the hospital setting poses important challenges to teamwork, including the use of large teams with membership that changes frequently because of the need to provide care around the clock. Furthermore, individual team members often have high workloads, care for multiple patients simultaneously, and are seldom in the same place at the same time.

Interdisciplinary rounds (IDR) are a microsystem‐level solution with the goal to share information, achieve mutual understanding, and collaboratively revise the plan of care within care teams. Though common, IDR look very different across hospitals, making studies that evaluate novel strategies to improve IDR and measure their impact of great interest to hospital medicine.

In this issue of the Journal of Hospital Medicine, Bhamidipati and colleagues present a systematic review of published studies evaluating the effect of IDR on patient outcomes.[3] The systematic review included 22 studies, including 12 experimental/quasiexperimental and 10 observational studies. Overall, 13 studies were of low to medium quality, and 9 were high quality. Importantly, relatively few studies reported the degree to which IDR were implemented as planned. The investigators found evidence that IDR had a positive effect on length of stay (LOS) and staff satisfaction, but little evidence to support an effect on patient safety or satisfaction. Furthermore, the investigators found significant variability in IDR design and team composition. Some of this variation is to be expected, as IDR, like other interventions to improve quality and safety of patient care in complex settings, should be implemented with an expectation that the team may need to make adaptations based on local contextual factors such as workload (eg, daily census), environment (eg, open vs closed intensive care unit), local politics (eg, uniquely strong support for/against the intervention), and prior experience (eg, prior failed, similar interventions).[4, 5] Moreover, objectives for IDR may differ across settings. Some hospitals may have room (and a need) to improve LOS, whereas others may prioritize improving patient safety or patient experience metrics.

Bhamidipati and colleagues explain that their review did not reveal a causal pathway between IDR design and outcomes. We believe this lack of association is because most of the included studies did not propose a causal pathway between the IDR components implemented and the outcomes assessed. That is, few studies referred to conceptual models that explain how components of the IDR intervention might influence downstream patient outcomes.

IDR have the potential to influence a number of patient outcomes, including those reflecting efficiency (eg, length of stay), patient safety (eg, adverse events), and patient centeredness (eg, patient satisfaction). However, these outcomes are influenced by many factors, including patient characteristics and other efforts to improve care. As explained by the investigators, the results of many of the included studies may have been confounded due to relatively weak study designs and statistical analyses. Importantly, few of the studies included in this review report the more proximal measure of teamwork. If we hypothesize that IDR improve patient outcomes, they do so by improving teamwork. After all, the purpose of IDR is to assemble team members so they can communicate about and coordinate care. Measuring teamwork behaviors is difficult, especially on medical services. Measuring teamwork climate, the measurable aspects of team culture, is relatively easy. A recent systematic review of teamwork climate assessments in internal medicine identified the Safety Attitudes Questionnaire and the Team Climate Inventory as having substantial validity evidence and association with improved patient outcomes.[6]

Bhamidipati and colleagues proposed a definition for IDR and taxonomy for IDR design and reporting based on their systematic review. Although very useful, the IDR definition may be too limiting as evidenced by the fact that very few studies would be included in a systematic review using this definition as the inclusion criteria. Their proposed taxonomy should serve as a useful framework for future research efforts and appropriately recommends reporting of site characteristics, components of IDR design, and outcomes.

The systematic review by Bhamidipati et al. must also be interpreted in conjunction with another recently published systematic review by Pannick and colleagues assessing the effect of interdisciplinary team care interventions on general medical wards.[7] Contrary to the findings of the Bhamidipati et al. study, Pannick and colleagues found that most interdisciplinary team care interventions had no effect on LOS, but that half of the studies found an improvement in complications of care. Importantly, Pannick and colleagues included only experimental and quasiexperimental studies in their systematic review (ie, no observational studies).

There is clearly more work to be done in researching IDR and other interventions to improve teamwork in general medical settings. Larger studies are needed to provide sufficient power to detect improvement in outcomes. Future studies need to report the degree to which interventions are implemented as planned and need to use stronger study designs (eg, cluster randomized control or interrupted time series) to avoid the influence of confounders. Qualitative methods should be used to assess the influence of contextual factors on the success of interventions.[4] Most importantly, future studies should be based on conceptual models that explain how components of the intervention influence proximal measures of teamwork and downstream patient outcomes.

In the meantime, what is a hospital leader to do? We believe efforts to improve IDR are warranted, but that IDR program leaders need to first specify their primary objective(s). For example, in some hospitals, there may be little room to further reduce LOS, so another goalreducing preventable readmissions or reducing adverse eventsmight be specified as the key performance indicator. This crucial first step of creating a shared goal informs the design, implementation, and evaluation of IDR. We also believe that geographic localization of physicians to specific units is foundational to improving IDR. Physicians cannot feasibly attend IDR if their patients are spread across multiple units (or buildings). Finally, hospital leaders also need to view IDR as part of a larger set of interventions to improve teamwork. Leaders need to assess the adequacy of staffing levels, workflow, and team composition.[8] Unit‐based interdisciplinary leadership models should be used to help link efforts at various levels within a larger system.[9] These models designate a unit medical director and nurse manager who are jointly responsible for unit performance.

In conclusion, IDR play an important role in improving patient outcomes, but only do so by improving teamwork. In redesigning IDR, leaders need to be thoughtful about what outcomes IDR can affect, how IDR affect them, and how IDR fit into larger‐scale efforts to improve performance.

Care of hospitalized patients requires effective teamwork within groups composed of physicians (eg, residents, hospitalists, specialists), advanced practice providers, nurses, patient‐care technicians, pharmacists, social workers, and therapists. Sadly, hospital‐based team members often fail to communicate. For example, 2 studies found that nurses and physicians communicated with one another on only 50% to 60% of their patients' hospital days, resulting in a lack of a mutual understanding of the plan of care.[1, 2]

Failure to communicate effectively may be because the hospital setting poses important challenges to teamwork, including the use of large teams with membership that changes frequently because of the need to provide care around the clock. Furthermore, individual team members often have high workloads, care for multiple patients simultaneously, and are seldom in the same place at the same time.

Interdisciplinary rounds (IDR) are a microsystem‐level solution with the goal to share information, achieve mutual understanding, and collaboratively revise the plan of care within care teams. Though common, IDR look very different across hospitals, making studies that evaluate novel strategies to improve IDR and measure their impact of great interest to hospital medicine.

In this issue of the Journal of Hospital Medicine, Bhamidipati and colleagues present a systematic review of published studies evaluating the effect of IDR on patient outcomes.[3] The systematic review included 22 studies, including 12 experimental/quasiexperimental and 10 observational studies. Overall, 13 studies were of low to medium quality, and 9 were high quality. Importantly, relatively few studies reported the degree to which IDR were implemented as planned. The investigators found evidence that IDR had a positive effect on length of stay (LOS) and staff satisfaction, but little evidence to support an effect on patient safety or satisfaction. Furthermore, the investigators found significant variability in IDR design and team composition. Some of this variation is to be expected, as IDR, like other interventions to improve quality and safety of patient care in complex settings, should be implemented with an expectation that the team may need to make adaptations based on local contextual factors such as workload (eg, daily census), environment (eg, open vs closed intensive care unit), local politics (eg, uniquely strong support for/against the intervention), and prior experience (eg, prior failed, similar interventions).[4, 5] Moreover, objectives for IDR may differ across settings. Some hospitals may have room (and a need) to improve LOS, whereas others may prioritize improving patient safety or patient experience metrics.

Bhamidipati and colleagues explain that their review did not reveal a causal pathway between IDR design and outcomes. We believe this lack of association is because most of the included studies did not propose a causal pathway between the IDR components implemented and the outcomes assessed. That is, few studies referred to conceptual models that explain how components of the IDR intervention might influence downstream patient outcomes.

IDR have the potential to influence a number of patient outcomes, including those reflecting efficiency (eg, length of stay), patient safety (eg, adverse events), and patient centeredness (eg, patient satisfaction). However, these outcomes are influenced by many factors, including patient characteristics and other efforts to improve care. As explained by the investigators, the results of many of the included studies may have been confounded due to relatively weak study designs and statistical analyses. Importantly, few of the studies included in this review report the more proximal measure of teamwork. If we hypothesize that IDR improve patient outcomes, they do so by improving teamwork. After all, the purpose of IDR is to assemble team members so they can communicate about and coordinate care. Measuring teamwork behaviors is difficult, especially on medical services. Measuring teamwork climate, the measurable aspects of team culture, is relatively easy. A recent systematic review of teamwork climate assessments in internal medicine identified the Safety Attitudes Questionnaire and the Team Climate Inventory as having substantial validity evidence and association with improved patient outcomes.[6]

Bhamidipati and colleagues proposed a definition for IDR and taxonomy for IDR design and reporting based on their systematic review. Although very useful, the IDR definition may be too limiting as evidenced by the fact that very few studies would be included in a systematic review using this definition as the inclusion criteria. Their proposed taxonomy should serve as a useful framework for future research efforts and appropriately recommends reporting of site characteristics, components of IDR design, and outcomes.

The systematic review by Bhamidipati et al. must also be interpreted in conjunction with another recently published systematic review by Pannick and colleagues assessing the effect of interdisciplinary team care interventions on general medical wards.[7] Contrary to the findings of the Bhamidipati et al. study, Pannick and colleagues found that most interdisciplinary team care interventions had no effect on LOS, but that half of the studies found an improvement in complications of care. Importantly, Pannick and colleagues included only experimental and quasiexperimental studies in their systematic review (ie, no observational studies).

There is clearly more work to be done in researching IDR and other interventions to improve teamwork in general medical settings. Larger studies are needed to provide sufficient power to detect improvement in outcomes. Future studies need to report the degree to which interventions are implemented as planned and need to use stronger study designs (eg, cluster randomized control or interrupted time series) to avoid the influence of confounders. Qualitative methods should be used to assess the influence of contextual factors on the success of interventions.[4] Most importantly, future studies should be based on conceptual models that explain how components of the intervention influence proximal measures of teamwork and downstream patient outcomes.

In the meantime, what is a hospital leader to do? We believe efforts to improve IDR are warranted, but that IDR program leaders need to first specify their primary objective(s). For example, in some hospitals, there may be little room to further reduce LOS, so another goalreducing preventable readmissions or reducing adverse eventsmight be specified as the key performance indicator. This crucial first step of creating a shared goal informs the design, implementation, and evaluation of IDR. We also believe that geographic localization of physicians to specific units is foundational to improving IDR. Physicians cannot feasibly attend IDR if their patients are spread across multiple units (or buildings). Finally, hospital leaders also need to view IDR as part of a larger set of interventions to improve teamwork. Leaders need to assess the adequacy of staffing levels, workflow, and team composition.[8] Unit‐based interdisciplinary leadership models should be used to help link efforts at various levels within a larger system.[9] These models designate a unit medical director and nurse manager who are jointly responsible for unit performance.

In conclusion, IDR play an important role in improving patient outcomes, but only do so by improving teamwork. In redesigning IDR, leaders need to be thoughtful about what outcomes IDR can affect, how IDR affect them, and how IDR fit into larger‐scale efforts to improve performance.

Care of hospitalized patients requires effective teamwork within groups composed of physicians (eg, residents, hospitalists, specialists), advanced practice providers, nurses, patient‐care technicians, pharmacists, social workers, and therapists. Sadly, hospital‐based team members often fail to communicate. For example, 2 studies found that nurses and physicians communicated with one another on only 50% to 60% of their patients' hospital days, resulting in a lack of a mutual understanding of the plan of care.[1, 2]

Failure to communicate effectively may be because the hospital setting poses important challenges to teamwork, including the use of large teams with membership that changes frequently because of the need to provide care around the clock. Furthermore, individual team members often have high workloads, care for multiple patients simultaneously, and are seldom in the same place at the same time.

Interdisciplinary rounds (IDR) are a microsystem‐level solution with the goal to share information, achieve mutual understanding, and collaboratively revise the plan of care within care teams. Though common, IDR look very different across hospitals, making studies that evaluate novel strategies to improve IDR and measure their impact of great interest to hospital medicine.

In this issue of the Journal of Hospital Medicine, Bhamidipati and colleagues present a systematic review of published studies evaluating the effect of IDR on patient outcomes.[3] The systematic review included 22 studies, including 12 experimental/quasiexperimental and 10 observational studies. Overall, 13 studies were of low to medium quality, and 9 were high quality. Importantly, relatively few studies reported the degree to which IDR were implemented as planned. The investigators found evidence that IDR had a positive effect on length of stay (LOS) and staff satisfaction, but little evidence to support an effect on patient safety or satisfaction. Furthermore, the investigators found significant variability in IDR design and team composition. Some of this variation is to be expected, as IDR, like other interventions to improve quality and safety of patient care in complex settings, should be implemented with an expectation that the team may need to make adaptations based on local contextual factors such as workload (eg, daily census), environment (eg, open vs closed intensive care unit), local politics (eg, uniquely strong support for/against the intervention), and prior experience (eg, prior failed, similar interventions).[4, 5] Moreover, objectives for IDR may differ across settings. Some hospitals may have room (and a need) to improve LOS, whereas others may prioritize improving patient safety or patient experience metrics.

Bhamidipati and colleagues explain that their review did not reveal a causal pathway between IDR design and outcomes. We believe this lack of association is because most of the included studies did not propose a causal pathway between the IDR components implemented and the outcomes assessed. That is, few studies referred to conceptual models that explain how components of the IDR intervention might influence downstream patient outcomes.

IDR have the potential to influence a number of patient outcomes, including those reflecting efficiency (eg, length of stay), patient safety (eg, adverse events), and patient centeredness (eg, patient satisfaction). However, these outcomes are influenced by many factors, including patient characteristics and other efforts to improve care. As explained by the investigators, the results of many of the included studies may have been confounded due to relatively weak study designs and statistical analyses. Importantly, few of the studies included in this review report the more proximal measure of teamwork. If we hypothesize that IDR improve patient outcomes, they do so by improving teamwork. After all, the purpose of IDR is to assemble team members so they can communicate about and coordinate care. Measuring teamwork behaviors is difficult, especially on medical services. Measuring teamwork climate, the measurable aspects of team culture, is relatively easy. A recent systematic review of teamwork climate assessments in internal medicine identified the Safety Attitudes Questionnaire and the Team Climate Inventory as having substantial validity evidence and association with improved patient outcomes.[6]

Bhamidipati and colleagues proposed a definition for IDR and taxonomy for IDR design and reporting based on their systematic review. Although very useful, the IDR definition may be too limiting as evidenced by the fact that very few studies would be included in a systematic review using this definition as the inclusion criteria. Their proposed taxonomy should serve as a useful framework for future research efforts and appropriately recommends reporting of site characteristics, components of IDR design, and outcomes.

The systematic review by Bhamidipati et al. must also be interpreted in conjunction with another recently published systematic review by Pannick and colleagues assessing the effect of interdisciplinary team care interventions on general medical wards.[7] Contrary to the findings of the Bhamidipati et al. study, Pannick and colleagues found that most interdisciplinary team care interventions had no effect on LOS, but that half of the studies found an improvement in complications of care. Importantly, Pannick and colleagues included only experimental and quasiexperimental studies in their systematic review (ie, no observational studies).

There is clearly more work to be done in researching IDR and other interventions to improve teamwork in general medical settings. Larger studies are needed to provide sufficient power to detect improvement in outcomes. Future studies need to report the degree to which interventions are implemented as planned and need to use stronger study designs (eg, cluster randomized control or interrupted time series) to avoid the influence of confounders. Qualitative methods should be used to assess the influence of contextual factors on the success of interventions.[4] Most importantly, future studies should be based on conceptual models that explain how components of the intervention influence proximal measures of teamwork and downstream patient outcomes.

In the meantime, what is a hospital leader to do? We believe efforts to improve IDR are warranted, but that IDR program leaders need to first specify their primary objective(s). For example, in some hospitals, there may be little room to further reduce LOS, so another goalreducing preventable readmissions or reducing adverse eventsmight be specified as the key performance indicator. This crucial first step of creating a shared goal informs the design, implementation, and evaluation of IDR. We also believe that geographic localization of physicians to specific units is foundational to improving IDR. Physicians cannot feasibly attend IDR if their patients are spread across multiple units (or buildings). Finally, hospital leaders also need to view IDR as part of a larger set of interventions to improve teamwork. Leaders need to assess the adequacy of staffing levels, workflow, and team composition.[8] Unit‐based interdisciplinary leadership models should be used to help link efforts at various levels within a larger system.[9] These models designate a unit medical director and nurse manager who are jointly responsible for unit performance.

In conclusion, IDR play an important role in improving patient outcomes, but only do so by improving teamwork. In redesigning IDR, leaders need to be thoughtful about what outcomes IDR can affect, how IDR affect them, and how IDR fit into larger‐scale efforts to improve performance.

Determining when residents are independently prepared to perform clinical care tasks safely is not easy or understood. Educators have struggled to identify robust ways to evaluate trainees and their preparedness to treat patients while unsupervised. Trust allows the trainee to experience increasing levels of participation and responsibility in the workplace in a way that builds competence for future practice. The breadth of knowledge and skills required to become a competent and safe physician, coupled with the busy workload confound this challenge. Notably, a technically proficient trainee may not have the clinical judgment to treat patients without supervision.

The Accreditation Council of Graduate Medical Education (ACGME) has previously outlined 6 core competencies for residency training: patient care, medical knowledge, practice‐based learning and improvement, interpersonal and communication skills, professionalism, and systems‐based practice.[1] A systematic literature review suggests that traditional trainee evaluation tools are difficult to use and unreliable in measuring the competencies independently from one another, whereas certain competencies are consistently difficult to quantify in a reliable and valid way.[2] The evaluation of trainees' clinical performance despite efforts to create objective tools remain strongly influenced by subjective measures and continues to be highly variable among different evaluators.[3] Objectively measuring resident autonomy and readiness to supervise junior colleagues remains imprecise.[4]

The ACGME's Next Accreditation System (NAS) incorporates educational milestones as part of the reporting of resident training outcomes.[5] The milestones allow for the translation of the core competencies into integrative and observable abilities. Furthermore, the milestone categories are stratified into tiers to allow progress to be measured longitudinally and by task complexity using a novel assessment strategy.

The development of trust between supervisors and trainees is a critical step in decisions to allow increased responsibility and the provision of autonomous decision making, which is an important aspect of physician training. Identifying the factors that influence the supervisors' evaluation of resident competency and capability is at the crux of trainee maturation as well as patient safety.[4] Trust, defined as believability and discernment by attendings of resident physicians, plays a large role in attending evaluations of residents during their clinical rotations.[3] Trust impacts the decisions of successful performance of entrustable professional activities (EPAs), or those tasks that require mastery prior to completion of training milestones.[6] A study of entrustment decisions made by attending anesthesiologists identified the factors that contribute to the amount of autonomy given to residents, such as trainee trustworthiness, medical knowledge, and level of training.[4] The aim of our study, building on this study, was 2‐fold: (1) use deductive qualitative analysis to apply this framework to existing resident and attending data, and (2) define the categories within this framework and describe how internal medicine attending and resident physician perceptions of trust can impact clinical decision making and patient care.

METHODS

We are reporting on a secondary data analysis of interview transcripts from a study conducted on the inpatient general medicine service at the University of Chicago, an academic tertiary care medical center. The methods for data collection and full consent have been outlined previously.[7, 8, 9] The institutional review board of the University of Chicago approved this study.

Briefly, between January 2006 and November 2006, all eligible internal medicine resident physicians, postgraduate year (PGY)‐2 or PGY‐3, and attending physicians, either generalists or hospitalists, were privately interviewed within 1 week of their final call night on the inpatient general medicine rotation to assess decision making and clinical supervision during the rotation. All interviews were conducted by 1 investigator (J.F.), and discussions were audio taped and transcribed for analysis. Interviews were conducted at the conclusion of the rotation to prevent any influence on resident and attending behavior during the rotation.

The critical incident technique, a procedure used for collecting direct observations of human behavior that have critical significance on the decision‐making process, was used to solicit examples of ineffective supervision, inquiring about 2 to 3 important clinical decisions made on the most recent call night, with probes to identify issues of trust, autonomy, and decision making.[10] A critical incident can be described as one that makes a significant contribution, either positively or negatively, on the process.

Appreciative inquiry, a technique that aims to uncover the best things about the clinical encounter being explored, was used to solicit examples of effective supervision. Probes are used to identify factors, either personal or situational, that influenced the withholding or provision of resident autonomy during periods of clinical care delivery.[11]

All identifiable information was removed from the interview transcripts to protect participant and patient confidentiality. Deductive qualitative analysis was performed using the conceptual EPA framework, which describes several factors that influence the attending physicians' decisions to deem a resident trustworthy to independently fulfill a specific clinical task.[4] These factors include (1) the nature of the task, (2) the qualities of the supervisor, (3) the qualities of the trainee and the quality of the relationship between the supervisor and the trainee, and (4) the circumstances surrounding the clinical task.

The deidentified, anonymous transcripts were reviewed by 2 investigators (K.J.C., J.M.F.) and analyzed using the constant comparative methods to deductively map the content to the existing framework and generate novel sub themes.[12, 13, 14] Novel categories within each of the domains were inductively generated. Two reviewers (K.J.C., J.M.F.) independently applied the themes to a randomly selected 10% portion of the interview transcripts to assess the inter‐rater reliability. The inter‐rater agreement was assessed using the generalized kappa statistic. The discrepancies between reviewers regarding assignment of codes were resolved via discussion and third party adjudication until consensus was achieved on thematic structure. The codes were then applied to the entire dataset.

RESULTS

Between January 2006 and November 2006, 46 of 50 (88%) attending physicians and 44 of 50 (92%) resident physicians were interviewed following the conclusion of their general medicine inpatient rotation. Of attending physicians, 55% were male, 45% were female, and 38% were academic faculty hospitalists. Of the residents who completed interviews, 47% were male, 53% were female, 52% were PGY‐2, and 45% were PGY‐3.

A total of 535 mentions of trust were abstracted from the transcripts. The 4 major domains that influence trusttrainee factors (Table 1), supervisor factors (Table 2), task factors (Table 3), and systems factors (Table 4)were deductively coded with several emerging novel categories and subthemes. The domains were consistent across the postgraduate year of trainee. No differences in themes were noted, other than those explicitly stated, between the postgraduate years.

DISCUSSION

Clinical supervisors must hold the quality of care constant while balancing the amount of supervision and autonomy provided to learners in procedural tasks and clinical decision making. We found that the development of trust is multifactorial and highly contextual. It occurs under the broad constructs of task, supervisor, trainee, and environmental factors, and is well described in prior work. We also demonstrate that often what determines these broader factors is highly subjective, frequently independent of objective measures of trainee performance. Many decisions are based on personal characteristics, such as the perception of honesty, disposition, perceived confidence or perceived overconfidence of the trainee, prior experience, and expressed future field of specialty.

Our findings are consistent with prior research, but go further in describing and demonstrating the existence and innovative use of factors, other than clinical knowledge and skill, in the formation of a multidimensional construct of trust. Kennedy et al. identified 4 dimensions of trust knowledge and skill, discernment, conscientiousness, and truthfulness[15]and demonstrated that supervising physicians rely on specific processes to assess trainee trustworthiness, specifically the use of double checks and language cues. This is consistent with our results, which demonstrate that many attending physicians independently verify information, such as laboratory findings, to inform their perceptions of trainee honesty, attention to detail, and ability to follow orders reliably. Furthermore, our subthemes of communication and the demonstration of logical clinical reasoning correspond to Kennedy's use of language cues.[15] We found that language cues are used as markers of trustworthiness, particularly early on in the rotation, as a litmus test to gauge the trainee's integrity and ability to assess and treat patients unsupervised.

To date, much has been written about the importance of direct observation in the evaluation of trainees.[16, 17, 18, 19] Our results demonstrate that supervising clinicians use a multifactorial, highly nuanced, and subjective process despite validated performance‐based assessment methods, such as the objective structured clinical exam or mini‐clinical evaluation exercise, to assess competence and grant entrustement.[3] Several factors utilized to determine trustworthiness in addition to direct observation are subjective in nature, specifically the trainee's prior experience and expressed career choice.

It is encouraging that attending physicians make use of direct observations to inform decisions of entrustment, albeit in an informal and unstructured way. They also seem to take into account the context and setting in which the observation occurs, and consider both the environmental factors as well as factors that relate to the task itself.[20] For example, attendings and residents reported that team dynamics played a large role in influencing trust decisions. We also found that attending physicians rely on indirect observation and will inquire among their colleagues and other senior residents to gain information about their trainees abilities and integrity. Evaluation tools that facilitate sharing of trainees' level of preparedness, prior feedback, and experience could facilitate the determination of readiness to complete EPAs as well as the reporting of achieved milestones in accordance with the ACGME NAS.

Sharing knowledge about trainees among attendings is common and of increasing importance in the context of attending physicians' shortened exposure to trainees due to the residency work‐hour restrictions and growing productivity pressures. In our study, attending physicians described work‐hour restrictions as detrimental to trainee trustworthiness, either in the context of decreased accountability for patient care or as intrinsic to the nature of forced absences that kept trainees from fully participating in daily ward activities and knowing their patients. Attending physicians felt that trainees did not know their patients well enough to be able to make independent decisions about care. The increased transition to a shift‐based structure of inpatient medicine may result in increasingly less time for direct observation and make it more difficult for attendings to justify their decisions about engendering trust. In addition, the increased fragmentation that is noted in training secondary to the work‐hour regulations may in fact have consequences on the development of clinical skill and decision making, such that increased attention to the need for supervision and longer lead to entrustment may be needed in certain circumstances. Attendings need guidance on how to improve their ability to observe trainees in the context of the new work environment, and how to role model decision making more effectively in the compressed time exposure to housestaff.

Our study has several limitations. The organizational structure and culture of our institution are unique to 1 academic setting. This may undermine our ability to generalize these research findings and analysis to the population at large.[21] In addition, recall bias may have played into the interpretation of the interview content given the timing with which they were performed after the conclusion of the rotation. The study interviews took place in 2006, and it is reasonable to believe that some perceptions concerning duty‐hour restrictions and competency‐based graduate medical education have changed. However, from our ongoing research over the past 5 years[4] and our personal experience with entrustment factors, we believe that the participants' perceptions of trust and competency are valid and have largely remained unchanged, given the similarity in findings to the accepted ten Cate framework. In addition, this work was done following the first iteration of the work‐hour regulations but prior to the implementation of explicit supervisory levels, so it may indeed represent a truer state of the supervisory relationship before external regulations were applied. Finally, this work represents an internal medicine residency training program and may not be generalizable to other specialties that posses different cultural factors that impact the decision for entrustment. However, the congruence of our data with that of the original work of ten Cate, which was done in gynecology,[6] and that of Sterkenberg et al. in anesthesiology,[4] supports our key factors being ubiquitous to all training programs.

In conclusion, we provide new insights into subjective factors that inform the perceptions of trust and entrustment decisions by supervising physicians, specifically subjective trainee characteristics, team dynamics, and informal observation. There was agreement among attendings about which elements of competence are considered most important in their entrustment decisions related to trainee, supervisor, task, and environmental factors. Rather than undervaluing the use of personal factors in the determination of trust, we believe that acknowledgement and appreciation of these factors may be important to give supervisors more confidence and better tools to assess resident physicians, and to understand how their personality traits relate to and impact their professional competence. Our findings are relevant for the development of assessment instruments to evaluate whether medical graduates are ready for safe practice without supervision.

Determining when residents are independently prepared to perform clinical care tasks safely is not easy or understood. Educators have struggled to identify robust ways to evaluate trainees and their preparedness to treat patients while unsupervised. Trust allows the trainee to experience increasing levels of participation and responsibility in the workplace in a way that builds competence for future practice. The breadth of knowledge and skills required to become a competent and safe physician, coupled with the busy workload confound this challenge. Notably, a technically proficient trainee may not have the clinical judgment to treat patients without supervision.

The Accreditation Council of Graduate Medical Education (ACGME) has previously outlined 6 core competencies for residency training: patient care, medical knowledge, practice‐based learning and improvement, interpersonal and communication skills, professionalism, and systems‐based practice.[1] A systematic literature review suggests that traditional trainee evaluation tools are difficult to use and unreliable in measuring the competencies independently from one another, whereas certain competencies are consistently difficult to quantify in a reliable and valid way.[2] The evaluation of trainees' clinical performance despite efforts to create objective tools remain strongly influenced by subjective measures and continues to be highly variable among different evaluators.[3] Objectively measuring resident autonomy and readiness to supervise junior colleagues remains imprecise.[4]

The ACGME's Next Accreditation System (NAS) incorporates educational milestones as part of the reporting of resident training outcomes.[5] The milestones allow for the translation of the core competencies into integrative and observable abilities. Furthermore, the milestone categories are stratified into tiers to allow progress to be measured longitudinally and by task complexity using a novel assessment strategy.

The development of trust between supervisors and trainees is a critical step in decisions to allow increased responsibility and the provision of autonomous decision making, which is an important aspect of physician training. Identifying the factors that influence the supervisors' evaluation of resident competency and capability is at the crux of trainee maturation as well as patient safety.[4] Trust, defined as believability and discernment by attendings of resident physicians, plays a large role in attending evaluations of residents during their clinical rotations.[3] Trust impacts the decisions of successful performance of entrustable professional activities (EPAs), or those tasks that require mastery prior to completion of training milestones.[6] A study of entrustment decisions made by attending anesthesiologists identified the factors that contribute to the amount of autonomy given to residents, such as trainee trustworthiness, medical knowledge, and level of training.[4] The aim of our study, building on this study, was 2‐fold: (1) use deductive qualitative analysis to apply this framework to existing resident and attending data, and (2) define the categories within this framework and describe how internal medicine attending and resident physician perceptions of trust can impact clinical decision making and patient care.

METHODS

We are reporting on a secondary data analysis of interview transcripts from a study conducted on the inpatient general medicine service at the University of Chicago, an academic tertiary care medical center. The methods for data collection and full consent have been outlined previously.[7, 8, 9] The institutional review board of the University of Chicago approved this study.

Briefly, between January 2006 and November 2006, all eligible internal medicine resident physicians, postgraduate year (PGY)‐2 or PGY‐3, and attending physicians, either generalists or hospitalists, were privately interviewed within 1 week of their final call night on the inpatient general medicine rotation to assess decision making and clinical supervision during the rotation. All interviews were conducted by 1 investigator (J.F.), and discussions were audio taped and transcribed for analysis. Interviews were conducted at the conclusion of the rotation to prevent any influence on resident and attending behavior during the rotation.

The critical incident technique, a procedure used for collecting direct observations of human behavior that have critical significance on the decision‐making process, was used to solicit examples of ineffective supervision, inquiring about 2 to 3 important clinical decisions made on the most recent call night, with probes to identify issues of trust, autonomy, and decision making.[10] A critical incident can be described as one that makes a significant contribution, either positively or negatively, on the process.

Appreciative inquiry, a technique that aims to uncover the best things about the clinical encounter being explored, was used to solicit examples of effective supervision. Probes are used to identify factors, either personal or situational, that influenced the withholding or provision of resident autonomy during periods of clinical care delivery.[11]

All identifiable information was removed from the interview transcripts to protect participant and patient confidentiality. Deductive qualitative analysis was performed using the conceptual EPA framework, which describes several factors that influence the attending physicians' decisions to deem a resident trustworthy to independently fulfill a specific clinical task.[4] These factors include (1) the nature of the task, (2) the qualities of the supervisor, (3) the qualities of the trainee and the quality of the relationship between the supervisor and the trainee, and (4) the circumstances surrounding the clinical task.

The deidentified, anonymous transcripts were reviewed by 2 investigators (K.J.C., J.M.F.) and analyzed using the constant comparative methods to deductively map the content to the existing framework and generate novel sub themes.[12, 13, 14] Novel categories within each of the domains were inductively generated. Two reviewers (K.J.C., J.M.F.) independently applied the themes to a randomly selected 10% portion of the interview transcripts to assess the inter‐rater reliability. The inter‐rater agreement was assessed using the generalized kappa statistic. The discrepancies between reviewers regarding assignment of codes were resolved via discussion and third party adjudication until consensus was achieved on thematic structure. The codes were then applied to the entire dataset.

RESULTS

Between January 2006 and November 2006, 46 of 50 (88%) attending physicians and 44 of 50 (92%) resident physicians were interviewed following the conclusion of their general medicine inpatient rotation. Of attending physicians, 55% were male, 45% were female, and 38% were academic faculty hospitalists. Of the residents who completed interviews, 47% were male, 53% were female, 52% were PGY‐2, and 45% were PGY‐3.

A total of 535 mentions of trust were abstracted from the transcripts. The 4 major domains that influence trusttrainee factors (Table 1), supervisor factors (Table 2), task factors (Table 3), and systems factors (Table 4)were deductively coded with several emerging novel categories and subthemes. The domains were consistent across the postgraduate year of trainee. No differences in themes were noted, other than those explicitly stated, between the postgraduate years.

DISCUSSION

Clinical supervisors must hold the quality of care constant while balancing the amount of supervision and autonomy provided to learners in procedural tasks and clinical decision making. We found that the development of trust is multifactorial and highly contextual. It occurs under the broad constructs of task, supervisor, trainee, and environmental factors, and is well described in prior work. We also demonstrate that often what determines these broader factors is highly subjective, frequently independent of objective measures of trainee performance. Many decisions are based on personal characteristics, such as the perception of honesty, disposition, perceived confidence or perceived overconfidence of the trainee, prior experience, and expressed future field of specialty.

Our findings are consistent with prior research, but go further in describing and demonstrating the existence and innovative use of factors, other than clinical knowledge and skill, in the formation of a multidimensional construct of trust. Kennedy et al. identified 4 dimensions of trust knowledge and skill, discernment, conscientiousness, and truthfulness[15]and demonstrated that supervising physicians rely on specific processes to assess trainee trustworthiness, specifically the use of double checks and language cues. This is consistent with our results, which demonstrate that many attending physicians independently verify information, such as laboratory findings, to inform their perceptions of trainee honesty, attention to detail, and ability to follow orders reliably. Furthermore, our subthemes of communication and the demonstration of logical clinical reasoning correspond to Kennedy's use of language cues.[15] We found that language cues are used as markers of trustworthiness, particularly early on in the rotation, as a litmus test to gauge the trainee's integrity and ability to assess and treat patients unsupervised.

To date, much has been written about the importance of direct observation in the evaluation of trainees.[16, 17, 18, 19] Our results demonstrate that supervising clinicians use a multifactorial, highly nuanced, and subjective process despite validated performance‐based assessment methods, such as the objective structured clinical exam or mini‐clinical evaluation exercise, to assess competence and grant entrustement.[3] Several factors utilized to determine trustworthiness in addition to direct observation are subjective in nature, specifically the trainee's prior experience and expressed career choice.

It is encouraging that attending physicians make use of direct observations to inform decisions of entrustment, albeit in an informal and unstructured way. They also seem to take into account the context and setting in which the observation occurs, and consider both the environmental factors as well as factors that relate to the task itself.[20] For example, attendings and residents reported that team dynamics played a large role in influencing trust decisions. We also found that attending physicians rely on indirect observation and will inquire among their colleagues and other senior residents to gain information about their trainees abilities and integrity. Evaluation tools that facilitate sharing of trainees' level of preparedness, prior feedback, and experience could facilitate the determination of readiness to complete EPAs as well as the reporting of achieved milestones in accordance with the ACGME NAS.

Sharing knowledge about trainees among attendings is common and of increasing importance in the context of attending physicians' shortened exposure to trainees due to the residency work‐hour restrictions and growing productivity pressures. In our study, attending physicians described work‐hour restrictions as detrimental to trainee trustworthiness, either in the context of decreased accountability for patient care or as intrinsic to the nature of forced absences that kept trainees from fully participating in daily ward activities and knowing their patients. Attending physicians felt that trainees did not know their patients well enough to be able to make independent decisions about care. The increased transition to a shift‐based structure of inpatient medicine may result in increasingly less time for direct observation and make it more difficult for attendings to justify their decisions about engendering trust. In addition, the increased fragmentation that is noted in training secondary to the work‐hour regulations may in fact have consequences on the development of clinical skill and decision making, such that increased attention to the need for supervision and longer lead to entrustment may be needed in certain circumstances. Attendings need guidance on how to improve their ability to observe trainees in the context of the new work environment, and how to role model decision making more effectively in the compressed time exposure to housestaff.

Our study has several limitations. The organizational structure and culture of our institution are unique to 1 academic setting. This may undermine our ability to generalize these research findings and analysis to the population at large.[21] In addition, recall bias may have played into the interpretation of the interview content given the timing with which they were performed after the conclusion of the rotation. The study interviews took place in 2006, and it is reasonable to believe that some perceptions concerning duty‐hour restrictions and competency‐based graduate medical education have changed. However, from our ongoing research over the past 5 years[4] and our personal experience with entrustment factors, we believe that the participants' perceptions of trust and competency are valid and have largely remained unchanged, given the similarity in findings to the accepted ten Cate framework. In addition, this work was done following the first iteration of the work‐hour regulations but prior to the implementation of explicit supervisory levels, so it may indeed represent a truer state of the supervisory relationship before external regulations were applied. Finally, this work represents an internal medicine residency training program and may not be generalizable to other specialties that posses different cultural factors that impact the decision for entrustment. However, the congruence of our data with that of the original work of ten Cate, which was done in gynecology,[6] and that of Sterkenberg et al. in anesthesiology,[4] supports our key factors being ubiquitous to all training programs.

In conclusion, we provide new insights into subjective factors that inform the perceptions of trust and entrustment decisions by supervising physicians, specifically subjective trainee characteristics, team dynamics, and informal observation. There was agreement among attendings about which elements of competence are considered most important in their entrustment decisions related to trainee, supervisor, task, and environmental factors. Rather than undervaluing the use of personal factors in the determination of trust, we believe that acknowledgement and appreciation of these factors may be important to give supervisors more confidence and better tools to assess resident physicians, and to understand how their personality traits relate to and impact their professional competence. Our findings are relevant for the development of assessment instruments to evaluate whether medical graduates are ready for safe practice without supervision.

Determining when residents are independently prepared to perform clinical care tasks safely is not easy or understood. Educators have struggled to identify robust ways to evaluate trainees and their preparedness to treat patients while unsupervised. Trust allows the trainee to experience increasing levels of participation and responsibility in the workplace in a way that builds competence for future practice. The breadth of knowledge and skills required to become a competent and safe physician, coupled with the busy workload confound this challenge. Notably, a technically proficient trainee may not have the clinical judgment to treat patients without supervision.

The Accreditation Council of Graduate Medical Education (ACGME) has previously outlined 6 core competencies for residency training: patient care, medical knowledge, practice‐based learning and improvement, interpersonal and communication skills, professionalism, and systems‐based practice.[1] A systematic literature review suggests that traditional trainee evaluation tools are difficult to use and unreliable in measuring the competencies independently from one another, whereas certain competencies are consistently difficult to quantify in a reliable and valid way.[2] The evaluation of trainees' clinical performance despite efforts to create objective tools remain strongly influenced by subjective measures and continues to be highly variable among different evaluators.[3] Objectively measuring resident autonomy and readiness to supervise junior colleagues remains imprecise.[4]

The ACGME's Next Accreditation System (NAS) incorporates educational milestones as part of the reporting of resident training outcomes.[5] The milestones allow for the translation of the core competencies into integrative and observable abilities. Furthermore, the milestone categories are stratified into tiers to allow progress to be measured longitudinally and by task complexity using a novel assessment strategy.

The development of trust between supervisors and trainees is a critical step in decisions to allow increased responsibility and the provision of autonomous decision making, which is an important aspect of physician training. Identifying the factors that influence the supervisors' evaluation of resident competency and capability is at the crux of trainee maturation as well as patient safety.[4] Trust, defined as believability and discernment by attendings of resident physicians, plays a large role in attending evaluations of residents during their clinical rotations.[3] Trust impacts the decisions of successful performance of entrustable professional activities (EPAs), or those tasks that require mastery prior to completion of training milestones.[6] A study of entrustment decisions made by attending anesthesiologists identified the factors that contribute to the amount of autonomy given to residents, such as trainee trustworthiness, medical knowledge, and level of training.[4] The aim of our study, building on this study, was 2‐fold: (1) use deductive qualitative analysis to apply this framework to existing resident and attending data, and (2) define the categories within this framework and describe how internal medicine attending and resident physician perceptions of trust can impact clinical decision making and patient care.

METHODS

We are reporting on a secondary data analysis of interview transcripts from a study conducted on the inpatient general medicine service at the University of Chicago, an academic tertiary care medical center. The methods for data collection and full consent have been outlined previously.[7, 8, 9] The institutional review board of the University of Chicago approved this study.

Briefly, between January 2006 and November 2006, all eligible internal medicine resident physicians, postgraduate year (PGY)‐2 or PGY‐3, and attending physicians, either generalists or hospitalists, were privately interviewed within 1 week of their final call night on the inpatient general medicine rotation to assess decision making and clinical supervision during the rotation. All interviews were conducted by 1 investigator (J.F.), and discussions were audio taped and transcribed for analysis. Interviews were conducted at the conclusion of the rotation to prevent any influence on resident and attending behavior during the rotation.

The critical incident technique, a procedure used for collecting direct observations of human behavior that have critical significance on the decision‐making process, was used to solicit examples of ineffective supervision, inquiring about 2 to 3 important clinical decisions made on the most recent call night, with probes to identify issues of trust, autonomy, and decision making.[10] A critical incident can be described as one that makes a significant contribution, either positively or negatively, on the process.

Appreciative inquiry, a technique that aims to uncover the best things about the clinical encounter being explored, was used to solicit examples of effective supervision. Probes are used to identify factors, either personal or situational, that influenced the withholding or provision of resident autonomy during periods of clinical care delivery.[11]

All identifiable information was removed from the interview transcripts to protect participant and patient confidentiality. Deductive qualitative analysis was performed using the conceptual EPA framework, which describes several factors that influence the attending physicians' decisions to deem a resident trustworthy to independently fulfill a specific clinical task.[4] These factors include (1) the nature of the task, (2) the qualities of the supervisor, (3) the qualities of the trainee and the quality of the relationship between the supervisor and the trainee, and (4) the circumstances surrounding the clinical task.

The deidentified, anonymous transcripts were reviewed by 2 investigators (K.J.C., J.M.F.) and analyzed using the constant comparative methods to deductively map the content to the existing framework and generate novel sub themes.[12, 13, 14] Novel categories within each of the domains were inductively generated. Two reviewers (K.J.C., J.M.F.) independently applied the themes to a randomly selected 10% portion of the interview transcripts to assess the inter‐rater reliability. The inter‐rater agreement was assessed using the generalized kappa statistic. The discrepancies between reviewers regarding assignment of codes were resolved via discussion and third party adjudication until consensus was achieved on thematic structure. The codes were then applied to the entire dataset.

RESULTS

Between January 2006 and November 2006, 46 of 50 (88%) attending physicians and 44 of 50 (92%) resident physicians were interviewed following the conclusion of their general medicine inpatient rotation. Of attending physicians, 55% were male, 45% were female, and 38% were academic faculty hospitalists. Of the residents who completed interviews, 47% were male, 53% were female, 52% were PGY‐2, and 45% were PGY‐3.

A total of 535 mentions of trust were abstracted from the transcripts. The 4 major domains that influence trusttrainee factors (Table 1), supervisor factors (Table 2), task factors (Table 3), and systems factors (Table 4)were deductively coded with several emerging novel categories and subthemes. The domains were consistent across the postgraduate year of trainee. No differences in themes were noted, other than those explicitly stated, between the postgraduate years.

DISCUSSION

Clinical supervisors must hold the quality of care constant while balancing the amount of supervision and autonomy provided to learners in procedural tasks and clinical decision making. We found that the development of trust is multifactorial and highly contextual. It occurs under the broad constructs of task, supervisor, trainee, and environmental factors, and is well described in prior work. We also demonstrate that often what determines these broader factors is highly subjective, frequently independent of objective measures of trainee performance. Many decisions are based on personal characteristics, such as the perception of honesty, disposition, perceived confidence or perceived overconfidence of the trainee, prior experience, and expressed future field of specialty.

Our findings are consistent with prior research, but go further in describing and demonstrating the existence and innovative use of factors, other than clinical knowledge and skill, in the formation of a multidimensional construct of trust. Kennedy et al. identified 4 dimensions of trust knowledge and skill, discernment, conscientiousness, and truthfulness[15]and demonstrated that supervising physicians rely on specific processes to assess trainee trustworthiness, specifically the use of double checks and language cues. This is consistent with our results, which demonstrate that many attending physicians independently verify information, such as laboratory findings, to inform their perceptions of trainee honesty, attention to detail, and ability to follow orders reliably. Furthermore, our subthemes of communication and the demonstration of logical clinical reasoning correspond to Kennedy's use of language cues.[15] We found that language cues are used as markers of trustworthiness, particularly early on in the rotation, as a litmus test to gauge the trainee's integrity and ability to assess and treat patients unsupervised.

To date, much has been written about the importance of direct observation in the evaluation of trainees.[16, 17, 18, 19] Our results demonstrate that supervising clinicians use a multifactorial, highly nuanced, and subjective process despite validated performance‐based assessment methods, such as the objective structured clinical exam or mini‐clinical evaluation exercise, to assess competence and grant entrustement.[3] Several factors utilized to determine trustworthiness in addition to direct observation are subjective in nature, specifically the trainee's prior experience and expressed career choice.

It is encouraging that attending physicians make use of direct observations to inform decisions of entrustment, albeit in an informal and unstructured way. They also seem to take into account the context and setting in which the observation occurs, and consider both the environmental factors as well as factors that relate to the task itself.[20] For example, attendings and residents reported that team dynamics played a large role in influencing trust decisions. We also found that attending physicians rely on indirect observation and will inquire among their colleagues and other senior residents to gain information about their trainees abilities and integrity. Evaluation tools that facilitate sharing of trainees' level of preparedness, prior feedback, and experience could facilitate the determination of readiness to complete EPAs as well as the reporting of achieved milestones in accordance with the ACGME NAS.

Sharing knowledge about trainees among attendings is common and of increasing importance in the context of attending physicians' shortened exposure to trainees due to the residency work‐hour restrictions and growing productivity pressures. In our study, attending physicians described work‐hour restrictions as detrimental to trainee trustworthiness, either in the context of decreased accountability for patient care or as intrinsic to the nature of forced absences that kept trainees from fully participating in daily ward activities and knowing their patients. Attending physicians felt that trainees did not know their patients well enough to be able to make independent decisions about care. The increased transition to a shift‐based structure of inpatient medicine may result in increasingly less time for direct observation and make it more difficult for attendings to justify their decisions about engendering trust. In addition, the increased fragmentation that is noted in training secondary to the work‐hour regulations may in fact have consequences on the development of clinical skill and decision making, such that increased attention to the need for supervision and longer lead to entrustment may be needed in certain circumstances. Attendings need guidance on how to improve their ability to observe trainees in the context of the new work environment, and how to role model decision making more effectively in the compressed time exposure to housestaff.

Our study has several limitations. The organizational structure and culture of our institution are unique to 1 academic setting. This may undermine our ability to generalize these research findings and analysis to the population at large.[21] In addition, recall bias may have played into the interpretation of the interview content given the timing with which they were performed after the conclusion of the rotation. The study interviews took place in 2006, and it is reasonable to believe that some perceptions concerning duty‐hour restrictions and competency‐based graduate medical education have changed. However, from our ongoing research over the past 5 years[4] and our personal experience with entrustment factors, we believe that the participants' perceptions of trust and competency are valid and have largely remained unchanged, given the similarity in findings to the accepted ten Cate framework. In addition, this work was done following the first iteration of the work‐hour regulations but prior to the implementation of explicit supervisory levels, so it may indeed represent a truer state of the supervisory relationship before external regulations were applied. Finally, this work represents an internal medicine residency training program and may not be generalizable to other specialties that posses different cultural factors that impact the decision for entrustment. However, the congruence of our data with that of the original work of ten Cate, which was done in gynecology,[6] and that of Sterkenberg et al. in anesthesiology,[4] supports our key factors being ubiquitous to all training programs.

In conclusion, we provide new insights into subjective factors that inform the perceptions of trust and entrustment decisions by supervising physicians, specifically subjective trainee characteristics, team dynamics, and informal observation. There was agreement among attendings about which elements of competence are considered most important in their entrustment decisions related to trainee, supervisor, task, and environmental factors. Rather than undervaluing the use of personal factors in the determination of trust, we believe that acknowledgement and appreciation of these factors may be important to give supervisors more confidence and better tools to assess resident physicians, and to understand how their personality traits relate to and impact their professional competence. Our findings are relevant for the development of assessment instruments to evaluate whether medical graduates are ready for safe practice without supervision.