Vikas Parekh, MD

The structure and function of academic hospital medicine programs (AHPs) has evolved significantly with the growth of hospital medicine.[1, 2, 3, 4] Many AHPs formed in response to regulatory and financial changes, which drove demand for increased trainee oversight, improved clinical efficiency, and growth in nonteaching services staffed by hospitalists. Differences in local organizational contexts and needs have contributed to great variability in AHP program design and operations. As AHPs have become more established, the need to engage academic hospitalists in scholarship and activities that support professional development and promotion has been recognized. Defining sustainable and successful positions for academic hospitalists is a priority called for by leaders in the field.[5, 6]

In this rapidly evolving context, AHPs have employed a variety of approaches to organizing clinical and academic faculty roles, without guiding evidence or consensus‐based performance benchmarks. A number of AHPs have achieved success along traditional academic metrics of research, scholarship, and education. Currently, it is not known whether specific approaches to AHP organization, structure, or definition of faculty roles are associated with achievement of more traditional markers of academic success.

The Academic Committee of the Society of Hospital Medicine (SHM), and the Academic Hospitalist Task Force of the Society of General Internal Medicine (SGIM) had separately initiated projects to explore characteristics associated with success in AHPs. In 2012, these organizations combined efforts to jointly develop and implement the SCHOLAR (SuCcessful HOspitaLists in Academics and Research) project. The goals were to identify successful AHPs using objective criteria, and to then study those groups in greater detail to generate insights that would be broadly relevant to the field. Efforts to clarify the factors within AHPs linked to success by traditional academic metrics will benefit hospitalists, their leaders, and key stakeholders striving to achieve optimal balance between clinical and academic roles. We describe the initial work of the SCHOLAR project, our definitions of academic success in AHPs, and the characteristics of a cohort of exemplary AHPs who achieved the highest levels on these metrics.

METHODS

RESULTS

Faculty Promotion

Figure 1 demonstrates the proportion of faculty at various academic ranks. The percent of faculty above the rank of assistant professor in the SCHOLAR programs exceeded those in the overall LAHP‐50 by 5% (17.9% vs 12.8%, P = 0.01). Of note, 6% of the hospitalists at AHPs in the SCHOLAR programs were on nonfaculty tracks.

Figure 1

DISCUSSION

Using a definition of academic success that incorporated metrics of grant funding, faculty promotion, and scholarly output, we identified a unique subset of successful AHPsthe SCHOLAR cohort. The programs represented in the SCHOLAR cohort were generally large and relatively mature. Despite this, the cohort consisted of mostly junior faculty, had a paucity of fellowship‐trained hospitalists, and not all reported grant funding.

Prior published work reported complementary findings.[6, 8, 9] A survey of 20 large, well‐established academic hospitalist programs in 2008 found that the majority of hospitalists were junior faculty with a limited publication portfolio. Of the 266 respondents in that study, 86% reported an academic rank at or below assistant professor; funding was not explored.[9] Our similar findings 4 years later add to this work by demonstrating trends over time, and suggest that progress toward creating successful pathways for academic advancement has been slow. In a 2012 survey of the SHM membership, 28% of hospitalists with academic appointments reported no current or future plans to engage in research.[8] These findings suggest that faculty in AHPs may define scholarship through nontraditional pathways, or in some cases choose not to pursue or prioritize scholarship altogether.

Our findings also add to the literature with regard to our assessment of funding, which was variable across the SCHOLAR group. The broad range of funding in the SCHOLAR programs for which we have data (grant dollars $0$15 million per program) suggests that opportunities to improve supported scholarship remain, even among a selected cohort of successful AHPs. The predominance of junior faculty in the SCHOLAR programs may be a reason for this variation. Junior faculty may be engaged in research with funding directed to senior mentors outside their AHP. Alternatively, they may pursue meaningful local hospital quality improvement or educational innovations not supported by external grants, or hold leadership roles in education, quality, or information technology that allow for advancement and promotion without external grant funding. As the scope and impact of these roles increases, senior leaders with alternate sources of support may rely less on research funds; this too may explain some of the differences. Our findings are congruent with results of a study that reviewed original research published by hospitalists, and concluded that the majority of hospitalist research was not externally funded.[8] Our approach for assessing grant funding by adjusting for FTE had the potential to inadvertently favor smaller well‐funded groups over larger ones; however, programs in our sample were similarly represented when ranked by funding/FTE or total grant dollars. As many successful AHPs do concentrate their research funding among a core of focused hospitalist researchers, our definition may not be the ideal metric for some programs.

We chose to define scholarship based on abstract output, rather than peer‐reviewed publications. Although this choice was necessary from a feasibility perspective, it may have excluded programs that prioritize peer‐reviewed publications over abstracts. Although we were unable to incorporate a search strategy to accurately and comprehensively track the publication output attributed specifically to hospitalist researchers and quantify it by program, others have since defined such an approach.[8] However, tracking abstracts theoretically allowed insights into a larger volume of innovative and creative work generated by top AHPs by potentially including work in the earlier stages of development.

We used a consensus‐based definition of success to define our SCHOLAR cohort. There are other ways to measure academic success, which if applied, may have yielded a different sample of programs. For example, over half of the original research articles published in the Journal of Hospital Medicine over a 7‐year span were generated from 5 academic centers.[8] This definition of success may be equally credible, though we note that 4 of these 5 programs were also included in the SCHOLAR cohort. We feel our broader approach was more reflective of the variety of pathways to success available to academic hospitalists. Before our metrics are applied as a benchmarking tool, however, they should ideally be combined with factors not measured in our study to ensure a more comprehensive or balanced reflection of academic success. Factors such as mentorship, level of hospitalist engagement,[10] prevalence of leadership opportunities, operational and fiscal infrastructure, and the impact of local quality, safety, and value efforts should be considered.

Comparison of successfully promoted faculty at AHPs across the country is inherently limited by the wide variation in promotion standards across different institutions; controlling for such differences was not possible with our methodology. For example, it appears that several programs with relatively few senior faculty may have met metrics leading to their inclusion in the SCHOLAR group because of their small program size. Future benchmarking efforts for promotion at AHPs should take scaling into account and consider both total number as well as percentage of senior faculty when evaluating success.

Our methodology has several limitations. Survey data were self‐reported and not independently validated, and as such are subject to recall and reporting biases. Response bias inherently excluded some AHPs that may have met our grant funding or promotions criteria had they participated in the initial LAHP‐50 survey, though we identified and included additional programs through our scholarship metric, increasing the representativeness of the SCHOLAR cohort. Given the dynamic nature of the field, the age of the data we relied upon for analysis limits the generalizability of our specific benchmarks to current practice. However, the development of academic success occurs over the long‐term, and published data on academic hospitalist productivity are consistent with this slower time course.[8] Despite these limitations, our data inform the general topic of gauging performance of AHPs, underscoring the challenges of developing and applying metrics of success, and highlight the variability of performance on selected metrics even among a relatively small group of 17 programs.

In conclusion, we have created a method to quantify academic success that may be useful to academic hospitalists and their group leaders as they set targets for improvement in the field. Even among our SCHOLAR cohort, room for ongoing improvement in development of funded scholarship and a core of senior faculty exists. Further investigation into the unique features of successful groups will offer insight to leaders in academic hospital medicine regarding infrastructure and processes that should be embraced to raise the bar for all AHPs. In addition, efforts to further define and validate nontraditional approaches to scholarship that allow for successful promotion at AHPs would be informative. We view our work less as a singular approach to benchmarking standards for AHPs, and more a call to action to continue efforts to balance scholarly activity and broad professional development of academic hospitalists with increasing clinical demands.

The structure and function of academic hospital medicine programs (AHPs) has evolved significantly with the growth of hospital medicine.[1, 2, 3, 4] Many AHPs formed in response to regulatory and financial changes, which drove demand for increased trainee oversight, improved clinical efficiency, and growth in nonteaching services staffed by hospitalists. Differences in local organizational contexts and needs have contributed to great variability in AHP program design and operations. As AHPs have become more established, the need to engage academic hospitalists in scholarship and activities that support professional development and promotion has been recognized. Defining sustainable and successful positions for academic hospitalists is a priority called for by leaders in the field.[5, 6]

In this rapidly evolving context, AHPs have employed a variety of approaches to organizing clinical and academic faculty roles, without guiding evidence or consensus‐based performance benchmarks. A number of AHPs have achieved success along traditional academic metrics of research, scholarship, and education. Currently, it is not known whether specific approaches to AHP organization, structure, or definition of faculty roles are associated with achievement of more traditional markers of academic success.

The Academic Committee of the Society of Hospital Medicine (SHM), and the Academic Hospitalist Task Force of the Society of General Internal Medicine (SGIM) had separately initiated projects to explore characteristics associated with success in AHPs. In 2012, these organizations combined efforts to jointly develop and implement the SCHOLAR (SuCcessful HOspitaLists in Academics and Research) project. The goals were to identify successful AHPs using objective criteria, and to then study those groups in greater detail to generate insights that would be broadly relevant to the field. Efforts to clarify the factors within AHPs linked to success by traditional academic metrics will benefit hospitalists, their leaders, and key stakeholders striving to achieve optimal balance between clinical and academic roles. We describe the initial work of the SCHOLAR project, our definitions of academic success in AHPs, and the characteristics of a cohort of exemplary AHPs who achieved the highest levels on these metrics.

METHODS

RESULTS

Faculty Promotion

Figure 1 demonstrates the proportion of faculty at various academic ranks. The percent of faculty above the rank of assistant professor in the SCHOLAR programs exceeded those in the overall LAHP‐50 by 5% (17.9% vs 12.8%, P = 0.01). Of note, 6% of the hospitalists at AHPs in the SCHOLAR programs were on nonfaculty tracks.

Figure 1

DISCUSSION

Using a definition of academic success that incorporated metrics of grant funding, faculty promotion, and scholarly output, we identified a unique subset of successful AHPsthe SCHOLAR cohort. The programs represented in the SCHOLAR cohort were generally large and relatively mature. Despite this, the cohort consisted of mostly junior faculty, had a paucity of fellowship‐trained hospitalists, and not all reported grant funding.

Prior published work reported complementary findings.[6, 8, 9] A survey of 20 large, well‐established academic hospitalist programs in 2008 found that the majority of hospitalists were junior faculty with a limited publication portfolio. Of the 266 respondents in that study, 86% reported an academic rank at or below assistant professor; funding was not explored.[9] Our similar findings 4 years later add to this work by demonstrating trends over time, and suggest that progress toward creating successful pathways for academic advancement has been slow. In a 2012 survey of the SHM membership, 28% of hospitalists with academic appointments reported no current or future plans to engage in research.[8] These findings suggest that faculty in AHPs may define scholarship through nontraditional pathways, or in some cases choose not to pursue or prioritize scholarship altogether.

Our findings also add to the literature with regard to our assessment of funding, which was variable across the SCHOLAR group. The broad range of funding in the SCHOLAR programs for which we have data (grant dollars $0$15 million per program) suggests that opportunities to improve supported scholarship remain, even among a selected cohort of successful AHPs. The predominance of junior faculty in the SCHOLAR programs may be a reason for this variation. Junior faculty may be engaged in research with funding directed to senior mentors outside their AHP. Alternatively, they may pursue meaningful local hospital quality improvement or educational innovations not supported by external grants, or hold leadership roles in education, quality, or information technology that allow for advancement and promotion without external grant funding. As the scope and impact of these roles increases, senior leaders with alternate sources of support may rely less on research funds; this too may explain some of the differences. Our findings are congruent with results of a study that reviewed original research published by hospitalists, and concluded that the majority of hospitalist research was not externally funded.[8] Our approach for assessing grant funding by adjusting for FTE had the potential to inadvertently favor smaller well‐funded groups over larger ones; however, programs in our sample were similarly represented when ranked by funding/FTE or total grant dollars. As many successful AHPs do concentrate their research funding among a core of focused hospitalist researchers, our definition may not be the ideal metric for some programs.

We chose to define scholarship based on abstract output, rather than peer‐reviewed publications. Although this choice was necessary from a feasibility perspective, it may have excluded programs that prioritize peer‐reviewed publications over abstracts. Although we were unable to incorporate a search strategy to accurately and comprehensively track the publication output attributed specifically to hospitalist researchers and quantify it by program, others have since defined such an approach.[8] However, tracking abstracts theoretically allowed insights into a larger volume of innovative and creative work generated by top AHPs by potentially including work in the earlier stages of development.

We used a consensus‐based definition of success to define our SCHOLAR cohort. There are other ways to measure academic success, which if applied, may have yielded a different sample of programs. For example, over half of the original research articles published in the Journal of Hospital Medicine over a 7‐year span were generated from 5 academic centers.[8] This definition of success may be equally credible, though we note that 4 of these 5 programs were also included in the SCHOLAR cohort. We feel our broader approach was more reflective of the variety of pathways to success available to academic hospitalists. Before our metrics are applied as a benchmarking tool, however, they should ideally be combined with factors not measured in our study to ensure a more comprehensive or balanced reflection of academic success. Factors such as mentorship, level of hospitalist engagement,[10] prevalence of leadership opportunities, operational and fiscal infrastructure, and the impact of local quality, safety, and value efforts should be considered.

Comparison of successfully promoted faculty at AHPs across the country is inherently limited by the wide variation in promotion standards across different institutions; controlling for such differences was not possible with our methodology. For example, it appears that several programs with relatively few senior faculty may have met metrics leading to their inclusion in the SCHOLAR group because of their small program size. Future benchmarking efforts for promotion at AHPs should take scaling into account and consider both total number as well as percentage of senior faculty when evaluating success.

Our methodology has several limitations. Survey data were self‐reported and not independently validated, and as such are subject to recall and reporting biases. Response bias inherently excluded some AHPs that may have met our grant funding or promotions criteria had they participated in the initial LAHP‐50 survey, though we identified and included additional programs through our scholarship metric, increasing the representativeness of the SCHOLAR cohort. Given the dynamic nature of the field, the age of the data we relied upon for analysis limits the generalizability of our specific benchmarks to current practice. However, the development of academic success occurs over the long‐term, and published data on academic hospitalist productivity are consistent with this slower time course.[8] Despite these limitations, our data inform the general topic of gauging performance of AHPs, underscoring the challenges of developing and applying metrics of success, and highlight the variability of performance on selected metrics even among a relatively small group of 17 programs.

In conclusion, we have created a method to quantify academic success that may be useful to academic hospitalists and their group leaders as they set targets for improvement in the field. Even among our SCHOLAR cohort, room for ongoing improvement in development of funded scholarship and a core of senior faculty exists. Further investigation into the unique features of successful groups will offer insight to leaders in academic hospital medicine regarding infrastructure and processes that should be embraced to raise the bar for all AHPs. In addition, efforts to further define and validate nontraditional approaches to scholarship that allow for successful promotion at AHPs would be informative. We view our work less as a singular approach to benchmarking standards for AHPs, and more a call to action to continue efforts to balance scholarly activity and broad professional development of academic hospitalists with increasing clinical demands.

The structure and function of academic hospital medicine programs (AHPs) has evolved significantly with the growth of hospital medicine.[1, 2, 3, 4] Many AHPs formed in response to regulatory and financial changes, which drove demand for increased trainee oversight, improved clinical efficiency, and growth in nonteaching services staffed by hospitalists. Differences in local organizational contexts and needs have contributed to great variability in AHP program design and operations. As AHPs have become more established, the need to engage academic hospitalists in scholarship and activities that support professional development and promotion has been recognized. Defining sustainable and successful positions for academic hospitalists is a priority called for by leaders in the field.[5, 6]

In this rapidly evolving context, AHPs have employed a variety of approaches to organizing clinical and academic faculty roles, without guiding evidence or consensus‐based performance benchmarks. A number of AHPs have achieved success along traditional academic metrics of research, scholarship, and education. Currently, it is not known whether specific approaches to AHP organization, structure, or definition of faculty roles are associated with achievement of more traditional markers of academic success.

The Academic Committee of the Society of Hospital Medicine (SHM), and the Academic Hospitalist Task Force of the Society of General Internal Medicine (SGIM) had separately initiated projects to explore characteristics associated with success in AHPs. In 2012, these organizations combined efforts to jointly develop and implement the SCHOLAR (SuCcessful HOspitaLists in Academics and Research) project. The goals were to identify successful AHPs using objective criteria, and to then study those groups in greater detail to generate insights that would be broadly relevant to the field. Efforts to clarify the factors within AHPs linked to success by traditional academic metrics will benefit hospitalists, their leaders, and key stakeholders striving to achieve optimal balance between clinical and academic roles. We describe the initial work of the SCHOLAR project, our definitions of academic success in AHPs, and the characteristics of a cohort of exemplary AHPs who achieved the highest levels on these metrics.

METHODS

RESULTS

Faculty Promotion

Figure 1 demonstrates the proportion of faculty at various academic ranks. The percent of faculty above the rank of assistant professor in the SCHOLAR programs exceeded those in the overall LAHP‐50 by 5% (17.9% vs 12.8%, P = 0.01). Of note, 6% of the hospitalists at AHPs in the SCHOLAR programs were on nonfaculty tracks.

Figure 1

DISCUSSION

Using a definition of academic success that incorporated metrics of grant funding, faculty promotion, and scholarly output, we identified a unique subset of successful AHPsthe SCHOLAR cohort. The programs represented in the SCHOLAR cohort were generally large and relatively mature. Despite this, the cohort consisted of mostly junior faculty, had a paucity of fellowship‐trained hospitalists, and not all reported grant funding.

Prior published work reported complementary findings.[6, 8, 9] A survey of 20 large, well‐established academic hospitalist programs in 2008 found that the majority of hospitalists were junior faculty with a limited publication portfolio. Of the 266 respondents in that study, 86% reported an academic rank at or below assistant professor; funding was not explored.[9] Our similar findings 4 years later add to this work by demonstrating trends over time, and suggest that progress toward creating successful pathways for academic advancement has been slow. In a 2012 survey of the SHM membership, 28% of hospitalists with academic appointments reported no current or future plans to engage in research.[8] These findings suggest that faculty in AHPs may define scholarship through nontraditional pathways, or in some cases choose not to pursue or prioritize scholarship altogether.

Our findings also add to the literature with regard to our assessment of funding, which was variable across the SCHOLAR group. The broad range of funding in the SCHOLAR programs for which we have data (grant dollars $0$15 million per program) suggests that opportunities to improve supported scholarship remain, even among a selected cohort of successful AHPs. The predominance of junior faculty in the SCHOLAR programs may be a reason for this variation. Junior faculty may be engaged in research with funding directed to senior mentors outside their AHP. Alternatively, they may pursue meaningful local hospital quality improvement or educational innovations not supported by external grants, or hold leadership roles in education, quality, or information technology that allow for advancement and promotion without external grant funding. As the scope and impact of these roles increases, senior leaders with alternate sources of support may rely less on research funds; this too may explain some of the differences. Our findings are congruent with results of a study that reviewed original research published by hospitalists, and concluded that the majority of hospitalist research was not externally funded.[8] Our approach for assessing grant funding by adjusting for FTE had the potential to inadvertently favor smaller well‐funded groups over larger ones; however, programs in our sample were similarly represented when ranked by funding/FTE or total grant dollars. As many successful AHPs do concentrate their research funding among a core of focused hospitalist researchers, our definition may not be the ideal metric for some programs.

We chose to define scholarship based on abstract output, rather than peer‐reviewed publications. Although this choice was necessary from a feasibility perspective, it may have excluded programs that prioritize peer‐reviewed publications over abstracts. Although we were unable to incorporate a search strategy to accurately and comprehensively track the publication output attributed specifically to hospitalist researchers and quantify it by program, others have since defined such an approach.[8] However, tracking abstracts theoretically allowed insights into a larger volume of innovative and creative work generated by top AHPs by potentially including work in the earlier stages of development.

We used a consensus‐based definition of success to define our SCHOLAR cohort. There are other ways to measure academic success, which if applied, may have yielded a different sample of programs. For example, over half of the original research articles published in the Journal of Hospital Medicine over a 7‐year span were generated from 5 academic centers.[8] This definition of success may be equally credible, though we note that 4 of these 5 programs were also included in the SCHOLAR cohort. We feel our broader approach was more reflective of the variety of pathways to success available to academic hospitalists. Before our metrics are applied as a benchmarking tool, however, they should ideally be combined with factors not measured in our study to ensure a more comprehensive or balanced reflection of academic success. Factors such as mentorship, level of hospitalist engagement,[10] prevalence of leadership opportunities, operational and fiscal infrastructure, and the impact of local quality, safety, and value efforts should be considered.

Comparison of successfully promoted faculty at AHPs across the country is inherently limited by the wide variation in promotion standards across different institutions; controlling for such differences was not possible with our methodology. For example, it appears that several programs with relatively few senior faculty may have met metrics leading to their inclusion in the SCHOLAR group because of their small program size. Future benchmarking efforts for promotion at AHPs should take scaling into account and consider both total number as well as percentage of senior faculty when evaluating success.

Our methodology has several limitations. Survey data were self‐reported and not independently validated, and as such are subject to recall and reporting biases. Response bias inherently excluded some AHPs that may have met our grant funding or promotions criteria had they participated in the initial LAHP‐50 survey, though we identified and included additional programs through our scholarship metric, increasing the representativeness of the SCHOLAR cohort. Given the dynamic nature of the field, the age of the data we relied upon for analysis limits the generalizability of our specific benchmarks to current practice. However, the development of academic success occurs over the long‐term, and published data on academic hospitalist productivity are consistent with this slower time course.[8] Despite these limitations, our data inform the general topic of gauging performance of AHPs, underscoring the challenges of developing and applying metrics of success, and highlight the variability of performance on selected metrics even among a relatively small group of 17 programs.

In conclusion, we have created a method to quantify academic success that may be useful to academic hospitalists and their group leaders as they set targets for improvement in the field. Even among our SCHOLAR cohort, room for ongoing improvement in development of funded scholarship and a core of senior faculty exists. Further investigation into the unique features of successful groups will offer insight to leaders in academic hospital medicine regarding infrastructure and processes that should be embraced to raise the bar for all AHPs. In addition, efforts to further define and validate nontraditional approaches to scholarship that allow for successful promotion at AHPs would be informative. We view our work less as a singular approach to benchmarking standards for AHPs, and more a call to action to continue efforts to balance scholarly activity and broad professional development of academic hospitalists with increasing clinical demands.

In 2003, the Accreditation Council for Graduate Medical Education (ACGME) announced the first in a series of guidelines related to the regulation and oversight of residency training.1 The initial iteration specifically focused on the total and consecutive numbers of duty hours worked by trainees. These limitations began a new era of shift work in internal medicine residency training. With decreases in housestaff admitting capacity, clinical work has frequently been offloaded to non‐teaching or attending‐only services, increasing the demand for hospitalists to fill the void in physician‐staffed care in the hospital.2, 3 Since the implementation of the 2003 ACGME guidelines and a growing focus on patient safety, there has been increased study of, and call for, oversight of trainees in medicine; among these was the 2008 Institute of Medicine report,4 calling for 24/7 attending‐level supervision. The updated ACGME requirements,5 effective July 1, 2011, mandate enhanced on‐site supervision of trainee physicians. These new regulations not only define varying levels of supervision for trainees, including direct supervision with the physical presence of a supervisor and the degree of availability of said supervisor, they also describe ensuring the quality of supervision provided.5 While continuous attending‐level supervision is not yet mandated, many residency programs look to their academic hospitalists to fill the supervisory void, particularly at night. However, what specific roles hospitalists play in the nighttime supervision of trainees or the impact of this supervision remains unclear. To date, no study has examined a broad sample of hospitalist programs in teaching hospitals and the types of resident oversight they provide. We aimed to describe the current state of academic hospitalists in the clinical supervision of housestaff, specifically during the overnight period, and hospitalist perceptions of how the new ACGME requirements would impact traineehospitalist interactions.

METHODS

The Housestaff Oversight Subcommittee, a working group of the Society of General Internal Medicine (SGIM) Academic Hospitalist Task Force, surveyed a sample of academic hospitalist program leaders to assess the current status of trainee supervision performed by hospitalists. Programs were considered academic if they were located in the primary hospital of a residency that participates in the National Resident Matching Program for Internal Medicine. To obtain a broad geographic spectrum of academic hospitalist programs, all programs, both university and community‐based, in 4 states and 2 metropolitan regions were sampled: Washington, Oregon, Texas, Maryland, and the Philadelphia and Chicago metropolitan areas. Hospitalist program leaders were identified by members of the Taskforce using individual program websites and by querying departmental leadership at eligible teaching hospitals. Respondents were contacted by e‐mail for participation. None of the authors of the manuscript were participants in the survey.

The survey was developed by consensus of the working group after reviewing the salient literature and included additional questions queried to internal medicine program directors.6 The 19‐item SurveyMonkey instrument included questions about hospitalists' role in trainees' education and evaluation. A Likert‐type scale was used to assess perceptions regarding the impact of on‐site hospitalist supervision on trainee autonomy and hospitalist workload (1 = strongly disagree to 5 = strongly agree). Descriptive statistics were performed and, where appropriate, t test and Fisher's exact test were performed to identify associations between program characteristics and perceptions. Stata SE was used (STATA Corp, College Station, TX) for all statistical analysis.

RESULTS

The survey was sent to 47 individuals identified as likely hospitalist program leaders and completed by 41 individuals (87%). However, 7 respondents turned out not to be program leaders and were therefore excluded, resulting in a 72% (34/47) survey response rate.

The programs for which we did not obtain responses were similar to respondent programs, and did not include a larger proportion of community‐based programs or overrepresent a specific geographic region. Twenty‐five (73%) of the 34 hospitalist program leaders were male, with an average age of 44.3 years, and an average of 12 years post‐residency training (range, 530 years). They reported leading groups with an average of 18 full‐time equivalent (FTE) faculty (range, 350 persons).

DISCUSSION

This study provides important information about how academic hospitalists currently contribute to the supervision of internal medicine residents. While academic hospitalist groups frequently have faculty providing clinical care on‐site at night, and often hospitalists provide overnight supervision of internal medicine trainees, formal supervision of trainees is not uniform, and few hospitalists groups have a mechanism to provide training or faculty development on how to effectively supervise resident trainees. Hospitalist leaders expressed concerns that creating additional formal overnight supervisory responsibilities may add to an already burdened overnight hospitalist. Formalizing this supervisory role, including explicit role definitions and faculty training for trainee supervision, is necessary.

Though our sample size is small, we captured a diverse geographic range of both university and community‐based academic hospitalist programs by surveying group leaders in several distinct regions. We are unable to comment on differences between responding and non‐responding hospitalist programs, but there does not appear to be a systematic difference between these groups.

Our findings are consistent with work describing a lack of structured conceptual frameworks in effectively supervising trainees,7, 8 and also, at times, nebulous expectations for hospitalist faculty. We found that the existence of a formal supervisory policy within the associated residency program, as well as defined roles for hospitalists, increases the likelihood of positive perceptions of the new ACGME supervisory recommendations. However, the existence of these requirements does not mean that all programs are capable of following them. While additional discussion is required to best delineate a formal overnight hospitalist role in trainee supervision, clearly defining expectations for both faculty and trainees, and their interactions, may alleviate the struggles that exist in programs with ill‐defined roles for hospitalist faculty supervision. While faculty duty hours standards do not exist, additional duties of nighttime coverage for hospitalists suggests that close attention should be paid to burn‐out.9 Faculty development on nighttime supervision and teaching may help maximize both learning and patient care efficiency, and provide a framework for this often unstructured educational time.

In 2003, the Accreditation Council for Graduate Medical Education (ACGME) announced the first in a series of guidelines related to the regulation and oversight of residency training.1 The initial iteration specifically focused on the total and consecutive numbers of duty hours worked by trainees. These limitations began a new era of shift work in internal medicine residency training. With decreases in housestaff admitting capacity, clinical work has frequently been offloaded to non‐teaching or attending‐only services, increasing the demand for hospitalists to fill the void in physician‐staffed care in the hospital.2, 3 Since the implementation of the 2003 ACGME guidelines and a growing focus on patient safety, there has been increased study of, and call for, oversight of trainees in medicine; among these was the 2008 Institute of Medicine report,4 calling for 24/7 attending‐level supervision. The updated ACGME requirements,5 effective July 1, 2011, mandate enhanced on‐site supervision of trainee physicians. These new regulations not only define varying levels of supervision for trainees, including direct supervision with the physical presence of a supervisor and the degree of availability of said supervisor, they also describe ensuring the quality of supervision provided.5 While continuous attending‐level supervision is not yet mandated, many residency programs look to their academic hospitalists to fill the supervisory void, particularly at night. However, what specific roles hospitalists play in the nighttime supervision of trainees or the impact of this supervision remains unclear. To date, no study has examined a broad sample of hospitalist programs in teaching hospitals and the types of resident oversight they provide. We aimed to describe the current state of academic hospitalists in the clinical supervision of housestaff, specifically during the overnight period, and hospitalist perceptions of how the new ACGME requirements would impact traineehospitalist interactions.

METHODS

The Housestaff Oversight Subcommittee, a working group of the Society of General Internal Medicine (SGIM) Academic Hospitalist Task Force, surveyed a sample of academic hospitalist program leaders to assess the current status of trainee supervision performed by hospitalists. Programs were considered academic if they were located in the primary hospital of a residency that participates in the National Resident Matching Program for Internal Medicine. To obtain a broad geographic spectrum of academic hospitalist programs, all programs, both university and community‐based, in 4 states and 2 metropolitan regions were sampled: Washington, Oregon, Texas, Maryland, and the Philadelphia and Chicago metropolitan areas. Hospitalist program leaders were identified by members of the Taskforce using individual program websites and by querying departmental leadership at eligible teaching hospitals. Respondents were contacted by e‐mail for participation. None of the authors of the manuscript were participants in the survey.

The survey was developed by consensus of the working group after reviewing the salient literature and included additional questions queried to internal medicine program directors.6 The 19‐item SurveyMonkey instrument included questions about hospitalists' role in trainees' education and evaluation. A Likert‐type scale was used to assess perceptions regarding the impact of on‐site hospitalist supervision on trainee autonomy and hospitalist workload (1 = strongly disagree to 5 = strongly agree). Descriptive statistics were performed and, where appropriate, t test and Fisher's exact test were performed to identify associations between program characteristics and perceptions. Stata SE was used (STATA Corp, College Station, TX) for all statistical analysis.

RESULTS

The survey was sent to 47 individuals identified as likely hospitalist program leaders and completed by 41 individuals (87%). However, 7 respondents turned out not to be program leaders and were therefore excluded, resulting in a 72% (34/47) survey response rate.

The programs for which we did not obtain responses were similar to respondent programs, and did not include a larger proportion of community‐based programs or overrepresent a specific geographic region. Twenty‐five (73%) of the 34 hospitalist program leaders were male, with an average age of 44.3 years, and an average of 12 years post‐residency training (range, 530 years). They reported leading groups with an average of 18 full‐time equivalent (FTE) faculty (range, 350 persons).

DISCUSSION

This study provides important information about how academic hospitalists currently contribute to the supervision of internal medicine residents. While academic hospitalist groups frequently have faculty providing clinical care on‐site at night, and often hospitalists provide overnight supervision of internal medicine trainees, formal supervision of trainees is not uniform, and few hospitalists groups have a mechanism to provide training or faculty development on how to effectively supervise resident trainees. Hospitalist leaders expressed concerns that creating additional formal overnight supervisory responsibilities may add to an already burdened overnight hospitalist. Formalizing this supervisory role, including explicit role definitions and faculty training for trainee supervision, is necessary.

Though our sample size is small, we captured a diverse geographic range of both university and community‐based academic hospitalist programs by surveying group leaders in several distinct regions. We are unable to comment on differences between responding and non‐responding hospitalist programs, but there does not appear to be a systematic difference between these groups.

Our findings are consistent with work describing a lack of structured conceptual frameworks in effectively supervising trainees,7, 8 and also, at times, nebulous expectations for hospitalist faculty. We found that the existence of a formal supervisory policy within the associated residency program, as well as defined roles for hospitalists, increases the likelihood of positive perceptions of the new ACGME supervisory recommendations. However, the existence of these requirements does not mean that all programs are capable of following them. While additional discussion is required to best delineate a formal overnight hospitalist role in trainee supervision, clearly defining expectations for both faculty and trainees, and their interactions, may alleviate the struggles that exist in programs with ill‐defined roles for hospitalist faculty supervision. While faculty duty hours standards do not exist, additional duties of nighttime coverage for hospitalists suggests that close attention should be paid to burn‐out.9 Faculty development on nighttime supervision and teaching may help maximize both learning and patient care efficiency, and provide a framework for this often unstructured educational time.

In 2003, the Accreditation Council for Graduate Medical Education (ACGME) announced the first in a series of guidelines related to the regulation and oversight of residency training.1 The initial iteration specifically focused on the total and consecutive numbers of duty hours worked by trainees. These limitations began a new era of shift work in internal medicine residency training. With decreases in housestaff admitting capacity, clinical work has frequently been offloaded to non‐teaching or attending‐only services, increasing the demand for hospitalists to fill the void in physician‐staffed care in the hospital.2, 3 Since the implementation of the 2003 ACGME guidelines and a growing focus on patient safety, there has been increased study of, and call for, oversight of trainees in medicine; among these was the 2008 Institute of Medicine report,4 calling for 24/7 attending‐level supervision. The updated ACGME requirements,5 effective July 1, 2011, mandate enhanced on‐site supervision of trainee physicians. These new regulations not only define varying levels of supervision for trainees, including direct supervision with the physical presence of a supervisor and the degree of availability of said supervisor, they also describe ensuring the quality of supervision provided.5 While continuous attending‐level supervision is not yet mandated, many residency programs look to their academic hospitalists to fill the supervisory void, particularly at night. However, what specific roles hospitalists play in the nighttime supervision of trainees or the impact of this supervision remains unclear. To date, no study has examined a broad sample of hospitalist programs in teaching hospitals and the types of resident oversight they provide. We aimed to describe the current state of academic hospitalists in the clinical supervision of housestaff, specifically during the overnight period, and hospitalist perceptions of how the new ACGME requirements would impact traineehospitalist interactions.

METHODS

The Housestaff Oversight Subcommittee, a working group of the Society of General Internal Medicine (SGIM) Academic Hospitalist Task Force, surveyed a sample of academic hospitalist program leaders to assess the current status of trainee supervision performed by hospitalists. Programs were considered academic if they were located in the primary hospital of a residency that participates in the National Resident Matching Program for Internal Medicine. To obtain a broad geographic spectrum of academic hospitalist programs, all programs, both university and community‐based, in 4 states and 2 metropolitan regions were sampled: Washington, Oregon, Texas, Maryland, and the Philadelphia and Chicago metropolitan areas. Hospitalist program leaders were identified by members of the Taskforce using individual program websites and by querying departmental leadership at eligible teaching hospitals. Respondents were contacted by e‐mail for participation. None of the authors of the manuscript were participants in the survey.

The survey was developed by consensus of the working group after reviewing the salient literature and included additional questions queried to internal medicine program directors.6 The 19‐item SurveyMonkey instrument included questions about hospitalists' role in trainees' education and evaluation. A Likert‐type scale was used to assess perceptions regarding the impact of on‐site hospitalist supervision on trainee autonomy and hospitalist workload (1 = strongly disagree to 5 = strongly agree). Descriptive statistics were performed and, where appropriate, t test and Fisher's exact test were performed to identify associations between program characteristics and perceptions. Stata SE was used (STATA Corp, College Station, TX) for all statistical analysis.

RESULTS

The survey was sent to 47 individuals identified as likely hospitalist program leaders and completed by 41 individuals (87%). However, 7 respondents turned out not to be program leaders and were therefore excluded, resulting in a 72% (34/47) survey response rate.

The programs for which we did not obtain responses were similar to respondent programs, and did not include a larger proportion of community‐based programs or overrepresent a specific geographic region. Twenty‐five (73%) of the 34 hospitalist program leaders were male, with an average age of 44.3 years, and an average of 12 years post‐residency training (range, 530 years). They reported leading groups with an average of 18 full‐time equivalent (FTE) faculty (range, 350 persons).

DISCUSSION

This study provides important information about how academic hospitalists currently contribute to the supervision of internal medicine residents. While academic hospitalist groups frequently have faculty providing clinical care on‐site at night, and often hospitalists provide overnight supervision of internal medicine trainees, formal supervision of trainees is not uniform, and few hospitalists groups have a mechanism to provide training or faculty development on how to effectively supervise resident trainees. Hospitalist leaders expressed concerns that creating additional formal overnight supervisory responsibilities may add to an already burdened overnight hospitalist. Formalizing this supervisory role, including explicit role definitions and faculty training for trainee supervision, is necessary.

Though our sample size is small, we captured a diverse geographic range of both university and community‐based academic hospitalist programs by surveying group leaders in several distinct regions. We are unable to comment on differences between responding and non‐responding hospitalist programs, but there does not appear to be a systematic difference between these groups.

Our findings are consistent with work describing a lack of structured conceptual frameworks in effectively supervising trainees,7, 8 and also, at times, nebulous expectations for hospitalist faculty. We found that the existence of a formal supervisory policy within the associated residency program, as well as defined roles for hospitalists, increases the likelihood of positive perceptions of the new ACGME supervisory recommendations. However, the existence of these requirements does not mean that all programs are capable of following them. While additional discussion is required to best delineate a formal overnight hospitalist role in trainee supervision, clearly defining expectations for both faculty and trainees, and their interactions, may alleviate the struggles that exist in programs with ill‐defined roles for hospitalist faculty supervision. While faculty duty hours standards do not exist, additional duties of nighttime coverage for hospitalists suggests that close attention should be paid to burn‐out.9 Faculty development on nighttime supervision and teaching may help maximize both learning and patient care efficiency, and provide a framework for this often unstructured educational time.

Patient safety can be understood in terms of the Swiss cheese model of systems accidents. This model implies that many holes must align before an adverse event occurs.1 The limitations on work hours instituted by the Accreditation Council for Graduate Medical Education (ACGME)2 sought to close one hole by reducing fatigue in residents. As programs comply with these regulations, new interventions are being implemented to limit resident hours. This has resulted in more handoffs of care and therefore less continuity. The ultimate result may be to increase patient care errors by opening up new holes, the opposite of the stated goal of this reform.

Some residency programs have reported on their experience with hour reductions, giving insight into residents' perceptions on the benefits and drawbacks of such interventions. Residents have reported concern about continuity of care after such interventions.37 However, some residents believed they provided better patient care after the interventions to reduce hours.8, 9 Few studies have actually documented changes in the incidence of adverse events or errors as a result of work hour limitations.10 One study conducted prior to implementation of the ACGME work hour rules demonstrated more complications in internal medicine patients after New York's Code 405 (a state regulation that limited resident work hours, similar to the ACGME rules) was implemented.11 In contrast, another study showed that errors committed by interns were reduced with scheduling changes that resulted in shorter shifts and reduced hours.12

Because residents are on the front lines of patient care, they are uniquely positioned to provide insight into the impact of the work hour rules on patient safety. We conducted this study to more fully understand the effect of the ACGME work hour limitations and other possible factors on patient care errors from the perspectives of internal medicine residents.

METHODS

Design

We conducted 4 focus groups in total (Appendix 1). The first 3 focus groups followed the same focus group guide, developed after a literature review. Focus groups 1 and 2 were conducted at the university‐based program. Focus group 3 was conducted at the community teaching hospitalaffiliated program. The first 3 focus groups were analyzed before the fourth focus group was conducted. A new focus group guide was developed for the fourth focus group to further explore themes identified in the first 3 focus groups (Fig. 1 and Appendix 2). The fourth focus group was conducted at the program affiliated with a freestanding medical college. All focus groups were audiotaped and transcribed verbatim. Each lasted approximately 90‐120 minutes.

Figure 1

RESULTS

Twenty‐eight residents participated. Some of these residents had experience in the prework hour era, and some did not. Average age was 28 years (range 26‐33 years); 18 were women, and 11 were interns (Table 2). The focus groups ranged in size from 5 to 9. A sample of the codes and their definitions can be found in Table 3.

The Model

The model (Fig. 2) illustrates resident‐perceived contributors to patient care mistakes related to the ACGME work hour rules. These contributors are in the center circle. They include fatigue, inexperience, sign‐out, not knowing their own patients well enough, entropy (which we defined as the amount of chaos in the system), and workload. They are not listed in order of importance. The boxes outside the circle are consequences of the ACGME work hour rules and their perceived impact on the contributors to patient care mistakes. At the top are the intended consequences, that is the specific goals of the ACGME: less resident time in the hospital (ie, reduced hours) and improved well‐being.17 At the bottom are the unintended consequences: more patient care discontinuity and compliance with the work hour rules becoming a goal equally important to providing high‐quality patient care. Of these 4 consequences, only improved well‐being was viewed by the residents as decreasing patient care mistakes. The other consequences were cited by residents as sometimes increasing patient care errors. Because of the complexity of the model, several factors not directly related to resident work hours were identified in the analysis but are not shown in the model. They include faculty involvement and team work (usually positive influences), nurses and information technology (could be positive or negative), and late‐night/early‐morning hours (negative).

Figure 2

The quotations below illustrate the relationships between the consequences of the work hour rules, resident‐perceived contributors to patient care mistakes, and actual patient care.

DISCUSSION

We conducted this study to better understand why internal medicine residents thought patient care mistakes occurred; we were particularly interested in how they perceived the impact of certain aspects of the ACGME work hour rules on patient care mistakes. Designing systems that achieve compliance with the work hour rules while minimizing patient risk can best be accomplished by fully understanding why errors occur.

Our study revealed that in the opinion of some interns and residents, the work hour rules had consequences for patient care. Like any intervention, this one had both intended and unintended consequences.18 The ACGME has stated that improvement in residents' quality of life was an intended consequence,17 and the participants in our study reported that this had occurred. Despite uncertainty about the overall impact on patient outcomes, residents were glad to have more time away from the hospital.

Our respondents reported that not knowing patients well was a factor that contributed to patient care errors. It is intuitive that working fewer hours often results in more handoffs of care,19 a situation characterized by not knowing patients well. However, residents also identified not getting to know their own patients well as a factor that led to patient care mistakes because of (1) incomplete knowledge of a patient's status, (2) delays in diagnosis, and (3) errors in management. They also described feelings of professional disappointment and frustration at not being able to perform certain aspects of patient care (eg, family meetings) because of the hour limits and the inflexibility of the rules. As we strive to redefine professionalism in the setting of reduced work hours,20 this phenomenon should be addressed.

Sign‐out was identified as another contributor to patient care errors. The effectiveness of sign‐outs is a concern across medicine, and the Joint Commission on Accreditation of Healthcare Organizations made sign‐out procedures one of its priority areas in 2006.21 Much has been written about resident sign‐out, emphasizing the relationship between poor‐quality sign‐outs and patient safety.19, 22 However, barriers to effective sign‐out processes persist,23 even though standardized sign‐out strategies have been described.24, 25 Even in a rigorous study of work hours and patient safety, the computerized sign‐out template for the residents was rarely used.12 Cross‐coverage, or the patient care that occurs after sign‐out is complete, has also been linked to a greater likelihood of adverse events.26

Several factors not related to resident work hours were noted to often mitigate patient care mistakes. Physician teamwork, nursing, information technology (eg, computerized medical records), and faculty supervision were the most prominent. For example, the information technology available at the VA hospitals often helped to facilitate patient care, but it also provided an overwhelming amount of information to sift through. It was clear that the influence of some of these factors varied from institution to institution, reflecting the cultures of different programs.

Our results are consistent with those reported from previous studies. Striking a balance between preventing resident fatigue and preserving continuity of care has been debated since the ACGME announced changes to resident work hour limits.27 Resident quality of life generally improves and fatigue decreases with work hour limits in place,28 but patient safety remains a concern.10 Our findings corroborate the benefits of improved resident well‐being and the persistent concerns about patient safety, identified in a recently published study at a different institution.29 However, our findings expand on those reported in the literature by offering additional empirical evidence, albeit qualitative, about the way that residents see the relationships among the consequences of work hour rules, resident‐reported contributors to patient care mistakes, and the mistakes themselves.

Our study should be interpreted in the context of several limitations. First, the use of qualitative methods did not allow us to generalize or quantify our findings. However, we purposely included 3 diverse institutions with differing responses to the work hour rules to enhance the external validity of our findings. Second, the last focus group was conducted a year after the first 3; by that point, the work hour rules had been in place for 20 months. We believe that this was both a strength and a limitation because it allowed us to gain a perspective after some of the initial growing pains were over. This time lag also allowed for analysis of the first 3 transcripts so we could revise the focus group guide and ultimately determine that thematic saturation had occurred. In addition, few of our questions were phrased to evaluate the ACGME rules; instead, they asked about links among discontinuity, scheduling, fatigue, and patient care. We therefore believe that even residents who were not in the programs before the work hour rules began were still able to knowledgeably participate in the conversation. One question directly referable to the ACGME rules asked residents to reflect on problems arising from them. This could have led residents to only reflect on the problems associated with the rules. However, in all 4 focus groups, residents commented on the positive impact of improved well‐being resulting from the work hour rules. This led us to believe the respondents felt they could voice their favorable feelings as well as their unfavorable feelings about the rules. An additional limitation is that the agreement between coders was only 45%. It is important to realize that assessing coding agreement in qualitative work is quite difficult because it is often difficult to assign a single code to a section of text. When the coders discussed a disagreement, it was almost always the case that the difference was subtle and that the coding of either investigator would made sense for that text. Finally, our results are based on the participation of 28 residents. To be certain we were not representing the opinions of only a few people, we presented iterations of this model to faculty and resident groups for their feedback. Importantly, the residents offered no substantial changes or criticisms of the model.

Limitations notwithstanding, we believe our findings have important policy implications. First, despite work hours successfully being reduced, residents perceived no decrease in the amount of work they did. This resulted in higher workload and more entropy, suggesting that residency programs may need to carefully evaluate the patient care responsibility carried by residents. Second, continued effort to educate residents to provide effective sign‐out is needed. As one participant pointed out, residency offers a unique opportunity to learn to manage discontinuity in a controlled setting. Another educational opportunity is the chance to teach physician teamwork. Participants believed that effective teamwork could ameliorate some of the discontinuity in patient care. This teamwork training should include faculty as well, although further work is needed to define how faculty can best add to patient continuity while still fostering resident autonomy. Finally, the impact of work hour rules on the professional development of residents should be further explored.

In conclusion, we have proposed a model to explain the major resident‐reported contributors to patient care mistakes with respect to resident work hour rules. Our results help to clarify the next steps needed: testing the proposed relationships between the factors and patient care mistakes and rigorously evaluating solutions that minimize the impact of these factors. Returning to the Swiss cheese framework for describing systems accidents, our results suggest that although resident work hour reductions may have sufficiently filled the hole caused by resident fatigue, other gaps may have actually widened as a result of the systems put into place to achieve compliance. Continued vigilance is therefore necessary to both identify the additional holes likely to appear and, more importantly, effectively close those holes before patient harm occurs.

FOCUS GROUP GUIDE (4TH FOCUS GROUP)

The IOM definition of a medical error is the failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim.

• 1

  Please describe the call structure at each institution where you do ward months (eg, non‐ICU months).

• 2

  What are some settings at the hospitals where you work where you have seen medical errors, mistakes, or adverse outcomes?

• 3

  How do you think that other nurses influence the occurrence of medical errors, mistakes, or adverse outcomes?

   

  • a

    Clerks?

  • b

    Other ancillary staff?

  • 4

    How would you describe the responsibilities of a cross‐covering resident or intern?

  • 5

    How do you think continuity of care impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

     

    • a

      What role do sign‐outs have?

    • 6

      How do you think that fatigue impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

    • 7

      How do you think that technology such as computerized physician order entry impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

       

      • a

        Electronic medical records?

      • b

        Palm pilots?

      • c

        Is there such a thing as too much information?

      • 8

        How do you think that experience (or inexperience) impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

      • 9

        Please describe how attendings supervise you when you are on a ward team. How do you think that attending supervision impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

         

        • a

          What about resident supervision of interns?

        • b

          What is the ideal role of an attending on a team?

        • c

          Can you think of a time when having attending input changed the plans or the course of a patient in a major way, good, bad, or neutral?

        • 10

          How do you think that time of day impacts patient care in terms of in terms of medical errors, mistakes, or adverse outcomes?

           

          • a

            How comfortable do you feel calling for help at night? What makes you more or less likely to do it (personal attributes of person to be called, situation, etc.)?

          • 11

            What do you think is an ideal workload? (eg, How many complex patients are typical of your hospitals?) Does that vary from the VA to St. Joe's to Froedtert? How many patients should be admitted in 1 night by an intern? How many should an intern have ongoing responsibility for? Is there such a thing as too few patients?

          • 12

            If one of your family members were to admitted to your hospital at night with a life‐threatening condition, which situation would you prefer for their care (all other things being equal): admission by night float with handoff to a new but well‐rested resident or admission by a resident who then continues to care for that family member the next day but has been awake for 24 hours, admitting and cross‐covering other patients? Why?

          • 13

            What do you think was the intent of the ACGME rules? Do you think that those goals have been accomplished? Why or why not? How have they affected you as residents? How do you think that the ACGME work hour rules have influenced patient care?

           

Patient safety can be understood in terms of the Swiss cheese model of systems accidents. This model implies that many holes must align before an adverse event occurs.1 The limitations on work hours instituted by the Accreditation Council for Graduate Medical Education (ACGME)2 sought to close one hole by reducing fatigue in residents. As programs comply with these regulations, new interventions are being implemented to limit resident hours. This has resulted in more handoffs of care and therefore less continuity. The ultimate result may be to increase patient care errors by opening up new holes, the opposite of the stated goal of this reform.

Some residency programs have reported on their experience with hour reductions, giving insight into residents' perceptions on the benefits and drawbacks of such interventions. Residents have reported concern about continuity of care after such interventions.37 However, some residents believed they provided better patient care after the interventions to reduce hours.8, 9 Few studies have actually documented changes in the incidence of adverse events or errors as a result of work hour limitations.10 One study conducted prior to implementation of the ACGME work hour rules demonstrated more complications in internal medicine patients after New York's Code 405 (a state regulation that limited resident work hours, similar to the ACGME rules) was implemented.11 In contrast, another study showed that errors committed by interns were reduced with scheduling changes that resulted in shorter shifts and reduced hours.12

Because residents are on the front lines of patient care, they are uniquely positioned to provide insight into the impact of the work hour rules on patient safety. We conducted this study to more fully understand the effect of the ACGME work hour limitations and other possible factors on patient care errors from the perspectives of internal medicine residents.

METHODS

Design

We conducted 4 focus groups in total (Appendix 1). The first 3 focus groups followed the same focus group guide, developed after a literature review. Focus groups 1 and 2 were conducted at the university‐based program. Focus group 3 was conducted at the community teaching hospitalaffiliated program. The first 3 focus groups were analyzed before the fourth focus group was conducted. A new focus group guide was developed for the fourth focus group to further explore themes identified in the first 3 focus groups (Fig. 1 and Appendix 2). The fourth focus group was conducted at the program affiliated with a freestanding medical college. All focus groups were audiotaped and transcribed verbatim. Each lasted approximately 90‐120 minutes.

Figure 1

RESULTS

Twenty‐eight residents participated. Some of these residents had experience in the prework hour era, and some did not. Average age was 28 years (range 26‐33 years); 18 were women, and 11 were interns (Table 2). The focus groups ranged in size from 5 to 9. A sample of the codes and their definitions can be found in Table 3.

The Model

The model (Fig. 2) illustrates resident‐perceived contributors to patient care mistakes related to the ACGME work hour rules. These contributors are in the center circle. They include fatigue, inexperience, sign‐out, not knowing their own patients well enough, entropy (which we defined as the amount of chaos in the system), and workload. They are not listed in order of importance. The boxes outside the circle are consequences of the ACGME work hour rules and their perceived impact on the contributors to patient care mistakes. At the top are the intended consequences, that is the specific goals of the ACGME: less resident time in the hospital (ie, reduced hours) and improved well‐being.17 At the bottom are the unintended consequences: more patient care discontinuity and compliance with the work hour rules becoming a goal equally important to providing high‐quality patient care. Of these 4 consequences, only improved well‐being was viewed by the residents as decreasing patient care mistakes. The other consequences were cited by residents as sometimes increasing patient care errors. Because of the complexity of the model, several factors not directly related to resident work hours were identified in the analysis but are not shown in the model. They include faculty involvement and team work (usually positive influences), nurses and information technology (could be positive or negative), and late‐night/early‐morning hours (negative).

Figure 2

The quotations below illustrate the relationships between the consequences of the work hour rules, resident‐perceived contributors to patient care mistakes, and actual patient care.

DISCUSSION

We conducted this study to better understand why internal medicine residents thought patient care mistakes occurred; we were particularly interested in how they perceived the impact of certain aspects of the ACGME work hour rules on patient care mistakes. Designing systems that achieve compliance with the work hour rules while minimizing patient risk can best be accomplished by fully understanding why errors occur.

Our study revealed that in the opinion of some interns and residents, the work hour rules had consequences for patient care. Like any intervention, this one had both intended and unintended consequences.18 The ACGME has stated that improvement in residents' quality of life was an intended consequence,17 and the participants in our study reported that this had occurred. Despite uncertainty about the overall impact on patient outcomes, residents were glad to have more time away from the hospital.

Our respondents reported that not knowing patients well was a factor that contributed to patient care errors. It is intuitive that working fewer hours often results in more handoffs of care,19 a situation characterized by not knowing patients well. However, residents also identified not getting to know their own patients well as a factor that led to patient care mistakes because of (1) incomplete knowledge of a patient's status, (2) delays in diagnosis, and (3) errors in management. They also described feelings of professional disappointment and frustration at not being able to perform certain aspects of patient care (eg, family meetings) because of the hour limits and the inflexibility of the rules. As we strive to redefine professionalism in the setting of reduced work hours,20 this phenomenon should be addressed.

Sign‐out was identified as another contributor to patient care errors. The effectiveness of sign‐outs is a concern across medicine, and the Joint Commission on Accreditation of Healthcare Organizations made sign‐out procedures one of its priority areas in 2006.21 Much has been written about resident sign‐out, emphasizing the relationship between poor‐quality sign‐outs and patient safety.19, 22 However, barriers to effective sign‐out processes persist,23 even though standardized sign‐out strategies have been described.24, 25 Even in a rigorous study of work hours and patient safety, the computerized sign‐out template for the residents was rarely used.12 Cross‐coverage, or the patient care that occurs after sign‐out is complete, has also been linked to a greater likelihood of adverse events.26

Several factors not related to resident work hours were noted to often mitigate patient care mistakes. Physician teamwork, nursing, information technology (eg, computerized medical records), and faculty supervision were the most prominent. For example, the information technology available at the VA hospitals often helped to facilitate patient care, but it also provided an overwhelming amount of information to sift through. It was clear that the influence of some of these factors varied from institution to institution, reflecting the cultures of different programs.

Our results are consistent with those reported from previous studies. Striking a balance between preventing resident fatigue and preserving continuity of care has been debated since the ACGME announced changes to resident work hour limits.27 Resident quality of life generally improves and fatigue decreases with work hour limits in place,28 but patient safety remains a concern.10 Our findings corroborate the benefits of improved resident well‐being and the persistent concerns about patient safety, identified in a recently published study at a different institution.29 However, our findings expand on those reported in the literature by offering additional empirical evidence, albeit qualitative, about the way that residents see the relationships among the consequences of work hour rules, resident‐reported contributors to patient care mistakes, and the mistakes themselves.

Our study should be interpreted in the context of several limitations. First, the use of qualitative methods did not allow us to generalize or quantify our findings. However, we purposely included 3 diverse institutions with differing responses to the work hour rules to enhance the external validity of our findings. Second, the last focus group was conducted a year after the first 3; by that point, the work hour rules had been in place for 20 months. We believe that this was both a strength and a limitation because it allowed us to gain a perspective after some of the initial growing pains were over. This time lag also allowed for analysis of the first 3 transcripts so we could revise the focus group guide and ultimately determine that thematic saturation had occurred. In addition, few of our questions were phrased to evaluate the ACGME rules; instead, they asked about links among discontinuity, scheduling, fatigue, and patient care. We therefore believe that even residents who were not in the programs before the work hour rules began were still able to knowledgeably participate in the conversation. One question directly referable to the ACGME rules asked residents to reflect on problems arising from them. This could have led residents to only reflect on the problems associated with the rules. However, in all 4 focus groups, residents commented on the positive impact of improved well‐being resulting from the work hour rules. This led us to believe the respondents felt they could voice their favorable feelings as well as their unfavorable feelings about the rules. An additional limitation is that the agreement between coders was only 45%. It is important to realize that assessing coding agreement in qualitative work is quite difficult because it is often difficult to assign a single code to a section of text. When the coders discussed a disagreement, it was almost always the case that the difference was subtle and that the coding of either investigator would made sense for that text. Finally, our results are based on the participation of 28 residents. To be certain we were not representing the opinions of only a few people, we presented iterations of this model to faculty and resident groups for their feedback. Importantly, the residents offered no substantial changes or criticisms of the model.

Limitations notwithstanding, we believe our findings have important policy implications. First, despite work hours successfully being reduced, residents perceived no decrease in the amount of work they did. This resulted in higher workload and more entropy, suggesting that residency programs may need to carefully evaluate the patient care responsibility carried by residents. Second, continued effort to educate residents to provide effective sign‐out is needed. As one participant pointed out, residency offers a unique opportunity to learn to manage discontinuity in a controlled setting. Another educational opportunity is the chance to teach physician teamwork. Participants believed that effective teamwork could ameliorate some of the discontinuity in patient care. This teamwork training should include faculty as well, although further work is needed to define how faculty can best add to patient continuity while still fostering resident autonomy. Finally, the impact of work hour rules on the professional development of residents should be further explored.

In conclusion, we have proposed a model to explain the major resident‐reported contributors to patient care mistakes with respect to resident work hour rules. Our results help to clarify the next steps needed: testing the proposed relationships between the factors and patient care mistakes and rigorously evaluating solutions that minimize the impact of these factors. Returning to the Swiss cheese framework for describing systems accidents, our results suggest that although resident work hour reductions may have sufficiently filled the hole caused by resident fatigue, other gaps may have actually widened as a result of the systems put into place to achieve compliance. Continued vigilance is therefore necessary to both identify the additional holes likely to appear and, more importantly, effectively close those holes before patient harm occurs.

FOCUS GROUP GUIDE (4TH FOCUS GROUP)

The IOM definition of a medical error is the failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim.

• 1

  Please describe the call structure at each institution where you do ward months (eg, non‐ICU months).

• 2

  What are some settings at the hospitals where you work where you have seen medical errors, mistakes, or adverse outcomes?

• 3

  How do you think that other nurses influence the occurrence of medical errors, mistakes, or adverse outcomes?

   

  • a

    Clerks?

  • b

    Other ancillary staff?

  • 4

    How would you describe the responsibilities of a cross‐covering resident or intern?

  • 5

    How do you think continuity of care impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

     

    • a

      What role do sign‐outs have?

    • 6

      How do you think that fatigue impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

    • 7

      How do you think that technology such as computerized physician order entry impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

       

      • a

        Electronic medical records?

      • b

        Palm pilots?

      • c

        Is there such a thing as too much information?

      • 8

        How do you think that experience (or inexperience) impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

      • 9

        Please describe how attendings supervise you when you are on a ward team. How do you think that attending supervision impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

         

        • a

          What about resident supervision of interns?

        • b

          What is the ideal role of an attending on a team?

        • c

          Can you think of a time when having attending input changed the plans or the course of a patient in a major way, good, bad, or neutral?

        • 10

          How do you think that time of day impacts patient care in terms of in terms of medical errors, mistakes, or adverse outcomes?

           

          • a

            How comfortable do you feel calling for help at night? What makes you more or less likely to do it (personal attributes of person to be called, situation, etc.)?

          • 11

            What do you think is an ideal workload? (eg, How many complex patients are typical of your hospitals?) Does that vary from the VA to St. Joe's to Froedtert? How many patients should be admitted in 1 night by an intern? How many should an intern have ongoing responsibility for? Is there such a thing as too few patients?

          • 12

            If one of your family members were to admitted to your hospital at night with a life‐threatening condition, which situation would you prefer for their care (all other things being equal): admission by night float with handoff to a new but well‐rested resident or admission by a resident who then continues to care for that family member the next day but has been awake for 24 hours, admitting and cross‐covering other patients? Why?

          • 13

            What do you think was the intent of the ACGME rules? Do you think that those goals have been accomplished? Why or why not? How have they affected you as residents? How do you think that the ACGME work hour rules have influenced patient care?

           

Patient safety can be understood in terms of the Swiss cheese model of systems accidents. This model implies that many holes must align before an adverse event occurs.1 The limitations on work hours instituted by the Accreditation Council for Graduate Medical Education (ACGME)2 sought to close one hole by reducing fatigue in residents. As programs comply with these regulations, new interventions are being implemented to limit resident hours. This has resulted in more handoffs of care and therefore less continuity. The ultimate result may be to increase patient care errors by opening up new holes, the opposite of the stated goal of this reform.

Some residency programs have reported on their experience with hour reductions, giving insight into residents' perceptions on the benefits and drawbacks of such interventions. Residents have reported concern about continuity of care after such interventions.37 However, some residents believed they provided better patient care after the interventions to reduce hours.8, 9 Few studies have actually documented changes in the incidence of adverse events or errors as a result of work hour limitations.10 One study conducted prior to implementation of the ACGME work hour rules demonstrated more complications in internal medicine patients after New York's Code 405 (a state regulation that limited resident work hours, similar to the ACGME rules) was implemented.11 In contrast, another study showed that errors committed by interns were reduced with scheduling changes that resulted in shorter shifts and reduced hours.12

Because residents are on the front lines of patient care, they are uniquely positioned to provide insight into the impact of the work hour rules on patient safety. We conducted this study to more fully understand the effect of the ACGME work hour limitations and other possible factors on patient care errors from the perspectives of internal medicine residents.

METHODS

Design

We conducted 4 focus groups in total (Appendix 1). The first 3 focus groups followed the same focus group guide, developed after a literature review. Focus groups 1 and 2 were conducted at the university‐based program. Focus group 3 was conducted at the community teaching hospitalaffiliated program. The first 3 focus groups were analyzed before the fourth focus group was conducted. A new focus group guide was developed for the fourth focus group to further explore themes identified in the first 3 focus groups (Fig. 1 and Appendix 2). The fourth focus group was conducted at the program affiliated with a freestanding medical college. All focus groups were audiotaped and transcribed verbatim. Each lasted approximately 90‐120 minutes.

Figure 1

RESULTS

Twenty‐eight residents participated. Some of these residents had experience in the prework hour era, and some did not. Average age was 28 years (range 26‐33 years); 18 were women, and 11 were interns (Table 2). The focus groups ranged in size from 5 to 9. A sample of the codes and their definitions can be found in Table 3.

The Model

The model (Fig. 2) illustrates resident‐perceived contributors to patient care mistakes related to the ACGME work hour rules. These contributors are in the center circle. They include fatigue, inexperience, sign‐out, not knowing their own patients well enough, entropy (which we defined as the amount of chaos in the system), and workload. They are not listed in order of importance. The boxes outside the circle are consequences of the ACGME work hour rules and their perceived impact on the contributors to patient care mistakes. At the top are the intended consequences, that is the specific goals of the ACGME: less resident time in the hospital (ie, reduced hours) and improved well‐being.17 At the bottom are the unintended consequences: more patient care discontinuity and compliance with the work hour rules becoming a goal equally important to providing high‐quality patient care. Of these 4 consequences, only improved well‐being was viewed by the residents as decreasing patient care mistakes. The other consequences were cited by residents as sometimes increasing patient care errors. Because of the complexity of the model, several factors not directly related to resident work hours were identified in the analysis but are not shown in the model. They include faculty involvement and team work (usually positive influences), nurses and information technology (could be positive or negative), and late‐night/early‐morning hours (negative).

Figure 2

The quotations below illustrate the relationships between the consequences of the work hour rules, resident‐perceived contributors to patient care mistakes, and actual patient care.

DISCUSSION

We conducted this study to better understand why internal medicine residents thought patient care mistakes occurred; we were particularly interested in how they perceived the impact of certain aspects of the ACGME work hour rules on patient care mistakes. Designing systems that achieve compliance with the work hour rules while minimizing patient risk can best be accomplished by fully understanding why errors occur.

Our study revealed that in the opinion of some interns and residents, the work hour rules had consequences for patient care. Like any intervention, this one had both intended and unintended consequences.18 The ACGME has stated that improvement in residents' quality of life was an intended consequence,17 and the participants in our study reported that this had occurred. Despite uncertainty about the overall impact on patient outcomes, residents were glad to have more time away from the hospital.

Our respondents reported that not knowing patients well was a factor that contributed to patient care errors. It is intuitive that working fewer hours often results in more handoffs of care,19 a situation characterized by not knowing patients well. However, residents also identified not getting to know their own patients well as a factor that led to patient care mistakes because of (1) incomplete knowledge of a patient's status, (2) delays in diagnosis, and (3) errors in management. They also described feelings of professional disappointment and frustration at not being able to perform certain aspects of patient care (eg, family meetings) because of the hour limits and the inflexibility of the rules. As we strive to redefine professionalism in the setting of reduced work hours,20 this phenomenon should be addressed.

Sign‐out was identified as another contributor to patient care errors. The effectiveness of sign‐outs is a concern across medicine, and the Joint Commission on Accreditation of Healthcare Organizations made sign‐out procedures one of its priority areas in 2006.21 Much has been written about resident sign‐out, emphasizing the relationship between poor‐quality sign‐outs and patient safety.19, 22 However, barriers to effective sign‐out processes persist,23 even though standardized sign‐out strategies have been described.24, 25 Even in a rigorous study of work hours and patient safety, the computerized sign‐out template for the residents was rarely used.12 Cross‐coverage, or the patient care that occurs after sign‐out is complete, has also been linked to a greater likelihood of adverse events.26

Several factors not related to resident work hours were noted to often mitigate patient care mistakes. Physician teamwork, nursing, information technology (eg, computerized medical records), and faculty supervision were the most prominent. For example, the information technology available at the VA hospitals often helped to facilitate patient care, but it also provided an overwhelming amount of information to sift through. It was clear that the influence of some of these factors varied from institution to institution, reflecting the cultures of different programs.

Our results are consistent with those reported from previous studies. Striking a balance between preventing resident fatigue and preserving continuity of care has been debated since the ACGME announced changes to resident work hour limits.27 Resident quality of life generally improves and fatigue decreases with work hour limits in place,28 but patient safety remains a concern.10 Our findings corroborate the benefits of improved resident well‐being and the persistent concerns about patient safety, identified in a recently published study at a different institution.29 However, our findings expand on those reported in the literature by offering additional empirical evidence, albeit qualitative, about the way that residents see the relationships among the consequences of work hour rules, resident‐reported contributors to patient care mistakes, and the mistakes themselves.

Our study should be interpreted in the context of several limitations. First, the use of qualitative methods did not allow us to generalize or quantify our findings. However, we purposely included 3 diverse institutions with differing responses to the work hour rules to enhance the external validity of our findings. Second, the last focus group was conducted a year after the first 3; by that point, the work hour rules had been in place for 20 months. We believe that this was both a strength and a limitation because it allowed us to gain a perspective after some of the initial growing pains were over. This time lag also allowed for analysis of the first 3 transcripts so we could revise the focus group guide and ultimately determine that thematic saturation had occurred. In addition, few of our questions were phrased to evaluate the ACGME rules; instead, they asked about links among discontinuity, scheduling, fatigue, and patient care. We therefore believe that even residents who were not in the programs before the work hour rules began were still able to knowledgeably participate in the conversation. One question directly referable to the ACGME rules asked residents to reflect on problems arising from them. This could have led residents to only reflect on the problems associated with the rules. However, in all 4 focus groups, residents commented on the positive impact of improved well‐being resulting from the work hour rules. This led us to believe the respondents felt they could voice their favorable feelings as well as their unfavorable feelings about the rules. An additional limitation is that the agreement between coders was only 45%. It is important to realize that assessing coding agreement in qualitative work is quite difficult because it is often difficult to assign a single code to a section of text. When the coders discussed a disagreement, it was almost always the case that the difference was subtle and that the coding of either investigator would made sense for that text. Finally, our results are based on the participation of 28 residents. To be certain we were not representing the opinions of only a few people, we presented iterations of this model to faculty and resident groups for their feedback. Importantly, the residents offered no substantial changes or criticisms of the model.

Limitations notwithstanding, we believe our findings have important policy implications. First, despite work hours successfully being reduced, residents perceived no decrease in the amount of work they did. This resulted in higher workload and more entropy, suggesting that residency programs may need to carefully evaluate the patient care responsibility carried by residents. Second, continued effort to educate residents to provide effective sign‐out is needed. As one participant pointed out, residency offers a unique opportunity to learn to manage discontinuity in a controlled setting. Another educational opportunity is the chance to teach physician teamwork. Participants believed that effective teamwork could ameliorate some of the discontinuity in patient care. This teamwork training should include faculty as well, although further work is needed to define how faculty can best add to patient continuity while still fostering resident autonomy. Finally, the impact of work hour rules on the professional development of residents should be further explored.

In conclusion, we have proposed a model to explain the major resident‐reported contributors to patient care mistakes with respect to resident work hour rules. Our results help to clarify the next steps needed: testing the proposed relationships between the factors and patient care mistakes and rigorously evaluating solutions that minimize the impact of these factors. Returning to the Swiss cheese framework for describing systems accidents, our results suggest that although resident work hour reductions may have sufficiently filled the hole caused by resident fatigue, other gaps may have actually widened as a result of the systems put into place to achieve compliance. Continued vigilance is therefore necessary to both identify the additional holes likely to appear and, more importantly, effectively close those holes before patient harm occurs.

FOCUS GROUP GUIDE (4TH FOCUS GROUP)

The IOM definition of a medical error is the failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim.

• 1

  Please describe the call structure at each institution where you do ward months (eg, non‐ICU months).

• 2

  What are some settings at the hospitals where you work where you have seen medical errors, mistakes, or adverse outcomes?

• 3

  How do you think that other nurses influence the occurrence of medical errors, mistakes, or adverse outcomes?

   

  • a

    Clerks?

  • b

    Other ancillary staff?

  • 4

    How would you describe the responsibilities of a cross‐covering resident or intern?

  • 5

    How do you think continuity of care impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

     

    • a

      What role do sign‐outs have?

    • 6

      How do you think that fatigue impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

    • 7

      How do you think that technology such as computerized physician order entry impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

       

      • a

        Electronic medical records?

      • b

        Palm pilots?

      • c

        Is there such a thing as too much information?

      • 8

        How do you think that experience (or inexperience) impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

      • 9

        Please describe how attendings supervise you when you are on a ward team. How do you think that attending supervision impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

         

        • a

          What about resident supervision of interns?

        • b

          What is the ideal role of an attending on a team?

        • c

          Can you think of a time when having attending input changed the plans or the course of a patient in a major way, good, bad, or neutral?

        • 10

          How do you think that time of day impacts patient care in terms of in terms of medical errors, mistakes, or adverse outcomes?

           

          • a

            How comfortable do you feel calling for help at night? What makes you more or less likely to do it (personal attributes of person to be called, situation, etc.)?

          • 11

            What do you think is an ideal workload? (eg, How many complex patients are typical of your hospitals?) Does that vary from the VA to St. Joe's to Froedtert? How many patients should be admitted in 1 night by an intern? How many should an intern have ongoing responsibility for? Is there such a thing as too few patients?

          • 12

            If one of your family members were to admitted to your hospital at night with a life‐threatening condition, which situation would you prefer for their care (all other things being equal): admission by night float with handoff to a new but well‐rested resident or admission by a resident who then continues to care for that family member the next day but has been awake for 24 hours, admitting and cross‐covering other patients? Why?

          • 13

            What do you think was the intent of the ACGME rules? Do you think that those goals have been accomplished? Why or why not? How have they affected you as residents? How do you think that the ACGME work hour rules have influenced patient care?