Diane B. Wayne, MD

Internal medicine (IM) residents and hospitalist physicians commonly conduct bedside thoracenteses for both diagnostic and therapeutic purposes.[1] The American Board of Internal Medicine only requires that certification candidates understand the indications, complications, and management of thoracenteses.[2] A disconnect between clinical practice patterns and board requirements may increase patient risk because poorly trained physicians are more likely to cause complications.[3] National practice patterns show that many thoracenteses are referred to interventional radiology (IR).[4] However, research links performance of bedside procedures to reduced hospital length of stay and lower costs, without increasing risk of complications.[1, 5, 6]

Simulation‐based education offers a controlled environment where trainees improve procedural knowledge and skills without patient harm.[7] Simulation‐based mastery learning (SBML) is a rigorous form of competency‐based education that improves clinical skills and reduces iatrogenic complications and healthcare costs.[5, 6, 8] SBML also is an effective method to boost thoracentesis skills among IM residents.[9] However, there are no data to show that thoracentesis skills acquired in the simulation laboratory transfer to clinical environments and affect referral patterns.

We hypothesized that a thoracentesis SBML intervention would improve skills and increase procedural self‐confidence while reducing procedure referrals. This study aimed to (1) assess the effect of thoracentesis SBML on a cohort of IM residents' simulated skills and (2) compare traditionally trained (nonSBML‐trained) residents, SBML‐trained residents, and hospitalist physicians regarding procedure referral patterns, self‐confidence, procedure experience, and reasons for referral.

METHODS AND MATERIALS

RESULTS

DISCUSSION

This study confirms earlier research showing that thoracentesis SBML improves residents' clinical skills, but is the first to use a randomized study design.[9] Use of the mastery model in health professions education ensures that all learners are competent to provide patient care including performing invasive procedures. Such rigorous education yields downstream translational outcomes including safety profiles comparable to experts.[1, 6]

This study also shows that SBML‐trained residents displayed higher self‐confidence and performed significantly more bedside procedures than traditionally trained residents and more experienced hospitalist physicians. Although the Society of Hospital Medicine considers thoracentesis skills a core competency for hospitalist physicians,[11] we speculate that some hospitalist physicians had not performed a thoracentesis in years. A recent national survey showed that only 44% of hospitalist physicians performed at least 1 thoracentesis within the past year.[10] Research also shows a shift in medical culture to refer procedures to specialty services, such as IR, by over 900% in the past 2 decades.[4] Our results provide novel information about procedure referrals because we show that SBML provides translational outcomes by improving skills and self‐confidence that influence referral patterns. SBML‐trained residents performed almost a quarter of procedures at the bedside. Although this only represents an 8% absolute difference in bedside procedures compared to traditionally trained residents, if a large number of residents are trained using SBML this results in a meaningful number of procedures shifted to the patient bedside. According to University HealthSystem Consortium data, in US teaching hospitals, approximately 35,325 thoracenteses are performed yearly.[1] Shifting even 8% of these procedures to the bedside would result in significant clinical benefit and cost savings. Reduced referrals increase additional bedside procedures that are safe, cost‐effective, and highly satisfying to patients.[1, 12, 13] Further study is required to determine the impact on referral patterns after providing SMBL training to attending physicians.

Our study also provides information about the rationale for procedure referrals. Earlier work speculates that financial incentive, training and time may explain high procedure referral rates.[10] One report on IM residents noted an 87% IR referral rate for thoracentesis, and confirmed that both training and time were major reasons.[14] Hospitalist physicians reported lack of time as the major factor leading to procedural referrals, which is problematic because bedside procedures yield similar clinical outcomes at lower costs.[1, 12] Attending preference also prevented 11 additional bedside procedures in the SBML‐trained group. Schedule adjustments and SBML training of hospitalist physicians should be considered, because bundled payments in the Affordable Care Act may favor shifting to the higher‐value approach of bedside thoracenteses.[15]

Our study has several limitations. First, we only performed surveys at 1 institution and the results may not be generalizable. Second, we relied on an electronic query to alert us to thoracenteses. Our query may have missed procedures that were unsuccessful or did not have EHR orders entered. Third, physicians may have been surveyed more than once for different or the same patient(s), but opinions may have shifted over time. Fourth, some items such as time needed to be written in the survey and were not specifically asked. This could have resulted in under‐reporting. Finally, we did not assess the clinical outcomes of thoracenteses in this study, although earlier work shows that residents who complete SBML have safety outcomes similar to IR.[1, 6]

In summary, IM residents who complete thoracentesis SBML demonstrate improved clinical skills and are more likely to perform bedside procedures. In an era of bundled payments, rethinking current care models to promote cost‐effective care is necessary. We believe providing additional education, training, and support to hospitalist physicians to promote bedside procedures is a promising strategy that warrants further study.

Internal medicine (IM) residents and hospitalist physicians commonly conduct bedside thoracenteses for both diagnostic and therapeutic purposes.[1] The American Board of Internal Medicine only requires that certification candidates understand the indications, complications, and management of thoracenteses.[2] A disconnect between clinical practice patterns and board requirements may increase patient risk because poorly trained physicians are more likely to cause complications.[3] National practice patterns show that many thoracenteses are referred to interventional radiology (IR).[4] However, research links performance of bedside procedures to reduced hospital length of stay and lower costs, without increasing risk of complications.[1, 5, 6]

Simulation‐based education offers a controlled environment where trainees improve procedural knowledge and skills without patient harm.[7] Simulation‐based mastery learning (SBML) is a rigorous form of competency‐based education that improves clinical skills and reduces iatrogenic complications and healthcare costs.[5, 6, 8] SBML also is an effective method to boost thoracentesis skills among IM residents.[9] However, there are no data to show that thoracentesis skills acquired in the simulation laboratory transfer to clinical environments and affect referral patterns.

We hypothesized that a thoracentesis SBML intervention would improve skills and increase procedural self‐confidence while reducing procedure referrals. This study aimed to (1) assess the effect of thoracentesis SBML on a cohort of IM residents' simulated skills and (2) compare traditionally trained (nonSBML‐trained) residents, SBML‐trained residents, and hospitalist physicians regarding procedure referral patterns, self‐confidence, procedure experience, and reasons for referral.

METHODS AND MATERIALS

RESULTS

DISCUSSION

This study confirms earlier research showing that thoracentesis SBML improves residents' clinical skills, but is the first to use a randomized study design.[9] Use of the mastery model in health professions education ensures that all learners are competent to provide patient care including performing invasive procedures. Such rigorous education yields downstream translational outcomes including safety profiles comparable to experts.[1, 6]

This study also shows that SBML‐trained residents displayed higher self‐confidence and performed significantly more bedside procedures than traditionally trained residents and more experienced hospitalist physicians. Although the Society of Hospital Medicine considers thoracentesis skills a core competency for hospitalist physicians,[11] we speculate that some hospitalist physicians had not performed a thoracentesis in years. A recent national survey showed that only 44% of hospitalist physicians performed at least 1 thoracentesis within the past year.[10] Research also shows a shift in medical culture to refer procedures to specialty services, such as IR, by over 900% in the past 2 decades.[4] Our results provide novel information about procedure referrals because we show that SBML provides translational outcomes by improving skills and self‐confidence that influence referral patterns. SBML‐trained residents performed almost a quarter of procedures at the bedside. Although this only represents an 8% absolute difference in bedside procedures compared to traditionally trained residents, if a large number of residents are trained using SBML this results in a meaningful number of procedures shifted to the patient bedside. According to University HealthSystem Consortium data, in US teaching hospitals, approximately 35,325 thoracenteses are performed yearly.[1] Shifting even 8% of these procedures to the bedside would result in significant clinical benefit and cost savings. Reduced referrals increase additional bedside procedures that are safe, cost‐effective, and highly satisfying to patients.[1, 12, 13] Further study is required to determine the impact on referral patterns after providing SMBL training to attending physicians.

Our study also provides information about the rationale for procedure referrals. Earlier work speculates that financial incentive, training and time may explain high procedure referral rates.[10] One report on IM residents noted an 87% IR referral rate for thoracentesis, and confirmed that both training and time were major reasons.[14] Hospitalist physicians reported lack of time as the major factor leading to procedural referrals, which is problematic because bedside procedures yield similar clinical outcomes at lower costs.[1, 12] Attending preference also prevented 11 additional bedside procedures in the SBML‐trained group. Schedule adjustments and SBML training of hospitalist physicians should be considered, because bundled payments in the Affordable Care Act may favor shifting to the higher‐value approach of bedside thoracenteses.[15]

Our study has several limitations. First, we only performed surveys at 1 institution and the results may not be generalizable. Second, we relied on an electronic query to alert us to thoracenteses. Our query may have missed procedures that were unsuccessful or did not have EHR orders entered. Third, physicians may have been surveyed more than once for different or the same patient(s), but opinions may have shifted over time. Fourth, some items such as time needed to be written in the survey and were not specifically asked. This could have resulted in under‐reporting. Finally, we did not assess the clinical outcomes of thoracenteses in this study, although earlier work shows that residents who complete SBML have safety outcomes similar to IR.[1, 6]

In summary, IM residents who complete thoracentesis SBML demonstrate improved clinical skills and are more likely to perform bedside procedures. In an era of bundled payments, rethinking current care models to promote cost‐effective care is necessary. We believe providing additional education, training, and support to hospitalist physicians to promote bedside procedures is a promising strategy that warrants further study.

Internal medicine (IM) residents and hospitalist physicians commonly conduct bedside thoracenteses for both diagnostic and therapeutic purposes.[1] The American Board of Internal Medicine only requires that certification candidates understand the indications, complications, and management of thoracenteses.[2] A disconnect between clinical practice patterns and board requirements may increase patient risk because poorly trained physicians are more likely to cause complications.[3] National practice patterns show that many thoracenteses are referred to interventional radiology (IR).[4] However, research links performance of bedside procedures to reduced hospital length of stay and lower costs, without increasing risk of complications.[1, 5, 6]

Simulation‐based education offers a controlled environment where trainees improve procedural knowledge and skills without patient harm.[7] Simulation‐based mastery learning (SBML) is a rigorous form of competency‐based education that improves clinical skills and reduces iatrogenic complications and healthcare costs.[5, 6, 8] SBML also is an effective method to boost thoracentesis skills among IM residents.[9] However, there are no data to show that thoracentesis skills acquired in the simulation laboratory transfer to clinical environments and affect referral patterns.

We hypothesized that a thoracentesis SBML intervention would improve skills and increase procedural self‐confidence while reducing procedure referrals. This study aimed to (1) assess the effect of thoracentesis SBML on a cohort of IM residents' simulated skills and (2) compare traditionally trained (nonSBML‐trained) residents, SBML‐trained residents, and hospitalist physicians regarding procedure referral patterns, self‐confidence, procedure experience, and reasons for referral.

METHODS AND MATERIALS

RESULTS

DISCUSSION

This study confirms earlier research showing that thoracentesis SBML improves residents' clinical skills, but is the first to use a randomized study design.[9] Use of the mastery model in health professions education ensures that all learners are competent to provide patient care including performing invasive procedures. Such rigorous education yields downstream translational outcomes including safety profiles comparable to experts.[1, 6]

This study also shows that SBML‐trained residents displayed higher self‐confidence and performed significantly more bedside procedures than traditionally trained residents and more experienced hospitalist physicians. Although the Society of Hospital Medicine considers thoracentesis skills a core competency for hospitalist physicians,[11] we speculate that some hospitalist physicians had not performed a thoracentesis in years. A recent national survey showed that only 44% of hospitalist physicians performed at least 1 thoracentesis within the past year.[10] Research also shows a shift in medical culture to refer procedures to specialty services, such as IR, by over 900% in the past 2 decades.[4] Our results provide novel information about procedure referrals because we show that SBML provides translational outcomes by improving skills and self‐confidence that influence referral patterns. SBML‐trained residents performed almost a quarter of procedures at the bedside. Although this only represents an 8% absolute difference in bedside procedures compared to traditionally trained residents, if a large number of residents are trained using SBML this results in a meaningful number of procedures shifted to the patient bedside. According to University HealthSystem Consortium data, in US teaching hospitals, approximately 35,325 thoracenteses are performed yearly.[1] Shifting even 8% of these procedures to the bedside would result in significant clinical benefit and cost savings. Reduced referrals increase additional bedside procedures that are safe, cost‐effective, and highly satisfying to patients.[1, 12, 13] Further study is required to determine the impact on referral patterns after providing SMBL training to attending physicians.

Our study also provides information about the rationale for procedure referrals. Earlier work speculates that financial incentive, training and time may explain high procedure referral rates.[10] One report on IM residents noted an 87% IR referral rate for thoracentesis, and confirmed that both training and time were major reasons.[14] Hospitalist physicians reported lack of time as the major factor leading to procedural referrals, which is problematic because bedside procedures yield similar clinical outcomes at lower costs.[1, 12] Attending preference also prevented 11 additional bedside procedures in the SBML‐trained group. Schedule adjustments and SBML training of hospitalist physicians should be considered, because bundled payments in the Affordable Care Act may favor shifting to the higher‐value approach of bedside thoracenteses.[15]

Our study has several limitations. First, we only performed surveys at 1 institution and the results may not be generalizable. Second, we relied on an electronic query to alert us to thoracenteses. Our query may have missed procedures that were unsuccessful or did not have EHR orders entered. Third, physicians may have been surveyed more than once for different or the same patient(s), but opinions may have shifted over time. Fourth, some items such as time needed to be written in the survey and were not specifically asked. This could have resulted in under‐reporting. Finally, we did not assess the clinical outcomes of thoracenteses in this study, although earlier work shows that residents who complete SBML have safety outcomes similar to IR.[1, 6]

In summary, IM residents who complete thoracentesis SBML demonstrate improved clinical skills and are more likely to perform bedside procedures. In an era of bundled payments, rethinking current care models to promote cost‐effective care is necessary. We believe providing additional education, training, and support to hospitalist physicians to promote bedside procedures is a promising strategy that warrants further study.

Cirrhosis affects up to 3% of the population and is 1 of the 10 most common causes of death in the United States.[1, 2, 3, 4] Paracentesis procedures are frequently performed in patients with liver disease and ascites for diagnostic and/or therapeutic purposes. These procedures can be performed safely by trained clinicians at the bedside or referred to interventional radiology (IR).[2, 3, 4]

National practice patterns show that paracentesis procedures are increasingly referred to IR rather than performed at the bedside by internal medicine or gastroenterology clinicians.[5, 6, 7] In fact, a recent study of Medicare beneficiaries showed that inpatient and outpatient paracentesis procedures performed by radiologists increased by 964% from 1993 to 2008.[7] Reasons for the decline in bedside procedures include the increased availability of IR, lack of sufficient reimbursement, and the time required to perform paracentesis procedures.[5, 6, 7, 8] Surveys of internal medicine and family medicine residents and gastroenterology fellows show trainees often lack the confidence and experience needed to perform the procedure safely.[9, 10, 11] Additionally, many clinicians do not have expertise with ultrasound use and may not have access to necessary equipment.

Inconsistent certification requirements may also impact the competence and experience of physicians to perform paracentesis procedures. Internal medicine residents are no longer required by the American Board of Internal Medicine (ABIM) to demonstrate competency in procedures such as paracentesis for certification.[12] However, the Accreditation Council for Graduate Medical Education (ACGME) requirements state that internal medicine programs must offer residents the opportunity to demonstrate competence in the performance of procedures such as paracentesis, thoracentesis, and central venous catheter insertion.[13] The American Board of Family Medicine (ABFM) does not outline specific procedural competence for initial certification.[14] The ACGME states that family medicine residents must receive training to perform those clinical procedures required for their future practices but allows each program to determine which procedures to require.[15] Due to this uncertainty, practicing hospitalists are likely to have variable training and competence in bedside procedures such as paracentesis.

We previously showed that internal medicine residents rotating on the hepatology service of an academic medical center performed 59% of paracentesis procedures at the bedside.[16] These findings are in contrast to national data showing that 74% of paracentesis procedures performed on Medicare beneficiaries were performed by radiologists.[7] Practice patterns at university hospitals may not be reflected in this data because the study was limited to Medicare beneficiaries and included ambulatory patients.[7] In addition to uncertainty about who is performing this procedure in inpatient settings, little is known about the effect of specialty on postparacentesis clinical outcomes.[16, 17]

The current study had 3 aims: (1) evaluate which clinical specialties perform paracentesis procedures at university hospitals; (2) model patient characteristics associated with procedures performed at the bedside versus those referred to IR; and (3) among patients with a similar likelihood of IR referral, evaluate length of stay (LOS) and hospital costs of patients undergoing procedures performed by different specialties.

METHODS

We performed an observational administrative database review of patients who underwent paracentesis procedures in hospitals participating in the University HealthSystem Consortium (UHC) Clinical Database from January 2010 through December 2012. UHC is an alliance of 120 nonprofit academic medical centers and their 290 affiliated hospitals. UHC maintains databases containing clinical, operational, financial, and patient safety data from affiliated hospitals. Using the UHC database, we described the characteristics of all patients who underwent paracentesis procedures by clinical specialty performing the procedure. We then modeled the effects of patient characteristics on decision‐making about IR referral. Finally, among patients with a homogeneous predicted probability of IR referral, we compared LOS and direct costs by specialty performing the procedure. The Northwestern University institutional review board approved this study.

RESULTS

Predicted Probabilities of IR Referral

Figure 1 presents the distribution of predicted probabilities for IR referral. Predicted probabilities were low overall, with very few patients having an equal chance of referralthe standard often used in comparative effectiveness analyses from observational data. Figure 1 indicates that IR referral probabilities were clustered in an unusual bimodal distribution. The cluster on the left, which centers around a 15% predicted probability of IR referral, consists of discharges with patient characteristics that were associated with a very low chance of an IR paracentesis. We therefore used this distribution to conduct comparative analyses of admission outcomes between clinical specialty groups, choosing to examine patients with a 20% or greater chance of IR referral.

Figure 1

Post hoc analysis revealed that the biggest factor driving low predicted probability of IR referral was whether patients experienced an ICU stay at any time during hospitalization. Among the discharges with a predicted probability 0.2 (n=26,615 discharges), there were only 87 discharges with ICU stays (0.3%). For the discharges with predicted probability <0.2 (n=7676), 91.9% (n=7055) had an ICU admission. We therefore used a threshold of 0.2 or greater to present the most comparable LOS and direct cost differences.

DISCUSSION

This study showed that internal medicine‐ and family medicine‐trained clinicians perform approximately half of the inpatient paracentesis procedures at university hospitals and their affiliates. This confirms findings from our earlier single‐institution study[16] but contrasts with previously published reports involving Medicare data. The earlier report, using Medicare claims and including ambulatory procedures, revealed that primary care physicians and gastroenterologists only performed approximately 10% of US paracentesis procedures in 2008.[7] Our findings suggest that practices are different at university hospitals, where patients with severe liver disease often seek care. Because we used the UHC database, it was not possible to determine if the clinicians who performed paracentesis procedures in this study were internal medicine or family medicine residents, fellows, or attending physicians. However, findings from our own institution show that the vast majority of bedside paracentesis procedures are performed by internal medicine residents.[16]

Our findings have implications for certification of internal medicine and family medicine trainees. In 2008, the ABIM removed the requirement that internal medicine residents demonstrate competency in paracentesis.[12] This decision was informed by a lack of standardized methods to determine procedural competency and published surveys showing that internal medicine and hospitalist physicians rarely performed bedside procedures.[5, 6] Despite this policy change, our findings show that current clinical practice at university hospitals does not reflect national practice patterns or certification requirements, because many internal medicine‐ and family medicine‐trained clinicians still perform paracentesis procedures. This is concerning because internal medicine and family medicine trainees report variable confidence, experience, expertise, and supervision regarding performance of invasive procedures.[9, 10, 21, 22, 23, 24] Furthermore, earlier research also demonstrates that graduating residents and fellows are not able to competently perform common bedside procedures such as thoracentesis, temporary hemodialysis catheter insertion, and lumbar puncture.[25, 26, 27]

The American Association for the Study of Liver Diseases (AASLD) recommends that trained clinicians perform paracentesis procedures.[3, 4] However, the AASLD provides no definition for how training should occur. Because competency in this procedure is not specifically required by the ABIM, ABFM, or ACGME, a paradoxical situation occurs in which internal medicine and family medicine residents, and internal medicine‐trained fellows and faculty continue to perform paracentesis procedures on highly complex patients, but are no longer required to be competent to do so.

In earlier research we showed that simulation‐based mastery learning (SBML) was an effective method to boost internal medicine residents' paracentesis skills.[28] In SBML, all trainees must meet or exceed a minimum passing score on a simulated procedure before performing one on an actual patient.[29] This approach improves clinical care and outcomes in procedures such as central venous catheter insertion[30, 31] and advanced cardiac life support.[32] SBML‐trained residents also performed safe paracentesis procedures with shorter hospital LOS, fewer ICU transfers, and fewer blood product transfusions than IR procedures.[16] Based on the results of this study, AASLD guidelines regarding training, and our experience with SBML, we recommend that all clinicians complete paracentesis SBML training before performing procedures on patients.

Using our propensity model we identified patient characteristics that were associated with IR referral. Patients with a liver transplant were more likely to be cared for in IR. This may be due to a belief that postoperative procedures are anatomically more complex or because surgical trainees do not commonly perform this procedure. The current study confirms findings from earlier work that obese and female patients are more likely to be referred to IR.[16] IR referral of obese patients is likely to occur because paracentesis procedures are technically more difficult. We have no explanation why female patients were more likely to be referred to IR, because most decisions appear to be discretionary. Prospective studies are needed to determine evidence‐based recommendations regarding paracentesis procedure location. Patients with more comorbidities (eg, ICU stay, awaiting liver transplant, coagulation disorders) were more likely to undergo bedside procedures. The complexity of patients undergoing bedside paracentesis procedures reinforces the need for rigorous skill assessment for clinicians who perform them because complications such as intraperitoneal bleeding can be fatal.

Finally, we showed that LOS was similar but hospital direct costs were $800 to $1300 lower for patients whose paracentesis procedure was performed by medicine or gastroenterology/hepatology compared to IR. Medical subspecialties and surgery procedures were more expensive than IR, consistent with the higher LOS seen in these groups. IR procedures add costs due to facility charges for space, personnel, and equipment.[33] At our institution, the hospital cost of an IR paracentesis in 2012 was $361. If we use this figure, and assume costs are similar across university hospitals, the resultant cost savings would be $10,257,454 (for the procedure alone) if all procedures referred to IR in this 2‐year study were instead performed at the bedside. This estimate is approximate because it does not consider factors such as cost of clinician staffing models, which may differ across UHC hospitals. As hospitals look to reduce costs, potential savings due to appropriate use of bedside and IR procedures should be considered. This is especially important because there is no evidence that the extra expense of IR procedures is justified due to improved patient outcomes.

This study has several limitations. First, this was an observational study. Although the database was large, we were limited by coding accuracy and could not control for all potential confounding factors such as Model for End‐Stage Liver Disease score,[34, 35] other specific laboratory values, amount of ascites fluid removed, or bedside procedure failures later referred to IR. However, we do know that only a small number of second, third, or fourth procedures were subsequently referred to IR after earlier ones were performed at the bedside. Additionally the UHC database does not include patient‐specific data, and therefore we could not adjust for multiple visits or procedures by the same patient. Second, we were unable to determine the level of teaching involvement at each UHC affiliated hospital. Community hospitals where attendings managed most of the patients without trainees could not be differentiated from university hospitals where trainees were involved in most patients' care. Third, we did not have specialty information for 9434 (9.7%) procedures and had to exclude these cases. We also excluded a large number of paracentesis procedures in our final outcomes analysis. However, this was necessary because we needed to perform a patient‐level analysis to ensure the propensity and outcomes models were accurate. Finally, we did not evaluate inpatient mortality or 30‐day hospital readmission rates. Mortality and readmission from complications of a paracentesis procedure are rare events.[3, 4, 36] However, mortality and hospital readmission among patients with liver disease are relatively common.[37, 38] It was impossible to link these outcomes to a paracentesis procedure without the ability to perform medical records review.

In conclusion, paracentesis procedures are performed frequently by internal medicine‐ and family medicine‐trained clinicians in university hospitals. Because of these findings regarding current practice patterns, we believe the ACGME, ABIM, and ABFM should clarify their policies to require that residents are competent to perform paracentesis procedures before performing them on patients. This may improve supervision and training for paracentesis procedures that are already occurring and possibly encourage performance of additional, less costly bedside procedures.

Cirrhosis affects up to 3% of the population and is 1 of the 10 most common causes of death in the United States.[1, 2, 3, 4] Paracentesis procedures are frequently performed in patients with liver disease and ascites for diagnostic and/or therapeutic purposes. These procedures can be performed safely by trained clinicians at the bedside or referred to interventional radiology (IR).[2, 3, 4]

National practice patterns show that paracentesis procedures are increasingly referred to IR rather than performed at the bedside by internal medicine or gastroenterology clinicians.[5, 6, 7] In fact, a recent study of Medicare beneficiaries showed that inpatient and outpatient paracentesis procedures performed by radiologists increased by 964% from 1993 to 2008.[7] Reasons for the decline in bedside procedures include the increased availability of IR, lack of sufficient reimbursement, and the time required to perform paracentesis procedures.[5, 6, 7, 8] Surveys of internal medicine and family medicine residents and gastroenterology fellows show trainees often lack the confidence and experience needed to perform the procedure safely.[9, 10, 11] Additionally, many clinicians do not have expertise with ultrasound use and may not have access to necessary equipment.

Inconsistent certification requirements may also impact the competence and experience of physicians to perform paracentesis procedures. Internal medicine residents are no longer required by the American Board of Internal Medicine (ABIM) to demonstrate competency in procedures such as paracentesis for certification.[12] However, the Accreditation Council for Graduate Medical Education (ACGME) requirements state that internal medicine programs must offer residents the opportunity to demonstrate competence in the performance of procedures such as paracentesis, thoracentesis, and central venous catheter insertion.[13] The American Board of Family Medicine (ABFM) does not outline specific procedural competence for initial certification.[14] The ACGME states that family medicine residents must receive training to perform those clinical procedures required for their future practices but allows each program to determine which procedures to require.[15] Due to this uncertainty, practicing hospitalists are likely to have variable training and competence in bedside procedures such as paracentesis.

We previously showed that internal medicine residents rotating on the hepatology service of an academic medical center performed 59% of paracentesis procedures at the bedside.[16] These findings are in contrast to national data showing that 74% of paracentesis procedures performed on Medicare beneficiaries were performed by radiologists.[7] Practice patterns at university hospitals may not be reflected in this data because the study was limited to Medicare beneficiaries and included ambulatory patients.[7] In addition to uncertainty about who is performing this procedure in inpatient settings, little is known about the effect of specialty on postparacentesis clinical outcomes.[16, 17]

The current study had 3 aims: (1) evaluate which clinical specialties perform paracentesis procedures at university hospitals; (2) model patient characteristics associated with procedures performed at the bedside versus those referred to IR; and (3) among patients with a similar likelihood of IR referral, evaluate length of stay (LOS) and hospital costs of patients undergoing procedures performed by different specialties.

METHODS

We performed an observational administrative database review of patients who underwent paracentesis procedures in hospitals participating in the University HealthSystem Consortium (UHC) Clinical Database from January 2010 through December 2012. UHC is an alliance of 120 nonprofit academic medical centers and their 290 affiliated hospitals. UHC maintains databases containing clinical, operational, financial, and patient safety data from affiliated hospitals. Using the UHC database, we described the characteristics of all patients who underwent paracentesis procedures by clinical specialty performing the procedure. We then modeled the effects of patient characteristics on decision‐making about IR referral. Finally, among patients with a homogeneous predicted probability of IR referral, we compared LOS and direct costs by specialty performing the procedure. The Northwestern University institutional review board approved this study.

RESULTS

Predicted Probabilities of IR Referral

Figure 1 presents the distribution of predicted probabilities for IR referral. Predicted probabilities were low overall, with very few patients having an equal chance of referralthe standard often used in comparative effectiveness analyses from observational data. Figure 1 indicates that IR referral probabilities were clustered in an unusual bimodal distribution. The cluster on the left, which centers around a 15% predicted probability of IR referral, consists of discharges with patient characteristics that were associated with a very low chance of an IR paracentesis. We therefore used this distribution to conduct comparative analyses of admission outcomes between clinical specialty groups, choosing to examine patients with a 20% or greater chance of IR referral.

Figure 1

Post hoc analysis revealed that the biggest factor driving low predicted probability of IR referral was whether patients experienced an ICU stay at any time during hospitalization. Among the discharges with a predicted probability 0.2 (n=26,615 discharges), there were only 87 discharges with ICU stays (0.3%). For the discharges with predicted probability <0.2 (n=7676), 91.9% (n=7055) had an ICU admission. We therefore used a threshold of 0.2 or greater to present the most comparable LOS and direct cost differences.

DISCUSSION

This study showed that internal medicine‐ and family medicine‐trained clinicians perform approximately half of the inpatient paracentesis procedures at university hospitals and their affiliates. This confirms findings from our earlier single‐institution study[16] but contrasts with previously published reports involving Medicare data. The earlier report, using Medicare claims and including ambulatory procedures, revealed that primary care physicians and gastroenterologists only performed approximately 10% of US paracentesis procedures in 2008.[7] Our findings suggest that practices are different at university hospitals, where patients with severe liver disease often seek care. Because we used the UHC database, it was not possible to determine if the clinicians who performed paracentesis procedures in this study were internal medicine or family medicine residents, fellows, or attending physicians. However, findings from our own institution show that the vast majority of bedside paracentesis procedures are performed by internal medicine residents.[16]

Our findings have implications for certification of internal medicine and family medicine trainees. In 2008, the ABIM removed the requirement that internal medicine residents demonstrate competency in paracentesis.[12] This decision was informed by a lack of standardized methods to determine procedural competency and published surveys showing that internal medicine and hospitalist physicians rarely performed bedside procedures.[5, 6] Despite this policy change, our findings show that current clinical practice at university hospitals does not reflect national practice patterns or certification requirements, because many internal medicine‐ and family medicine‐trained clinicians still perform paracentesis procedures. This is concerning because internal medicine and family medicine trainees report variable confidence, experience, expertise, and supervision regarding performance of invasive procedures.[9, 10, 21, 22, 23, 24] Furthermore, earlier research also demonstrates that graduating residents and fellows are not able to competently perform common bedside procedures such as thoracentesis, temporary hemodialysis catheter insertion, and lumbar puncture.[25, 26, 27]

The American Association for the Study of Liver Diseases (AASLD) recommends that trained clinicians perform paracentesis procedures.[3, 4] However, the AASLD provides no definition for how training should occur. Because competency in this procedure is not specifically required by the ABIM, ABFM, or ACGME, a paradoxical situation occurs in which internal medicine and family medicine residents, and internal medicine‐trained fellows and faculty continue to perform paracentesis procedures on highly complex patients, but are no longer required to be competent to do so.

In earlier research we showed that simulation‐based mastery learning (SBML) was an effective method to boost internal medicine residents' paracentesis skills.[28] In SBML, all trainees must meet or exceed a minimum passing score on a simulated procedure before performing one on an actual patient.[29] This approach improves clinical care and outcomes in procedures such as central venous catheter insertion[30, 31] and advanced cardiac life support.[32] SBML‐trained residents also performed safe paracentesis procedures with shorter hospital LOS, fewer ICU transfers, and fewer blood product transfusions than IR procedures.[16] Based on the results of this study, AASLD guidelines regarding training, and our experience with SBML, we recommend that all clinicians complete paracentesis SBML training before performing procedures on patients.

Using our propensity model we identified patient characteristics that were associated with IR referral. Patients with a liver transplant were more likely to be cared for in IR. This may be due to a belief that postoperative procedures are anatomically more complex or because surgical trainees do not commonly perform this procedure. The current study confirms findings from earlier work that obese and female patients are more likely to be referred to IR.[16] IR referral of obese patients is likely to occur because paracentesis procedures are technically more difficult. We have no explanation why female patients were more likely to be referred to IR, because most decisions appear to be discretionary. Prospective studies are needed to determine evidence‐based recommendations regarding paracentesis procedure location. Patients with more comorbidities (eg, ICU stay, awaiting liver transplant, coagulation disorders) were more likely to undergo bedside procedures. The complexity of patients undergoing bedside paracentesis procedures reinforces the need for rigorous skill assessment for clinicians who perform them because complications such as intraperitoneal bleeding can be fatal.

Finally, we showed that LOS was similar but hospital direct costs were $800 to $1300 lower for patients whose paracentesis procedure was performed by medicine or gastroenterology/hepatology compared to IR. Medical subspecialties and surgery procedures were more expensive than IR, consistent with the higher LOS seen in these groups. IR procedures add costs due to facility charges for space, personnel, and equipment.[33] At our institution, the hospital cost of an IR paracentesis in 2012 was $361. If we use this figure, and assume costs are similar across university hospitals, the resultant cost savings would be $10,257,454 (for the procedure alone) if all procedures referred to IR in this 2‐year study were instead performed at the bedside. This estimate is approximate because it does not consider factors such as cost of clinician staffing models, which may differ across UHC hospitals. As hospitals look to reduce costs, potential savings due to appropriate use of bedside and IR procedures should be considered. This is especially important because there is no evidence that the extra expense of IR procedures is justified due to improved patient outcomes.

This study has several limitations. First, this was an observational study. Although the database was large, we were limited by coding accuracy and could not control for all potential confounding factors such as Model for End‐Stage Liver Disease score,[34, 35] other specific laboratory values, amount of ascites fluid removed, or bedside procedure failures later referred to IR. However, we do know that only a small number of second, third, or fourth procedures were subsequently referred to IR after earlier ones were performed at the bedside. Additionally the UHC database does not include patient‐specific data, and therefore we could not adjust for multiple visits or procedures by the same patient. Second, we were unable to determine the level of teaching involvement at each UHC affiliated hospital. Community hospitals where attendings managed most of the patients without trainees could not be differentiated from university hospitals where trainees were involved in most patients' care. Third, we did not have specialty information for 9434 (9.7%) procedures and had to exclude these cases. We also excluded a large number of paracentesis procedures in our final outcomes analysis. However, this was necessary because we needed to perform a patient‐level analysis to ensure the propensity and outcomes models were accurate. Finally, we did not evaluate inpatient mortality or 30‐day hospital readmission rates. Mortality and readmission from complications of a paracentesis procedure are rare events.[3, 4, 36] However, mortality and hospital readmission among patients with liver disease are relatively common.[37, 38] It was impossible to link these outcomes to a paracentesis procedure without the ability to perform medical records review.

In conclusion, paracentesis procedures are performed frequently by internal medicine‐ and family medicine‐trained clinicians in university hospitals. Because of these findings regarding current practice patterns, we believe the ACGME, ABIM, and ABFM should clarify their policies to require that residents are competent to perform paracentesis procedures before performing them on patients. This may improve supervision and training for paracentesis procedures that are already occurring and possibly encourage performance of additional, less costly bedside procedures.

Cirrhosis affects up to 3% of the population and is 1 of the 10 most common causes of death in the United States.[1, 2, 3, 4] Paracentesis procedures are frequently performed in patients with liver disease and ascites for diagnostic and/or therapeutic purposes. These procedures can be performed safely by trained clinicians at the bedside or referred to interventional radiology (IR).[2, 3, 4]

National practice patterns show that paracentesis procedures are increasingly referred to IR rather than performed at the bedside by internal medicine or gastroenterology clinicians.[5, 6, 7] In fact, a recent study of Medicare beneficiaries showed that inpatient and outpatient paracentesis procedures performed by radiologists increased by 964% from 1993 to 2008.[7] Reasons for the decline in bedside procedures include the increased availability of IR, lack of sufficient reimbursement, and the time required to perform paracentesis procedures.[5, 6, 7, 8] Surveys of internal medicine and family medicine residents and gastroenterology fellows show trainees often lack the confidence and experience needed to perform the procedure safely.[9, 10, 11] Additionally, many clinicians do not have expertise with ultrasound use and may not have access to necessary equipment.

Inconsistent certification requirements may also impact the competence and experience of physicians to perform paracentesis procedures. Internal medicine residents are no longer required by the American Board of Internal Medicine (ABIM) to demonstrate competency in procedures such as paracentesis for certification.[12] However, the Accreditation Council for Graduate Medical Education (ACGME) requirements state that internal medicine programs must offer residents the opportunity to demonstrate competence in the performance of procedures such as paracentesis, thoracentesis, and central venous catheter insertion.[13] The American Board of Family Medicine (ABFM) does not outline specific procedural competence for initial certification.[14] The ACGME states that family medicine residents must receive training to perform those clinical procedures required for their future practices but allows each program to determine which procedures to require.[15] Due to this uncertainty, practicing hospitalists are likely to have variable training and competence in bedside procedures such as paracentesis.

We previously showed that internal medicine residents rotating on the hepatology service of an academic medical center performed 59% of paracentesis procedures at the bedside.[16] These findings are in contrast to national data showing that 74% of paracentesis procedures performed on Medicare beneficiaries were performed by radiologists.[7] Practice patterns at university hospitals may not be reflected in this data because the study was limited to Medicare beneficiaries and included ambulatory patients.[7] In addition to uncertainty about who is performing this procedure in inpatient settings, little is known about the effect of specialty on postparacentesis clinical outcomes.[16, 17]

The current study had 3 aims: (1) evaluate which clinical specialties perform paracentesis procedures at university hospitals; (2) model patient characteristics associated with procedures performed at the bedside versus those referred to IR; and (3) among patients with a similar likelihood of IR referral, evaluate length of stay (LOS) and hospital costs of patients undergoing procedures performed by different specialties.

METHODS

We performed an observational administrative database review of patients who underwent paracentesis procedures in hospitals participating in the University HealthSystem Consortium (UHC) Clinical Database from January 2010 through December 2012. UHC is an alliance of 120 nonprofit academic medical centers and their 290 affiliated hospitals. UHC maintains databases containing clinical, operational, financial, and patient safety data from affiliated hospitals. Using the UHC database, we described the characteristics of all patients who underwent paracentesis procedures by clinical specialty performing the procedure. We then modeled the effects of patient characteristics on decision‐making about IR referral. Finally, among patients with a homogeneous predicted probability of IR referral, we compared LOS and direct costs by specialty performing the procedure. The Northwestern University institutional review board approved this study.

RESULTS

Predicted Probabilities of IR Referral

Figure 1 presents the distribution of predicted probabilities for IR referral. Predicted probabilities were low overall, with very few patients having an equal chance of referralthe standard often used in comparative effectiveness analyses from observational data. Figure 1 indicates that IR referral probabilities were clustered in an unusual bimodal distribution. The cluster on the left, which centers around a 15% predicted probability of IR referral, consists of discharges with patient characteristics that were associated with a very low chance of an IR paracentesis. We therefore used this distribution to conduct comparative analyses of admission outcomes between clinical specialty groups, choosing to examine patients with a 20% or greater chance of IR referral.

Figure 1

Post hoc analysis revealed that the biggest factor driving low predicted probability of IR referral was whether patients experienced an ICU stay at any time during hospitalization. Among the discharges with a predicted probability 0.2 (n=26,615 discharges), there were only 87 discharges with ICU stays (0.3%). For the discharges with predicted probability <0.2 (n=7676), 91.9% (n=7055) had an ICU admission. We therefore used a threshold of 0.2 or greater to present the most comparable LOS and direct cost differences.

DISCUSSION

This study showed that internal medicine‐ and family medicine‐trained clinicians perform approximately half of the inpatient paracentesis procedures at university hospitals and their affiliates. This confirms findings from our earlier single‐institution study[16] but contrasts with previously published reports involving Medicare data. The earlier report, using Medicare claims and including ambulatory procedures, revealed that primary care physicians and gastroenterologists only performed approximately 10% of US paracentesis procedures in 2008.[7] Our findings suggest that practices are different at university hospitals, where patients with severe liver disease often seek care. Because we used the UHC database, it was not possible to determine if the clinicians who performed paracentesis procedures in this study were internal medicine or family medicine residents, fellows, or attending physicians. However, findings from our own institution show that the vast majority of bedside paracentesis procedures are performed by internal medicine residents.[16]

Our findings have implications for certification of internal medicine and family medicine trainees. In 2008, the ABIM removed the requirement that internal medicine residents demonstrate competency in paracentesis.[12] This decision was informed by a lack of standardized methods to determine procedural competency and published surveys showing that internal medicine and hospitalist physicians rarely performed bedside procedures.[5, 6] Despite this policy change, our findings show that current clinical practice at university hospitals does not reflect national practice patterns or certification requirements, because many internal medicine‐ and family medicine‐trained clinicians still perform paracentesis procedures. This is concerning because internal medicine and family medicine trainees report variable confidence, experience, expertise, and supervision regarding performance of invasive procedures.[9, 10, 21, 22, 23, 24] Furthermore, earlier research also demonstrates that graduating residents and fellows are not able to competently perform common bedside procedures such as thoracentesis, temporary hemodialysis catheter insertion, and lumbar puncture.[25, 26, 27]

The American Association for the Study of Liver Diseases (AASLD) recommends that trained clinicians perform paracentesis procedures.[3, 4] However, the AASLD provides no definition for how training should occur. Because competency in this procedure is not specifically required by the ABIM, ABFM, or ACGME, a paradoxical situation occurs in which internal medicine and family medicine residents, and internal medicine‐trained fellows and faculty continue to perform paracentesis procedures on highly complex patients, but are no longer required to be competent to do so.

In earlier research we showed that simulation‐based mastery learning (SBML) was an effective method to boost internal medicine residents' paracentesis skills.[28] In SBML, all trainees must meet or exceed a minimum passing score on a simulated procedure before performing one on an actual patient.[29] This approach improves clinical care and outcomes in procedures such as central venous catheter insertion[30, 31] and advanced cardiac life support.[32] SBML‐trained residents also performed safe paracentesis procedures with shorter hospital LOS, fewer ICU transfers, and fewer blood product transfusions than IR procedures.[16] Based on the results of this study, AASLD guidelines regarding training, and our experience with SBML, we recommend that all clinicians complete paracentesis SBML training before performing procedures on patients.

Using our propensity model we identified patient characteristics that were associated with IR referral. Patients with a liver transplant were more likely to be cared for in IR. This may be due to a belief that postoperative procedures are anatomically more complex or because surgical trainees do not commonly perform this procedure. The current study confirms findings from earlier work that obese and female patients are more likely to be referred to IR.[16] IR referral of obese patients is likely to occur because paracentesis procedures are technically more difficult. We have no explanation why female patients were more likely to be referred to IR, because most decisions appear to be discretionary. Prospective studies are needed to determine evidence‐based recommendations regarding paracentesis procedure location. Patients with more comorbidities (eg, ICU stay, awaiting liver transplant, coagulation disorders) were more likely to undergo bedside procedures. The complexity of patients undergoing bedside paracentesis procedures reinforces the need for rigorous skill assessment for clinicians who perform them because complications such as intraperitoneal bleeding can be fatal.

Finally, we showed that LOS was similar but hospital direct costs were $800 to $1300 lower for patients whose paracentesis procedure was performed by medicine or gastroenterology/hepatology compared to IR. Medical subspecialties and surgery procedures were more expensive than IR, consistent with the higher LOS seen in these groups. IR procedures add costs due to facility charges for space, personnel, and equipment.[33] At our institution, the hospital cost of an IR paracentesis in 2012 was $361. If we use this figure, and assume costs are similar across university hospitals, the resultant cost savings would be $10,257,454 (for the procedure alone) if all procedures referred to IR in this 2‐year study were instead performed at the bedside. This estimate is approximate because it does not consider factors such as cost of clinician staffing models, which may differ across UHC hospitals. As hospitals look to reduce costs, potential savings due to appropriate use of bedside and IR procedures should be considered. This is especially important because there is no evidence that the extra expense of IR procedures is justified due to improved patient outcomes.

This study has several limitations. First, this was an observational study. Although the database was large, we were limited by coding accuracy and could not control for all potential confounding factors such as Model for End‐Stage Liver Disease score,[34, 35] other specific laboratory values, amount of ascites fluid removed, or bedside procedure failures later referred to IR. However, we do know that only a small number of second, third, or fourth procedures were subsequently referred to IR after earlier ones were performed at the bedside. Additionally the UHC database does not include patient‐specific data, and therefore we could not adjust for multiple visits or procedures by the same patient. Second, we were unable to determine the level of teaching involvement at each UHC affiliated hospital. Community hospitals where attendings managed most of the patients without trainees could not be differentiated from university hospitals where trainees were involved in most patients' care. Third, we did not have specialty information for 9434 (9.7%) procedures and had to exclude these cases. We also excluded a large number of paracentesis procedures in our final outcomes analysis. However, this was necessary because we needed to perform a patient‐level analysis to ensure the propensity and outcomes models were accurate. Finally, we did not evaluate inpatient mortality or 30‐day hospital readmission rates. Mortality and readmission from complications of a paracentesis procedure are rare events.[3, 4, 36] However, mortality and hospital readmission among patients with liver disease are relatively common.[37, 38] It was impossible to link these outcomes to a paracentesis procedure without the ability to perform medical records review.

In conclusion, paracentesis procedures are performed frequently by internal medicine‐ and family medicine‐trained clinicians in university hospitals. Because of these findings regarding current practice patterns, we believe the ACGME, ABIM, and ABFM should clarify their policies to require that residents are competent to perform paracentesis procedures before performing them on patients. This may improve supervision and training for paracentesis procedures that are already occurring and possibly encourage performance of additional, less costly bedside procedures.

Decreased efficiency at the beginning of residency training likely results in preventable harm for patients, a phenomenon known as the July Effect.[1, 2] Postgraduate year (PGY)1 residents enter training with a variety of clinical skills and experiences, and concerns exist regarding their preparation to enter graduate medical education (GME).[3] Electrocardiogram (ECG) interpretation is a core clinical skill that residents must have on the first day of training to manage patients, recognize emergencies, and develop evidence‐based and cost‐effective treatment plans. We assessed incoming PGY‐1 residents' ability to interpret common ECG findings as part of a rigorous boot camp experience.[4]

METHODS

This was an institutional review board‐approved pre‐post study of 81 new PGY‐1 residents' ECG interpretation skills. Subjects represented all trainees from internal medicine (n=47), emergency medicine (n=13), anesthesiology (n=11), and general surgery (n=10), who entered GME at Northwestern University in June 2013. Residents completed a pretest, followed by a 60‐minute interactive small group tutorial and a post‐test. Program faculty and expert cardiologists selected 10 common ECG findings for the study, many representing medical emergencies requiring immediate treatment. The diagnoses were: normal sinus rhythm, hyperkalemia, right bundle branch block (RBBB), left bundle branch block (LBBB), complete heart block, lateral wall myocardial infarction (MI), anterior wall MI, atrial fibrillation, ventricular paced rhythm, and ventricular tachycardia (VT). ECGs were selected from an online reference set (www.ecg.bidmc.harvard.edu; Harvard University, Cambridge, MA) using different pre‐ and post‐test ECGs in varying order to represent the same 10 common findings.

RESULTS

All 81 residents completed the study. The mean age was 27 years, and 56% were male. Eighty (99%) graduated from a US medical school. The mean United States Medical Licensing Examination scores were step 1: 243.8 (14.4) and step 2: 251.8 (13.6). Twenty‐six (32%) completed a cardiology rotation in medical school. Before the pretest, residents self‐assessed their ECG interpretation skills as a mean of 61.8 (standard deviation 17.2) using a scale of 0 (not confident) to 100 (very confident). Pretest results ranged from 60.5% correct (complete heart block) to 96.3% correct (normal sinus rhythm). Eighteen residents (22%) did not recognize hyperkalemia, 20 (25%) were unable to identify RBBB, and 15 (18%) LBBB. Twenty‐two (27%) could not discern a lateral wall MI, and 8 residents (10%) missed an anterior wall MI. Sixteen (20%) could not diagnose atrial fibrillation, 18 (22%) could not identify a ventricular paced rhythm, and 13 (16%) did not recognize VT. Mean post‐test scores improved significantly for 5 cases (P<0.05), but did not rise significantly for normal sinus rhythm, lateral wall MI, anterior wall MI, hyperkalemia, and ventricular paced rhythm 1.

Figure 1

DISCUSSION

PGY‐1 residents from multiple specialties were not confident regarding their ability to interpret ECGs and could not reliably identify 10 basic findings. This is despite graduating almost exclusively from US medical schools and performing at high levels on standardized tests. Although boot camp improved recognition of important ECG findings, including VT and bundle branch blocks, identification of emergent diagnoses such as lateral/anterior MI and hyperkalemia require additional training and close supervision during patient care. This study provides further evidence that the preparation of PGY‐1 residents to enter GME is lacking. Recent calls for inclusion of cost‐consciousness and stewardship of resources as a seventh competency for residents[5] are challenging, because incoming trainees do not uniformly possess the basic clinical skills needed to make these judgments.[3, 4] If residents cannot reliably interpret ECGs, it is not possible to determine cost‐effective testing strategies for patients with cardiac conditions. Based on the result of this study and others,[3, 4] we believe medical schools should agree upon specific graduation requirements to ensure all students have mastered core competencies and are prepared to enter GME.

Decreased efficiency at the beginning of residency training likely results in preventable harm for patients, a phenomenon known as the July Effect.[1, 2] Postgraduate year (PGY)1 residents enter training with a variety of clinical skills and experiences, and concerns exist regarding their preparation to enter graduate medical education (GME).[3] Electrocardiogram (ECG) interpretation is a core clinical skill that residents must have on the first day of training to manage patients, recognize emergencies, and develop evidence‐based and cost‐effective treatment plans. We assessed incoming PGY‐1 residents' ability to interpret common ECG findings as part of a rigorous boot camp experience.[4]

METHODS

This was an institutional review board‐approved pre‐post study of 81 new PGY‐1 residents' ECG interpretation skills. Subjects represented all trainees from internal medicine (n=47), emergency medicine (n=13), anesthesiology (n=11), and general surgery (n=10), who entered GME at Northwestern University in June 2013. Residents completed a pretest, followed by a 60‐minute interactive small group tutorial and a post‐test. Program faculty and expert cardiologists selected 10 common ECG findings for the study, many representing medical emergencies requiring immediate treatment. The diagnoses were: normal sinus rhythm, hyperkalemia, right bundle branch block (RBBB), left bundle branch block (LBBB), complete heart block, lateral wall myocardial infarction (MI), anterior wall MI, atrial fibrillation, ventricular paced rhythm, and ventricular tachycardia (VT). ECGs were selected from an online reference set (www.ecg.bidmc.harvard.edu; Harvard University, Cambridge, MA) using different pre‐ and post‐test ECGs in varying order to represent the same 10 common findings.

RESULTS

All 81 residents completed the study. The mean age was 27 years, and 56% were male. Eighty (99%) graduated from a US medical school. The mean United States Medical Licensing Examination scores were step 1: 243.8 (14.4) and step 2: 251.8 (13.6). Twenty‐six (32%) completed a cardiology rotation in medical school. Before the pretest, residents self‐assessed their ECG interpretation skills as a mean of 61.8 (standard deviation 17.2) using a scale of 0 (not confident) to 100 (very confident). Pretest results ranged from 60.5% correct (complete heart block) to 96.3% correct (normal sinus rhythm). Eighteen residents (22%) did not recognize hyperkalemia, 20 (25%) were unable to identify RBBB, and 15 (18%) LBBB. Twenty‐two (27%) could not discern a lateral wall MI, and 8 residents (10%) missed an anterior wall MI. Sixteen (20%) could not diagnose atrial fibrillation, 18 (22%) could not identify a ventricular paced rhythm, and 13 (16%) did not recognize VT. Mean post‐test scores improved significantly for 5 cases (P<0.05), but did not rise significantly for normal sinus rhythm, lateral wall MI, anterior wall MI, hyperkalemia, and ventricular paced rhythm 1.

Figure 1

DISCUSSION

PGY‐1 residents from multiple specialties were not confident regarding their ability to interpret ECGs and could not reliably identify 10 basic findings. This is despite graduating almost exclusively from US medical schools and performing at high levels on standardized tests. Although boot camp improved recognition of important ECG findings, including VT and bundle branch blocks, identification of emergent diagnoses such as lateral/anterior MI and hyperkalemia require additional training and close supervision during patient care. This study provides further evidence that the preparation of PGY‐1 residents to enter GME is lacking. Recent calls for inclusion of cost‐consciousness and stewardship of resources as a seventh competency for residents[5] are challenging, because incoming trainees do not uniformly possess the basic clinical skills needed to make these judgments.[3, 4] If residents cannot reliably interpret ECGs, it is not possible to determine cost‐effective testing strategies for patients with cardiac conditions. Based on the result of this study and others,[3, 4] we believe medical schools should agree upon specific graduation requirements to ensure all students have mastered core competencies and are prepared to enter GME.

Decreased efficiency at the beginning of residency training likely results in preventable harm for patients, a phenomenon known as the July Effect.[1, 2] Postgraduate year (PGY)1 residents enter training with a variety of clinical skills and experiences, and concerns exist regarding their preparation to enter graduate medical education (GME).[3] Electrocardiogram (ECG) interpretation is a core clinical skill that residents must have on the first day of training to manage patients, recognize emergencies, and develop evidence‐based and cost‐effective treatment plans. We assessed incoming PGY‐1 residents' ability to interpret common ECG findings as part of a rigorous boot camp experience.[4]

METHODS

This was an institutional review board‐approved pre‐post study of 81 new PGY‐1 residents' ECG interpretation skills. Subjects represented all trainees from internal medicine (n=47), emergency medicine (n=13), anesthesiology (n=11), and general surgery (n=10), who entered GME at Northwestern University in June 2013. Residents completed a pretest, followed by a 60‐minute interactive small group tutorial and a post‐test. Program faculty and expert cardiologists selected 10 common ECG findings for the study, many representing medical emergencies requiring immediate treatment. The diagnoses were: normal sinus rhythm, hyperkalemia, right bundle branch block (RBBB), left bundle branch block (LBBB), complete heart block, lateral wall myocardial infarction (MI), anterior wall MI, atrial fibrillation, ventricular paced rhythm, and ventricular tachycardia (VT). ECGs were selected from an online reference set (www.ecg.bidmc.harvard.edu; Harvard University, Cambridge, MA) using different pre‐ and post‐test ECGs in varying order to represent the same 10 common findings.

RESULTS

All 81 residents completed the study. The mean age was 27 years, and 56% were male. Eighty (99%) graduated from a US medical school. The mean United States Medical Licensing Examination scores were step 1: 243.8 (14.4) and step 2: 251.8 (13.6). Twenty‐six (32%) completed a cardiology rotation in medical school. Before the pretest, residents self‐assessed their ECG interpretation skills as a mean of 61.8 (standard deviation 17.2) using a scale of 0 (not confident) to 100 (very confident). Pretest results ranged from 60.5% correct (complete heart block) to 96.3% correct (normal sinus rhythm). Eighteen residents (22%) did not recognize hyperkalemia, 20 (25%) were unable to identify RBBB, and 15 (18%) LBBB. Twenty‐two (27%) could not discern a lateral wall MI, and 8 residents (10%) missed an anterior wall MI. Sixteen (20%) could not diagnose atrial fibrillation, 18 (22%) could not identify a ventricular paced rhythm, and 13 (16%) did not recognize VT. Mean post‐test scores improved significantly for 5 cases (P<0.05), but did not rise significantly for normal sinus rhythm, lateral wall MI, anterior wall MI, hyperkalemia, and ventricular paced rhythm 1.

Figure 1

DISCUSSION

PGY‐1 residents from multiple specialties were not confident regarding their ability to interpret ECGs and could not reliably identify 10 basic findings. This is despite graduating almost exclusively from US medical schools and performing at high levels on standardized tests. Although boot camp improved recognition of important ECG findings, including VT and bundle branch blocks, identification of emergent diagnoses such as lateral/anterior MI and hyperkalemia require additional training and close supervision during patient care. This study provides further evidence that the preparation of PGY‐1 residents to enter GME is lacking. Recent calls for inclusion of cost‐consciousness and stewardship of resources as a seventh competency for residents[5] are challenging, because incoming trainees do not uniformly possess the basic clinical skills needed to make these judgments.[3, 4] If residents cannot reliably interpret ECGs, it is not possible to determine cost‐effective testing strategies for patients with cardiac conditions. Based on the result of this study and others,[3, 4] we believe medical schools should agree upon specific graduation requirements to ensure all students have mastered core competencies and are prepared to enter GME.

Unprofessional behavior in the inpatient setting has the potential to impact care delivery and the quality of trainee's educational experience. These behaviors, from disparaging colleagues to blocking admissions, can negatively impact the learning environment. The learning environment or conditions created by the patient care team's actions play a critical role in the development of trainees.[1, 2] The rising presence of hospitalists in the inpatient setting raises the question of how their actions impact the learning environment. Professional behavior has been defined as a core competency for hospitalists by the Society of Hospital Medicine.[3] Professional behavior of all team members, from faculty to trainee, can impact the learning environment and patient safety.[4, 5] However, few educational materials exist to train faculty and housestaff on recognizing and ameliorating unprofessional behaviors.

A prior assessment regarding hospitalists' lapses in professionalism identified scenarios that demonstrated increased participation by hospitalists at 3 institutions.[6] Participants reported observation or participation in specific unprofessional behaviors and rated their perception of these behaviors. Additional work within those residency environments demonstrated that residents' perceptions of and participation in these behaviors increased throughout training, with environmental characteristics, specifically faculty behavior, influencing trainee professional development and acclimation of these behaviors.[7, 8]

Although overall participation in egregious behavior was low, resident participation in 3 categories of unprofessional behavior increased during internship. Those scenarios included disparaging the emergency room or primary care physician for missed findings or management decisions, blocking or not taking admissions appropriate for the service in question, and misrepresenting a test as urgent to expedite obtaining the test. We developed our intervention focused on these areas to address professionalism lapses that occur during internship. Our earlier work showed faculty role models influenced trainee behavior. For this reason, we provided education to both residents and hospitalists to maximize the impact of the intervention.

We present here a novel, interactive, video‐based workshop curriculum for faculty and trainees that aims to illustrate unprofessional behaviors and outlines the role faculty may play in promoting such behaviors. In addition, we review the result of postworkshop evaluation on intent to change behavior and satisfaction.

METHODS

A grant from the American Board of Internal Medicine Foundation supported this project. The working group that resulted, the Chicago Professional Practice Project and Outcomes, included faculty representation from 3 Chicago‐area hospitals: the University of Chicago, Northwestern University, and NorthShore University HealthSystem. Academic hospitalists at these sites were invited to participate. Each site also has an internal medicine residency program in which hospitalists were expected to attend the teaching service. Given this, resident trainees at all participating sites, and 1 community teaching affiliate program (Mercy Hospital and Medical Center) where academic hospitalists at the University of Chicago rotate, were recruited for participation. Faculty champions were identified for each site, and 1 internal and external faculty representative from the working group served to debrief and facilitate. Trainee workshops were administered by 1 internal and external collaborator, and for the community site, 2 external faculty members. Workshops were held during established educational conference times, and lunch was provided.

Scripts highlighting each of the behaviors identified in the prior survey were developed and peer reviewed for clarity and face validity across the 3 sites. Medical student and resident actors were trained utilizing the finalized scripts, and a performance artist affiliated with the Screen Actors Guild assisted in their preparation for filming. All videos were filmed at the University of Chicago Pritzker School of Medicine Clinical Performance Center. The final videos ranged in length from 4 to 7 minutes and included title, cast, and funding source. As an example, 1 video highlighted the unprofessional behavior of misrepresenting a test as urgent to prioritize one's patient in the queue. This video included a resident, intern, and attending on inpatient rounds during which the resident encouraged the intern to misrepresent the patient's status to expedite obtaining the study and facilitate the patient's discharge. The resident stressed that he would be in the clinic and had many patients to see, highlighting the impact of workload on unprofessional behavior, and aggressively persuaded the intern to sell her test to have it performed the same day. When this occurred, the attending applauded the intern for her strong work.

A moderator guide and debriefing tools were developed to facilitate discussion. The duration of each of the workshops was approximately 60 minutes. After welcoming remarks, participants were provided tools to utilize during the viewing of each video. These checklists noted the roles of those depicted in the video, asked to identify positive or negative behaviors displayed, and included questions regarding how behaviors could be detrimental and how the situation could have been prevented. After viewing the videos, participants divided into small groups to discuss the individual exhibiting the unprofessional behavior, their perceived motivation for said behavior, and its impact on the team culture and patient care. Following a small‐group discussion, large‐group debriefing was performed, addressing the barriers and facilitators to professional behavior. Two videos were shown at each workshop, and participants completed a postworkshop evaluation. Videos chosen for viewing were based upon preworkshop survey results that highlighted areas of concern at that specific site.

Postworkshop paper‐based evaluations assessed participants' perception of displayed behaviors on a Likert‐type scale (1=unprofessional to 5=professional) utilizing items validated in prior work,[6, 7, 8] their level of agreement regarding the impact of video‐based exercises, and intent to change behavior using a Likert‐type scale (1=strongly disagree to 5=strongly agree). A constructed‐response section for comments regarding their experience was included. Descriptive statistics and Wilcoxon rank sum analyses were performed.

RESULTS

Forty‐four academic hospitalist faculty members (44/83; 53%) and 244 resident trainees (244/356; 68%) participated. When queried regarding their perception of the displayed behaviors in the videos, nearly 100% of faculty and trainees felt disparaging the emergency department or primary care physician for missed findings or clinical decisions was somewhat unprofessional or unprofessional. Ninety percent of hospitalists and 93% of trainees rated celebrating a blocked admission as somewhat unprofessional or unprofessional (Table 1).

The scenarios portrayed were well received, with more than 85% of faculty and trainees agreeing that the behaviors displayed were realistic. Those who perceived videos as very realistic were more likely to report intent to change behavior (93% vs 53%, P=0.01). Nearly two‐thirds of faculty and 67% of housestaff expressed agreement that they intended to change behavior based upon the experience (Table 2).

Qualitative comments in the constructed‐response portion of the evaluation noted the effectiveness of the interactive materials. In addition, the need for focused faculty development was identified by 1 respondent who stated: If unprofessional behavior is the unwritten curriculum, there needs to be an explicit, written curriculum to address it. Finally, the aim of facilitating self‐reflection is echoed in this faculty respondent's comment: Always good to be reminded of our behaviors and the influence they have on others and from this resident physician It helps to re‐evaluate how you talk to people.

CONCLUSIONS

Faculty can be a large determinant of the learning environment and impact trainees' professional development.[9] Hospitalists should be encouraged to embrace faculty role‐modeling of effective professional behaviors, especially given their increased presence in the inpatient learning environment. In addition, resident trainees and their behaviors contribute to the learning environment and influence the further professional development of more junior trainees.[10] Targeting professionalism education toward previously identified and prevalent unprofessional behaviors in the inpatient care of patients may serve to affect the most change among providers who practice in this setting. Individualized assessment of the learning environment may aid in identifying common scenarios that may plague a specific learning culture, allowing for relevant and targeted discussion of factors that promote and perpetuate such behaviors.[11]

Interactive, video‐based modules provided an effective way to promote interactive reflection and robust discussion. This model of experiential learning is an effective form of professional development as it engages the learner and stimulates ongoing incorporation of the topics addressed.[12, 13] Creating a shared concrete experience among targeted learners, using the video‐based scenarios, stimulates reflective observation, and ultimately experimentation, or incorporation into practice.[14]

There are several limitations to our evaluation including that we focused solely on academic hospitalist programs, and our sample size for faculty and residents was small. Also, we only addressed a small, though representative, sample of unprofessional behaviors and have not yet linked intervention to actual behavior change. Finally, the script scenarios that we used in this study were not previously published as they were created specifically for this intervention. Validity evidence for these scenarios include that they were based upon the results of earlier work from our institutions and underwent thorough peer review for content and clarity. Further studies will be required to do this. However, we do believe that these are positive findings for utilizing this type of interactive curriculum for professionalism education to promote self‐reflection and behavior change.

Video‐based professionalism education is a feasible, interactive mechanism to encourage self‐reflection and intent to change behavior among faculty and resident physicians. Future study is underway to conduct longitudinal assessments of the learning environments at the participating institutions to assess culture change, perceptions of behaviors, and sustainability of this type of intervention.

Disclosures: The authors acknowledge funding from the American Board of Internal Medicine. The funders had no role in the design of the study; the collection, analysis, and interpretation of the data; or the decision to approve publication of the finished manuscript. Results from this work have been presented at the Midwest Society of General Internal Medicine Regional Meeting, Chicago, Illinois, September 2011; Midwest Society of Hospital Medicine Regional Meeting, Chicago, Illinois, October 2011, and Society of Hospital Medicine Annual Meeting, San Diego, California, April 2012. The authors declare that they do not have any conflicts of interest to disclose.

Unprofessional behavior in the inpatient setting has the potential to impact care delivery and the quality of trainee's educational experience. These behaviors, from disparaging colleagues to blocking admissions, can negatively impact the learning environment. The learning environment or conditions created by the patient care team's actions play a critical role in the development of trainees.[1, 2] The rising presence of hospitalists in the inpatient setting raises the question of how their actions impact the learning environment. Professional behavior has been defined as a core competency for hospitalists by the Society of Hospital Medicine.[3] Professional behavior of all team members, from faculty to trainee, can impact the learning environment and patient safety.[4, 5] However, few educational materials exist to train faculty and housestaff on recognizing and ameliorating unprofessional behaviors.

A prior assessment regarding hospitalists' lapses in professionalism identified scenarios that demonstrated increased participation by hospitalists at 3 institutions.[6] Participants reported observation or participation in specific unprofessional behaviors and rated their perception of these behaviors. Additional work within those residency environments demonstrated that residents' perceptions of and participation in these behaviors increased throughout training, with environmental characteristics, specifically faculty behavior, influencing trainee professional development and acclimation of these behaviors.[7, 8]

Although overall participation in egregious behavior was low, resident participation in 3 categories of unprofessional behavior increased during internship. Those scenarios included disparaging the emergency room or primary care physician for missed findings or management decisions, blocking or not taking admissions appropriate for the service in question, and misrepresenting a test as urgent to expedite obtaining the test. We developed our intervention focused on these areas to address professionalism lapses that occur during internship. Our earlier work showed faculty role models influenced trainee behavior. For this reason, we provided education to both residents and hospitalists to maximize the impact of the intervention.

We present here a novel, interactive, video‐based workshop curriculum for faculty and trainees that aims to illustrate unprofessional behaviors and outlines the role faculty may play in promoting such behaviors. In addition, we review the result of postworkshop evaluation on intent to change behavior and satisfaction.

METHODS

A grant from the American Board of Internal Medicine Foundation supported this project. The working group that resulted, the Chicago Professional Practice Project and Outcomes, included faculty representation from 3 Chicago‐area hospitals: the University of Chicago, Northwestern University, and NorthShore University HealthSystem. Academic hospitalists at these sites were invited to participate. Each site also has an internal medicine residency program in which hospitalists were expected to attend the teaching service. Given this, resident trainees at all participating sites, and 1 community teaching affiliate program (Mercy Hospital and Medical Center) where academic hospitalists at the University of Chicago rotate, were recruited for participation. Faculty champions were identified for each site, and 1 internal and external faculty representative from the working group served to debrief and facilitate. Trainee workshops were administered by 1 internal and external collaborator, and for the community site, 2 external faculty members. Workshops were held during established educational conference times, and lunch was provided.

Scripts highlighting each of the behaviors identified in the prior survey were developed and peer reviewed for clarity and face validity across the 3 sites. Medical student and resident actors were trained utilizing the finalized scripts, and a performance artist affiliated with the Screen Actors Guild assisted in their preparation for filming. All videos were filmed at the University of Chicago Pritzker School of Medicine Clinical Performance Center. The final videos ranged in length from 4 to 7 minutes and included title, cast, and funding source. As an example, 1 video highlighted the unprofessional behavior of misrepresenting a test as urgent to prioritize one's patient in the queue. This video included a resident, intern, and attending on inpatient rounds during which the resident encouraged the intern to misrepresent the patient's status to expedite obtaining the study and facilitate the patient's discharge. The resident stressed that he would be in the clinic and had many patients to see, highlighting the impact of workload on unprofessional behavior, and aggressively persuaded the intern to sell her test to have it performed the same day. When this occurred, the attending applauded the intern for her strong work.

A moderator guide and debriefing tools were developed to facilitate discussion. The duration of each of the workshops was approximately 60 minutes. After welcoming remarks, participants were provided tools to utilize during the viewing of each video. These checklists noted the roles of those depicted in the video, asked to identify positive or negative behaviors displayed, and included questions regarding how behaviors could be detrimental and how the situation could have been prevented. After viewing the videos, participants divided into small groups to discuss the individual exhibiting the unprofessional behavior, their perceived motivation for said behavior, and its impact on the team culture and patient care. Following a small‐group discussion, large‐group debriefing was performed, addressing the barriers and facilitators to professional behavior. Two videos were shown at each workshop, and participants completed a postworkshop evaluation. Videos chosen for viewing were based upon preworkshop survey results that highlighted areas of concern at that specific site.

Postworkshop paper‐based evaluations assessed participants' perception of displayed behaviors on a Likert‐type scale (1=unprofessional to 5=professional) utilizing items validated in prior work,[6, 7, 8] their level of agreement regarding the impact of video‐based exercises, and intent to change behavior using a Likert‐type scale (1=strongly disagree to 5=strongly agree). A constructed‐response section for comments regarding their experience was included. Descriptive statistics and Wilcoxon rank sum analyses were performed.

RESULTS

Forty‐four academic hospitalist faculty members (44/83; 53%) and 244 resident trainees (244/356; 68%) participated. When queried regarding their perception of the displayed behaviors in the videos, nearly 100% of faculty and trainees felt disparaging the emergency department or primary care physician for missed findings or clinical decisions was somewhat unprofessional or unprofessional. Ninety percent of hospitalists and 93% of trainees rated celebrating a blocked admission as somewhat unprofessional or unprofessional (Table 1).

The scenarios portrayed were well received, with more than 85% of faculty and trainees agreeing that the behaviors displayed were realistic. Those who perceived videos as very realistic were more likely to report intent to change behavior (93% vs 53%, P=0.01). Nearly two‐thirds of faculty and 67% of housestaff expressed agreement that they intended to change behavior based upon the experience (Table 2).

Qualitative comments in the constructed‐response portion of the evaluation noted the effectiveness of the interactive materials. In addition, the need for focused faculty development was identified by 1 respondent who stated: If unprofessional behavior is the unwritten curriculum, there needs to be an explicit, written curriculum to address it. Finally, the aim of facilitating self‐reflection is echoed in this faculty respondent's comment: Always good to be reminded of our behaviors and the influence they have on others and from this resident physician It helps to re‐evaluate how you talk to people.

CONCLUSIONS

Faculty can be a large determinant of the learning environment and impact trainees' professional development.[9] Hospitalists should be encouraged to embrace faculty role‐modeling of effective professional behaviors, especially given their increased presence in the inpatient learning environment. In addition, resident trainees and their behaviors contribute to the learning environment and influence the further professional development of more junior trainees.[10] Targeting professionalism education toward previously identified and prevalent unprofessional behaviors in the inpatient care of patients may serve to affect the most change among providers who practice in this setting. Individualized assessment of the learning environment may aid in identifying common scenarios that may plague a specific learning culture, allowing for relevant and targeted discussion of factors that promote and perpetuate such behaviors.[11]

Interactive, video‐based modules provided an effective way to promote interactive reflection and robust discussion. This model of experiential learning is an effective form of professional development as it engages the learner and stimulates ongoing incorporation of the topics addressed.[12, 13] Creating a shared concrete experience among targeted learners, using the video‐based scenarios, stimulates reflective observation, and ultimately experimentation, or incorporation into practice.[14]

There are several limitations to our evaluation including that we focused solely on academic hospitalist programs, and our sample size for faculty and residents was small. Also, we only addressed a small, though representative, sample of unprofessional behaviors and have not yet linked intervention to actual behavior change. Finally, the script scenarios that we used in this study were not previously published as they were created specifically for this intervention. Validity evidence for these scenarios include that they were based upon the results of earlier work from our institutions and underwent thorough peer review for content and clarity. Further studies will be required to do this. However, we do believe that these are positive findings for utilizing this type of interactive curriculum for professionalism education to promote self‐reflection and behavior change.

Video‐based professionalism education is a feasible, interactive mechanism to encourage self‐reflection and intent to change behavior among faculty and resident physicians. Future study is underway to conduct longitudinal assessments of the learning environments at the participating institutions to assess culture change, perceptions of behaviors, and sustainability of this type of intervention.

Disclosures: The authors acknowledge funding from the American Board of Internal Medicine. The funders had no role in the design of the study; the collection, analysis, and interpretation of the data; or the decision to approve publication of the finished manuscript. Results from this work have been presented at the Midwest Society of General Internal Medicine Regional Meeting, Chicago, Illinois, September 2011; Midwest Society of Hospital Medicine Regional Meeting, Chicago, Illinois, October 2011, and Society of Hospital Medicine Annual Meeting, San Diego, California, April 2012. The authors declare that they do not have any conflicts of interest to disclose.

Unprofessional behavior in the inpatient setting has the potential to impact care delivery and the quality of trainee's educational experience. These behaviors, from disparaging colleagues to blocking admissions, can negatively impact the learning environment. The learning environment or conditions created by the patient care team's actions play a critical role in the development of trainees.[1, 2] The rising presence of hospitalists in the inpatient setting raises the question of how their actions impact the learning environment. Professional behavior has been defined as a core competency for hospitalists by the Society of Hospital Medicine.[3] Professional behavior of all team members, from faculty to trainee, can impact the learning environment and patient safety.[4, 5] However, few educational materials exist to train faculty and housestaff on recognizing and ameliorating unprofessional behaviors.

A prior assessment regarding hospitalists' lapses in professionalism identified scenarios that demonstrated increased participation by hospitalists at 3 institutions.[6] Participants reported observation or participation in specific unprofessional behaviors and rated their perception of these behaviors. Additional work within those residency environments demonstrated that residents' perceptions of and participation in these behaviors increased throughout training, with environmental characteristics, specifically faculty behavior, influencing trainee professional development and acclimation of these behaviors.[7, 8]

Although overall participation in egregious behavior was low, resident participation in 3 categories of unprofessional behavior increased during internship. Those scenarios included disparaging the emergency room or primary care physician for missed findings or management decisions, blocking or not taking admissions appropriate for the service in question, and misrepresenting a test as urgent to expedite obtaining the test. We developed our intervention focused on these areas to address professionalism lapses that occur during internship. Our earlier work showed faculty role models influenced trainee behavior. For this reason, we provided education to both residents and hospitalists to maximize the impact of the intervention.

We present here a novel, interactive, video‐based workshop curriculum for faculty and trainees that aims to illustrate unprofessional behaviors and outlines the role faculty may play in promoting such behaviors. In addition, we review the result of postworkshop evaluation on intent to change behavior and satisfaction.

METHODS

A grant from the American Board of Internal Medicine Foundation supported this project. The working group that resulted, the Chicago Professional Practice Project and Outcomes, included faculty representation from 3 Chicago‐area hospitals: the University of Chicago, Northwestern University, and NorthShore University HealthSystem. Academic hospitalists at these sites were invited to participate. Each site also has an internal medicine residency program in which hospitalists were expected to attend the teaching service. Given this, resident trainees at all participating sites, and 1 community teaching affiliate program (Mercy Hospital and Medical Center) where academic hospitalists at the University of Chicago rotate, were recruited for participation. Faculty champions were identified for each site, and 1 internal and external faculty representative from the working group served to debrief and facilitate. Trainee workshops were administered by 1 internal and external collaborator, and for the community site, 2 external faculty members. Workshops were held during established educational conference times, and lunch was provided.

Scripts highlighting each of the behaviors identified in the prior survey were developed and peer reviewed for clarity and face validity across the 3 sites. Medical student and resident actors were trained utilizing the finalized scripts, and a performance artist affiliated with the Screen Actors Guild assisted in their preparation for filming. All videos were filmed at the University of Chicago Pritzker School of Medicine Clinical Performance Center. The final videos ranged in length from 4 to 7 minutes and included title, cast, and funding source. As an example, 1 video highlighted the unprofessional behavior of misrepresenting a test as urgent to prioritize one's patient in the queue. This video included a resident, intern, and attending on inpatient rounds during which the resident encouraged the intern to misrepresent the patient's status to expedite obtaining the study and facilitate the patient's discharge. The resident stressed that he would be in the clinic and had many patients to see, highlighting the impact of workload on unprofessional behavior, and aggressively persuaded the intern to sell her test to have it performed the same day. When this occurred, the attending applauded the intern for her strong work.

A moderator guide and debriefing tools were developed to facilitate discussion. The duration of each of the workshops was approximately 60 minutes. After welcoming remarks, participants were provided tools to utilize during the viewing of each video. These checklists noted the roles of those depicted in the video, asked to identify positive or negative behaviors displayed, and included questions regarding how behaviors could be detrimental and how the situation could have been prevented. After viewing the videos, participants divided into small groups to discuss the individual exhibiting the unprofessional behavior, their perceived motivation for said behavior, and its impact on the team culture and patient care. Following a small‐group discussion, large‐group debriefing was performed, addressing the barriers and facilitators to professional behavior. Two videos were shown at each workshop, and participants completed a postworkshop evaluation. Videos chosen for viewing were based upon preworkshop survey results that highlighted areas of concern at that specific site.

Postworkshop paper‐based evaluations assessed participants' perception of displayed behaviors on a Likert‐type scale (1=unprofessional to 5=professional) utilizing items validated in prior work,[6, 7, 8] their level of agreement regarding the impact of video‐based exercises, and intent to change behavior using a Likert‐type scale (1=strongly disagree to 5=strongly agree). A constructed‐response section for comments regarding their experience was included. Descriptive statistics and Wilcoxon rank sum analyses were performed.

RESULTS

Forty‐four academic hospitalist faculty members (44/83; 53%) and 244 resident trainees (244/356; 68%) participated. When queried regarding their perception of the displayed behaviors in the videos, nearly 100% of faculty and trainees felt disparaging the emergency department or primary care physician for missed findings or clinical decisions was somewhat unprofessional or unprofessional. Ninety percent of hospitalists and 93% of trainees rated celebrating a blocked admission as somewhat unprofessional or unprofessional (Table 1).

The scenarios portrayed were well received, with more than 85% of faculty and trainees agreeing that the behaviors displayed were realistic. Those who perceived videos as very realistic were more likely to report intent to change behavior (93% vs 53%, P=0.01). Nearly two‐thirds of faculty and 67% of housestaff expressed agreement that they intended to change behavior based upon the experience (Table 2).

Qualitative comments in the constructed‐response portion of the evaluation noted the effectiveness of the interactive materials. In addition, the need for focused faculty development was identified by 1 respondent who stated: If unprofessional behavior is the unwritten curriculum, there needs to be an explicit, written curriculum to address it. Finally, the aim of facilitating self‐reflection is echoed in this faculty respondent's comment: Always good to be reminded of our behaviors and the influence they have on others and from this resident physician It helps to re‐evaluate how you talk to people.

CONCLUSIONS

Faculty can be a large determinant of the learning environment and impact trainees' professional development.[9] Hospitalists should be encouraged to embrace faculty role‐modeling of effective professional behaviors, especially given their increased presence in the inpatient learning environment. In addition, resident trainees and their behaviors contribute to the learning environment and influence the further professional development of more junior trainees.[10] Targeting professionalism education toward previously identified and prevalent unprofessional behaviors in the inpatient care of patients may serve to affect the most change among providers who practice in this setting. Individualized assessment of the learning environment may aid in identifying common scenarios that may plague a specific learning culture, allowing for relevant and targeted discussion of factors that promote and perpetuate such behaviors.[11]

Interactive, video‐based modules provided an effective way to promote interactive reflection and robust discussion. This model of experiential learning is an effective form of professional development as it engages the learner and stimulates ongoing incorporation of the topics addressed.[12, 13] Creating a shared concrete experience among targeted learners, using the video‐based scenarios, stimulates reflective observation, and ultimately experimentation, or incorporation into practice.[14]

There are several limitations to our evaluation including that we focused solely on academic hospitalist programs, and our sample size for faculty and residents was small. Also, we only addressed a small, though representative, sample of unprofessional behaviors and have not yet linked intervention to actual behavior change. Finally, the script scenarios that we used in this study were not previously published as they were created specifically for this intervention. Validity evidence for these scenarios include that they were based upon the results of earlier work from our institutions and underwent thorough peer review for content and clarity. Further studies will be required to do this. However, we do believe that these are positive findings for utilizing this type of interactive curriculum for professionalism education to promote self‐reflection and behavior change.

Video‐based professionalism education is a feasible, interactive mechanism to encourage self‐reflection and intent to change behavior among faculty and resident physicians. Future study is underway to conduct longitudinal assessments of the learning environments at the participating institutions to assess culture change, perceptions of behaviors, and sustainability of this type of intervention.

Disclosures: The authors acknowledge funding from the American Board of Internal Medicine. The funders had no role in the design of the study; the collection, analysis, and interpretation of the data; or the decision to approve publication of the finished manuscript. Results from this work have been presented at the Midwest Society of General Internal Medicine Regional Meeting, Chicago, Illinois, September 2011; Midwest Society of Hospital Medicine Regional Meeting, Chicago, Illinois, October 2011, and Society of Hospital Medicine Annual Meeting, San Diego, California, April 2012. The authors declare that they do not have any conflicts of interest to disclose.

The discrepancy between what is taught about professionalism in formal medical education and what is witnessed in the hospital has received increasing attention.17 This latter aspect of medical education contributes to the hidden curriculum and impacts medical trainees' views on professionalism.8 The hidden curriculum refers to the lessons trainees learn through informal interactions within the multilayered educational learning environment.9 A growing body of work examines how the hidden curriculum and disruptive physicians impact the learning environment.9, 10 In response, regulatory agencies, such as the Liaison Committee on Medical Education (LCME) and Accreditation Council for Graduate Medical Education (ACGME), require training programs and medical schools to maintain standards of professionalism, and to regularly evaluate the learning environment and its impact on professionalism.11, 12 The ACGME in 2011 expanded its standards regarding professionalism by making certain that the program director and institution ensure a culture of professionalism that supports patient safety and personal responsibility.11 Given this increasing focus on professionalism in medical school and residency training programs, it is critical to examine faculty perceptions and actions that may perpetuate the discrepancy between the formal and hidden curriculum.13 This early exposure is especially significant because unprofessional behavior in medical school is strongly associated with later disciplinary action by a medical board.14, 15 Certain unprofessional behaviors can also compromise patient care and safety, and can detract from the hospital working environment.1618

In our previous work, we demonstrated that internal medicine interns reported increased participation in unprofessional behaviors regarding on‐call etiquette during internship.19, 20 Examples of these behaviors include refusing an admission (ie, blocking) and misrepresenting a test as urgent. Interestingly, students and residents have highlighted the powerful role of supervising faculty physicians in condoning or inhibiting such behavior. Given the increasing role of hospitalists as resident supervisors, it is important to consider the perceptions and actions of hospitalists with respect to perpetuating or hindering some unprofessional behaviors. Although hospital medicine is a relatively new specialty, many hospitalists are in frequent contact with medical trainees, perhaps because many residency programs and medical schools have a strong inpatient focus.2123 It is thus possible that hospitalists have a major influence on residents' behaviors and views of professionalism. In fact, the Society of Hospital Medicine's Core Competencies for Hospital Medicine explicitly state that hospitalists are expected to serve as a role model for professional and ethical conduct to house staff, medical students and other members of the interdisciplinary team.24

Therefore, the current study had 2 aims: first, to measure internal medicine hospitalists' perceptions of, and participation in, unprofessional behaviors using a previously validated survey; and second, to examine associations between job characteristics and participation in unprofessional behaviors.

METHODS

RESULTS

Seventy‐seven of the 101 hospitalists (76.2%) at 3 sites completed the survey. While response rates varied by site (site 1, 67%; site 2, 74%; site 3, 86%), the differences were not statistically significant (² = 2.9, P = 0.24). Most hospitalists (79.2%) completed residency after 2000. Over half (57.1%) of participants were male, and over half (61%) reported having worked with their current hospitalist group from 1 to 4 years. Almost 60% (59.7%) reported being unfamiliar with residents in the program. Over 40% of hospitalists did not do any night work. Hospitalists were largely clinical, one‐quarter of hospitalists reported working over 50% FTE, and the median was 80% FTE. While 78% of hospitalists reported some teaching time, median time on teaching service was low at 10% (Table 1).

Hospitalists perceived almost all behaviors as unprofessional (unprofessional or somewhat unprofessional on a 5‐point Likert Scale). The only behavior rated as professional with a mean of 4.25 (95% CI 4.014.49) was staying past shift limit to complete a patient‐care task that could have been signed out. This behavior also had the highest level of participation by hospitalists (81.7%). Hospitalists were most ambivalent when rating professionalism of attending an industry‐sponsored dinner or social event (mean 3.20, 95% CI 2.983.41) (Table 2).

Participation in egregious behaviors, such as falsifying patient records (6.49%) and performing medical or surgical procedures on a patient beyond self‐perceived level of skill (2.60%), was very low. The most common behaviors rated as unprofessional that hospitalists reported participating in were having nonmedical/personal conversations in patient corridors (67.1%), ordering a routine test as urgent to expedite care (62.3%), and making fun of other physicians to colleagues (40.3%). Forty percent of participants reported disparaging the emergency room (ER) team or primary care physician for findings later discovered, signing out over the phone when it could have been done in person, and texting or using smartphones during educational conferences. In particular, participation in unprofessional behaviors related to trainees was close to zero (eg, asking a student to discuss, with patients, medical or surgical information which is perceived to be beyond their level of knowledge). The least common behaviors that hospitalists reported participating in were discharging a patient before they are ready to go home in order to reduce one's census (2.56%) and reporting patient information as normal when uncertain of the true results (2.60%). Like previous studies of unprofessional behaviors, those that reported participation were less likely to report the behavior as unprofessional.8, 19

Observation of behaviors ranged from 4% to 80%. In all cases, observation of the behavior was reported at a higher level than participation. Correlation between observation and participation was also high, with the exception of a few behaviors that had zero or near zero participation rates (ie, reporting patient information as normal when unsure of true results.)

After performing factor analysis, 4 factors had eigenvalues greater than 1 and were therefore retained and extracted for further analysis. These 4 factors accounted for 76% of the variance in responses reported on the survey. By examining which items or groups of items most strongly loaded on each factor, the factors were named accordingly: factor 1 referred to behaviors related to making fun of others, factor 2 referred to workload management, factor 3 referred to behaviors related to the learning environment, and factor 4 referred to behaviors related to time pressure (Table 3).

Using site‐adjusted multivariate regression, certain hospitalist job characteristics were associated with certain patterns of participating in unprofessional behavior (Table 4). Those with less clinical time (<50% FTE) were more likely to participate in unprofessional behaviors related to making fun of others (factor 1, value = 0.94, 95% CI 0.32 to 1.56, P value <0.05). Hospitalists who had any administrative time ( value = 0.61, 95% CI 0.111.10, P value <0.05) were more likely to report participation in behaviors related to workload management. Hospitalists engaged in any night work were more likely to report participation in unprofessional behaviors related to time pressure ( value = 0.67, 95% CI 0.171.17, P value <0.05). Time devoted to teaching or research was not associated with greater participation in any of the domains of unprofessional behavior surveyed.

The only demographic characteristic that was significantly associated with unprofessional behavior was age. Specifically, those who were born after 1970 were more likely to participate in unprofessional behaviors related to workload management ( value = 0.87, 95% CI 0.071.67, P value <0.05). Site differences were also present. Specifically, one site was more likely to report participation in unprofessional behaviors related to workload management ( value site 1 = 1.01, 95% CI 0.15 to 1.86, P value <0.05), while another site was less likely to report participation in behaviors related to the learning environment ( value site 3 = 0.70, 95% CI 1.31 to 0.09, P value <0.05). Gender and familiarity with residents were not significant predictors of participation in unprofessional behaviors. Results remained robust in sensitivity analyses using different cutoffs of clinical time and teaching time.

DISCUSSION

This multisite study adds to what is known about the perceptions of, and participation in, unprofessional behaviors among internal medicine hospitalists. Hospitalists perceived almost all surveyed behaviors as unprofessional. Participation in egregious and trainee‐related unprofessional behaviors was very low. Four categories appeared to explain the variability in how hospitalists reported participation in unprofessional behaviors: making fun of others, workload management, learning environment, and time pressure. Participation in behaviors within these factors was associated with certain job characteristics, such as clinical time, administrative time, and night work, as well as age and site.

It is reassuring that participation in, and trainee‐related, unprofessional behaviors is very low, and it is noteworthy that attending an industry‐sponsored dinner is not considered unprofessional. This was surprising in the setting of increased external pressures to report and ban such interactions.28 Perception that attending such dinners is acceptable may reflect a lag between current practice and national recommendations.

It is important to explore why certain job characteristics are associated with participation in unprofessional behaviors. For example, those with less clinical time were more likely to participate in making fun of others. It may be the case that hospitalists with more clinical time may make a larger effort to develop and maintain positive relationships. Another possible explanation is that hospitalists with less clinical time are more easily influenced by those in the learning environment who make fun of others, such as residents who they are supervising for only a brief period.

For unprofessional behaviors related to workload management, those who were younger, and those with any administrative time, were more likely to participate in behaviors such as celebrating a blocked‐admission. Our prior work shows that behaviors related to workload management are more widespread in residency, and therefore younger hospitalists, who are often recent residency graduates, may be more prone to participating in these behaviors. While unproven, it is possible that those with more administrative time may have competing priorities with their administrative roles, which motivate them to more actively manage their workload, leading them to participate in workload management behaviors.

Hospitalists who did any night work were more likely to participate in unprofessional behaviors related to time pressure. This could reflect the high workloads that night hospitalists may face and the pressure they feel to wrap up work, resulting in a hasty handoff (ie, over the phone) or to defer work (ie, family questions). Site differences were also observed for participation in behaviors related to the learning environment, speaking to the importance of institutional culture.

It is worth mentioning that hospitalists who were teachers were not any less likely to report participating in certain behaviors. While 78% of hospitalists reported some level of teaching, the median reported percentage of teaching was 10% FTE. This level of teaching likely reflects the diverse nature of work in which hospitalists engage. While hospitalists spend some time working with trainees, services that are not staffed with residents (eg, uncovered services) are becoming increasingly common due to stricter resident duty hour restrictions. This may explain why 60% of hospitalists reported being unfamiliar with residents. We also used a high bar for familiarity, which we defined as knowing half of residents by name, and served as a proxy for those who may have trained at the institution where they currently work. In spite of hospitalists reporting a low fraction of their total clinical time devoted to resident services, a significant fraction of resident services were staffed by hospitalists at all sites, making them a natural target for interventions.

These results have implications for future work to assess and improve professionalism in the hospital learning environment. First, interventions to address unprofessional behaviors should focus on behaviors with the highest participation rates. Like our earlier studies of residents, participation is high in certain behaviors, such as misrepresenting a test as urgent, or disparaging the ER or primary care physician (PCP) for a missed finding.19, 20 While blocking an admission was common in our studies of residents, reported participation among hospitalists was low. Similar to a prior study of clinical year medical students at one of our sites, 1 in 5 hospitalists reported not correcting someone who mistakes a student for a physician, highlighting the role that hospitalists may have in perpetuating this behavior.8 Additionally, addressing the behaviors identified in this study, through novel curricular tools, may help to teach residents many of the interpersonal and communication skills called for in the 2011 ACGME Common Program Requirements.11 The ACGME requirements also include the expectation that faculty model how to manage their time before, during, and after clinical assignments, and recognize that transferring a patient to a rested provider is best. Given that most hospitalists believe staying past shift limit is professional, these requirements will be difficult to adopt without widespread culture change.

Moreover, interventions could be tailored to hospitalists with certain job characteristics. Interventions may be educational or systems based. An example of the former is stressing the impact of the learning and working environment on trainees, and an example of the latter is streamlining the process in which ordered tests are executed to result in a more timely completion of tests. This may result in fewer physicians misrepresenting a test as urgent in order to have the test done in a timely manner. Additionally, hospitalists with less clinical time could receive education on their impact as a role model for trainees. Hospitalists who are younger or with administrative commitments could be trained on the importance of avoiding behaviors related to workload management, such as blocking or turfing patients. Lastly, given the site differences, critical examination of institutional culture and policies is also important. With funding from the American Board of Internal Medicine (ABIM) Foundation, we are currently creating an educational intervention, targeting those behaviors that were most frequent among hospitalists and residents at our institutions to promote dialogue and critical reflection, with the hope of reducing the most prevalent behaviors encountered.

There are several limitations to this study. Despite the anonymity of the survey, participants may have inaccurately reported their participation in unprofessional behaviors due to socially desirable response. In addition, because we used factor analysis and multivariate regression models with a small sample size, item nonresponse limited the sample for regression analyses and raises the concern for response bias. However, all significant associations remained so after performing backwards stepwise elimination of covariates that were P > 0.10 in models that were larger (ranging from 65 to 69). Because we used self‐report and not direct observation of participation in unprofessional behaviors, it is not possible to validate the responses given. Future work could rely on the use of 360 degree evaluations or other methods to validate responses given by self‐report. It is also important to consider assessing whether these behaviors are associated with actual patient outcomes, such as length of stay or readmission. Some items may not always be unprofessional. For example, texting during an educational conference might be to advance care, which would not necessarily be unprofessional. The order in which the questions were asked could have led to bias. We asked about participation before perception to try to limit bias reporting in participation. Changing the order of these questions would potentially have resulted in under‐reporting participation in behaviors that one perceived to be unprofessional. This study was conducted at 3 institutions located in Chicago, limiting generalizability to institutions outside of this area. Only internal medicine hospitalists were surveyed, which also limits generalizability to other disciplines and specialties within internal medicine. Lastly, it is important to highlight that hospitalists are not the sole teachers on inpatient services, since residents encounter a variety of faculty who serve as teaching attendings. Future work should expand to other centers and other specialties.

In conclusion, in this multi‐institutional study of hospitalists, participation in egregious behaviors was low. Four factors or patterns underlie hospitalists' reports of participation in unprofessional behavior: making fun of others, learning environment, workload management, and time pressure. Job characteristics (clinical time, administrative time, night work), age, and site were all associated with different patterns of unprofessional behavior. Specifically, hospitalists with less clinical time were more likely to make fun of others. Hospitalists who were younger in age, as well as those who had any administrative work, were more likely to participate in behaviors related to workload management. Hospitalists who work nights were more likely to report behaviors related to time pressure. Interventions to promote professionalism should take institutional culture into account and should focus on behaviors with the highest participation rates. Efforts should also be made to address underlying reasons for participation in these behaviors.

The discrepancy between what is taught about professionalism in formal medical education and what is witnessed in the hospital has received increasing attention.17 This latter aspect of medical education contributes to the hidden curriculum and impacts medical trainees' views on professionalism.8 The hidden curriculum refers to the lessons trainees learn through informal interactions within the multilayered educational learning environment.9 A growing body of work examines how the hidden curriculum and disruptive physicians impact the learning environment.9, 10 In response, regulatory agencies, such as the Liaison Committee on Medical Education (LCME) and Accreditation Council for Graduate Medical Education (ACGME), require training programs and medical schools to maintain standards of professionalism, and to regularly evaluate the learning environment and its impact on professionalism.11, 12 The ACGME in 2011 expanded its standards regarding professionalism by making certain that the program director and institution ensure a culture of professionalism that supports patient safety and personal responsibility.11 Given this increasing focus on professionalism in medical school and residency training programs, it is critical to examine faculty perceptions and actions that may perpetuate the discrepancy between the formal and hidden curriculum.13 This early exposure is especially significant because unprofessional behavior in medical school is strongly associated with later disciplinary action by a medical board.14, 15 Certain unprofessional behaviors can also compromise patient care and safety, and can detract from the hospital working environment.1618

In our previous work, we demonstrated that internal medicine interns reported increased participation in unprofessional behaviors regarding on‐call etiquette during internship.19, 20 Examples of these behaviors include refusing an admission (ie, blocking) and misrepresenting a test as urgent. Interestingly, students and residents have highlighted the powerful role of supervising faculty physicians in condoning or inhibiting such behavior. Given the increasing role of hospitalists as resident supervisors, it is important to consider the perceptions and actions of hospitalists with respect to perpetuating or hindering some unprofessional behaviors. Although hospital medicine is a relatively new specialty, many hospitalists are in frequent contact with medical trainees, perhaps because many residency programs and medical schools have a strong inpatient focus.2123 It is thus possible that hospitalists have a major influence on residents' behaviors and views of professionalism. In fact, the Society of Hospital Medicine's Core Competencies for Hospital Medicine explicitly state that hospitalists are expected to serve as a role model for professional and ethical conduct to house staff, medical students and other members of the interdisciplinary team.24

Therefore, the current study had 2 aims: first, to measure internal medicine hospitalists' perceptions of, and participation in, unprofessional behaviors using a previously validated survey; and second, to examine associations between job characteristics and participation in unprofessional behaviors.

METHODS

RESULTS

Seventy‐seven of the 101 hospitalists (76.2%) at 3 sites completed the survey. While response rates varied by site (site 1, 67%; site 2, 74%; site 3, 86%), the differences were not statistically significant (² = 2.9, P = 0.24). Most hospitalists (79.2%) completed residency after 2000. Over half (57.1%) of participants were male, and over half (61%) reported having worked with their current hospitalist group from 1 to 4 years. Almost 60% (59.7%) reported being unfamiliar with residents in the program. Over 40% of hospitalists did not do any night work. Hospitalists were largely clinical, one‐quarter of hospitalists reported working over 50% FTE, and the median was 80% FTE. While 78% of hospitalists reported some teaching time, median time on teaching service was low at 10% (Table 1).