William C. McGaghie, PhD

Internal medicine (IM) residents and hospitalist physicians commonly conduct bedside thoracenteses for both diagnostic and therapeutic purposes.[1] The American Board of Internal Medicine only requires that certification candidates understand the indications, complications, and management of thoracenteses.[2] A disconnect between clinical practice patterns and board requirements may increase patient risk because poorly trained physicians are more likely to cause complications.[3] National practice patterns show that many thoracenteses are referred to interventional radiology (IR).[4] However, research links performance of bedside procedures to reduced hospital length of stay and lower costs, without increasing risk of complications.[1, 5, 6]

Simulation‐based education offers a controlled environment where trainees improve procedural knowledge and skills without patient harm.[7] Simulation‐based mastery learning (SBML) is a rigorous form of competency‐based education that improves clinical skills and reduces iatrogenic complications and healthcare costs.[5, 6, 8] SBML also is an effective method to boost thoracentesis skills among IM residents.[9] However, there are no data to show that thoracentesis skills acquired in the simulation laboratory transfer to clinical environments and affect referral patterns.

We hypothesized that a thoracentesis SBML intervention would improve skills and increase procedural self‐confidence while reducing procedure referrals. This study aimed to (1) assess the effect of thoracentesis SBML on a cohort of IM residents' simulated skills and (2) compare traditionally trained (nonSBML‐trained) residents, SBML‐trained residents, and hospitalist physicians regarding procedure referral patterns, self‐confidence, procedure experience, and reasons for referral.

METHODS AND MATERIALS

RESULTS

DISCUSSION

This study confirms earlier research showing that thoracentesis SBML improves residents' clinical skills, but is the first to use a randomized study design.[9] Use of the mastery model in health professions education ensures that all learners are competent to provide patient care including performing invasive procedures. Such rigorous education yields downstream translational outcomes including safety profiles comparable to experts.[1, 6]

This study also shows that SBML‐trained residents displayed higher self‐confidence and performed significantly more bedside procedures than traditionally trained residents and more experienced hospitalist physicians. Although the Society of Hospital Medicine considers thoracentesis skills a core competency for hospitalist physicians,[11] we speculate that some hospitalist physicians had not performed a thoracentesis in years. A recent national survey showed that only 44% of hospitalist physicians performed at least 1 thoracentesis within the past year.[10] Research also shows a shift in medical culture to refer procedures to specialty services, such as IR, by over 900% in the past 2 decades.[4] Our results provide novel information about procedure referrals because we show that SBML provides translational outcomes by improving skills and self‐confidence that influence referral patterns. SBML‐trained residents performed almost a quarter of procedures at the bedside. Although this only represents an 8% absolute difference in bedside procedures compared to traditionally trained residents, if a large number of residents are trained using SBML this results in a meaningful number of procedures shifted to the patient bedside. According to University HealthSystem Consortium data, in US teaching hospitals, approximately 35,325 thoracenteses are performed yearly.[1] Shifting even 8% of these procedures to the bedside would result in significant clinical benefit and cost savings. Reduced referrals increase additional bedside procedures that are safe, cost‐effective, and highly satisfying to patients.[1, 12, 13] Further study is required to determine the impact on referral patterns after providing SMBL training to attending physicians.

Our study also provides information about the rationale for procedure referrals. Earlier work speculates that financial incentive, training and time may explain high procedure referral rates.[10] One report on IM residents noted an 87% IR referral rate for thoracentesis, and confirmed that both training and time were major reasons.[14] Hospitalist physicians reported lack of time as the major factor leading to procedural referrals, which is problematic because bedside procedures yield similar clinical outcomes at lower costs.[1, 12] Attending preference also prevented 11 additional bedside procedures in the SBML‐trained group. Schedule adjustments and SBML training of hospitalist physicians should be considered, because bundled payments in the Affordable Care Act may favor shifting to the higher‐value approach of bedside thoracenteses.[15]

Our study has several limitations. First, we only performed surveys at 1 institution and the results may not be generalizable. Second, we relied on an electronic query to alert us to thoracenteses. Our query may have missed procedures that were unsuccessful or did not have EHR orders entered. Third, physicians may have been surveyed more than once for different or the same patient(s), but opinions may have shifted over time. Fourth, some items such as time needed to be written in the survey and were not specifically asked. This could have resulted in under‐reporting. Finally, we did not assess the clinical outcomes of thoracenteses in this study, although earlier work shows that residents who complete SBML have safety outcomes similar to IR.[1, 6]

In summary, IM residents who complete thoracentesis SBML demonstrate improved clinical skills and are more likely to perform bedside procedures. In an era of bundled payments, rethinking current care models to promote cost‐effective care is necessary. We believe providing additional education, training, and support to hospitalist physicians to promote bedside procedures is a promising strategy that warrants further study.

Internal medicine (IM) residents and hospitalist physicians commonly conduct bedside thoracenteses for both diagnostic and therapeutic purposes.[1] The American Board of Internal Medicine only requires that certification candidates understand the indications, complications, and management of thoracenteses.[2] A disconnect between clinical practice patterns and board requirements may increase patient risk because poorly trained physicians are more likely to cause complications.[3] National practice patterns show that many thoracenteses are referred to interventional radiology (IR).[4] However, research links performance of bedside procedures to reduced hospital length of stay and lower costs, without increasing risk of complications.[1, 5, 6]

Simulation‐based education offers a controlled environment where trainees improve procedural knowledge and skills without patient harm.[7] Simulation‐based mastery learning (SBML) is a rigorous form of competency‐based education that improves clinical skills and reduces iatrogenic complications and healthcare costs.[5, 6, 8] SBML also is an effective method to boost thoracentesis skills among IM residents.[9] However, there are no data to show that thoracentesis skills acquired in the simulation laboratory transfer to clinical environments and affect referral patterns.

We hypothesized that a thoracentesis SBML intervention would improve skills and increase procedural self‐confidence while reducing procedure referrals. This study aimed to (1) assess the effect of thoracentesis SBML on a cohort of IM residents' simulated skills and (2) compare traditionally trained (nonSBML‐trained) residents, SBML‐trained residents, and hospitalist physicians regarding procedure referral patterns, self‐confidence, procedure experience, and reasons for referral.

METHODS AND MATERIALS

RESULTS

DISCUSSION

This study confirms earlier research showing that thoracentesis SBML improves residents' clinical skills, but is the first to use a randomized study design.[9] Use of the mastery model in health professions education ensures that all learners are competent to provide patient care including performing invasive procedures. Such rigorous education yields downstream translational outcomes including safety profiles comparable to experts.[1, 6]

This study also shows that SBML‐trained residents displayed higher self‐confidence and performed significantly more bedside procedures than traditionally trained residents and more experienced hospitalist physicians. Although the Society of Hospital Medicine considers thoracentesis skills a core competency for hospitalist physicians,[11] we speculate that some hospitalist physicians had not performed a thoracentesis in years. A recent national survey showed that only 44% of hospitalist physicians performed at least 1 thoracentesis within the past year.[10] Research also shows a shift in medical culture to refer procedures to specialty services, such as IR, by over 900% in the past 2 decades.[4] Our results provide novel information about procedure referrals because we show that SBML provides translational outcomes by improving skills and self‐confidence that influence referral patterns. SBML‐trained residents performed almost a quarter of procedures at the bedside. Although this only represents an 8% absolute difference in bedside procedures compared to traditionally trained residents, if a large number of residents are trained using SBML this results in a meaningful number of procedures shifted to the patient bedside. According to University HealthSystem Consortium data, in US teaching hospitals, approximately 35,325 thoracenteses are performed yearly.[1] Shifting even 8% of these procedures to the bedside would result in significant clinical benefit and cost savings. Reduced referrals increase additional bedside procedures that are safe, cost‐effective, and highly satisfying to patients.[1, 12, 13] Further study is required to determine the impact on referral patterns after providing SMBL training to attending physicians.

Our study also provides information about the rationale for procedure referrals. Earlier work speculates that financial incentive, training and time may explain high procedure referral rates.[10] One report on IM residents noted an 87% IR referral rate for thoracentesis, and confirmed that both training and time were major reasons.[14] Hospitalist physicians reported lack of time as the major factor leading to procedural referrals, which is problematic because bedside procedures yield similar clinical outcomes at lower costs.[1, 12] Attending preference also prevented 11 additional bedside procedures in the SBML‐trained group. Schedule adjustments and SBML training of hospitalist physicians should be considered, because bundled payments in the Affordable Care Act may favor shifting to the higher‐value approach of bedside thoracenteses.[15]

Our study has several limitations. First, we only performed surveys at 1 institution and the results may not be generalizable. Second, we relied on an electronic query to alert us to thoracenteses. Our query may have missed procedures that were unsuccessful or did not have EHR orders entered. Third, physicians may have been surveyed more than once for different or the same patient(s), but opinions may have shifted over time. Fourth, some items such as time needed to be written in the survey and were not specifically asked. This could have resulted in under‐reporting. Finally, we did not assess the clinical outcomes of thoracenteses in this study, although earlier work shows that residents who complete SBML have safety outcomes similar to IR.[1, 6]

In summary, IM residents who complete thoracentesis SBML demonstrate improved clinical skills and are more likely to perform bedside procedures. In an era of bundled payments, rethinking current care models to promote cost‐effective care is necessary. We believe providing additional education, training, and support to hospitalist physicians to promote bedside procedures is a promising strategy that warrants further study.

Internal medicine (IM) residents and hospitalist physicians commonly conduct bedside thoracenteses for both diagnostic and therapeutic purposes.[1] The American Board of Internal Medicine only requires that certification candidates understand the indications, complications, and management of thoracenteses.[2] A disconnect between clinical practice patterns and board requirements may increase patient risk because poorly trained physicians are more likely to cause complications.[3] National practice patterns show that many thoracenteses are referred to interventional radiology (IR).[4] However, research links performance of bedside procedures to reduced hospital length of stay and lower costs, without increasing risk of complications.[1, 5, 6]

Simulation‐based education offers a controlled environment where trainees improve procedural knowledge and skills without patient harm.[7] Simulation‐based mastery learning (SBML) is a rigorous form of competency‐based education that improves clinical skills and reduces iatrogenic complications and healthcare costs.[5, 6, 8] SBML also is an effective method to boost thoracentesis skills among IM residents.[9] However, there are no data to show that thoracentesis skills acquired in the simulation laboratory transfer to clinical environments and affect referral patterns.

We hypothesized that a thoracentesis SBML intervention would improve skills and increase procedural self‐confidence while reducing procedure referrals. This study aimed to (1) assess the effect of thoracentesis SBML on a cohort of IM residents' simulated skills and (2) compare traditionally trained (nonSBML‐trained) residents, SBML‐trained residents, and hospitalist physicians regarding procedure referral patterns, self‐confidence, procedure experience, and reasons for referral.

METHODS AND MATERIALS

RESULTS

DISCUSSION

This study confirms earlier research showing that thoracentesis SBML improves residents' clinical skills, but is the first to use a randomized study design.[9] Use of the mastery model in health professions education ensures that all learners are competent to provide patient care including performing invasive procedures. Such rigorous education yields downstream translational outcomes including safety profiles comparable to experts.[1, 6]

This study also shows that SBML‐trained residents displayed higher self‐confidence and performed significantly more bedside procedures than traditionally trained residents and more experienced hospitalist physicians. Although the Society of Hospital Medicine considers thoracentesis skills a core competency for hospitalist physicians,[11] we speculate that some hospitalist physicians had not performed a thoracentesis in years. A recent national survey showed that only 44% of hospitalist physicians performed at least 1 thoracentesis within the past year.[10] Research also shows a shift in medical culture to refer procedures to specialty services, such as IR, by over 900% in the past 2 decades.[4] Our results provide novel information about procedure referrals because we show that SBML provides translational outcomes by improving skills and self‐confidence that influence referral patterns. SBML‐trained residents performed almost a quarter of procedures at the bedside. Although this only represents an 8% absolute difference in bedside procedures compared to traditionally trained residents, if a large number of residents are trained using SBML this results in a meaningful number of procedures shifted to the patient bedside. According to University HealthSystem Consortium data, in US teaching hospitals, approximately 35,325 thoracenteses are performed yearly.[1] Shifting even 8% of these procedures to the bedside would result in significant clinical benefit and cost savings. Reduced referrals increase additional bedside procedures that are safe, cost‐effective, and highly satisfying to patients.[1, 12, 13] Further study is required to determine the impact on referral patterns after providing SMBL training to attending physicians.

Our study also provides information about the rationale for procedure referrals. Earlier work speculates that financial incentive, training and time may explain high procedure referral rates.[10] One report on IM residents noted an 87% IR referral rate for thoracentesis, and confirmed that both training and time were major reasons.[14] Hospitalist physicians reported lack of time as the major factor leading to procedural referrals, which is problematic because bedside procedures yield similar clinical outcomes at lower costs.[1, 12] Attending preference also prevented 11 additional bedside procedures in the SBML‐trained group. Schedule adjustments and SBML training of hospitalist physicians should be considered, because bundled payments in the Affordable Care Act may favor shifting to the higher‐value approach of bedside thoracenteses.[15]

Our study has several limitations. First, we only performed surveys at 1 institution and the results may not be generalizable. Second, we relied on an electronic query to alert us to thoracenteses. Our query may have missed procedures that were unsuccessful or did not have EHR orders entered. Third, physicians may have been surveyed more than once for different or the same patient(s), but opinions may have shifted over time. Fourth, some items such as time needed to be written in the survey and were not specifically asked. This could have resulted in under‐reporting. Finally, we did not assess the clinical outcomes of thoracenteses in this study, although earlier work shows that residents who complete SBML have safety outcomes similar to IR.[1, 6]

In summary, IM residents who complete thoracentesis SBML demonstrate improved clinical skills and are more likely to perform bedside procedures. In an era of bundled payments, rethinking current care models to promote cost‐effective care is necessary. We believe providing additional education, training, and support to hospitalist physicians to promote bedside procedures is a promising strategy that warrants further study.

Central venous catheter (CVC) insertions are commonly performed at the bedside in medical intensive care unit (MICU) settings. Internal medicine residents are required to demonstrate knowledge regarding CVC indications, complications, and sterile technique,1 and often perform the procedure during training. Education in CVC insertion is needed because many internal medicine residents are uncomfortable performing this procedure.2 CVC insertion also carries the risk of potentially life‐threatening complications including infection, pneumothorax, arterial puncture, deep vein thrombosis, and bleeding. Education and training may also contribute to improved patient care because increased physician experience with CVC insertion reduces complication risk.3, 4 Similarly, a higher number of needle passes or attempts during CVC insertion correlates with mechanical complications such as pneumothorax or arterial punctures.48 Pneumothorax rates for internal jugular (IJ) CVCs have been reported to range from 0% to 0.2% and for subclavian (SC) CVCs from 1.5% to 3.1%.4, 5 The arterial puncture rate for IJ CVCs ranges from 5.0% to 9.4% and for SC CVCs from 3.1% to 4.9%.4, 5 Proper use of ultrasound to assist with IJ CVC insertion has been shown to decrease these mechanical complications.4, 5 However, studies of ultrasound use with SC CVC insertion have mixed results.4

Simulation‐based training has been used in medical education to increase knowledge, provide opportunities for deliberate and safe practice, and shape the development of clinical skills.9, 10 We previously used simulation‐based mastery learning to improve the thoracentesis and advanced cardiac life support (ACLS) skills of internal medicine residents.11, 12 Although a few small studies have linked simulation‐based interventions to improved quality of care,1319 more work is needed to show that results from a simulated environment transfer to actual patient care.

This study had 2 aims. The first was to expand our simulation‐based mastery learning to CVC insertion using a CVC simulator and ultrasound device. The second was to assess quality indicators (number of needle passes, pneumothorax, arterial punctures, and need for catheter adjustment) and resident confidence related to actual CVC insertions in the MICU before and after an educational intervention.

Materials and Methods

Design

This was a cohort study20 of IJ and SC CVC insertions by 41 second‐ and third‐year internal medicine residents rotating through the MICU in a university‐affiliated program from October 2006 to February 2007. The Northwestern University Institutional Review Board approved the study. All study participants were required to give informed consent prior to participation.

Thirteen residents rotated through the MICU during a 6‐week preintervention phase. These residents served as a traditionally trained group that did not receive CVC insertion simulator training. Simultaneously, 28 residents who rotated through the MICU later in the study period received simulation‐based training in CVC insertion and served as the simulator‐trained group (Figure 1). Demographic data were obtained from the participants including age, gender, ethnicity, year of training, and scores on the United States Medical Licensing Examination (USMLE) Steps 1 and 2.

Figure 1

Simulator‐trained residents underwent baseline skill assessment (pretest) using a 27‐item checklist in IJ and SC CVC insertions (see Appendix). Checklists were developed by one author (J.H.B.) using appropriate references4, 5 and a step‐by‐step process,21 and reviewed for completeness by another author with expertise in checklist development (D.B.W.). Each skill or other action was listed in order and given equal weight. A dichotomous scoring scale of 1 = done correctly and 0 = done incorrectly/not done was imposed for each item. Assessments were performed using Simulab's CentralLineMan. This model features realistic tissue with ultrasound compatibility, an arterial pulse, and self‐sealing veins and skins. Needles, dilators, and guidewires can be inserted and realistic venous and arterial pressures demonstrated (Figure 2).

Figure 2

Residents in the simulator‐trained group received two, 2‐hour education sessions featuring a lecture, ultrasound training, deliberate practice with the CVC simulator, and feedback.22 Education sessions contained standardized didactic material on CVC indications and complications, as well as a stepwise demonstration of IJ and SC CVC insertions using ultrasound and landmark techniques. These sessions were supervised by a senior hospitalist faculty member with expertise in CVC insertions (J.H.B.). Residents were expected to use the ultrasound device for all IJ CVC insertions. However, its use was optional for SC CVC insertion. After training, residents were retested (posttest) and required to meet or exceed a minimum passing score (MPS) set by an expert panel for both IJ and SC procedures.23 This 11 member expert panel provided item‐based (Angoff) and group‐based (Hofstee) judgments on the 27‐item checklists as described previously.23

Residents who did not achieve the MPS had more deliberate practice and were retested until the MPS was reached; the key feature of mastery learning.24 After completing simulation‐based mastery learning in CVC insertion, the 28 simulator‐trained residents rotated through the MICU.

Results

All eligible residents participated in the study and completed the entire protocol. There was no significant difference in age, gender, ethnicity, year of training, or USMLE Step 1 and 2 scores between the traditionally‐trained and simulator‐trained groups.

Interrater reliability measured by the mean kappa coefficient was very high (_n = 0.94) across the 27 IJ and SC checklist items. No resident met the MPS (79.1%) for CVC insertion at baseline testing. In the simulator‐trained group, 25 of 28 (89%) residents achieved SC skill mastery and 27 of 28 (96%) achieved IJ skill mastery within the standard four hour curriculum. All residents subsequently reached the MPS with less than one hour of additional practice time. A graphic portrait of the residents' pretest and posttest performance on the simulated CVC clinical skills examination with descriptive statistics is shown in Figure 3. After the educational intervention, posttest scores significantly improved (p < 0.001), to meet or exceed the MPS.

Figure 3

Traditionally trained and simulator‐trained residents independently inserted 46 CVCs during the study period. Simulator‐trained residents required significantly fewer needle passes to insert all actual CVCs in the MICU compared to traditionally trained residents: mean (M) = 1.79, standard deviation (SD) = 1.03 versus M = 2.78, SD = 1.77 (p = 0.04). As shown in Table 1, the groups did not differ in pneumothorax, arterial puncture, or mean number of CVC adjustments. In addition, the groups did not differ in use of ultrasound for IJ or SC CVC insertions. One IJ CVC was inserted without ultrasound in the traditionally‐trained group; 2 were inserted without ultrasound in the simulator‐trained group. Ultrasound was not used during any SC CVC insertions in the traditionally‐trained group and was used for 1 SC CVC insertion in the simulator‐trained group.

Simulator‐trained residents displayed more self‐confidence about their procedural skills than traditionally‐trained residents (M = 81, SD = 11 versus M = 68, SD = 20, p = 0.02). Spearman correlations showed no practical association between resident self‐confidence and performance on CVC insertion quality indicators.

Discussion

This study demonstrates the use of a mastery learning model to develop CVC insertion skills to a high achievement level among internal medicine residents. Our data support prior work showing that procedural skills that are poor at baseline can be increased significantly using simulation‐based training and deliberate practice.1118, 28 This report on CVC insertion adds to the growing body of literature showing that simulation training complements standard medical education,1119, 28 and expands the clinical application of the mastery model beyond thoracentesis and ACLS.11, 12 Use of the mastery model described in this study also has important implications for patients. In our training program, residents are required to demonstrate procedural mastery in a simulated environment before independently performing a CVC insertion on an actual patient. This is in sharp contrast to the traditional clinical model of procedural training at the bedside, and may be used in other training programs and with other invasive procedures.

The second aim of our study was to determine the impact of simulation‐based training on actual clinical practice by residents in the MICU. To our knowledge, no prior study has demonstrated that simulation‐based training in CVC insertion improves patient outcomes. We believe our results advance what is known about the impact of simulation‐based training because simulator‐trained residents in this study performed actual CVC insertions in the MICU using significantly fewer needle passes. Needle passes have been used by other investigators as a surrogate measure for reduced CVC‐associated complications because mechanical complications rise exponentially with more than two insertion attempts.47, 29 We believe this finding demonstrates transfer of skill acquired from simulation‐based training to the actual clinical environment. It is possible that ultrasound training accounts for the improvement in the simulator‐trained group. However, we do not believe that ultrasound training is entirely responsible as prior work has shown that deliberate practice using mastery learning without ultrasound significantly improved resident performance of thoracentesis11 and ACLS12, 19 procedures. We did not show a significant reduction in complications such as pneumothorax or arterial puncture. This is likely due to the small sample size and the low number of procedures and complications during the study period.

Our results also show that resident self‐confidence regarding actual CVC insertions improved after simulation training. These findings are similar to prior reports linking improved confidence among trainees after simulation‐based training in CVC insertion.29, 30 Our results did not reveal a correlation between improved self‐confidence and clinical skill acquisition. Linking improved self‐confidence to improved clinical skill is important because self‐assessment does not always correlate with performance ability.31, 32

More study is needed to evaluate the impact of simulation‐based training on the quality of CVC insertions by trainees. Mechanisms shown to decrease complications of CVC placement include use of ultrasound,4, 7, 3336 full sterile barrier technique,3739 chlorhexidine skin preparations,4042 and nurse‐physician education.43 Our simulation‐training program incorporates each of these elements. We plan to expand our simulation‐based training intervention to a larger sample size to determine its impact on mechanical and infectious complication rates linked to CVC insertion.

This study has several limitations. It was performed at a single institution over a short time period. However, demonstration of significantly fewer needle passes and improved resident self‐confidence after simulator training are important findings that warrant further study. It was impossible to blind raters during the skills assessment examination about whether the resident was performing a pretest or posttest. This was accounted for by using a second rater, who was blind to the pretest and posttest status of the examinee. The arterial puncture rate of 7% among simulator‐trained residents was higher than expected, although it remains within published ranges.4, 5 Also, a low total number of CVCs were evaluated during the study. This is likely due to strict exclusion criteria employed in order to study the impact of simulation training. For example, CVC insertions were only evaluated if they were actually performed by study residents (supervised insertions were excluded) and femoral catheters were not evaluated. We did not track clinical experience with CVC insertion by residents before the study. Residents who were simulator‐trained may have had more clinical experience with CVC insertion and this may have impacted their performance. However, residents did not differ in year of training or clinical rotations, and there is clear evidence that clinical training is not a proxy for skill acquisition.44 Finally, outcome data were measured via resident questionnaires that relied on resident recall about CVC insertion rather than observer ratings. This method was selected because observer ratings could not be standardized given the large number of clinical supervisors in the MICU over the study period. Information about needle passes and arterial puncture also may not be documented in procedural notes and could not be obtained by medical record review. We attempted to minimize recall bias by surveying residents within 24 hours of CVC placement.

In conclusion, this study demonstrates that simulation‐based training and deliberate practice in a mastery learning setting improves performance of both simulated and actual CVC insertions by internal medicine residents. Procedural training remains an important component of internal medicine training, although internists are performing fewer invasive procedures now than in years past.45, 46 Use of a mastery model of CVC insertion requires that trainees demonstrate skill in a simulated environment before independently performing this invasive procedure on patients. Further study is needed to assess clinical outcomes such as reduced CVC‐related infections and mechanical complications after simulation‐based training.

Central venous catheter (CVC) insertions are commonly performed at the bedside in medical intensive care unit (MICU) settings. Internal medicine residents are required to demonstrate knowledge regarding CVC indications, complications, and sterile technique,1 and often perform the procedure during training. Education in CVC insertion is needed because many internal medicine residents are uncomfortable performing this procedure.2 CVC insertion also carries the risk of potentially life‐threatening complications including infection, pneumothorax, arterial puncture, deep vein thrombosis, and bleeding. Education and training may also contribute to improved patient care because increased physician experience with CVC insertion reduces complication risk.3, 4 Similarly, a higher number of needle passes or attempts during CVC insertion correlates with mechanical complications such as pneumothorax or arterial punctures.48 Pneumothorax rates for internal jugular (IJ) CVCs have been reported to range from 0% to 0.2% and for subclavian (SC) CVCs from 1.5% to 3.1%.4, 5 The arterial puncture rate for IJ CVCs ranges from 5.0% to 9.4% and for SC CVCs from 3.1% to 4.9%.4, 5 Proper use of ultrasound to assist with IJ CVC insertion has been shown to decrease these mechanical complications.4, 5 However, studies of ultrasound use with SC CVC insertion have mixed results.4

Simulation‐based training has been used in medical education to increase knowledge, provide opportunities for deliberate and safe practice, and shape the development of clinical skills.9, 10 We previously used simulation‐based mastery learning to improve the thoracentesis and advanced cardiac life support (ACLS) skills of internal medicine residents.11, 12 Although a few small studies have linked simulation‐based interventions to improved quality of care,1319 more work is needed to show that results from a simulated environment transfer to actual patient care.

This study had 2 aims. The first was to expand our simulation‐based mastery learning to CVC insertion using a CVC simulator and ultrasound device. The second was to assess quality indicators (number of needle passes, pneumothorax, arterial punctures, and need for catheter adjustment) and resident confidence related to actual CVC insertions in the MICU before and after an educational intervention.

Materials and Methods

Design

This was a cohort study20 of IJ and SC CVC insertions by 41 second‐ and third‐year internal medicine residents rotating through the MICU in a university‐affiliated program from October 2006 to February 2007. The Northwestern University Institutional Review Board approved the study. All study participants were required to give informed consent prior to participation.

Thirteen residents rotated through the MICU during a 6‐week preintervention phase. These residents served as a traditionally trained group that did not receive CVC insertion simulator training. Simultaneously, 28 residents who rotated through the MICU later in the study period received simulation‐based training in CVC insertion and served as the simulator‐trained group (Figure 1). Demographic data were obtained from the participants including age, gender, ethnicity, year of training, and scores on the United States Medical Licensing Examination (USMLE) Steps 1 and 2.

Figure 1

Simulator‐trained residents underwent baseline skill assessment (pretest) using a 27‐item checklist in IJ and SC CVC insertions (see Appendix). Checklists were developed by one author (J.H.B.) using appropriate references4, 5 and a step‐by‐step process,21 and reviewed for completeness by another author with expertise in checklist development (D.B.W.). Each skill or other action was listed in order and given equal weight. A dichotomous scoring scale of 1 = done correctly and 0 = done incorrectly/not done was imposed for each item. Assessments were performed using Simulab's CentralLineMan. This model features realistic tissue with ultrasound compatibility, an arterial pulse, and self‐sealing veins and skins. Needles, dilators, and guidewires can be inserted and realistic venous and arterial pressures demonstrated (Figure 2).

Figure 2

Residents in the simulator‐trained group received two, 2‐hour education sessions featuring a lecture, ultrasound training, deliberate practice with the CVC simulator, and feedback.22 Education sessions contained standardized didactic material on CVC indications and complications, as well as a stepwise demonstration of IJ and SC CVC insertions using ultrasound and landmark techniques. These sessions were supervised by a senior hospitalist faculty member with expertise in CVC insertions (J.H.B.). Residents were expected to use the ultrasound device for all IJ CVC insertions. However, its use was optional for SC CVC insertion. After training, residents were retested (posttest) and required to meet or exceed a minimum passing score (MPS) set by an expert panel for both IJ and SC procedures.23 This 11 member expert panel provided item‐based (Angoff) and group‐based (Hofstee) judgments on the 27‐item checklists as described previously.23

Residents who did not achieve the MPS had more deliberate practice and were retested until the MPS was reached; the key feature of mastery learning.24 After completing simulation‐based mastery learning in CVC insertion, the 28 simulator‐trained residents rotated through the MICU.

Results

All eligible residents participated in the study and completed the entire protocol. There was no significant difference in age, gender, ethnicity, year of training, or USMLE Step 1 and 2 scores between the traditionally‐trained and simulator‐trained groups.

Interrater reliability measured by the mean kappa coefficient was very high (_n = 0.94) across the 27 IJ and SC checklist items. No resident met the MPS (79.1%) for CVC insertion at baseline testing. In the simulator‐trained group, 25 of 28 (89%) residents achieved SC skill mastery and 27 of 28 (96%) achieved IJ skill mastery within the standard four hour curriculum. All residents subsequently reached the MPS with less than one hour of additional practice time. A graphic portrait of the residents' pretest and posttest performance on the simulated CVC clinical skills examination with descriptive statistics is shown in Figure 3. After the educational intervention, posttest scores significantly improved (p < 0.001), to meet or exceed the MPS.

Figure 3

Traditionally trained and simulator‐trained residents independently inserted 46 CVCs during the study period. Simulator‐trained residents required significantly fewer needle passes to insert all actual CVCs in the MICU compared to traditionally trained residents: mean (M) = 1.79, standard deviation (SD) = 1.03 versus M = 2.78, SD = 1.77 (p = 0.04). As shown in Table 1, the groups did not differ in pneumothorax, arterial puncture, or mean number of CVC adjustments. In addition, the groups did not differ in use of ultrasound for IJ or SC CVC insertions. One IJ CVC was inserted without ultrasound in the traditionally‐trained group; 2 were inserted without ultrasound in the simulator‐trained group. Ultrasound was not used during any SC CVC insertions in the traditionally‐trained group and was used for 1 SC CVC insertion in the simulator‐trained group.

Simulator‐trained residents displayed more self‐confidence about their procedural skills than traditionally‐trained residents (M = 81, SD = 11 versus M = 68, SD = 20, p = 0.02). Spearman correlations showed no practical association between resident self‐confidence and performance on CVC insertion quality indicators.

Discussion

This study demonstrates the use of a mastery learning model to develop CVC insertion skills to a high achievement level among internal medicine residents. Our data support prior work showing that procedural skills that are poor at baseline can be increased significantly using simulation‐based training and deliberate practice.1118, 28 This report on CVC insertion adds to the growing body of literature showing that simulation training complements standard medical education,1119, 28 and expands the clinical application of the mastery model beyond thoracentesis and ACLS.11, 12 Use of the mastery model described in this study also has important implications for patients. In our training program, residents are required to demonstrate procedural mastery in a simulated environment before independently performing a CVC insertion on an actual patient. This is in sharp contrast to the traditional clinical model of procedural training at the bedside, and may be used in other training programs and with other invasive procedures.

The second aim of our study was to determine the impact of simulation‐based training on actual clinical practice by residents in the MICU. To our knowledge, no prior study has demonstrated that simulation‐based training in CVC insertion improves patient outcomes. We believe our results advance what is known about the impact of simulation‐based training because simulator‐trained residents in this study performed actual CVC insertions in the MICU using significantly fewer needle passes. Needle passes have been used by other investigators as a surrogate measure for reduced CVC‐associated complications because mechanical complications rise exponentially with more than two insertion attempts.47, 29 We believe this finding demonstrates transfer of skill acquired from simulation‐based training to the actual clinical environment. It is possible that ultrasound training accounts for the improvement in the simulator‐trained group. However, we do not believe that ultrasound training is entirely responsible as prior work has shown that deliberate practice using mastery learning without ultrasound significantly improved resident performance of thoracentesis11 and ACLS12, 19 procedures. We did not show a significant reduction in complications such as pneumothorax or arterial puncture. This is likely due to the small sample size and the low number of procedures and complications during the study period.

Our results also show that resident self‐confidence regarding actual CVC insertions improved after simulation training. These findings are similar to prior reports linking improved confidence among trainees after simulation‐based training in CVC insertion.29, 30 Our results did not reveal a correlation between improved self‐confidence and clinical skill acquisition. Linking improved self‐confidence to improved clinical skill is important because self‐assessment does not always correlate with performance ability.31, 32

More study is needed to evaluate the impact of simulation‐based training on the quality of CVC insertions by trainees. Mechanisms shown to decrease complications of CVC placement include use of ultrasound,4, 7, 3336 full sterile barrier technique,3739 chlorhexidine skin preparations,4042 and nurse‐physician education.43 Our simulation‐training program incorporates each of these elements. We plan to expand our simulation‐based training intervention to a larger sample size to determine its impact on mechanical and infectious complication rates linked to CVC insertion.

This study has several limitations. It was performed at a single institution over a short time period. However, demonstration of significantly fewer needle passes and improved resident self‐confidence after simulator training are important findings that warrant further study. It was impossible to blind raters during the skills assessment examination about whether the resident was performing a pretest or posttest. This was accounted for by using a second rater, who was blind to the pretest and posttest status of the examinee. The arterial puncture rate of 7% among simulator‐trained residents was higher than expected, although it remains within published ranges.4, 5 Also, a low total number of CVCs were evaluated during the study. This is likely due to strict exclusion criteria employed in order to study the impact of simulation training. For example, CVC insertions were only evaluated if they were actually performed by study residents (supervised insertions were excluded) and femoral catheters were not evaluated. We did not track clinical experience with CVC insertion by residents before the study. Residents who were simulator‐trained may have had more clinical experience with CVC insertion and this may have impacted their performance. However, residents did not differ in year of training or clinical rotations, and there is clear evidence that clinical training is not a proxy for skill acquisition.44 Finally, outcome data were measured via resident questionnaires that relied on resident recall about CVC insertion rather than observer ratings. This method was selected because observer ratings could not be standardized given the large number of clinical supervisors in the MICU over the study period. Information about needle passes and arterial puncture also may not be documented in procedural notes and could not be obtained by medical record review. We attempted to minimize recall bias by surveying residents within 24 hours of CVC placement.

In conclusion, this study demonstrates that simulation‐based training and deliberate practice in a mastery learning setting improves performance of both simulated and actual CVC insertions by internal medicine residents. Procedural training remains an important component of internal medicine training, although internists are performing fewer invasive procedures now than in years past.45, 46 Use of a mastery model of CVC insertion requires that trainees demonstrate skill in a simulated environment before independently performing this invasive procedure on patients. Further study is needed to assess clinical outcomes such as reduced CVC‐related infections and mechanical complications after simulation‐based training.

Central venous catheter (CVC) insertions are commonly performed at the bedside in medical intensive care unit (MICU) settings. Internal medicine residents are required to demonstrate knowledge regarding CVC indications, complications, and sterile technique,1 and often perform the procedure during training. Education in CVC insertion is needed because many internal medicine residents are uncomfortable performing this procedure.2 CVC insertion also carries the risk of potentially life‐threatening complications including infection, pneumothorax, arterial puncture, deep vein thrombosis, and bleeding. Education and training may also contribute to improved patient care because increased physician experience with CVC insertion reduces complication risk.3, 4 Similarly, a higher number of needle passes or attempts during CVC insertion correlates with mechanical complications such as pneumothorax or arterial punctures.48 Pneumothorax rates for internal jugular (IJ) CVCs have been reported to range from 0% to 0.2% and for subclavian (SC) CVCs from 1.5% to 3.1%.4, 5 The arterial puncture rate for IJ CVCs ranges from 5.0% to 9.4% and for SC CVCs from 3.1% to 4.9%.4, 5 Proper use of ultrasound to assist with IJ CVC insertion has been shown to decrease these mechanical complications.4, 5 However, studies of ultrasound use with SC CVC insertion have mixed results.4

Simulation‐based training has been used in medical education to increase knowledge, provide opportunities for deliberate and safe practice, and shape the development of clinical skills.9, 10 We previously used simulation‐based mastery learning to improve the thoracentesis and advanced cardiac life support (ACLS) skills of internal medicine residents.11, 12 Although a few small studies have linked simulation‐based interventions to improved quality of care,1319 more work is needed to show that results from a simulated environment transfer to actual patient care.

This study had 2 aims. The first was to expand our simulation‐based mastery learning to CVC insertion using a CVC simulator and ultrasound device. The second was to assess quality indicators (number of needle passes, pneumothorax, arterial punctures, and need for catheter adjustment) and resident confidence related to actual CVC insertions in the MICU before and after an educational intervention.

Materials and Methods

Design

This was a cohort study20 of IJ and SC CVC insertions by 41 second‐ and third‐year internal medicine residents rotating through the MICU in a university‐affiliated program from October 2006 to February 2007. The Northwestern University Institutional Review Board approved the study. All study participants were required to give informed consent prior to participation.

Thirteen residents rotated through the MICU during a 6‐week preintervention phase. These residents served as a traditionally trained group that did not receive CVC insertion simulator training. Simultaneously, 28 residents who rotated through the MICU later in the study period received simulation‐based training in CVC insertion and served as the simulator‐trained group (Figure 1). Demographic data were obtained from the participants including age, gender, ethnicity, year of training, and scores on the United States Medical Licensing Examination (USMLE) Steps 1 and 2.

Figure 1

Simulator‐trained residents underwent baseline skill assessment (pretest) using a 27‐item checklist in IJ and SC CVC insertions (see Appendix). Checklists were developed by one author (J.H.B.) using appropriate references4, 5 and a step‐by‐step process,21 and reviewed for completeness by another author with expertise in checklist development (D.B.W.). Each skill or other action was listed in order and given equal weight. A dichotomous scoring scale of 1 = done correctly and 0 = done incorrectly/not done was imposed for each item. Assessments were performed using Simulab's CentralLineMan. This model features realistic tissue with ultrasound compatibility, an arterial pulse, and self‐sealing veins and skins. Needles, dilators, and guidewires can be inserted and realistic venous and arterial pressures demonstrated (Figure 2).

Figure 2

Residents in the simulator‐trained group received two, 2‐hour education sessions featuring a lecture, ultrasound training, deliberate practice with the CVC simulator, and feedback.22 Education sessions contained standardized didactic material on CVC indications and complications, as well as a stepwise demonstration of IJ and SC CVC insertions using ultrasound and landmark techniques. These sessions were supervised by a senior hospitalist faculty member with expertise in CVC insertions (J.H.B.). Residents were expected to use the ultrasound device for all IJ CVC insertions. However, its use was optional for SC CVC insertion. After training, residents were retested (posttest) and required to meet or exceed a minimum passing score (MPS) set by an expert panel for both IJ and SC procedures.23 This 11 member expert panel provided item‐based (Angoff) and group‐based (Hofstee) judgments on the 27‐item checklists as described previously.23

Residents who did not achieve the MPS had more deliberate practice and were retested until the MPS was reached; the key feature of mastery learning.24 After completing simulation‐based mastery learning in CVC insertion, the 28 simulator‐trained residents rotated through the MICU.

Results

All eligible residents participated in the study and completed the entire protocol. There was no significant difference in age, gender, ethnicity, year of training, or USMLE Step 1 and 2 scores between the traditionally‐trained and simulator‐trained groups.

Interrater reliability measured by the mean kappa coefficient was very high (_n = 0.94) across the 27 IJ and SC checklist items. No resident met the MPS (79.1%) for CVC insertion at baseline testing. In the simulator‐trained group, 25 of 28 (89%) residents achieved SC skill mastery and 27 of 28 (96%) achieved IJ skill mastery within the standard four hour curriculum. All residents subsequently reached the MPS with less than one hour of additional practice time. A graphic portrait of the residents' pretest and posttest performance on the simulated CVC clinical skills examination with descriptive statistics is shown in Figure 3. After the educational intervention, posttest scores significantly improved (p < 0.001), to meet or exceed the MPS.

Figure 3

Traditionally trained and simulator‐trained residents independently inserted 46 CVCs during the study period. Simulator‐trained residents required significantly fewer needle passes to insert all actual CVCs in the MICU compared to traditionally trained residents: mean (M) = 1.79, standard deviation (SD) = 1.03 versus M = 2.78, SD = 1.77 (p = 0.04). As shown in Table 1, the groups did not differ in pneumothorax, arterial puncture, or mean number of CVC adjustments. In addition, the groups did not differ in use of ultrasound for IJ or SC CVC insertions. One IJ CVC was inserted without ultrasound in the traditionally‐trained group; 2 were inserted without ultrasound in the simulator‐trained group. Ultrasound was not used during any SC CVC insertions in the traditionally‐trained group and was used for 1 SC CVC insertion in the simulator‐trained group.

Simulator‐trained residents displayed more self‐confidence about their procedural skills than traditionally‐trained residents (M = 81, SD = 11 versus M = 68, SD = 20, p = 0.02). Spearman correlations showed no practical association between resident self‐confidence and performance on CVC insertion quality indicators.

Discussion

This study demonstrates the use of a mastery learning model to develop CVC insertion skills to a high achievement level among internal medicine residents. Our data support prior work showing that procedural skills that are poor at baseline can be increased significantly using simulation‐based training and deliberate practice.1118, 28 This report on CVC insertion adds to the growing body of literature showing that simulation training complements standard medical education,1119, 28 and expands the clinical application of the mastery model beyond thoracentesis and ACLS.11, 12 Use of the mastery model described in this study also has important implications for patients. In our training program, residents are required to demonstrate procedural mastery in a simulated environment before independently performing a CVC insertion on an actual patient. This is in sharp contrast to the traditional clinical model of procedural training at the bedside, and may be used in other training programs and with other invasive procedures.

The second aim of our study was to determine the impact of simulation‐based training on actual clinical practice by residents in the MICU. To our knowledge, no prior study has demonstrated that simulation‐based training in CVC insertion improves patient outcomes. We believe our results advance what is known about the impact of simulation‐based training because simulator‐trained residents in this study performed actual CVC insertions in the MICU using significantly fewer needle passes. Needle passes have been used by other investigators as a surrogate measure for reduced CVC‐associated complications because mechanical complications rise exponentially with more than two insertion attempts.47, 29 We believe this finding demonstrates transfer of skill acquired from simulation‐based training to the actual clinical environment. It is possible that ultrasound training accounts for the improvement in the simulator‐trained group. However, we do not believe that ultrasound training is entirely responsible as prior work has shown that deliberate practice using mastery learning without ultrasound significantly improved resident performance of thoracentesis11 and ACLS12, 19 procedures. We did not show a significant reduction in complications such as pneumothorax or arterial puncture. This is likely due to the small sample size and the low number of procedures and complications during the study period.

Our results also show that resident self‐confidence regarding actual CVC insertions improved after simulation training. These findings are similar to prior reports linking improved confidence among trainees after simulation‐based training in CVC insertion.29, 30 Our results did not reveal a correlation between improved self‐confidence and clinical skill acquisition. Linking improved self‐confidence to improved clinical skill is important because self‐assessment does not always correlate with performance ability.31, 32

More study is needed to evaluate the impact of simulation‐based training on the quality of CVC insertions by trainees. Mechanisms shown to decrease complications of CVC placement include use of ultrasound,4, 7, 3336 full sterile barrier technique,3739 chlorhexidine skin preparations,4042 and nurse‐physician education.43 Our simulation‐training program incorporates each of these elements. We plan to expand our simulation‐based training intervention to a larger sample size to determine its impact on mechanical and infectious complication rates linked to CVC insertion.

This study has several limitations. It was performed at a single institution over a short time period. However, demonstration of significantly fewer needle passes and improved resident self‐confidence after simulator training are important findings that warrant further study. It was impossible to blind raters during the skills assessment examination about whether the resident was performing a pretest or posttest. This was accounted for by using a second rater, who was blind to the pretest and posttest status of the examinee. The arterial puncture rate of 7% among simulator‐trained residents was higher than expected, although it remains within published ranges.4, 5 Also, a low total number of CVCs were evaluated during the study. This is likely due to strict exclusion criteria employed in order to study the impact of simulation training. For example, CVC insertions were only evaluated if they were actually performed by study residents (supervised insertions were excluded) and femoral catheters were not evaluated. We did not track clinical experience with CVC insertion by residents before the study. Residents who were simulator‐trained may have had more clinical experience with CVC insertion and this may have impacted their performance. However, residents did not differ in year of training or clinical rotations, and there is clear evidence that clinical training is not a proxy for skill acquisition.44 Finally, outcome data were measured via resident questionnaires that relied on resident recall about CVC insertion rather than observer ratings. This method was selected because observer ratings could not be standardized given the large number of clinical supervisors in the MICU over the study period. Information about needle passes and arterial puncture also may not be documented in procedural notes and could not be obtained by medical record review. We attempted to minimize recall bias by surveying residents within 24 hours of CVC placement.

In conclusion, this study demonstrates that simulation‐based training and deliberate practice in a mastery learning setting improves performance of both simulated and actual CVC insertions by internal medicine residents. Procedural training remains an important component of internal medicine training, although internists are performing fewer invasive procedures now than in years past.45, 46 Use of a mastery model of CVC insertion requires that trainees demonstrate skill in a simulated environment before independently performing this invasive procedure on patients. Further study is needed to assess clinical outcomes such as reduced CVC‐related infections and mechanical complications after simulation‐based training.

In a supplement to its inaugural issue, the Journal of Hospital Medicine published core competencies for hospitalists covering 3 areas: clinical conditions, systems in health care, and procedures.1 Completion of a traditional internal medicine residency may not provide hospitalists with the skills necessary to safely perform necessary procedures such as thoracentesis. A recent article reported that most internal medicine residents surveyed were uncomfortable performing common procedures, and their discomfort was higher for thoracentesis than for central line insertion, lumbar puncture, or paracentesis.2 This confirmed a previous report that family practice residents had low confidence in performing thoracenteses.3 Thoracentesis also carries the risk of the potentially life‐threatening complication of pneumothorax, which may be increased when performed by physicians‐in‐training.4

One method for improving training and assessment is the use of simulation technology. Simulation has been used to increase knowledge, provide opportunities for deliberate and safe practice, and shape the development of clinical skills.5, 6 Simulation has also been advocated for assessing competence in procedures including carotid angiography,7 emergency airway management,8 basic bronchoscopy,9 and advanced cardiac life support (ACLS).10, 11

Recently, we used simulation technology to help residents reach mastery learning standards for ACLS.11 Mastery learning,12 an extreme form of competency‐based education,13 implies that learners have acquired the clinical knowledge and skill measured against rigorous achievement standards. In mastery learning, educational results are equivalent, whereas educational practice time differs. To demonstrate mastery learning, we first documented a 38% improvement in skill after a simulation‐based educational intervention10 and used a multidisciplinary panel to determine mastery achievement standards for ACLS skills in 6 clinical scenarios.14 These standards were used in a study in which the amount of time needed to achieve skill mastery was allowed to vary while the skill outcomes of the residents were identical clinically.11

The present study had 4 aims. The first was to assess the baseline skill and knowledge of third‐year residents in thoracentesis. The second was to compare the thoracentesis‐related knowledge and skills of residents before and after an educational intervention. The third was to assess the correlation of medical knowledge and clinical experience with performance on a clinical skills examination after simulation training. The last was to document the feasibility of incorporating simulation‐based education into a training program.

METHODS

RESULTS

All residents consented to participate and completed the entire training protocol. Table 1 presents demographic data about the residents. Most had limited experience performing and supervising thoracenteses.

Interrater reliability for the thoracentesis checklist data was calculated at pretest. Across the 25 checklist items, the mean kappa coefficient was very high (_n = .94). The MPS used as the mastery achievement standard was 80% (eg, 20 of 25 checklist items). This was the mean of the Angoff and Hofstee ratings obtained from the first judgment of the expert panel and is displayed in Figure 1.

Figure 1

No resident achieved mastery at pretest. However, 37 of the 40 medicine residents (93%) achieved mastery within the standard 4‐hour thoracentesis curriculum. The remaining 3 residents (7%) needed extra time ranging from 20 to 90 minutes to reach mastery.

Figure 1 is a graphic portrait with descriptive statistics of the residents' pretest and posttest performance on the thoracentesis written and clinical skills exams. For the written exam, the mean score rose from 57.63% to 89.75%, a statistically significant improvement of 56% from pretest to posttest (t_[39] = 17.0, P < .0001). The clinical skills exam also showed a highly significant 71% pretest‐to‐posttest gain, as the mean score rose from 51.70% to 88.3% (t_[39] = 15.6, P < .0001).

Results from the regression analysis indicate that neither pretest performance, medical knowledge measured by local or USMLE examinations, nor thoracentesis clinical experience was correlated with the posttest measure of thoracentesis clinical skills. However, the need for additional practice to reach the mastery standard on the posttest was a powerful negative predictor of posttest performance: b = .27 (95% CI = .46 to .09; P < .006; r² = .28). For those residents who required extra practice time, the initial clinical skills posttest score was 20% lower than that of their peers. Although the need for extra deliberate practice was associated with relatively lower initial posttest scores, all residents ultimately met or exceeded the rigorous thoracentesis MPS.

The responses of the 40 residents on a course evaluation questionnaire were uniformly positive. Responses were recorded on a Likert scale where 1 = strongly disagree, 2 = disagree, 3 = uncertain, 4 = agree, and 5 = strongly agree (Table 2). The data show that residents strongly agreed that practice with the medical simulator boosts clinical skills and self‐confidence, that they received useful feedback from the training sessions, and that deliberate practice using the simulator is a valuable educational experience. Residents were uncertain whether practice with the medical simulator has more educational value than patient care.

DISCUSSION

This study demonstrates the use of a mastery learning model to develop the thoracentesis skills of internal medicine residents to a high level. Use of a thoracentesis model in a structured educational program offering an opportunity for deliberate practice with feedback produced large and consistent improvements in residents' skills. An important finding of our study is that despite having completed most of their internal medicine training, residents displayed poor knowledge and clinical skill in thoracentesis procedures at baseline. This is similar to previous studies showing that the procedural skills and knowledge of physicians at all stages of training are often poor. Examples of areas in which significant gaps were found include basic skills such as chest radiography,27 emergency airway management,8 and pulmonary auscultation.28 In contrast, after the mastery learning program, all the residents met or exceeded the MPS for the thoracentesis clinical procedure and scored much higher on the posttest written examination.

Our data also demonstrate that medical knowledge measured by procedure‐specific pretests and posttests and USMLE Steps 1 and 2 scores were not correlated with thoracentesis skill acquisition. This reinforces findings from our previous studies of ACLS skill acquisition10, 11 and supports the difference between professional and academic achievement. Pretest skill performance and clinical experience also were not correlated with posttest outcomes. However, the amount of deliberate practice needed to reach the mastery standard was a powerful negative predictor of posttest thoracentesis skill scores, replicating our research on ACLS.11 We believe that clinical experience was not correlated with posttest outcomes because residents infrequently performed thoracenteses procedures during their training.

This project demonstrates a practical model for outcomes‐based education, certification, and program accreditation. Given the need to move procedural training in internal medicine beyond such historical methods as see one, do one, teach one,29 extension of the mastery model to other invasive procedures deserves further study. At our institution we have been encouraged by the ability of simulation‐based education in ACLS to promote long‐term skill retention30 and improvement in the quality of actual patient care.31 In addition to studying these outcomes for thoracentesis, we plan to incorporate the use of ultrasound when training residents to perform procedures such as thoracentesis and central venous catheter insertion.

Given concerns about the quality of resident preparation to perform invasive procedures, programs such as this should be considered as part of the procedural certification process. As shown by our experience with several classes of residents (n = 158), use of simulation technology to reach high procedural skill levels is effective and feasible in internal medicine residency training. In addition, our residents have consistently enjoyed participating in the simulated training programs. Postcourse questionnaires show that residents agree that deliberate practice with simulation technology complements but does not replace patient care in graduate medical education.5, 10

An important question needing more research is whether performance in a simulated environment transfers to actual clinical settings. Several small studies have demonstrated such a relationship,8, 9, 31, 32 yet the transfer of simulated training to clinical practice requires further study. More work should also be done to assess long‐term retention of skills30 and to determine the utility and benefit of simulation‐based training in procedural certification and credentialing.

This study had several limitations. It was conducted in 1 training program at a single medical center. The sample size (n = 40) was relatively small. The thoracentesis model was used for both education and testing, potentially confounding the events. However, these limitations do not diminish the pronounced impact that the simulation‐based training had on the skills and knowledge of our residents.

In conclusion, this study has demonstrated the ability of deliberate practice using a thoracentesis model to produce high‐level performance of simulated thoracenteses. The project received high ratings from learners and provides reliable assessments of procedural competence. Although internists are performing fewer invasive procedures now than in years past, procedural training is still an important component of internal medicine training.29, 33 Attainment of high procedural skill levels may be especially important for residents who plan to practice hospital medicine. We believe that simulation‐based training using deliberate practice should be a key contributor to future internal medicine residency education, certification, and accreditation.

In a supplement to its inaugural issue, the Journal of Hospital Medicine published core competencies for hospitalists covering 3 areas: clinical conditions, systems in health care, and procedures.1 Completion of a traditional internal medicine residency may not provide hospitalists with the skills necessary to safely perform necessary procedures such as thoracentesis. A recent article reported that most internal medicine residents surveyed were uncomfortable performing common procedures, and their discomfort was higher for thoracentesis than for central line insertion, lumbar puncture, or paracentesis.2 This confirmed a previous report that family practice residents had low confidence in performing thoracenteses.3 Thoracentesis also carries the risk of the potentially life‐threatening complication of pneumothorax, which may be increased when performed by physicians‐in‐training.4

One method for improving training and assessment is the use of simulation technology. Simulation has been used to increase knowledge, provide opportunities for deliberate and safe practice, and shape the development of clinical skills.5, 6 Simulation has also been advocated for assessing competence in procedures including carotid angiography,7 emergency airway management,8 basic bronchoscopy,9 and advanced cardiac life support (ACLS).10, 11

Recently, we used simulation technology to help residents reach mastery learning standards for ACLS.11 Mastery learning,12 an extreme form of competency‐based education,13 implies that learners have acquired the clinical knowledge and skill measured against rigorous achievement standards. In mastery learning, educational results are equivalent, whereas educational practice time differs. To demonstrate mastery learning, we first documented a 38% improvement in skill after a simulation‐based educational intervention10 and used a multidisciplinary panel to determine mastery achievement standards for ACLS skills in 6 clinical scenarios.14 These standards were used in a study in which the amount of time needed to achieve skill mastery was allowed to vary while the skill outcomes of the residents were identical clinically.11

The present study had 4 aims. The first was to assess the baseline skill and knowledge of third‐year residents in thoracentesis. The second was to compare the thoracentesis‐related knowledge and skills of residents before and after an educational intervention. The third was to assess the correlation of medical knowledge and clinical experience with performance on a clinical skills examination after simulation training. The last was to document the feasibility of incorporating simulation‐based education into a training program.

METHODS

RESULTS

All residents consented to participate and completed the entire training protocol. Table 1 presents demographic data about the residents. Most had limited experience performing and supervising thoracenteses.

Interrater reliability for the thoracentesis checklist data was calculated at pretest. Across the 25 checklist items, the mean kappa coefficient was very high (_n = .94). The MPS used as the mastery achievement standard was 80% (eg, 20 of 25 checklist items). This was the mean of the Angoff and Hofstee ratings obtained from the first judgment of the expert panel and is displayed in Figure 1.

Figure 1

No resident achieved mastery at pretest. However, 37 of the 40 medicine residents (93%) achieved mastery within the standard 4‐hour thoracentesis curriculum. The remaining 3 residents (7%) needed extra time ranging from 20 to 90 minutes to reach mastery.

Figure 1 is a graphic portrait with descriptive statistics of the residents' pretest and posttest performance on the thoracentesis written and clinical skills exams. For the written exam, the mean score rose from 57.63% to 89.75%, a statistically significant improvement of 56% from pretest to posttest (t_[39] = 17.0, P < .0001). The clinical skills exam also showed a highly significant 71% pretest‐to‐posttest gain, as the mean score rose from 51.70% to 88.3% (t_[39] = 15.6, P < .0001).

Results from the regression analysis indicate that neither pretest performance, medical knowledge measured by local or USMLE examinations, nor thoracentesis clinical experience was correlated with the posttest measure of thoracentesis clinical skills. However, the need for additional practice to reach the mastery standard on the posttest was a powerful negative predictor of posttest performance: b = .27 (95% CI = .46 to .09; P < .006; r² = .28). For those residents who required extra practice time, the initial clinical skills posttest score was 20% lower than that of their peers. Although the need for extra deliberate practice was associated with relatively lower initial posttest scores, all residents ultimately met or exceeded the rigorous thoracentesis MPS.

The responses of the 40 residents on a course evaluation questionnaire were uniformly positive. Responses were recorded on a Likert scale where 1 = strongly disagree, 2 = disagree, 3 = uncertain, 4 = agree, and 5 = strongly agree (Table 2). The data show that residents strongly agreed that practice with the medical simulator boosts clinical skills and self‐confidence, that they received useful feedback from the training sessions, and that deliberate practice using the simulator is a valuable educational experience. Residents were uncertain whether practice with the medical simulator has more educational value than patient care.

DISCUSSION

This study demonstrates the use of a mastery learning model to develop the thoracentesis skills of internal medicine residents to a high level. Use of a thoracentesis model in a structured educational program offering an opportunity for deliberate practice with feedback produced large and consistent improvements in residents' skills. An important finding of our study is that despite having completed most of their internal medicine training, residents displayed poor knowledge and clinical skill in thoracentesis procedures at baseline. This is similar to previous studies showing that the procedural skills and knowledge of physicians at all stages of training are often poor. Examples of areas in which significant gaps were found include basic skills such as chest radiography,27 emergency airway management,8 and pulmonary auscultation.28 In contrast, after the mastery learning program, all the residents met or exceeded the MPS for the thoracentesis clinical procedure and scored much higher on the posttest written examination.

Our data also demonstrate that medical knowledge measured by procedure‐specific pretests and posttests and USMLE Steps 1 and 2 scores were not correlated with thoracentesis skill acquisition. This reinforces findings from our previous studies of ACLS skill acquisition10, 11 and supports the difference between professional and academic achievement. Pretest skill performance and clinical experience also were not correlated with posttest outcomes. However, the amount of deliberate practice needed to reach the mastery standard was a powerful negative predictor of posttest thoracentesis skill scores, replicating our research on ACLS.11 We believe that clinical experience was not correlated with posttest outcomes because residents infrequently performed thoracenteses procedures during their training.

This project demonstrates a practical model for outcomes‐based education, certification, and program accreditation. Given the need to move procedural training in internal medicine beyond such historical methods as see one, do one, teach one,29 extension of the mastery model to other invasive procedures deserves further study. At our institution we have been encouraged by the ability of simulation‐based education in ACLS to promote long‐term skill retention30 and improvement in the quality of actual patient care.31 In addition to studying these outcomes for thoracentesis, we plan to incorporate the use of ultrasound when training residents to perform procedures such as thoracentesis and central venous catheter insertion.

Given concerns about the quality of resident preparation to perform invasive procedures, programs such as this should be considered as part of the procedural certification process. As shown by our experience with several classes of residents (n = 158), use of simulation technology to reach high procedural skill levels is effective and feasible in internal medicine residency training. In addition, our residents have consistently enjoyed participating in the simulated training programs. Postcourse questionnaires show that residents agree that deliberate practice with simulation technology complements but does not replace patient care in graduate medical education.5, 10

An important question needing more research is whether performance in a simulated environment transfers to actual clinical settings. Several small studies have demonstrated such a relationship,8, 9, 31, 32 yet the transfer of simulated training to clinical practice requires further study. More work should also be done to assess long‐term retention of skills30 and to determine the utility and benefit of simulation‐based training in procedural certification and credentialing.

This study had several limitations. It was conducted in 1 training program at a single medical center. The sample size (n = 40) was relatively small. The thoracentesis model was used for both education and testing, potentially confounding the events. However, these limitations do not diminish the pronounced impact that the simulation‐based training had on the skills and knowledge of our residents.

In conclusion, this study has demonstrated the ability of deliberate practice using a thoracentesis model to produce high‐level performance of simulated thoracenteses. The project received high ratings from learners and provides reliable assessments of procedural competence. Although internists are performing fewer invasive procedures now than in years past, procedural training is still an important component of internal medicine training.29, 33 Attainment of high procedural skill levels may be especially important for residents who plan to practice hospital medicine. We believe that simulation‐based training using deliberate practice should be a key contributor to future internal medicine residency education, certification, and accreditation.

In a supplement to its inaugural issue, the Journal of Hospital Medicine published core competencies for hospitalists covering 3 areas: clinical conditions, systems in health care, and procedures.1 Completion of a traditional internal medicine residency may not provide hospitalists with the skills necessary to safely perform necessary procedures such as thoracentesis. A recent article reported that most internal medicine residents surveyed were uncomfortable performing common procedures, and their discomfort was higher for thoracentesis than for central line insertion, lumbar puncture, or paracentesis.2 This confirmed a previous report that family practice residents had low confidence in performing thoracenteses.3 Thoracentesis also carries the risk of the potentially life‐threatening complication of pneumothorax, which may be increased when performed by physicians‐in‐training.4

One method for improving training and assessment is the use of simulation technology. Simulation has been used to increase knowledge, provide opportunities for deliberate and safe practice, and shape the development of clinical skills.5, 6 Simulation has also been advocated for assessing competence in procedures including carotid angiography,7 emergency airway management,8 basic bronchoscopy,9 and advanced cardiac life support (ACLS).10, 11

Recently, we used simulation technology to help residents reach mastery learning standards for ACLS.11 Mastery learning,12 an extreme form of competency‐based education,13 implies that learners have acquired the clinical knowledge and skill measured against rigorous achievement standards. In mastery learning, educational results are equivalent, whereas educational practice time differs. To demonstrate mastery learning, we first documented a 38% improvement in skill after a simulation‐based educational intervention10 and used a multidisciplinary panel to determine mastery achievement standards for ACLS skills in 6 clinical scenarios.14 These standards were used in a study in which the amount of time needed to achieve skill mastery was allowed to vary while the skill outcomes of the residents were identical clinically.11

The present study had 4 aims. The first was to assess the baseline skill and knowledge of third‐year residents in thoracentesis. The second was to compare the thoracentesis‐related knowledge and skills of residents before and after an educational intervention. The third was to assess the correlation of medical knowledge and clinical experience with performance on a clinical skills examination after simulation training. The last was to document the feasibility of incorporating simulation‐based education into a training program.

METHODS

RESULTS

All residents consented to participate and completed the entire training protocol. Table 1 presents demographic data about the residents. Most had limited experience performing and supervising thoracenteses.

Interrater reliability for the thoracentesis checklist data was calculated at pretest. Across the 25 checklist items, the mean kappa coefficient was very high (_n = .94). The MPS used as the mastery achievement standard was 80% (eg, 20 of 25 checklist items). This was the mean of the Angoff and Hofstee ratings obtained from the first judgment of the expert panel and is displayed in Figure 1.

Figure 1

No resident achieved mastery at pretest. However, 37 of the 40 medicine residents (93%) achieved mastery within the standard 4‐hour thoracentesis curriculum. The remaining 3 residents (7%) needed extra time ranging from 20 to 90 minutes to reach mastery.

Figure 1 is a graphic portrait with descriptive statistics of the residents' pretest and posttest performance on the thoracentesis written and clinical skills exams. For the written exam, the mean score rose from 57.63% to 89.75%, a statistically significant improvement of 56% from pretest to posttest (t_[39] = 17.0, P < .0001). The clinical skills exam also showed a highly significant 71% pretest‐to‐posttest gain, as the mean score rose from 51.70% to 88.3% (t_[39] = 15.6, P < .0001).

Results from the regression analysis indicate that neither pretest performance, medical knowledge measured by local or USMLE examinations, nor thoracentesis clinical experience was correlated with the posttest measure of thoracentesis clinical skills. However, the need for additional practice to reach the mastery standard on the posttest was a powerful negative predictor of posttest performance: b = .27 (95% CI = .46 to .09; P < .006; r² = .28). For those residents who required extra practice time, the initial clinical skills posttest score was 20% lower than that of their peers. Although the need for extra deliberate practice was associated with relatively lower initial posttest scores, all residents ultimately met or exceeded the rigorous thoracentesis MPS.

The responses of the 40 residents on a course evaluation questionnaire were uniformly positive. Responses were recorded on a Likert scale where 1 = strongly disagree, 2 = disagree, 3 = uncertain, 4 = agree, and 5 = strongly agree (Table 2). The data show that residents strongly agreed that practice with the medical simulator boosts clinical skills and self‐confidence, that they received useful feedback from the training sessions, and that deliberate practice using the simulator is a valuable educational experience. Residents were uncertain whether practice with the medical simulator has more educational value than patient care.

DISCUSSION

This study demonstrates the use of a mastery learning model to develop the thoracentesis skills of internal medicine residents to a high level. Use of a thoracentesis model in a structured educational program offering an opportunity for deliberate practice with feedback produced large and consistent improvements in residents' skills. An important finding of our study is that despite having completed most of their internal medicine training, residents displayed poor knowledge and clinical skill in thoracentesis procedures at baseline. This is similar to previous studies showing that the procedural skills and knowledge of physicians at all stages of training are often poor. Examples of areas in which significant gaps were found include basic skills such as chest radiography,27 emergency airway management,8 and pulmonary auscultation.28 In contrast, after the mastery learning program, all the residents met or exceeded the MPS for the thoracentesis clinical procedure and scored much higher on the posttest written examination.

Our data also demonstrate that medical knowledge measured by procedure‐specific pretests and posttests and USMLE Steps 1 and 2 scores were not correlated with thoracentesis skill acquisition. This reinforces findings from our previous studies of ACLS skill acquisition10, 11 and supports the difference between professional and academic achievement. Pretest skill performance and clinical experience also were not correlated with posttest outcomes. However, the amount of deliberate practice needed to reach the mastery standard was a powerful negative predictor of posttest thoracentesis skill scores, replicating our research on ACLS.11 We believe that clinical experience was not correlated with posttest outcomes because residents infrequently performed thoracenteses procedures during their training.

This project demonstrates a practical model for outcomes‐based education, certification, and program accreditation. Given the need to move procedural training in internal medicine beyond such historical methods as see one, do one, teach one,29 extension of the mastery model to other invasive procedures deserves further study. At our institution we have been encouraged by the ability of simulation‐based education in ACLS to promote long‐term skill retention30 and improvement in the quality of actual patient care.31 In addition to studying these outcomes for thoracentesis, we plan to incorporate the use of ultrasound when training residents to perform procedures such as thoracentesis and central venous catheter insertion.

Given concerns about the quality of resident preparation to perform invasive procedures, programs such as this should be considered as part of the procedural certification process. As shown by our experience with several classes of residents (n = 158), use of simulation technology to reach high procedural skill levels is effective and feasible in internal medicine residency training. In addition, our residents have consistently enjoyed participating in the simulated training programs. Postcourse questionnaires show that residents agree that deliberate practice with simulation technology complements but does not replace patient care in graduate medical education.5, 10

An important question needing more research is whether performance in a simulated environment transfers to actual clinical settings. Several small studies have demonstrated such a relationship,8, 9, 31, 32 yet the transfer of simulated training to clinical practice requires further study. More work should also be done to assess long‐term retention of skills30 and to determine the utility and benefit of simulation‐based training in procedural certification and credentialing.

This study had several limitations. It was conducted in 1 training program at a single medical center. The sample size (n = 40) was relatively small. The thoracentesis model was used for both education and testing, potentially confounding the events. However, these limitations do not diminish the pronounced impact that the simulation‐based training had on the skills and knowledge of our residents.

In conclusion, this study has demonstrated the ability of deliberate practice using a thoracentesis model to produce high‐level performance of simulated thoracenteses. The project received high ratings from learners and provides reliable assessments of procedural competence. Although internists are performing fewer invasive procedures now than in years past, procedural training is still an important component of internal medicine training.29, 33 Attainment of high procedural skill levels may be especially important for residents who plan to practice hospital medicine. We believe that simulation‐based training using deliberate practice should be a key contributor to future internal medicine residency education, certification, and accreditation.

See one, do one, teach one is a refrain familiar to all physicians. Historically, most procedural training has occurred at the bedside. In this model, senior residents, subspecialty fellows, or faculty members would demonstrate procedural skills to junior trainees, who would subsequently practice the procedures on patients, often with uneven, risky results. Acquisition of procedural skills by residents and fellows on inpatient wards is suboptimal for at least 2 reasons beyond the risks to patient safety: (1) clinical priorities are more important than educational priorities in this setting, and (2) the patient, not the medical learner, is the most important person in the room.

Recently, several new factors have challenged the traditional medical education model. For a variety of reasons, general internists currently perform far fewer invasive procedures than they used to.1 A heightened focus on patient safety and quality raises questions about the qualifications needed to perform invasive procedures. Assessment requirements have also become more stringent. The Accreditation Council for Graduate Medical Education (ACGME) now requires the use of measures that yield reliable and valid data to document the competence of trainees performing invasive procedures.2 In 2006 these factors, and the challenge to educate, assess, and certify residents, prompted the American Board of Internal Medicine to revise its certification requirements and remove the need for technical proficiency in several procedures including paracentesis, central venous catheter placement, and thoracentesis.3, 4

Two studies reported in this issue of the Journal of Hospital Medicine highlight important issues about preparing residents to perform invasive procedures. These include the educational limits of routine clinical care and the challenge to design rigorous educational interventions that improve residents' skills. Miranda and colleagues5 designed a clinical trial to evaluate an educational intervention in which residents practiced insertion of subclavian and internal jugular venous catheters under the supervision of a hospitalist faculty member. The goal was to reduce the frequency of femoral venous catheters placed at their institution. Although residents demonstrated increased knowledge and confidence after the educational intervention, the actual number of subclavian and internal jugular venous catheter insertions was lower in the intervention group, and was rare overall. The intervention did not achieve the stated goal of reducing the number of femoral venous catheters placed by residents. This research highlights that residents cannot be trained to perform invasive procedures through clinical experience alone. In addition, it demonstrates that brief educational interventions are also insufficient. Whether a longer and more robust educational intervention might have shown different results is uncertain, but many experts believe that opportunities for deliberate practice6 using standardized and sustained treatments7 can be a powerful tool to boost the procedural skills of physicians.

At the same institution, Lucas and colleagues studied the impact of a procedural service on the number of invasive procedures performed on a general medicine inpatient service.8 They found a 48% increase in procedure attempts when the procedure service staffed by an experienced faulty member was available. However, no improvement in success rate or reduction in complications was demonstrated. Thus, opportunities for trainees to perform procedures increased, but the presence of a faculty member to provide direct supervision did not improve the quality of the procedures accomplished.

Together these reports highlight challenges and opportunities in training residents to perform invasive procedures. Both studies involved the procedural skills of residents. One used an educational intervention, the other featured faculty supervision. Both studies produced outcomes that suggest improved procedural training, but neither improved the actual quality of delivered care. A brief educational intervention increased resident confidence and knowledge but did not increase the quality or number of procedures performed by residents. Opportunities to perform invasive procedures increased dramatically when an experienced attending physician was available to supervise residents. However, more education was not provided, and the quality of procedures performed did not improve.

Given these limitations, how should physicians learn to perform invasive procedures? We endorse a systematic approach to achieve high levels of procedural skills in resident physicians. First, procedures should be carefully selected. Only those essential to future practice should be required. If possible, opportunities should be available for selected trainees to develop skills in performing additional procedures relevant to their future careers. An example would be the opportunity for residents in a hospitalist track to develop proficiency in central venous catheter insertion through clinical experience, didactic education, and rigorous skill assessment. Second, dedicated programs are needed to train and assess residents in procedural skills. Reliance on clinical experience alone is inadequate because of the low frequency at which most procedures are performed and the inability to standardize assessments in routine clinical practice.

Simulation technology is a powerful adjunct to traditional clinical training and has been demonstrated to be highly effective in developing procedural skills in disciplines such as endoscopy9 and laparoscopic surgery.10 At our institution, a simulation‐based training program has been used to help residents achieve11 and maintain12 a high level of skill in performing advanced cardiac life support procedures. We use simulation to provide opportunities for deliberate practice in a controlled environment in which immediate feedback is emphasized and mastery levels are reached. The rigorous curriculum is standardized, but learner progress is individualized depending on the practice time needed to achieve competency standards.

Most important, when training physicians to perform invasive procedures, it is critical to use interventions and training programs that can be linked to improvements in actual clinical care. The studies by Miranda et al. and Lucas et al. highlight the utility of focused educational programs to complement clinical training as well as the positive impact of direct faculty supervision. These results are important starting points for programs to consider as they train and certify residents in required procedural skills. However, much work remains to be done. These studies have revealed that improvements in patient care outcomes are not likely to occur unless robust, learner‐centered educational programs are combined with adequate opportunities for residents to perform procedures under appropriate supervision.

See one, do one, teach one is a refrain familiar to all physicians. Historically, most procedural training has occurred at the bedside. In this model, senior residents, subspecialty fellows, or faculty members would demonstrate procedural skills to junior trainees, who would subsequently practice the procedures on patients, often with uneven, risky results. Acquisition of procedural skills by residents and fellows on inpatient wards is suboptimal for at least 2 reasons beyond the risks to patient safety: (1) clinical priorities are more important than educational priorities in this setting, and (2) the patient, not the medical learner, is the most important person in the room.

Recently, several new factors have challenged the traditional medical education model. For a variety of reasons, general internists currently perform far fewer invasive procedures than they used to.1 A heightened focus on patient safety and quality raises questions about the qualifications needed to perform invasive procedures. Assessment requirements have also become more stringent. The Accreditation Council for Graduate Medical Education (ACGME) now requires the use of measures that yield reliable and valid data to document the competence of trainees performing invasive procedures.2 In 2006 these factors, and the challenge to educate, assess, and certify residents, prompted the American Board of Internal Medicine to revise its certification requirements and remove the need for technical proficiency in several procedures including paracentesis, central venous catheter placement, and thoracentesis.3, 4

Two studies reported in this issue of the Journal of Hospital Medicine highlight important issues about preparing residents to perform invasive procedures. These include the educational limits of routine clinical care and the challenge to design rigorous educational interventions that improve residents' skills. Miranda and colleagues5 designed a clinical trial to evaluate an educational intervention in which residents practiced insertion of subclavian and internal jugular venous catheters under the supervision of a hospitalist faculty member. The goal was to reduce the frequency of femoral venous catheters placed at their institution. Although residents demonstrated increased knowledge and confidence after the educational intervention, the actual number of subclavian and internal jugular venous catheter insertions was lower in the intervention group, and was rare overall. The intervention did not achieve the stated goal of reducing the number of femoral venous catheters placed by residents. This research highlights that residents cannot be trained to perform invasive procedures through clinical experience alone. In addition, it demonstrates that brief educational interventions are also insufficient. Whether a longer and more robust educational intervention might have shown different results is uncertain, but many experts believe that opportunities for deliberate practice6 using standardized and sustained treatments7 can be a powerful tool to boost the procedural skills of physicians.

At the same institution, Lucas and colleagues studied the impact of a procedural service on the number of invasive procedures performed on a general medicine inpatient service.8 They found a 48% increase in procedure attempts when the procedure service staffed by an experienced faulty member was available. However, no improvement in success rate or reduction in complications was demonstrated. Thus, opportunities for trainees to perform procedures increased, but the presence of a faculty member to provide direct supervision did not improve the quality of the procedures accomplished.

Together these reports highlight challenges and opportunities in training residents to perform invasive procedures. Both studies involved the procedural skills of residents. One used an educational intervention, the other featured faculty supervision. Both studies produced outcomes that suggest improved procedural training, but neither improved the actual quality of delivered care. A brief educational intervention increased resident confidence and knowledge but did not increase the quality or number of procedures performed by residents. Opportunities to perform invasive procedures increased dramatically when an experienced attending physician was available to supervise residents. However, more education was not provided, and the quality of procedures performed did not improve.

Given these limitations, how should physicians learn to perform invasive procedures? We endorse a systematic approach to achieve high levels of procedural skills in resident physicians. First, procedures should be carefully selected. Only those essential to future practice should be required. If possible, opportunities should be available for selected trainees to develop skills in performing additional procedures relevant to their future careers. An example would be the opportunity for residents in a hospitalist track to develop proficiency in central venous catheter insertion through clinical experience, didactic education, and rigorous skill assessment. Second, dedicated programs are needed to train and assess residents in procedural skills. Reliance on clinical experience alone is inadequate because of the low frequency at which most procedures are performed and the inability to standardize assessments in routine clinical practice.

Simulation technology is a powerful adjunct to traditional clinical training and has been demonstrated to be highly effective in developing procedural skills in disciplines such as endoscopy9 and laparoscopic surgery.10 At our institution, a simulation‐based training program has been used to help residents achieve11 and maintain12 a high level of skill in performing advanced cardiac life support procedures. We use simulation to provide opportunities for deliberate practice in a controlled environment in which immediate feedback is emphasized and mastery levels are reached. The rigorous curriculum is standardized, but learner progress is individualized depending on the practice time needed to achieve competency standards.

Most important, when training physicians to perform invasive procedures, it is critical to use interventions and training programs that can be linked to improvements in actual clinical care. The studies by Miranda et al. and Lucas et al. highlight the utility of focused educational programs to complement clinical training as well as the positive impact of direct faculty supervision. These results are important starting points for programs to consider as they train and certify residents in required procedural skills. However, much work remains to be done. These studies have revealed that improvements in patient care outcomes are not likely to occur unless robust, learner‐centered educational programs are combined with adequate opportunities for residents to perform procedures under appropriate supervision.

See one, do one, teach one is a refrain familiar to all physicians. Historically, most procedural training has occurred at the bedside. In this model, senior residents, subspecialty fellows, or faculty members would demonstrate procedural skills to junior trainees, who would subsequently practice the procedures on patients, often with uneven, risky results. Acquisition of procedural skills by residents and fellows on inpatient wards is suboptimal for at least 2 reasons beyond the risks to patient safety: (1) clinical priorities are more important than educational priorities in this setting, and (2) the patient, not the medical learner, is the most important person in the room.

Recently, several new factors have challenged the traditional medical education model. For a variety of reasons, general internists currently perform far fewer invasive procedures than they used to.1 A heightened focus on patient safety and quality raises questions about the qualifications needed to perform invasive procedures. Assessment requirements have also become more stringent. The Accreditation Council for Graduate Medical Education (ACGME) now requires the use of measures that yield reliable and valid data to document the competence of trainees performing invasive procedures.2 In 2006 these factors, and the challenge to educate, assess, and certify residents, prompted the American Board of Internal Medicine to revise its certification requirements and remove the need for technical proficiency in several procedures including paracentesis, central venous catheter placement, and thoracentesis.3, 4

Two studies reported in this issue of the Journal of Hospital Medicine highlight important issues about preparing residents to perform invasive procedures. These include the educational limits of routine clinical care and the challenge to design rigorous educational interventions that improve residents' skills. Miranda and colleagues5 designed a clinical trial to evaluate an educational intervention in which residents practiced insertion of subclavian and internal jugular venous catheters under the supervision of a hospitalist faculty member. The goal was to reduce the frequency of femoral venous catheters placed at their institution. Although residents demonstrated increased knowledge and confidence after the educational intervention, the actual number of subclavian and internal jugular venous catheter insertions was lower in the intervention group, and was rare overall. The intervention did not achieve the stated goal of reducing the number of femoral venous catheters placed by residents. This research highlights that residents cannot be trained to perform invasive procedures through clinical experience alone. In addition, it demonstrates that brief educational interventions are also insufficient. Whether a longer and more robust educational intervention might have shown different results is uncertain, but many experts believe that opportunities for deliberate practice6 using standardized and sustained treatments7 can be a powerful tool to boost the procedural skills of physicians.

At the same institution, Lucas and colleagues studied the impact of a procedural service on the number of invasive procedures performed on a general medicine inpatient service.8 They found a 48% increase in procedure attempts when the procedure service staffed by an experienced faulty member was available. However, no improvement in success rate or reduction in complications was demonstrated. Thus, opportunities for trainees to perform procedures increased, but the presence of a faculty member to provide direct supervision did not improve the quality of the procedures accomplished.

Together these reports highlight challenges and opportunities in training residents to perform invasive procedures. Both studies involved the procedural skills of residents. One used an educational intervention, the other featured faculty supervision. Both studies produced outcomes that suggest improved procedural training, but neither improved the actual quality of delivered care. A brief educational intervention increased resident confidence and knowledge but did not increase the quality or number of procedures performed by residents. Opportunities to perform invasive procedures increased dramatically when an experienced attending physician was available to supervise residents. However, more education was not provided, and the quality of procedures performed did not improve.

Given these limitations, how should physicians learn to perform invasive procedures? We endorse a systematic approach to achieve high levels of procedural skills in resident physicians. First, procedures should be carefully selected. Only those essential to future practice should be required. If possible, opportunities should be available for selected trainees to develop skills in performing additional procedures relevant to their future careers. An example would be the opportunity for residents in a hospitalist track to develop proficiency in central venous catheter insertion through clinical experience, didactic education, and rigorous skill assessment. Second, dedicated programs are needed to train and assess residents in procedural skills. Reliance on clinical experience alone is inadequate because of the low frequency at which most procedures are performed and the inability to standardize assessments in routine clinical practice.

Simulation technology is a powerful adjunct to traditional clinical training and has been demonstrated to be highly effective in developing procedural skills in disciplines such as endoscopy9 and laparoscopic surgery.10 At our institution, a simulation‐based training program has been used to help residents achieve11 and maintain12 a high level of skill in performing advanced cardiac life support procedures. We use simulation to provide opportunities for deliberate practice in a controlled environment in which immediate feedback is emphasized and mastery levels are reached. The rigorous curriculum is standardized, but learner progress is individualized depending on the practice time needed to achieve competency standards.

Most important, when training physicians to perform invasive procedures, it is critical to use interventions and training programs that can be linked to improvements in actual clinical care. The studies by Miranda et al. and Lucas et al. highlight the utility of focused educational programs to complement clinical training as well as the positive impact of direct faculty supervision. These results are important starting points for programs to consider as they train and certify residents in required procedural skills. However, much work remains to be done. These studies have revealed that improvements in patient care outcomes are not likely to occur unless robust, learner‐centered educational programs are combined with adequate opportunities for residents to perform procedures under appropriate supervision.