User login
Central venous catheter (CVC) insertions are commonly performed at the bedside in medical intensive care unit (MICU) settings. Internal medicine residents are required to demonstrate knowledge regarding CVC indications, complications, and sterile technique,1 and often perform the procedure during training. Education in CVC insertion is needed because many internal medicine residents are uncomfortable performing this procedure.2 CVC insertion also carries the risk of potentially life‐threatening complications including infection, pneumothorax, arterial puncture, deep vein thrombosis, and bleeding. Education and training may also contribute to improved patient care because increased physician experience with CVC insertion reduces complication risk.3, 4 Similarly, a higher number of needle passes or attempts during CVC insertion correlates with mechanical complications such as pneumothorax or arterial punctures.48 Pneumothorax rates for internal jugular (IJ) CVCs have been reported to range from 0% to 0.2% and for subclavian (SC) CVCs from 1.5% to 3.1%.4, 5 The arterial puncture rate for IJ CVCs ranges from 5.0% to 9.4% and for SC CVCs from 3.1% to 4.9%.4, 5 Proper use of ultrasound to assist with IJ CVC insertion has been shown to decrease these mechanical complications.4, 5 However, studies of ultrasound use with SC CVC insertion have mixed results.4
Simulation‐based training has been used in medical education to increase knowledge, provide opportunities for deliberate and safe practice, and shape the development of clinical skills.9, 10 We previously used simulation‐based mastery learning to improve the thoracentesis and advanced cardiac life support (ACLS) skills of internal medicine residents.11, 12 Although a few small studies have linked simulation‐based interventions to improved quality of care,1319 more work is needed to show that results from a simulated environment transfer to actual patient care.
This study had 2 aims. The first was to expand our simulation‐based mastery learning to CVC insertion using a CVC simulator and ultrasound device. The second was to assess quality indicators (number of needle passes, pneumothorax, arterial punctures, and need for catheter adjustment) and resident confidence related to actual CVC insertions in the MICU before and after an educational intervention.
Materials and Methods
Design
This was a cohort study20 of IJ and SC CVC insertions by 41 second‐ and third‐year internal medicine residents rotating through the MICU in a university‐affiliated program from October 2006 to February 2007. The Northwestern University Institutional Review Board approved the study. All study participants were required to give informed consent prior to participation.
Thirteen residents rotated through the MICU during a 6‐week preintervention phase. These residents served as a traditionally trained group that did not receive CVC insertion simulator training. Simultaneously, 28 residents who rotated through the MICU later in the study period received simulation‐based training in CVC insertion and served as the simulator‐trained group (Figure 1). Demographic data were obtained from the participants including age, gender, ethnicity, year of training, and scores on the United States Medical Licensing Examination (USMLE) Steps 1 and 2.
Simulator‐trained residents underwent baseline skill assessment (pretest) using a 27‐item checklist in IJ and SC CVC insertions (see Appendix). Checklists were developed by one author (J.H.B.) using appropriate references4, 5 and a step‐by‐step process,21 and reviewed for completeness by another author with expertise in checklist development (D.B.W.). Each skill or other action was listed in order and given equal weight. A dichotomous scoring scale of 1 = done correctly and 0 = done incorrectly/not done was imposed for each item. Assessments were performed using Simulab's CentralLineMan. This model features realistic tissue with ultrasound compatibility, an arterial pulse, and self‐sealing veins and skins. Needles, dilators, and guidewires can be inserted and realistic venous and arterial pressures demonstrated (Figure 2).
Residents in the simulator‐trained group received two, 2‐hour education sessions featuring a lecture, ultrasound training, deliberate practice with the CVC simulator, and feedback.22 Education sessions contained standardized didactic material on CVC indications and complications, as well as a stepwise demonstration of IJ and SC CVC insertions using ultrasound and landmark techniques. These sessions were supervised by a senior hospitalist faculty member with expertise in CVC insertions (J.H.B.). Residents were expected to use the ultrasound device for all IJ CVC insertions. However, its use was optional for SC CVC insertion. After training, residents were retested (posttest) and required to meet or exceed a minimum passing score (MPS) set by an expert panel for both IJ and SC procedures.23 This 11 member expert panel provided item‐based (Angoff) and group‐based (Hofstee) judgments on the 27‐item checklists as described previously.23
Residents who did not achieve the MPS had more deliberate practice and were retested until the MPS was reached; the key feature of mastery learning.24 After completing simulation‐based mastery learning in CVC insertion, the 28 simulator‐trained residents rotated through the MICU.
Data Collection
All pretests and posttests (using the 27‐item checklist) were graded by a single unblinded instructor (J.H.B.) and were videotaped. Another faculty instructor with expertise in scoring clinical skills examinations and blind to pre‐post status (D.B.W.) rescored a random 50% sample of the tests to assess interrater reliability.
Data regarding actual CVC insertions in the MICU were collected by contacting all MICU residents daily during the study period. This allowed for CVC insertions to be identified within 24 hours. All survey data were collected anonymously. The primary inserter of each CVC was questioned about quality indicators and procedural self‐confidence concerning CVC placement. CVCs primarily inserted by nonstudy subjects (first‐year residents, emergency medicine residents, pulmonary‐critical care medicine faculty members, and subspecialty fellows) or CVC placements that were supervised, but not directly placed by study participants, were excluded.
Outcome Measures
Pretest and posttest checklist scores from simulator‐trained residents were compared to measure the impact of training sessions. Residents rotating through the MICU were asked about several quality indicators related to actual CVC insertions. Quality indicators include: (1) number of needle passes required during the procedure (skin punctures); (2) presence of complications including pneumothorax and arterial puncture; and (3) need for CVC adjustment after chest x‐ray. Participants were also questioned regarding their confidence in CVC insertion using a 100 point scale (0 = not confident and 100 = very confident). Survey results from the 28 simulator‐trained residents were compared to results from the 13 traditionally‐trained residents.
Data Analysis
Checklist score reliability was estimated by calculating interrater reliability, the preferred method for assessments that depend on human judges, using the kappa () coefficient adjusted25, 26 using the formula of Brennan and Prediger.27 Within‐group differences from pretest (baseline) to posttest (outcome) were analyzed using paired t‐tests.
MICU survey results were compared using t‐tests. Traditionally‐trained and simulator‐trained groups were assessed for demographic differences using t‐tests and the chi‐square statistic. Spearman's rank correlation coefficient was used to assess for relationships between resident self‐confidence and quality indicators. All analyses were preformed using SPSS statistical software, version 16.0 (SPSS, Inc., Chicago, IL).
Results
All eligible residents participated in the study and completed the entire protocol. There was no significant difference in age, gender, ethnicity, year of training, or USMLE Step 1 and 2 scores between the traditionally‐trained and simulator‐trained groups.
Interrater reliability measured by the mean kappa coefficient was very high (n = 0.94) across the 27 IJ and SC checklist items. No resident met the MPS (79.1%) for CVC insertion at baseline testing. In the simulator‐trained group, 25 of 28 (89%) residents achieved SC skill mastery and 27 of 28 (96%) achieved IJ skill mastery within the standard four hour curriculum. All residents subsequently reached the MPS with less than one hour of additional practice time. A graphic portrait of the residents' pretest and posttest performance on the simulated CVC clinical skills examination with descriptive statistics is shown in Figure 3. After the educational intervention, posttest scores significantly improved (p < 0.001), to meet or exceed the MPS.
Traditionally trained and simulator‐trained residents independently inserted 46 CVCs during the study period. Simulator‐trained residents required significantly fewer needle passes to insert all actual CVCs in the MICU compared to traditionally trained residents: mean (M) = 1.79, standard deviation (SD) = 1.03 versus M = 2.78, SD = 1.77 (p = 0.04). As shown in Table 1, the groups did not differ in pneumothorax, arterial puncture, or mean number of CVC adjustments. In addition, the groups did not differ in use of ultrasound for IJ or SC CVC insertions. One IJ CVC was inserted without ultrasound in the traditionally‐trained group; 2 were inserted without ultrasound in the simulator‐trained group. Ultrasound was not used during any SC CVC insertions in the traditionally‐trained group and was used for 1 SC CVC insertion in the simulator‐trained group.
| Internal Jugular and Subclavian CVCs | |||
|---|---|---|---|
| Traditionally Trained Residents | Simulator Trained Residents | P value | |
| |||
| Number of attempts during insertion [mean (SD)] | 2.78 (1.77) | 1.79 (1.03) | 0.04* |
| Pneumothorax (number) | 0 | 0 | n/a |
| Arterial puncture (%) | 11 | 7 | 0.65 |
| CVC adjustment (%) | 15 | 8 | 0.52 |
| Confidence (%) [mean (SD)] | 68 (20) | 81 (11) | 0.02* |
Simulator‐trained residents displayed more self‐confidence about their procedural skills than traditionally‐trained residents (M = 81, SD = 11 versus M = 68, SD = 20, p = 0.02). Spearman correlations showed no practical association between resident self‐confidence and performance on CVC insertion quality indicators.
Discussion
This study demonstrates the use of a mastery learning model to develop CVC insertion skills to a high achievement level among internal medicine residents. Our data support prior work showing that procedural skills that are poor at baseline can be increased significantly using simulation‐based training and deliberate practice.1118, 28 This report on CVC insertion adds to the growing body of literature showing that simulation training complements standard medical education,1119, 28 and expands the clinical application of the mastery model beyond thoracentesis and ACLS.11, 12 Use of the mastery model described in this study also has important implications for patients. In our training program, residents are required to demonstrate procedural mastery in a simulated environment before independently performing a CVC insertion on an actual patient. This is in sharp contrast to the traditional clinical model of procedural training at the bedside, and may be used in other training programs and with other invasive procedures.
The second aim of our study was to determine the impact of simulation‐based training on actual clinical practice by residents in the MICU. To our knowledge, no prior study has demonstrated that simulation‐based training in CVC insertion improves patient outcomes. We believe our results advance what is known about the impact of simulation‐based training because simulator‐trained residents in this study performed actual CVC insertions in the MICU using significantly fewer needle passes. Needle passes have been used by other investigators as a surrogate measure for reduced CVC‐associated complications because mechanical complications rise exponentially with more than two insertion attempts.47, 29 We believe this finding demonstrates transfer of skill acquired from simulation‐based training to the actual clinical environment. It is possible that ultrasound training accounts for the improvement in the simulator‐trained group. However, we do not believe that ultrasound training is entirely responsible as prior work has shown that deliberate practice using mastery learning without ultrasound significantly improved resident performance of thoracentesis11 and ACLS12, 19 procedures. We did not show a significant reduction in complications such as pneumothorax or arterial puncture. This is likely due to the small sample size and the low number of procedures and complications during the study period.
Our results also show that resident self‐confidence regarding actual CVC insertions improved after simulation training. These findings are similar to prior reports linking improved confidence among trainees after simulation‐based training in CVC insertion.29, 30 Our results did not reveal a correlation between improved self‐confidence and clinical skill acquisition. Linking improved self‐confidence to improved clinical skill is important because self‐assessment does not always correlate with performance ability.31, 32
More study is needed to evaluate the impact of simulation‐based training on the quality of CVC insertions by trainees. Mechanisms shown to decrease complications of CVC placement include use of ultrasound,4, 7, 3336 full sterile barrier technique,3739 chlorhexidine skin preparations,4042 and nurse‐physician education.43 Our simulation‐training program incorporates each of these elements. We plan to expand our simulation‐based training intervention to a larger sample size to determine its impact on mechanical and infectious complication rates linked to CVC insertion.
This study has several limitations. It was performed at a single institution over a short time period. However, demonstration of significantly fewer needle passes and improved resident self‐confidence after simulator training are important findings that warrant further study. It was impossible to blind raters during the skills assessment examination about whether the resident was performing a pretest or posttest. This was accounted for by using a second rater, who was blind to the pretest and posttest status of the examinee. The arterial puncture rate of 7% among simulator‐trained residents was higher than expected, although it remains within published ranges.4, 5 Also, a low total number of CVCs were evaluated during the study. This is likely due to strict exclusion criteria employed in order to study the impact of simulation training. For example, CVC insertions were only evaluated if they were actually performed by study residents (supervised insertions were excluded) and femoral catheters were not evaluated. We did not track clinical experience with CVC insertion by residents before the study. Residents who were simulator‐trained may have had more clinical experience with CVC insertion and this may have impacted their performance. However, residents did not differ in year of training or clinical rotations, and there is clear evidence that clinical training is not a proxy for skill acquisition.44 Finally, outcome data were measured via resident questionnaires that relied on resident recall about CVC insertion rather than observer ratings. This method was selected because observer ratings could not be standardized given the large number of clinical supervisors in the MICU over the study period. Information about needle passes and arterial puncture also may not be documented in procedural notes and could not be obtained by medical record review. We attempted to minimize recall bias by surveying residents within 24 hours of CVC placement.
In conclusion, this study demonstrates that simulation‐based training and deliberate practice in a mastery learning setting improves performance of both simulated and actual CVC insertions by internal medicine residents. Procedural training remains an important component of internal medicine training, although internists are performing fewer invasive procedures now than in years past.45, 46 Use of a mastery model of CVC insertion requires that trainees demonstrate skill in a simulated environment before independently performing this invasive procedure on patients. Further study is needed to assess clinical outcomes such as reduced CVC‐related infections and mechanical complications after simulation‐based training.
Acknowledgements
The authors thank the Northwestern University internal medicine residents for their dedication to education and patient care. They acknowledge Drs. J. Larry Jameson and Charles Watts for their support and encouragement of this work.
Appendix
Central Venous Catheter Insertion Checklists for Simulation‐based Education 0, 0
| |||
| Informed consent obtained: must do all | A | B | C |
| Benefits | |||
| Risks | |||
| Consent given | |||
| Place the patient in slight Trendelenburg position | A | B | C |
| Flush the ports on the catheter with sterile saline | A | B | C |
| Clamp each port (ok to keep brown port open) | A | B | C |
| Remove brown port from end of catheter to accommodate wire | A | B | C |
| Area is cleaned with chlorhexadine | A | B | C |
| Resident gets in sterile gown, gloves, hat and mask | A | B | C |
| Area is draped in usual sterile fashion (must be full body drape) | A | B | C |
| The ultrasound (US) probe is properly set up with sterile sheath and sonographic gel | A | B | C |
| The vein is localized using anatomical landmarks with the US machine | A | B | C |
| If no US is used this is wrong | |||
| The skin is anesthetized with 1% lidocaine in a small wheal | A | B | C |
| The deeper structures are anesthetized | A | B | C |
| Localize the vein with this needle (optional) | A | B | C |
| Using the large needle or catheter‐ syringe complex, cannulate the vein while aspirating (must be done with US) | A | B | C |
| Remove the syringe from the needle or advance the catheter into the vein removing both the syringe and needle | A | B | C |
| Advance the guidewire into the vein no more than approximately 1215 cm | A | B | C |
| Knick the skin with the scalpel to advance the dilator | A | B | C |
| Advance the dilator over the guidewire and dilate the vein | A | B | C |
| Advance the triple lumen over the guidewire | A | B | C |
| Never let go of the guidewire | A | B | C |
| Once the catheter is inserted remove the guidewire in its entirety | A | B | C |
| Advance the catheter to approx to 1416cm on the right side, 1618 cm on the left side | A | B | C |
| Ensure there is blood flow/flush each port | A | B | C |
| Secure the catheter in place (suture or staple) | A | B | C |
| Place dressing over catheter | A | B | C |
| Get a chest x‐ray | A | B | C |
| Notify that the catheter is ok to use | A | B | C |
| Maintain sterile technique | A | B | C |
| |||
| Informed consent obtained: must do all | A | B | C |
| Benefits | |||
| Risks | |||
| Consent given | |||
| Place the patient in slight Trendelenburg position | A | B | C |
| Flush the ports on the catheter with sterile saline | A | B | C |
| Clamp each port (ok to leave brown port open) | A | B | C |
| Remove brown port from end of catheter to accommodate wire | A | B | C |
| Area is cleaned with chlorhexadine | A | B | C |
| Resident gets in sterile gown, gloves, hat and mask | A | B | C |
| Area is draped in usual sterile fashion (must be full body drape) | A | B | C |
| **The US probe is properly set up with sterile sheath and sonographic gel . (MUST DO if use US) | A | B | C |
| The vein is localized using US machine or anatomical landmarks are verbalized | A | B | C |
| The skin is anesthetized with 1% lidocaine in a small wheal | A | B | C |
| The deeper structures are anesthetized using a larger needle (must verbalize they anesthetize the clavicle) | A | B | C |
| Localize the vein with this needle (optional) | A | B | C |
| Using the large needle or catheter syringe complex cannulate the vein while aspirating (optional confirmed by US) | A | B | C |
| If US was not used then expected to state they are directing the needle to the sternal notch | A | B | C |
| Remove the syringe from the needle or advance the catheter into the vein removing both the syringe and needle | A | B | C |
| Advance the guidewire into the vein no more than approximately 1215 cm | A | B | C |
| Knick the skin with the scalpel to advance the dilator | A | B | C |
| Advance the dilator over the guidewire and dilate the vein | A | B | C |
| Advance the triple lumen over the guidewire | A | B | C |
| Never let go of the guidewire | A | B | C |
| Once the catheter is inserted remove the guidewire in its entirety | A | B | C |
| Advance the catheter to approx to 1416cm on the right side, 1618 cm on the left side | A | B | C |
| Ensure there is blood flow/flush each port | A | B | C |
| Secure the catheter in place (suture or staple) | A | B | C |
| Place dressing over catheter | A | B | C |
| Get a chest x‐ray | A | B | C |
| Notify that the catheter is ok to use | A | B | C |
| Maintain sterile technique | A | B | C |
- American Board of Internal Medicine. Procedures Required for Internal Medicine. Available at: http://www.abim.org/certification/policies/imss/im.aspx. Accessed January 28, 2009.
- ,,, et al.Beyond the comfort zone: residents assess their comfort performing inpatient medical procedures.Am J Med.2006;119:71.e17–e24.
- ,,, et al.Central vein catheterization. Failure and complication rates by three percutaneous approaches.Arch Intern Med.1986;146:259–261.
- ,.Preventing complications of central venous catheterization.N Engl J Med.2003;348:1123–1133.
- ,,, et al.Mechanical complications of central venous catheters.J Intensive Care Med.2006;21:40–46.
- ,,, et al.Risk factors of failure and immediate complication of subclavian vein catheterization in critically ill patients.Intensive Care Med.2002;28:1036–1041.
- ,,, et al.Complications and failures of subclavian‐vein catheterization.N Engl J Med.1994;331:1735–1738.
- .Central venous catheterization: better and worse.J Intensive Care Med.2006;21:51–53.
- ,,, et al.Reliability and validity of a simulation‐based acute care skills assessment for medical students and residents.Anesthesiology.2003;99:1270–1280.
- ,,, et al.Simulation technology for health care professional skills training and assessment.JAMA.1999;282:861–866.
- ,,, et al.Mastery learning of thoracentesis skills by internal medicine residents using simulation technology and deliberate practice.J Hosp Med.2008;3:48–54.
- ,,, et al.Mastery learning of advanced cardiac life support skills by internal medicine residents using simulation technology and deliberate practice.J Gen Intern Med.2006;21:251–256.
- ,,, et al.Laparoscopic skills are improved with LapMentor training: results of a randomized, double‐blinded study.Ann Surg.2006;243:854–860.
- ,,.Bronchoscopy simulator effectively prepares junior residents to competently perform basic clinical bronchoscopy.Ann Thorac Surg.2004;78:287–291.
- ,,, et al.Multicenter, randomized, controlled trial of virtual‐reality simulator training in acquisition of competency in colonoscopy.Gastrointest Endosc.2006;64:361–368.
- ,,, et al.Achieving house staff competence in emergency airway management: results of a teaching program using a computerized patient simulator.Crit Care Med.2004;32:2422–2427.
- ,.Computer simulator training enhances the competency of gastroenterology fellows at colonoscopy: results of a pilot study.Am J Gastroenterol.2004;99:33–37.
- ,,, et al.Virtual reality training improves operating room performance: results of a randomized, double‐blinded study.Ann Surg.2002;236:458–463.
- ,,, et al.Simulation‐based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: a case‐control study.CHEST.2008;133:56–61.
- ,.Clinical Epidemiology: the Essentials.4th ed.Philadelphia:Lippincott Williams 2005.
- . The Checklists Development Checklist. Western Michigan University Evaluation Center, July2000. Available at: http://www. wmich.edu/evalctr/checklists/cdc.htm. Accessed May 15, 2006.
- .Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains.Acad Med.2004;79:S70–S81.
- ,,, et al.Do baseline data influence standard setting for a clinical skills examination?Acad Med.2007;82:S105–S108.
- ,,, et al.Lessons for Continuing Medical Education from simulation research in undergraduate and graduate medical education.CHEST.2009;135.
- .Reliability: on the reproducibility of assessment data.Med Educ.2004;38:1006–1012.
- ,,.Statistical Methods for Rates and Proportions.3rd ed.New York:John Wiley 41:687–699.
- ,,, et al.Simulation‐based training of internal medicine residents in advanced cardiac life support protocols: a randomized trial.Teach Learn Med.2005;17:202–208.
- ,,.Central catheter simulation: a new training algorithm.Am Surg.2007;73:680–682.
- ,,.The use of tissue models for vascular access training. Phase I of the procedural patient safety initiative.J Gen Intern Med.2006;21:514–517.
- ,,, et al.The use of simulation in emergency medicine: a research agenda.Acad Emerg Med.2007;14:353–363.
- ,,, et al.Graduating internal medicine residents' self‐assessment and performance of advanced cardiac life support skills.Med Teach.2006;28:365–369.
- ,.Bedside ultrasonography in the ICU: Part 2.CHEST.2005;128:1766–1781.
- ,,, et al.Pulsed Doppler ultrasonography guidance for catheterization of the subclavian vein: a randomized study.Anesthesiology.1998;88:1195–1201.
- ,,, et al.Ultrasound guidance versus the landmark technique for the placement of central venous catheters in the emergency department.Acad Emerg Med.2002;9:800–805.
- ,,, et al.Ultrasound guidance for placement of central venous catheters: a meta‐analysis of the literature.Crit Care Med.1996;24:2053–2058.
- ,,, et al.Eliminating catheter‐related bloodstream infections in the intensive care unit.Crit Care Med.2004;32:2014–2020.
- ,,, et al.An intervention to decrease catheter‐related bloodstream infections in the ICU.N Engl J Med.2006;355:2725–2732.
- ,,, et al.Education of physicians‐in‐training can decrease the risk for vascular catheter infection.Ann Intern Med.2000;132:641–648.
- ,,, et al.Chlorhexidine compared with povidone‐iodine solution for vascular catheter‐site care: a meta‐analysis.Ann Intern Med.2002;136:792–801.
- ,,.Prospective randomised trial of povidone‐iodine, alcohol, and chlorhexidine for prevention of infection associated with central venous and arterial catheters.Lancet.1991;338:339–343.
- ,,, et al.Prospective, randomized trial of two antiseptic solutions for prevention of central venous or arterial catheter colonization and infection in intensive care unit patients.Crit Care Med.1996;24:1818–1823.
- ,,, et al.The effect of an education program on the incidence of central venous catheter‐associated bloodstream infection in a medical ICU.CHEST.2004;126:1612–1618.
- ,,.Systematic review: the relationship between clinical experience and quality of health care.Ann Intern Med.2005;142:260–273.
- ,.What procedures should internists do?Ann Intern Med.2007;146:392–393.
- ,.The declining number and variety of procedures done by general internists: a resurvey of members of the American College of Physicians.Ann Intern Med.2007;146:355–360.
Central venous catheter (CVC) insertions are commonly performed at the bedside in medical intensive care unit (MICU) settings. Internal medicine residents are required to demonstrate knowledge regarding CVC indications, complications, and sterile technique,1 and often perform the procedure during training. Education in CVC insertion is needed because many internal medicine residents are uncomfortable performing this procedure.2 CVC insertion also carries the risk of potentially life‐threatening complications including infection, pneumothorax, arterial puncture, deep vein thrombosis, and bleeding. Education and training may also contribute to improved patient care because increased physician experience with CVC insertion reduces complication risk.3, 4 Similarly, a higher number of needle passes or attempts during CVC insertion correlates with mechanical complications such as pneumothorax or arterial punctures.48 Pneumothorax rates for internal jugular (IJ) CVCs have been reported to range from 0% to 0.2% and for subclavian (SC) CVCs from 1.5% to 3.1%.4, 5 The arterial puncture rate for IJ CVCs ranges from 5.0% to 9.4% and for SC CVCs from 3.1% to 4.9%.4, 5 Proper use of ultrasound to assist with IJ CVC insertion has been shown to decrease these mechanical complications.4, 5 However, studies of ultrasound use with SC CVC insertion have mixed results.4
Simulation‐based training has been used in medical education to increase knowledge, provide opportunities for deliberate and safe practice, and shape the development of clinical skills.9, 10 We previously used simulation‐based mastery learning to improve the thoracentesis and advanced cardiac life support (ACLS) skills of internal medicine residents.11, 12 Although a few small studies have linked simulation‐based interventions to improved quality of care,1319 more work is needed to show that results from a simulated environment transfer to actual patient care.
This study had 2 aims. The first was to expand our simulation‐based mastery learning to CVC insertion using a CVC simulator and ultrasound device. The second was to assess quality indicators (number of needle passes, pneumothorax, arterial punctures, and need for catheter adjustment) and resident confidence related to actual CVC insertions in the MICU before and after an educational intervention.
Materials and Methods
Design
This was a cohort study20 of IJ and SC CVC insertions by 41 second‐ and third‐year internal medicine residents rotating through the MICU in a university‐affiliated program from October 2006 to February 2007. The Northwestern University Institutional Review Board approved the study. All study participants were required to give informed consent prior to participation.
Thirteen residents rotated through the MICU during a 6‐week preintervention phase. These residents served as a traditionally trained group that did not receive CVC insertion simulator training. Simultaneously, 28 residents who rotated through the MICU later in the study period received simulation‐based training in CVC insertion and served as the simulator‐trained group (Figure 1). Demographic data were obtained from the participants including age, gender, ethnicity, year of training, and scores on the United States Medical Licensing Examination (USMLE) Steps 1 and 2.
Simulator‐trained residents underwent baseline skill assessment (pretest) using a 27‐item checklist in IJ and SC CVC insertions (see Appendix). Checklists were developed by one author (J.H.B.) using appropriate references4, 5 and a step‐by‐step process,21 and reviewed for completeness by another author with expertise in checklist development (D.B.W.). Each skill or other action was listed in order and given equal weight. A dichotomous scoring scale of 1 = done correctly and 0 = done incorrectly/not done was imposed for each item. Assessments were performed using Simulab's CentralLineMan. This model features realistic tissue with ultrasound compatibility, an arterial pulse, and self‐sealing veins and skins. Needles, dilators, and guidewires can be inserted and realistic venous and arterial pressures demonstrated (Figure 2).
Residents in the simulator‐trained group received two, 2‐hour education sessions featuring a lecture, ultrasound training, deliberate practice with the CVC simulator, and feedback.22 Education sessions contained standardized didactic material on CVC indications and complications, as well as a stepwise demonstration of IJ and SC CVC insertions using ultrasound and landmark techniques. These sessions were supervised by a senior hospitalist faculty member with expertise in CVC insertions (J.H.B.). Residents were expected to use the ultrasound device for all IJ CVC insertions. However, its use was optional for SC CVC insertion. After training, residents were retested (posttest) and required to meet or exceed a minimum passing score (MPS) set by an expert panel for both IJ and SC procedures.23 This 11 member expert panel provided item‐based (Angoff) and group‐based (Hofstee) judgments on the 27‐item checklists as described previously.23
Residents who did not achieve the MPS had more deliberate practice and were retested until the MPS was reached; the key feature of mastery learning.24 After completing simulation‐based mastery learning in CVC insertion, the 28 simulator‐trained residents rotated through the MICU.
Data Collection
All pretests and posttests (using the 27‐item checklist) were graded by a single unblinded instructor (J.H.B.) and were videotaped. Another faculty instructor with expertise in scoring clinical skills examinations and blind to pre‐post status (D.B.W.) rescored a random 50% sample of the tests to assess interrater reliability.
Data regarding actual CVC insertions in the MICU were collected by contacting all MICU residents daily during the study period. This allowed for CVC insertions to be identified within 24 hours. All survey data were collected anonymously. The primary inserter of each CVC was questioned about quality indicators and procedural self‐confidence concerning CVC placement. CVCs primarily inserted by nonstudy subjects (first‐year residents, emergency medicine residents, pulmonary‐critical care medicine faculty members, and subspecialty fellows) or CVC placements that were supervised, but not directly placed by study participants, were excluded.
Outcome Measures
Pretest and posttest checklist scores from simulator‐trained residents were compared to measure the impact of training sessions. Residents rotating through the MICU were asked about several quality indicators related to actual CVC insertions. Quality indicators include: (1) number of needle passes required during the procedure (skin punctures); (2) presence of complications including pneumothorax and arterial puncture; and (3) need for CVC adjustment after chest x‐ray. Participants were also questioned regarding their confidence in CVC insertion using a 100 point scale (0 = not confident and 100 = very confident). Survey results from the 28 simulator‐trained residents were compared to results from the 13 traditionally‐trained residents.
Data Analysis
Checklist score reliability was estimated by calculating interrater reliability, the preferred method for assessments that depend on human judges, using the kappa () coefficient adjusted25, 26 using the formula of Brennan and Prediger.27 Within‐group differences from pretest (baseline) to posttest (outcome) were analyzed using paired t‐tests.
MICU survey results were compared using t‐tests. Traditionally‐trained and simulator‐trained groups were assessed for demographic differences using t‐tests and the chi‐square statistic. Spearman's rank correlation coefficient was used to assess for relationships between resident self‐confidence and quality indicators. All analyses were preformed using SPSS statistical software, version 16.0 (SPSS, Inc., Chicago, IL).
Results
All eligible residents participated in the study and completed the entire protocol. There was no significant difference in age, gender, ethnicity, year of training, or USMLE Step 1 and 2 scores between the traditionally‐trained and simulator‐trained groups.
Interrater reliability measured by the mean kappa coefficient was very high (n = 0.94) across the 27 IJ and SC checklist items. No resident met the MPS (79.1%) for CVC insertion at baseline testing. In the simulator‐trained group, 25 of 28 (89%) residents achieved SC skill mastery and 27 of 28 (96%) achieved IJ skill mastery within the standard four hour curriculum. All residents subsequently reached the MPS with less than one hour of additional practice time. A graphic portrait of the residents' pretest and posttest performance on the simulated CVC clinical skills examination with descriptive statistics is shown in Figure 3. After the educational intervention, posttest scores significantly improved (p < 0.001), to meet or exceed the MPS.
Traditionally trained and simulator‐trained residents independently inserted 46 CVCs during the study period. Simulator‐trained residents required significantly fewer needle passes to insert all actual CVCs in the MICU compared to traditionally trained residents: mean (M) = 1.79, standard deviation (SD) = 1.03 versus M = 2.78, SD = 1.77 (p = 0.04). As shown in Table 1, the groups did not differ in pneumothorax, arterial puncture, or mean number of CVC adjustments. In addition, the groups did not differ in use of ultrasound for IJ or SC CVC insertions. One IJ CVC was inserted without ultrasound in the traditionally‐trained group; 2 were inserted without ultrasound in the simulator‐trained group. Ultrasound was not used during any SC CVC insertions in the traditionally‐trained group and was used for 1 SC CVC insertion in the simulator‐trained group.
| Internal Jugular and Subclavian CVCs | |||
|---|---|---|---|
| Traditionally Trained Residents | Simulator Trained Residents | P value | |
| |||
| Number of attempts during insertion [mean (SD)] | 2.78 (1.77) | 1.79 (1.03) | 0.04* |
| Pneumothorax (number) | 0 | 0 | n/a |
| Arterial puncture (%) | 11 | 7 | 0.65 |
| CVC adjustment (%) | 15 | 8 | 0.52 |
| Confidence (%) [mean (SD)] | 68 (20) | 81 (11) | 0.02* |
Simulator‐trained residents displayed more self‐confidence about their procedural skills than traditionally‐trained residents (M = 81, SD = 11 versus M = 68, SD = 20, p = 0.02). Spearman correlations showed no practical association between resident self‐confidence and performance on CVC insertion quality indicators.
Discussion
This study demonstrates the use of a mastery learning model to develop CVC insertion skills to a high achievement level among internal medicine residents. Our data support prior work showing that procedural skills that are poor at baseline can be increased significantly using simulation‐based training and deliberate practice.1118, 28 This report on CVC insertion adds to the growing body of literature showing that simulation training complements standard medical education,1119, 28 and expands the clinical application of the mastery model beyond thoracentesis and ACLS.11, 12 Use of the mastery model described in this study also has important implications for patients. In our training program, residents are required to demonstrate procedural mastery in a simulated environment before independently performing a CVC insertion on an actual patient. This is in sharp contrast to the traditional clinical model of procedural training at the bedside, and may be used in other training programs and with other invasive procedures.
The second aim of our study was to determine the impact of simulation‐based training on actual clinical practice by residents in the MICU. To our knowledge, no prior study has demonstrated that simulation‐based training in CVC insertion improves patient outcomes. We believe our results advance what is known about the impact of simulation‐based training because simulator‐trained residents in this study performed actual CVC insertions in the MICU using significantly fewer needle passes. Needle passes have been used by other investigators as a surrogate measure for reduced CVC‐associated complications because mechanical complications rise exponentially with more than two insertion attempts.47, 29 We believe this finding demonstrates transfer of skill acquired from simulation‐based training to the actual clinical environment. It is possible that ultrasound training accounts for the improvement in the simulator‐trained group. However, we do not believe that ultrasound training is entirely responsible as prior work has shown that deliberate practice using mastery learning without ultrasound significantly improved resident performance of thoracentesis11 and ACLS12, 19 procedures. We did not show a significant reduction in complications such as pneumothorax or arterial puncture. This is likely due to the small sample size and the low number of procedures and complications during the study period.
Our results also show that resident self‐confidence regarding actual CVC insertions improved after simulation training. These findings are similar to prior reports linking improved confidence among trainees after simulation‐based training in CVC insertion.29, 30 Our results did not reveal a correlation between improved self‐confidence and clinical skill acquisition. Linking improved self‐confidence to improved clinical skill is important because self‐assessment does not always correlate with performance ability.31, 32
More study is needed to evaluate the impact of simulation‐based training on the quality of CVC insertions by trainees. Mechanisms shown to decrease complications of CVC placement include use of ultrasound,4, 7, 3336 full sterile barrier technique,3739 chlorhexidine skin preparations,4042 and nurse‐physician education.43 Our simulation‐training program incorporates each of these elements. We plan to expand our simulation‐based training intervention to a larger sample size to determine its impact on mechanical and infectious complication rates linked to CVC insertion.
This study has several limitations. It was performed at a single institution over a short time period. However, demonstration of significantly fewer needle passes and improved resident self‐confidence after simulator training are important findings that warrant further study. It was impossible to blind raters during the skills assessment examination about whether the resident was performing a pretest or posttest. This was accounted for by using a second rater, who was blind to the pretest and posttest status of the examinee. The arterial puncture rate of 7% among simulator‐trained residents was higher than expected, although it remains within published ranges.4, 5 Also, a low total number of CVCs were evaluated during the study. This is likely due to strict exclusion criteria employed in order to study the impact of simulation training. For example, CVC insertions were only evaluated if they were actually performed by study residents (supervised insertions were excluded) and femoral catheters were not evaluated. We did not track clinical experience with CVC insertion by residents before the study. Residents who were simulator‐trained may have had more clinical experience with CVC insertion and this may have impacted their performance. However, residents did not differ in year of training or clinical rotations, and there is clear evidence that clinical training is not a proxy for skill acquisition.44 Finally, outcome data were measured via resident questionnaires that relied on resident recall about CVC insertion rather than observer ratings. This method was selected because observer ratings could not be standardized given the large number of clinical supervisors in the MICU over the study period. Information about needle passes and arterial puncture also may not be documented in procedural notes and could not be obtained by medical record review. We attempted to minimize recall bias by surveying residents within 24 hours of CVC placement.
In conclusion, this study demonstrates that simulation‐based training and deliberate practice in a mastery learning setting improves performance of both simulated and actual CVC insertions by internal medicine residents. Procedural training remains an important component of internal medicine training, although internists are performing fewer invasive procedures now than in years past.45, 46 Use of a mastery model of CVC insertion requires that trainees demonstrate skill in a simulated environment before independently performing this invasive procedure on patients. Further study is needed to assess clinical outcomes such as reduced CVC‐related infections and mechanical complications after simulation‐based training.
Acknowledgements
The authors thank the Northwestern University internal medicine residents for their dedication to education and patient care. They acknowledge Drs. J. Larry Jameson and Charles Watts for their support and encouragement of this work.
Appendix
Central Venous Catheter Insertion Checklists for Simulation‐based Education 0, 0
| |||
| Informed consent obtained: must do all | A | B | C |
| Benefits | |||
| Risks | |||
| Consent given | |||
| Place the patient in slight Trendelenburg position | A | B | C |
| Flush the ports on the catheter with sterile saline | A | B | C |
| Clamp each port (ok to keep brown port open) | A | B | C |
| Remove brown port from end of catheter to accommodate wire | A | B | C |
| Area is cleaned with chlorhexadine | A | B | C |
| Resident gets in sterile gown, gloves, hat and mask | A | B | C |
| Area is draped in usual sterile fashion (must be full body drape) | A | B | C |
| The ultrasound (US) probe is properly set up with sterile sheath and sonographic gel | A | B | C |
| The vein is localized using anatomical landmarks with the US machine | A | B | C |
| If no US is used this is wrong | |||
| The skin is anesthetized with 1% lidocaine in a small wheal | A | B | C |
| The deeper structures are anesthetized | A | B | C |
| Localize the vein with this needle (optional) | A | B | C |
| Using the large needle or catheter‐ syringe complex, cannulate the vein while aspirating (must be done with US) | A | B | C |
| Remove the syringe from the needle or advance the catheter into the vein removing both the syringe and needle | A | B | C |
| Advance the guidewire into the vein no more than approximately 1215 cm | A | B | C |
| Knick the skin with the scalpel to advance the dilator | A | B | C |
| Advance the dilator over the guidewire and dilate the vein | A | B | C |
| Advance the triple lumen over the guidewire | A | B | C |
| Never let go of the guidewire | A | B | C |
| Once the catheter is inserted remove the guidewire in its entirety | A | B | C |
| Advance the catheter to approx to 1416cm on the right side, 1618 cm on the left side | A | B | C |
| Ensure there is blood flow/flush each port | A | B | C |
| Secure the catheter in place (suture or staple) | A | B | C |
| Place dressing over catheter | A | B | C |
| Get a chest x‐ray | A | B | C |
| Notify that the catheter is ok to use | A | B | C |
| Maintain sterile technique | A | B | C |
| |||
| Informed consent obtained: must do all | A | B | C |
| Benefits | |||
| Risks | |||
| Consent given | |||
| Place the patient in slight Trendelenburg position | A | B | C |
| Flush the ports on the catheter with sterile saline | A | B | C |
| Clamp each port (ok to leave brown port open) | A | B | C |
| Remove brown port from end of catheter to accommodate wire | A | B | C |
| Area is cleaned with chlorhexadine | A | B | C |
| Resident gets in sterile gown, gloves, hat and mask | A | B | C |
| Area is draped in usual sterile fashion (must be full body drape) | A | B | C |
| **The US probe is properly set up with sterile sheath and sonographic gel . (MUST DO if use US) | A | B | C |
| The vein is localized using US machine or anatomical landmarks are verbalized | A | B | C |
| The skin is anesthetized with 1% lidocaine in a small wheal | A | B | C |
| The deeper structures are anesthetized using a larger needle (must verbalize they anesthetize the clavicle) | A | B | C |
| Localize the vein with this needle (optional) | A | B | C |
| Using the large needle or catheter syringe complex cannulate the vein while aspirating (optional confirmed by US) | A | B | C |
| If US was not used then expected to state they are directing the needle to the sternal notch | A | B | C |
| Remove the syringe from the needle or advance the catheter into the vein removing both the syringe and needle | A | B | C |
| Advance the guidewire into the vein no more than approximately 1215 cm | A | B | C |
| Knick the skin with the scalpel to advance the dilator | A | B | C |
| Advance the dilator over the guidewire and dilate the vein | A | B | C |
| Advance the triple lumen over the guidewire | A | B | C |
| Never let go of the guidewire | A | B | C |
| Once the catheter is inserted remove the guidewire in its entirety | A | B | C |
| Advance the catheter to approx to 1416cm on the right side, 1618 cm on the left side | A | B | C |
| Ensure there is blood flow/flush each port | A | B | C |
| Secure the catheter in place (suture or staple) | A | B | C |
| Place dressing over catheter | A | B | C |
| Get a chest x‐ray | A | B | C |
| Notify that the catheter is ok to use | A | B | C |
| Maintain sterile technique | A | B | C |
Central venous catheter (CVC) insertions are commonly performed at the bedside in medical intensive care unit (MICU) settings. Internal medicine residents are required to demonstrate knowledge regarding CVC indications, complications, and sterile technique,1 and often perform the procedure during training. Education in CVC insertion is needed because many internal medicine residents are uncomfortable performing this procedure.2 CVC insertion also carries the risk of potentially life‐threatening complications including infection, pneumothorax, arterial puncture, deep vein thrombosis, and bleeding. Education and training may also contribute to improved patient care because increased physician experience with CVC insertion reduces complication risk.3, 4 Similarly, a higher number of needle passes or attempts during CVC insertion correlates with mechanical complications such as pneumothorax or arterial punctures.48 Pneumothorax rates for internal jugular (IJ) CVCs have been reported to range from 0% to 0.2% and for subclavian (SC) CVCs from 1.5% to 3.1%.4, 5 The arterial puncture rate for IJ CVCs ranges from 5.0% to 9.4% and for SC CVCs from 3.1% to 4.9%.4, 5 Proper use of ultrasound to assist with IJ CVC insertion has been shown to decrease these mechanical complications.4, 5 However, studies of ultrasound use with SC CVC insertion have mixed results.4
Simulation‐based training has been used in medical education to increase knowledge, provide opportunities for deliberate and safe practice, and shape the development of clinical skills.9, 10 We previously used simulation‐based mastery learning to improve the thoracentesis and advanced cardiac life support (ACLS) skills of internal medicine residents.11, 12 Although a few small studies have linked simulation‐based interventions to improved quality of care,1319 more work is needed to show that results from a simulated environment transfer to actual patient care.
This study had 2 aims. The first was to expand our simulation‐based mastery learning to CVC insertion using a CVC simulator and ultrasound device. The second was to assess quality indicators (number of needle passes, pneumothorax, arterial punctures, and need for catheter adjustment) and resident confidence related to actual CVC insertions in the MICU before and after an educational intervention.
Materials and Methods
Design
This was a cohort study20 of IJ and SC CVC insertions by 41 second‐ and third‐year internal medicine residents rotating through the MICU in a university‐affiliated program from October 2006 to February 2007. The Northwestern University Institutional Review Board approved the study. All study participants were required to give informed consent prior to participation.
Thirteen residents rotated through the MICU during a 6‐week preintervention phase. These residents served as a traditionally trained group that did not receive CVC insertion simulator training. Simultaneously, 28 residents who rotated through the MICU later in the study period received simulation‐based training in CVC insertion and served as the simulator‐trained group (Figure 1). Demographic data were obtained from the participants including age, gender, ethnicity, year of training, and scores on the United States Medical Licensing Examination (USMLE) Steps 1 and 2.
Simulator‐trained residents underwent baseline skill assessment (pretest) using a 27‐item checklist in IJ and SC CVC insertions (see Appendix). Checklists were developed by one author (J.H.B.) using appropriate references4, 5 and a step‐by‐step process,21 and reviewed for completeness by another author with expertise in checklist development (D.B.W.). Each skill or other action was listed in order and given equal weight. A dichotomous scoring scale of 1 = done correctly and 0 = done incorrectly/not done was imposed for each item. Assessments were performed using Simulab's CentralLineMan. This model features realistic tissue with ultrasound compatibility, an arterial pulse, and self‐sealing veins and skins. Needles, dilators, and guidewires can be inserted and realistic venous and arterial pressures demonstrated (Figure 2).
Residents in the simulator‐trained group received two, 2‐hour education sessions featuring a lecture, ultrasound training, deliberate practice with the CVC simulator, and feedback.22 Education sessions contained standardized didactic material on CVC indications and complications, as well as a stepwise demonstration of IJ and SC CVC insertions using ultrasound and landmark techniques. These sessions were supervised by a senior hospitalist faculty member with expertise in CVC insertions (J.H.B.). Residents were expected to use the ultrasound device for all IJ CVC insertions. However, its use was optional for SC CVC insertion. After training, residents were retested (posttest) and required to meet or exceed a minimum passing score (MPS) set by an expert panel for both IJ and SC procedures.23 This 11 member expert panel provided item‐based (Angoff) and group‐based (Hofstee) judgments on the 27‐item checklists as described previously.23
Residents who did not achieve the MPS had more deliberate practice and were retested until the MPS was reached; the key feature of mastery learning.24 After completing simulation‐based mastery learning in CVC insertion, the 28 simulator‐trained residents rotated through the MICU.
Data Collection
All pretests and posttests (using the 27‐item checklist) were graded by a single unblinded instructor (J.H.B.) and were videotaped. Another faculty instructor with expertise in scoring clinical skills examinations and blind to pre‐post status (D.B.W.) rescored a random 50% sample of the tests to assess interrater reliability.
Data regarding actual CVC insertions in the MICU were collected by contacting all MICU residents daily during the study period. This allowed for CVC insertions to be identified within 24 hours. All survey data were collected anonymously. The primary inserter of each CVC was questioned about quality indicators and procedural self‐confidence concerning CVC placement. CVCs primarily inserted by nonstudy subjects (first‐year residents, emergency medicine residents, pulmonary‐critical care medicine faculty members, and subspecialty fellows) or CVC placements that were supervised, but not directly placed by study participants, were excluded.
Outcome Measures
Pretest and posttest checklist scores from simulator‐trained residents were compared to measure the impact of training sessions. Residents rotating through the MICU were asked about several quality indicators related to actual CVC insertions. Quality indicators include: (1) number of needle passes required during the procedure (skin punctures); (2) presence of complications including pneumothorax and arterial puncture; and (3) need for CVC adjustment after chest x‐ray. Participants were also questioned regarding their confidence in CVC insertion using a 100 point scale (0 = not confident and 100 = very confident). Survey results from the 28 simulator‐trained residents were compared to results from the 13 traditionally‐trained residents.
Data Analysis
Checklist score reliability was estimated by calculating interrater reliability, the preferred method for assessments that depend on human judges, using the kappa () coefficient adjusted25, 26 using the formula of Brennan and Prediger.27 Within‐group differences from pretest (baseline) to posttest (outcome) were analyzed using paired t‐tests.
MICU survey results were compared using t‐tests. Traditionally‐trained and simulator‐trained groups were assessed for demographic differences using t‐tests and the chi‐square statistic. Spearman's rank correlation coefficient was used to assess for relationships between resident self‐confidence and quality indicators. All analyses were preformed using SPSS statistical software, version 16.0 (SPSS, Inc., Chicago, IL).
Results
All eligible residents participated in the study and completed the entire protocol. There was no significant difference in age, gender, ethnicity, year of training, or USMLE Step 1 and 2 scores between the traditionally‐trained and simulator‐trained groups.
Interrater reliability measured by the mean kappa coefficient was very high (n = 0.94) across the 27 IJ and SC checklist items. No resident met the MPS (79.1%) for CVC insertion at baseline testing. In the simulator‐trained group, 25 of 28 (89%) residents achieved SC skill mastery and 27 of 28 (96%) achieved IJ skill mastery within the standard four hour curriculum. All residents subsequently reached the MPS with less than one hour of additional practice time. A graphic portrait of the residents' pretest and posttest performance on the simulated CVC clinical skills examination with descriptive statistics is shown in Figure 3. After the educational intervention, posttest scores significantly improved (p < 0.001), to meet or exceed the MPS.
Traditionally trained and simulator‐trained residents independently inserted 46 CVCs during the study period. Simulator‐trained residents required significantly fewer needle passes to insert all actual CVCs in the MICU compared to traditionally trained residents: mean (M) = 1.79, standard deviation (SD) = 1.03 versus M = 2.78, SD = 1.77 (p = 0.04). As shown in Table 1, the groups did not differ in pneumothorax, arterial puncture, or mean number of CVC adjustments. In addition, the groups did not differ in use of ultrasound for IJ or SC CVC insertions. One IJ CVC was inserted without ultrasound in the traditionally‐trained group; 2 were inserted without ultrasound in the simulator‐trained group. Ultrasound was not used during any SC CVC insertions in the traditionally‐trained group and was used for 1 SC CVC insertion in the simulator‐trained group.
| Internal Jugular and Subclavian CVCs | |||
|---|---|---|---|
| Traditionally Trained Residents | Simulator Trained Residents | P value | |
| |||
| Number of attempts during insertion [mean (SD)] | 2.78 (1.77) | 1.79 (1.03) | 0.04* |
| Pneumothorax (number) | 0 | 0 | n/a |
| Arterial puncture (%) | 11 | 7 | 0.65 |
| CVC adjustment (%) | 15 | 8 | 0.52 |
| Confidence (%) [mean (SD)] | 68 (20) | 81 (11) | 0.02* |
Simulator‐trained residents displayed more self‐confidence about their procedural skills than traditionally‐trained residents (M = 81, SD = 11 versus M = 68, SD = 20, p = 0.02). Spearman correlations showed no practical association between resident self‐confidence and performance on CVC insertion quality indicators.
Discussion
This study demonstrates the use of a mastery learning model to develop CVC insertion skills to a high achievement level among internal medicine residents. Our data support prior work showing that procedural skills that are poor at baseline can be increased significantly using simulation‐based training and deliberate practice.1118, 28 This report on CVC insertion adds to the growing body of literature showing that simulation training complements standard medical education,1119, 28 and expands the clinical application of the mastery model beyond thoracentesis and ACLS.11, 12 Use of the mastery model described in this study also has important implications for patients. In our training program, residents are required to demonstrate procedural mastery in a simulated environment before independently performing a CVC insertion on an actual patient. This is in sharp contrast to the traditional clinical model of procedural training at the bedside, and may be used in other training programs and with other invasive procedures.
The second aim of our study was to determine the impact of simulation‐based training on actual clinical practice by residents in the MICU. To our knowledge, no prior study has demonstrated that simulation‐based training in CVC insertion improves patient outcomes. We believe our results advance what is known about the impact of simulation‐based training because simulator‐trained residents in this study performed actual CVC insertions in the MICU using significantly fewer needle passes. Needle passes have been used by other investigators as a surrogate measure for reduced CVC‐associated complications because mechanical complications rise exponentially with more than two insertion attempts.47, 29 We believe this finding demonstrates transfer of skill acquired from simulation‐based training to the actual clinical environment. It is possible that ultrasound training accounts for the improvement in the simulator‐trained group. However, we do not believe that ultrasound training is entirely responsible as prior work has shown that deliberate practice using mastery learning without ultrasound significantly improved resident performance of thoracentesis11 and ACLS12, 19 procedures. We did not show a significant reduction in complications such as pneumothorax or arterial puncture. This is likely due to the small sample size and the low number of procedures and complications during the study period.
Our results also show that resident self‐confidence regarding actual CVC insertions improved after simulation training. These findings are similar to prior reports linking improved confidence among trainees after simulation‐based training in CVC insertion.29, 30 Our results did not reveal a correlation between improved self‐confidence and clinical skill acquisition. Linking improved self‐confidence to improved clinical skill is important because self‐assessment does not always correlate with performance ability.31, 32
More study is needed to evaluate the impact of simulation‐based training on the quality of CVC insertions by trainees. Mechanisms shown to decrease complications of CVC placement include use of ultrasound,4, 7, 3336 full sterile barrier technique,3739 chlorhexidine skin preparations,4042 and nurse‐physician education.43 Our simulation‐training program incorporates each of these elements. We plan to expand our simulation‐based training intervention to a larger sample size to determine its impact on mechanical and infectious complication rates linked to CVC insertion.
This study has several limitations. It was performed at a single institution over a short time period. However, demonstration of significantly fewer needle passes and improved resident self‐confidence after simulator training are important findings that warrant further study. It was impossible to blind raters during the skills assessment examination about whether the resident was performing a pretest or posttest. This was accounted for by using a second rater, who was blind to the pretest and posttest status of the examinee. The arterial puncture rate of 7% among simulator‐trained residents was higher than expected, although it remains within published ranges.4, 5 Also, a low total number of CVCs were evaluated during the study. This is likely due to strict exclusion criteria employed in order to study the impact of simulation training. For example, CVC insertions were only evaluated if they were actually performed by study residents (supervised insertions were excluded) and femoral catheters were not evaluated. We did not track clinical experience with CVC insertion by residents before the study. Residents who were simulator‐trained may have had more clinical experience with CVC insertion and this may have impacted their performance. However, residents did not differ in year of training or clinical rotations, and there is clear evidence that clinical training is not a proxy for skill acquisition.44 Finally, outcome data were measured via resident questionnaires that relied on resident recall about CVC insertion rather than observer ratings. This method was selected because observer ratings could not be standardized given the large number of clinical supervisors in the MICU over the study period. Information about needle passes and arterial puncture also may not be documented in procedural notes and could not be obtained by medical record review. We attempted to minimize recall bias by surveying residents within 24 hours of CVC placement.
In conclusion, this study demonstrates that simulation‐based training and deliberate practice in a mastery learning setting improves performance of both simulated and actual CVC insertions by internal medicine residents. Procedural training remains an important component of internal medicine training, although internists are performing fewer invasive procedures now than in years past.45, 46 Use of a mastery model of CVC insertion requires that trainees demonstrate skill in a simulated environment before independently performing this invasive procedure on patients. Further study is needed to assess clinical outcomes such as reduced CVC‐related infections and mechanical complications after simulation‐based training.
Acknowledgements
The authors thank the Northwestern University internal medicine residents for their dedication to education and patient care. They acknowledge Drs. J. Larry Jameson and Charles Watts for their support and encouragement of this work.
Appendix
Central Venous Catheter Insertion Checklists for Simulation‐based Education 0, 0
| |||
| Informed consent obtained: must do all | A | B | C |
| Benefits | |||
| Risks | |||
| Consent given | |||
| Place the patient in slight Trendelenburg position | A | B | C |
| Flush the ports on the catheter with sterile saline | A | B | C |
| Clamp each port (ok to keep brown port open) | A | B | C |
| Remove brown port from end of catheter to accommodate wire | A | B | C |
| Area is cleaned with chlorhexadine | A | B | C |
| Resident gets in sterile gown, gloves, hat and mask | A | B | C |
| Area is draped in usual sterile fashion (must be full body drape) | A | B | C |
| The ultrasound (US) probe is properly set up with sterile sheath and sonographic gel | A | B | C |
| The vein is localized using anatomical landmarks with the US machine | A | B | C |
| If no US is used this is wrong | |||
| The skin is anesthetized with 1% lidocaine in a small wheal | A | B | C |
| The deeper structures are anesthetized | A | B | C |
| Localize the vein with this needle (optional) | A | B | C |
| Using the large needle or catheter‐ syringe complex, cannulate the vein while aspirating (must be done with US) | A | B | C |
| Remove the syringe from the needle or advance the catheter into the vein removing both the syringe and needle | A | B | C |
| Advance the guidewire into the vein no more than approximately 1215 cm | A | B | C |
| Knick the skin with the scalpel to advance the dilator | A | B | C |
| Advance the dilator over the guidewire and dilate the vein | A | B | C |
| Advance the triple lumen over the guidewire | A | B | C |
| Never let go of the guidewire | A | B | C |
| Once the catheter is inserted remove the guidewire in its entirety | A | B | C |
| Advance the catheter to approx to 1416cm on the right side, 1618 cm on the left side | A | B | C |
| Ensure there is blood flow/flush each port | A | B | C |
| Secure the catheter in place (suture or staple) | A | B | C |
| Place dressing over catheter | A | B | C |
| Get a chest x‐ray | A | B | C |
| Notify that the catheter is ok to use | A | B | C |
| Maintain sterile technique | A | B | C |
| |||
| Informed consent obtained: must do all | A | B | C |
| Benefits | |||
| Risks | |||
| Consent given | |||
| Place the patient in slight Trendelenburg position | A | B | C |
| Flush the ports on the catheter with sterile saline | A | B | C |
| Clamp each port (ok to leave brown port open) | A | B | C |
| Remove brown port from end of catheter to accommodate wire | A | B | C |
| Area is cleaned with chlorhexadine | A | B | C |
| Resident gets in sterile gown, gloves, hat and mask | A | B | C |
| Area is draped in usual sterile fashion (must be full body drape) | A | B | C |
| **The US probe is properly set up with sterile sheath and sonographic gel . (MUST DO if use US) | A | B | C |
| The vein is localized using US machine or anatomical landmarks are verbalized | A | B | C |
| The skin is anesthetized with 1% lidocaine in a small wheal | A | B | C |
| The deeper structures are anesthetized using a larger needle (must verbalize they anesthetize the clavicle) | A | B | C |
| Localize the vein with this needle (optional) | A | B | C |
| Using the large needle or catheter syringe complex cannulate the vein while aspirating (optional confirmed by US) | A | B | C |
| If US was not used then expected to state they are directing the needle to the sternal notch | A | B | C |
| Remove the syringe from the needle or advance the catheter into the vein removing both the syringe and needle | A | B | C |
| Advance the guidewire into the vein no more than approximately 1215 cm | A | B | C |
| Knick the skin with the scalpel to advance the dilator | A | B | C |
| Advance the dilator over the guidewire and dilate the vein | A | B | C |
| Advance the triple lumen over the guidewire | A | B | C |
| Never let go of the guidewire | A | B | C |
| Once the catheter is inserted remove the guidewire in its entirety | A | B | C |
| Advance the catheter to approx to 1416cm on the right side, 1618 cm on the left side | A | B | C |
| Ensure there is blood flow/flush each port | A | B | C |
| Secure the catheter in place (suture or staple) | A | B | C |
| Place dressing over catheter | A | B | C |
| Get a chest x‐ray | A | B | C |
| Notify that the catheter is ok to use | A | B | C |
| Maintain sterile technique | A | B | C |
- American Board of Internal Medicine. Procedures Required for Internal Medicine. Available at: http://www.abim.org/certification/policies/imss/im.aspx. Accessed January 28, 2009.
- ,,, et al.Beyond the comfort zone: residents assess their comfort performing inpatient medical procedures.Am J Med.2006;119:71.e17–e24.
- ,,, et al.Central vein catheterization. Failure and complication rates by three percutaneous approaches.Arch Intern Med.1986;146:259–261.
- ,.Preventing complications of central venous catheterization.N Engl J Med.2003;348:1123–1133.
- ,,, et al.Mechanical complications of central venous catheters.J Intensive Care Med.2006;21:40–46.
- ,,, et al.Risk factors of failure and immediate complication of subclavian vein catheterization in critically ill patients.Intensive Care Med.2002;28:1036–1041.
- ,,, et al.Complications and failures of subclavian‐vein catheterization.N Engl J Med.1994;331:1735–1738.
- .Central venous catheterization: better and worse.J Intensive Care Med.2006;21:51–53.
- ,,, et al.Reliability and validity of a simulation‐based acute care skills assessment for medical students and residents.Anesthesiology.2003;99:1270–1280.
- ,,, et al.Simulation technology for health care professional skills training and assessment.JAMA.1999;282:861–866.
- ,,, et al.Mastery learning of thoracentesis skills by internal medicine residents using simulation technology and deliberate practice.J Hosp Med.2008;3:48–54.
- ,,, et al.Mastery learning of advanced cardiac life support skills by internal medicine residents using simulation technology and deliberate practice.J Gen Intern Med.2006;21:251–256.
- ,,, et al.Laparoscopic skills are improved with LapMentor training: results of a randomized, double‐blinded study.Ann Surg.2006;243:854–860.
- ,,.Bronchoscopy simulator effectively prepares junior residents to competently perform basic clinical bronchoscopy.Ann Thorac Surg.2004;78:287–291.
- ,,, et al.Multicenter, randomized, controlled trial of virtual‐reality simulator training in acquisition of competency in colonoscopy.Gastrointest Endosc.2006;64:361–368.
- ,,, et al.Achieving house staff competence in emergency airway management: results of a teaching program using a computerized patient simulator.Crit Care Med.2004;32:2422–2427.
- ,.Computer simulator training enhances the competency of gastroenterology fellows at colonoscopy: results of a pilot study.Am J Gastroenterol.2004;99:33–37.
- ,,, et al.Virtual reality training improves operating room performance: results of a randomized, double‐blinded study.Ann Surg.2002;236:458–463.
- ,,, et al.Simulation‐based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: a case‐control study.CHEST.2008;133:56–61.
- ,.Clinical Epidemiology: the Essentials.4th ed.Philadelphia:Lippincott Williams 2005.
- . The Checklists Development Checklist. Western Michigan University Evaluation Center, July2000. Available at: http://www. wmich.edu/evalctr/checklists/cdc.htm. Accessed May 15, 2006.
- .Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains.Acad Med.2004;79:S70–S81.
- ,,, et al.Do baseline data influence standard setting for a clinical skills examination?Acad Med.2007;82:S105–S108.
- ,,, et al.Lessons for Continuing Medical Education from simulation research in undergraduate and graduate medical education.CHEST.2009;135.
- .Reliability: on the reproducibility of assessment data.Med Educ.2004;38:1006–1012.
- ,,.Statistical Methods for Rates and Proportions.3rd ed.New York:John Wiley 41:687–699.
- ,,, et al.Simulation‐based training of internal medicine residents in advanced cardiac life support protocols: a randomized trial.Teach Learn Med.2005;17:202–208.
- ,,.Central catheter simulation: a new training algorithm.Am Surg.2007;73:680–682.
- ,,.The use of tissue models for vascular access training. Phase I of the procedural patient safety initiative.J Gen Intern Med.2006;21:514–517.
- ,,, et al.The use of simulation in emergency medicine: a research agenda.Acad Emerg Med.2007;14:353–363.
- ,,, et al.Graduating internal medicine residents' self‐assessment and performance of advanced cardiac life support skills.Med Teach.2006;28:365–369.
- ,.Bedside ultrasonography in the ICU: Part 2.CHEST.2005;128:1766–1781.
- ,,, et al.Pulsed Doppler ultrasonography guidance for catheterization of the subclavian vein: a randomized study.Anesthesiology.1998;88:1195–1201.
- ,,, et al.Ultrasound guidance versus the landmark technique for the placement of central venous catheters in the emergency department.Acad Emerg Med.2002;9:800–805.
- ,,, et al.Ultrasound guidance for placement of central venous catheters: a meta‐analysis of the literature.Crit Care Med.1996;24:2053–2058.
- ,,, et al.Eliminating catheter‐related bloodstream infections in the intensive care unit.Crit Care Med.2004;32:2014–2020.
- ,,, et al.An intervention to decrease catheter‐related bloodstream infections in the ICU.N Engl J Med.2006;355:2725–2732.
- ,,, et al.Education of physicians‐in‐training can decrease the risk for vascular catheter infection.Ann Intern Med.2000;132:641–648.
- ,,, et al.Chlorhexidine compared with povidone‐iodine solution for vascular catheter‐site care: a meta‐analysis.Ann Intern Med.2002;136:792–801.
- ,,.Prospective randomised trial of povidone‐iodine, alcohol, and chlorhexidine for prevention of infection associated with central venous and arterial catheters.Lancet.1991;338:339–343.
- ,,, et al.Prospective, randomized trial of two antiseptic solutions for prevention of central venous or arterial catheter colonization and infection in intensive care unit patients.Crit Care Med.1996;24:1818–1823.
- ,,, et al.The effect of an education program on the incidence of central venous catheter‐associated bloodstream infection in a medical ICU.CHEST.2004;126:1612–1618.
- ,,.Systematic review: the relationship between clinical experience and quality of health care.Ann Intern Med.2005;142:260–273.
- ,.What procedures should internists do?Ann Intern Med.2007;146:392–393.
- ,.The declining number and variety of procedures done by general internists: a resurvey of members of the American College of Physicians.Ann Intern Med.2007;146:355–360.
- American Board of Internal Medicine. Procedures Required for Internal Medicine. Available at: http://www.abim.org/certification/policies/imss/im.aspx. Accessed January 28, 2009.
- ,,, et al.Beyond the comfort zone: residents assess their comfort performing inpatient medical procedures.Am J Med.2006;119:71.e17–e24.
- ,,, et al.Central vein catheterization. Failure and complication rates by three percutaneous approaches.Arch Intern Med.1986;146:259–261.
- ,.Preventing complications of central venous catheterization.N Engl J Med.2003;348:1123–1133.
- ,,, et al.Mechanical complications of central venous catheters.J Intensive Care Med.2006;21:40–46.
- ,,, et al.Risk factors of failure and immediate complication of subclavian vein catheterization in critically ill patients.Intensive Care Med.2002;28:1036–1041.
- ,,, et al.Complications and failures of subclavian‐vein catheterization.N Engl J Med.1994;331:1735–1738.
- .Central venous catheterization: better and worse.J Intensive Care Med.2006;21:51–53.
- ,,, et al.Reliability and validity of a simulation‐based acute care skills assessment for medical students and residents.Anesthesiology.2003;99:1270–1280.
- ,,, et al.Simulation technology for health care professional skills training and assessment.JAMA.1999;282:861–866.
- ,,, et al.Mastery learning of thoracentesis skills by internal medicine residents using simulation technology and deliberate practice.J Hosp Med.2008;3:48–54.
- ,,, et al.Mastery learning of advanced cardiac life support skills by internal medicine residents using simulation technology and deliberate practice.J Gen Intern Med.2006;21:251–256.
- ,,, et al.Laparoscopic skills are improved with LapMentor training: results of a randomized, double‐blinded study.Ann Surg.2006;243:854–860.
- ,,.Bronchoscopy simulator effectively prepares junior residents to competently perform basic clinical bronchoscopy.Ann Thorac Surg.2004;78:287–291.
- ,,, et al.Multicenter, randomized, controlled trial of virtual‐reality simulator training in acquisition of competency in colonoscopy.Gastrointest Endosc.2006;64:361–368.
- ,,, et al.Achieving house staff competence in emergency airway management: results of a teaching program using a computerized patient simulator.Crit Care Med.2004;32:2422–2427.
- ,.Computer simulator training enhances the competency of gastroenterology fellows at colonoscopy: results of a pilot study.Am J Gastroenterol.2004;99:33–37.
- ,,, et al.Virtual reality training improves operating room performance: results of a randomized, double‐blinded study.Ann Surg.2002;236:458–463.
- ,,, et al.Simulation‐based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: a case‐control study.CHEST.2008;133:56–61.
- ,.Clinical Epidemiology: the Essentials.4th ed.Philadelphia:Lippincott Williams 2005.
- . The Checklists Development Checklist. Western Michigan University Evaluation Center, July2000. Available at: http://www. wmich.edu/evalctr/checklists/cdc.htm. Accessed May 15, 2006.
- .Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains.Acad Med.2004;79:S70–S81.
- ,,, et al.Do baseline data influence standard setting for a clinical skills examination?Acad Med.2007;82:S105–S108.
- ,,, et al.Lessons for Continuing Medical Education from simulation research in undergraduate and graduate medical education.CHEST.2009;135.
- .Reliability: on the reproducibility of assessment data.Med Educ.2004;38:1006–1012.
- ,,.Statistical Methods for Rates and Proportions.3rd ed.New York:John Wiley 41:687–699.
- ,,, et al.Simulation‐based training of internal medicine residents in advanced cardiac life support protocols: a randomized trial.Teach Learn Med.2005;17:202–208.
- ,,.Central catheter simulation: a new training algorithm.Am Surg.2007;73:680–682.
- ,,.The use of tissue models for vascular access training. Phase I of the procedural patient safety initiative.J Gen Intern Med.2006;21:514–517.
- ,,, et al.The use of simulation in emergency medicine: a research agenda.Acad Emerg Med.2007;14:353–363.
- ,,, et al.Graduating internal medicine residents' self‐assessment and performance of advanced cardiac life support skills.Med Teach.2006;28:365–369.
- ,.Bedside ultrasonography in the ICU: Part 2.CHEST.2005;128:1766–1781.
- ,,, et al.Pulsed Doppler ultrasonography guidance for catheterization of the subclavian vein: a randomized study.Anesthesiology.1998;88:1195–1201.
- ,,, et al.Ultrasound guidance versus the landmark technique for the placement of central venous catheters in the emergency department.Acad Emerg Med.2002;9:800–805.
- ,,, et al.Ultrasound guidance for placement of central venous catheters: a meta‐analysis of the literature.Crit Care Med.1996;24:2053–2058.
- ,,, et al.Eliminating catheter‐related bloodstream infections in the intensive care unit.Crit Care Med.2004;32:2014–2020.
- ,,, et al.An intervention to decrease catheter‐related bloodstream infections in the ICU.N Engl J Med.2006;355:2725–2732.
- ,,, et al.Education of physicians‐in‐training can decrease the risk for vascular catheter infection.Ann Intern Med.2000;132:641–648.
- ,,, et al.Chlorhexidine compared with povidone‐iodine solution for vascular catheter‐site care: a meta‐analysis.Ann Intern Med.2002;136:792–801.
- ,,.Prospective randomised trial of povidone‐iodine, alcohol, and chlorhexidine for prevention of infection associated with central venous and arterial catheters.Lancet.1991;338:339–343.
- ,,, et al.Prospective, randomized trial of two antiseptic solutions for prevention of central venous or arterial catheter colonization and infection in intensive care unit patients.Crit Care Med.1996;24:1818–1823.
- ,,, et al.The effect of an education program on the incidence of central venous catheter‐associated bloodstream infection in a medical ICU.CHEST.2004;126:1612–1618.
- ,,.Systematic review: the relationship between clinical experience and quality of health care.Ann Intern Med.2005;142:260–273.
- ,.What procedures should internists do?Ann Intern Med.2007;146:392–393.
- ,.The declining number and variety of procedures done by general internists: a resurvey of members of the American College of Physicians.Ann Intern Med.2007;146:355–360.
Copyright © 2009 Society of Hospital Medicine
