User login
To Err Is Human revealed the underappreciated tension between the enormous benefits of medical care and the potential for harm.1 Following this report, there has been an explosion of research and commentary detailing quality improvement (QI) opportunities. One area of growing emphasis has been resident physician training.2, 3 If medical care is dangerous, then a substantial contributor to the hazard must be the apprentice‐style process of physician training and the novice skill set of the trainees.4, 5 Many resident training programs have devised efforts to decrease the errors committed by physicians‐in‐training,6 change the culture of residency training,7 engage residents in quality improvement,8, 9 and improve resident training in quality improvement.10
Many of the programs devised to teach QI in the residency setting require substantial funding, a large pool of QI experts, or redesign of resident training programs.410 While effective, these programs are not feasible for many resource‐constrained residency programs. A less intense program, using resident‐led, hospitalist‐facilitated, limited root cause analysis (RCA), has been adopted at the Internal Medicine Residency Program at the Mount Sinai Hospital (MSH). We describe our 2‐year experience using this technique, including cases discussed, improvement strategies suggested, projects implemented, and resident perceptions.
METHODS
Setting
Departmental QI leaders developed this initiative in the Internal Medicine Residency Program at the MSH in New York City, New York. This residency program trains over 140 residents annually in categorical, preliminary, and research track positions, as well as an affiliated medicine/pediatrics program. The program's residents rotate at 3 clinic sites: a tertiary care hospital, a public safety‐net hospital, and a Veterans Affairs hospital. The QI program was only implemented at the MSH. Over 90% of the program's graduates go on to complete a subspecialty fellowship.
Intervention Description
The QI program was designed around a noon‐time quality improvement conference (QIC) occurring once every 4 weeks. In the weeks prior to the session, chief residents and a hospitalist mentor selected a case related to an inpatient care issue. Potential cases were solicited, and/or offered, from a range of sources including attending physicians, nurse managers, residents, and quality officers. Only cases from the teaching services were chosen. To ensure that participants on the case were able to recall relevant details, preference was given to more recent cases. A third‐year resident on an elective or outpatient block was chosen to investigate the case. To maximize the objectivity of the investigation, every effort was made to select a resident who was not involved in the care of the patient.
The resident was instructed to complete a limited RCA (fewer fact‐finding interviews and only 1 group meeting) and was directed to online resources.11 Each resident presenter worked closely with the chief residents and hospitalist mentor to identify appropriate strategies for collecting data and interviewing involved parties. If necessary, either due to volume of work or sensitivity of the case, the chief resident or hospitalist would assist with the data gathering. The resident contacted multiple parties involved in the patient care issue including, nurses, residents, attendings, pharmacists, social workers, and, if appropriate, the patient. The resident constructed a timeline for each case, and identified specific points in the patient care experience, where errors, near misses, or misunderstandings occurred. During the QIC, these findings were presented to Internal Medicine residents, chief residents, representatives from the Chief Medical Officer's office, attending physicians overseeing the residents on inpatient rotations, and representatives from any group (social work nursing, housekeeping, pharmacy, etc) that may have impacted patient care for the particular case being investigated. On average, 50 healthcare providers attended the QIC. Lunch was provided.
After the findings were presented, a chief resident and a lead hospitalist facilitated a group discussion on the circumstances surrounding the case. Discussions were focused on identifying system‐wide failures and proposing systems‐based solutions. Great efforts were made to remind all participants to refrain from individual blame. At the end of each QIC, participants summarized and prioritized suggestions to reduce the discussed error. Interested residents were invited to form improvement committees for cases with viable solutions. Each committee attempted to implement improvements discussed during the QIC. Committees, led by a representative from the Division of Hospital Medicine, included 2 to 4 residents as well as healthcare workers from other disciplines if appropriate. For all improvement efforts, the focus was on the interventions which appeared high yield with low cost.
Intervention Evaluation
The program was exempt from Institutional Review Board review as a part of the Department of Medicine's quality improvement and assurance portfolio.
The results of the QICs were tracked. After each case, a QI team consisting of chief residents and representatives from the Division of Hospital Medicine recorded the cases presented, and interventions suggested for each case, in an online database. After implementation, the success of each intervention was recorded. To evaluate the types of interventions suggested by residents, the 3 physician‐authors, who regularly attend these conferences and have a focused career interest in QI, grouped all suggestions into 4 broad categories: Educational, Reminder Systems, Design Changes (protocol‐based), and Design Changes (Information Technology [IT]‐based). Design change interventions (IT‐based) consisted of an adjustment to electronic systems, such as displaying specific lab results on a medication ordering system. Design changes (protocol‐based) consisted of changes made to standing protocols such as nursing protocols for reporting abnormal lab values. Reminder system interventions were endeavors such as a checklist for discharge planning. Educational interventions focused on providing additional training sessions or conferences.
The 3 physician‐authors independently reviewed each suggested intervention to determine its success. They first evaluated whether the change was attempted or not. For all attempted interventions, the reviewer then assessed if there was either objective permanent system‐wide change, subjective behavior change, or no change. To meet the objective change threshold, the intervention either had to have permanently changed provider workflow or have data demonstrating behavior change or improved outcome. Interventions with anecdotal evidence that behavior was improved or modified, but lacking systematic data, were qualified as subjective behavior change. For each assessment, 2 of the 3 reviewers needed to agree for an intervention to be recorded as a success.
Resident views on the monthly conferences were solicited via an anonymous and voluntary questionnaire. A first survey was designed to assess whether residents felt that the conferences provided them with the ability to recognize and improve systems errors which compromise patient care. This survey was administered at the conclusion of the first year of the program to residents who attended the final 2 QICs. A second survey assessed whether the tone of the conferences was constructive and blame‐free. This survey was administered at the conclusion of the second year of the program to residents who attended the year's final 2 QICs.
RESULTS
Over the first 22 months of the program, 20 conferences were held (Table 1). The topics covered ranged considerably and included: deficits in supervision, medication errors, patient satisfaction, staff safety, and 30‐day readmissions. Forty‐six distinct interventions were suggested during these conferences. Of those, an attempt was made to initiate 25 (54%) of these suggestions (Table 2). Of the 25 interventions that were initiated, 18 (72%) were determined to be successful. Eight resulted in objective permanent system‐wide change and 10 resulted in subjective behavior change among residents.
| QIC Topic | Interventions Suggested by Residents | Suggestion Results (Attempted/Not Attempted, Successful/Unsuccessful) |
|---|---|---|
| ||
| Central venous catheter guide wire lost during code placement | Improved supervision and training for line placement | Attempted, but unsuccessful |
| Avoid unnecessary line placement during codes | Attempted, but unsuccessful | |
| Inappropriate administration of warfarin | Decision support providing real‐time coagulation profile | Attempted and successful |
| Central line bloodstream infection | Clarified and encouraged use of line service | Attempted and successful |
| Daily documentation of catheter placement date | Not attempted | |
| Delayed administration of pain medication | Training nurses to use text paging communication system | Attempted and successful |
| Patient discharged on wrong medication dose | Do not use abbreviations | Not attempted |
| Electronic medication reconciliation | Attempted and successful | |
| Confusion over code status | Clarification of various forms used for DNR | Not attempted |
| Better communication of code status during signout | Not attempted | |
| Patient received hydromorphone IV instead of PO during verbal order at end‐of‐life | Verbal orders should have talk back verification | Attempted, but unsuccessful |
| Encourage informing patients of medical errors | Attempted, but unsuccessful | |
| Premature closure of diagnosis during transfer from MICU | Improve comfort level disagreeing with supervisors | Attempted, but unsuccessful |
| Reassessment of patient prior to late‐day MICU transfers | Not attempted | |
| Patient erroneously received clopidogrel bisulfate (Plavix) for years due to poor medication reconciliation | Improved discharge summary interface | Attempted and successful |
| Encourage physicians to call PCP on discharge | Attempted and successful | |
| Modified barium swallow ordered incorrectly, resulting in patient aspiration | Simplify electronic order entry system to clearly identify tests | Not attempted |
| Change radiology requisition form to facilitate communication | Not attempted | |
| Fingersticks leading to blood exposure | Train PGY1s on the needles used at all 3 hospitals | Not attempted |
| Improve mask with face shields and gown availability | Attempted and successful | |
| Patient discharged with central venous catheter still in place | Check list for lines and Foleys | Not attempted |
| Improved discharge documentation | Not attempted | |
| 30‐Day readmission | Mandatory discharge summary completion prior to discharge | Attempted and successful |
| Discharge summary training during intern year | Attempted and successful | |
| DKA developed in house when insulin not administered | Improve communication between floor and dialysis RNs | Not attempted |
| Better PA supervision by residents regarding order writing | Attempted and successful | |
| Compromised patient satisfaction | Patient handouts with name and role of each care team member | Attempted, but unsuccessful |
| Patient satisfaction coaching | Attempted and successful | |
| Elevated PTT and poor documentation | Improved feedback to residents regarding daily notes | Not attempted |
| Nurses must call physicians with alert values | Not attempted | |
| Hospital‐acquired MRSA | Improve availability of contact precaution gowns | Attempted, but unsuccessful |
| Direct observation of hand washing on morning rounds | Attempted and successful | |
| Staff safety with deranged family member | Education of staff regarding safety protocols | Attempted, but unsuccessful |
| Transfer of unstable patient from outside hospital ICU to general medicine floor | Standardization of OSH transfer guidelines | Not attempted |
| Improved documentation of transferring MD contract data | Attempted and successful | |
| Consult called, patient not seen by attending | Education of faculty on existing institutional consult policy | Attempted, but unsuccessful |
| Clarification of violations reporting process for hospital consults | Attempted, but unsuccessful | |
| Type of Intervention | No. of Interventions Suggested | No. of Interventions Implemented (%) | Of Implemented Interventions, No. Which Were Successful (%) | No. of Attempted Interventions With Objective Change (%) | No. of Attempted Interventions With Subjective Change (%) |
|---|---|---|---|---|---|
| Design changes: information technology‐based | 5 | 2 (40) | 2 (100%) | 2 (100) | 0 (0) |
| Design changes: protocol‐based | 17 | 10 (59) | 8 (80%) | 5 (50) | 3 (30) |
| Educational | 20 | 11 (55) | 7 (64) | 1 (9) | 6 (55) |
| Reminder systems | 4 | 2 (50) | 1 (50) | 0 (0) | 1 (50) |
| Total | 46 | 25 (54) | 18 (72) | 8 (32) | 10 (40) |
Two IT‐based system design changes were implemented; both resulted in objective system‐wide change. Eight protocol‐based design changes were implemented successfully, 5 objectively, and 3 subjectively. Seven educational interventions and 1 reminder system intervention were initiated.
The most successful intervention to come from these conferences was the implementation of an electronic medication reconciliation program. The reconciliation program was suggested following a conference on a patient who was discharged home on the wrong dose of a medication. The institution's paper‐based medication reconciliation process, particularly for heart‐failure patients, had long been known to be deficient. The QIC brought this issue to life by highlighting a cases that may have been ameliorated with a more robust medication reconciliation process. Enthusiastic residents were invited to build a case for medication reconciliation to the Chief Medical Officer, and this helped garner resources for the hospital‐wide project. Another successful IT‐based intervention was initiated after a case of inappropriate administration of warfarin to a patient with an already elevated international normalized ratio (INR). The computerized order entry system was changed so that, at the point of ordering warfarin, the most recent coagulation profile and platelet values appear before an order can be finalized.
An example of a protocol‐based intervention came from a conference that focused on poor communication at the time of discharge, which resulted in a 30‐day readmission. As a result, resident work flow was changed so that discharge summaries are expected to be completed at the time of discharge. Along with this protocol change was an educational initiative to improve the quality of discharge summaries by including essential data for the transition of care.
Overall, residents reviewed the conferences very positively (Table 3). The response rate for the first year survey was 40% (56/140) and the second year survey was 18% (26/143). The vast majority of participants felt that the conferences were of high quality (96%) and that the exercise could lead to improvement in quality (98%). Residents felt that the conference focused more on system issues than individual shortcomings (92%). A majority felt comfortable expressing their opinions during the conferences (77%).
| Overall Conference Quality | ||
|---|---|---|
| Question | Mean Score (n = 53) | Rating Question a 4 or 5 |
| Conference Tone | ||
| Question | Mean Score (n = 26) | Rating Question With a 4 or 5 |
| ||
| Please rate the overall quality of the QIC conferences. | 4.49* | 98% |
| The case highlighted an issue that is highly relevant to the quality of patient care. | 4.81 | 100% |
| Solutions discussed at this conference could lead to improved patient care and/or patient satisfaction. | 4.65 | 96% |
| My knowledge of issues related to hospital quality and patient safety has been enhanced by this conference. | 4.61 | 96% |
| The QIC focused on individuals, individual actions, or omissions, which compromised high quality care. | 3.35 | 50% |
| The QIC focused on system failures that compromised high quality care. | 4.35 | 92% |
| I felt comfortable sharing my honest opinions about the medical events presented during the conferences. | 4.15 | 77% |
| I avoided expressing my opinions about the medical events presented during the conferences because I did not want to criticize my peers. | 2.5 | 19% |
DISCUSSION
The first 20 sessions from this resident‐led, hospitalist‐facilitated QI program provided evidence that residents can contribute to patient safety within a large tertiary care center. The role of residents in actively addressing errors and unsatisfactory outcomes in the hospital has not been a traditional QI focus.12 Involvement has typically been a passive process for physician trainees, while more senior clinical staff members decide on and prioritize QI activities. We have observed that empowering residents to take a more active role in performance improvement yields significant change and does more than simply educate about basic QI methodology.
One reason for the success of these conferences was leveraging insights of residents as key front line providers. Residents spend more time than perhaps any other category of hospital employee working within clinical care systems. They are deeply aware of the quality struggles inherent to large healthcare organizations, and this insight can lead to high impact suggestions for improvement. Often, suggestions were simple proposals that were overlooked or unappreciated by other administrative leaders. An example of this type of contribution was when residents brought the lack of infection control equipment, on certain units, to the attention of the infection control staff and facility engineers. At a separate conference, residents informed the transfer office staff that valuable contact information for physicians accepting outside hospital transfers was not being collected. Both of these observations led to quick change, with better infection control gown availability and improved documentation by transfer office staff.
Our program also demonstrated that including residents in QI provides momentum for either a training program or an institution to pursue solutions that might have otherwise been resisted. The improvement suggestion to complete discharge summaries prior to the patient leaving the hospital had long been a goal for the residency program leadership, but there was hesitation to force this work flow change on the residents. After a QI conference, when a number of the residents themselves made the suggestion, implementing the change was much easier. Similarly, after several cases of clear errors relating to a suboptimal process of medication reconciliation, the institution dedicated scarce IT personnel to work with providers to develop a robust, user‐friendly medication reconciliation application to decrease transition of care errors.
Through this program, residents also demonstrated their ability to deconstruct patient care problems. For each case, resident session leaders interviewed physician providers, physician extenders, nurses, nurse managers, pharmacists, security staff, engineering staff, and administrative staff. They gathered crucial information regarding the patient care event and the gaps or errors that led to a poor outcome. After many of the conferences, the resident presenters commented on how the investigative exercise left them more appreciative of the complexity of the medical system and interested in fixing the problems uncovered.
The feedback from the resident surveys demonstrated that residents valued the QI program. The data collected also shows that such programs can be executed in a manner which highlights system flaws. Our data do, however, suggest that there is room to improve the tone of the conference to further decrease the sense from residents that quality discussions focus on individuals. Residents often struggle to master the myriad new expectations inherent in the transition from student to physician.13 A quality process which discourages already overworked and uncertain trainees, by creating a process which assigns blame for unintentional quality shortcomings, would be counterproductive.
Lessons Learned
While this QI program has had success uncovering clinical care issues, and creating a climate and process for resident participation in improvement, there has been a number of limitations and lessons learned. Most importantly, including busy residents in any process that requires regular participation and follow‐through is difficult. A number of suggested improvements which created substantial interest and early momentum were ultimately left unfinished, as residents and even faculty facilitators became overwhelmed by clinical responsibilities. In fact, the majority of suggestions have not been successfully implemented and even fewer have created lasting change. This must be carefully monitored, as experiencing multiple failures can undermine the empowerment that such QI programs are created to foster.
Regular reflection on the successful and unsuccessful projects yielded several important insights that resulted in changes over the course of the program. Suggestions were more likely to move from idea generation to execution if the QIC was attended by administrators with decision‐making authority. Several of the suggestionsimproved medication reconciliation, better transfer documentation, and improved availability of infection control productswere able to be acted upon because conference attendees were administrators with purview over these issues. Many times, these leaders were more than willing to implement helpful suggestions, but simply needed them to be brought to their attention. As a result, we have been more attentive to inviting as many stakeholders as possible to the QICs.
It was also clear that suggestions would not be realized without a physician leader and were more successful when resident interest was substantial. After each QIC, residents who had made promising suggestions were approached to continue to participate. If the residents agreed, the projects were pursued and a faculty or chief resident leader was assigned. Lastly, we have also made use of one of the department's QI data analysts to assist with project completion. This individual has been made available to provide administrative support (organizing meetings, paperwork, etc) but also to provide data for projects, should the need arise.
Another important finding is that the tone of the QI program must be constantly monitored. Despite reminding residents at each session that the exercise was for the purpose of identifying systems barriers to delivering high quality care, there were times when residents felt targeted or blamed. At one point, a number of residents voiced their concerns that the conferences had spent too much time highlighting quality failures without recognizing the many positive performances on the teaching service. As a result, subsequent conferences often began by highlighting quality improvements made. Additionally, a part of 1 session each year had been dedicated to reading letters and e‐mails sent by patients or families which highlight memorably positive performances by the residents. Finally, care was taken to make sure invited guests to the sessions were reminded of the session's blame‐free ground rules.
Care must be taken when investigating clinical cases. On several occasions, attending physicians expressed discomfort with having residents scrutinize a clinical event. Although this process was protected under the QI umbrella and faculty names were never shared at the conferences, some faculty believed that this process was the purview of departmental or hospital QI staff, not untrained residents. Given the support provided for this program by the department chair and program director, as well as the professional nature with which the residents conducted their inquiries, there was little difficulty rejecting this line of objection. This feedback did lead supervisors to be more involved with the resident presenters, coaching them regarding data gathering and interviewing. If a case appears that it will be particularly sensitive, the hospitalist mentor or chief resident will reach out to involved residents and faculty to notify them that the case will be reviewed.
A final development secured, in part, as a result of this quality program has been more protected faculty time. At the start of this program, all faculty time was donated time on top of other administrative and patient care responsibilities. After the first 18 months of the QIC program, the residency program named an assistant program director for quality. At the time of writing this manuscript, the program further invested in quality by naming both an assistant and associate program director for quality. These positions combined amount to at least 0.4 full‐time equivalents (FTE). Of that, roughly 0.1 FTE is spent working on the QICs and subsequent project implementation.
Limitations
The evaluation of the success of the interventions potentially biased our findings. The qualitative method of using multiple reviewers, all of whom were invested in the program's outcomes, to gauge the success of initiated interventions may have resulted in an overestimate of the project's effectiveness. Furthermore, the category of subjective change lacks measurable criteria, making replication of the findings difficult.
The results presented here are from a single institution, conceived of and executed by a group of dedicated faculty. Moreover, both the chair of the department and the program director were very supportive of this endeavor. Possibly, because of these aspects, the findings presented here would not be readily replicated at another institution.
The percentage of residents who completed the feedback surveys was low. This may result in an overestimate of quality, value, and tone of the conferences, as well as potentially missing an opportunity for improving the program. We will address this issue through more rigorous quantitative and qualitative feedback at the end of the third year of the program.
CONCLUSIONS
Residents are willing and effective participants in a QI program. As front line providers, their experiences are valuable and their willingness to share insights can be an impetus for change. Finally, a process which includes modest investigation by third year residents, has faculty support and oversight, and provides minimal administrative support can overcome the difficulty of involving overworked residents in quality efforts.
Acknowledgements
The authors acknowledge Michael Pourdehnad for his role in developing the quality program.
- ,,.To Err Is Human: Building a Safer Health System.Washington, DC:National Academy Press;1999.
- ,,,,.Redesigning residency education in internal medicine: a position paper from the Association of Program Directors in Internal Medicine.Ann Intern Med.2006;144:920–926.
- Accreditation Council for Graduate Medical Education. Program directors guide to the common program requirements. Available at: http://www.acgme.org/acWebsite/navPages/commonpr_documents/ CompleteGuide_v2%20.pdf. Accessed May 5,2010.
- ,,,.Medical errors involving trainees: a study of closed malpractice claims from 5 insurers.Arch Intern Med.2007;167:2030–2036.
- ,,,,,.Residents report on adverse events and their causes.Arch Intern Med.2005;165:2607–2613.
- ,.A system of analyzing medical errors to improve GMA curricula and programs.Acad Med.2001;76:125–133.
- ,,, et al.Changing conversations: teaching safety and quality in residency training.Acad Med.2008;83(11):1080–1087.
- ,,.Practice‐based learning and improvement: a curriculum in continuous quality improvement for surgery residents.Arch Surg.2007;142:479–483.
- .Involving residents in quality improvement: contrasting “top‐down” and “bottom‐up” approaches. Accreditation Council for Graduate Medical Education and Institute for Healthcare Improvement‐day project.ACGME Bulletin. August2008.
- ,,,,.Creating a quality improvement elective for medical house officers.Gen Intern Med.2004;19(8):861–867.
- National Center for Patient Safety. United States Department for Veteran Affairs. Root cause analysis tools. Available at: http://www.patientsafety.gov/CogAids/RCA/. Accessed August 17,2010.
- ,,, et al.Residents' engagement in quality improvement: a systematic review of the literature.Acad Med.2009;84:1757–1764.
- ,,.Graduates from a traditional medical curriculum evaluate the effectiveness of their medical curriculum through interviews.BMC Med Educ.2009;9:64.
To Err Is Human revealed the underappreciated tension between the enormous benefits of medical care and the potential for harm.1 Following this report, there has been an explosion of research and commentary detailing quality improvement (QI) opportunities. One area of growing emphasis has been resident physician training.2, 3 If medical care is dangerous, then a substantial contributor to the hazard must be the apprentice‐style process of physician training and the novice skill set of the trainees.4, 5 Many resident training programs have devised efforts to decrease the errors committed by physicians‐in‐training,6 change the culture of residency training,7 engage residents in quality improvement,8, 9 and improve resident training in quality improvement.10
Many of the programs devised to teach QI in the residency setting require substantial funding, a large pool of QI experts, or redesign of resident training programs.410 While effective, these programs are not feasible for many resource‐constrained residency programs. A less intense program, using resident‐led, hospitalist‐facilitated, limited root cause analysis (RCA), has been adopted at the Internal Medicine Residency Program at the Mount Sinai Hospital (MSH). We describe our 2‐year experience using this technique, including cases discussed, improvement strategies suggested, projects implemented, and resident perceptions.
METHODS
Setting
Departmental QI leaders developed this initiative in the Internal Medicine Residency Program at the MSH in New York City, New York. This residency program trains over 140 residents annually in categorical, preliminary, and research track positions, as well as an affiliated medicine/pediatrics program. The program's residents rotate at 3 clinic sites: a tertiary care hospital, a public safety‐net hospital, and a Veterans Affairs hospital. The QI program was only implemented at the MSH. Over 90% of the program's graduates go on to complete a subspecialty fellowship.
Intervention Description
The QI program was designed around a noon‐time quality improvement conference (QIC) occurring once every 4 weeks. In the weeks prior to the session, chief residents and a hospitalist mentor selected a case related to an inpatient care issue. Potential cases were solicited, and/or offered, from a range of sources including attending physicians, nurse managers, residents, and quality officers. Only cases from the teaching services were chosen. To ensure that participants on the case were able to recall relevant details, preference was given to more recent cases. A third‐year resident on an elective or outpatient block was chosen to investigate the case. To maximize the objectivity of the investigation, every effort was made to select a resident who was not involved in the care of the patient.
The resident was instructed to complete a limited RCA (fewer fact‐finding interviews and only 1 group meeting) and was directed to online resources.11 Each resident presenter worked closely with the chief residents and hospitalist mentor to identify appropriate strategies for collecting data and interviewing involved parties. If necessary, either due to volume of work or sensitivity of the case, the chief resident or hospitalist would assist with the data gathering. The resident contacted multiple parties involved in the patient care issue including, nurses, residents, attendings, pharmacists, social workers, and, if appropriate, the patient. The resident constructed a timeline for each case, and identified specific points in the patient care experience, where errors, near misses, or misunderstandings occurred. During the QIC, these findings were presented to Internal Medicine residents, chief residents, representatives from the Chief Medical Officer's office, attending physicians overseeing the residents on inpatient rotations, and representatives from any group (social work nursing, housekeeping, pharmacy, etc) that may have impacted patient care for the particular case being investigated. On average, 50 healthcare providers attended the QIC. Lunch was provided.
After the findings were presented, a chief resident and a lead hospitalist facilitated a group discussion on the circumstances surrounding the case. Discussions were focused on identifying system‐wide failures and proposing systems‐based solutions. Great efforts were made to remind all participants to refrain from individual blame. At the end of each QIC, participants summarized and prioritized suggestions to reduce the discussed error. Interested residents were invited to form improvement committees for cases with viable solutions. Each committee attempted to implement improvements discussed during the QIC. Committees, led by a representative from the Division of Hospital Medicine, included 2 to 4 residents as well as healthcare workers from other disciplines if appropriate. For all improvement efforts, the focus was on the interventions which appeared high yield with low cost.
Intervention Evaluation
The program was exempt from Institutional Review Board review as a part of the Department of Medicine's quality improvement and assurance portfolio.
The results of the QICs were tracked. After each case, a QI team consisting of chief residents and representatives from the Division of Hospital Medicine recorded the cases presented, and interventions suggested for each case, in an online database. After implementation, the success of each intervention was recorded. To evaluate the types of interventions suggested by residents, the 3 physician‐authors, who regularly attend these conferences and have a focused career interest in QI, grouped all suggestions into 4 broad categories: Educational, Reminder Systems, Design Changes (protocol‐based), and Design Changes (Information Technology [IT]‐based). Design change interventions (IT‐based) consisted of an adjustment to electronic systems, such as displaying specific lab results on a medication ordering system. Design changes (protocol‐based) consisted of changes made to standing protocols such as nursing protocols for reporting abnormal lab values. Reminder system interventions were endeavors such as a checklist for discharge planning. Educational interventions focused on providing additional training sessions or conferences.
The 3 physician‐authors independently reviewed each suggested intervention to determine its success. They first evaluated whether the change was attempted or not. For all attempted interventions, the reviewer then assessed if there was either objective permanent system‐wide change, subjective behavior change, or no change. To meet the objective change threshold, the intervention either had to have permanently changed provider workflow or have data demonstrating behavior change or improved outcome. Interventions with anecdotal evidence that behavior was improved or modified, but lacking systematic data, were qualified as subjective behavior change. For each assessment, 2 of the 3 reviewers needed to agree for an intervention to be recorded as a success.
Resident views on the monthly conferences were solicited via an anonymous and voluntary questionnaire. A first survey was designed to assess whether residents felt that the conferences provided them with the ability to recognize and improve systems errors which compromise patient care. This survey was administered at the conclusion of the first year of the program to residents who attended the final 2 QICs. A second survey assessed whether the tone of the conferences was constructive and blame‐free. This survey was administered at the conclusion of the second year of the program to residents who attended the year's final 2 QICs.
RESULTS
Over the first 22 months of the program, 20 conferences were held (Table 1). The topics covered ranged considerably and included: deficits in supervision, medication errors, patient satisfaction, staff safety, and 30‐day readmissions. Forty‐six distinct interventions were suggested during these conferences. Of those, an attempt was made to initiate 25 (54%) of these suggestions (Table 2). Of the 25 interventions that were initiated, 18 (72%) were determined to be successful. Eight resulted in objective permanent system‐wide change and 10 resulted in subjective behavior change among residents.
| QIC Topic | Interventions Suggested by Residents | Suggestion Results (Attempted/Not Attempted, Successful/Unsuccessful) |
|---|---|---|
| ||
| Central venous catheter guide wire lost during code placement | Improved supervision and training for line placement | Attempted, but unsuccessful |
| Avoid unnecessary line placement during codes | Attempted, but unsuccessful | |
| Inappropriate administration of warfarin | Decision support providing real‐time coagulation profile | Attempted and successful |
| Central line bloodstream infection | Clarified and encouraged use of line service | Attempted and successful |
| Daily documentation of catheter placement date | Not attempted | |
| Delayed administration of pain medication | Training nurses to use text paging communication system | Attempted and successful |
| Patient discharged on wrong medication dose | Do not use abbreviations | Not attempted |
| Electronic medication reconciliation | Attempted and successful | |
| Confusion over code status | Clarification of various forms used for DNR | Not attempted |
| Better communication of code status during signout | Not attempted | |
| Patient received hydromorphone IV instead of PO during verbal order at end‐of‐life | Verbal orders should have talk back verification | Attempted, but unsuccessful |
| Encourage informing patients of medical errors | Attempted, but unsuccessful | |
| Premature closure of diagnosis during transfer from MICU | Improve comfort level disagreeing with supervisors | Attempted, but unsuccessful |
| Reassessment of patient prior to late‐day MICU transfers | Not attempted | |
| Patient erroneously received clopidogrel bisulfate (Plavix) for years due to poor medication reconciliation | Improved discharge summary interface | Attempted and successful |
| Encourage physicians to call PCP on discharge | Attempted and successful | |
| Modified barium swallow ordered incorrectly, resulting in patient aspiration | Simplify electronic order entry system to clearly identify tests | Not attempted |
| Change radiology requisition form to facilitate communication | Not attempted | |
| Fingersticks leading to blood exposure | Train PGY1s on the needles used at all 3 hospitals | Not attempted |
| Improve mask with face shields and gown availability | Attempted and successful | |
| Patient discharged with central venous catheter still in place | Check list for lines and Foleys | Not attempted |
| Improved discharge documentation | Not attempted | |
| 30‐Day readmission | Mandatory discharge summary completion prior to discharge | Attempted and successful |
| Discharge summary training during intern year | Attempted and successful | |
| DKA developed in house when insulin not administered | Improve communication between floor and dialysis RNs | Not attempted |
| Better PA supervision by residents regarding order writing | Attempted and successful | |
| Compromised patient satisfaction | Patient handouts with name and role of each care team member | Attempted, but unsuccessful |
| Patient satisfaction coaching | Attempted and successful | |
| Elevated PTT and poor documentation | Improved feedback to residents regarding daily notes | Not attempted |
| Nurses must call physicians with alert values | Not attempted | |
| Hospital‐acquired MRSA | Improve availability of contact precaution gowns | Attempted, but unsuccessful |
| Direct observation of hand washing on morning rounds | Attempted and successful | |
| Staff safety with deranged family member | Education of staff regarding safety protocols | Attempted, but unsuccessful |
| Transfer of unstable patient from outside hospital ICU to general medicine floor | Standardization of OSH transfer guidelines | Not attempted |
| Improved documentation of transferring MD contract data | Attempted and successful | |
| Consult called, patient not seen by attending | Education of faculty on existing institutional consult policy | Attempted, but unsuccessful |
| Clarification of violations reporting process for hospital consults | Attempted, but unsuccessful | |
| Type of Intervention | No. of Interventions Suggested | No. of Interventions Implemented (%) | Of Implemented Interventions, No. Which Were Successful (%) | No. of Attempted Interventions With Objective Change (%) | No. of Attempted Interventions With Subjective Change (%) |
|---|---|---|---|---|---|
| Design changes: information technology‐based | 5 | 2 (40) | 2 (100%) | 2 (100) | 0 (0) |
| Design changes: protocol‐based | 17 | 10 (59) | 8 (80%) | 5 (50) | 3 (30) |
| Educational | 20 | 11 (55) | 7 (64) | 1 (9) | 6 (55) |
| Reminder systems | 4 | 2 (50) | 1 (50) | 0 (0) | 1 (50) |
| Total | 46 | 25 (54) | 18 (72) | 8 (32) | 10 (40) |
Two IT‐based system design changes were implemented; both resulted in objective system‐wide change. Eight protocol‐based design changes were implemented successfully, 5 objectively, and 3 subjectively. Seven educational interventions and 1 reminder system intervention were initiated.
The most successful intervention to come from these conferences was the implementation of an electronic medication reconciliation program. The reconciliation program was suggested following a conference on a patient who was discharged home on the wrong dose of a medication. The institution's paper‐based medication reconciliation process, particularly for heart‐failure patients, had long been known to be deficient. The QIC brought this issue to life by highlighting a cases that may have been ameliorated with a more robust medication reconciliation process. Enthusiastic residents were invited to build a case for medication reconciliation to the Chief Medical Officer, and this helped garner resources for the hospital‐wide project. Another successful IT‐based intervention was initiated after a case of inappropriate administration of warfarin to a patient with an already elevated international normalized ratio (INR). The computerized order entry system was changed so that, at the point of ordering warfarin, the most recent coagulation profile and platelet values appear before an order can be finalized.
An example of a protocol‐based intervention came from a conference that focused on poor communication at the time of discharge, which resulted in a 30‐day readmission. As a result, resident work flow was changed so that discharge summaries are expected to be completed at the time of discharge. Along with this protocol change was an educational initiative to improve the quality of discharge summaries by including essential data for the transition of care.
Overall, residents reviewed the conferences very positively (Table 3). The response rate for the first year survey was 40% (56/140) and the second year survey was 18% (26/143). The vast majority of participants felt that the conferences were of high quality (96%) and that the exercise could lead to improvement in quality (98%). Residents felt that the conference focused more on system issues than individual shortcomings (92%). A majority felt comfortable expressing their opinions during the conferences (77%).
| Overall Conference Quality | ||
|---|---|---|
| Question | Mean Score (n = 53) | Rating Question a 4 or 5 |
| Conference Tone | ||
| Question | Mean Score (n = 26) | Rating Question With a 4 or 5 |
| ||
| Please rate the overall quality of the QIC conferences. | 4.49* | 98% |
| The case highlighted an issue that is highly relevant to the quality of patient care. | 4.81 | 100% |
| Solutions discussed at this conference could lead to improved patient care and/or patient satisfaction. | 4.65 | 96% |
| My knowledge of issues related to hospital quality and patient safety has been enhanced by this conference. | 4.61 | 96% |
| The QIC focused on individuals, individual actions, or omissions, which compromised high quality care. | 3.35 | 50% |
| The QIC focused on system failures that compromised high quality care. | 4.35 | 92% |
| I felt comfortable sharing my honest opinions about the medical events presented during the conferences. | 4.15 | 77% |
| I avoided expressing my opinions about the medical events presented during the conferences because I did not want to criticize my peers. | 2.5 | 19% |
DISCUSSION
The first 20 sessions from this resident‐led, hospitalist‐facilitated QI program provided evidence that residents can contribute to patient safety within a large tertiary care center. The role of residents in actively addressing errors and unsatisfactory outcomes in the hospital has not been a traditional QI focus.12 Involvement has typically been a passive process for physician trainees, while more senior clinical staff members decide on and prioritize QI activities. We have observed that empowering residents to take a more active role in performance improvement yields significant change and does more than simply educate about basic QI methodology.
One reason for the success of these conferences was leveraging insights of residents as key front line providers. Residents spend more time than perhaps any other category of hospital employee working within clinical care systems. They are deeply aware of the quality struggles inherent to large healthcare organizations, and this insight can lead to high impact suggestions for improvement. Often, suggestions were simple proposals that were overlooked or unappreciated by other administrative leaders. An example of this type of contribution was when residents brought the lack of infection control equipment, on certain units, to the attention of the infection control staff and facility engineers. At a separate conference, residents informed the transfer office staff that valuable contact information for physicians accepting outside hospital transfers was not being collected. Both of these observations led to quick change, with better infection control gown availability and improved documentation by transfer office staff.
Our program also demonstrated that including residents in QI provides momentum for either a training program or an institution to pursue solutions that might have otherwise been resisted. The improvement suggestion to complete discharge summaries prior to the patient leaving the hospital had long been a goal for the residency program leadership, but there was hesitation to force this work flow change on the residents. After a QI conference, when a number of the residents themselves made the suggestion, implementing the change was much easier. Similarly, after several cases of clear errors relating to a suboptimal process of medication reconciliation, the institution dedicated scarce IT personnel to work with providers to develop a robust, user‐friendly medication reconciliation application to decrease transition of care errors.
Through this program, residents also demonstrated their ability to deconstruct patient care problems. For each case, resident session leaders interviewed physician providers, physician extenders, nurses, nurse managers, pharmacists, security staff, engineering staff, and administrative staff. They gathered crucial information regarding the patient care event and the gaps or errors that led to a poor outcome. After many of the conferences, the resident presenters commented on how the investigative exercise left them more appreciative of the complexity of the medical system and interested in fixing the problems uncovered.
The feedback from the resident surveys demonstrated that residents valued the QI program. The data collected also shows that such programs can be executed in a manner which highlights system flaws. Our data do, however, suggest that there is room to improve the tone of the conference to further decrease the sense from residents that quality discussions focus on individuals. Residents often struggle to master the myriad new expectations inherent in the transition from student to physician.13 A quality process which discourages already overworked and uncertain trainees, by creating a process which assigns blame for unintentional quality shortcomings, would be counterproductive.
Lessons Learned
While this QI program has had success uncovering clinical care issues, and creating a climate and process for resident participation in improvement, there has been a number of limitations and lessons learned. Most importantly, including busy residents in any process that requires regular participation and follow‐through is difficult. A number of suggested improvements which created substantial interest and early momentum were ultimately left unfinished, as residents and even faculty facilitators became overwhelmed by clinical responsibilities. In fact, the majority of suggestions have not been successfully implemented and even fewer have created lasting change. This must be carefully monitored, as experiencing multiple failures can undermine the empowerment that such QI programs are created to foster.
Regular reflection on the successful and unsuccessful projects yielded several important insights that resulted in changes over the course of the program. Suggestions were more likely to move from idea generation to execution if the QIC was attended by administrators with decision‐making authority. Several of the suggestionsimproved medication reconciliation, better transfer documentation, and improved availability of infection control productswere able to be acted upon because conference attendees were administrators with purview over these issues. Many times, these leaders were more than willing to implement helpful suggestions, but simply needed them to be brought to their attention. As a result, we have been more attentive to inviting as many stakeholders as possible to the QICs.
It was also clear that suggestions would not be realized without a physician leader and were more successful when resident interest was substantial. After each QIC, residents who had made promising suggestions were approached to continue to participate. If the residents agreed, the projects were pursued and a faculty or chief resident leader was assigned. Lastly, we have also made use of one of the department's QI data analysts to assist with project completion. This individual has been made available to provide administrative support (organizing meetings, paperwork, etc) but also to provide data for projects, should the need arise.
Another important finding is that the tone of the QI program must be constantly monitored. Despite reminding residents at each session that the exercise was for the purpose of identifying systems barriers to delivering high quality care, there were times when residents felt targeted or blamed. At one point, a number of residents voiced their concerns that the conferences had spent too much time highlighting quality failures without recognizing the many positive performances on the teaching service. As a result, subsequent conferences often began by highlighting quality improvements made. Additionally, a part of 1 session each year had been dedicated to reading letters and e‐mails sent by patients or families which highlight memorably positive performances by the residents. Finally, care was taken to make sure invited guests to the sessions were reminded of the session's blame‐free ground rules.
Care must be taken when investigating clinical cases. On several occasions, attending physicians expressed discomfort with having residents scrutinize a clinical event. Although this process was protected under the QI umbrella and faculty names were never shared at the conferences, some faculty believed that this process was the purview of departmental or hospital QI staff, not untrained residents. Given the support provided for this program by the department chair and program director, as well as the professional nature with which the residents conducted their inquiries, there was little difficulty rejecting this line of objection. This feedback did lead supervisors to be more involved with the resident presenters, coaching them regarding data gathering and interviewing. If a case appears that it will be particularly sensitive, the hospitalist mentor or chief resident will reach out to involved residents and faculty to notify them that the case will be reviewed.
A final development secured, in part, as a result of this quality program has been more protected faculty time. At the start of this program, all faculty time was donated time on top of other administrative and patient care responsibilities. After the first 18 months of the QIC program, the residency program named an assistant program director for quality. At the time of writing this manuscript, the program further invested in quality by naming both an assistant and associate program director for quality. These positions combined amount to at least 0.4 full‐time equivalents (FTE). Of that, roughly 0.1 FTE is spent working on the QICs and subsequent project implementation.
Limitations
The evaluation of the success of the interventions potentially biased our findings. The qualitative method of using multiple reviewers, all of whom were invested in the program's outcomes, to gauge the success of initiated interventions may have resulted in an overestimate of the project's effectiveness. Furthermore, the category of subjective change lacks measurable criteria, making replication of the findings difficult.
The results presented here are from a single institution, conceived of and executed by a group of dedicated faculty. Moreover, both the chair of the department and the program director were very supportive of this endeavor. Possibly, because of these aspects, the findings presented here would not be readily replicated at another institution.
The percentage of residents who completed the feedback surveys was low. This may result in an overestimate of quality, value, and tone of the conferences, as well as potentially missing an opportunity for improving the program. We will address this issue through more rigorous quantitative and qualitative feedback at the end of the third year of the program.
CONCLUSIONS
Residents are willing and effective participants in a QI program. As front line providers, their experiences are valuable and their willingness to share insights can be an impetus for change. Finally, a process which includes modest investigation by third year residents, has faculty support and oversight, and provides minimal administrative support can overcome the difficulty of involving overworked residents in quality efforts.
Acknowledgements
The authors acknowledge Michael Pourdehnad for his role in developing the quality program.
To Err Is Human revealed the underappreciated tension between the enormous benefits of medical care and the potential for harm.1 Following this report, there has been an explosion of research and commentary detailing quality improvement (QI) opportunities. One area of growing emphasis has been resident physician training.2, 3 If medical care is dangerous, then a substantial contributor to the hazard must be the apprentice‐style process of physician training and the novice skill set of the trainees.4, 5 Many resident training programs have devised efforts to decrease the errors committed by physicians‐in‐training,6 change the culture of residency training,7 engage residents in quality improvement,8, 9 and improve resident training in quality improvement.10
Many of the programs devised to teach QI in the residency setting require substantial funding, a large pool of QI experts, or redesign of resident training programs.410 While effective, these programs are not feasible for many resource‐constrained residency programs. A less intense program, using resident‐led, hospitalist‐facilitated, limited root cause analysis (RCA), has been adopted at the Internal Medicine Residency Program at the Mount Sinai Hospital (MSH). We describe our 2‐year experience using this technique, including cases discussed, improvement strategies suggested, projects implemented, and resident perceptions.
METHODS
Setting
Departmental QI leaders developed this initiative in the Internal Medicine Residency Program at the MSH in New York City, New York. This residency program trains over 140 residents annually in categorical, preliminary, and research track positions, as well as an affiliated medicine/pediatrics program. The program's residents rotate at 3 clinic sites: a tertiary care hospital, a public safety‐net hospital, and a Veterans Affairs hospital. The QI program was only implemented at the MSH. Over 90% of the program's graduates go on to complete a subspecialty fellowship.
Intervention Description
The QI program was designed around a noon‐time quality improvement conference (QIC) occurring once every 4 weeks. In the weeks prior to the session, chief residents and a hospitalist mentor selected a case related to an inpatient care issue. Potential cases were solicited, and/or offered, from a range of sources including attending physicians, nurse managers, residents, and quality officers. Only cases from the teaching services were chosen. To ensure that participants on the case were able to recall relevant details, preference was given to more recent cases. A third‐year resident on an elective or outpatient block was chosen to investigate the case. To maximize the objectivity of the investigation, every effort was made to select a resident who was not involved in the care of the patient.
The resident was instructed to complete a limited RCA (fewer fact‐finding interviews and only 1 group meeting) and was directed to online resources.11 Each resident presenter worked closely with the chief residents and hospitalist mentor to identify appropriate strategies for collecting data and interviewing involved parties. If necessary, either due to volume of work or sensitivity of the case, the chief resident or hospitalist would assist with the data gathering. The resident contacted multiple parties involved in the patient care issue including, nurses, residents, attendings, pharmacists, social workers, and, if appropriate, the patient. The resident constructed a timeline for each case, and identified specific points in the patient care experience, where errors, near misses, or misunderstandings occurred. During the QIC, these findings were presented to Internal Medicine residents, chief residents, representatives from the Chief Medical Officer's office, attending physicians overseeing the residents on inpatient rotations, and representatives from any group (social work nursing, housekeeping, pharmacy, etc) that may have impacted patient care for the particular case being investigated. On average, 50 healthcare providers attended the QIC. Lunch was provided.
After the findings were presented, a chief resident and a lead hospitalist facilitated a group discussion on the circumstances surrounding the case. Discussions were focused on identifying system‐wide failures and proposing systems‐based solutions. Great efforts were made to remind all participants to refrain from individual blame. At the end of each QIC, participants summarized and prioritized suggestions to reduce the discussed error. Interested residents were invited to form improvement committees for cases with viable solutions. Each committee attempted to implement improvements discussed during the QIC. Committees, led by a representative from the Division of Hospital Medicine, included 2 to 4 residents as well as healthcare workers from other disciplines if appropriate. For all improvement efforts, the focus was on the interventions which appeared high yield with low cost.
Intervention Evaluation
The program was exempt from Institutional Review Board review as a part of the Department of Medicine's quality improvement and assurance portfolio.
The results of the QICs were tracked. After each case, a QI team consisting of chief residents and representatives from the Division of Hospital Medicine recorded the cases presented, and interventions suggested for each case, in an online database. After implementation, the success of each intervention was recorded. To evaluate the types of interventions suggested by residents, the 3 physician‐authors, who regularly attend these conferences and have a focused career interest in QI, grouped all suggestions into 4 broad categories: Educational, Reminder Systems, Design Changes (protocol‐based), and Design Changes (Information Technology [IT]‐based). Design change interventions (IT‐based) consisted of an adjustment to electronic systems, such as displaying specific lab results on a medication ordering system. Design changes (protocol‐based) consisted of changes made to standing protocols such as nursing protocols for reporting abnormal lab values. Reminder system interventions were endeavors such as a checklist for discharge planning. Educational interventions focused on providing additional training sessions or conferences.
The 3 physician‐authors independently reviewed each suggested intervention to determine its success. They first evaluated whether the change was attempted or not. For all attempted interventions, the reviewer then assessed if there was either objective permanent system‐wide change, subjective behavior change, or no change. To meet the objective change threshold, the intervention either had to have permanently changed provider workflow or have data demonstrating behavior change or improved outcome. Interventions with anecdotal evidence that behavior was improved or modified, but lacking systematic data, were qualified as subjective behavior change. For each assessment, 2 of the 3 reviewers needed to agree for an intervention to be recorded as a success.
Resident views on the monthly conferences were solicited via an anonymous and voluntary questionnaire. A first survey was designed to assess whether residents felt that the conferences provided them with the ability to recognize and improve systems errors which compromise patient care. This survey was administered at the conclusion of the first year of the program to residents who attended the final 2 QICs. A second survey assessed whether the tone of the conferences was constructive and blame‐free. This survey was administered at the conclusion of the second year of the program to residents who attended the year's final 2 QICs.
RESULTS
Over the first 22 months of the program, 20 conferences were held (Table 1). The topics covered ranged considerably and included: deficits in supervision, medication errors, patient satisfaction, staff safety, and 30‐day readmissions. Forty‐six distinct interventions were suggested during these conferences. Of those, an attempt was made to initiate 25 (54%) of these suggestions (Table 2). Of the 25 interventions that were initiated, 18 (72%) were determined to be successful. Eight resulted in objective permanent system‐wide change and 10 resulted in subjective behavior change among residents.
| QIC Topic | Interventions Suggested by Residents | Suggestion Results (Attempted/Not Attempted, Successful/Unsuccessful) |
|---|---|---|
| ||
| Central venous catheter guide wire lost during code placement | Improved supervision and training for line placement | Attempted, but unsuccessful |
| Avoid unnecessary line placement during codes | Attempted, but unsuccessful | |
| Inappropriate administration of warfarin | Decision support providing real‐time coagulation profile | Attempted and successful |
| Central line bloodstream infection | Clarified and encouraged use of line service | Attempted and successful |
| Daily documentation of catheter placement date | Not attempted | |
| Delayed administration of pain medication | Training nurses to use text paging communication system | Attempted and successful |
| Patient discharged on wrong medication dose | Do not use abbreviations | Not attempted |
| Electronic medication reconciliation | Attempted and successful | |
| Confusion over code status | Clarification of various forms used for DNR | Not attempted |
| Better communication of code status during signout | Not attempted | |
| Patient received hydromorphone IV instead of PO during verbal order at end‐of‐life | Verbal orders should have talk back verification | Attempted, but unsuccessful |
| Encourage informing patients of medical errors | Attempted, but unsuccessful | |
| Premature closure of diagnosis during transfer from MICU | Improve comfort level disagreeing with supervisors | Attempted, but unsuccessful |
| Reassessment of patient prior to late‐day MICU transfers | Not attempted | |
| Patient erroneously received clopidogrel bisulfate (Plavix) for years due to poor medication reconciliation | Improved discharge summary interface | Attempted and successful |
| Encourage physicians to call PCP on discharge | Attempted and successful | |
| Modified barium swallow ordered incorrectly, resulting in patient aspiration | Simplify electronic order entry system to clearly identify tests | Not attempted |
| Change radiology requisition form to facilitate communication | Not attempted | |
| Fingersticks leading to blood exposure | Train PGY1s on the needles used at all 3 hospitals | Not attempted |
| Improve mask with face shields and gown availability | Attempted and successful | |
| Patient discharged with central venous catheter still in place | Check list for lines and Foleys | Not attempted |
| Improved discharge documentation | Not attempted | |
| 30‐Day readmission | Mandatory discharge summary completion prior to discharge | Attempted and successful |
| Discharge summary training during intern year | Attempted and successful | |
| DKA developed in house when insulin not administered | Improve communication between floor and dialysis RNs | Not attempted |
| Better PA supervision by residents regarding order writing | Attempted and successful | |
| Compromised patient satisfaction | Patient handouts with name and role of each care team member | Attempted, but unsuccessful |
| Patient satisfaction coaching | Attempted and successful | |
| Elevated PTT and poor documentation | Improved feedback to residents regarding daily notes | Not attempted |
| Nurses must call physicians with alert values | Not attempted | |
| Hospital‐acquired MRSA | Improve availability of contact precaution gowns | Attempted, but unsuccessful |
| Direct observation of hand washing on morning rounds | Attempted and successful | |
| Staff safety with deranged family member | Education of staff regarding safety protocols | Attempted, but unsuccessful |
| Transfer of unstable patient from outside hospital ICU to general medicine floor | Standardization of OSH transfer guidelines | Not attempted |
| Improved documentation of transferring MD contract data | Attempted and successful | |
| Consult called, patient not seen by attending | Education of faculty on existing institutional consult policy | Attempted, but unsuccessful |
| Clarification of violations reporting process for hospital consults | Attempted, but unsuccessful | |
| Type of Intervention | No. of Interventions Suggested | No. of Interventions Implemented (%) | Of Implemented Interventions, No. Which Were Successful (%) | No. of Attempted Interventions With Objective Change (%) | No. of Attempted Interventions With Subjective Change (%) |
|---|---|---|---|---|---|
| Design changes: information technology‐based | 5 | 2 (40) | 2 (100%) | 2 (100) | 0 (0) |
| Design changes: protocol‐based | 17 | 10 (59) | 8 (80%) | 5 (50) | 3 (30) |
| Educational | 20 | 11 (55) | 7 (64) | 1 (9) | 6 (55) |
| Reminder systems | 4 | 2 (50) | 1 (50) | 0 (0) | 1 (50) |
| Total | 46 | 25 (54) | 18 (72) | 8 (32) | 10 (40) |
Two IT‐based system design changes were implemented; both resulted in objective system‐wide change. Eight protocol‐based design changes were implemented successfully, 5 objectively, and 3 subjectively. Seven educational interventions and 1 reminder system intervention were initiated.
The most successful intervention to come from these conferences was the implementation of an electronic medication reconciliation program. The reconciliation program was suggested following a conference on a patient who was discharged home on the wrong dose of a medication. The institution's paper‐based medication reconciliation process, particularly for heart‐failure patients, had long been known to be deficient. The QIC brought this issue to life by highlighting a cases that may have been ameliorated with a more robust medication reconciliation process. Enthusiastic residents were invited to build a case for medication reconciliation to the Chief Medical Officer, and this helped garner resources for the hospital‐wide project. Another successful IT‐based intervention was initiated after a case of inappropriate administration of warfarin to a patient with an already elevated international normalized ratio (INR). The computerized order entry system was changed so that, at the point of ordering warfarin, the most recent coagulation profile and platelet values appear before an order can be finalized.
An example of a protocol‐based intervention came from a conference that focused on poor communication at the time of discharge, which resulted in a 30‐day readmission. As a result, resident work flow was changed so that discharge summaries are expected to be completed at the time of discharge. Along with this protocol change was an educational initiative to improve the quality of discharge summaries by including essential data for the transition of care.
Overall, residents reviewed the conferences very positively (Table 3). The response rate for the first year survey was 40% (56/140) and the second year survey was 18% (26/143). The vast majority of participants felt that the conferences were of high quality (96%) and that the exercise could lead to improvement in quality (98%). Residents felt that the conference focused more on system issues than individual shortcomings (92%). A majority felt comfortable expressing their opinions during the conferences (77%).
| Overall Conference Quality | ||
|---|---|---|
| Question | Mean Score (n = 53) | Rating Question a 4 or 5 |
| Conference Tone | ||
| Question | Mean Score (n = 26) | Rating Question With a 4 or 5 |
| ||
| Please rate the overall quality of the QIC conferences. | 4.49* | 98% |
| The case highlighted an issue that is highly relevant to the quality of patient care. | 4.81 | 100% |
| Solutions discussed at this conference could lead to improved patient care and/or patient satisfaction. | 4.65 | 96% |
| My knowledge of issues related to hospital quality and patient safety has been enhanced by this conference. | 4.61 | 96% |
| The QIC focused on individuals, individual actions, or omissions, which compromised high quality care. | 3.35 | 50% |
| The QIC focused on system failures that compromised high quality care. | 4.35 | 92% |
| I felt comfortable sharing my honest opinions about the medical events presented during the conferences. | 4.15 | 77% |
| I avoided expressing my opinions about the medical events presented during the conferences because I did not want to criticize my peers. | 2.5 | 19% |
DISCUSSION
The first 20 sessions from this resident‐led, hospitalist‐facilitated QI program provided evidence that residents can contribute to patient safety within a large tertiary care center. The role of residents in actively addressing errors and unsatisfactory outcomes in the hospital has not been a traditional QI focus.12 Involvement has typically been a passive process for physician trainees, while more senior clinical staff members decide on and prioritize QI activities. We have observed that empowering residents to take a more active role in performance improvement yields significant change and does more than simply educate about basic QI methodology.
One reason for the success of these conferences was leveraging insights of residents as key front line providers. Residents spend more time than perhaps any other category of hospital employee working within clinical care systems. They are deeply aware of the quality struggles inherent to large healthcare organizations, and this insight can lead to high impact suggestions for improvement. Often, suggestions were simple proposals that were overlooked or unappreciated by other administrative leaders. An example of this type of contribution was when residents brought the lack of infection control equipment, on certain units, to the attention of the infection control staff and facility engineers. At a separate conference, residents informed the transfer office staff that valuable contact information for physicians accepting outside hospital transfers was not being collected. Both of these observations led to quick change, with better infection control gown availability and improved documentation by transfer office staff.
Our program also demonstrated that including residents in QI provides momentum for either a training program or an institution to pursue solutions that might have otherwise been resisted. The improvement suggestion to complete discharge summaries prior to the patient leaving the hospital had long been a goal for the residency program leadership, but there was hesitation to force this work flow change on the residents. After a QI conference, when a number of the residents themselves made the suggestion, implementing the change was much easier. Similarly, after several cases of clear errors relating to a suboptimal process of medication reconciliation, the institution dedicated scarce IT personnel to work with providers to develop a robust, user‐friendly medication reconciliation application to decrease transition of care errors.
Through this program, residents also demonstrated their ability to deconstruct patient care problems. For each case, resident session leaders interviewed physician providers, physician extenders, nurses, nurse managers, pharmacists, security staff, engineering staff, and administrative staff. They gathered crucial information regarding the patient care event and the gaps or errors that led to a poor outcome. After many of the conferences, the resident presenters commented on how the investigative exercise left them more appreciative of the complexity of the medical system and interested in fixing the problems uncovered.
The feedback from the resident surveys demonstrated that residents valued the QI program. The data collected also shows that such programs can be executed in a manner which highlights system flaws. Our data do, however, suggest that there is room to improve the tone of the conference to further decrease the sense from residents that quality discussions focus on individuals. Residents often struggle to master the myriad new expectations inherent in the transition from student to physician.13 A quality process which discourages already overworked and uncertain trainees, by creating a process which assigns blame for unintentional quality shortcomings, would be counterproductive.
Lessons Learned
While this QI program has had success uncovering clinical care issues, and creating a climate and process for resident participation in improvement, there has been a number of limitations and lessons learned. Most importantly, including busy residents in any process that requires regular participation and follow‐through is difficult. A number of suggested improvements which created substantial interest and early momentum were ultimately left unfinished, as residents and even faculty facilitators became overwhelmed by clinical responsibilities. In fact, the majority of suggestions have not been successfully implemented and even fewer have created lasting change. This must be carefully monitored, as experiencing multiple failures can undermine the empowerment that such QI programs are created to foster.
Regular reflection on the successful and unsuccessful projects yielded several important insights that resulted in changes over the course of the program. Suggestions were more likely to move from idea generation to execution if the QIC was attended by administrators with decision‐making authority. Several of the suggestionsimproved medication reconciliation, better transfer documentation, and improved availability of infection control productswere able to be acted upon because conference attendees were administrators with purview over these issues. Many times, these leaders were more than willing to implement helpful suggestions, but simply needed them to be brought to their attention. As a result, we have been more attentive to inviting as many stakeholders as possible to the QICs.
It was also clear that suggestions would not be realized without a physician leader and were more successful when resident interest was substantial. After each QIC, residents who had made promising suggestions were approached to continue to participate. If the residents agreed, the projects were pursued and a faculty or chief resident leader was assigned. Lastly, we have also made use of one of the department's QI data analysts to assist with project completion. This individual has been made available to provide administrative support (organizing meetings, paperwork, etc) but also to provide data for projects, should the need arise.
Another important finding is that the tone of the QI program must be constantly monitored. Despite reminding residents at each session that the exercise was for the purpose of identifying systems barriers to delivering high quality care, there were times when residents felt targeted or blamed. At one point, a number of residents voiced their concerns that the conferences had spent too much time highlighting quality failures without recognizing the many positive performances on the teaching service. As a result, subsequent conferences often began by highlighting quality improvements made. Additionally, a part of 1 session each year had been dedicated to reading letters and e‐mails sent by patients or families which highlight memorably positive performances by the residents. Finally, care was taken to make sure invited guests to the sessions were reminded of the session's blame‐free ground rules.
Care must be taken when investigating clinical cases. On several occasions, attending physicians expressed discomfort with having residents scrutinize a clinical event. Although this process was protected under the QI umbrella and faculty names were never shared at the conferences, some faculty believed that this process was the purview of departmental or hospital QI staff, not untrained residents. Given the support provided for this program by the department chair and program director, as well as the professional nature with which the residents conducted their inquiries, there was little difficulty rejecting this line of objection. This feedback did lead supervisors to be more involved with the resident presenters, coaching them regarding data gathering and interviewing. If a case appears that it will be particularly sensitive, the hospitalist mentor or chief resident will reach out to involved residents and faculty to notify them that the case will be reviewed.
A final development secured, in part, as a result of this quality program has been more protected faculty time. At the start of this program, all faculty time was donated time on top of other administrative and patient care responsibilities. After the first 18 months of the QIC program, the residency program named an assistant program director for quality. At the time of writing this manuscript, the program further invested in quality by naming both an assistant and associate program director for quality. These positions combined amount to at least 0.4 full‐time equivalents (FTE). Of that, roughly 0.1 FTE is spent working on the QICs and subsequent project implementation.
Limitations
The evaluation of the success of the interventions potentially biased our findings. The qualitative method of using multiple reviewers, all of whom were invested in the program's outcomes, to gauge the success of initiated interventions may have resulted in an overestimate of the project's effectiveness. Furthermore, the category of subjective change lacks measurable criteria, making replication of the findings difficult.
The results presented here are from a single institution, conceived of and executed by a group of dedicated faculty. Moreover, both the chair of the department and the program director were very supportive of this endeavor. Possibly, because of these aspects, the findings presented here would not be readily replicated at another institution.
The percentage of residents who completed the feedback surveys was low. This may result in an overestimate of quality, value, and tone of the conferences, as well as potentially missing an opportunity for improving the program. We will address this issue through more rigorous quantitative and qualitative feedback at the end of the third year of the program.
CONCLUSIONS
Residents are willing and effective participants in a QI program. As front line providers, their experiences are valuable and their willingness to share insights can be an impetus for change. Finally, a process which includes modest investigation by third year residents, has faculty support and oversight, and provides minimal administrative support can overcome the difficulty of involving overworked residents in quality efforts.
Acknowledgements
The authors acknowledge Michael Pourdehnad for his role in developing the quality program.
- ,,.To Err Is Human: Building a Safer Health System.Washington, DC:National Academy Press;1999.
- ,,,,.Redesigning residency education in internal medicine: a position paper from the Association of Program Directors in Internal Medicine.Ann Intern Med.2006;144:920–926.
- Accreditation Council for Graduate Medical Education. Program directors guide to the common program requirements. Available at: http://www.acgme.org/acWebsite/navPages/commonpr_documents/ CompleteGuide_v2%20.pdf. Accessed May 5,2010.
- ,,,.Medical errors involving trainees: a study of closed malpractice claims from 5 insurers.Arch Intern Med.2007;167:2030–2036.
- ,,,,,.Residents report on adverse events and their causes.Arch Intern Med.2005;165:2607–2613.
- ,.A system of analyzing medical errors to improve GMA curricula and programs.Acad Med.2001;76:125–133.
- ,,, et al.Changing conversations: teaching safety and quality in residency training.Acad Med.2008;83(11):1080–1087.
- ,,.Practice‐based learning and improvement: a curriculum in continuous quality improvement for surgery residents.Arch Surg.2007;142:479–483.
- .Involving residents in quality improvement: contrasting “top‐down” and “bottom‐up” approaches. Accreditation Council for Graduate Medical Education and Institute for Healthcare Improvement‐day project.ACGME Bulletin. August2008.
- ,,,,.Creating a quality improvement elective for medical house officers.Gen Intern Med.2004;19(8):861–867.
- National Center for Patient Safety. United States Department for Veteran Affairs. Root cause analysis tools. Available at: http://www.patientsafety.gov/CogAids/RCA/. Accessed August 17,2010.
- ,,, et al.Residents' engagement in quality improvement: a systematic review of the literature.Acad Med.2009;84:1757–1764.
- ,,.Graduates from a traditional medical curriculum evaluate the effectiveness of their medical curriculum through interviews.BMC Med Educ.2009;9:64.
- ,,.To Err Is Human: Building a Safer Health System.Washington, DC:National Academy Press;1999.
- ,,,,.Redesigning residency education in internal medicine: a position paper from the Association of Program Directors in Internal Medicine.Ann Intern Med.2006;144:920–926.
- Accreditation Council for Graduate Medical Education. Program directors guide to the common program requirements. Available at: http://www.acgme.org/acWebsite/navPages/commonpr_documents/ CompleteGuide_v2%20.pdf. Accessed May 5,2010.
- ,,,.Medical errors involving trainees: a study of closed malpractice claims from 5 insurers.Arch Intern Med.2007;167:2030–2036.
- ,,,,,.Residents report on adverse events and their causes.Arch Intern Med.2005;165:2607–2613.
- ,.A system of analyzing medical errors to improve GMA curricula and programs.Acad Med.2001;76:125–133.
- ,,, et al.Changing conversations: teaching safety and quality in residency training.Acad Med.2008;83(11):1080–1087.
- ,,.Practice‐based learning and improvement: a curriculum in continuous quality improvement for surgery residents.Arch Surg.2007;142:479–483.
- .Involving residents in quality improvement: contrasting “top‐down” and “bottom‐up” approaches. Accreditation Council for Graduate Medical Education and Institute for Healthcare Improvement‐day project.ACGME Bulletin. August2008.
- ,,,,.Creating a quality improvement elective for medical house officers.Gen Intern Med.2004;19(8):861–867.
- National Center for Patient Safety. United States Department for Veteran Affairs. Root cause analysis tools. Available at: http://www.patientsafety.gov/CogAids/RCA/. Accessed August 17,2010.
- ,,, et al.Residents' engagement in quality improvement: a systematic review of the literature.Acad Med.2009;84:1757–1764.
- ,,.Graduates from a traditional medical curriculum evaluate the effectiveness of their medical curriculum through interviews.BMC Med Educ.2009;9:64.