Leigh Anne Bakel, MD

The Intervention Bundle

A multidisciplinary task force developed a set of Best Practice Guidelines for Writing Progress Notes in the EHR (see Supporting Information, Appendix 1, in the online version of this article). They were designed to promote cognitive review of data, reduce note clutter, promote synthesis of data, and discourage copy‐paste. For example, the guidelines recommended either the phrase, Vital signs from the last 24 hours have been reviewed and are pertinent for or a link that included minimum/maximum values rather than including multiple sets of data. We next developed a note template aligned with these guidelines (see Supporting Information, Appendix 2, in the online version of this article) using features and links that already existed within the EHR. Interns received classroom teaching about the best practices and instruction in use of the template.

Study Design

The study was a retrospective pre‐/postintervention. An audit tool designed to assess compliance with the guidelines was used to score 25 progress notes written by pediatric interns in August 2010 and August 2011 during the pre‐ and postintervention periods, respectively (see Supporting Information, Appendix 3, in the online version of this article).

Progress notes were eligible based on the following criteria: (1) written on any day subsequent to the admission date, (2) written by a pediatric intern, and (3) progress note from the previous day available for comparison. It was not required that 2 consecutive notes be written by the same resident. Eligible notes were identified using a computer‐generated report, reviewed by a study member to ensure eligibility, and assigned a number.

Notes were scored on a scale of 0 to 17, with each question having a range of possible scores from 0 to 2. Some questions related to inappropriate copy‐paste (questions 2, 9, 10) and a question related to discrete diagnostic language for abnormal labs (question 11) were weighted more heavily in the tool, as compliance with these components of the guideline was felt to be of greater importance. Several questions within the audit tool refer to clutter. We defined clutter as any additional data not endorsed by the guidelines or not explicitly stated as relevant to the patient's care for that day.

Raters were trained to score notes through practice sessions, during which they all scored the same note and compared findings. To rectify inter‐rater scoring discrepancies identified during these sessions, a reference manual was created to assist raters in scoring notes (see Supporting Information, Appendix 4, in the online version of this article). Each preintervention note was then systematically assigned to 2 raters, comprised of a physician and 3 staff from health information management. Each rater scored the note individually without discussion. The inter‐rater reliability was determined to be excellent, with kappa indices ranging from 88% to 100% for the 13 questions; each note in the postintervention period was therefore assigned to only 1 rater. Total and individual questions' scores were sent to the statistician for analysis.

Statistical Analysis

Inter‐rater reliability of the audit tool was evaluated by calculating the intraclass correlation (ICC) coefficient using a multilevel random intercept model to account for the rater effect.[18] The study was powered to detect an anticipated ICC of at least 0.75 at the 1‐sided 0.05 significance level, assuming a null hypothesis that the ICC is 0.4 or less. The total score was summarized in terms of means and standard deviation. Individual item responses were summarized using percentages and compared between the pre‐ and postintervention assessment using the Fisher exact test. The analysis of response patterns for individual item scores was considered exploratory. The Benjamini‐Hochberg false discovery rate method was utilized to control the false‐positive rate when comparing individual item scores.[19] All P values were 2‐sided and considered statistically significant at <0.05. Statistical analyses were conducted using SAS software version 9.2 (SAS Institute Inc., Cary, NC).

Principal Findings

Improvements in electronic note writing, particularly in reducing note clutter, were achieved after the implementation of a bundled intervention. Because the intervention is a bundle, we cannot definitively identify which component had the greatest impact. Given the improvements seen in some areas with very low baseline performance, we hypothesize that these are most attributable to the creation of a compliant note template that (1) guided authors in using data links that were less cluttered and (2) eliminated the use of unnecessary links (eg, pain scores and daily weights). The lack of similar improvements in reducing input/output and lab clutter may be due to the fact that even with changes to the template suggesting a more narrative approach to these components, residents still felt compelled to use data links. Because our EHR does not easily allow for the inclusion of individual data elements, such as specific drain output or hemoglobin as opposed to a complete blood count, residents continued to use links that included more data than necessary. Although not significant findings, there was an observed decline in the proportion of notes containing a physical exam not entirely copied from the previous day and containing an assessment that was entirely new. These findings may be attributable to having a small sample of authors, a few of whom in the postintervention period were particularly prone to using copy‐paste.

Relationship to Other Evidence

The observed decline in quality of provider documentation after implementation of the EHR has led to a robust discussion in the literature about what really constitutes a quality provider note.[7, 8, 9, 10, 20] The absence of a defined gold standard makes research in this area challenging. It is our observation that when physicians refer to a decline in quality documentation in the EHR, they are frequently referring to the fact that electronically generated notes are often unattractive, difficult to read, and seem to lack clinical narrative.

Several publications have attempted to define note quality. Payne et al. described physical characteristics of electronically generated notes that were deemed more attractive to a reader, including a large proportion of narrative free text.[15] Hanson performed a qualitative study to describe outpatient clinical notes from the perspective of multiple stakeholders, resulting in a description of the characteristics of a quality note.[21] This formed the basis for the QNOTE, a validated tool to measure the quality of outpatient notes.[22] Similar work has not been done to rigorously define quality for inpatient documentation. Stetson did develop an instrument, the Physician Documentation Quality Instrument (PDQI‐9) to assess inpatient notes across 9 attributes; however, the validation method relied on a gold standard of a general impression score of 7 physician leaders.[23, 24]

Although these tools aim to address overall note quality, an advantage provided by our audit tool is that it directly addresses the problems most attributable to documenting in an EHR, namely note clutter and copy‐paste. A second advantage is that clinicians and nonclinicians can score notes objectively. The QNOTE and PDQI‐9 still rely on subjective assessment and require that the evaluator be a clinician.

There has also been little published about how to achieve notes of high quality. In 2013, Shoolin et al. did publish a consensus statement from the Association of Medical Directors of Information Systems outlining some guidelines for inpatient EHR documentation.[25] Optimal strategies for implementing such guidelines, however, and the overall impact such an implementation would have on improving note writing has not previously been studied. This study, therefore, adds to the existing body of literature by providing an example of an intervention that may lead to improvements in note writing.

Limitations

Our study has several limitations. The sample size of notes and authors was small. The short duration of the study and the assessment of notes soon after the intervention prevented an assessment of whether improvements were sustained over time.

Unfortunately, we were not evaluating the same group of interns in the pre‐ and postintervention periods. Interns were chosen as subjects as there was an existing opportunity to do large group training during new intern orientation. Furthermore, we were concerned that more note‐writing experience alone would influence the outcome if we examined the same interns later in the year.

The audit tool was also a first attempt at measuring compliance with the guidelines. Determination of an optimal score/weight for each item requires further investigation as part of a larger scale validation study. In addition, the cognitive review and synthesis of data encouraged in our guideline were more difficult to measure using the audit tool, as they require some clinical knowledge about the patient and an assessment of the author's medical decision making. We do not assert, therefore, that compliance with the guidelines or a higher total score necessarily translates into overall note quality, as we recognize these limitations of the tool.

Future Directions

In conclusion, this report is a first effort to improve the quality of note writing in the EHR. Much more work is necessary, particularly in improving the clinical narrative and inappropriate copy‐paste. The examination of other interventions, such as the impact of structured feedback to the note author, whether by way of a validated scoring tool and/or narrative comments, is a logical next step for investigation.

The Intervention Bundle

A multidisciplinary task force developed a set of Best Practice Guidelines for Writing Progress Notes in the EHR (see Supporting Information, Appendix 1, in the online version of this article). They were designed to promote cognitive review of data, reduce note clutter, promote synthesis of data, and discourage copy‐paste. For example, the guidelines recommended either the phrase, Vital signs from the last 24 hours have been reviewed and are pertinent for or a link that included minimum/maximum values rather than including multiple sets of data. We next developed a note template aligned with these guidelines (see Supporting Information, Appendix 2, in the online version of this article) using features and links that already existed within the EHR. Interns received classroom teaching about the best practices and instruction in use of the template.

Study Design

The study was a retrospective pre‐/postintervention. An audit tool designed to assess compliance with the guidelines was used to score 25 progress notes written by pediatric interns in August 2010 and August 2011 during the pre‐ and postintervention periods, respectively (see Supporting Information, Appendix 3, in the online version of this article).

Progress notes were eligible based on the following criteria: (1) written on any day subsequent to the admission date, (2) written by a pediatric intern, and (3) progress note from the previous day available for comparison. It was not required that 2 consecutive notes be written by the same resident. Eligible notes were identified using a computer‐generated report, reviewed by a study member to ensure eligibility, and assigned a number.

Notes were scored on a scale of 0 to 17, with each question having a range of possible scores from 0 to 2. Some questions related to inappropriate copy‐paste (questions 2, 9, 10) and a question related to discrete diagnostic language for abnormal labs (question 11) were weighted more heavily in the tool, as compliance with these components of the guideline was felt to be of greater importance. Several questions within the audit tool refer to clutter. We defined clutter as any additional data not endorsed by the guidelines or not explicitly stated as relevant to the patient's care for that day.

Raters were trained to score notes through practice sessions, during which they all scored the same note and compared findings. To rectify inter‐rater scoring discrepancies identified during these sessions, a reference manual was created to assist raters in scoring notes (see Supporting Information, Appendix 4, in the online version of this article). Each preintervention note was then systematically assigned to 2 raters, comprised of a physician and 3 staff from health information management. Each rater scored the note individually without discussion. The inter‐rater reliability was determined to be excellent, with kappa indices ranging from 88% to 100% for the 13 questions; each note in the postintervention period was therefore assigned to only 1 rater. Total and individual questions' scores were sent to the statistician for analysis.

Statistical Analysis

Inter‐rater reliability of the audit tool was evaluated by calculating the intraclass correlation (ICC) coefficient using a multilevel random intercept model to account for the rater effect.[18] The study was powered to detect an anticipated ICC of at least 0.75 at the 1‐sided 0.05 significance level, assuming a null hypothesis that the ICC is 0.4 or less. The total score was summarized in terms of means and standard deviation. Individual item responses were summarized using percentages and compared between the pre‐ and postintervention assessment using the Fisher exact test. The analysis of response patterns for individual item scores was considered exploratory. The Benjamini‐Hochberg false discovery rate method was utilized to control the false‐positive rate when comparing individual item scores.[19] All P values were 2‐sided and considered statistically significant at <0.05. Statistical analyses were conducted using SAS software version 9.2 (SAS Institute Inc., Cary, NC).

Principal Findings

Improvements in electronic note writing, particularly in reducing note clutter, were achieved after the implementation of a bundled intervention. Because the intervention is a bundle, we cannot definitively identify which component had the greatest impact. Given the improvements seen in some areas with very low baseline performance, we hypothesize that these are most attributable to the creation of a compliant note template that (1) guided authors in using data links that were less cluttered and (2) eliminated the use of unnecessary links (eg, pain scores and daily weights). The lack of similar improvements in reducing input/output and lab clutter may be due to the fact that even with changes to the template suggesting a more narrative approach to these components, residents still felt compelled to use data links. Because our EHR does not easily allow for the inclusion of individual data elements, such as specific drain output or hemoglobin as opposed to a complete blood count, residents continued to use links that included more data than necessary. Although not significant findings, there was an observed decline in the proportion of notes containing a physical exam not entirely copied from the previous day and containing an assessment that was entirely new. These findings may be attributable to having a small sample of authors, a few of whom in the postintervention period were particularly prone to using copy‐paste.

Relationship to Other Evidence

The observed decline in quality of provider documentation after implementation of the EHR has led to a robust discussion in the literature about what really constitutes a quality provider note.[7, 8, 9, 10, 20] The absence of a defined gold standard makes research in this area challenging. It is our observation that when physicians refer to a decline in quality documentation in the EHR, they are frequently referring to the fact that electronically generated notes are often unattractive, difficult to read, and seem to lack clinical narrative.

Several publications have attempted to define note quality. Payne et al. described physical characteristics of electronically generated notes that were deemed more attractive to a reader, including a large proportion of narrative free text.[15] Hanson performed a qualitative study to describe outpatient clinical notes from the perspective of multiple stakeholders, resulting in a description of the characteristics of a quality note.[21] This formed the basis for the QNOTE, a validated tool to measure the quality of outpatient notes.[22] Similar work has not been done to rigorously define quality for inpatient documentation. Stetson did develop an instrument, the Physician Documentation Quality Instrument (PDQI‐9) to assess inpatient notes across 9 attributes; however, the validation method relied on a gold standard of a general impression score of 7 physician leaders.[23, 24]

Although these tools aim to address overall note quality, an advantage provided by our audit tool is that it directly addresses the problems most attributable to documenting in an EHR, namely note clutter and copy‐paste. A second advantage is that clinicians and nonclinicians can score notes objectively. The QNOTE and PDQI‐9 still rely on subjective assessment and require that the evaluator be a clinician.

There has also been little published about how to achieve notes of high quality. In 2013, Shoolin et al. did publish a consensus statement from the Association of Medical Directors of Information Systems outlining some guidelines for inpatient EHR documentation.[25] Optimal strategies for implementing such guidelines, however, and the overall impact such an implementation would have on improving note writing has not previously been studied. This study, therefore, adds to the existing body of literature by providing an example of an intervention that may lead to improvements in note writing.

Limitations

Our study has several limitations. The sample size of notes and authors was small. The short duration of the study and the assessment of notes soon after the intervention prevented an assessment of whether improvements were sustained over time.

Unfortunately, we were not evaluating the same group of interns in the pre‐ and postintervention periods. Interns were chosen as subjects as there was an existing opportunity to do large group training during new intern orientation. Furthermore, we were concerned that more note‐writing experience alone would influence the outcome if we examined the same interns later in the year.

The audit tool was also a first attempt at measuring compliance with the guidelines. Determination of an optimal score/weight for each item requires further investigation as part of a larger scale validation study. In addition, the cognitive review and synthesis of data encouraged in our guideline were more difficult to measure using the audit tool, as they require some clinical knowledge about the patient and an assessment of the author's medical decision making. We do not assert, therefore, that compliance with the guidelines or a higher total score necessarily translates into overall note quality, as we recognize these limitations of the tool.

Future Directions

In conclusion, this report is a first effort to improve the quality of note writing in the EHR. Much more work is necessary, particularly in improving the clinical narrative and inappropriate copy‐paste. The examination of other interventions, such as the impact of structured feedback to the note author, whether by way of a validated scoring tool and/or narrative comments, is a logical next step for investigation.

The Intervention Bundle

A multidisciplinary task force developed a set of Best Practice Guidelines for Writing Progress Notes in the EHR (see Supporting Information, Appendix 1, in the online version of this article). They were designed to promote cognitive review of data, reduce note clutter, promote synthesis of data, and discourage copy‐paste. For example, the guidelines recommended either the phrase, Vital signs from the last 24 hours have been reviewed and are pertinent for or a link that included minimum/maximum values rather than including multiple sets of data. We next developed a note template aligned with these guidelines (see Supporting Information, Appendix 2, in the online version of this article) using features and links that already existed within the EHR. Interns received classroom teaching about the best practices and instruction in use of the template.

Study Design

The study was a retrospective pre‐/postintervention. An audit tool designed to assess compliance with the guidelines was used to score 25 progress notes written by pediatric interns in August 2010 and August 2011 during the pre‐ and postintervention periods, respectively (see Supporting Information, Appendix 3, in the online version of this article).

Progress notes were eligible based on the following criteria: (1) written on any day subsequent to the admission date, (2) written by a pediatric intern, and (3) progress note from the previous day available for comparison. It was not required that 2 consecutive notes be written by the same resident. Eligible notes were identified using a computer‐generated report, reviewed by a study member to ensure eligibility, and assigned a number.

Notes were scored on a scale of 0 to 17, with each question having a range of possible scores from 0 to 2. Some questions related to inappropriate copy‐paste (questions 2, 9, 10) and a question related to discrete diagnostic language for abnormal labs (question 11) were weighted more heavily in the tool, as compliance with these components of the guideline was felt to be of greater importance. Several questions within the audit tool refer to clutter. We defined clutter as any additional data not endorsed by the guidelines or not explicitly stated as relevant to the patient's care for that day.

Raters were trained to score notes through practice sessions, during which they all scored the same note and compared findings. To rectify inter‐rater scoring discrepancies identified during these sessions, a reference manual was created to assist raters in scoring notes (see Supporting Information, Appendix 4, in the online version of this article). Each preintervention note was then systematically assigned to 2 raters, comprised of a physician and 3 staff from health information management. Each rater scored the note individually without discussion. The inter‐rater reliability was determined to be excellent, with kappa indices ranging from 88% to 100% for the 13 questions; each note in the postintervention period was therefore assigned to only 1 rater. Total and individual questions' scores were sent to the statistician for analysis.

Statistical Analysis

Inter‐rater reliability of the audit tool was evaluated by calculating the intraclass correlation (ICC) coefficient using a multilevel random intercept model to account for the rater effect.[18] The study was powered to detect an anticipated ICC of at least 0.75 at the 1‐sided 0.05 significance level, assuming a null hypothesis that the ICC is 0.4 or less. The total score was summarized in terms of means and standard deviation. Individual item responses were summarized using percentages and compared between the pre‐ and postintervention assessment using the Fisher exact test. The analysis of response patterns for individual item scores was considered exploratory. The Benjamini‐Hochberg false discovery rate method was utilized to control the false‐positive rate when comparing individual item scores.[19] All P values were 2‐sided and considered statistically significant at <0.05. Statistical analyses were conducted using SAS software version 9.2 (SAS Institute Inc., Cary, NC).

Principal Findings

Improvements in electronic note writing, particularly in reducing note clutter, were achieved after the implementation of a bundled intervention. Because the intervention is a bundle, we cannot definitively identify which component had the greatest impact. Given the improvements seen in some areas with very low baseline performance, we hypothesize that these are most attributable to the creation of a compliant note template that (1) guided authors in using data links that were less cluttered and (2) eliminated the use of unnecessary links (eg, pain scores and daily weights). The lack of similar improvements in reducing input/output and lab clutter may be due to the fact that even with changes to the template suggesting a more narrative approach to these components, residents still felt compelled to use data links. Because our EHR does not easily allow for the inclusion of individual data elements, such as specific drain output or hemoglobin as opposed to a complete blood count, residents continued to use links that included more data than necessary. Although not significant findings, there was an observed decline in the proportion of notes containing a physical exam not entirely copied from the previous day and containing an assessment that was entirely new. These findings may be attributable to having a small sample of authors, a few of whom in the postintervention period were particularly prone to using copy‐paste.

Relationship to Other Evidence

The observed decline in quality of provider documentation after implementation of the EHR has led to a robust discussion in the literature about what really constitutes a quality provider note.[7, 8, 9, 10, 20] The absence of a defined gold standard makes research in this area challenging. It is our observation that when physicians refer to a decline in quality documentation in the EHR, they are frequently referring to the fact that electronically generated notes are often unattractive, difficult to read, and seem to lack clinical narrative.

Several publications have attempted to define note quality. Payne et al. described physical characteristics of electronically generated notes that were deemed more attractive to a reader, including a large proportion of narrative free text.[15] Hanson performed a qualitative study to describe outpatient clinical notes from the perspective of multiple stakeholders, resulting in a description of the characteristics of a quality note.[21] This formed the basis for the QNOTE, a validated tool to measure the quality of outpatient notes.[22] Similar work has not been done to rigorously define quality for inpatient documentation. Stetson did develop an instrument, the Physician Documentation Quality Instrument (PDQI‐9) to assess inpatient notes across 9 attributes; however, the validation method relied on a gold standard of a general impression score of 7 physician leaders.[23, 24]

Although these tools aim to address overall note quality, an advantage provided by our audit tool is that it directly addresses the problems most attributable to documenting in an EHR, namely note clutter and copy‐paste. A second advantage is that clinicians and nonclinicians can score notes objectively. The QNOTE and PDQI‐9 still rely on subjective assessment and require that the evaluator be a clinician.

There has also been little published about how to achieve notes of high quality. In 2013, Shoolin et al. did publish a consensus statement from the Association of Medical Directors of Information Systems outlining some guidelines for inpatient EHR documentation.[25] Optimal strategies for implementing such guidelines, however, and the overall impact such an implementation would have on improving note writing has not previously been studied. This study, therefore, adds to the existing body of literature by providing an example of an intervention that may lead to improvements in note writing.

Limitations

Our study has several limitations. The sample size of notes and authors was small. The short duration of the study and the assessment of notes soon after the intervention prevented an assessment of whether improvements were sustained over time.

Unfortunately, we were not evaluating the same group of interns in the pre‐ and postintervention periods. Interns were chosen as subjects as there was an existing opportunity to do large group training during new intern orientation. Furthermore, we were concerned that more note‐writing experience alone would influence the outcome if we examined the same interns later in the year.

The audit tool was also a first attempt at measuring compliance with the guidelines. Determination of an optimal score/weight for each item requires further investigation as part of a larger scale validation study. In addition, the cognitive review and synthesis of data encouraged in our guideline were more difficult to measure using the audit tool, as they require some clinical knowledge about the patient and an assessment of the author's medical decision making. We do not assert, therefore, that compliance with the guidelines or a higher total score necessarily translates into overall note quality, as we recognize these limitations of the tool.

Future Directions

In conclusion, this report is a first effort to improve the quality of note writing in the EHR. Much more work is necessary, particularly in improving the clinical narrative and inappropriate copy‐paste. The examination of other interventions, such as the impact of structured feedback to the note author, whether by way of a validated scoring tool and/or narrative comments, is a logical next step for investigation.

Setting

Children's Hospital Colorado is a 413‐bed freestanding tertiary care teaching hospital with over 13,000 inpatient admissions annually and an average patient length of stay of 5.7 days. We were the first children's hospital to fully implement a single EHR (Epic Systems, Madison, WI) in 2006. This discharge improvement initiative emerged from our hospital's involvement in the Children's Hospital Association Discharge Collaborative between October 2011 and October 2012. We were 1 of 12 participating hospitals and developed several different projects within the framework of the initiative.

Improvement Team

Our multidisciplinary project team included hospitalist physicians, case managers, social workers, respiratory therapists, pharmacists, medical interpreters, process improvement specialists, clinical application specialists whose daily role is management of our hospital's EHR software, and resident liaisons whose daily role is working with residents to facilitate care coordination.

Ethics

The project was determined to be QI work by the Children's Hospital Colorado Organizational Research Risk and Quality Improvement Review Panel.

Understanding the Problem

To understand the perspectives of each discipline involved in discharge planning, the lead hospitalist physician and a process improvement specialist interviewed key representatives from each group. Key informant interviews were conducted with hospitalist physicians, case managers, nurses, social workers, resident liaisons, respiratory therapists, pharmacists, medical interpreters, and residents. We inquired about their informational needs, their methods for obtaining relevant information, and whether the information was currently documented in the EHR. We then used process mapping to learn each disciplines' workflow related to discharge planning. Finally, we gathered key stakeholders together for a group session where discharge planning was mapped using the example of a patient admitted with asthma. From this session, we created a detailed multidisciplinary swim lane process map, a flowchart displaying the sequence of events in the overall discharge process grouped visually by placing the events in lanes. Each lane represented a discipline involved in patient discharge, and the arrows between lanes showed how information is passed between the various disciplines. Using this diagram, the team was able to fully understand provider interdependence in discharge planning and longitudinal timing of discharge‐related tasks during the patient's hospitalization.

We learned that: (1) discharge planning is complex, and there were often multiple provider types involved in the discharge of a single patient; (2) communication and coordination between the multitude of providers was often suboptimal; and (3) many of the tasks related to discharge were left to the last minute, resulting in unnecessary delays. Underlying these problems was a clear lack of organized and visible discharge planning information within the EHR.

There were many examples of obscure and siloed discharge processes. Physicians were aware of discharge criteria, but did not document these criteria for others to see. Case management assessments of home health needs were conveyed verbally to other team members, creating the potential for omissions, mistakes, or delays in appropriate home health planning. Social workers helped families to navigate financial hurdles (eg, assistance with payments for prescription medications). However, the presence of financial or insurance problems was not readily apparent to front‐line clinicians making discharge decisions. Other factors with potential significance for discharge planning, such as English‐language proficiency or a family's geographic distance from the hospital, were buried in disparate flow sheets or reports and not available or apparent to all health team members. There were also clear examples of discharge‐related tasks occurring at the end of hospitalization that could easily have been completed earlier in the admission such as identifying a primary care provider (PCP), scheduling follow‐up appointments, and completing work/subhool excuses because of lack of care team awareness that these items were needed.

Planning the Intervention

Based on our learning, we developed a key driver diagram (Figure 1). Our aim was to create a DRR that organized important discharge‐related information into 1 easily accessible report. Key drivers that were identified as relevant to the content of the DRR included: barriers to discharge, discharge criteria, home care, postdischarge care, and last minute delays. We also identified secondary drivers related to the design of the DRR. We hypothesized that addressing the secondary drivers would be essential to end user adoption of the tool. The secondary drivers included: accessibility, relevance, ease of updating, automation, and readability.

Figure 1

With the swim lane diagram as well as our primary and secondary drivers in mind, we created a mock DRR on paper. We conducted multiple patient discharge simulations with representatives from all disciplines, walking through each step of a patient hospitalization from registration to discharge. This allowed us to map out how preexisting, yet disparate, EHR data could be channeled into 1 report. A few changes were made to processes involving data collection and documentation to facilitate timely transfer of information to the report. For example, questions addressing potential barriers to discharge and whether a school/work excuse was needed were added to the admission nursing assessment.

We then moved the paper DRR to the electronic environment. Data elements that were pulled automatically into the report included: potential barriers to discharge collected during nursing intake, case management information on home care needs, discharge criteria entered by resident and attending physicians, PCP, home pharmacy, follow‐up appointments, school/work excuse information gathered by resident liaisons, and active patient problems drawn from the problem list section. These data were organized into 4 distinct domains within the final DRR: potential barriers, transitional care, home care, and discharge criteria (Table 1).

Additional features potentially important to discharge planning were also incorporated into the report based on end user feedback. These included hyperlinks to discharge orders, home oxygen prescriptions, and the after‐visit summary for families, and the patient's home care company (if present). To facilitate discharge and transitional care related communication between the primary team and subspecialty teams, consults involved during the hospitalization were included on the report. As home care arrangements often involve care for active lines and drains, they were added to the report (Figure 2).

Figure 2

Implementation

The report was activated within the EHR in June 2012. The team focused initial promotion and education efforts on medical floors. Education was widely disseminated via email and in‐person presentations.

The DRR was incorporated into daily CCRs for medical patients in July 2012. These multidisciplinary rounds occurred after medical‐team bedside rounds, focusing on care coordination and discharge planning. For each patient discussed, the DRR was projected onto a large screen, allowing all team members to view and discuss relevant discharge information. A process improvement (PI) specialist attended CCRs daily for several months, educating participants and monitoring use of the DRR. The PI specialist solicited feedback on ways to improve the DRR, and timed rounds to measure whether use of the DRR prolonged CCRs.

In the first weeks postimplementation, the use of the DRR prolonged rounds by as much as 1 minute per patient. Based on direct observation, the team focused interventions on barriers to the efficient use of the report during CCRs including: the need to scroll through the report, which was not visible on 1 screen; the need to navigate between patients; the need to quickly update the report based on discussion; and the need to update discharge criteria (Figure 3).

Figure 3

Setting

Children's Hospital Colorado is a 413‐bed freestanding tertiary care teaching hospital with over 13,000 inpatient admissions annually and an average patient length of stay of 5.7 days. We were the first children's hospital to fully implement a single EHR (Epic Systems, Madison, WI) in 2006. This discharge improvement initiative emerged from our hospital's involvement in the Children's Hospital Association Discharge Collaborative between October 2011 and October 2012. We were 1 of 12 participating hospitals and developed several different projects within the framework of the initiative.

Improvement Team

Our multidisciplinary project team included hospitalist physicians, case managers, social workers, respiratory therapists, pharmacists, medical interpreters, process improvement specialists, clinical application specialists whose daily role is management of our hospital's EHR software, and resident liaisons whose daily role is working with residents to facilitate care coordination.

Ethics

The project was determined to be QI work by the Children's Hospital Colorado Organizational Research Risk and Quality Improvement Review Panel.

Understanding the Problem

To understand the perspectives of each discipline involved in discharge planning, the lead hospitalist physician and a process improvement specialist interviewed key representatives from each group. Key informant interviews were conducted with hospitalist physicians, case managers, nurses, social workers, resident liaisons, respiratory therapists, pharmacists, medical interpreters, and residents. We inquired about their informational needs, their methods for obtaining relevant information, and whether the information was currently documented in the EHR. We then used process mapping to learn each disciplines' workflow related to discharge planning. Finally, we gathered key stakeholders together for a group session where discharge planning was mapped using the example of a patient admitted with asthma. From this session, we created a detailed multidisciplinary swim lane process map, a flowchart displaying the sequence of events in the overall discharge process grouped visually by placing the events in lanes. Each lane represented a discipline involved in patient discharge, and the arrows between lanes showed how information is passed between the various disciplines. Using this diagram, the team was able to fully understand provider interdependence in discharge planning and longitudinal timing of discharge‐related tasks during the patient's hospitalization.

We learned that: (1) discharge planning is complex, and there were often multiple provider types involved in the discharge of a single patient; (2) communication and coordination between the multitude of providers was often suboptimal; and (3) many of the tasks related to discharge were left to the last minute, resulting in unnecessary delays. Underlying these problems was a clear lack of organized and visible discharge planning information within the EHR.

There were many examples of obscure and siloed discharge processes. Physicians were aware of discharge criteria, but did not document these criteria for others to see. Case management assessments of home health needs were conveyed verbally to other team members, creating the potential for omissions, mistakes, or delays in appropriate home health planning. Social workers helped families to navigate financial hurdles (eg, assistance with payments for prescription medications). However, the presence of financial or insurance problems was not readily apparent to front‐line clinicians making discharge decisions. Other factors with potential significance for discharge planning, such as English‐language proficiency or a family's geographic distance from the hospital, were buried in disparate flow sheets or reports and not available or apparent to all health team members. There were also clear examples of discharge‐related tasks occurring at the end of hospitalization that could easily have been completed earlier in the admission such as identifying a primary care provider (PCP), scheduling follow‐up appointments, and completing work/subhool excuses because of lack of care team awareness that these items were needed.

Planning the Intervention

Based on our learning, we developed a key driver diagram (Figure 1). Our aim was to create a DRR that organized important discharge‐related information into 1 easily accessible report. Key drivers that were identified as relevant to the content of the DRR included: barriers to discharge, discharge criteria, home care, postdischarge care, and last minute delays. We also identified secondary drivers related to the design of the DRR. We hypothesized that addressing the secondary drivers would be essential to end user adoption of the tool. The secondary drivers included: accessibility, relevance, ease of updating, automation, and readability.

Figure 1

With the swim lane diagram as well as our primary and secondary drivers in mind, we created a mock DRR on paper. We conducted multiple patient discharge simulations with representatives from all disciplines, walking through each step of a patient hospitalization from registration to discharge. This allowed us to map out how preexisting, yet disparate, EHR data could be channeled into 1 report. A few changes were made to processes involving data collection and documentation to facilitate timely transfer of information to the report. For example, questions addressing potential barriers to discharge and whether a school/work excuse was needed were added to the admission nursing assessment.

We then moved the paper DRR to the electronic environment. Data elements that were pulled automatically into the report included: potential barriers to discharge collected during nursing intake, case management information on home care needs, discharge criteria entered by resident and attending physicians, PCP, home pharmacy, follow‐up appointments, school/work excuse information gathered by resident liaisons, and active patient problems drawn from the problem list section. These data were organized into 4 distinct domains within the final DRR: potential barriers, transitional care, home care, and discharge criteria (Table 1).

Additional features potentially important to discharge planning were also incorporated into the report based on end user feedback. These included hyperlinks to discharge orders, home oxygen prescriptions, and the after‐visit summary for families, and the patient's home care company (if present). To facilitate discharge and transitional care related communication between the primary team and subspecialty teams, consults involved during the hospitalization were included on the report. As home care arrangements often involve care for active lines and drains, they were added to the report (Figure 2).

Figure 2

Implementation

The report was activated within the EHR in June 2012. The team focused initial promotion and education efforts on medical floors. Education was widely disseminated via email and in‐person presentations.

The DRR was incorporated into daily CCRs for medical patients in July 2012. These multidisciplinary rounds occurred after medical‐team bedside rounds, focusing on care coordination and discharge planning. For each patient discussed, the DRR was projected onto a large screen, allowing all team members to view and discuss relevant discharge information. A process improvement (PI) specialist attended CCRs daily for several months, educating participants and monitoring use of the DRR. The PI specialist solicited feedback on ways to improve the DRR, and timed rounds to measure whether use of the DRR prolonged CCRs.

In the first weeks postimplementation, the use of the DRR prolonged rounds by as much as 1 minute per patient. Based on direct observation, the team focused interventions on barriers to the efficient use of the report during CCRs including: the need to scroll through the report, which was not visible on 1 screen; the need to navigate between patients; the need to quickly update the report based on discussion; and the need to update discharge criteria (Figure 3).

Figure 3

Setting

Children's Hospital Colorado is a 413‐bed freestanding tertiary care teaching hospital with over 13,000 inpatient admissions annually and an average patient length of stay of 5.7 days. We were the first children's hospital to fully implement a single EHR (Epic Systems, Madison, WI) in 2006. This discharge improvement initiative emerged from our hospital's involvement in the Children's Hospital Association Discharge Collaborative between October 2011 and October 2012. We were 1 of 12 participating hospitals and developed several different projects within the framework of the initiative.

Improvement Team

Our multidisciplinary project team included hospitalist physicians, case managers, social workers, respiratory therapists, pharmacists, medical interpreters, process improvement specialists, clinical application specialists whose daily role is management of our hospital's EHR software, and resident liaisons whose daily role is working with residents to facilitate care coordination.

Ethics

The project was determined to be QI work by the Children's Hospital Colorado Organizational Research Risk and Quality Improvement Review Panel.

Understanding the Problem

To understand the perspectives of each discipline involved in discharge planning, the lead hospitalist physician and a process improvement specialist interviewed key representatives from each group. Key informant interviews were conducted with hospitalist physicians, case managers, nurses, social workers, resident liaisons, respiratory therapists, pharmacists, medical interpreters, and residents. We inquired about their informational needs, their methods for obtaining relevant information, and whether the information was currently documented in the EHR. We then used process mapping to learn each disciplines' workflow related to discharge planning. Finally, we gathered key stakeholders together for a group session where discharge planning was mapped using the example of a patient admitted with asthma. From this session, we created a detailed multidisciplinary swim lane process map, a flowchart displaying the sequence of events in the overall discharge process grouped visually by placing the events in lanes. Each lane represented a discipline involved in patient discharge, and the arrows between lanes showed how information is passed between the various disciplines. Using this diagram, the team was able to fully understand provider interdependence in discharge planning and longitudinal timing of discharge‐related tasks during the patient's hospitalization.

We learned that: (1) discharge planning is complex, and there were often multiple provider types involved in the discharge of a single patient; (2) communication and coordination between the multitude of providers was often suboptimal; and (3) many of the tasks related to discharge were left to the last minute, resulting in unnecessary delays. Underlying these problems was a clear lack of organized and visible discharge planning information within the EHR.

There were many examples of obscure and siloed discharge processes. Physicians were aware of discharge criteria, but did not document these criteria for others to see. Case management assessments of home health needs were conveyed verbally to other team members, creating the potential for omissions, mistakes, or delays in appropriate home health planning. Social workers helped families to navigate financial hurdles (eg, assistance with payments for prescription medications). However, the presence of financial or insurance problems was not readily apparent to front‐line clinicians making discharge decisions. Other factors with potential significance for discharge planning, such as English‐language proficiency or a family's geographic distance from the hospital, were buried in disparate flow sheets or reports and not available or apparent to all health team members. There were also clear examples of discharge‐related tasks occurring at the end of hospitalization that could easily have been completed earlier in the admission such as identifying a primary care provider (PCP), scheduling follow‐up appointments, and completing work/subhool excuses because of lack of care team awareness that these items were needed.

Planning the Intervention

Based on our learning, we developed a key driver diagram (Figure 1). Our aim was to create a DRR that organized important discharge‐related information into 1 easily accessible report. Key drivers that were identified as relevant to the content of the DRR included: barriers to discharge, discharge criteria, home care, postdischarge care, and last minute delays. We also identified secondary drivers related to the design of the DRR. We hypothesized that addressing the secondary drivers would be essential to end user adoption of the tool. The secondary drivers included: accessibility, relevance, ease of updating, automation, and readability.

Figure 1

With the swim lane diagram as well as our primary and secondary drivers in mind, we created a mock DRR on paper. We conducted multiple patient discharge simulations with representatives from all disciplines, walking through each step of a patient hospitalization from registration to discharge. This allowed us to map out how preexisting, yet disparate, EHR data could be channeled into 1 report. A few changes were made to processes involving data collection and documentation to facilitate timely transfer of information to the report. For example, questions addressing potential barriers to discharge and whether a school/work excuse was needed were added to the admission nursing assessment.

We then moved the paper DRR to the electronic environment. Data elements that were pulled automatically into the report included: potential barriers to discharge collected during nursing intake, case management information on home care needs, discharge criteria entered by resident and attending physicians, PCP, home pharmacy, follow‐up appointments, school/work excuse information gathered by resident liaisons, and active patient problems drawn from the problem list section. These data were organized into 4 distinct domains within the final DRR: potential barriers, transitional care, home care, and discharge criteria (Table 1).

Additional features potentially important to discharge planning were also incorporated into the report based on end user feedback. These included hyperlinks to discharge orders, home oxygen prescriptions, and the after‐visit summary for families, and the patient's home care company (if present). To facilitate discharge and transitional care related communication between the primary team and subspecialty teams, consults involved during the hospitalization were included on the report. As home care arrangements often involve care for active lines and drains, they were added to the report (Figure 2).

Figure 2

Implementation

The report was activated within the EHR in June 2012. The team focused initial promotion and education efforts on medical floors. Education was widely disseminated via email and in‐person presentations.

The DRR was incorporated into daily CCRs for medical patients in July 2012. These multidisciplinary rounds occurred after medical‐team bedside rounds, focusing on care coordination and discharge planning. For each patient discussed, the DRR was projected onto a large screen, allowing all team members to view and discuss relevant discharge information. A process improvement (PI) specialist attended CCRs daily for several months, educating participants and monitoring use of the DRR. The PI specialist solicited feedback on ways to improve the DRR, and timed rounds to measure whether use of the DRR prolonged CCRs.

In the first weeks postimplementation, the use of the DRR prolonged rounds by as much as 1 minute per patient. Based on direct observation, the team focused interventions on barriers to the efficient use of the report during CCRs including: the need to scroll through the report, which was not visible on 1 screen; the need to navigate between patients; the need to quickly update the report based on discussion; and the need to update discharge criteria (Figure 3).

Figure 3