Karen Jerardi, MD, MED

The American Board of Medical Specialties approved subspecialty designation for the field of pediatric hospital medicine (PHM) in 2016.¹ For those who started independent practice prior to July 2019, there were two options for board eligibility: the “practice pathway” or completion of a PHM fellowship. The practice pathway allows for pediatric and combined internal medicine–pediatric (med-peds) providers who graduated by July 2019 to sit for the PHM board-certification examination if they meet specific criteria in their pediatric practice.² For pediatric and med-peds residents who graduated after July 2019, PHM board eligibility is available only through completion of a PHM fellowship.

PHM subspecialty designation with fellowship training requirements may pose unique challenges to med-peds residents interested in practicing both pediatric and adult hospital medicine (HM).^3,4 Each year, an estimated 25% of med-peds residency graduates go on to practice HM.⁵ The majority (62%-83%) of currently practicing med-peds–trained hospitalists care for both adults and children.^5,6 Further, med-peds–trained hospitalists comprise at least 10% of the PHM workforce⁵ and play an important role in caring for adult survivors of childhood diseases.³

Limited existing data suggest that the future practice patterns of med-peds residents may be affected by PHM fellowship requirements. One previous survey study indicated that, although med-peds residents see value in additional training opportunities offered by fellowship, the majority are less likely to pursue PHM as a result of the new requirements.⁴ Prominent factors dissuading residents from pursuing PHM fellowship included forfeited earnings during fellowship, student loan obligations, family obligations, and the perception that training received during residency was sufficient. Although these data provide important insights into potential changes in practice patterns, they do not explore qualities of PHM fellowship that may make additional training more appealing to med-peds residents and promote retention of med-peds–trained providers in the PHM workforce.

Further, there is no existing literature exploring if and how PHM fellowship programs are equipped to support the needs of med-peds–trained fellows. Other subspecialties have supported med-peds trainees in combined fellowship training programs, including rheumatology, neurology, pediatric emergency medicine, allergy/immunology, physical medicine and rehabilitation, and psychiatry.^7,8 However, the extent to which PHM fellowships follow a similar model to accommodate the career goals of med-peds participants is unclear.

Given the large numbers of med-peds residents who go on to practice combined PHM and adult HM, it is crucial to understand the training needs of this group within the context of PHM fellowship and board certification. The primary objectives of this study were to understand (1) the perceived PHM fellowship needs of med-peds residents interested in HM, and (2) how the current PHM fellowship training environment can meet those needs. Understanding that additional training requirements to practice PHM may affect the career trajectory of residents interested in HM, secondary objectives included describing perceptions of med-peds residents on PHM specialty designation and whether designation affected their career plans.

Study Design

This cross-sectional study took place over a 3-month period from May to July 2019 and included two surveys of different populations to develop a comprehensive understanding of stakeholder perceptions of PHM fellowship. The first survey (resident survey) invited med-peds residents who were members of the National Med-Peds Residents’ Association (NMPRA)⁹ in 2019 and who were interested in HM. The second survey (fellowship director [FD] survey) included PHM FDs. The study was determined to be exempt by the University of Pittsburgh Institutional Review Board.

Study Population and Recruitment

Resident Survey

Two attempts were made to elicit participation via the NMPRA electronic mailing list. The NMPRA membership includes med-peds residents and chief residents from US med-peds residency programs. As of May 2019, 77 med-peds residency programs and their residents were members of NMPRA, which encompassed all med-peds programs in the United States and its territories. NMPRA maintains a listserv for all members, and all existing US/territory programs were members at the time of the survey. Med-peds interns, residents, and chief residents interested in HM were invited to participate in this study.

FD Survey

Forty-eight FDs, representing member institutions of the PHM Fellowship Directors’ Council, were surveyed via the PHM Fellowship Directors listserv.

Survey Instruments

We constructed two de novo surveys consisting of multiple-choice and short-answer questions (Appendix 1 and Appendix 2). To enhance the validity of survey responses, questions were designed and tested using an iterative consensus process among authors and additional participants, including current med-peds PHM fellows, PHM fellowship program directors, med-peds residency program directors, and current med-peds residents. These revisions were repeated for a total of four cycles. Items were created to increase knowledge on the following key areas: resident-perceived needs in fellowship training, impact of PHM subspecialty designation on career choices related to HM, health system structure of fellowship programs, and ability to accommodate med-peds clinical training within a PHM fellowship. A combined med-peds fellowship, as defined in the survey and referenced in this study, is a “combined internal medicine–pediatrics hospital medicine fellowship whereby you would remain eligible for PHM board certification.” To ensure a broad and inclusive view of potential needs of med-peds trainees considering fellowship, all respondents were asked to complete questions pertaining to anticipated fellowship needs regardless of their indicated interest in fellowship.

Data Collection

Survey completion was voluntary. Email identifiers were not linked to completed surveys. Study data were collected and managed by using Qualtrics XM. Only completed survey entries were included in analysis.

Statistical Methods and Data Analysis

R software version 4.0.2 (R Foundation for Statistical Computing) was used for statistical analysis. Demographic data were summarized using frequency distributions. The intent of the free-text questions for both surveys was qualitative explanatory thematic analysis. Authors EB, HL, and AJ used a deductive approach to identify common themes that elucidated med-peds resident–anticipated needs in fellowship and PHM program strategies and barriers to accommodate these needs. Preliminary themes and action items were reviewed and discussed among the full authorship team until consensus was reached.

Demographic Data

Resident Survey

A total of 466 med-peds residents completed the resident survey. There are approximately 1300 med-peds residents annually, creating an estimated response rate of 35.8% of all US med-peds residents. The majority (n = 380, 81.5%) of respondents were med-peds postgraduate years 1 through 3 and thus only eligible for PHM board certification via the PHM fellowship pathway (Table 1). Most (n = 446, 95.7%) respondents had considered a career in adult, pediatric, or combined HM at some point. Of those med-peds residents who considered a career in HM (Appendix Table 1), 92.8% (n = 414) would prefer to practice combined adult HM and PHM.

FD Survey

Twenty-eight FDs completed the FD survey, representing 58.3% of 2019 PHM fellowship programs. Of the responding programs, 23 (82.1%) were associated with a freestanding children’s hospital, and 24 (85.7%) were integrated or affiliated with a health system that provides adult inpatient care (Table 2). Sixteen (57.1%) programs had a med-peds residency program at their institution.

Med-Peds Resident Perceptions of PHM Fellowship

In considering the importance of PHM board certification for physicians practicing PHM, 59.0% (n= 275) of respondents rated board certification as “not at all important” (Appendix Table 2). Most (n = 420, 90.1%) med-peds trainees responded that PHM subspecialty designation “decreased” or “significantly decreased” their desire to pursue a career that includes PHM. Of the respondents who reported no interest in hospital medicine, eight (40%) reported that PHM subspecialty status dissuaded them from a career in HM at least a moderate amount (Appendix Table 3). Roughly one third (n=158, 33.9%) of respondents reported that PHM subspecialty designation increased or significantly increased their desire to pursue a career that includes adult HM (Appenidx Table 2). Finally, although the majority (n = 275, 59%) of respondents said they had no interest in a HM fellowship, 114 (24.5%) indicated interest in a combined med-peds HM fellowship (Appendix Table 1). Short-answer questions revealed that commitment to additional training on top of a 4-year residency program was a possible deterring factor, particularly in light of student loan debt and family obligations. Respondents reported adequate clinical training during residency as another deterring factor.

Med-Peds Resident–Perceived Needs in PHM Fellowship

Regardless of interest in completing a PHM fellowship, all resident survey respondents were asked how their ideal PHM fellowship should be structured. Almost all (n = 456, 97.9%) respondents indicated that they would prefer to complete a combined med-peds HM fellowship (Table 3), and most preferred to complete a fellowship in 2 years. Only 10 (2.1%) respondents preferred to complete a PHM fellowship alone in 2 or 3 years. More than half (n=253, 54.3%) of respondents indicated that it would be ideal to obtain a master’s degree as part of fellowship.

Three quarters (n = 355, 75.8%) of med-peds residents reported that they would want 41% or more of clinical time in an ideal fellowship dedicated to adult HM. Importantly, most (n = 322, 69.1%) of the med-peds residents did not consider moonlighting alone in either PHM or adult HM to be enough to maintain training. In addition, many (n = 366, 78.5%) respondents felt that it was important or very important for scholarly work during fellowship to bridge pediatrics and internal medicine.

Short-answer questions indicated that the ability to practice both internal medicine and pediatrics during fellowship emerged as an important deciding factor, with emphasis on adequate opportunities to maintain internal medicine knowledge base (Figure). Similarly, access to med-peds mentorship was an important component of the decision. Compensation both during fellowship and potential future earnings was also a prominent consideration.

Capacity of PHM Programs to Support Med-Peds Fellows

Fifteen (53.6%) FDs reported that their programs were able to accommodate both PHM and adult HM clinical time during fellowship, 11 (39.3%) were unsure, and 2 (7.1%) were unable to accommodate both (Table 2).

The options for adult HM clinical time varied by institution and included precepted time on adult HM, full attending privileges on adult HM, and adult HM time through moonlighting only. Short-answer responses from FDs with experience training med-peds fellows cited using PHM elective time for adult HM and offering moonlighting in adult HM as ways to address career goals of med-peds trainees. Scholarship time for fellows was preserved by decreasing required time on pediatric intensive care unit and complex care services.

Accessibility of Med-Peds Mentorship

As noted above, med-peds residents identified mentorship as an important factor in consideration of PHM fellowship. A total of 23 (82.1%) FDs reported their programs had med-peds faculty members within their PHM team (Table 2). The majority (n = 21, 91.3%) of those med-peds faculty had both PHM and adult HM clinical time.

This study characterized the ideal PHM fellowship structure from the perspective of med-peds residents and described the current ability of PHM fellowships to support med-peds residents. The majority of residents stated that they had no interest in an HM fellowship. However, for med-peds residents who considered a career in HM, 88.8% preferred to complete a combined internal medicine and pediatrics HM fellowship with close to half of clinical time dedicated to adult HM. Just over half (53.6%) of programs reported that they could currently accommodate both PHM and adult clinical time during fellowship, and all but two programs reported that they could accommodate both PHM and HM time in the future.

PHM subspecialty designation with associated fellowship training requirements decreased desire to practice HM among med-peds residents who responded to our survey. This reflects findings from a recently published study that evaluated whether PHM fellowship requirements for board certification influenced pediatric and med-peds residents’ decision to pursue PHM in 2018.⁴ Additionally, Chandrasekar et al⁴ found that 87% of respondents indicated that sufficient residency training was an important factor in discouraging them from pursuing PHM fellowship. We noted similar findings in our open-ended survey responses, which indicate that med-peds respondents perceived that the intended purpose of PHM fellowship was to provide additional clinical training, and that served as a deterrent for fellowship. However, the survey by Chandrasekar et al⁴ assessed only four factors for understanding what was important in encouraging pursuit of a PHM fellowship: opportunity to gain new skills, potential increase in salary, opportunity for a master’s degree, and increased prestige. Our survey expands on med-peds residents’ needs, indicating that med-peds residents want a combined med-peds/HM fellowship that allows them to meet PHM board-eligibility requirements while also continuing to develop their adult HM clinical practice and other nonclinical training objectives in a way that combines both adult HM and PHM. Both surveys demonstrate the role that residency program directors and other resident mentors can have in counseling trainees on the nonclinical training objectives of PHM fellowship, including research, quality improvement, medical education, and leadership and clinical operations. Additional emphasis can be placed on opportunities for an individualized curriculum to address the specific career aims of each resident.

In this study, med-peds trainees viewed distribution of clinical time during fellowship as an important factor in pursuing PHM fellowship. The perceived importance of balancing clinical time is not surprising considering that most survey respondents interested in HM ultimately intend to practice both PHM and adult HM. This finding corresponds with current practice patterns of med-peds hospitalists, the majority of whom care for both children and adults.^4,5 Moonlighting in adult medicine was not considered sufficient, suggesting desire for mentorship and training integration on the internal medicine side. Opportunities for trainees to maintain and expand their internal medicine knowledge base and clinical decision-making outside of moonlighting will be key to meeting the needs of med-peds residents in PHM fellowship.

Fortunately, more than half of responding programs reported that they could allow for adult HM practice during PHM fellowship. Twelve programs were unsure if they could accommodate adult HM clinical time, and only two programs reported they could not. We suspect that the ability to support this training with clinical time in both adult HM and PHM is more likely available at programs with established internal medicine relationships, often in the form of med-peds residency programs and med-peds faculty. Further, these established relationships may be more common at pediatric health systems that are integrated or affiliated with an adult health system. Most PHM fellowships surveyed indicated that their pediatric institution had an affiliation with an adult facility, and most had med-peds HM faculty.

Precedent for supporting med-peds fellows is somewhat limited given that only five of the responding PHM fellowship programs reported having fellows with med-peds residency training. However, discrepancies between the expressed needs of med-peds residents and the current Accreditation Council for Graduate Medical Education (ACGME)–accredited PHM fellowship structure highlight opportunities to tailor fellowship training to support the career goals of med-peds residents. The current PHM fellowship structure consists of 26 educational units, with each unit representing 4 calendar weeks. A minimum of eight units are spent on each of the following: core clinical rotations, systems and scholarship, and individualized curriculum.^10,11 The Society of Hospital Medicine has published core competencies for both PHM and adult HM, which highlight significant overlap in each field’s skill competency, particularly in areas such as quality improvement, legal issues and risk management, and handoffs and transitions of care.^12,13 We contend that competencies addressed within PHM fellowship core clinical rotations may overlap with adult HM. Training in adult HM could be completed as part of the individualized curriculum with the ACGME, allowing adult HM practice to count toward this requirement. This would offer med-peds fellows the option to maintain their adult HM knowledge base without eliminating all elective time. Ultimately, it will be important to be creative in how training is accomplished and skills are acquired during both core clinical and individualized training blocks for med-peds trainees completing PHM fellowship.

In order to meet the expressed needs of med-peds residents interested in incorporating both adult HM and PHM into their future careers through PHM fellowship, we offer key recommendations for consideration by the ACGME, PHM FDs, and med-peds program directors (Figure). We encourage current PHM fellowship programs to establish relationships with adult HM programs to develop structured clinical opportunities that will allow fellows to gain the additional clinical training desired.

There were important limitations in this study. First, our estimated response rate for the resident survey was 35.8% of all med-peds residents in 2019, which may be interpreted as low. However, it is important to note that the survey was targeted to residents interested in HM. More than 25% of med-peds residents pursue a career in HM,⁵ suggesting our response rate may be attributed to residents who did not complete the survey because they were interested in other fields. The program director survey response rate was higher at 58.3%, though it is possible that response bias resulted in a higher response rate from programs with the ability to support med-peds trainees. Regardless, data from programs with the ability to support med-peds trainees are highly valuable in describing how PHM fellowship can be inclusive of med-peds–trained physicians interested in pursuing HM.

Both surveys were completed in 2019, prior to the ACGME accreditation of PHM fellowship, which likely presents new, unique challenges to fellowship programs trying to support the needs of med-peds fellows. However, insights noted above from programs with experience training med-peds fellows are still applicable within the constraints of ACGME requirements.

Many med-peds residents express strong interest in practicing HM and including PHM as part of their future hospitalist practice. With the introduction of PHM subspecialty board certification through the American Board of Pediatrics, med-peds residents face new considerations when choosing a career path after residency. The majority of resident respondents express the desire to spend a substantial portion of their clinical practice and/or fellowship practicing adult HM. A majority of PHM fellowships can or are willing to explore how to provide both pediatric and adult hospitalist training to med-peds residency–trained fellows. Understanding the facilitators and barriers to recruiting med-peds trainees for PHM fellowship ultimately has significant implications for the future of the PHM workforce. Incorporating the recommendations noted in this study may increase retention of med-peds providers in PHM by enabling fellowship training and ultimately board certification. Collaboration among the ACGME, PHM program directors, and med-peds residency program directors could help to develop PHM fellowship training programs that will meet the needs of med-peds residents interested in practicing PHM while still meeting ACGME requirements for PHM board eligibility.

The authors thank Dr Anoop Agrawal of National Med-Peds Residents’ Association (NMPRA).

The American Board of Medical Specialties approved subspecialty designation for the field of pediatric hospital medicine (PHM) in 2016.¹ For those who started independent practice prior to July 2019, there were two options for board eligibility: the “practice pathway” or completion of a PHM fellowship. The practice pathway allows for pediatric and combined internal medicine–pediatric (med-peds) providers who graduated by July 2019 to sit for the PHM board-certification examination if they meet specific criteria in their pediatric practice.² For pediatric and med-peds residents who graduated after July 2019, PHM board eligibility is available only through completion of a PHM fellowship.

PHM subspecialty designation with fellowship training requirements may pose unique challenges to med-peds residents interested in practicing both pediatric and adult hospital medicine (HM).^3,4 Each year, an estimated 25% of med-peds residency graduates go on to practice HM.⁵ The majority (62%-83%) of currently practicing med-peds–trained hospitalists care for both adults and children.^5,6 Further, med-peds–trained hospitalists comprise at least 10% of the PHM workforce⁵ and play an important role in caring for adult survivors of childhood diseases.³

Limited existing data suggest that the future practice patterns of med-peds residents may be affected by PHM fellowship requirements. One previous survey study indicated that, although med-peds residents see value in additional training opportunities offered by fellowship, the majority are less likely to pursue PHM as a result of the new requirements.⁴ Prominent factors dissuading residents from pursuing PHM fellowship included forfeited earnings during fellowship, student loan obligations, family obligations, and the perception that training received during residency was sufficient. Although these data provide important insights into potential changes in practice patterns, they do not explore qualities of PHM fellowship that may make additional training more appealing to med-peds residents and promote retention of med-peds–trained providers in the PHM workforce.

Further, there is no existing literature exploring if and how PHM fellowship programs are equipped to support the needs of med-peds–trained fellows. Other subspecialties have supported med-peds trainees in combined fellowship training programs, including rheumatology, neurology, pediatric emergency medicine, allergy/immunology, physical medicine and rehabilitation, and psychiatry.^7,8 However, the extent to which PHM fellowships follow a similar model to accommodate the career goals of med-peds participants is unclear.

Given the large numbers of med-peds residents who go on to practice combined PHM and adult HM, it is crucial to understand the training needs of this group within the context of PHM fellowship and board certification. The primary objectives of this study were to understand (1) the perceived PHM fellowship needs of med-peds residents interested in HM, and (2) how the current PHM fellowship training environment can meet those needs. Understanding that additional training requirements to practice PHM may affect the career trajectory of residents interested in HM, secondary objectives included describing perceptions of med-peds residents on PHM specialty designation and whether designation affected their career plans.

Study Design

This cross-sectional study took place over a 3-month period from May to July 2019 and included two surveys of different populations to develop a comprehensive understanding of stakeholder perceptions of PHM fellowship. The first survey (resident survey) invited med-peds residents who were members of the National Med-Peds Residents’ Association (NMPRA)⁹ in 2019 and who were interested in HM. The second survey (fellowship director [FD] survey) included PHM FDs. The study was determined to be exempt by the University of Pittsburgh Institutional Review Board.

Study Population and Recruitment

Resident Survey

Two attempts were made to elicit participation via the NMPRA electronic mailing list. The NMPRA membership includes med-peds residents and chief residents from US med-peds residency programs. As of May 2019, 77 med-peds residency programs and their residents were members of NMPRA, which encompassed all med-peds programs in the United States and its territories. NMPRA maintains a listserv for all members, and all existing US/territory programs were members at the time of the survey. Med-peds interns, residents, and chief residents interested in HM were invited to participate in this study.

FD Survey

Forty-eight FDs, representing member institutions of the PHM Fellowship Directors’ Council, were surveyed via the PHM Fellowship Directors listserv.

Survey Instruments

We constructed two de novo surveys consisting of multiple-choice and short-answer questions (Appendix 1 and Appendix 2). To enhance the validity of survey responses, questions were designed and tested using an iterative consensus process among authors and additional participants, including current med-peds PHM fellows, PHM fellowship program directors, med-peds residency program directors, and current med-peds residents. These revisions were repeated for a total of four cycles. Items were created to increase knowledge on the following key areas: resident-perceived needs in fellowship training, impact of PHM subspecialty designation on career choices related to HM, health system structure of fellowship programs, and ability to accommodate med-peds clinical training within a PHM fellowship. A combined med-peds fellowship, as defined in the survey and referenced in this study, is a “combined internal medicine–pediatrics hospital medicine fellowship whereby you would remain eligible for PHM board certification.” To ensure a broad and inclusive view of potential needs of med-peds trainees considering fellowship, all respondents were asked to complete questions pertaining to anticipated fellowship needs regardless of their indicated interest in fellowship.

Data Collection

Survey completion was voluntary. Email identifiers were not linked to completed surveys. Study data were collected and managed by using Qualtrics XM. Only completed survey entries were included in analysis.

Statistical Methods and Data Analysis

R software version 4.0.2 (R Foundation for Statistical Computing) was used for statistical analysis. Demographic data were summarized using frequency distributions. The intent of the free-text questions for both surveys was qualitative explanatory thematic analysis. Authors EB, HL, and AJ used a deductive approach to identify common themes that elucidated med-peds resident–anticipated needs in fellowship and PHM program strategies and barriers to accommodate these needs. Preliminary themes and action items were reviewed and discussed among the full authorship team until consensus was reached.

Demographic Data

Resident Survey

A total of 466 med-peds residents completed the resident survey. There are approximately 1300 med-peds residents annually, creating an estimated response rate of 35.8% of all US med-peds residents. The majority (n = 380, 81.5%) of respondents were med-peds postgraduate years 1 through 3 and thus only eligible for PHM board certification via the PHM fellowship pathway (Table 1). Most (n = 446, 95.7%) respondents had considered a career in adult, pediatric, or combined HM at some point. Of those med-peds residents who considered a career in HM (Appendix Table 1), 92.8% (n = 414) would prefer to practice combined adult HM and PHM.

FD Survey

Twenty-eight FDs completed the FD survey, representing 58.3% of 2019 PHM fellowship programs. Of the responding programs, 23 (82.1%) were associated with a freestanding children’s hospital, and 24 (85.7%) were integrated or affiliated with a health system that provides adult inpatient care (Table 2). Sixteen (57.1%) programs had a med-peds residency program at their institution.

Med-Peds Resident Perceptions of PHM Fellowship

In considering the importance of PHM board certification for physicians practicing PHM, 59.0% (n= 275) of respondents rated board certification as “not at all important” (Appendix Table 2). Most (n = 420, 90.1%) med-peds trainees responded that PHM subspecialty designation “decreased” or “significantly decreased” their desire to pursue a career that includes PHM. Of the respondents who reported no interest in hospital medicine, eight (40%) reported that PHM subspecialty status dissuaded them from a career in HM at least a moderate amount (Appendix Table 3). Roughly one third (n=158, 33.9%) of respondents reported that PHM subspecialty designation increased or significantly increased their desire to pursue a career that includes adult HM (Appenidx Table 2). Finally, although the majority (n = 275, 59%) of respondents said they had no interest in a HM fellowship, 114 (24.5%) indicated interest in a combined med-peds HM fellowship (Appendix Table 1). Short-answer questions revealed that commitment to additional training on top of a 4-year residency program was a possible deterring factor, particularly in light of student loan debt and family obligations. Respondents reported adequate clinical training during residency as another deterring factor.

Med-Peds Resident–Perceived Needs in PHM Fellowship

Regardless of interest in completing a PHM fellowship, all resident survey respondents were asked how their ideal PHM fellowship should be structured. Almost all (n = 456, 97.9%) respondents indicated that they would prefer to complete a combined med-peds HM fellowship (Table 3), and most preferred to complete a fellowship in 2 years. Only 10 (2.1%) respondents preferred to complete a PHM fellowship alone in 2 or 3 years. More than half (n=253, 54.3%) of respondents indicated that it would be ideal to obtain a master’s degree as part of fellowship.

Three quarters (n = 355, 75.8%) of med-peds residents reported that they would want 41% or more of clinical time in an ideal fellowship dedicated to adult HM. Importantly, most (n = 322, 69.1%) of the med-peds residents did not consider moonlighting alone in either PHM or adult HM to be enough to maintain training. In addition, many (n = 366, 78.5%) respondents felt that it was important or very important for scholarly work during fellowship to bridge pediatrics and internal medicine.

Short-answer questions indicated that the ability to practice both internal medicine and pediatrics during fellowship emerged as an important deciding factor, with emphasis on adequate opportunities to maintain internal medicine knowledge base (Figure). Similarly, access to med-peds mentorship was an important component of the decision. Compensation both during fellowship and potential future earnings was also a prominent consideration.

Capacity of PHM Programs to Support Med-Peds Fellows

Fifteen (53.6%) FDs reported that their programs were able to accommodate both PHM and adult HM clinical time during fellowship, 11 (39.3%) were unsure, and 2 (7.1%) were unable to accommodate both (Table 2).

The options for adult HM clinical time varied by institution and included precepted time on adult HM, full attending privileges on adult HM, and adult HM time through moonlighting only. Short-answer responses from FDs with experience training med-peds fellows cited using PHM elective time for adult HM and offering moonlighting in adult HM as ways to address career goals of med-peds trainees. Scholarship time for fellows was preserved by decreasing required time on pediatric intensive care unit and complex care services.

Accessibility of Med-Peds Mentorship

As noted above, med-peds residents identified mentorship as an important factor in consideration of PHM fellowship. A total of 23 (82.1%) FDs reported their programs had med-peds faculty members within their PHM team (Table 2). The majority (n = 21, 91.3%) of those med-peds faculty had both PHM and adult HM clinical time.

This study characterized the ideal PHM fellowship structure from the perspective of med-peds residents and described the current ability of PHM fellowships to support med-peds residents. The majority of residents stated that they had no interest in an HM fellowship. However, for med-peds residents who considered a career in HM, 88.8% preferred to complete a combined internal medicine and pediatrics HM fellowship with close to half of clinical time dedicated to adult HM. Just over half (53.6%) of programs reported that they could currently accommodate both PHM and adult clinical time during fellowship, and all but two programs reported that they could accommodate both PHM and HM time in the future.

PHM subspecialty designation with associated fellowship training requirements decreased desire to practice HM among med-peds residents who responded to our survey. This reflects findings from a recently published study that evaluated whether PHM fellowship requirements for board certification influenced pediatric and med-peds residents’ decision to pursue PHM in 2018.⁴ Additionally, Chandrasekar et al⁴ found that 87% of respondents indicated that sufficient residency training was an important factor in discouraging them from pursuing PHM fellowship. We noted similar findings in our open-ended survey responses, which indicate that med-peds respondents perceived that the intended purpose of PHM fellowship was to provide additional clinical training, and that served as a deterrent for fellowship. However, the survey by Chandrasekar et al⁴ assessed only four factors for understanding what was important in encouraging pursuit of a PHM fellowship: opportunity to gain new skills, potential increase in salary, opportunity for a master’s degree, and increased prestige. Our survey expands on med-peds residents’ needs, indicating that med-peds residents want a combined med-peds/HM fellowship that allows them to meet PHM board-eligibility requirements while also continuing to develop their adult HM clinical practice and other nonclinical training objectives in a way that combines both adult HM and PHM. Both surveys demonstrate the role that residency program directors and other resident mentors can have in counseling trainees on the nonclinical training objectives of PHM fellowship, including research, quality improvement, medical education, and leadership and clinical operations. Additional emphasis can be placed on opportunities for an individualized curriculum to address the specific career aims of each resident.

In this study, med-peds trainees viewed distribution of clinical time during fellowship as an important factor in pursuing PHM fellowship. The perceived importance of balancing clinical time is not surprising considering that most survey respondents interested in HM ultimately intend to practice both PHM and adult HM. This finding corresponds with current practice patterns of med-peds hospitalists, the majority of whom care for both children and adults.^4,5 Moonlighting in adult medicine was not considered sufficient, suggesting desire for mentorship and training integration on the internal medicine side. Opportunities for trainees to maintain and expand their internal medicine knowledge base and clinical decision-making outside of moonlighting will be key to meeting the needs of med-peds residents in PHM fellowship.

Fortunately, more than half of responding programs reported that they could allow for adult HM practice during PHM fellowship. Twelve programs were unsure if they could accommodate adult HM clinical time, and only two programs reported they could not. We suspect that the ability to support this training with clinical time in both adult HM and PHM is more likely available at programs with established internal medicine relationships, often in the form of med-peds residency programs and med-peds faculty. Further, these established relationships may be more common at pediatric health systems that are integrated or affiliated with an adult health system. Most PHM fellowships surveyed indicated that their pediatric institution had an affiliation with an adult facility, and most had med-peds HM faculty.

Precedent for supporting med-peds fellows is somewhat limited given that only five of the responding PHM fellowship programs reported having fellows with med-peds residency training. However, discrepancies between the expressed needs of med-peds residents and the current Accreditation Council for Graduate Medical Education (ACGME)–accredited PHM fellowship structure highlight opportunities to tailor fellowship training to support the career goals of med-peds residents. The current PHM fellowship structure consists of 26 educational units, with each unit representing 4 calendar weeks. A minimum of eight units are spent on each of the following: core clinical rotations, systems and scholarship, and individualized curriculum.^10,11 The Society of Hospital Medicine has published core competencies for both PHM and adult HM, which highlight significant overlap in each field’s skill competency, particularly in areas such as quality improvement, legal issues and risk management, and handoffs and transitions of care.^12,13 We contend that competencies addressed within PHM fellowship core clinical rotations may overlap with adult HM. Training in adult HM could be completed as part of the individualized curriculum with the ACGME, allowing adult HM practice to count toward this requirement. This would offer med-peds fellows the option to maintain their adult HM knowledge base without eliminating all elective time. Ultimately, it will be important to be creative in how training is accomplished and skills are acquired during both core clinical and individualized training blocks for med-peds trainees completing PHM fellowship.

In order to meet the expressed needs of med-peds residents interested in incorporating both adult HM and PHM into their future careers through PHM fellowship, we offer key recommendations for consideration by the ACGME, PHM FDs, and med-peds program directors (Figure). We encourage current PHM fellowship programs to establish relationships with adult HM programs to develop structured clinical opportunities that will allow fellows to gain the additional clinical training desired.

There were important limitations in this study. First, our estimated response rate for the resident survey was 35.8% of all med-peds residents in 2019, which may be interpreted as low. However, it is important to note that the survey was targeted to residents interested in HM. More than 25% of med-peds residents pursue a career in HM,⁵ suggesting our response rate may be attributed to residents who did not complete the survey because they were interested in other fields. The program director survey response rate was higher at 58.3%, though it is possible that response bias resulted in a higher response rate from programs with the ability to support med-peds trainees. Regardless, data from programs with the ability to support med-peds trainees are highly valuable in describing how PHM fellowship can be inclusive of med-peds–trained physicians interested in pursuing HM.

Both surveys were completed in 2019, prior to the ACGME accreditation of PHM fellowship, which likely presents new, unique challenges to fellowship programs trying to support the needs of med-peds fellows. However, insights noted above from programs with experience training med-peds fellows are still applicable within the constraints of ACGME requirements.

Many med-peds residents express strong interest in practicing HM and including PHM as part of their future hospitalist practice. With the introduction of PHM subspecialty board certification through the American Board of Pediatrics, med-peds residents face new considerations when choosing a career path after residency. The majority of resident respondents express the desire to spend a substantial portion of their clinical practice and/or fellowship practicing adult HM. A majority of PHM fellowships can or are willing to explore how to provide both pediatric and adult hospitalist training to med-peds residency–trained fellows. Understanding the facilitators and barriers to recruiting med-peds trainees for PHM fellowship ultimately has significant implications for the future of the PHM workforce. Incorporating the recommendations noted in this study may increase retention of med-peds providers in PHM by enabling fellowship training and ultimately board certification. Collaboration among the ACGME, PHM program directors, and med-peds residency program directors could help to develop PHM fellowship training programs that will meet the needs of med-peds residents interested in practicing PHM while still meeting ACGME requirements for PHM board eligibility.

The authors thank Dr Anoop Agrawal of National Med-Peds Residents’ Association (NMPRA).

The American Board of Medical Specialties approved subspecialty designation for the field of pediatric hospital medicine (PHM) in 2016.¹ For those who started independent practice prior to July 2019, there were two options for board eligibility: the “practice pathway” or completion of a PHM fellowship. The practice pathway allows for pediatric and combined internal medicine–pediatric (med-peds) providers who graduated by July 2019 to sit for the PHM board-certification examination if they meet specific criteria in their pediatric practice.² For pediatric and med-peds residents who graduated after July 2019, PHM board eligibility is available only through completion of a PHM fellowship.

PHM subspecialty designation with fellowship training requirements may pose unique challenges to med-peds residents interested in practicing both pediatric and adult hospital medicine (HM).^3,4 Each year, an estimated 25% of med-peds residency graduates go on to practice HM.⁵ The majority (62%-83%) of currently practicing med-peds–trained hospitalists care for both adults and children.^5,6 Further, med-peds–trained hospitalists comprise at least 10% of the PHM workforce⁵ and play an important role in caring for adult survivors of childhood diseases.³

Limited existing data suggest that the future practice patterns of med-peds residents may be affected by PHM fellowship requirements. One previous survey study indicated that, although med-peds residents see value in additional training opportunities offered by fellowship, the majority are less likely to pursue PHM as a result of the new requirements.⁴ Prominent factors dissuading residents from pursuing PHM fellowship included forfeited earnings during fellowship, student loan obligations, family obligations, and the perception that training received during residency was sufficient. Although these data provide important insights into potential changes in practice patterns, they do not explore qualities of PHM fellowship that may make additional training more appealing to med-peds residents and promote retention of med-peds–trained providers in the PHM workforce.

Further, there is no existing literature exploring if and how PHM fellowship programs are equipped to support the needs of med-peds–trained fellows. Other subspecialties have supported med-peds trainees in combined fellowship training programs, including rheumatology, neurology, pediatric emergency medicine, allergy/immunology, physical medicine and rehabilitation, and psychiatry.^7,8 However, the extent to which PHM fellowships follow a similar model to accommodate the career goals of med-peds participants is unclear.

Given the large numbers of med-peds residents who go on to practice combined PHM and adult HM, it is crucial to understand the training needs of this group within the context of PHM fellowship and board certification. The primary objectives of this study were to understand (1) the perceived PHM fellowship needs of med-peds residents interested in HM, and (2) how the current PHM fellowship training environment can meet those needs. Understanding that additional training requirements to practice PHM may affect the career trajectory of residents interested in HM, secondary objectives included describing perceptions of med-peds residents on PHM specialty designation and whether designation affected their career plans.

Study Design

This cross-sectional study took place over a 3-month period from May to July 2019 and included two surveys of different populations to develop a comprehensive understanding of stakeholder perceptions of PHM fellowship. The first survey (resident survey) invited med-peds residents who were members of the National Med-Peds Residents’ Association (NMPRA)⁹ in 2019 and who were interested in HM. The second survey (fellowship director [FD] survey) included PHM FDs. The study was determined to be exempt by the University of Pittsburgh Institutional Review Board.

Study Population and Recruitment

Resident Survey

Two attempts were made to elicit participation via the NMPRA electronic mailing list. The NMPRA membership includes med-peds residents and chief residents from US med-peds residency programs. As of May 2019, 77 med-peds residency programs and their residents were members of NMPRA, which encompassed all med-peds programs in the United States and its territories. NMPRA maintains a listserv for all members, and all existing US/territory programs were members at the time of the survey. Med-peds interns, residents, and chief residents interested in HM were invited to participate in this study.

FD Survey

Forty-eight FDs, representing member institutions of the PHM Fellowship Directors’ Council, were surveyed via the PHM Fellowship Directors listserv.

Survey Instruments

We constructed two de novo surveys consisting of multiple-choice and short-answer questions (Appendix 1 and Appendix 2). To enhance the validity of survey responses, questions were designed and tested using an iterative consensus process among authors and additional participants, including current med-peds PHM fellows, PHM fellowship program directors, med-peds residency program directors, and current med-peds residents. These revisions were repeated for a total of four cycles. Items were created to increase knowledge on the following key areas: resident-perceived needs in fellowship training, impact of PHM subspecialty designation on career choices related to HM, health system structure of fellowship programs, and ability to accommodate med-peds clinical training within a PHM fellowship. A combined med-peds fellowship, as defined in the survey and referenced in this study, is a “combined internal medicine–pediatrics hospital medicine fellowship whereby you would remain eligible for PHM board certification.” To ensure a broad and inclusive view of potential needs of med-peds trainees considering fellowship, all respondents were asked to complete questions pertaining to anticipated fellowship needs regardless of their indicated interest in fellowship.

Data Collection

Survey completion was voluntary. Email identifiers were not linked to completed surveys. Study data were collected and managed by using Qualtrics XM. Only completed survey entries were included in analysis.

Statistical Methods and Data Analysis

R software version 4.0.2 (R Foundation for Statistical Computing) was used for statistical analysis. Demographic data were summarized using frequency distributions. The intent of the free-text questions for both surveys was qualitative explanatory thematic analysis. Authors EB, HL, and AJ used a deductive approach to identify common themes that elucidated med-peds resident–anticipated needs in fellowship and PHM program strategies and barriers to accommodate these needs. Preliminary themes and action items were reviewed and discussed among the full authorship team until consensus was reached.

Demographic Data

Resident Survey

A total of 466 med-peds residents completed the resident survey. There are approximately 1300 med-peds residents annually, creating an estimated response rate of 35.8% of all US med-peds residents. The majority (n = 380, 81.5%) of respondents were med-peds postgraduate years 1 through 3 and thus only eligible for PHM board certification via the PHM fellowship pathway (Table 1). Most (n = 446, 95.7%) respondents had considered a career in adult, pediatric, or combined HM at some point. Of those med-peds residents who considered a career in HM (Appendix Table 1), 92.8% (n = 414) would prefer to practice combined adult HM and PHM.

FD Survey

Twenty-eight FDs completed the FD survey, representing 58.3% of 2019 PHM fellowship programs. Of the responding programs, 23 (82.1%) were associated with a freestanding children’s hospital, and 24 (85.7%) were integrated or affiliated with a health system that provides adult inpatient care (Table 2). Sixteen (57.1%) programs had a med-peds residency program at their institution.

Med-Peds Resident Perceptions of PHM Fellowship

In considering the importance of PHM board certification for physicians practicing PHM, 59.0% (n= 275) of respondents rated board certification as “not at all important” (Appendix Table 2). Most (n = 420, 90.1%) med-peds trainees responded that PHM subspecialty designation “decreased” or “significantly decreased” their desire to pursue a career that includes PHM. Of the respondents who reported no interest in hospital medicine, eight (40%) reported that PHM subspecialty status dissuaded them from a career in HM at least a moderate amount (Appendix Table 3). Roughly one third (n=158, 33.9%) of respondents reported that PHM subspecialty designation increased or significantly increased their desire to pursue a career that includes adult HM (Appenidx Table 2). Finally, although the majority (n = 275, 59%) of respondents said they had no interest in a HM fellowship, 114 (24.5%) indicated interest in a combined med-peds HM fellowship (Appendix Table 1). Short-answer questions revealed that commitment to additional training on top of a 4-year residency program was a possible deterring factor, particularly in light of student loan debt and family obligations. Respondents reported adequate clinical training during residency as another deterring factor.

Med-Peds Resident–Perceived Needs in PHM Fellowship

Regardless of interest in completing a PHM fellowship, all resident survey respondents were asked how their ideal PHM fellowship should be structured. Almost all (n = 456, 97.9%) respondents indicated that they would prefer to complete a combined med-peds HM fellowship (Table 3), and most preferred to complete a fellowship in 2 years. Only 10 (2.1%) respondents preferred to complete a PHM fellowship alone in 2 or 3 years. More than half (n=253, 54.3%) of respondents indicated that it would be ideal to obtain a master’s degree as part of fellowship.

Three quarters (n = 355, 75.8%) of med-peds residents reported that they would want 41% or more of clinical time in an ideal fellowship dedicated to adult HM. Importantly, most (n = 322, 69.1%) of the med-peds residents did not consider moonlighting alone in either PHM or adult HM to be enough to maintain training. In addition, many (n = 366, 78.5%) respondents felt that it was important or very important for scholarly work during fellowship to bridge pediatrics and internal medicine.

Short-answer questions indicated that the ability to practice both internal medicine and pediatrics during fellowship emerged as an important deciding factor, with emphasis on adequate opportunities to maintain internal medicine knowledge base (Figure). Similarly, access to med-peds mentorship was an important component of the decision. Compensation both during fellowship and potential future earnings was also a prominent consideration.

Capacity of PHM Programs to Support Med-Peds Fellows

Fifteen (53.6%) FDs reported that their programs were able to accommodate both PHM and adult HM clinical time during fellowship, 11 (39.3%) were unsure, and 2 (7.1%) were unable to accommodate both (Table 2).

The options for adult HM clinical time varied by institution and included precepted time on adult HM, full attending privileges on adult HM, and adult HM time through moonlighting only. Short-answer responses from FDs with experience training med-peds fellows cited using PHM elective time for adult HM and offering moonlighting in adult HM as ways to address career goals of med-peds trainees. Scholarship time for fellows was preserved by decreasing required time on pediatric intensive care unit and complex care services.

Accessibility of Med-Peds Mentorship

As noted above, med-peds residents identified mentorship as an important factor in consideration of PHM fellowship. A total of 23 (82.1%) FDs reported their programs had med-peds faculty members within their PHM team (Table 2). The majority (n = 21, 91.3%) of those med-peds faculty had both PHM and adult HM clinical time.

This study characterized the ideal PHM fellowship structure from the perspective of med-peds residents and described the current ability of PHM fellowships to support med-peds residents. The majority of residents stated that they had no interest in an HM fellowship. However, for med-peds residents who considered a career in HM, 88.8% preferred to complete a combined internal medicine and pediatrics HM fellowship with close to half of clinical time dedicated to adult HM. Just over half (53.6%) of programs reported that they could currently accommodate both PHM and adult clinical time during fellowship, and all but two programs reported that they could accommodate both PHM and HM time in the future.

PHM subspecialty designation with associated fellowship training requirements decreased desire to practice HM among med-peds residents who responded to our survey. This reflects findings from a recently published study that evaluated whether PHM fellowship requirements for board certification influenced pediatric and med-peds residents’ decision to pursue PHM in 2018.⁴ Additionally, Chandrasekar et al⁴ found that 87% of respondents indicated that sufficient residency training was an important factor in discouraging them from pursuing PHM fellowship. We noted similar findings in our open-ended survey responses, which indicate that med-peds respondents perceived that the intended purpose of PHM fellowship was to provide additional clinical training, and that served as a deterrent for fellowship. However, the survey by Chandrasekar et al⁴ assessed only four factors for understanding what was important in encouraging pursuit of a PHM fellowship: opportunity to gain new skills, potential increase in salary, opportunity for a master’s degree, and increased prestige. Our survey expands on med-peds residents’ needs, indicating that med-peds residents want a combined med-peds/HM fellowship that allows them to meet PHM board-eligibility requirements while also continuing to develop their adult HM clinical practice and other nonclinical training objectives in a way that combines both adult HM and PHM. Both surveys demonstrate the role that residency program directors and other resident mentors can have in counseling trainees on the nonclinical training objectives of PHM fellowship, including research, quality improvement, medical education, and leadership and clinical operations. Additional emphasis can be placed on opportunities for an individualized curriculum to address the specific career aims of each resident.

In this study, med-peds trainees viewed distribution of clinical time during fellowship as an important factor in pursuing PHM fellowship. The perceived importance of balancing clinical time is not surprising considering that most survey respondents interested in HM ultimately intend to practice both PHM and adult HM. This finding corresponds with current practice patterns of med-peds hospitalists, the majority of whom care for both children and adults.^4,5 Moonlighting in adult medicine was not considered sufficient, suggesting desire for mentorship and training integration on the internal medicine side. Opportunities for trainees to maintain and expand their internal medicine knowledge base and clinical decision-making outside of moonlighting will be key to meeting the needs of med-peds residents in PHM fellowship.

Fortunately, more than half of responding programs reported that they could allow for adult HM practice during PHM fellowship. Twelve programs were unsure if they could accommodate adult HM clinical time, and only two programs reported they could not. We suspect that the ability to support this training with clinical time in both adult HM and PHM is more likely available at programs with established internal medicine relationships, often in the form of med-peds residency programs and med-peds faculty. Further, these established relationships may be more common at pediatric health systems that are integrated or affiliated with an adult health system. Most PHM fellowships surveyed indicated that their pediatric institution had an affiliation with an adult facility, and most had med-peds HM faculty.

Precedent for supporting med-peds fellows is somewhat limited given that only five of the responding PHM fellowship programs reported having fellows with med-peds residency training. However, discrepancies between the expressed needs of med-peds residents and the current Accreditation Council for Graduate Medical Education (ACGME)–accredited PHM fellowship structure highlight opportunities to tailor fellowship training to support the career goals of med-peds residents. The current PHM fellowship structure consists of 26 educational units, with each unit representing 4 calendar weeks. A minimum of eight units are spent on each of the following: core clinical rotations, systems and scholarship, and individualized curriculum.^10,11 The Society of Hospital Medicine has published core competencies for both PHM and adult HM, which highlight significant overlap in each field’s skill competency, particularly in areas such as quality improvement, legal issues and risk management, and handoffs and transitions of care.^12,13 We contend that competencies addressed within PHM fellowship core clinical rotations may overlap with adult HM. Training in adult HM could be completed as part of the individualized curriculum with the ACGME, allowing adult HM practice to count toward this requirement. This would offer med-peds fellows the option to maintain their adult HM knowledge base without eliminating all elective time. Ultimately, it will be important to be creative in how training is accomplished and skills are acquired during both core clinical and individualized training blocks for med-peds trainees completing PHM fellowship.

In order to meet the expressed needs of med-peds residents interested in incorporating both adult HM and PHM into their future careers through PHM fellowship, we offer key recommendations for consideration by the ACGME, PHM FDs, and med-peds program directors (Figure). We encourage current PHM fellowship programs to establish relationships with adult HM programs to develop structured clinical opportunities that will allow fellows to gain the additional clinical training desired.

There were important limitations in this study. First, our estimated response rate for the resident survey was 35.8% of all med-peds residents in 2019, which may be interpreted as low. However, it is important to note that the survey was targeted to residents interested in HM. More than 25% of med-peds residents pursue a career in HM,⁵ suggesting our response rate may be attributed to residents who did not complete the survey because they were interested in other fields. The program director survey response rate was higher at 58.3%, though it is possible that response bias resulted in a higher response rate from programs with the ability to support med-peds trainees. Regardless, data from programs with the ability to support med-peds trainees are highly valuable in describing how PHM fellowship can be inclusive of med-peds–trained physicians interested in pursuing HM.

Both surveys were completed in 2019, prior to the ACGME accreditation of PHM fellowship, which likely presents new, unique challenges to fellowship programs trying to support the needs of med-peds fellows. However, insights noted above from programs with experience training med-peds fellows are still applicable within the constraints of ACGME requirements.

Many med-peds residents express strong interest in practicing HM and including PHM as part of their future hospitalist practice. With the introduction of PHM subspecialty board certification through the American Board of Pediatrics, med-peds residents face new considerations when choosing a career path after residency. The majority of resident respondents express the desire to spend a substantial portion of their clinical practice and/or fellowship practicing adult HM. A majority of PHM fellowships can or are willing to explore how to provide both pediatric and adult hospitalist training to med-peds residency–trained fellows. Understanding the facilitators and barriers to recruiting med-peds trainees for PHM fellowship ultimately has significant implications for the future of the PHM workforce. Incorporating the recommendations noted in this study may increase retention of med-peds providers in PHM by enabling fellowship training and ultimately board certification. Collaboration among the ACGME, PHM program directors, and med-peds residency program directors could help to develop PHM fellowship training programs that will meet the needs of med-peds residents interested in practicing PHM while still meeting ACGME requirements for PHM board eligibility.

The authors thank Dr Anoop Agrawal of National Med-Peds Residents’ Association (NMPRA).

Febrile infants aged 0 to 60 days often undergo diagnostic testing to evaluate for invasive bacterial infections (IBI; ie, bacteremia and meningitis) and are subsequently hospitalized pending culture results. Only 1% to 2% of infants 0 to 60 days old have an IBI,^1-3 and most hospitalized infants are discharged once physicians feel confident that pathogens are unlikely to be isolated from blood and cerebrospinal fluid (CSF) cultures. Practice regarding duration of hospitalization while awaiting blood and CSF culture results is not standardized in this population. Longer hospitalizations can lead to increased costs and familial stress, including difficulty with breastfeeding and anxiety in newly postpartum mothers.^4,5

In 2010, an institutional evidence-based guideline for the management of febrile infants aged 0 to 60 days recommended discharge after 36 hours of observation if all cultures were negative.⁶ However, recent studies demonstrate that 85% to 93% of pathogens in blood and CSF cultures grow within 24 hours of incubation.^7-9 Assuming a 2% prevalence of IBI, if 15% of pathogens were identified after 24 hours of incubation, only one out of 333 infants would have an IBI identified after 24 hours of hospital observation.⁷

Furthermore, a review of our institution’s electronic health records (EHR) over the past 5 years revealed that an observation period of 24 hours would have resulted in the discharge of three infants with an IBI. Two infants had bacteremia; both were discharged from the emergency department (ED) without antibiotics, returned to care after cultures were reported positive at 27 hours, and had no adverse outcomes. The third infant had meningitis, but also had an abnormal CSF Gram stain, which led to a longer hospitalization.

In 2019, our institution appraised the emerging literature and institutional data supporting the low absolute risk of missed IBI, and also leveraged local consensus among key stakeholders to update its evidence-based guideline for the evaluation and management of febrile infants aged 60 days and younger. The updated guideline recommends that clinicians consider discharging well-appearing neonates and infants if blood and CSF cultures remain negative at 24 hours.¹⁰ The objective of this study was to decrease the average hospital culture observation time (COT; culture incubation to hospital discharge) from 38 to 30 hours over a 12-month period in febrile infants aged 0 to 60 days.

Context

Improvement efforts were conducted at Cincinnati Children’s Hospital Medical Center (CCHMC), a large, urban, academic hospital that admitted more than 8,000 noncritically ill patients to the hospital medicine (HM) service from July 1, 2018, through June 30, 2019. Hospital medicine teams, located at both the main and satellite campuses, are staffed by attending physicians, fellows, residents, medical students, and nurse practitioners. The two campuses, which are about 20 miles apart, share clinician providers but have distinct nursing pools.

Microbiology services for all CCHMC patients are provided at the main campus. Blood and CSF cultures at the satellite campus are transported to the main campus for incubation and monitoring via an urgent courier service. The microbiology laboratory at CCHMC uses a continuous monitoring system for blood cultures (BACT/ALERT Virtuo, BioMérieux). The system automatically alerts laboratory technicians of positive cultures; these results are reported to clinical providers within 30 minutes of detection. Laboratory technicians manually evaluate CSF cultures once daily for 5 days.

Improvement Team

Our improvement team included three HM attending physicians; two HM fellows; a pediatric chief resident; two nurses, who represented nursing pools at the main and satellite campuses; and a clinical pharmacist, who is a co-leader of the antimicrobial stewardship program at CCHMC. Supporting members for the improvement team included the CCHMC laboratory director; the microbiology laboratory director; an infectious disease physician, who is a co-leader of the antimicrobial stewardship program; and nursing directors of the HM units at both campuses.

Evidence-Based Guideline

Our improvement initiative was based on recommendations from the updated CCHMC Evidence-Based Care Guideline for Management of Infants 0 to 60 days with Fever of Unknown Source.¹⁰ This guideline, published in May 2019, was developed by a multidisciplinary working group composed of key stakeholders from HM, community pediatrics, emergency medicine, the pediatric residency program, infectious disease, and laboratory medicine. Several improvement team members were participants on the committee that published the evidence-based guideline. The committee first performed a systematic literature review and critical appraisal of the literature. Care recommendations were formulated via a consensus process directed by best evidence, patient and family preferences, and clinical expertise; the recommendations were subsequently reviewed and approved by clinical experts who were not involved in the development process.

Based on evidence review and multistakeholder consensus, the updated guideline recommends clinicians consider discharging neonates and infants aged 60 days and younger if there is no culture growth after an observation period of 24 hours (as documented in the EHR) and patients are otherwise medically ready for discharge (ie, well appearing with adequate oral intake).^10,11 In addition, prior to discharge, there must be a documented working phone number on file for the patient’s parents/guardians, an established outpatient follow-up plan within 24 hours, and communication with the primary pediatrician who is in agreement with discharge at 24 hours.

Study Population

Infants 0 to 60 days old who had a documented or reported fever without an apparent source based on history and physical exam upon presentation to the ED, and who were subsequently admitted to the HM service at CCHMC between October 30, 2018, and July 10, 2020, were eligible for inclusion. We excluded infants who were admitted to other clinical services (eg, intensive care unit); had organisms identified on blood, urine, or CSF culture within 24 hours of incubation; had positive herpes simplex virus testing; had skin/soft tissue infections or another clearly documented source of bacterial infection; or had an alternative indication for hospitalization (eg, need for intravenous fluid or deep suctioning) after cultures had incubated for 24 hours. Infants who had a positive blood, urine, or CSF culture result after 24 hours of incubation were included in the study population. Organisms were classified as pathogen or contaminant based on treatment decisions made by the care team.

Improvement Activities

Key drivers critical to success of the improvement efforts were: (1) clearly defined standard of care for duration of observation in febrile infants 0 to 60 days old; (2) improved understanding of microbiology lab procedures; (3) effective communication of discharge criteria between providers and nurses; and (4) transparency of data with feedback (Figure 1).

Education and Structured Dissemination of Evidence-Based Guideline

The CCHMC febrile infant guideline¹⁰ was disseminated to HM physicians, residents, and nurses via the following means: (1) in-person announcements at staff meetings and educational conferences, (2) published highlights from the guideline in weekly newsletters, and (3) email announcements. Additionally, members of the study team educated HM attending physicians, nursing staff from the medical units at both campuses, and resident physicians about recent studies demonstrating safety of shorter length of stay (LOS) in febrile infants aged 0 to 60 days. The study team also provided residents, physicians, and nurses with data on the number of positive blood and CSF cultures and outcomes of patients at CCHMC within the past 5 years. In addition, team members led a journal club for residents discussing an article⁷ describing time-to-positivity of blood and CSF cultures in febrile infants. For ongoing engagement, the evidence-based guideline and a detailed explanation of microbiology procedures were published in the resident handbook, an internal resource that includes vital clinical pearls and practice guidelines across specialties. (Each resident receives an updated hard copy each year, and there is also an online link to the resource in the EHR.) Information about the guideline and COT was also included in the monthly chief resident’s orientation script, which is relayed to all residents on the first day of their HM rotation.

Clear Communication of Microbiology Procedures

Team members created a detailed process map describing the processing protocols for blood and CSF cultures collected at both CCHMC campuses. This information was shared with HM attending physicians and nurses via in-person announcements at staff meetings, flyers in team workrooms, and email communications. Residents received information on microbiology protocols via in-person announcements at educational conferences and dissemination in the weekly residency newsletter.Important information communicated included:

1. Definition of culture start time. We conveyed that there may be a delay of up to 4 hours between culture collection at the satellite campus and culture incubation at the main campus laboratory. As a result, the time of blood or CSF sample arrival to the main campus laboratory was a more accurate reflection of the culture incubation start time than the culture collection time.

2. Explanation of CSF culture processing. We discussed the process by which these cultures are plated upon arrival at the microbiology laboratory and read once per day in the morning. Therefore, a culture incubated at midnight would be evaluated once at 9 hours and not again until 33 hours.

Modification of Febrile Infant Order Set

Enhancements to the febrile infant order set improved communication and cultivated a shared mental model regarding discharge goals among all members of the care team. The EHR order set for febrile infants was updated as follows: (1) mandatory free-text fields that established the culture start time for blood and CSF cultures were added, (2) culture start time was clearly defined (ie, the time culture arrives at the main campus laboratory), and (3) a change was made in the default discharge criteria¹¹ to “culture observation for 24 hours,” with the ability to modify COT (Appendix Figure 1). We embedded hyperlinks to the guideline and microbiology process map within the updated order set, which allowed providers to easily access this information and refresh their knowledge of the recommendations (Appendix Figure 1).

Identification of Failures and Follow-up With Near-Time Feedback

All cases of febrile infants were tracked weekly. For infants hospitalized longer than 24 hours, the study team contacted the discharging clinicians to discuss reasons for prolonged hospitalization, with an emphasis on identifying system-level barriers to earlier discharge.

Study of the Interventions

The institutional microbiology database was queried weekly to identify all infants 0 to 60 days old who had a blood culture obtained and were hospitalized on the HM service. Study team members conducted targeted EHR review to determine whether patients met exclusion criteria and to identify reasons for prolonged COT. Baseline data were collected retrospectively for a 3-month period prior to initiation of improvement activities. During the study period, queries were conducted weekly and reviewed by study team members to evaluate the impact of improvement activities and to inform new interventions.

Measures

Our primary outcome measure was COT, defined as the hours between final culture incubation and hospital discharge. The operational definition for “final culture incubation” was the documented time of arrival of the last collected culture to the microbiology laboratory. Our goal COT was 30 hours to account for a subset of patients whose blood and/or CSF culture were obtained overnight (ie, after 9 pm), since subsequent discharge times would likely and practically be delayed beyond 24 hours. Our secondary outcome measure was LOS, defined as the time between ED arrival and hospital discharge. Process measures included the proportion of patients for whom the febrile infant EHR order set was used and the proportion of patients for whom medical discharge criteria (ie, blood and CSF culture observed for ”xx” hours) and culture incubation start times were entered using the order set. Balancing measures included identification of IBI after hospital discharge, 48-hour ED revisits, and 7-day hospital readmissions.

Analysis

Measures were evaluated using statistical process control charts and run charts, and Western Electric rules were employed to determine special cause variation.¹² Annotated X-bar S control charts tracked the impact of improvement activities on average COT and LOS for all infants. Given that a relatively small number of patients (ie, two to four) met inclusion criteria each week, average COT was calculated per five patients.

This study was considered exempt from review by the CCHMC Institutional Review Board.

Of the 184 infants in this study, 46 were included as part of baseline data collection, and 138 were included during the intervention period. The median age was 26.6 days (range, 3-59 days); 52% of patients were female; two-thirds were non-Hispanic White; 22% were Black, and 5% were Hispanic (Appendix Table).

Average COT decreased from 38 hours to 32 hours with improvement activities (Figure 2) and was sustained for a total of 17 months. There were small decreases in COT after initial education was provided to attendings, nurses, and residents.

After the launch of the updated order set, median usage of the EHR order set increased from 50% to 80%. Medical discharge criteria were entered for 80 (96%) of the 83 patients for whom the updated order set was applied; culture incubation start times were entered for 78 (94%) of these patients.

No infants in our cohort were found to have IBI after hospital discharge. There were no ED revisits within 48 hours of discharge, and there were no hospital readmissions within 7 days of index discharge. Furthermore, none of the patients included in the study had growth of a pathogenic organism after 24 hours.

Of the 138 infants hospitalized during the intervention period, 77 (56%) had a COT greater than 30 hours. Among these 77 patients, 49 (64%) had their final culture incubated between 9 pm and 4 am; Furthermore, 11 (14%) had missing, abnormal, pretreated, or uninterpretable CSF studies, 7 (9%) had ongoing fevers, and 4 (5%) remained hospitalized due to family preference or inability to obtain timely outpatient follow-up.

Our study aimed to decrease the average COT from 38 hours to 30 hours among hospitalized infants aged 60 days and younger over a period of 12 months. An intervention featuring implementation of an evidence-based guideline through education, laboratory procedure transparency, creation of a standardized EHR order set, and near-time feedback was associated with a shorter average COT of 32 hours, sustained over a 17-month period. No infants with bacteremia or meningitis were inappropriately discharged during this study.

Interpretation

Prior to our improvement efforts, most febrile infants at CCHMC were observed for at least 36 hours based on a prior institutional guideline,⁶ despite recent evidence suggesting that most pathogens in blood and CSF cultures grow within 24 hours of incubation.^7-9 The goal of this improvement initiative was to bridge the gap between emerging evidence and clinical practice by developing and disseminating an updated evidence-based guideline to safely decrease the hospital observation time in febrile infants aged 60 days and younger.

Similar to previous studies aimed at improving diagnosis and management among febrile infants,^13-16 generation and structured dissemination of an institutional evidence-based guideline was crucial to safely shortening COT in our population. These prior studies established a goal COT of 36 to 42 hours for hospitalized febrile infants.^13,15,16 Our study incorporated emerging evidence and local experience into an updated evidence-based practice guideline to further reduce COT to 32 hours for hospitalized infants. Key factors contributing to our success included multidisciplinary engagement, specifically partnering with nurses and resident physicians in designing and implementing our initiatives. Furthermore, improved transparency of culture monitoring practices allowed clinicians to better understand the recommended observation periods. Finally, we employed a standardized EHR order set as a no-cost, one-time, high-reliability intervention to establish 24 hours of culture monitoring as the default and to enhance transparency around start time for culture incubation.

Average COT remained stable at 32 hours for 17 months after initiation of the intervention. During the intervention period, 64% patients with hospital stays longer than 30 hours had cultures obtained between 9 pm to 4 am. These patients often remained hospitalized for longer than 30 hours to allow for a daytime hospital discharge. Additionally, CSF cultures were only monitored manually once per day between 8 am and 10 am. As a result, CSF cultures obtained in the evening (eg, 9 pm) would be evaluated once at roughly 12 hours of incubation, and then the following morning at 36 hours of incubation. In cases where CSF studies (eg, cell count, protein, Gram stain) were abnormal, uninterpretable, or could not be obtained, clinicians monitored CSF cultures closer to 36 hours from incubation. While evidence-based guidelines and local data support safe early discharge of febrile infants, clinicians presented with incomplete or uninterpretable data were appropriately more likely to observe infants for longer periods to confirm negative cultures.

Limitations

The study has several limitations. First, this single-center study was conducted at a quaternary care medical center with a robust quality improvement infrastructure. Our interventions took advantage of the existing processes in place that ensure timely discharge of medically ready patients.¹¹ Furthermore, microbiology laboratory practices are unique to our institution. These factors limit the generalizability of this work. Second, due to small numbers of eligible infants, analyses were conducted per five patients. Infrequent hospitalizations limited our ability to learn quickly from PDSA cycles. Finally, we did not measure cost savings attributable to shorter hospital stays. However, in addition to financial savings from charges and decreased nonmedical costs such as lost earnings and childcare,¹⁷ shorter hospitalizations have many additional benefits, such as promoting bonding and breastfeeding and decreasing exposure to nosocomial infections. Shorter hospitalizations, with clearly communicated discharge times, also serve to optimize patient throughput.

Implementation of a clinical practice guideline resulted in reduction of average COT from 38 to 32 hours in febrile infants aged 60 days and younger, with no cases of missed IBI. Engagement of multidisciplinary stakeholders in the generation and structured dissemination of the evidence-based guideline, improved transparency of the microbiological blood and CSF culture process, and standardization of EHR order sets were crucial to the success of this work. Cultures incubated overnight and daily CSF culture-monitoring practices primarily contributed to an average LOS of more than 30 hours.

Future work will include collaboration with emergency physicians to improve evaluation efficiency and decrease LOS in the ED for febrile infants. Additionally, creation of an automated data dashboard of COT and LOS will provide clinicians with real-time feedback on hospitalization practices.

The authors thank Dr Jeffrey Simmons, MD, MSc, as well as the members of the 2019 Fever of Uncertain Source Evidence-Based Guideline Committee. We also thank the James M Anderson Center for Health System Excellence and the Rapid Cycle Improvement Collaborative for their support with guideline development as well as design and execution of our improvement efforts.

Febrile infants aged 0 to 60 days often undergo diagnostic testing to evaluate for invasive bacterial infections (IBI; ie, bacteremia and meningitis) and are subsequently hospitalized pending culture results. Only 1% to 2% of infants 0 to 60 days old have an IBI,^1-3 and most hospitalized infants are discharged once physicians feel confident that pathogens are unlikely to be isolated from blood and cerebrospinal fluid (CSF) cultures. Practice regarding duration of hospitalization while awaiting blood and CSF culture results is not standardized in this population. Longer hospitalizations can lead to increased costs and familial stress, including difficulty with breastfeeding and anxiety in newly postpartum mothers.^4,5

In 2010, an institutional evidence-based guideline for the management of febrile infants aged 0 to 60 days recommended discharge after 36 hours of observation if all cultures were negative.⁶ However, recent studies demonstrate that 85% to 93% of pathogens in blood and CSF cultures grow within 24 hours of incubation.^7-9 Assuming a 2% prevalence of IBI, if 15% of pathogens were identified after 24 hours of incubation, only one out of 333 infants would have an IBI identified after 24 hours of hospital observation.⁷

Furthermore, a review of our institution’s electronic health records (EHR) over the past 5 years revealed that an observation period of 24 hours would have resulted in the discharge of three infants with an IBI. Two infants had bacteremia; both were discharged from the emergency department (ED) without antibiotics, returned to care after cultures were reported positive at 27 hours, and had no adverse outcomes. The third infant had meningitis, but also had an abnormal CSF Gram stain, which led to a longer hospitalization.

In 2019, our institution appraised the emerging literature and institutional data supporting the low absolute risk of missed IBI, and also leveraged local consensus among key stakeholders to update its evidence-based guideline for the evaluation and management of febrile infants aged 60 days and younger. The updated guideline recommends that clinicians consider discharging well-appearing neonates and infants if blood and CSF cultures remain negative at 24 hours.¹⁰ The objective of this study was to decrease the average hospital culture observation time (COT; culture incubation to hospital discharge) from 38 to 30 hours over a 12-month period in febrile infants aged 0 to 60 days.

Context

Improvement efforts were conducted at Cincinnati Children’s Hospital Medical Center (CCHMC), a large, urban, academic hospital that admitted more than 8,000 noncritically ill patients to the hospital medicine (HM) service from July 1, 2018, through June 30, 2019. Hospital medicine teams, located at both the main and satellite campuses, are staffed by attending physicians, fellows, residents, medical students, and nurse practitioners. The two campuses, which are about 20 miles apart, share clinician providers but have distinct nursing pools.

Microbiology services for all CCHMC patients are provided at the main campus. Blood and CSF cultures at the satellite campus are transported to the main campus for incubation and monitoring via an urgent courier service. The microbiology laboratory at CCHMC uses a continuous monitoring system for blood cultures (BACT/ALERT Virtuo, BioMérieux). The system automatically alerts laboratory technicians of positive cultures; these results are reported to clinical providers within 30 minutes of detection. Laboratory technicians manually evaluate CSF cultures once daily for 5 days.

Improvement Team

Our improvement team included three HM attending physicians; two HM fellows; a pediatric chief resident; two nurses, who represented nursing pools at the main and satellite campuses; and a clinical pharmacist, who is a co-leader of the antimicrobial stewardship program at CCHMC. Supporting members for the improvement team included the CCHMC laboratory director; the microbiology laboratory director; an infectious disease physician, who is a co-leader of the antimicrobial stewardship program; and nursing directors of the HM units at both campuses.

Evidence-Based Guideline

Our improvement initiative was based on recommendations from the updated CCHMC Evidence-Based Care Guideline for Management of Infants 0 to 60 days with Fever of Unknown Source.¹⁰ This guideline, published in May 2019, was developed by a multidisciplinary working group composed of key stakeholders from HM, community pediatrics, emergency medicine, the pediatric residency program, infectious disease, and laboratory medicine. Several improvement team members were participants on the committee that published the evidence-based guideline. The committee first performed a systematic literature review and critical appraisal of the literature. Care recommendations were formulated via a consensus process directed by best evidence, patient and family preferences, and clinical expertise; the recommendations were subsequently reviewed and approved by clinical experts who were not involved in the development process.

Based on evidence review and multistakeholder consensus, the updated guideline recommends clinicians consider discharging neonates and infants aged 60 days and younger if there is no culture growth after an observation period of 24 hours (as documented in the EHR) and patients are otherwise medically ready for discharge (ie, well appearing with adequate oral intake).^10,11 In addition, prior to discharge, there must be a documented working phone number on file for the patient’s parents/guardians, an established outpatient follow-up plan within 24 hours, and communication with the primary pediatrician who is in agreement with discharge at 24 hours.

Study Population

Infants 0 to 60 days old who had a documented or reported fever without an apparent source based on history and physical exam upon presentation to the ED, and who were subsequently admitted to the HM service at CCHMC between October 30, 2018, and July 10, 2020, were eligible for inclusion. We excluded infants who were admitted to other clinical services (eg, intensive care unit); had organisms identified on blood, urine, or CSF culture within 24 hours of incubation; had positive herpes simplex virus testing; had skin/soft tissue infections or another clearly documented source of bacterial infection; or had an alternative indication for hospitalization (eg, need for intravenous fluid or deep suctioning) after cultures had incubated for 24 hours. Infants who had a positive blood, urine, or CSF culture result after 24 hours of incubation were included in the study population. Organisms were classified as pathogen or contaminant based on treatment decisions made by the care team.

Improvement Activities

Key drivers critical to success of the improvement efforts were: (1) clearly defined standard of care for duration of observation in febrile infants 0 to 60 days old; (2) improved understanding of microbiology lab procedures; (3) effective communication of discharge criteria between providers and nurses; and (4) transparency of data with feedback (Figure 1).

Education and Structured Dissemination of Evidence-Based Guideline

The CCHMC febrile infant guideline¹⁰ was disseminated to HM physicians, residents, and nurses via the following means: (1) in-person announcements at staff meetings and educational conferences, (2) published highlights from the guideline in weekly newsletters, and (3) email announcements. Additionally, members of the study team educated HM attending physicians, nursing staff from the medical units at both campuses, and resident physicians about recent studies demonstrating safety of shorter length of stay (LOS) in febrile infants aged 0 to 60 days. The study team also provided residents, physicians, and nurses with data on the number of positive blood and CSF cultures and outcomes of patients at CCHMC within the past 5 years. In addition, team members led a journal club for residents discussing an article⁷ describing time-to-positivity of blood and CSF cultures in febrile infants. For ongoing engagement, the evidence-based guideline and a detailed explanation of microbiology procedures were published in the resident handbook, an internal resource that includes vital clinical pearls and practice guidelines across specialties. (Each resident receives an updated hard copy each year, and there is also an online link to the resource in the EHR.) Information about the guideline and COT was also included in the monthly chief resident’s orientation script, which is relayed to all residents on the first day of their HM rotation.

Clear Communication of Microbiology Procedures

Team members created a detailed process map describing the processing protocols for blood and CSF cultures collected at both CCHMC campuses. This information was shared with HM attending physicians and nurses via in-person announcements at staff meetings, flyers in team workrooms, and email communications. Residents received information on microbiology protocols via in-person announcements at educational conferences and dissemination in the weekly residency newsletter.Important information communicated included:

1. Definition of culture start time. We conveyed that there may be a delay of up to 4 hours between culture collection at the satellite campus and culture incubation at the main campus laboratory. As a result, the time of blood or CSF sample arrival to the main campus laboratory was a more accurate reflection of the culture incubation start time than the culture collection time.

2. Explanation of CSF culture processing. We discussed the process by which these cultures are plated upon arrival at the microbiology laboratory and read once per day in the morning. Therefore, a culture incubated at midnight would be evaluated once at 9 hours and not again until 33 hours.

Modification of Febrile Infant Order Set

Enhancements to the febrile infant order set improved communication and cultivated a shared mental model regarding discharge goals among all members of the care team. The EHR order set for febrile infants was updated as follows: (1) mandatory free-text fields that established the culture start time for blood and CSF cultures were added, (2) culture start time was clearly defined (ie, the time culture arrives at the main campus laboratory), and (3) a change was made in the default discharge criteria¹¹ to “culture observation for 24 hours,” with the ability to modify COT (Appendix Figure 1). We embedded hyperlinks to the guideline and microbiology process map within the updated order set, which allowed providers to easily access this information and refresh their knowledge of the recommendations (Appendix Figure 1).

Identification of Failures and Follow-up With Near-Time Feedback

All cases of febrile infants were tracked weekly. For infants hospitalized longer than 24 hours, the study team contacted the discharging clinicians to discuss reasons for prolonged hospitalization, with an emphasis on identifying system-level barriers to earlier discharge.

Study of the Interventions

The institutional microbiology database was queried weekly to identify all infants 0 to 60 days old who had a blood culture obtained and were hospitalized on the HM service. Study team members conducted targeted EHR review to determine whether patients met exclusion criteria and to identify reasons for prolonged COT. Baseline data were collected retrospectively for a 3-month period prior to initiation of improvement activities. During the study period, queries were conducted weekly and reviewed by study team members to evaluate the impact of improvement activities and to inform new interventions.

Measures

Our primary outcome measure was COT, defined as the hours between final culture incubation and hospital discharge. The operational definition for “final culture incubation” was the documented time of arrival of the last collected culture to the microbiology laboratory. Our goal COT was 30 hours to account for a subset of patients whose blood and/or CSF culture were obtained overnight (ie, after 9 pm), since subsequent discharge times would likely and practically be delayed beyond 24 hours. Our secondary outcome measure was LOS, defined as the time between ED arrival and hospital discharge. Process measures included the proportion of patients for whom the febrile infant EHR order set was used and the proportion of patients for whom medical discharge criteria (ie, blood and CSF culture observed for ”xx” hours) and culture incubation start times were entered using the order set. Balancing measures included identification of IBI after hospital discharge, 48-hour ED revisits, and 7-day hospital readmissions.

Analysis

Measures were evaluated using statistical process control charts and run charts, and Western Electric rules were employed to determine special cause variation.¹² Annotated X-bar S control charts tracked the impact of improvement activities on average COT and LOS for all infants. Given that a relatively small number of patients (ie, two to four) met inclusion criteria each week, average COT was calculated per five patients.

This study was considered exempt from review by the CCHMC Institutional Review Board.

Of the 184 infants in this study, 46 were included as part of baseline data collection, and 138 were included during the intervention period. The median age was 26.6 days (range, 3-59 days); 52% of patients were female; two-thirds were non-Hispanic White; 22% were Black, and 5% were Hispanic (Appendix Table).

Average COT decreased from 38 hours to 32 hours with improvement activities (Figure 2) and was sustained for a total of 17 months. There were small decreases in COT after initial education was provided to attendings, nurses, and residents.

After the launch of the updated order set, median usage of the EHR order set increased from 50% to 80%. Medical discharge criteria were entered for 80 (96%) of the 83 patients for whom the updated order set was applied; culture incubation start times were entered for 78 (94%) of these patients.

No infants in our cohort were found to have IBI after hospital discharge. There were no ED revisits within 48 hours of discharge, and there were no hospital readmissions within 7 days of index discharge. Furthermore, none of the patients included in the study had growth of a pathogenic organism after 24 hours.

Of the 138 infants hospitalized during the intervention period, 77 (56%) had a COT greater than 30 hours. Among these 77 patients, 49 (64%) had their final culture incubated between 9 pm and 4 am; Furthermore, 11 (14%) had missing, abnormal, pretreated, or uninterpretable CSF studies, 7 (9%) had ongoing fevers, and 4 (5%) remained hospitalized due to family preference or inability to obtain timely outpatient follow-up.

Our study aimed to decrease the average COT from 38 hours to 30 hours among hospitalized infants aged 60 days and younger over a period of 12 months. An intervention featuring implementation of an evidence-based guideline through education, laboratory procedure transparency, creation of a standardized EHR order set, and near-time feedback was associated with a shorter average COT of 32 hours, sustained over a 17-month period. No infants with bacteremia or meningitis were inappropriately discharged during this study.

Interpretation

Prior to our improvement efforts, most febrile infants at CCHMC were observed for at least 36 hours based on a prior institutional guideline,⁶ despite recent evidence suggesting that most pathogens in blood and CSF cultures grow within 24 hours of incubation.^7-9 The goal of this improvement initiative was to bridge the gap between emerging evidence and clinical practice by developing and disseminating an updated evidence-based guideline to safely decrease the hospital observation time in febrile infants aged 60 days and younger.

Similar to previous studies aimed at improving diagnosis and management among febrile infants,^13-16 generation and structured dissemination of an institutional evidence-based guideline was crucial to safely shortening COT in our population. These prior studies established a goal COT of 36 to 42 hours for hospitalized febrile infants.^13,15,16 Our study incorporated emerging evidence and local experience into an updated evidence-based practice guideline to further reduce COT to 32 hours for hospitalized infants. Key factors contributing to our success included multidisciplinary engagement, specifically partnering with nurses and resident physicians in designing and implementing our initiatives. Furthermore, improved transparency of culture monitoring practices allowed clinicians to better understand the recommended observation periods. Finally, we employed a standardized EHR order set as a no-cost, one-time, high-reliability intervention to establish 24 hours of culture monitoring as the default and to enhance transparency around start time for culture incubation.

Average COT remained stable at 32 hours for 17 months after initiation of the intervention. During the intervention period, 64% patients with hospital stays longer than 30 hours had cultures obtained between 9 pm to 4 am. These patients often remained hospitalized for longer than 30 hours to allow for a daytime hospital discharge. Additionally, CSF cultures were only monitored manually once per day between 8 am and 10 am. As a result, CSF cultures obtained in the evening (eg, 9 pm) would be evaluated once at roughly 12 hours of incubation, and then the following morning at 36 hours of incubation. In cases where CSF studies (eg, cell count, protein, Gram stain) were abnormal, uninterpretable, or could not be obtained, clinicians monitored CSF cultures closer to 36 hours from incubation. While evidence-based guidelines and local data support safe early discharge of febrile infants, clinicians presented with incomplete or uninterpretable data were appropriately more likely to observe infants for longer periods to confirm negative cultures.

Limitations

The study has several limitations. First, this single-center study was conducted at a quaternary care medical center with a robust quality improvement infrastructure. Our interventions took advantage of the existing processes in place that ensure timely discharge of medically ready patients.¹¹ Furthermore, microbiology laboratory practices are unique to our institution. These factors limit the generalizability of this work. Second, due to small numbers of eligible infants, analyses were conducted per five patients. Infrequent hospitalizations limited our ability to learn quickly from PDSA cycles. Finally, we did not measure cost savings attributable to shorter hospital stays. However, in addition to financial savings from charges and decreased nonmedical costs such as lost earnings and childcare,¹⁷ shorter hospitalizations have many additional benefits, such as promoting bonding and breastfeeding and decreasing exposure to nosocomial infections. Shorter hospitalizations, with clearly communicated discharge times, also serve to optimize patient throughput.

Implementation of a clinical practice guideline resulted in reduction of average COT from 38 to 32 hours in febrile infants aged 60 days and younger, with no cases of missed IBI. Engagement of multidisciplinary stakeholders in the generation and structured dissemination of the evidence-based guideline, improved transparency of the microbiological blood and CSF culture process, and standardization of EHR order sets were crucial to the success of this work. Cultures incubated overnight and daily CSF culture-monitoring practices primarily contributed to an average LOS of more than 30 hours.

Future work will include collaboration with emergency physicians to improve evaluation efficiency and decrease LOS in the ED for febrile infants. Additionally, creation of an automated data dashboard of COT and LOS will provide clinicians with real-time feedback on hospitalization practices.

The authors thank Dr Jeffrey Simmons, MD, MSc, as well as the members of the 2019 Fever of Uncertain Source Evidence-Based Guideline Committee. We also thank the James M Anderson Center for Health System Excellence and the Rapid Cycle Improvement Collaborative for their support with guideline development as well as design and execution of our improvement efforts.

Febrile infants aged 0 to 60 days often undergo diagnostic testing to evaluate for invasive bacterial infections (IBI; ie, bacteremia and meningitis) and are subsequently hospitalized pending culture results. Only 1% to 2% of infants 0 to 60 days old have an IBI,^1-3 and most hospitalized infants are discharged once physicians feel confident that pathogens are unlikely to be isolated from blood and cerebrospinal fluid (CSF) cultures. Practice regarding duration of hospitalization while awaiting blood and CSF culture results is not standardized in this population. Longer hospitalizations can lead to increased costs and familial stress, including difficulty with breastfeeding and anxiety in newly postpartum mothers.^4,5

In 2010, an institutional evidence-based guideline for the management of febrile infants aged 0 to 60 days recommended discharge after 36 hours of observation if all cultures were negative.⁶ However, recent studies demonstrate that 85% to 93% of pathogens in blood and CSF cultures grow within 24 hours of incubation.^7-9 Assuming a 2% prevalence of IBI, if 15% of pathogens were identified after 24 hours of incubation, only one out of 333 infants would have an IBI identified after 24 hours of hospital observation.⁷

Furthermore, a review of our institution’s electronic health records (EHR) over the past 5 years revealed that an observation period of 24 hours would have resulted in the discharge of three infants with an IBI. Two infants had bacteremia; both were discharged from the emergency department (ED) without antibiotics, returned to care after cultures were reported positive at 27 hours, and had no adverse outcomes. The third infant had meningitis, but also had an abnormal CSF Gram stain, which led to a longer hospitalization.

In 2019, our institution appraised the emerging literature and institutional data supporting the low absolute risk of missed IBI, and also leveraged local consensus among key stakeholders to update its evidence-based guideline for the evaluation and management of febrile infants aged 60 days and younger. The updated guideline recommends that clinicians consider discharging well-appearing neonates and infants if blood and CSF cultures remain negative at 24 hours.¹⁰ The objective of this study was to decrease the average hospital culture observation time (COT; culture incubation to hospital discharge) from 38 to 30 hours over a 12-month period in febrile infants aged 0 to 60 days.

Context

Improvement efforts were conducted at Cincinnati Children’s Hospital Medical Center (CCHMC), a large, urban, academic hospital that admitted more than 8,000 noncritically ill patients to the hospital medicine (HM) service from July 1, 2018, through June 30, 2019. Hospital medicine teams, located at both the main and satellite campuses, are staffed by attending physicians, fellows, residents, medical students, and nurse practitioners. The two campuses, which are about 20 miles apart, share clinician providers but have distinct nursing pools.

Microbiology services for all CCHMC patients are provided at the main campus. Blood and CSF cultures at the satellite campus are transported to the main campus for incubation and monitoring via an urgent courier service. The microbiology laboratory at CCHMC uses a continuous monitoring system for blood cultures (BACT/ALERT Virtuo, BioMérieux). The system automatically alerts laboratory technicians of positive cultures; these results are reported to clinical providers within 30 minutes of detection. Laboratory technicians manually evaluate CSF cultures once daily for 5 days.

Improvement Team

Our improvement team included three HM attending physicians; two HM fellows; a pediatric chief resident; two nurses, who represented nursing pools at the main and satellite campuses; and a clinical pharmacist, who is a co-leader of the antimicrobial stewardship program at CCHMC. Supporting members for the improvement team included the CCHMC laboratory director; the microbiology laboratory director; an infectious disease physician, who is a co-leader of the antimicrobial stewardship program; and nursing directors of the HM units at both campuses.

Evidence-Based Guideline

Our improvement initiative was based on recommendations from the updated CCHMC Evidence-Based Care Guideline for Management of Infants 0 to 60 days with Fever of Unknown Source.¹⁰ This guideline, published in May 2019, was developed by a multidisciplinary working group composed of key stakeholders from HM, community pediatrics, emergency medicine, the pediatric residency program, infectious disease, and laboratory medicine. Several improvement team members were participants on the committee that published the evidence-based guideline. The committee first performed a systematic literature review and critical appraisal of the literature. Care recommendations were formulated via a consensus process directed by best evidence, patient and family preferences, and clinical expertise; the recommendations were subsequently reviewed and approved by clinical experts who were not involved in the development process.

Based on evidence review and multistakeholder consensus, the updated guideline recommends clinicians consider discharging neonates and infants aged 60 days and younger if there is no culture growth after an observation period of 24 hours (as documented in the EHR) and patients are otherwise medically ready for discharge (ie, well appearing with adequate oral intake).^10,11 In addition, prior to discharge, there must be a documented working phone number on file for the patient’s parents/guardians, an established outpatient follow-up plan within 24 hours, and communication with the primary pediatrician who is in agreement with discharge at 24 hours.

Study Population

Infants 0 to 60 days old who had a documented or reported fever without an apparent source based on history and physical exam upon presentation to the ED, and who were subsequently admitted to the HM service at CCHMC between October 30, 2018, and July 10, 2020, were eligible for inclusion. We excluded infants who were admitted to other clinical services (eg, intensive care unit); had organisms identified on blood, urine, or CSF culture within 24 hours of incubation; had positive herpes simplex virus testing; had skin/soft tissue infections or another clearly documented source of bacterial infection; or had an alternative indication for hospitalization (eg, need for intravenous fluid or deep suctioning) after cultures had incubated for 24 hours. Infants who had a positive blood, urine, or CSF culture result after 24 hours of incubation were included in the study population. Organisms were classified as pathogen or contaminant based on treatment decisions made by the care team.

Improvement Activities

Key drivers critical to success of the improvement efforts were: (1) clearly defined standard of care for duration of observation in febrile infants 0 to 60 days old; (2) improved understanding of microbiology lab procedures; (3) effective communication of discharge criteria between providers and nurses; and (4) transparency of data with feedback (Figure 1).

Education and Structured Dissemination of Evidence-Based Guideline

The CCHMC febrile infant guideline¹⁰ was disseminated to HM physicians, residents, and nurses via the following means: (1) in-person announcements at staff meetings and educational conferences, (2) published highlights from the guideline in weekly newsletters, and (3) email announcements. Additionally, members of the study team educated HM attending physicians, nursing staff from the medical units at both campuses, and resident physicians about recent studies demonstrating safety of shorter length of stay (LOS) in febrile infants aged 0 to 60 days. The study team also provided residents, physicians, and nurses with data on the number of positive blood and CSF cultures and outcomes of patients at CCHMC within the past 5 years. In addition, team members led a journal club for residents discussing an article⁷ describing time-to-positivity of blood and CSF cultures in febrile infants. For ongoing engagement, the evidence-based guideline and a detailed explanation of microbiology procedures were published in the resident handbook, an internal resource that includes vital clinical pearls and practice guidelines across specialties. (Each resident receives an updated hard copy each year, and there is also an online link to the resource in the EHR.) Information about the guideline and COT was also included in the monthly chief resident’s orientation script, which is relayed to all residents on the first day of their HM rotation.

Clear Communication of Microbiology Procedures

Team members created a detailed process map describing the processing protocols for blood and CSF cultures collected at both CCHMC campuses. This information was shared with HM attending physicians and nurses via in-person announcements at staff meetings, flyers in team workrooms, and email communications. Residents received information on microbiology protocols via in-person announcements at educational conferences and dissemination in the weekly residency newsletter.Important information communicated included:

1. Definition of culture start time. We conveyed that there may be a delay of up to 4 hours between culture collection at the satellite campus and culture incubation at the main campus laboratory. As a result, the time of blood or CSF sample arrival to the main campus laboratory was a more accurate reflection of the culture incubation start time than the culture collection time.

2. Explanation of CSF culture processing. We discussed the process by which these cultures are plated upon arrival at the microbiology laboratory and read once per day in the morning. Therefore, a culture incubated at midnight would be evaluated once at 9 hours and not again until 33 hours.

Modification of Febrile Infant Order Set

Enhancements to the febrile infant order set improved communication and cultivated a shared mental model regarding discharge goals among all members of the care team. The EHR order set for febrile infants was updated as follows: (1) mandatory free-text fields that established the culture start time for blood and CSF cultures were added, (2) culture start time was clearly defined (ie, the time culture arrives at the main campus laboratory), and (3) a change was made in the default discharge criteria¹¹ to “culture observation for 24 hours,” with the ability to modify COT (Appendix Figure 1). We embedded hyperlinks to the guideline and microbiology process map within the updated order set, which allowed providers to easily access this information and refresh their knowledge of the recommendations (Appendix Figure 1).

Identification of Failures and Follow-up With Near-Time Feedback

All cases of febrile infants were tracked weekly. For infants hospitalized longer than 24 hours, the study team contacted the discharging clinicians to discuss reasons for prolonged hospitalization, with an emphasis on identifying system-level barriers to earlier discharge.

Study of the Interventions

The institutional microbiology database was queried weekly to identify all infants 0 to 60 days old who had a blood culture obtained and were hospitalized on the HM service. Study team members conducted targeted EHR review to determine whether patients met exclusion criteria and to identify reasons for prolonged COT. Baseline data were collected retrospectively for a 3-month period prior to initiation of improvement activities. During the study period, queries were conducted weekly and reviewed by study team members to evaluate the impact of improvement activities and to inform new interventions.

Measures

Our primary outcome measure was COT, defined as the hours between final culture incubation and hospital discharge. The operational definition for “final culture incubation” was the documented time of arrival of the last collected culture to the microbiology laboratory. Our goal COT was 30 hours to account for a subset of patients whose blood and/or CSF culture were obtained overnight (ie, after 9 pm), since subsequent discharge times would likely and practically be delayed beyond 24 hours. Our secondary outcome measure was LOS, defined as the time between ED arrival and hospital discharge. Process measures included the proportion of patients for whom the febrile infant EHR order set was used and the proportion of patients for whom medical discharge criteria (ie, blood and CSF culture observed for ”xx” hours) and culture incubation start times were entered using the order set. Balancing measures included identification of IBI after hospital discharge, 48-hour ED revisits, and 7-day hospital readmissions.

Analysis

Measures were evaluated using statistical process control charts and run charts, and Western Electric rules were employed to determine special cause variation.¹² Annotated X-bar S control charts tracked the impact of improvement activities on average COT and LOS for all infants. Given that a relatively small number of patients (ie, two to four) met inclusion criteria each week, average COT was calculated per five patients.

This study was considered exempt from review by the CCHMC Institutional Review Board.

Of the 184 infants in this study, 46 were included as part of baseline data collection, and 138 were included during the intervention period. The median age was 26.6 days (range, 3-59 days); 52% of patients were female; two-thirds were non-Hispanic White; 22% were Black, and 5% were Hispanic (Appendix Table).

Average COT decreased from 38 hours to 32 hours with improvement activities (Figure 2) and was sustained for a total of 17 months. There were small decreases in COT after initial education was provided to attendings, nurses, and residents.

After the launch of the updated order set, median usage of the EHR order set increased from 50% to 80%. Medical discharge criteria were entered for 80 (96%) of the 83 patients for whom the updated order set was applied; culture incubation start times were entered for 78 (94%) of these patients.

No infants in our cohort were found to have IBI after hospital discharge. There were no ED revisits within 48 hours of discharge, and there were no hospital readmissions within 7 days of index discharge. Furthermore, none of the patients included in the study had growth of a pathogenic organism after 24 hours.

Of the 138 infants hospitalized during the intervention period, 77 (56%) had a COT greater than 30 hours. Among these 77 patients, 49 (64%) had their final culture incubated between 9 pm and 4 am; Furthermore, 11 (14%) had missing, abnormal, pretreated, or uninterpretable CSF studies, 7 (9%) had ongoing fevers, and 4 (5%) remained hospitalized due to family preference or inability to obtain timely outpatient follow-up.

Our study aimed to decrease the average COT from 38 hours to 30 hours among hospitalized infants aged 60 days and younger over a period of 12 months. An intervention featuring implementation of an evidence-based guideline through education, laboratory procedure transparency, creation of a standardized EHR order set, and near-time feedback was associated with a shorter average COT of 32 hours, sustained over a 17-month period. No infants with bacteremia or meningitis were inappropriately discharged during this study.

Interpretation

Prior to our improvement efforts, most febrile infants at CCHMC were observed for at least 36 hours based on a prior institutional guideline,⁶ despite recent evidence suggesting that most pathogens in blood and CSF cultures grow within 24 hours of incubation.^7-9 The goal of this improvement initiative was to bridge the gap between emerging evidence and clinical practice by developing and disseminating an updated evidence-based guideline to safely decrease the hospital observation time in febrile infants aged 60 days and younger.

Similar to previous studies aimed at improving diagnosis and management among febrile infants,^13-16 generation and structured dissemination of an institutional evidence-based guideline was crucial to safely shortening COT in our population. These prior studies established a goal COT of 36 to 42 hours for hospitalized febrile infants.^13,15,16 Our study incorporated emerging evidence and local experience into an updated evidence-based practice guideline to further reduce COT to 32 hours for hospitalized infants. Key factors contributing to our success included multidisciplinary engagement, specifically partnering with nurses and resident physicians in designing and implementing our initiatives. Furthermore, improved transparency of culture monitoring practices allowed clinicians to better understand the recommended observation periods. Finally, we employed a standardized EHR order set as a no-cost, one-time, high-reliability intervention to establish 24 hours of culture monitoring as the default and to enhance transparency around start time for culture incubation.

Average COT remained stable at 32 hours for 17 months after initiation of the intervention. During the intervention period, 64% patients with hospital stays longer than 30 hours had cultures obtained between 9 pm to 4 am. These patients often remained hospitalized for longer than 30 hours to allow for a daytime hospital discharge. Additionally, CSF cultures were only monitored manually once per day between 8 am and 10 am. As a result, CSF cultures obtained in the evening (eg, 9 pm) would be evaluated once at roughly 12 hours of incubation, and then the following morning at 36 hours of incubation. In cases where CSF studies (eg, cell count, protein, Gram stain) were abnormal, uninterpretable, or could not be obtained, clinicians monitored CSF cultures closer to 36 hours from incubation. While evidence-based guidelines and local data support safe early discharge of febrile infants, clinicians presented with incomplete or uninterpretable data were appropriately more likely to observe infants for longer periods to confirm negative cultures.

Limitations

The study has several limitations. First, this single-center study was conducted at a quaternary care medical center with a robust quality improvement infrastructure. Our interventions took advantage of the existing processes in place that ensure timely discharge of medically ready patients.¹¹ Furthermore, microbiology laboratory practices are unique to our institution. These factors limit the generalizability of this work. Second, due to small numbers of eligible infants, analyses were conducted per five patients. Infrequent hospitalizations limited our ability to learn quickly from PDSA cycles. Finally, we did not measure cost savings attributable to shorter hospital stays. However, in addition to financial savings from charges and decreased nonmedical costs such as lost earnings and childcare,¹⁷ shorter hospitalizations have many additional benefits, such as promoting bonding and breastfeeding and decreasing exposure to nosocomial infections. Shorter hospitalizations, with clearly communicated discharge times, also serve to optimize patient throughput.

Implementation of a clinical practice guideline resulted in reduction of average COT from 38 to 32 hours in febrile infants aged 60 days and younger, with no cases of missed IBI. Engagement of multidisciplinary stakeholders in the generation and structured dissemination of the evidence-based guideline, improved transparency of the microbiological blood and CSF culture process, and standardization of EHR order sets were crucial to the success of this work. Cultures incubated overnight and daily CSF culture-monitoring practices primarily contributed to an average LOS of more than 30 hours.

Future work will include collaboration with emergency physicians to improve evaluation efficiency and decrease LOS in the ED for febrile infants. Additionally, creation of an automated data dashboard of COT and LOS will provide clinicians with real-time feedback on hospitalization practices.

The authors thank Dr Jeffrey Simmons, MD, MSc, as well as the members of the 2019 Fever of Uncertain Source Evidence-Based Guideline Committee. We also thank the James M Anderson Center for Health System Excellence and the Rapid Cycle Improvement Collaborative for their support with guideline development as well as design and execution of our improvement efforts.

In July 2020, the revision of The Pediatric Hospital Medicine Core Competencies was published, bringing to fruition three years of meticulous work.¹ The working group produced 66 chapters outlining the knowledge, skills, and attitudes needed for competent pediatric hospitalist practice. The arrival of these competencies is especially prescient given pediatric hospital medicine’s (PHM’s) relatively new standing as an American Board of Medical Specialties certified subspecialty, as the competencies can serve as a guide for improvement of fellowship curricula, assessment systems, and faculty development. The competencies also represent an opportunity for PHM to take a bold step forward in the world of graduate medical education (GME) by realizing a key tenet of competency-based medical education (CBME)—competency-based, time-variable training (CBTVT), in which learners train until competence is achieved rather than for a predetermined duration.^2,3 In this perspective, we describe how medical education in the United States adopted a time-based training paradigm (in which time-in-training is a surrogate for competence), how CBME has brought time-variable training to the fore, and how PHM has an opportunity to be on the leading edge of education innovation.

In the 1800s, during the time of the “Wild West,” medical education in the United States matched this moniker. There was little standardization across the multiple training pathways to become a practicing physician, including apprenticeships, lecture series, and university courses.⁴ Predictably, this led to significant heterogeneity in the quality of medical care that a patient of the day received. This problem became clearer as Americans traveled to Europe and witnessed more structured and rigorous training programs, only to return to the comparatively poor state of medical education back home.⁵ There was a clear need for curricular standardization.

In 1876, the American Medical College Association (which later became the Association of American Medical Colleges [AAMC]) was founded to meet this need, and in 1905 the Association adopted a set of minimum standards for medical training that included the now-familiar two years of basic sciences and two years of clinical training.⁶ Two subsequent national surveys in the United States were commissioned to evaluate whether medical schools met this new standard, with both surveys finding that roughly half of existing programs passed muster.^7,8 As a result, nearly half of US medical schools had closed by 1920 in a crusade to standardize curricula and produce competent physicians. By the time the American Medical Association established initial standards for internship (an archetype of GME),⁴ time-based medical training was the dominant paradigm. This historical perspective highlights the rationale for standardization of education processes and curricula, particularly in terms of accountability to the American public. But heralded by the 1978 landmark paper by McGaghie et al,⁹ the paradigm began to shift in the late twentieth century from a focus on the process of physician training to outcomes.

In contrast to the process-focused model of the early 1900s, CBME starts by identifying patient and healthcare system needs, defining competencies required to meet those needs, and then designing curricular and assessment processes to help learners achieve those competencies.² This outcomes-based approach grew as a response to calls for greater accountability to the public due to evidence that some graduates were unprepared for unsupervised practice, raising concerns that strictly time-based training was no longer defensible.¹⁰ CBME aims to mitigate these concerns by starting with desired outcomes of training and working backward to ensure those outcomes are met.

While many programs have attempted to implement CBME, most still rely heavily on time-in-training to determine competence. Learners participate in structured curricula and, unless they are extreme outliers, are deemed ready for unsupervised practice after a predetermined duration. This model presumes that competence and time are related in a fixed, predictable manner and that learners gain competence at a uniform rate. However, learners do not, in fact, progress uniformly. A study by Schumacher et al¹¹ involving 23 pediatric residency programs showed significant interlearner variability in rates of entrustment (used as a surrogate for competence), leading the authors to call for time-variable training in GME. Significant interlearner variation in rates of competence attainment have been shown in other specialties as well.¹² As more CBME studies on training outcomes emerge, the evidence is mounting that not all learners need the same duration of training to become competent providers. Time-in-training and competence attainment are not related in a fixed manner. As Dr Jason¹³ wrote in 1969, “By making time a constant, we make achievement a variable.” Variable achievement (competence, outcomes) was the very driver for medical education’s shift to a competency-based approach. If variable competence was not acceptable then, why should it be now? The goal of CBTVT is not shorter training, but rather flexible, individualized training both in terms of content and duration. While this also means some learners may need to extend their training, this should already be part of GME programs that are required to have remediation policies for learners who are not progressing as expected.

Time variability is an oft-cited tenet of CBME,^2,3 but one that is being piloted by relatively few programs in the United States, mostly in undergraduate medical education (UME).^14-16 The Education in Pediatrics Across the Continuum (EPAC), a consortium consisting of four institutions piloting CBTVT in UME,¹⁴ has shown early evidence of feasibility¹⁷ and that UME graduates from CBTVT programs enter residency with levels of competence similar to those of graduates of traditional time-based programs.¹⁸ We believe that PHM can take a step toward truly realizing CBME by implementing CBTVT in fellowship programs.

There are multiple reasons why this is an opportune time for PHM fellowships to consider CBTVT. First, PHM is a relatively new board-certified subspecialty with a recently revised set of core competencies¹ that are likely to catalyze programmatic innovation. A key step in change management is building on previous efforts to generate more change.¹⁹ Programs can leverage the momentum from current and impending change initiatives to innovate and implement CBTVT. Second, the revised PHM competencies provide the first crucial step in implementing a CBME program by defining desired training outcomes necessary to deliver high-quality patient care. With PHM competencies now well defined, programs can focus on developing programs of assessment and corresponding faculty development, which can help deliver valid, defensible decisions about fellow competence.

Finally, PHM has a workforce that can support CBTVT. A major barrier to time-variable training in GME is the need for trainees-as-workforce. In many GME programs, residents and fellows provide a relatively inexpensive, renewable workforce. Trainees’ clinical rotations are often scheduled up to 1 year in advance to ensure care teams are fully staffed, particularly in the inpatient setting, creating a system where flexibility in training is impossible without creating gaps in clinical coverage. However, many PHM fellowships do not completely rely on fellows to cover clinical service lines. PHM fellows spend 32 weeks over 2 years in core clinical rotations with faculty supervision, in accordance with the Accreditation Council for Graduate Medical Education program requirements, both for 2- and 3-year programs. Some CBME experts estimate (based on previous and ongoing CBTVT pilots) that training duration is likely to vary by roughly 20% from current time-based practices when CBTVT is initially implemented.²⁰ Thus, only a small number of clinical service weeks are likely to be affected. If a fellow were deemed ready for unsupervised practice before finishing 2 years of fellowship in a CBTVT program, the corresponding faculty supervisor could use the time previously assigned for supervision to pursue other priorities, such as education, scholarship, or quality improvement. Why provide supervision if a clinical competency committee has deemed a fellow ready for unsupervised practice? Some level of observation and formative feedback could continue, but full supervision would be redundant and unnecessary. CBTVT would allow for some fellows to experience the uncertainty that comes with unsupervised decision-making while still in an environment with trusted fellowship mentors and advisors.

PHM fellowship programs likely cannot flip a switch to “turn on” CBTVT immediately, but they can take steps toward making the transition. Validity, or defensibility of decisions, will be crucial for assessment in CBTVT systems. Programs will need to develop robust assessment systems that collect myriad data to answer the question, “When is this learner competent to deliver high-quality care without supervision?” Programs can align assessment instruments, faculty-development initiatives, and clinical competency committee (CCC) processes with the 2020 PHM competencies to provide a defensible answer. Program leaders should then seek validity evidence, either in existing literature or through novel scholarly initiatives, to support these summative decisions. Engaging all fellowship stakeholders in transitions to CBTVT will be important and should include fellows, program directors, CCC members, clinical leadership, and members from accrediting and credentialing bodies.

As fellowship programs review and revise curricula and assessment systems around the updated PHM core competencies, they should also consider what changes are necessary to implement CBTVT. Time variability is not a novelty but, rather, is a corollary to the outcomes-based approach of CBME. PHM fellowships should strike while the iron is hot and build on current change initiatives prompted by the growth of our specialty to be leaders in CBTVT.

In July 2020, the revision of The Pediatric Hospital Medicine Core Competencies was published, bringing to fruition three years of meticulous work.¹ The working group produced 66 chapters outlining the knowledge, skills, and attitudes needed for competent pediatric hospitalist practice. The arrival of these competencies is especially prescient given pediatric hospital medicine’s (PHM’s) relatively new standing as an American Board of Medical Specialties certified subspecialty, as the competencies can serve as a guide for improvement of fellowship curricula, assessment systems, and faculty development. The competencies also represent an opportunity for PHM to take a bold step forward in the world of graduate medical education (GME) by realizing a key tenet of competency-based medical education (CBME)—competency-based, time-variable training (CBTVT), in which learners train until competence is achieved rather than for a predetermined duration.^2,3 In this perspective, we describe how medical education in the United States adopted a time-based training paradigm (in which time-in-training is a surrogate for competence), how CBME has brought time-variable training to the fore, and how PHM has an opportunity to be on the leading edge of education innovation.

In the 1800s, during the time of the “Wild West,” medical education in the United States matched this moniker. There was little standardization across the multiple training pathways to become a practicing physician, including apprenticeships, lecture series, and university courses.⁴ Predictably, this led to significant heterogeneity in the quality of medical care that a patient of the day received. This problem became clearer as Americans traveled to Europe and witnessed more structured and rigorous training programs, only to return to the comparatively poor state of medical education back home.⁵ There was a clear need for curricular standardization.

In 1876, the American Medical College Association (which later became the Association of American Medical Colleges [AAMC]) was founded to meet this need, and in 1905 the Association adopted a set of minimum standards for medical training that included the now-familiar two years of basic sciences and two years of clinical training.⁶ Two subsequent national surveys in the United States were commissioned to evaluate whether medical schools met this new standard, with both surveys finding that roughly half of existing programs passed muster.^7,8 As a result, nearly half of US medical schools had closed by 1920 in a crusade to standardize curricula and produce competent physicians. By the time the American Medical Association established initial standards for internship (an archetype of GME),⁴ time-based medical training was the dominant paradigm. This historical perspective highlights the rationale for standardization of education processes and curricula, particularly in terms of accountability to the American public. But heralded by the 1978 landmark paper by McGaghie et al,⁹ the paradigm began to shift in the late twentieth century from a focus on the process of physician training to outcomes.

In contrast to the process-focused model of the early 1900s, CBME starts by identifying patient and healthcare system needs, defining competencies required to meet those needs, and then designing curricular and assessment processes to help learners achieve those competencies.² This outcomes-based approach grew as a response to calls for greater accountability to the public due to evidence that some graduates were unprepared for unsupervised practice, raising concerns that strictly time-based training was no longer defensible.¹⁰ CBME aims to mitigate these concerns by starting with desired outcomes of training and working backward to ensure those outcomes are met.

While many programs have attempted to implement CBME, most still rely heavily on time-in-training to determine competence. Learners participate in structured curricula and, unless they are extreme outliers, are deemed ready for unsupervised practice after a predetermined duration. This model presumes that competence and time are related in a fixed, predictable manner and that learners gain competence at a uniform rate. However, learners do not, in fact, progress uniformly. A study by Schumacher et al¹¹ involving 23 pediatric residency programs showed significant interlearner variability in rates of entrustment (used as a surrogate for competence), leading the authors to call for time-variable training in GME. Significant interlearner variation in rates of competence attainment have been shown in other specialties as well.¹² As more CBME studies on training outcomes emerge, the evidence is mounting that not all learners need the same duration of training to become competent providers. Time-in-training and competence attainment are not related in a fixed manner. As Dr Jason¹³ wrote in 1969, “By making time a constant, we make achievement a variable.” Variable achievement (competence, outcomes) was the very driver for medical education’s shift to a competency-based approach. If variable competence was not acceptable then, why should it be now? The goal of CBTVT is not shorter training, but rather flexible, individualized training both in terms of content and duration. While this also means some learners may need to extend their training, this should already be part of GME programs that are required to have remediation policies for learners who are not progressing as expected.

Time variability is an oft-cited tenet of CBME,^2,3 but one that is being piloted by relatively few programs in the United States, mostly in undergraduate medical education (UME).^14-16 The Education in Pediatrics Across the Continuum (EPAC), a consortium consisting of four institutions piloting CBTVT in UME,¹⁴ has shown early evidence of feasibility¹⁷ and that UME graduates from CBTVT programs enter residency with levels of competence similar to those of graduates of traditional time-based programs.¹⁸ We believe that PHM can take a step toward truly realizing CBME by implementing CBTVT in fellowship programs.

There are multiple reasons why this is an opportune time for PHM fellowships to consider CBTVT. First, PHM is a relatively new board-certified subspecialty with a recently revised set of core competencies¹ that are likely to catalyze programmatic innovation. A key step in change management is building on previous efforts to generate more change.¹⁹ Programs can leverage the momentum from current and impending change initiatives to innovate and implement CBTVT. Second, the revised PHM competencies provide the first crucial step in implementing a CBME program by defining desired training outcomes necessary to deliver high-quality patient care. With PHM competencies now well defined, programs can focus on developing programs of assessment and corresponding faculty development, which can help deliver valid, defensible decisions about fellow competence.

Finally, PHM has a workforce that can support CBTVT. A major barrier to time-variable training in GME is the need for trainees-as-workforce. In many GME programs, residents and fellows provide a relatively inexpensive, renewable workforce. Trainees’ clinical rotations are often scheduled up to 1 year in advance to ensure care teams are fully staffed, particularly in the inpatient setting, creating a system where flexibility in training is impossible without creating gaps in clinical coverage. However, many PHM fellowships do not completely rely on fellows to cover clinical service lines. PHM fellows spend 32 weeks over 2 years in core clinical rotations with faculty supervision, in accordance with the Accreditation Council for Graduate Medical Education program requirements, both for 2- and 3-year programs. Some CBME experts estimate (based on previous and ongoing CBTVT pilots) that training duration is likely to vary by roughly 20% from current time-based practices when CBTVT is initially implemented.²⁰ Thus, only a small number of clinical service weeks are likely to be affected. If a fellow were deemed ready for unsupervised practice before finishing 2 years of fellowship in a CBTVT program, the corresponding faculty supervisor could use the time previously assigned for supervision to pursue other priorities, such as education, scholarship, or quality improvement. Why provide supervision if a clinical competency committee has deemed a fellow ready for unsupervised practice? Some level of observation and formative feedback could continue, but full supervision would be redundant and unnecessary. CBTVT would allow for some fellows to experience the uncertainty that comes with unsupervised decision-making while still in an environment with trusted fellowship mentors and advisors.

PHM fellowship programs likely cannot flip a switch to “turn on” CBTVT immediately, but they can take steps toward making the transition. Validity, or defensibility of decisions, will be crucial for assessment in CBTVT systems. Programs will need to develop robust assessment systems that collect myriad data to answer the question, “When is this learner competent to deliver high-quality care without supervision?” Programs can align assessment instruments, faculty-development initiatives, and clinical competency committee (CCC) processes with the 2020 PHM competencies to provide a defensible answer. Program leaders should then seek validity evidence, either in existing literature or through novel scholarly initiatives, to support these summative decisions. Engaging all fellowship stakeholders in transitions to CBTVT will be important and should include fellows, program directors, CCC members, clinical leadership, and members from accrediting and credentialing bodies.

As fellowship programs review and revise curricula and assessment systems around the updated PHM core competencies, they should also consider what changes are necessary to implement CBTVT. Time variability is not a novelty but, rather, is a corollary to the outcomes-based approach of CBME. PHM fellowships should strike while the iron is hot and build on current change initiatives prompted by the growth of our specialty to be leaders in CBTVT.

In July 2020, the revision of The Pediatric Hospital Medicine Core Competencies was published, bringing to fruition three years of meticulous work.¹ The working group produced 66 chapters outlining the knowledge, skills, and attitudes needed for competent pediatric hospitalist practice. The arrival of these competencies is especially prescient given pediatric hospital medicine’s (PHM’s) relatively new standing as an American Board of Medical Specialties certified subspecialty, as the competencies can serve as a guide for improvement of fellowship curricula, assessment systems, and faculty development. The competencies also represent an opportunity for PHM to take a bold step forward in the world of graduate medical education (GME) by realizing a key tenet of competency-based medical education (CBME)—competency-based, time-variable training (CBTVT), in which learners train until competence is achieved rather than for a predetermined duration.^2,3 In this perspective, we describe how medical education in the United States adopted a time-based training paradigm (in which time-in-training is a surrogate for competence), how CBME has brought time-variable training to the fore, and how PHM has an opportunity to be on the leading edge of education innovation.

In the 1800s, during the time of the “Wild West,” medical education in the United States matched this moniker. There was little standardization across the multiple training pathways to become a practicing physician, including apprenticeships, lecture series, and university courses.⁴ Predictably, this led to significant heterogeneity in the quality of medical care that a patient of the day received. This problem became clearer as Americans traveled to Europe and witnessed more structured and rigorous training programs, only to return to the comparatively poor state of medical education back home.⁵ There was a clear need for curricular standardization.

In 1876, the American Medical College Association (which later became the Association of American Medical Colleges [AAMC]) was founded to meet this need, and in 1905 the Association adopted a set of minimum standards for medical training that included the now-familiar two years of basic sciences and two years of clinical training.⁶ Two subsequent national surveys in the United States were commissioned to evaluate whether medical schools met this new standard, with both surveys finding that roughly half of existing programs passed muster.^7,8 As a result, nearly half of US medical schools had closed by 1920 in a crusade to standardize curricula and produce competent physicians. By the time the American Medical Association established initial standards for internship (an archetype of GME),⁴ time-based medical training was the dominant paradigm. This historical perspective highlights the rationale for standardization of education processes and curricula, particularly in terms of accountability to the American public. But heralded by the 1978 landmark paper by McGaghie et al,⁹ the paradigm began to shift in the late twentieth century from a focus on the process of physician training to outcomes.

In contrast to the process-focused model of the early 1900s, CBME starts by identifying patient and healthcare system needs, defining competencies required to meet those needs, and then designing curricular and assessment processes to help learners achieve those competencies.² This outcomes-based approach grew as a response to calls for greater accountability to the public due to evidence that some graduates were unprepared for unsupervised practice, raising concerns that strictly time-based training was no longer defensible.¹⁰ CBME aims to mitigate these concerns by starting with desired outcomes of training and working backward to ensure those outcomes are met.

While many programs have attempted to implement CBME, most still rely heavily on time-in-training to determine competence. Learners participate in structured curricula and, unless they are extreme outliers, are deemed ready for unsupervised practice after a predetermined duration. This model presumes that competence and time are related in a fixed, predictable manner and that learners gain competence at a uniform rate. However, learners do not, in fact, progress uniformly. A study by Schumacher et al¹¹ involving 23 pediatric residency programs showed significant interlearner variability in rates of entrustment (used as a surrogate for competence), leading the authors to call for time-variable training in GME. Significant interlearner variation in rates of competence attainment have been shown in other specialties as well.¹² As more CBME studies on training outcomes emerge, the evidence is mounting that not all learners need the same duration of training to become competent providers. Time-in-training and competence attainment are not related in a fixed manner. As Dr Jason¹³ wrote in 1969, “By making time a constant, we make achievement a variable.” Variable achievement (competence, outcomes) was the very driver for medical education’s shift to a competency-based approach. If variable competence was not acceptable then, why should it be now? The goal of CBTVT is not shorter training, but rather flexible, individualized training both in terms of content and duration. While this also means some learners may need to extend their training, this should already be part of GME programs that are required to have remediation policies for learners who are not progressing as expected.

Time variability is an oft-cited tenet of CBME,^2,3 but one that is being piloted by relatively few programs in the United States, mostly in undergraduate medical education (UME).^14-16 The Education in Pediatrics Across the Continuum (EPAC), a consortium consisting of four institutions piloting CBTVT in UME,¹⁴ has shown early evidence of feasibility¹⁷ and that UME graduates from CBTVT programs enter residency with levels of competence similar to those of graduates of traditional time-based programs.¹⁸ We believe that PHM can take a step toward truly realizing CBME by implementing CBTVT in fellowship programs.

There are multiple reasons why this is an opportune time for PHM fellowships to consider CBTVT. First, PHM is a relatively new board-certified subspecialty with a recently revised set of core competencies¹ that are likely to catalyze programmatic innovation. A key step in change management is building on previous efforts to generate more change.¹⁹ Programs can leverage the momentum from current and impending change initiatives to innovate and implement CBTVT. Second, the revised PHM competencies provide the first crucial step in implementing a CBME program by defining desired training outcomes necessary to deliver high-quality patient care. With PHM competencies now well defined, programs can focus on developing programs of assessment and corresponding faculty development, which can help deliver valid, defensible decisions about fellow competence.

Finally, PHM has a workforce that can support CBTVT. A major barrier to time-variable training in GME is the need for trainees-as-workforce. In many GME programs, residents and fellows provide a relatively inexpensive, renewable workforce. Trainees’ clinical rotations are often scheduled up to 1 year in advance to ensure care teams are fully staffed, particularly in the inpatient setting, creating a system where flexibility in training is impossible without creating gaps in clinical coverage. However, many PHM fellowships do not completely rely on fellows to cover clinical service lines. PHM fellows spend 32 weeks over 2 years in core clinical rotations with faculty supervision, in accordance with the Accreditation Council for Graduate Medical Education program requirements, both for 2- and 3-year programs. Some CBME experts estimate (based on previous and ongoing CBTVT pilots) that training duration is likely to vary by roughly 20% from current time-based practices when CBTVT is initially implemented.²⁰ Thus, only a small number of clinical service weeks are likely to be affected. If a fellow were deemed ready for unsupervised practice before finishing 2 years of fellowship in a CBTVT program, the corresponding faculty supervisor could use the time previously assigned for supervision to pursue other priorities, such as education, scholarship, or quality improvement. Why provide supervision if a clinical competency committee has deemed a fellow ready for unsupervised practice? Some level of observation and formative feedback could continue, but full supervision would be redundant and unnecessary. CBTVT would allow for some fellows to experience the uncertainty that comes with unsupervised decision-making while still in an environment with trusted fellowship mentors and advisors.

PHM fellowship programs likely cannot flip a switch to “turn on” CBTVT immediately, but they can take steps toward making the transition. Validity, or defensibility of decisions, will be crucial for assessment in CBTVT systems. Programs will need to develop robust assessment systems that collect myriad data to answer the question, “When is this learner competent to deliver high-quality care without supervision?” Programs can align assessment instruments, faculty-development initiatives, and clinical competency committee (CCC) processes with the 2020 PHM competencies to provide a defensible answer. Program leaders should then seek validity evidence, either in existing literature or through novel scholarly initiatives, to support these summative decisions. Engaging all fellowship stakeholders in transitions to CBTVT will be important and should include fellows, program directors, CCC members, clinical leadership, and members from accrediting and credentialing bodies.

As fellowship programs review and revise curricula and assessment systems around the updated PHM core competencies, they should also consider what changes are necessary to implement CBTVT. Time variability is not a novelty but, rather, is a corollary to the outcomes-based approach of CBME. PHM fellowships should strike while the iron is hot and build on current change initiatives prompted by the growth of our specialty to be leaders in CBTVT.

The transition from hospital to home can be overwhelming for caregivers.¹ Stress of hospitalization coupled with the expectation of families to execute postdischarge care plans make understandable discharge communication critical. Communication failures, inadequate education, absence of caregiver confidence, and lack of clarity regarding care plans may prohibit smooth transitions and lead to adverse postdischarge outcomes.^2-4

Health literacy plays a pivotal role in caregivers’ capacity to navigate the healthcare system, comprehend, and execute care plans. An estimated 90 million Americans have limited health literacy that may negatively impact the provision of safe and quality care^5,6 and be a risk factor for poor outcomes, including increased emergency department (ED) utilization and readmission rates.^7-9 Readability strongly influences the effectiveness of written materials.¹⁰ However, written medical information for patients and families are frequently between the 10^th and 12^th grade reading levels; more than 75% of all pediatric health information is written at or above 10^th grade reading level.¹¹ Government agencies recommend between a 6^th and 8^th grade reading level, for written material;^5,12,13 written discharge instructions have been identified as an important quality metric for hospital-to-home transitions.^14-16

At our center, we found that discharge instructions were commonly written at high reading levels and often incomplete.¹⁷ Poor discharge instructions may contribute to increased readmission rates and unnecessary ED visits.^9,18 Our global aim targeted improved health-literate written information, including understandability and completeness.

Our specific aim was to increase the percentage of discharge instructions written at or below the 7th grade level for hospital medicine (HM) patients on a community hospital pediatric unit from 13% to 80% in 6 months.

Context

The improvement work took place at a 42-bed inpatient pediatric unit at a community satellite of our large, urban, academic hospital. The unit is staffed by medical providers including attendings, fellows, nurse practitioners (NPs), and senior pediatric residents, and had more than 1000 HM discharges in fiscal year 2016. Children with common general pediatric diagnoses are admitted to this service; postsurgical patients are not admitted primarily to the HM service. In Cincinnati, the neighborhood-level high school drop-out rates are as high as 64%.¹⁹ Discharge instructions are written by medical providers in the electronic health record (EHR). A printed copy is given to families and verbally reviewed by a bedside nurse prior to discharge. Quality improvement (QI) efforts focused on discharge instructions were ignited by a prior review of 200 discharge instructions that showed they were difficult to read (median reading level of 10^th grade), poorly understandable (36% of instructions met the threshold of understandability as measured by the Patient Education Materials Assessment Tool²⁰) and were missing key elements of information.¹⁷

Improvement Team

The improvement team consisted of 4 pediatric hospitalists, 2 NPs, 1 nurse educator with health literacy expertise, 1 pediatric resident, 1 fourth-year medical student, 1 QI consultant, and 2 parents who had first-hand experience on the HM service. The improvement team observed the discharge process, including roles of the provider, nurse and family, outlined a process map, and created a modified failure mode and effect analysis.²¹ Prior to our work, discharge instructions written by providers often occurred as a last step, and the content was created as free text or from nonstandardized templates. Key drivers that informed interventions were determined and revised over time (Figure 1). The study was reviewed by our institutional review board and deemed not human subjects research.

Improvement Activities

Key drivers were identified, and interventions were executed using Plan-Do Study-Act cycles.²² The key drivers thought to be critical for the success of the QI efforts were family engagement; standardization of discharge instructions; medical staff engagement; and audit and feedback of data. The corresponding interventions were as follows:

Family Engagement

Understanding the discharge information families desired. Prior to testing, 10 families admitted to the HM service were asked about the discharge experience. We asked families about information they wanted in written discharge instructions: 1) reasons to call your primary doctor or return to the hospital; 2) when to see your primary doctor for a follow-up visit; 3) the phone number to reach your child’s doctor; 4) more information about why your child was admitted; 5) information about new medications; and 6) what to do to help your child continue to recover at home.

Development of templates. We engaged families throughout the process of creating general and disease-specific discharge templates. After a specific template was created and reviewed by the parents on our team, it was sent to members of the institutional Patient Education Committee, which includes parents and local health literacy experts, to review and critique. Feedback from the reviewers was incorporated into the templates prior to use in the EHR.

Postdischarge phone calls.A convenience sample of families discharged from the satellite campus was called 24 to 48 hours after discharge over a 2-week period in January, 2016. A member of our improvement team solicited feedback from families about the quality of the discharge instructions. Families were asked if discharge instructions were reviewed with them prior to going home, if they were given a copy of the instructions, how they would rate the ability to read and use the information, and if there were additional pieces of information that would have improved the instructions.

Standardization of Instructions

Education. A presentation was created and shared with medical providers; it was re-disseminated monthly to new residents rotating onto the service and to the attendings, fellows, and NPs scheduled for shifts during the month. This education continued for the duration of the study. The presentation included the definition of health literacy, scope of the problem, examples of poorly written discharge instructions, and tips on how to write readable and understandable instructions. Laminated cards that included tips on how to write instructions were also placed on work stations.

While the general template was an important first step, the content relied heavily on free text by providers, which could still lead to instructions written at a high reading level. Thus, disease-specific discharge instruction templates were created with prepopulated information that was written at a reading level at or below 7^th grade level (Figure 2). The diseases were prioritized based on the most common diagnoses on our HM service. Each template included information under each of the subheadings noted in the general template. Twelve disease-specific templates were tested and ultimately embedded in the EHR; the general template remained for use when the discharge diagnosis was not covered by a disease-specific template.

Medical Staff Engagement

Previously described tests of change also aimed to enhance staff engagement. These included frequent e-mails, discussion of the QI efforts at specific team meetings, and the creation of visual cues posted at computer work stations, which prompted staff to begin to work on discharge instructions soon after admission.

Audit and Feedback of Data

Weekly phone calls. One team updated clinicians through a regularly scheduled bi-weekly phone conference. The phone conference was established prior to our work and was designed to relay pertinent information to attendings and NPs who work at the satellite hospital. During the phone conferences, clinicians were notified of current performance on discharge instruction readability and specific tests of change for the week. Additionally, providers gave feedback about the improvement efforts. These updates continued for the first 6 months of the project until sustained improvements were observed.

E-mails. Weekly e-mails were sent to all providers scheduled for clinical time at the satellite campus. The e-mail contained information on current tests of change, a list of discharge instruction templates that were available in the EHR, and the annotated run chart illustrating readability levels over time.

Additionally, individual e-mails were sent to each provider after review of the written discharge instructions for the week. Providers were given information on the number of discharge instructions they personally composed, the percentage of those instructions that were written at or below 7^th grade level, and specific feedback on how their written instructions could be improved. We also encouraged feedback from each provider to better identify barriers to achieving our goal.

Study of the Interventions

Baseline data included a review of all instructions for patients discharged from the satellite campus from the end of April 2015 through mid-September 2015. The time period for testing of interventions during the fall and winter months allowed for rapid cycle learning due to higher patient census and predictability of admissions for specific diagnosis (ie, asthma and bronchiolitis). An automated report was generated from the EHR weekly with specific demographics and identifiers for patient discharged over the past 7 days, including patient age, gender, length of stay, discharge diagnosis, and insurance classification. Data was collected during the intervention period via structured review of the discharge instructions in the EHR by the principal investigator or a trained research coordinator. Discharge instructions for medically cleared mental health patients admitted to hospital medicine while awaiting psychiatric bed availability and patients and parents who were non-English speaking were excluded from review. All other instructions for patients discharged from the HM service at our Liberty Campus were included for review.

Measures

Readability, our primary measure of interest, was calculated using the mean score from the following formulas: Flesch Kincaid Grade Level,²³ Simple Measure of Gobbledygook Index,²⁴ Coleman-Liau Index,²⁵ Gunning-Fog Index,²⁶ and Automated Readability Index²⁷ by means of an online platform (https://readability-score.com).²⁸ This platform was chosen because it incorporated a variety of formulas, was user-friendly, and required minimal data cleaning. Each of the readability formulas have been used to assesses readability of health information given to patients and families.^29,30 The threshold of 7th grade is in alignment with our institutional policy for educational materials and with recommendations from several government agencies.^5,12

Analysis

A statistical process control p-chart was used to analyze our primary measure of readability, dichotomized as percent discharge instructions written at or below 7th grade level. Run charts were used to follow mean reading level of discharge instructions and our process measure of percent of discharge instruction written with a general or disease-specific standardized template. Run chart and control chart rules for identifying special cause were used for midline shifts.³¹

The Table includes the demographic and clinical information of patients included in our analyses. Through sequential interventions, the percentage of discharge instructions written at or below 7th grade readability level increased from a mean of 13% to more than 80% in 3 months (Figure 3). Furthermore, the mean was sustained above 90% for 10 months and at 98% for the last 4 months. The use of 1 of the 13 EHR templates increased from 0% to 96% and was associated with the largest impact on the overall improvements (Supplemental Figure 1). Additionally, the average reading level of the discharge instructions decreased from 10th grade to 6th grade level (Supplemental Figure 2).

Qualitative comments from providers about the discharge instructions included:

“Are these [discharge instructions] available at base??  Great resource for interns.”
“These [discharge] instructions make the [discharge] process so easy!!! Love these...”
“Also feel like they have helped my discharge teaching in the room!”

Qualitative comments from families postdischarge included:
“I thought the instructions were very clear and easy to read. I especially thought that highlighting the important areas really helped.”
“I think this form looks great, and I really like the idea of having your child’s name on it.”

Through sequential Plan-Do Study-Act cycles, we increased the percentage of discharge instructions written at or below 7^th grade reading level from 13% to 98%. Our most impactful intervention was the creation and dissemination of standardized disease-specific discharge instruction templates. Our findings complement evidence in the adult and pediatric literature that the use of standardized, disease-specific discharge instruction templates may improve readability of instructions.^32,33 And, while quality improvement efforts have been employed to improve the discharge process for patients,^34-36 this is the first study in the inpatient setting that, to our knowledge, specifically addresses discharge instructions using quality improvement methods.

Our work targeted the critical intersection between individual health literacy, an individual’s capacity to acquire, interpret, and use health information, and the necessary changes needed within our healthcare system to ensure that appropriately written instructions are given to patients and families.^17,37 Our efforts focused on improving discharge instructions answer the call to consider health literacy a modifiable clinical risk factor.³⁷ Furthermore, we address the 6 aims for quality healthcare delivery: 1) safe, timely, efficient and equitable delivery of care through the creation and dissemination of standardized instructions that are written at the appropriate reading level for families to ease hospital-to-home transitions and streamline the workflow of medical providers; 2) effective education of medical providers on health literacy concepts; and 3) family-centeredness through the involvement of families in our QI efforts. While previous QI efforts to improve hospital-to-home transitions have focused on medication reconciliation, communication with primary care physicians, follow-up appointments, and timely discharges of patients, none have specifically focused on the quality of discharge instructions.^34-36

Most physicians do not receive education about how to write information that is readable and understandable; more than half of providers desired more education in this area.³⁸ Furthermore, pediatric providers may overestimate parental health literacy levels,³⁹ which may contribute to variability in the readability of written health materials. While education alone can contribute to a provider’s ability to create readable instructions, we note the improvement after the introduction of disease templates to demonstrate the importance of workflow-integrated higher reliability interventions to sustain improvements.

Our baseline poor readability rates were due to limited knowledge by frontline providers composing the instructions and a system in which an important element for successful hospital-to-home transitions was not tackled until patients were ready for discharge. Streamlining of the discharge process, including the creation of discharge instructions, may lead to improved efficiency, fewer discrepancies, more effective communication, and an enhanced family experience. Moreover, the success of our improvement work was due to key stakeholders, including parents, being a part of the team and the notable buy-in from providers.

Our work was not without limitations. We excluded non-English speaking families from the study. We were unable to measure reading level of our population directly and instead based our goals on national estimates. Our primary measure was readability, which is only 1 piece contributing to quality discharge instructions. Understandability and actionability are also important considerations; ^17,20,29,40 however, improvements in these areas were limited by our design options within the EHR. Our efforts focused on children with common general pediatric diagnoses, and it is unclear how our interventions would generalize to medically complex patients with more volume of information to communicate at discharge and with uncommon diagnoses that are less readily incorporated into standardized templates. Relatedly, our work occurred at the satellite campus of our tertiary care center and may not represent generalizable material or methods to implement templates at our main campus location or at other hospitals. To begin to better understand this, we have spread to HM patients at our main campus, including medically complex patients with technology dependence and/or neurological impairments. Standardized, disease-specific templates most relevant to this population as well as several patient specific templates, for those with frequent readmissions due to medical complexity, have been created and are actively being tested.

In conclusion, in using interventions targeted at standardization of discharge instructions and timely feedback to staff, we saw rapid, dramatic, and sustained improvement in the readability of discharge instructions. Next steps include adaptation and spread to other patient populations and care teams, collaborations with other centers, and assessing the impact of effectively written discharge instructions on patient outcomes, such as adverse drug events, readmission rates, and family experience.

Disclosure

No external funding was secured for this study. Dr. Brady is supported by a Patient-Centered Outcomes Research Mentored Clinical Investigator Award from the Agency for Healthcare Research and Quality, Award Number K08HS023827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding organizations. The funding organization had no role in the design, preparation, review, or approval of this paper; nor the decision to submit the manuscript for publication. The authors have no financial relationships relevant to this article to disclose.

The transition from hospital to home can be overwhelming for caregivers.¹ Stress of hospitalization coupled with the expectation of families to execute postdischarge care plans make understandable discharge communication critical. Communication failures, inadequate education, absence of caregiver confidence, and lack of clarity regarding care plans may prohibit smooth transitions and lead to adverse postdischarge outcomes.^2-4

Health literacy plays a pivotal role in caregivers’ capacity to navigate the healthcare system, comprehend, and execute care plans. An estimated 90 million Americans have limited health literacy that may negatively impact the provision of safe and quality care^5,6 and be a risk factor for poor outcomes, including increased emergency department (ED) utilization and readmission rates.^7-9 Readability strongly influences the effectiveness of written materials.¹⁰ However, written medical information for patients and families are frequently between the 10^th and 12^th grade reading levels; more than 75% of all pediatric health information is written at or above 10^th grade reading level.¹¹ Government agencies recommend between a 6^th and 8^th grade reading level, for written material;^5,12,13 written discharge instructions have been identified as an important quality metric for hospital-to-home transitions.^14-16

At our center, we found that discharge instructions were commonly written at high reading levels and often incomplete.¹⁷ Poor discharge instructions may contribute to increased readmission rates and unnecessary ED visits.^9,18 Our global aim targeted improved health-literate written information, including understandability and completeness.

Our specific aim was to increase the percentage of discharge instructions written at or below the 7th grade level for hospital medicine (HM) patients on a community hospital pediatric unit from 13% to 80% in 6 months.

Context

The improvement work took place at a 42-bed inpatient pediatric unit at a community satellite of our large, urban, academic hospital. The unit is staffed by medical providers including attendings, fellows, nurse practitioners (NPs), and senior pediatric residents, and had more than 1000 HM discharges in fiscal year 2016. Children with common general pediatric diagnoses are admitted to this service; postsurgical patients are not admitted primarily to the HM service. In Cincinnati, the neighborhood-level high school drop-out rates are as high as 64%.¹⁹ Discharge instructions are written by medical providers in the electronic health record (EHR). A printed copy is given to families and verbally reviewed by a bedside nurse prior to discharge. Quality improvement (QI) efforts focused on discharge instructions were ignited by a prior review of 200 discharge instructions that showed they were difficult to read (median reading level of 10^th grade), poorly understandable (36% of instructions met the threshold of understandability as measured by the Patient Education Materials Assessment Tool²⁰) and were missing key elements of information.¹⁷

Improvement Team

The improvement team consisted of 4 pediatric hospitalists, 2 NPs, 1 nurse educator with health literacy expertise, 1 pediatric resident, 1 fourth-year medical student, 1 QI consultant, and 2 parents who had first-hand experience on the HM service. The improvement team observed the discharge process, including roles of the provider, nurse and family, outlined a process map, and created a modified failure mode and effect analysis.²¹ Prior to our work, discharge instructions written by providers often occurred as a last step, and the content was created as free text or from nonstandardized templates. Key drivers that informed interventions were determined and revised over time (Figure 1). The study was reviewed by our institutional review board and deemed not human subjects research.

Improvement Activities

Key drivers were identified, and interventions were executed using Plan-Do Study-Act cycles.²² The key drivers thought to be critical for the success of the QI efforts were family engagement; standardization of discharge instructions; medical staff engagement; and audit and feedback of data. The corresponding interventions were as follows:

Family Engagement

Understanding the discharge information families desired. Prior to testing, 10 families admitted to the HM service were asked about the discharge experience. We asked families about information they wanted in written discharge instructions: 1) reasons to call your primary doctor or return to the hospital; 2) when to see your primary doctor for a follow-up visit; 3) the phone number to reach your child’s doctor; 4) more information about why your child was admitted; 5) information about new medications; and 6) what to do to help your child continue to recover at home.

Development of templates. We engaged families throughout the process of creating general and disease-specific discharge templates. After a specific template was created and reviewed by the parents on our team, it was sent to members of the institutional Patient Education Committee, which includes parents and local health literacy experts, to review and critique. Feedback from the reviewers was incorporated into the templates prior to use in the EHR.

Postdischarge phone calls.A convenience sample of families discharged from the satellite campus was called 24 to 48 hours after discharge over a 2-week period in January, 2016. A member of our improvement team solicited feedback from families about the quality of the discharge instructions. Families were asked if discharge instructions were reviewed with them prior to going home, if they were given a copy of the instructions, how they would rate the ability to read and use the information, and if there were additional pieces of information that would have improved the instructions.

Standardization of Instructions

Education. A presentation was created and shared with medical providers; it was re-disseminated monthly to new residents rotating onto the service and to the attendings, fellows, and NPs scheduled for shifts during the month. This education continued for the duration of the study. The presentation included the definition of health literacy, scope of the problem, examples of poorly written discharge instructions, and tips on how to write readable and understandable instructions. Laminated cards that included tips on how to write instructions were also placed on work stations.

While the general template was an important first step, the content relied heavily on free text by providers, which could still lead to instructions written at a high reading level. Thus, disease-specific discharge instruction templates were created with prepopulated information that was written at a reading level at or below 7^th grade level (Figure 2). The diseases were prioritized based on the most common diagnoses on our HM service. Each template included information under each of the subheadings noted in the general template. Twelve disease-specific templates were tested and ultimately embedded in the EHR; the general template remained for use when the discharge diagnosis was not covered by a disease-specific template.

Medical Staff Engagement

Previously described tests of change also aimed to enhance staff engagement. These included frequent e-mails, discussion of the QI efforts at specific team meetings, and the creation of visual cues posted at computer work stations, which prompted staff to begin to work on discharge instructions soon after admission.

Audit and Feedback of Data

Weekly phone calls. One team updated clinicians through a regularly scheduled bi-weekly phone conference. The phone conference was established prior to our work and was designed to relay pertinent information to attendings and NPs who work at the satellite hospital. During the phone conferences, clinicians were notified of current performance on discharge instruction readability and specific tests of change for the week. Additionally, providers gave feedback about the improvement efforts. These updates continued for the first 6 months of the project until sustained improvements were observed.

E-mails. Weekly e-mails were sent to all providers scheduled for clinical time at the satellite campus. The e-mail contained information on current tests of change, a list of discharge instruction templates that were available in the EHR, and the annotated run chart illustrating readability levels over time.

Additionally, individual e-mails were sent to each provider after review of the written discharge instructions for the week. Providers were given information on the number of discharge instructions they personally composed, the percentage of those instructions that were written at or below 7^th grade level, and specific feedback on how their written instructions could be improved. We also encouraged feedback from each provider to better identify barriers to achieving our goal.

Study of the Interventions

Baseline data included a review of all instructions for patients discharged from the satellite campus from the end of April 2015 through mid-September 2015. The time period for testing of interventions during the fall and winter months allowed for rapid cycle learning due to higher patient census and predictability of admissions for specific diagnosis (ie, asthma and bronchiolitis). An automated report was generated from the EHR weekly with specific demographics and identifiers for patient discharged over the past 7 days, including patient age, gender, length of stay, discharge diagnosis, and insurance classification. Data was collected during the intervention period via structured review of the discharge instructions in the EHR by the principal investigator or a trained research coordinator. Discharge instructions for medically cleared mental health patients admitted to hospital medicine while awaiting psychiatric bed availability and patients and parents who were non-English speaking were excluded from review. All other instructions for patients discharged from the HM service at our Liberty Campus were included for review.

Measures

Readability, our primary measure of interest, was calculated using the mean score from the following formulas: Flesch Kincaid Grade Level,²³ Simple Measure of Gobbledygook Index,²⁴ Coleman-Liau Index,²⁵ Gunning-Fog Index,²⁶ and Automated Readability Index²⁷ by means of an online platform (https://readability-score.com).²⁸ This platform was chosen because it incorporated a variety of formulas, was user-friendly, and required minimal data cleaning. Each of the readability formulas have been used to assesses readability of health information given to patients and families.^29,30 The threshold of 7th grade is in alignment with our institutional policy for educational materials and with recommendations from several government agencies.^5,12

Analysis

A statistical process control p-chart was used to analyze our primary measure of readability, dichotomized as percent discharge instructions written at or below 7th grade level. Run charts were used to follow mean reading level of discharge instructions and our process measure of percent of discharge instruction written with a general or disease-specific standardized template. Run chart and control chart rules for identifying special cause were used for midline shifts.³¹

The Table includes the demographic and clinical information of patients included in our analyses. Through sequential interventions, the percentage of discharge instructions written at or below 7th grade readability level increased from a mean of 13% to more than 80% in 3 months (Figure 3). Furthermore, the mean was sustained above 90% for 10 months and at 98% for the last 4 months. The use of 1 of the 13 EHR templates increased from 0% to 96% and was associated with the largest impact on the overall improvements (Supplemental Figure 1). Additionally, the average reading level of the discharge instructions decreased from 10th grade to 6th grade level (Supplemental Figure 2).

Qualitative comments from providers about the discharge instructions included:

“Are these [discharge instructions] available at base??  Great resource for interns.”
“These [discharge] instructions make the [discharge] process so easy!!! Love these...”
“Also feel like they have helped my discharge teaching in the room!”

Qualitative comments from families postdischarge included:
“I thought the instructions were very clear and easy to read. I especially thought that highlighting the important areas really helped.”
“I think this form looks great, and I really like the idea of having your child’s name on it.”

Through sequential Plan-Do Study-Act cycles, we increased the percentage of discharge instructions written at or below 7^th grade reading level from 13% to 98%. Our most impactful intervention was the creation and dissemination of standardized disease-specific discharge instruction templates. Our findings complement evidence in the adult and pediatric literature that the use of standardized, disease-specific discharge instruction templates may improve readability of instructions.^32,33 And, while quality improvement efforts have been employed to improve the discharge process for patients,^34-36 this is the first study in the inpatient setting that, to our knowledge, specifically addresses discharge instructions using quality improvement methods.

Our work targeted the critical intersection between individual health literacy, an individual’s capacity to acquire, interpret, and use health information, and the necessary changes needed within our healthcare system to ensure that appropriately written instructions are given to patients and families.^17,37 Our efforts focused on improving discharge instructions answer the call to consider health literacy a modifiable clinical risk factor.³⁷ Furthermore, we address the 6 aims for quality healthcare delivery: 1) safe, timely, efficient and equitable delivery of care through the creation and dissemination of standardized instructions that are written at the appropriate reading level for families to ease hospital-to-home transitions and streamline the workflow of medical providers; 2) effective education of medical providers on health literacy concepts; and 3) family-centeredness through the involvement of families in our QI efforts. While previous QI efforts to improve hospital-to-home transitions have focused on medication reconciliation, communication with primary care physicians, follow-up appointments, and timely discharges of patients, none have specifically focused on the quality of discharge instructions.^34-36

Most physicians do not receive education about how to write information that is readable and understandable; more than half of providers desired more education in this area.³⁸ Furthermore, pediatric providers may overestimate parental health literacy levels,³⁹ which may contribute to variability in the readability of written health materials. While education alone can contribute to a provider’s ability to create readable instructions, we note the improvement after the introduction of disease templates to demonstrate the importance of workflow-integrated higher reliability interventions to sustain improvements.

Our baseline poor readability rates were due to limited knowledge by frontline providers composing the instructions and a system in which an important element for successful hospital-to-home transitions was not tackled until patients were ready for discharge. Streamlining of the discharge process, including the creation of discharge instructions, may lead to improved efficiency, fewer discrepancies, more effective communication, and an enhanced family experience. Moreover, the success of our improvement work was due to key stakeholders, including parents, being a part of the team and the notable buy-in from providers.

Our work was not without limitations. We excluded non-English speaking families from the study. We were unable to measure reading level of our population directly and instead based our goals on national estimates. Our primary measure was readability, which is only 1 piece contributing to quality discharge instructions. Understandability and actionability are also important considerations; ^17,20,29,40 however, improvements in these areas were limited by our design options within the EHR. Our efforts focused on children with common general pediatric diagnoses, and it is unclear how our interventions would generalize to medically complex patients with more volume of information to communicate at discharge and with uncommon diagnoses that are less readily incorporated into standardized templates. Relatedly, our work occurred at the satellite campus of our tertiary care center and may not represent generalizable material or methods to implement templates at our main campus location or at other hospitals. To begin to better understand this, we have spread to HM patients at our main campus, including medically complex patients with technology dependence and/or neurological impairments. Standardized, disease-specific templates most relevant to this population as well as several patient specific templates, for those with frequent readmissions due to medical complexity, have been created and are actively being tested.

In conclusion, in using interventions targeted at standardization of discharge instructions and timely feedback to staff, we saw rapid, dramatic, and sustained improvement in the readability of discharge instructions. Next steps include adaptation and spread to other patient populations and care teams, collaborations with other centers, and assessing the impact of effectively written discharge instructions on patient outcomes, such as adverse drug events, readmission rates, and family experience.

Disclosure

No external funding was secured for this study. Dr. Brady is supported by a Patient-Centered Outcomes Research Mentored Clinical Investigator Award from the Agency for Healthcare Research and Quality, Award Number K08HS023827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding organizations. The funding organization had no role in the design, preparation, review, or approval of this paper; nor the decision to submit the manuscript for publication. The authors have no financial relationships relevant to this article to disclose.

The transition from hospital to home can be overwhelming for caregivers.¹ Stress of hospitalization coupled with the expectation of families to execute postdischarge care plans make understandable discharge communication critical. Communication failures, inadequate education, absence of caregiver confidence, and lack of clarity regarding care plans may prohibit smooth transitions and lead to adverse postdischarge outcomes.^2-4

Health literacy plays a pivotal role in caregivers’ capacity to navigate the healthcare system, comprehend, and execute care plans. An estimated 90 million Americans have limited health literacy that may negatively impact the provision of safe and quality care^5,6 and be a risk factor for poor outcomes, including increased emergency department (ED) utilization and readmission rates.^7-9 Readability strongly influences the effectiveness of written materials.¹⁰ However, written medical information for patients and families are frequently between the 10^th and 12^th grade reading levels; more than 75% of all pediatric health information is written at or above 10^th grade reading level.¹¹ Government agencies recommend between a 6^th and 8^th grade reading level, for written material;^5,12,13 written discharge instructions have been identified as an important quality metric for hospital-to-home transitions.^14-16

At our center, we found that discharge instructions were commonly written at high reading levels and often incomplete.¹⁷ Poor discharge instructions may contribute to increased readmission rates and unnecessary ED visits.^9,18 Our global aim targeted improved health-literate written information, including understandability and completeness.

Our specific aim was to increase the percentage of discharge instructions written at or below the 7th grade level for hospital medicine (HM) patients on a community hospital pediatric unit from 13% to 80% in 6 months.

Context

The improvement work took place at a 42-bed inpatient pediatric unit at a community satellite of our large, urban, academic hospital. The unit is staffed by medical providers including attendings, fellows, nurse practitioners (NPs), and senior pediatric residents, and had more than 1000 HM discharges in fiscal year 2016. Children with common general pediatric diagnoses are admitted to this service; postsurgical patients are not admitted primarily to the HM service. In Cincinnati, the neighborhood-level high school drop-out rates are as high as 64%.¹⁹ Discharge instructions are written by medical providers in the electronic health record (EHR). A printed copy is given to families and verbally reviewed by a bedside nurse prior to discharge. Quality improvement (QI) efforts focused on discharge instructions were ignited by a prior review of 200 discharge instructions that showed they were difficult to read (median reading level of 10^th grade), poorly understandable (36% of instructions met the threshold of understandability as measured by the Patient Education Materials Assessment Tool²⁰) and were missing key elements of information.¹⁷

Improvement Team

The improvement team consisted of 4 pediatric hospitalists, 2 NPs, 1 nurse educator with health literacy expertise, 1 pediatric resident, 1 fourth-year medical student, 1 QI consultant, and 2 parents who had first-hand experience on the HM service. The improvement team observed the discharge process, including roles of the provider, nurse and family, outlined a process map, and created a modified failure mode and effect analysis.²¹ Prior to our work, discharge instructions written by providers often occurred as a last step, and the content was created as free text or from nonstandardized templates. Key drivers that informed interventions were determined and revised over time (Figure 1). The study was reviewed by our institutional review board and deemed not human subjects research.

Improvement Activities

Key drivers were identified, and interventions were executed using Plan-Do Study-Act cycles.²² The key drivers thought to be critical for the success of the QI efforts were family engagement; standardization of discharge instructions; medical staff engagement; and audit and feedback of data. The corresponding interventions were as follows:

Family Engagement

Understanding the discharge information families desired. Prior to testing, 10 families admitted to the HM service were asked about the discharge experience. We asked families about information they wanted in written discharge instructions: 1) reasons to call your primary doctor or return to the hospital; 2) when to see your primary doctor for a follow-up visit; 3) the phone number to reach your child’s doctor; 4) more information about why your child was admitted; 5) information about new medications; and 6) what to do to help your child continue to recover at home.

Development of templates. We engaged families throughout the process of creating general and disease-specific discharge templates. After a specific template was created and reviewed by the parents on our team, it was sent to members of the institutional Patient Education Committee, which includes parents and local health literacy experts, to review and critique. Feedback from the reviewers was incorporated into the templates prior to use in the EHR.

Postdischarge phone calls.A convenience sample of families discharged from the satellite campus was called 24 to 48 hours after discharge over a 2-week period in January, 2016. A member of our improvement team solicited feedback from families about the quality of the discharge instructions. Families were asked if discharge instructions were reviewed with them prior to going home, if they were given a copy of the instructions, how they would rate the ability to read and use the information, and if there were additional pieces of information that would have improved the instructions.

Standardization of Instructions

Education. A presentation was created and shared with medical providers; it was re-disseminated monthly to new residents rotating onto the service and to the attendings, fellows, and NPs scheduled for shifts during the month. This education continued for the duration of the study. The presentation included the definition of health literacy, scope of the problem, examples of poorly written discharge instructions, and tips on how to write readable and understandable instructions. Laminated cards that included tips on how to write instructions were also placed on work stations.

While the general template was an important first step, the content relied heavily on free text by providers, which could still lead to instructions written at a high reading level. Thus, disease-specific discharge instruction templates were created with prepopulated information that was written at a reading level at or below 7^th grade level (Figure 2). The diseases were prioritized based on the most common diagnoses on our HM service. Each template included information under each of the subheadings noted in the general template. Twelve disease-specific templates were tested and ultimately embedded in the EHR; the general template remained for use when the discharge diagnosis was not covered by a disease-specific template.

Medical Staff Engagement

Previously described tests of change also aimed to enhance staff engagement. These included frequent e-mails, discussion of the QI efforts at specific team meetings, and the creation of visual cues posted at computer work stations, which prompted staff to begin to work on discharge instructions soon after admission.

Audit and Feedback of Data

Weekly phone calls. One team updated clinicians through a regularly scheduled bi-weekly phone conference. The phone conference was established prior to our work and was designed to relay pertinent information to attendings and NPs who work at the satellite hospital. During the phone conferences, clinicians were notified of current performance on discharge instruction readability and specific tests of change for the week. Additionally, providers gave feedback about the improvement efforts. These updates continued for the first 6 months of the project until sustained improvements were observed.

E-mails. Weekly e-mails were sent to all providers scheduled for clinical time at the satellite campus. The e-mail contained information on current tests of change, a list of discharge instruction templates that were available in the EHR, and the annotated run chart illustrating readability levels over time.

Additionally, individual e-mails were sent to each provider after review of the written discharge instructions for the week. Providers were given information on the number of discharge instructions they personally composed, the percentage of those instructions that were written at or below 7^th grade level, and specific feedback on how their written instructions could be improved. We also encouraged feedback from each provider to better identify barriers to achieving our goal.

Study of the Interventions

Baseline data included a review of all instructions for patients discharged from the satellite campus from the end of April 2015 through mid-September 2015. The time period for testing of interventions during the fall and winter months allowed for rapid cycle learning due to higher patient census and predictability of admissions for specific diagnosis (ie, asthma and bronchiolitis). An automated report was generated from the EHR weekly with specific demographics and identifiers for patient discharged over the past 7 days, including patient age, gender, length of stay, discharge diagnosis, and insurance classification. Data was collected during the intervention period via structured review of the discharge instructions in the EHR by the principal investigator or a trained research coordinator. Discharge instructions for medically cleared mental health patients admitted to hospital medicine while awaiting psychiatric bed availability and patients and parents who were non-English speaking were excluded from review. All other instructions for patients discharged from the HM service at our Liberty Campus were included for review.

Measures

Readability, our primary measure of interest, was calculated using the mean score from the following formulas: Flesch Kincaid Grade Level,²³ Simple Measure of Gobbledygook Index,²⁴ Coleman-Liau Index,²⁵ Gunning-Fog Index,²⁶ and Automated Readability Index²⁷ by means of an online platform (https://readability-score.com).²⁸ This platform was chosen because it incorporated a variety of formulas, was user-friendly, and required minimal data cleaning. Each of the readability formulas have been used to assesses readability of health information given to patients and families.^29,30 The threshold of 7th grade is in alignment with our institutional policy for educational materials and with recommendations from several government agencies.^5,12

Analysis

A statistical process control p-chart was used to analyze our primary measure of readability, dichotomized as percent discharge instructions written at or below 7th grade level. Run charts were used to follow mean reading level of discharge instructions and our process measure of percent of discharge instruction written with a general or disease-specific standardized template. Run chart and control chart rules for identifying special cause were used for midline shifts.³¹

The Table includes the demographic and clinical information of patients included in our analyses. Through sequential interventions, the percentage of discharge instructions written at or below 7th grade readability level increased from a mean of 13% to more than 80% in 3 months (Figure 3). Furthermore, the mean was sustained above 90% for 10 months and at 98% for the last 4 months. The use of 1 of the 13 EHR templates increased from 0% to 96% and was associated with the largest impact on the overall improvements (Supplemental Figure 1). Additionally, the average reading level of the discharge instructions decreased from 10th grade to 6th grade level (Supplemental Figure 2).

Qualitative comments from providers about the discharge instructions included:

“Are these [discharge instructions] available at base??  Great resource for interns.”
“These [discharge] instructions make the [discharge] process so easy!!! Love these...”
“Also feel like they have helped my discharge teaching in the room!”

Qualitative comments from families postdischarge included:
“I thought the instructions were very clear and easy to read. I especially thought that highlighting the important areas really helped.”
“I think this form looks great, and I really like the idea of having your child’s name on it.”

Through sequential Plan-Do Study-Act cycles, we increased the percentage of discharge instructions written at or below 7^th grade reading level from 13% to 98%. Our most impactful intervention was the creation and dissemination of standardized disease-specific discharge instruction templates. Our findings complement evidence in the adult and pediatric literature that the use of standardized, disease-specific discharge instruction templates may improve readability of instructions.^32,33 And, while quality improvement efforts have been employed to improve the discharge process for patients,^34-36 this is the first study in the inpatient setting that, to our knowledge, specifically addresses discharge instructions using quality improvement methods.

Our work targeted the critical intersection between individual health literacy, an individual’s capacity to acquire, interpret, and use health information, and the necessary changes needed within our healthcare system to ensure that appropriately written instructions are given to patients and families.^17,37 Our efforts focused on improving discharge instructions answer the call to consider health literacy a modifiable clinical risk factor.³⁷ Furthermore, we address the 6 aims for quality healthcare delivery: 1) safe, timely, efficient and equitable delivery of care through the creation and dissemination of standardized instructions that are written at the appropriate reading level for families to ease hospital-to-home transitions and streamline the workflow of medical providers; 2) effective education of medical providers on health literacy concepts; and 3) family-centeredness through the involvement of families in our QI efforts. While previous QI efforts to improve hospital-to-home transitions have focused on medication reconciliation, communication with primary care physicians, follow-up appointments, and timely discharges of patients, none have specifically focused on the quality of discharge instructions.^34-36

Most physicians do not receive education about how to write information that is readable and understandable; more than half of providers desired more education in this area.³⁸ Furthermore, pediatric providers may overestimate parental health literacy levels,³⁹ which may contribute to variability in the readability of written health materials. While education alone can contribute to a provider’s ability to create readable instructions, we note the improvement after the introduction of disease templates to demonstrate the importance of workflow-integrated higher reliability interventions to sustain improvements.

Our baseline poor readability rates were due to limited knowledge by frontline providers composing the instructions and a system in which an important element for successful hospital-to-home transitions was not tackled until patients were ready for discharge. Streamlining of the discharge process, including the creation of discharge instructions, may lead to improved efficiency, fewer discrepancies, more effective communication, and an enhanced family experience. Moreover, the success of our improvement work was due to key stakeholders, including parents, being a part of the team and the notable buy-in from providers.

Our work was not without limitations. We excluded non-English speaking families from the study. We were unable to measure reading level of our population directly and instead based our goals on national estimates. Our primary measure was readability, which is only 1 piece contributing to quality discharge instructions. Understandability and actionability are also important considerations; ^17,20,29,40 however, improvements in these areas were limited by our design options within the EHR. Our efforts focused on children with common general pediatric diagnoses, and it is unclear how our interventions would generalize to medically complex patients with more volume of information to communicate at discharge and with uncommon diagnoses that are less readily incorporated into standardized templates. Relatedly, our work occurred at the satellite campus of our tertiary care center and may not represent generalizable material or methods to implement templates at our main campus location or at other hospitals. To begin to better understand this, we have spread to HM patients at our main campus, including medically complex patients with technology dependence and/or neurological impairments. Standardized, disease-specific templates most relevant to this population as well as several patient specific templates, for those with frequent readmissions due to medical complexity, have been created and are actively being tested.

In conclusion, in using interventions targeted at standardization of discharge instructions and timely feedback to staff, we saw rapid, dramatic, and sustained improvement in the readability of discharge instructions. Next steps include adaptation and spread to other patient populations and care teams, collaborations with other centers, and assessing the impact of effectively written discharge instructions on patient outcomes, such as adverse drug events, readmission rates, and family experience.

Disclosure

No external funding was secured for this study. Dr. Brady is supported by a Patient-Centered Outcomes Research Mentored Clinical Investigator Award from the Agency for Healthcare Research and Quality, Award Number K08HS023827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding organizations. The funding organization had no role in the design, preparation, review, or approval of this paper; nor the decision to submit the manuscript for publication. The authors have no financial relationships relevant to this article to disclose.