Patrick W Brady, MD, MSc

Continuous pulse oximetry monitoring (cSpO₂) in children with bronchiolitis is associated with increased rates of hospital admission, longer lengths of stay, more frequent treatment with supplemental oxygen, alarm fatigue, and higher hospital cost. There is no evidence that it improves clinical outcomes.^1-7 The safety of reducing cSpO₂ for stable bronchiolitis patients (ie, those who are clinically well and not requiring supplemental oxygen) has been assessed in quality improvement initiatives^8-10 and a randomized controlled trial.² These studies showed no increase in intensive care unit transfers, codes, or readmissions associated with reduced cSpO₂. Current national guidelines from the American Academy of Pediatrics⁵ and the Society of Hospital Medicine Choosing Wisely in Pediatric Hospital Medicine workgroup⁴ support limiting monitoring of children with bronchiolitis. Despite this, the practice of cSpO₂ in stable bronchiolitis patients off supplemental oxygen remains widespread.^11,12

Deimplementation, defined as reducing or stopping low-value or ineffective healthcare practices,^13,14 is a discrete focus area within implementation science. Deimplementation research involves the reduction of unnecessary and overused services for which there is potential for harm or no benefit.^15,16 In pediatrics, there are a number of potential targets for deimplementation,^4,17-20 including cSpO₂ for stable infants with bronchiolitis, but efforts to reduce low-value practices have met limited success to date. ^21,22

Implementation science offers rigorous methods for advancing the development and evaluation of strategies for deimplementation.²³ In particular, implementation science frameworks can facilitate our understanding of relevant contextual factors that may hinder or help efforts to deimplement low-value practices. To develop broadly applicable strategies to reduce monitoring overuse, it is important to understand the barriers, facilitators, and contextual factors (eg, clinical, political, interpersonal) that contribute to guideline-discordant cSpO₂ in hospitalized bronchiolitis patients. Further, the process by which one can develop a rigorous understanding of these factors and how they may impact deimplementation efforts could generalize to other scenarios in pediatrics where overuse remains an issue.

The goal of this study was to use semistructured interviews, informed by an established implementation science framework, specifically the Consolidated Framework for Implementation Research (CFIR),²⁴ to (1) identify barriers and facilitators to deimplementing unnecessary cSpO₂, and (2) develop strategies to deimplement cSpO₂ in a multicenter cohort of hospital-based clinician and administrative stakeholders.

Study Setting

This multicenter qualitative study using semistructured interviews took place within the Eliminating Monitor Overuse (EMO) SpO₂ study. The EMO SpO₂ study established rates of cSpO₂ in bronchiolitis patients not receiving supplemental oxygen or not receiving room air flow at 56 hospitals across the United States and in Canada from December 1, 2018, through March 31, 2019.¹² The study identified hospital-level risk-adjusted cSpO₂ rates ranging from 6% to 82%. A description of the EMO SpO₂ study methods²⁵ and its findings¹² have been published elsewhere.

Participants

We approached EMO study site principal investigators at 12 hospitals: the two highest- and two lowest-use hospitals within three hospital types (ie, freestanding children’s hospitals, children’s hospitals within large general hospitals, and community hospitals). We collaborated with the participating site principal investigators (n = 12), who were primarily hospitalist physicians in leadership roles, to recruit a purposive sample of additional stakeholders including bedside nurses (n = 12), hospitalist physicians (n = 15), respiratory therapists (n = 9), and hospital administrators (n = 8) to participate in semistructured interviews. Interviews were conducted until we achieved thematic saturation within each stakeholder group and within the high and low performing strata (total 56 interviews). Participants were asked to self-report basic demographic information (see Appendix, interview guide) as required by the study funder and to allow us to comment on the representativeness of the participant group.

Procedure

The interview guide was informed by the CFIR, a comprehensive framework detailing contextual factors that require consideration when planning for the implementation of a health service intervention. Table 1 details the CFIR domains with study-related examples. The interview guide (Appendix) provided limited clinical context apart from the age, diagnosis, and oxygen requirement for the population of interest to promote a broad array of responses and to avoid anchoring on specific clinical scenarios. Interviews were conducted by master’s degree or doctoral-level research coordinators with qualitative interviewing experience and supervised by a medical anthropologist and qualitative methods expert (F.K.B.). Prior to engaging in audio recorded phone interviews, the interviewer explained the risks and benefits of participating. Participants were compensated $50. Audio recordings were transcribed, deidentified, and uploaded to NVivo 12 Plus (QSR International) for data management.

The Institutional Review Boards of Children’s Hospital of Philadelphia, Pennsylvania, and the University of Pennsylvania in Philadelphia determined that the study met eligibility criteria for IRB exemption.

Data Analysis

Using an integrated approach to codebook development,²⁶ a priori codes were developed using constructs from the CFIR. Additional codes were added by the research team following a close reading of the first five transcripts.^27,28 Each code was defined, including decision rules for its application. Two research coordinators independently coded each transcript. Using the intercoder reliability function within NVivo, the coders established strong interrater reliability accordance scores (k > .8) by double coding 20% of the transcripts. Data were stratified by sites with low and high use of cSpO₂ to examine differences in barriers and facilitators to deimplementation. Each code was subcoded, summarized, and examined for patterns within and across participating disciplines, which yielded themes related to barriers and facilitators. We conducted member checking and reviewed our conclusions with a multidisciplinary group of clinical stakeholders (n = 13) to validate our analyses.

Barriers and facilitators to deimplementation were identified in multiple domains of the CFIR: outer setting, inner setting, characteristics of the individuals, and intervention characteristics (Table 1). Participants also suggested strategies to facilitate deimplementation in response to some identified barriers. See Table 2 for participant demographics and Table 3 for illustrative participant quotations.

Barriers

Outer Setting: Clinician Perceptions of Parental Discomfort With Discontinuing Monitoring

Participants mentioned parental preferences as a barrier to discontinuing cSpO₂, noting that parents seem to take comfort in watching the numbers on the monitor screen and are reluctant to have it withdrawn. Clinicians noted that parents sometimes put the monitor back on their child after a clinician removed it or have expressed concern that their unmonitored child was not receiving the same level of care as other patients who were being monitored. In these scenarios, clinicians reported they have found it helpful to educate caregivers about when cSpO₂ is and is not appropriate.

Inner Setting: Unclear or Nonexistent Guideline to Discontinue cSpO₂

Guidelines to discontinue cSpO₂ reportedly did not exist at all institutions. If a guideline did exist, lack of clarity or conflicting guidelines about when to use oxygen presented a barrier. Participants suggested that a clear guideline or additional oversight to ensure all clinicians are informed of the procedure for discontinuing cSpO₂ may help prevent miscommunication. Participants noted that their electronic health record (EHR) order sets commonly included cSpO₂ orders and that removing that option would facilitate deimplementation.

Inner Setting: Difficulty Educating All Staff

Participants noted difficulty with incorporating education about discontinuing cSpO₂ to all clinicians, particularly to those who are nightshift only or to rotating staff or trainees. This created barriers for frequent re-education because these staff are not familiar with the policies and procedures of the unit, which is crucial to developing a culture that supports the deimplementation of cSpO₂. Participants suggested that recurring education about procedures for discontinuing cSpO₂ should target trainees, new nurses, and overnight nurses. This would help to ensure that the guideline is uniformly followed.

Inner Setting: Culture of High cSpO₂ Use

Participants from high-use sites discussed a culture driven by readily available monitoring features or an expectation that monitoring indicates higher-quality care. Participants from low-use sites discussed increased cSpO₂ driven by clinicians who were accustomed to caring for higher-acuity patients, for whom continuous monitoring is likely appropriate, and were simultaneously caring for stable bronchiolitis patients.

Some suggested that visual cues would be useful to clinicians to sustain awareness about a cSpO₂ deimplementation guideline. It was also suggested that audit and feedback techniques like posting unit deimplementation statistics and creating a competition among units by posting unit performance could facilitate deimplementation. Additionally, some noted that visual aids in common spaces would be useful to remind clinicians and to engage caregivers about discontinuing cSpO₂.

Characteristics of Individuals: Clinician Discomfort Discontinuing cSpO₂

One frequently cited barrier across participants is that cSpO₂ provides “peace of mind” to alert clinicians to patients with low oxygen saturations that might otherwise be missed. Participants identified that clinician discomfort with reducing cSpO₂ may be driven by inexperienced clinicians less familiar with the bronchiolitis disease process, such as trainees, new nurses, or rotating clinicians unaccustomed to pediatric care. Trainees and new nurses were perceived as being more likely to work at night when there are fewer clinicians to provide patient care. Additionally, participants perceived that night shift clinicians favored cSpO₂ because they could measure vital signs without waking patients and families.

Clinicians discussed that discontinuing cSpO₂ would require alternative methods for assessing patient status, particularly for night shift nurses. Participants suggested strategies including changes to pulse oximetry assessment procedures to include more frequent “spot checks,” incorporation of assessments during sleep events (eg, naps) to ensure the patient does not experience desaturations during sleep, and training nurses to become more comfortable with suctioning patients. Suggestions also included education on the typical features of transient oxygen desaturations in otherwise stable patients with bronchiolitis² to bolster clinical confidence for clinicians unfamiliar with caring for bronchiolitis patients. Participants perceived that education about appropriate vs inappropriate use may help to empower clinicians to employ cSpO₂ appropriately.

Facilitators

Outer Setting: Standards and Evidence From Research, Professional Organizations, and Leaders in the Field

Many participants expressed the importance of consistent guidelines that are advocated by thought leaders in the field, supported by robust evidence, and consistent with approaches at peer hospitals. The more authoritative support a guideline has, the more comfortable people are adopting it and taking it seriously. Additionally, consistent education about guidelines was desired. Participants noted that all clinicians should be receiving education related to the American Academy of Pediatrics (AAP) Bronchiolitis and Choosing Wisely^® guidelines, ranging from a one-time update to annually. Continual updates and re-education sessions for clinicians who shared evidence about how cSpO₂ deimplementation could improve the quality of patient care by shortening hospital length of stay and lowering cost were suggested strategies.

Inner Setting: Leadership

Participants noted that successful deimplementation depends upon the presence of a champion or educator who will be able to lead the institutional charge in making practice change. This is typically an individual who is trusted at the institution, experienced in their field, or already doing implementation work. This could be either a single individual (champion) or a team. The most commonly noted clinician roles to engage in a leadership role or team were physicians and nurses.

Participants noted that a change in related clinical care pathways or EHR order sets would require cooperation from multiple clinical disciplines, administrators, and information technology leaders and explained that messaging and education about the value of the change would facilitate buy-in from those clinicians.

Inner Setting: EHR Support for Guidelines

Participants often endorsed the use of an order set within the EHR that supports guidelines and includes reminders to decrease cSpO₂. These reminders could come up when supplemental oxygen is discontinued or occur regularly throughout the patient’s stay to prompt the clinician to consider discontinuing cSpO₂.

Intervention Characteristics/Inner Setting: Clear Bronchiolitis Guidelines

The presence of a well-articulated hospital policy that delineates the appropriate and inappropriate use of cSpO₂ in bronchiolitis was mentioned as another facilitator of deimplementation.

Results of this qualitative study of stakeholders across hospitals with high and low cSpO₂ use illustrated the complexities involved with deimplementation of cSpO₂ in pediatric patients hospitalized with bronchiolitis. We identified numerous barriers spanning the CFIR constructs, including unclear or absent guidelines for stopping cSpO₂, clinician knowledge and comfort with bronchiolitis disease features, and unit culture. This suggests that multicomponent strategies that target various domains and a variety of stakeholders are needed to deimplement cSpO₂ use for stable bronchiolitis patients. Participants also identified facilitators, including clear cSpO₂ guidelines, supportive leaders and champions, and EHR modifications, that provide insight into strategies that may help sites reduce their use of cSpO₂. Additionally, participants also provided concrete, actionable suggestions for ways to reduce unnecessary monitoring that will be useful in informing promising deimplementation strategies for subsequent trials.

The importance of having specific and well-known guidelines from trusted sources, such as the AAP, about cSpO₂ and bronchiolitis treatment that are thoughtfully integrated in the EHR came through in multiple themes of our analysis. Prior studies on the effect of guidelines on clinical practice have suggested that rigorously designed guidelines can positively impact practice.²⁹ Participants also noted that cSpO₂ guidelines should be authoritative and that knowledge of guideline adoption by peer institutions was a facilitator of adoption. Usability issues negatively impact clinicians’ ability to follow guidelines.³⁰ Further, prior studies have demonstrated that EHR integration of guidelines can change practice.^31-33 Based on our findings, incorporating clear guidelines into commonly used formats, such as EHR order sets, could be an important deimplementation tool for cSpO₂ in stable bronchiolitis patients.

Education about and awareness of cSpO₂ guidelines was described as an important facilitator for appropriate cSpO₂ use and was suggested as a potential deimplementation strategy. Participants noted that educational need may vary by stakeholder group. For example, education may facilitate obtaining buy-in from hospital leaders, which is necessary to support changes to the EHR. Education incorporating information on the typical features of bronchiolitis and examples of appropriate and inappropriate cSpO₂ use was suggested for clinical team members. The limitations of education as a stand-alone deimplementation strategy were also noted, and participants highlighted challenges such as time needed for education and the need for ongoing education for rotating trainees. Inner and outer setting barriers, such as a perceived “culture of high pulse oximetryuse” and patient and family expectations, could also make education less effective as a stand-alone strategy. That—coupled with evidence that education and training alone are generally insufficient for producing reliable, sustained behavior change^34,35—suggests that a multifaceted approach will be important.

Our respondents consider parental perceptions and preference in their practice, which provides nuance to recent studies suggesting that parents prefer continuous monitors when their child is hospitalized with bronchiolitis. Chi et al described the impact of a brief educational intervention on parental preferences for monitoring children hospitalized for bronchiolitis.³⁶ This work suggests that educational interventions aimed at families should be considered in future (de)implementation studies because they may indirectly impact clinician behavior. Future studies should directly assess parental discomfort with discontinuing monitoring. Participants highlighted the link between knowledge and confidence in caring for typical bronchiolitis patients and monitoring practice, perceiving that less experienced clinicians are more likely to rely on cSpO₂. Participants at high-use sites emphasized the expectation that monitoring should occur during hospitalizations. This reflection is particularly pertinent for bronchiolitis, a disease characterized by frequent, self-resolving desaturations even after hospital discharge.³ This may reinforce a perceived need to capture and react to these desaturation events even though they are expected in bronchiolitis and can occur in healthy infants.³⁷ Some participants suggested that continuous monitoring be replaced with “nap tests” (ie, assessment for desaturations during a nap prior to discharge); however, like cSpO₂ in stable infants with bronchiolitis, this is another low-value practice. Otherwise healthy infants with mild to moderate disease are unlikely to subsequently worsen after showing signs of clinical improvement.³⁸ Nap tests are likely to lead to infants who are clinically improving being placed unnecessarily back on oxygen in reaction to the transient desaturations. Participants’ perception about the importance of cSpO₂ in bronchiolitis management, despite evidence suggesting it is a low-value practice, underscores the importance of not simply telling clinicians to stop cSpO₂. Employing strategies that replace continuous monitoring with another acceptable and feasible alternative (eg, regular clinician assessments including intermittent pulse oximetry checks) should be considered when planning for deimplementation.³⁹

Previous studies indicate that continuous monitoring can affect clinician decision-making, independent of other factors,^6,40 despite limited evidence that continuous monitors improve patient outcomes.^1-7 Studies have demonstrated noticeable increase in admissions based purely on pulse oximetry values,⁴⁰ with no evidence that this type of admission changes outcomes for bronchiolitis patients.⁶ One previous, single-center study identified inexperience as a potential driver for monitor use,⁴¹ and studies in adult populations have suggested that clinicians overestimate the value that continuous monitoring contributes to patient care,^42,43 which promotes guideline-discordant use. Our study provides novel insight into the issue of monitoring in bronchiolitis. Our results suggest that there is a need to shift organizational cultures around monitoring (which likely vary based on a range of factors) and that educational strategies addressing typical disease course, especially desaturations, in bronchiolitis will be an essential component in any deimplementation effort.

This study is strengthened by its sample of diverse stakeholder groups from multiple US health systems. Additionally, we interviewed individuals at sites with high cSpO₂ rates and at sites with low rates, as well as from community hospitals, children’s hospitals within general hospitals, and freestanding children’s hospitals, which allows us to understand barriers high-use sites encounter and facilitators of lower cSpO₂ rates at low-use sites. We also employed an interview approach informed by an established implementation science framework. Nonetheless, several limitations exist. First, participants at low-use sites did not necessarily have direct experience with a previous deimplementation effort to reduce cSpO₂. Additionally, participants were predominantly White and female; more diverse perspectives would strengthen confidence in the generalizability of our findings. While thematic saturation was achieved within each stakeholder group and within the high- and low-use strata, we interviewed fewer administrators and respiratory therapists relative to other stakeholder groups. Nevertheless, our conclusions were validated by our interdisciplinary stakeholder panel. As noted by participants, family preferences may influence clinician practice, and parents were not interviewed for this study. The information gleaned from the present study will inform the development of strategies to deimplement unnecessary cSpO₂ in pediatric hospitals, which we aim to rigorously evaluate in a future trial.

We identified barriers and facilitators to deimplementation of cSpO₂ for stable patients with bronchiolitis across children’s hospitals with high and low utilization of cSpO₂. These themes map to multiple CFIR domains and, along with participant-suggested strategies, can directly inform an approach to cSpO₂ deimplementation in a range of inpatient settings. Based on these data, future deimplementation efforts should focus on clear protocols for use and discontinuation of cSpO₂, EHR changes, and regular bronchiolitis education for hospital staff that emphasizes reducing unnecessary cSpO₂ utilization.

We acknowledge the NHLBI scientists who contributed their expertise to this project as part of the U01 Cooperative Agreement funding mechanism as federal employees conducting their official job duties: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD. We thank the Executive Council of the Pediatric Research in Inpatient Settings (PRIS) Network for their contributions to the early scientific development of this project. The Network assessed a Collaborative Support Fee for access to the hospitals and support of this project. We thank the PRIS Network collaborators for their major contributions to data collection measuring utilization to identify the hospitals we subsequently chose for this project. We thank Claire Bocage and the Mixed Methods Research Lab for major help in data management and data analysis.

Continuous pulse oximetry monitoring (cSpO₂) in children with bronchiolitis is associated with increased rates of hospital admission, longer lengths of stay, more frequent treatment with supplemental oxygen, alarm fatigue, and higher hospital cost. There is no evidence that it improves clinical outcomes.^1-7 The safety of reducing cSpO₂ for stable bronchiolitis patients (ie, those who are clinically well and not requiring supplemental oxygen) has been assessed in quality improvement initiatives^8-10 and a randomized controlled trial.² These studies showed no increase in intensive care unit transfers, codes, or readmissions associated with reduced cSpO₂. Current national guidelines from the American Academy of Pediatrics⁵ and the Society of Hospital Medicine Choosing Wisely in Pediatric Hospital Medicine workgroup⁴ support limiting monitoring of children with bronchiolitis. Despite this, the practice of cSpO₂ in stable bronchiolitis patients off supplemental oxygen remains widespread.^11,12

Deimplementation, defined as reducing or stopping low-value or ineffective healthcare practices,^13,14 is a discrete focus area within implementation science. Deimplementation research involves the reduction of unnecessary and overused services for which there is potential for harm or no benefit.^15,16 In pediatrics, there are a number of potential targets for deimplementation,^4,17-20 including cSpO₂ for stable infants with bronchiolitis, but efforts to reduce low-value practices have met limited success to date. ^21,22

Implementation science offers rigorous methods for advancing the development and evaluation of strategies for deimplementation.²³ In particular, implementation science frameworks can facilitate our understanding of relevant contextual factors that may hinder or help efforts to deimplement low-value practices. To develop broadly applicable strategies to reduce monitoring overuse, it is important to understand the barriers, facilitators, and contextual factors (eg, clinical, political, interpersonal) that contribute to guideline-discordant cSpO₂ in hospitalized bronchiolitis patients. Further, the process by which one can develop a rigorous understanding of these factors and how they may impact deimplementation efforts could generalize to other scenarios in pediatrics where overuse remains an issue.

The goal of this study was to use semistructured interviews, informed by an established implementation science framework, specifically the Consolidated Framework for Implementation Research (CFIR),²⁴ to (1) identify barriers and facilitators to deimplementing unnecessary cSpO₂, and (2) develop strategies to deimplement cSpO₂ in a multicenter cohort of hospital-based clinician and administrative stakeholders.

Study Setting

This multicenter qualitative study using semistructured interviews took place within the Eliminating Monitor Overuse (EMO) SpO₂ study. The EMO SpO₂ study established rates of cSpO₂ in bronchiolitis patients not receiving supplemental oxygen or not receiving room air flow at 56 hospitals across the United States and in Canada from December 1, 2018, through March 31, 2019.¹² The study identified hospital-level risk-adjusted cSpO₂ rates ranging from 6% to 82%. A description of the EMO SpO₂ study methods²⁵ and its findings¹² have been published elsewhere.

Participants

We approached EMO study site principal investigators at 12 hospitals: the two highest- and two lowest-use hospitals within three hospital types (ie, freestanding children’s hospitals, children’s hospitals within large general hospitals, and community hospitals). We collaborated with the participating site principal investigators (n = 12), who were primarily hospitalist physicians in leadership roles, to recruit a purposive sample of additional stakeholders including bedside nurses (n = 12), hospitalist physicians (n = 15), respiratory therapists (n = 9), and hospital administrators (n = 8) to participate in semistructured interviews. Interviews were conducted until we achieved thematic saturation within each stakeholder group and within the high and low performing strata (total 56 interviews). Participants were asked to self-report basic demographic information (see Appendix, interview guide) as required by the study funder and to allow us to comment on the representativeness of the participant group.

Procedure

The interview guide was informed by the CFIR, a comprehensive framework detailing contextual factors that require consideration when planning for the implementation of a health service intervention. Table 1 details the CFIR domains with study-related examples. The interview guide (Appendix) provided limited clinical context apart from the age, diagnosis, and oxygen requirement for the population of interest to promote a broad array of responses and to avoid anchoring on specific clinical scenarios. Interviews were conducted by master’s degree or doctoral-level research coordinators with qualitative interviewing experience and supervised by a medical anthropologist and qualitative methods expert (F.K.B.). Prior to engaging in audio recorded phone interviews, the interviewer explained the risks and benefits of participating. Participants were compensated $50. Audio recordings were transcribed, deidentified, and uploaded to NVivo 12 Plus (QSR International) for data management.

The Institutional Review Boards of Children’s Hospital of Philadelphia, Pennsylvania, and the University of Pennsylvania in Philadelphia determined that the study met eligibility criteria for IRB exemption.

Data Analysis

Using an integrated approach to codebook development,²⁶ a priori codes were developed using constructs from the CFIR. Additional codes were added by the research team following a close reading of the first five transcripts.^27,28 Each code was defined, including decision rules for its application. Two research coordinators independently coded each transcript. Using the intercoder reliability function within NVivo, the coders established strong interrater reliability accordance scores (k > .8) by double coding 20% of the transcripts. Data were stratified by sites with low and high use of cSpO₂ to examine differences in barriers and facilitators to deimplementation. Each code was subcoded, summarized, and examined for patterns within and across participating disciplines, which yielded themes related to barriers and facilitators. We conducted member checking and reviewed our conclusions with a multidisciplinary group of clinical stakeholders (n = 13) to validate our analyses.

Barriers and facilitators to deimplementation were identified in multiple domains of the CFIR: outer setting, inner setting, characteristics of the individuals, and intervention characteristics (Table 1). Participants also suggested strategies to facilitate deimplementation in response to some identified barriers. See Table 2 for participant demographics and Table 3 for illustrative participant quotations.

Barriers

Outer Setting: Clinician Perceptions of Parental Discomfort With Discontinuing Monitoring

Participants mentioned parental preferences as a barrier to discontinuing cSpO₂, noting that parents seem to take comfort in watching the numbers on the monitor screen and are reluctant to have it withdrawn. Clinicians noted that parents sometimes put the monitor back on their child after a clinician removed it or have expressed concern that their unmonitored child was not receiving the same level of care as other patients who were being monitored. In these scenarios, clinicians reported they have found it helpful to educate caregivers about when cSpO₂ is and is not appropriate.

Inner Setting: Unclear or Nonexistent Guideline to Discontinue cSpO₂

Guidelines to discontinue cSpO₂ reportedly did not exist at all institutions. If a guideline did exist, lack of clarity or conflicting guidelines about when to use oxygen presented a barrier. Participants suggested that a clear guideline or additional oversight to ensure all clinicians are informed of the procedure for discontinuing cSpO₂ may help prevent miscommunication. Participants noted that their electronic health record (EHR) order sets commonly included cSpO₂ orders and that removing that option would facilitate deimplementation.

Inner Setting: Difficulty Educating All Staff

Participants noted difficulty with incorporating education about discontinuing cSpO₂ to all clinicians, particularly to those who are nightshift only or to rotating staff or trainees. This created barriers for frequent re-education because these staff are not familiar with the policies and procedures of the unit, which is crucial to developing a culture that supports the deimplementation of cSpO₂. Participants suggested that recurring education about procedures for discontinuing cSpO₂ should target trainees, new nurses, and overnight nurses. This would help to ensure that the guideline is uniformly followed.

Inner Setting: Culture of High cSpO₂ Use

Participants from high-use sites discussed a culture driven by readily available monitoring features or an expectation that monitoring indicates higher-quality care. Participants from low-use sites discussed increased cSpO₂ driven by clinicians who were accustomed to caring for higher-acuity patients, for whom continuous monitoring is likely appropriate, and were simultaneously caring for stable bronchiolitis patients.

Some suggested that visual cues would be useful to clinicians to sustain awareness about a cSpO₂ deimplementation guideline. It was also suggested that audit and feedback techniques like posting unit deimplementation statistics and creating a competition among units by posting unit performance could facilitate deimplementation. Additionally, some noted that visual aids in common spaces would be useful to remind clinicians and to engage caregivers about discontinuing cSpO₂.

Characteristics of Individuals: Clinician Discomfort Discontinuing cSpO₂

One frequently cited barrier across participants is that cSpO₂ provides “peace of mind” to alert clinicians to patients with low oxygen saturations that might otherwise be missed. Participants identified that clinician discomfort with reducing cSpO₂ may be driven by inexperienced clinicians less familiar with the bronchiolitis disease process, such as trainees, new nurses, or rotating clinicians unaccustomed to pediatric care. Trainees and new nurses were perceived as being more likely to work at night when there are fewer clinicians to provide patient care. Additionally, participants perceived that night shift clinicians favored cSpO₂ because they could measure vital signs without waking patients and families.

Clinicians discussed that discontinuing cSpO₂ would require alternative methods for assessing patient status, particularly for night shift nurses. Participants suggested strategies including changes to pulse oximetry assessment procedures to include more frequent “spot checks,” incorporation of assessments during sleep events (eg, naps) to ensure the patient does not experience desaturations during sleep, and training nurses to become more comfortable with suctioning patients. Suggestions also included education on the typical features of transient oxygen desaturations in otherwise stable patients with bronchiolitis² to bolster clinical confidence for clinicians unfamiliar with caring for bronchiolitis patients. Participants perceived that education about appropriate vs inappropriate use may help to empower clinicians to employ cSpO₂ appropriately.

Facilitators

Outer Setting: Standards and Evidence From Research, Professional Organizations, and Leaders in the Field

Many participants expressed the importance of consistent guidelines that are advocated by thought leaders in the field, supported by robust evidence, and consistent with approaches at peer hospitals. The more authoritative support a guideline has, the more comfortable people are adopting it and taking it seriously. Additionally, consistent education about guidelines was desired. Participants noted that all clinicians should be receiving education related to the American Academy of Pediatrics (AAP) Bronchiolitis and Choosing Wisely^® guidelines, ranging from a one-time update to annually. Continual updates and re-education sessions for clinicians who shared evidence about how cSpO₂ deimplementation could improve the quality of patient care by shortening hospital length of stay and lowering cost were suggested strategies.

Inner Setting: Leadership

Participants noted that successful deimplementation depends upon the presence of a champion or educator who will be able to lead the institutional charge in making practice change. This is typically an individual who is trusted at the institution, experienced in their field, or already doing implementation work. This could be either a single individual (champion) or a team. The most commonly noted clinician roles to engage in a leadership role or team were physicians and nurses.

Participants noted that a change in related clinical care pathways or EHR order sets would require cooperation from multiple clinical disciplines, administrators, and information technology leaders and explained that messaging and education about the value of the change would facilitate buy-in from those clinicians.

Inner Setting: EHR Support for Guidelines

Participants often endorsed the use of an order set within the EHR that supports guidelines and includes reminders to decrease cSpO₂. These reminders could come up when supplemental oxygen is discontinued or occur regularly throughout the patient’s stay to prompt the clinician to consider discontinuing cSpO₂.

Intervention Characteristics/Inner Setting: Clear Bronchiolitis Guidelines

The presence of a well-articulated hospital policy that delineates the appropriate and inappropriate use of cSpO₂ in bronchiolitis was mentioned as another facilitator of deimplementation.

Results of this qualitative study of stakeholders across hospitals with high and low cSpO₂ use illustrated the complexities involved with deimplementation of cSpO₂ in pediatric patients hospitalized with bronchiolitis. We identified numerous barriers spanning the CFIR constructs, including unclear or absent guidelines for stopping cSpO₂, clinician knowledge and comfort with bronchiolitis disease features, and unit culture. This suggests that multicomponent strategies that target various domains and a variety of stakeholders are needed to deimplement cSpO₂ use for stable bronchiolitis patients. Participants also identified facilitators, including clear cSpO₂ guidelines, supportive leaders and champions, and EHR modifications, that provide insight into strategies that may help sites reduce their use of cSpO₂. Additionally, participants also provided concrete, actionable suggestions for ways to reduce unnecessary monitoring that will be useful in informing promising deimplementation strategies for subsequent trials.

The importance of having specific and well-known guidelines from trusted sources, such as the AAP, about cSpO₂ and bronchiolitis treatment that are thoughtfully integrated in the EHR came through in multiple themes of our analysis. Prior studies on the effect of guidelines on clinical practice have suggested that rigorously designed guidelines can positively impact practice.²⁹ Participants also noted that cSpO₂ guidelines should be authoritative and that knowledge of guideline adoption by peer institutions was a facilitator of adoption. Usability issues negatively impact clinicians’ ability to follow guidelines.³⁰ Further, prior studies have demonstrated that EHR integration of guidelines can change practice.^31-33 Based on our findings, incorporating clear guidelines into commonly used formats, such as EHR order sets, could be an important deimplementation tool for cSpO₂ in stable bronchiolitis patients.

Education about and awareness of cSpO₂ guidelines was described as an important facilitator for appropriate cSpO₂ use and was suggested as a potential deimplementation strategy. Participants noted that educational need may vary by stakeholder group. For example, education may facilitate obtaining buy-in from hospital leaders, which is necessary to support changes to the EHR. Education incorporating information on the typical features of bronchiolitis and examples of appropriate and inappropriate cSpO₂ use was suggested for clinical team members. The limitations of education as a stand-alone deimplementation strategy were also noted, and participants highlighted challenges such as time needed for education and the need for ongoing education for rotating trainees. Inner and outer setting barriers, such as a perceived “culture of high pulse oximetryuse” and patient and family expectations, could also make education less effective as a stand-alone strategy. That—coupled with evidence that education and training alone are generally insufficient for producing reliable, sustained behavior change^34,35—suggests that a multifaceted approach will be important.

Our respondents consider parental perceptions and preference in their practice, which provides nuance to recent studies suggesting that parents prefer continuous monitors when their child is hospitalized with bronchiolitis. Chi et al described the impact of a brief educational intervention on parental preferences for monitoring children hospitalized for bronchiolitis.³⁶ This work suggests that educational interventions aimed at families should be considered in future (de)implementation studies because they may indirectly impact clinician behavior. Future studies should directly assess parental discomfort with discontinuing monitoring. Participants highlighted the link between knowledge and confidence in caring for typical bronchiolitis patients and monitoring practice, perceiving that less experienced clinicians are more likely to rely on cSpO₂. Participants at high-use sites emphasized the expectation that monitoring should occur during hospitalizations. This reflection is particularly pertinent for bronchiolitis, a disease characterized by frequent, self-resolving desaturations even after hospital discharge.³ This may reinforce a perceived need to capture and react to these desaturation events even though they are expected in bronchiolitis and can occur in healthy infants.³⁷ Some participants suggested that continuous monitoring be replaced with “nap tests” (ie, assessment for desaturations during a nap prior to discharge); however, like cSpO₂ in stable infants with bronchiolitis, this is another low-value practice. Otherwise healthy infants with mild to moderate disease are unlikely to subsequently worsen after showing signs of clinical improvement.³⁸ Nap tests are likely to lead to infants who are clinically improving being placed unnecessarily back on oxygen in reaction to the transient desaturations. Participants’ perception about the importance of cSpO₂ in bronchiolitis management, despite evidence suggesting it is a low-value practice, underscores the importance of not simply telling clinicians to stop cSpO₂. Employing strategies that replace continuous monitoring with another acceptable and feasible alternative (eg, regular clinician assessments including intermittent pulse oximetry checks) should be considered when planning for deimplementation.³⁹

Previous studies indicate that continuous monitoring can affect clinician decision-making, independent of other factors,^6,40 despite limited evidence that continuous monitors improve patient outcomes.^1-7 Studies have demonstrated noticeable increase in admissions based purely on pulse oximetry values,⁴⁰ with no evidence that this type of admission changes outcomes for bronchiolitis patients.⁶ One previous, single-center study identified inexperience as a potential driver for monitor use,⁴¹ and studies in adult populations have suggested that clinicians overestimate the value that continuous monitoring contributes to patient care,^42,43 which promotes guideline-discordant use. Our study provides novel insight into the issue of monitoring in bronchiolitis. Our results suggest that there is a need to shift organizational cultures around monitoring (which likely vary based on a range of factors) and that educational strategies addressing typical disease course, especially desaturations, in bronchiolitis will be an essential component in any deimplementation effort.

This study is strengthened by its sample of diverse stakeholder groups from multiple US health systems. Additionally, we interviewed individuals at sites with high cSpO₂ rates and at sites with low rates, as well as from community hospitals, children’s hospitals within general hospitals, and freestanding children’s hospitals, which allows us to understand barriers high-use sites encounter and facilitators of lower cSpO₂ rates at low-use sites. We also employed an interview approach informed by an established implementation science framework. Nonetheless, several limitations exist. First, participants at low-use sites did not necessarily have direct experience with a previous deimplementation effort to reduce cSpO₂. Additionally, participants were predominantly White and female; more diverse perspectives would strengthen confidence in the generalizability of our findings. While thematic saturation was achieved within each stakeholder group and within the high- and low-use strata, we interviewed fewer administrators and respiratory therapists relative to other stakeholder groups. Nevertheless, our conclusions were validated by our interdisciplinary stakeholder panel. As noted by participants, family preferences may influence clinician practice, and parents were not interviewed for this study. The information gleaned from the present study will inform the development of strategies to deimplement unnecessary cSpO₂ in pediatric hospitals, which we aim to rigorously evaluate in a future trial.

We identified barriers and facilitators to deimplementation of cSpO₂ for stable patients with bronchiolitis across children’s hospitals with high and low utilization of cSpO₂. These themes map to multiple CFIR domains and, along with participant-suggested strategies, can directly inform an approach to cSpO₂ deimplementation in a range of inpatient settings. Based on these data, future deimplementation efforts should focus on clear protocols for use and discontinuation of cSpO₂, EHR changes, and regular bronchiolitis education for hospital staff that emphasizes reducing unnecessary cSpO₂ utilization.

We acknowledge the NHLBI scientists who contributed their expertise to this project as part of the U01 Cooperative Agreement funding mechanism as federal employees conducting their official job duties: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD. We thank the Executive Council of the Pediatric Research in Inpatient Settings (PRIS) Network for their contributions to the early scientific development of this project. The Network assessed a Collaborative Support Fee for access to the hospitals and support of this project. We thank the PRIS Network collaborators for their major contributions to data collection measuring utilization to identify the hospitals we subsequently chose for this project. We thank Claire Bocage and the Mixed Methods Research Lab for major help in data management and data analysis.

Continuous pulse oximetry monitoring (cSpO₂) in children with bronchiolitis is associated with increased rates of hospital admission, longer lengths of stay, more frequent treatment with supplemental oxygen, alarm fatigue, and higher hospital cost. There is no evidence that it improves clinical outcomes.^1-7 The safety of reducing cSpO₂ for stable bronchiolitis patients (ie, those who are clinically well and not requiring supplemental oxygen) has been assessed in quality improvement initiatives^8-10 and a randomized controlled trial.² These studies showed no increase in intensive care unit transfers, codes, or readmissions associated with reduced cSpO₂. Current national guidelines from the American Academy of Pediatrics⁵ and the Society of Hospital Medicine Choosing Wisely in Pediatric Hospital Medicine workgroup⁴ support limiting monitoring of children with bronchiolitis. Despite this, the practice of cSpO₂ in stable bronchiolitis patients off supplemental oxygen remains widespread.^11,12

Deimplementation, defined as reducing or stopping low-value or ineffective healthcare practices,^13,14 is a discrete focus area within implementation science. Deimplementation research involves the reduction of unnecessary and overused services for which there is potential for harm or no benefit.^15,16 In pediatrics, there are a number of potential targets for deimplementation,^4,17-20 including cSpO₂ for stable infants with bronchiolitis, but efforts to reduce low-value practices have met limited success to date. ^21,22

Implementation science offers rigorous methods for advancing the development and evaluation of strategies for deimplementation.²³ In particular, implementation science frameworks can facilitate our understanding of relevant contextual factors that may hinder or help efforts to deimplement low-value practices. To develop broadly applicable strategies to reduce monitoring overuse, it is important to understand the barriers, facilitators, and contextual factors (eg, clinical, political, interpersonal) that contribute to guideline-discordant cSpO₂ in hospitalized bronchiolitis patients. Further, the process by which one can develop a rigorous understanding of these factors and how they may impact deimplementation efforts could generalize to other scenarios in pediatrics where overuse remains an issue.

The goal of this study was to use semistructured interviews, informed by an established implementation science framework, specifically the Consolidated Framework for Implementation Research (CFIR),²⁴ to (1) identify barriers and facilitators to deimplementing unnecessary cSpO₂, and (2) develop strategies to deimplement cSpO₂ in a multicenter cohort of hospital-based clinician and administrative stakeholders.

Study Setting

This multicenter qualitative study using semistructured interviews took place within the Eliminating Monitor Overuse (EMO) SpO₂ study. The EMO SpO₂ study established rates of cSpO₂ in bronchiolitis patients not receiving supplemental oxygen or not receiving room air flow at 56 hospitals across the United States and in Canada from December 1, 2018, through March 31, 2019.¹² The study identified hospital-level risk-adjusted cSpO₂ rates ranging from 6% to 82%. A description of the EMO SpO₂ study methods²⁵ and its findings¹² have been published elsewhere.

Participants

We approached EMO study site principal investigators at 12 hospitals: the two highest- and two lowest-use hospitals within three hospital types (ie, freestanding children’s hospitals, children’s hospitals within large general hospitals, and community hospitals). We collaborated with the participating site principal investigators (n = 12), who were primarily hospitalist physicians in leadership roles, to recruit a purposive sample of additional stakeholders including bedside nurses (n = 12), hospitalist physicians (n = 15), respiratory therapists (n = 9), and hospital administrators (n = 8) to participate in semistructured interviews. Interviews were conducted until we achieved thematic saturation within each stakeholder group and within the high and low performing strata (total 56 interviews). Participants were asked to self-report basic demographic information (see Appendix, interview guide) as required by the study funder and to allow us to comment on the representativeness of the participant group.

Procedure

The interview guide was informed by the CFIR, a comprehensive framework detailing contextual factors that require consideration when planning for the implementation of a health service intervention. Table 1 details the CFIR domains with study-related examples. The interview guide (Appendix) provided limited clinical context apart from the age, diagnosis, and oxygen requirement for the population of interest to promote a broad array of responses and to avoid anchoring on specific clinical scenarios. Interviews were conducted by master’s degree or doctoral-level research coordinators with qualitative interviewing experience and supervised by a medical anthropologist and qualitative methods expert (F.K.B.). Prior to engaging in audio recorded phone interviews, the interviewer explained the risks and benefits of participating. Participants were compensated $50. Audio recordings were transcribed, deidentified, and uploaded to NVivo 12 Plus (QSR International) for data management.

The Institutional Review Boards of Children’s Hospital of Philadelphia, Pennsylvania, and the University of Pennsylvania in Philadelphia determined that the study met eligibility criteria for IRB exemption.

Data Analysis

Using an integrated approach to codebook development,²⁶ a priori codes were developed using constructs from the CFIR. Additional codes were added by the research team following a close reading of the first five transcripts.^27,28 Each code was defined, including decision rules for its application. Two research coordinators independently coded each transcript. Using the intercoder reliability function within NVivo, the coders established strong interrater reliability accordance scores (k > .8) by double coding 20% of the transcripts. Data were stratified by sites with low and high use of cSpO₂ to examine differences in barriers and facilitators to deimplementation. Each code was subcoded, summarized, and examined for patterns within and across participating disciplines, which yielded themes related to barriers and facilitators. We conducted member checking and reviewed our conclusions with a multidisciplinary group of clinical stakeholders (n = 13) to validate our analyses.

Barriers and facilitators to deimplementation were identified in multiple domains of the CFIR: outer setting, inner setting, characteristics of the individuals, and intervention characteristics (Table 1). Participants also suggested strategies to facilitate deimplementation in response to some identified barriers. See Table 2 for participant demographics and Table 3 for illustrative participant quotations.

Barriers

Outer Setting: Clinician Perceptions of Parental Discomfort With Discontinuing Monitoring

Participants mentioned parental preferences as a barrier to discontinuing cSpO₂, noting that parents seem to take comfort in watching the numbers on the monitor screen and are reluctant to have it withdrawn. Clinicians noted that parents sometimes put the monitor back on their child after a clinician removed it or have expressed concern that their unmonitored child was not receiving the same level of care as other patients who were being monitored. In these scenarios, clinicians reported they have found it helpful to educate caregivers about when cSpO₂ is and is not appropriate.

Inner Setting: Unclear or Nonexistent Guideline to Discontinue cSpO₂

Guidelines to discontinue cSpO₂ reportedly did not exist at all institutions. If a guideline did exist, lack of clarity or conflicting guidelines about when to use oxygen presented a barrier. Participants suggested that a clear guideline or additional oversight to ensure all clinicians are informed of the procedure for discontinuing cSpO₂ may help prevent miscommunication. Participants noted that their electronic health record (EHR) order sets commonly included cSpO₂ orders and that removing that option would facilitate deimplementation.

Inner Setting: Difficulty Educating All Staff

Participants noted difficulty with incorporating education about discontinuing cSpO₂ to all clinicians, particularly to those who are nightshift only or to rotating staff or trainees. This created barriers for frequent re-education because these staff are not familiar with the policies and procedures of the unit, which is crucial to developing a culture that supports the deimplementation of cSpO₂. Participants suggested that recurring education about procedures for discontinuing cSpO₂ should target trainees, new nurses, and overnight nurses. This would help to ensure that the guideline is uniformly followed.

Inner Setting: Culture of High cSpO₂ Use

Participants from high-use sites discussed a culture driven by readily available monitoring features or an expectation that monitoring indicates higher-quality care. Participants from low-use sites discussed increased cSpO₂ driven by clinicians who were accustomed to caring for higher-acuity patients, for whom continuous monitoring is likely appropriate, and were simultaneously caring for stable bronchiolitis patients.

Some suggested that visual cues would be useful to clinicians to sustain awareness about a cSpO₂ deimplementation guideline. It was also suggested that audit and feedback techniques like posting unit deimplementation statistics and creating a competition among units by posting unit performance could facilitate deimplementation. Additionally, some noted that visual aids in common spaces would be useful to remind clinicians and to engage caregivers about discontinuing cSpO₂.

Characteristics of Individuals: Clinician Discomfort Discontinuing cSpO₂

One frequently cited barrier across participants is that cSpO₂ provides “peace of mind” to alert clinicians to patients with low oxygen saturations that might otherwise be missed. Participants identified that clinician discomfort with reducing cSpO₂ may be driven by inexperienced clinicians less familiar with the bronchiolitis disease process, such as trainees, new nurses, or rotating clinicians unaccustomed to pediatric care. Trainees and new nurses were perceived as being more likely to work at night when there are fewer clinicians to provide patient care. Additionally, participants perceived that night shift clinicians favored cSpO₂ because they could measure vital signs without waking patients and families.

Clinicians discussed that discontinuing cSpO₂ would require alternative methods for assessing patient status, particularly for night shift nurses. Participants suggested strategies including changes to pulse oximetry assessment procedures to include more frequent “spot checks,” incorporation of assessments during sleep events (eg, naps) to ensure the patient does not experience desaturations during sleep, and training nurses to become more comfortable with suctioning patients. Suggestions also included education on the typical features of transient oxygen desaturations in otherwise stable patients with bronchiolitis² to bolster clinical confidence for clinicians unfamiliar with caring for bronchiolitis patients. Participants perceived that education about appropriate vs inappropriate use may help to empower clinicians to employ cSpO₂ appropriately.

Facilitators

Outer Setting: Standards and Evidence From Research, Professional Organizations, and Leaders in the Field

Many participants expressed the importance of consistent guidelines that are advocated by thought leaders in the field, supported by robust evidence, and consistent with approaches at peer hospitals. The more authoritative support a guideline has, the more comfortable people are adopting it and taking it seriously. Additionally, consistent education about guidelines was desired. Participants noted that all clinicians should be receiving education related to the American Academy of Pediatrics (AAP) Bronchiolitis and Choosing Wisely^® guidelines, ranging from a one-time update to annually. Continual updates and re-education sessions for clinicians who shared evidence about how cSpO₂ deimplementation could improve the quality of patient care by shortening hospital length of stay and lowering cost were suggested strategies.

Inner Setting: Leadership

Participants noted that successful deimplementation depends upon the presence of a champion or educator who will be able to lead the institutional charge in making practice change. This is typically an individual who is trusted at the institution, experienced in their field, or already doing implementation work. This could be either a single individual (champion) or a team. The most commonly noted clinician roles to engage in a leadership role or team were physicians and nurses.

Participants noted that a change in related clinical care pathways or EHR order sets would require cooperation from multiple clinical disciplines, administrators, and information technology leaders and explained that messaging and education about the value of the change would facilitate buy-in from those clinicians.

Inner Setting: EHR Support for Guidelines

Participants often endorsed the use of an order set within the EHR that supports guidelines and includes reminders to decrease cSpO₂. These reminders could come up when supplemental oxygen is discontinued or occur regularly throughout the patient’s stay to prompt the clinician to consider discontinuing cSpO₂.

Intervention Characteristics/Inner Setting: Clear Bronchiolitis Guidelines

The presence of a well-articulated hospital policy that delineates the appropriate and inappropriate use of cSpO₂ in bronchiolitis was mentioned as another facilitator of deimplementation.

Results of this qualitative study of stakeholders across hospitals with high and low cSpO₂ use illustrated the complexities involved with deimplementation of cSpO₂ in pediatric patients hospitalized with bronchiolitis. We identified numerous barriers spanning the CFIR constructs, including unclear or absent guidelines for stopping cSpO₂, clinician knowledge and comfort with bronchiolitis disease features, and unit culture. This suggests that multicomponent strategies that target various domains and a variety of stakeholders are needed to deimplement cSpO₂ use for stable bronchiolitis patients. Participants also identified facilitators, including clear cSpO₂ guidelines, supportive leaders and champions, and EHR modifications, that provide insight into strategies that may help sites reduce their use of cSpO₂. Additionally, participants also provided concrete, actionable suggestions for ways to reduce unnecessary monitoring that will be useful in informing promising deimplementation strategies for subsequent trials.

The importance of having specific and well-known guidelines from trusted sources, such as the AAP, about cSpO₂ and bronchiolitis treatment that are thoughtfully integrated in the EHR came through in multiple themes of our analysis. Prior studies on the effect of guidelines on clinical practice have suggested that rigorously designed guidelines can positively impact practice.²⁹ Participants also noted that cSpO₂ guidelines should be authoritative and that knowledge of guideline adoption by peer institutions was a facilitator of adoption. Usability issues negatively impact clinicians’ ability to follow guidelines.³⁰ Further, prior studies have demonstrated that EHR integration of guidelines can change practice.^31-33 Based on our findings, incorporating clear guidelines into commonly used formats, such as EHR order sets, could be an important deimplementation tool for cSpO₂ in stable bronchiolitis patients.

Education about and awareness of cSpO₂ guidelines was described as an important facilitator for appropriate cSpO₂ use and was suggested as a potential deimplementation strategy. Participants noted that educational need may vary by stakeholder group. For example, education may facilitate obtaining buy-in from hospital leaders, which is necessary to support changes to the EHR. Education incorporating information on the typical features of bronchiolitis and examples of appropriate and inappropriate cSpO₂ use was suggested for clinical team members. The limitations of education as a stand-alone deimplementation strategy were also noted, and participants highlighted challenges such as time needed for education and the need for ongoing education for rotating trainees. Inner and outer setting barriers, such as a perceived “culture of high pulse oximetryuse” and patient and family expectations, could also make education less effective as a stand-alone strategy. That—coupled with evidence that education and training alone are generally insufficient for producing reliable, sustained behavior change^34,35—suggests that a multifaceted approach will be important.

Our respondents consider parental perceptions and preference in their practice, which provides nuance to recent studies suggesting that parents prefer continuous monitors when their child is hospitalized with bronchiolitis. Chi et al described the impact of a brief educational intervention on parental preferences for monitoring children hospitalized for bronchiolitis.³⁶ This work suggests that educational interventions aimed at families should be considered in future (de)implementation studies because they may indirectly impact clinician behavior. Future studies should directly assess parental discomfort with discontinuing monitoring. Participants highlighted the link between knowledge and confidence in caring for typical bronchiolitis patients and monitoring practice, perceiving that less experienced clinicians are more likely to rely on cSpO₂. Participants at high-use sites emphasized the expectation that monitoring should occur during hospitalizations. This reflection is particularly pertinent for bronchiolitis, a disease characterized by frequent, self-resolving desaturations even after hospital discharge.³ This may reinforce a perceived need to capture and react to these desaturation events even though they are expected in bronchiolitis and can occur in healthy infants.³⁷ Some participants suggested that continuous monitoring be replaced with “nap tests” (ie, assessment for desaturations during a nap prior to discharge); however, like cSpO₂ in stable infants with bronchiolitis, this is another low-value practice. Otherwise healthy infants with mild to moderate disease are unlikely to subsequently worsen after showing signs of clinical improvement.³⁸ Nap tests are likely to lead to infants who are clinically improving being placed unnecessarily back on oxygen in reaction to the transient desaturations. Participants’ perception about the importance of cSpO₂ in bronchiolitis management, despite evidence suggesting it is a low-value practice, underscores the importance of not simply telling clinicians to stop cSpO₂. Employing strategies that replace continuous monitoring with another acceptable and feasible alternative (eg, regular clinician assessments including intermittent pulse oximetry checks) should be considered when planning for deimplementation.³⁹

Previous studies indicate that continuous monitoring can affect clinician decision-making, independent of other factors,^6,40 despite limited evidence that continuous monitors improve patient outcomes.^1-7 Studies have demonstrated noticeable increase in admissions based purely on pulse oximetry values,⁴⁰ with no evidence that this type of admission changes outcomes for bronchiolitis patients.⁶ One previous, single-center study identified inexperience as a potential driver for monitor use,⁴¹ and studies in adult populations have suggested that clinicians overestimate the value that continuous monitoring contributes to patient care,^42,43 which promotes guideline-discordant use. Our study provides novel insight into the issue of monitoring in bronchiolitis. Our results suggest that there is a need to shift organizational cultures around monitoring (which likely vary based on a range of factors) and that educational strategies addressing typical disease course, especially desaturations, in bronchiolitis will be an essential component in any deimplementation effort.

This study is strengthened by its sample of diverse stakeholder groups from multiple US health systems. Additionally, we interviewed individuals at sites with high cSpO₂ rates and at sites with low rates, as well as from community hospitals, children’s hospitals within general hospitals, and freestanding children’s hospitals, which allows us to understand barriers high-use sites encounter and facilitators of lower cSpO₂ rates at low-use sites. We also employed an interview approach informed by an established implementation science framework. Nonetheless, several limitations exist. First, participants at low-use sites did not necessarily have direct experience with a previous deimplementation effort to reduce cSpO₂. Additionally, participants were predominantly White and female; more diverse perspectives would strengthen confidence in the generalizability of our findings. While thematic saturation was achieved within each stakeholder group and within the high- and low-use strata, we interviewed fewer administrators and respiratory therapists relative to other stakeholder groups. Nevertheless, our conclusions were validated by our interdisciplinary stakeholder panel. As noted by participants, family preferences may influence clinician practice, and parents were not interviewed for this study. The information gleaned from the present study will inform the development of strategies to deimplement unnecessary cSpO₂ in pediatric hospitals, which we aim to rigorously evaluate in a future trial.

We identified barriers and facilitators to deimplementation of cSpO₂ for stable patients with bronchiolitis across children’s hospitals with high and low utilization of cSpO₂. These themes map to multiple CFIR domains and, along with participant-suggested strategies, can directly inform an approach to cSpO₂ deimplementation in a range of inpatient settings. Based on these data, future deimplementation efforts should focus on clear protocols for use and discontinuation of cSpO₂, EHR changes, and regular bronchiolitis education for hospital staff that emphasizes reducing unnecessary cSpO₂ utilization.

We acknowledge the NHLBI scientists who contributed their expertise to this project as part of the U01 Cooperative Agreement funding mechanism as federal employees conducting their official job duties: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD. We thank the Executive Council of the Pediatric Research in Inpatient Settings (PRIS) Network for their contributions to the early scientific development of this project. The Network assessed a Collaborative Support Fee for access to the hospitals and support of this project. We thank the PRIS Network collaborators for their major contributions to data collection measuring utilization to identify the hospitals we subsequently chose for this project. We thank Claire Bocage and the Mixed Methods Research Lab for major help in data management and data analysis.

Unrecognized in-hospital clinical deterioration can lead to substantial morbidity and mortality.¹ As a result, hospitals have implemented systems to identify and mitigate this form of potentially preventable harm.^2-4 Cardiopulmonary arrest rates are useful metrics to evaluate the effectiveness of systems designed to identify and respond to deteriorating adult patients.⁵ Pediatric arrests outside of the intensive care unit (ICU) are rare; therefore, the identification of valid and more frequent proximal measures of deterioration is critical to the assessment of current systems and to guide future improvement efforts.⁶

Bonafide et al developed and validated the critical deterioration event (CDE) metric, demonstrating that children who were transferred to the ICU and who received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of transfer had an over 13-fold increased risk of in-hospital mortality.⁷ Implementation of a rapid response system was subsequently associated with a decrease in the trajectory of CDEs.² At Cincinnati Children’s Hospital Medical Center (CCHMC), an additional proximal outcome measure was developed for unrecognized clinical deterioration: emergency transfers (ETs).^8,9 An event meets criteria for an ET when the patient undergoes intubation, inotropic support, or three or more fluid boluses in the first hour after arrival or prior to ICU transfer.⁹ Recently, ETs were associated with an increased in-hospital mortality, ICU length of stay, and post-transfer hospital length of stay when compared with nonemergent transfers.^10,11

While both CDEs and ETs were associated with adverse outcomes in children and may be modifiable through better rapid response systems, researchers have not previously compared the extent to which CDEs and ETs capture similar versus distinct events. Furthermore, the ability of focused situation awareness interventions to identify high-risk patients has not previously been assessed. Situation awareness is defined as the perception of elements in the environment, the comprehension of their meaning, and the projection of their status in the near future.¹² Clinically, improved situation awareness can lead to earlier recognition of deterioration and a reduction in failure to rescue events.⁹ The objectives of this study were to (1) describe CDEs and ETs and assess for similarities, differences, and trends, and (2) evaluate the utility of situation awareness interventions to detect patients who experience these events.

Setting and Inclusion Criteria

We conducted a retrospective cross-sectional study at CCHMC, a free-standing tertiary care children’s hospital. We included all patients cared for outside of the ICU during their hospitalization from January 2016 to July 2018. Transfer to the ICU included the pediatric and the cardiac ICUs.

Study Definitions

CDEs were events in which a patient received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of ICU transfer (Figure).⁷ ETs were events in which a patient underwent intubation, inotropes, or three or more fluid boluses in the first hour after arrival or before transfer (Figure).⁹ We examined two distinct situation awareness interventions: watcher identification and the pediatric early warning score (PEWS). A watcher is a situation awareness concern based on clinician perception, or “gut feeling,” that the patient is at high risk for deterioration.^9,13 When clinicians designate a patient as a watcher in the electronic medical record, they establish an action plan, reassessment timeline, and objective criteria for activation of the rapid response team to assess the patient. Watcher patients are discussed at institution-wide safety huddles three times daily. The PEWS is a reproducible assessment of the patient’s status based on physiologic parameters, including behavior, cardiovascular, and respiratory assessments.^3,4 At CCHMC, a Monaghan PEWS score is calculated with each assessment of vital signs.¹⁴ The bedside nurse calls the physician or advanced practice provider to assess the patient for a score of 4 or greater.

Event Identification and Classification

Two trained research nurses (C.F. and D.H.) manually reviewed all ICU transfers during the study period to determine if CDE criteria were met. Events meeting CDE criteria were classified as respiratory (requiring noninvasive or invasive ventilation), cardiac (requiring inotropes), or cardiopulmonary resuscitation (CPR) in which cardiac and respiratory interventions were initiated simultaneously. Additional information obtained included the time the patient met CDE criteria relative to the time of ICU transfer, watcher identification prior to the event, and the highest PEWS documented within 12 hours of the event. A physician (T.S.) performed manual chart review of each CDE as an additional validation step. ETs during the study period were obtained from an existing institutional database. ICU transfers meeting ET criteria are entered into this database in nearly real time by the inpatient nurse manager; this nurse attends all rapid response team calls and is aware of the disposition for each event. A physician (T.S.) performed manual chart review of each ET to determine event classification by intervention type, watcher identification, and the highest PEWS documented within 12 hours of the event. All CDEs and ETs were cross-referenced to determine overlap.

Outcome Measures and Statistical Analysis

The primary outcomes were CDEs and ETs, calculated as absolute counts and number of events per 10,000 non-ICU patient days. Events were classified by (1) category of intervention, (2) watcher identification prior to the event, and (3) PEWS of 4 or greater documented in the 12 hours prior to the event.

Incidence and Overlap of CDEs and ETs

There were 1,828 ICU transfers during the study period, of which 365 (20%) met criteria for a CDE, ET, or both. Among events captured, 359 (98.4%) met criteria for a CDE, occurring at a rate of 16.7 per 10,000 non-ICU patient days, and 88 (24.1%) met criteria for an ET, occurring at a rate of 4.1 per 10,000 non-ICU patient days (Table). Of the 88 ETs, 82 also met criteria for a CDE.

Timing and Categorization of CDEs and ETs

Despite the 12-hour time horizon, most CDEs (62.1%) met criteria within 1 hour of ICU transfer, and 79.9% met criteria within 3 hours (Figure). Respiratory events were most common for both CDEs (80.5%) and ETs (47.7%) (Table). Of respiratory CDEs, 67.4% required noninvasive ventilation, and 32.5% required invasive ventilation. Fluid or inotrope support were responsible for 11.7% of CDEs and nearly one-third of ETs; of note, the CDE definition does not include fluid boluses. Less than 10% of CDEs were characterized by CPR, whereas this accounted for 22.7% of ETs.

Identification of Events by Situation Awareness Interventions

The Table depicts the identification of events by watcher status and PEWS. All events were included for watcher identification, and events with a documented score in the 12 hours prior to transfer were included for PEWS. While half or less of the events were captured by watcher or PEWS separately, over 85% of events were captured by either one or both of the situation awareness interventions. The situation awareness interventions identified CDEs and ETs similarly.

This study is the first to classify and compare two proximal measures of clinical deterioration in children. Given that children with escalating respiratory symptoms are often treated successfully outside of the ICU, the findings that most events are respiratory in nature and occur within 1 hour of transfer are not unexpected. The analysis of situation awareness interventions suggests that neither watcher identification nor PEWS is independently sufficient to predict future deterioration. These findings support the necessity of both a clinician “gut feeling” and objective vital sign and physical exam findings to indicate a patient’s clinical status.⁹ Initiatives to improve the early recognition and mitigation of patient deterioration should focus on both tools to initiate an escalation of care, and work to understand gaps in these identification systems, which currently miss approximately 15% of acutely deteriorating patients. Although most patients had watcher identification or elevated PEWS prior to the event, they still required emergent life-sustaining care, which suggests that opportunities exist to improve mitigation and escalation pathways as a critical prevention effort.^7,10

It is likely that CDEs and ETs are important outcome metrics in the evaluation of pediatric escalation systems, including rapid response systems.¹⁵ ETs are less common and more specific for unrecognized deterioration, which makes them a more feasible early metric for assessment. CDEs, which are likely more sensitive, may be useful in settings in which deterioration is rare or a more common outcome enhances power to detect the effect of interventions.¹⁰

This study has limitations and lends itself to future work. While CDEs and ETs are more common than cardiopulmonary arrest, they remain relatively uncommon. This was a single-site study at a large, tertiary care, free-standing children’s hospital, so generalizability to centers with different characteristics and patient populations may be limited. Future work should focus on comparing patient-level outcomes of CDEs and ETs, including length of stay and mortality. The determination of specific diagnoses and conditions associated with CDEs and ETs may inform targeted preventive improvement science interventions.

CDEs were roughly fourfold more common than ETs, with most CDEs occurring within 1 hour of ICU transfer. Most patients were identified by either watcher status or elevated PEWS, suggesting that these tools, when utilized as complementary situation awareness interventions, are important for identifying patients at risk for deterioration. Opportunities exist for improved escalation plans for patients identified as high-risk to prevent the need for emergent life-sustaining intervention.

Unrecognized in-hospital clinical deterioration can lead to substantial morbidity and mortality.¹ As a result, hospitals have implemented systems to identify and mitigate this form of potentially preventable harm.^2-4 Cardiopulmonary arrest rates are useful metrics to evaluate the effectiveness of systems designed to identify and respond to deteriorating adult patients.⁵ Pediatric arrests outside of the intensive care unit (ICU) are rare; therefore, the identification of valid and more frequent proximal measures of deterioration is critical to the assessment of current systems and to guide future improvement efforts.⁶

Bonafide et al developed and validated the critical deterioration event (CDE) metric, demonstrating that children who were transferred to the ICU and who received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of transfer had an over 13-fold increased risk of in-hospital mortality.⁷ Implementation of a rapid response system was subsequently associated with a decrease in the trajectory of CDEs.² At Cincinnati Children’s Hospital Medical Center (CCHMC), an additional proximal outcome measure was developed for unrecognized clinical deterioration: emergency transfers (ETs).^8,9 An event meets criteria for an ET when the patient undergoes intubation, inotropic support, or three or more fluid boluses in the first hour after arrival or prior to ICU transfer.⁹ Recently, ETs were associated with an increased in-hospital mortality, ICU length of stay, and post-transfer hospital length of stay when compared with nonemergent transfers.^10,11

While both CDEs and ETs were associated with adverse outcomes in children and may be modifiable through better rapid response systems, researchers have not previously compared the extent to which CDEs and ETs capture similar versus distinct events. Furthermore, the ability of focused situation awareness interventions to identify high-risk patients has not previously been assessed. Situation awareness is defined as the perception of elements in the environment, the comprehension of their meaning, and the projection of their status in the near future.¹² Clinically, improved situation awareness can lead to earlier recognition of deterioration and a reduction in failure to rescue events.⁹ The objectives of this study were to (1) describe CDEs and ETs and assess for similarities, differences, and trends, and (2) evaluate the utility of situation awareness interventions to detect patients who experience these events.

Setting and Inclusion Criteria

We conducted a retrospective cross-sectional study at CCHMC, a free-standing tertiary care children’s hospital. We included all patients cared for outside of the ICU during their hospitalization from January 2016 to July 2018. Transfer to the ICU included the pediatric and the cardiac ICUs.

Study Definitions

CDEs were events in which a patient received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of ICU transfer (Figure).⁷ ETs were events in which a patient underwent intubation, inotropes, or three or more fluid boluses in the first hour after arrival or before transfer (Figure).⁹ We examined two distinct situation awareness interventions: watcher identification and the pediatric early warning score (PEWS). A watcher is a situation awareness concern based on clinician perception, or “gut feeling,” that the patient is at high risk for deterioration.^9,13 When clinicians designate a patient as a watcher in the electronic medical record, they establish an action plan, reassessment timeline, and objective criteria for activation of the rapid response team to assess the patient. Watcher patients are discussed at institution-wide safety huddles three times daily. The PEWS is a reproducible assessment of the patient’s status based on physiologic parameters, including behavior, cardiovascular, and respiratory assessments.^3,4 At CCHMC, a Monaghan PEWS score is calculated with each assessment of vital signs.¹⁴ The bedside nurse calls the physician or advanced practice provider to assess the patient for a score of 4 or greater.

Event Identification and Classification

Two trained research nurses (C.F. and D.H.) manually reviewed all ICU transfers during the study period to determine if CDE criteria were met. Events meeting CDE criteria were classified as respiratory (requiring noninvasive or invasive ventilation), cardiac (requiring inotropes), or cardiopulmonary resuscitation (CPR) in which cardiac and respiratory interventions were initiated simultaneously. Additional information obtained included the time the patient met CDE criteria relative to the time of ICU transfer, watcher identification prior to the event, and the highest PEWS documented within 12 hours of the event. A physician (T.S.) performed manual chart review of each CDE as an additional validation step. ETs during the study period were obtained from an existing institutional database. ICU transfers meeting ET criteria are entered into this database in nearly real time by the inpatient nurse manager; this nurse attends all rapid response team calls and is aware of the disposition for each event. A physician (T.S.) performed manual chart review of each ET to determine event classification by intervention type, watcher identification, and the highest PEWS documented within 12 hours of the event. All CDEs and ETs were cross-referenced to determine overlap.

Outcome Measures and Statistical Analysis

The primary outcomes were CDEs and ETs, calculated as absolute counts and number of events per 10,000 non-ICU patient days. Events were classified by (1) category of intervention, (2) watcher identification prior to the event, and (3) PEWS of 4 or greater documented in the 12 hours prior to the event.

Incidence and Overlap of CDEs and ETs

There were 1,828 ICU transfers during the study period, of which 365 (20%) met criteria for a CDE, ET, or both. Among events captured, 359 (98.4%) met criteria for a CDE, occurring at a rate of 16.7 per 10,000 non-ICU patient days, and 88 (24.1%) met criteria for an ET, occurring at a rate of 4.1 per 10,000 non-ICU patient days (Table). Of the 88 ETs, 82 also met criteria for a CDE.

Timing and Categorization of CDEs and ETs

Despite the 12-hour time horizon, most CDEs (62.1%) met criteria within 1 hour of ICU transfer, and 79.9% met criteria within 3 hours (Figure). Respiratory events were most common for both CDEs (80.5%) and ETs (47.7%) (Table). Of respiratory CDEs, 67.4% required noninvasive ventilation, and 32.5% required invasive ventilation. Fluid or inotrope support were responsible for 11.7% of CDEs and nearly one-third of ETs; of note, the CDE definition does not include fluid boluses. Less than 10% of CDEs were characterized by CPR, whereas this accounted for 22.7% of ETs.

Identification of Events by Situation Awareness Interventions

The Table depicts the identification of events by watcher status and PEWS. All events were included for watcher identification, and events with a documented score in the 12 hours prior to transfer were included for PEWS. While half or less of the events were captured by watcher or PEWS separately, over 85% of events were captured by either one or both of the situation awareness interventions. The situation awareness interventions identified CDEs and ETs similarly.

This study is the first to classify and compare two proximal measures of clinical deterioration in children. Given that children with escalating respiratory symptoms are often treated successfully outside of the ICU, the findings that most events are respiratory in nature and occur within 1 hour of transfer are not unexpected. The analysis of situation awareness interventions suggests that neither watcher identification nor PEWS is independently sufficient to predict future deterioration. These findings support the necessity of both a clinician “gut feeling” and objective vital sign and physical exam findings to indicate a patient’s clinical status.⁹ Initiatives to improve the early recognition and mitigation of patient deterioration should focus on both tools to initiate an escalation of care, and work to understand gaps in these identification systems, which currently miss approximately 15% of acutely deteriorating patients. Although most patients had watcher identification or elevated PEWS prior to the event, they still required emergent life-sustaining care, which suggests that opportunities exist to improve mitigation and escalation pathways as a critical prevention effort.^7,10

It is likely that CDEs and ETs are important outcome metrics in the evaluation of pediatric escalation systems, including rapid response systems.¹⁵ ETs are less common and more specific for unrecognized deterioration, which makes them a more feasible early metric for assessment. CDEs, which are likely more sensitive, may be useful in settings in which deterioration is rare or a more common outcome enhances power to detect the effect of interventions.¹⁰

This study has limitations and lends itself to future work. While CDEs and ETs are more common than cardiopulmonary arrest, they remain relatively uncommon. This was a single-site study at a large, tertiary care, free-standing children’s hospital, so generalizability to centers with different characteristics and patient populations may be limited. Future work should focus on comparing patient-level outcomes of CDEs and ETs, including length of stay and mortality. The determination of specific diagnoses and conditions associated with CDEs and ETs may inform targeted preventive improvement science interventions.

CDEs were roughly fourfold more common than ETs, with most CDEs occurring within 1 hour of ICU transfer. Most patients were identified by either watcher status or elevated PEWS, suggesting that these tools, when utilized as complementary situation awareness interventions, are important for identifying patients at risk for deterioration. Opportunities exist for improved escalation plans for patients identified as high-risk to prevent the need for emergent life-sustaining intervention.

Unrecognized in-hospital clinical deterioration can lead to substantial morbidity and mortality.¹ As a result, hospitals have implemented systems to identify and mitigate this form of potentially preventable harm.^2-4 Cardiopulmonary arrest rates are useful metrics to evaluate the effectiveness of systems designed to identify and respond to deteriorating adult patients.⁵ Pediatric arrests outside of the intensive care unit (ICU) are rare; therefore, the identification of valid and more frequent proximal measures of deterioration is critical to the assessment of current systems and to guide future improvement efforts.⁶

Bonafide et al developed and validated the critical deterioration event (CDE) metric, demonstrating that children who were transferred to the ICU and who received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of transfer had an over 13-fold increased risk of in-hospital mortality.⁷ Implementation of a rapid response system was subsequently associated with a decrease in the trajectory of CDEs.² At Cincinnati Children’s Hospital Medical Center (CCHMC), an additional proximal outcome measure was developed for unrecognized clinical deterioration: emergency transfers (ETs).^8,9 An event meets criteria for an ET when the patient undergoes intubation, inotropic support, or three or more fluid boluses in the first hour after arrival or prior to ICU transfer.⁹ Recently, ETs were associated with an increased in-hospital mortality, ICU length of stay, and post-transfer hospital length of stay when compared with nonemergent transfers.^10,11

While both CDEs and ETs were associated with adverse outcomes in children and may be modifiable through better rapid response systems, researchers have not previously compared the extent to which CDEs and ETs capture similar versus distinct events. Furthermore, the ability of focused situation awareness interventions to identify high-risk patients has not previously been assessed. Situation awareness is defined as the perception of elements in the environment, the comprehension of their meaning, and the projection of their status in the near future.¹² Clinically, improved situation awareness can lead to earlier recognition of deterioration and a reduction in failure to rescue events.⁹ The objectives of this study were to (1) describe CDEs and ETs and assess for similarities, differences, and trends, and (2) evaluate the utility of situation awareness interventions to detect patients who experience these events.

Setting and Inclusion Criteria

We conducted a retrospective cross-sectional study at CCHMC, a free-standing tertiary care children’s hospital. We included all patients cared for outside of the ICU during their hospitalization from January 2016 to July 2018. Transfer to the ICU included the pediatric and the cardiac ICUs.

Study Definitions

CDEs were events in which a patient received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of ICU transfer (Figure).⁷ ETs were events in which a patient underwent intubation, inotropes, or three or more fluid boluses in the first hour after arrival or before transfer (Figure).⁹ We examined two distinct situation awareness interventions: watcher identification and the pediatric early warning score (PEWS). A watcher is a situation awareness concern based on clinician perception, or “gut feeling,” that the patient is at high risk for deterioration.^9,13 When clinicians designate a patient as a watcher in the electronic medical record, they establish an action plan, reassessment timeline, and objective criteria for activation of the rapid response team to assess the patient. Watcher patients are discussed at institution-wide safety huddles three times daily. The PEWS is a reproducible assessment of the patient’s status based on physiologic parameters, including behavior, cardiovascular, and respiratory assessments.^3,4 At CCHMC, a Monaghan PEWS score is calculated with each assessment of vital signs.¹⁴ The bedside nurse calls the physician or advanced practice provider to assess the patient for a score of 4 or greater.

Event Identification and Classification

Two trained research nurses (C.F. and D.H.) manually reviewed all ICU transfers during the study period to determine if CDE criteria were met. Events meeting CDE criteria were classified as respiratory (requiring noninvasive or invasive ventilation), cardiac (requiring inotropes), or cardiopulmonary resuscitation (CPR) in which cardiac and respiratory interventions were initiated simultaneously. Additional information obtained included the time the patient met CDE criteria relative to the time of ICU transfer, watcher identification prior to the event, and the highest PEWS documented within 12 hours of the event. A physician (T.S.) performed manual chart review of each CDE as an additional validation step. ETs during the study period were obtained from an existing institutional database. ICU transfers meeting ET criteria are entered into this database in nearly real time by the inpatient nurse manager; this nurse attends all rapid response team calls and is aware of the disposition for each event. A physician (T.S.) performed manual chart review of each ET to determine event classification by intervention type, watcher identification, and the highest PEWS documented within 12 hours of the event. All CDEs and ETs were cross-referenced to determine overlap.

Outcome Measures and Statistical Analysis

The primary outcomes were CDEs and ETs, calculated as absolute counts and number of events per 10,000 non-ICU patient days. Events were classified by (1) category of intervention, (2) watcher identification prior to the event, and (3) PEWS of 4 or greater documented in the 12 hours prior to the event.

Incidence and Overlap of CDEs and ETs

There were 1,828 ICU transfers during the study period, of which 365 (20%) met criteria for a CDE, ET, or both. Among events captured, 359 (98.4%) met criteria for a CDE, occurring at a rate of 16.7 per 10,000 non-ICU patient days, and 88 (24.1%) met criteria for an ET, occurring at a rate of 4.1 per 10,000 non-ICU patient days (Table). Of the 88 ETs, 82 also met criteria for a CDE.

Timing and Categorization of CDEs and ETs

Despite the 12-hour time horizon, most CDEs (62.1%) met criteria within 1 hour of ICU transfer, and 79.9% met criteria within 3 hours (Figure). Respiratory events were most common for both CDEs (80.5%) and ETs (47.7%) (Table). Of respiratory CDEs, 67.4% required noninvasive ventilation, and 32.5% required invasive ventilation. Fluid or inotrope support were responsible for 11.7% of CDEs and nearly one-third of ETs; of note, the CDE definition does not include fluid boluses. Less than 10% of CDEs were characterized by CPR, whereas this accounted for 22.7% of ETs.

Identification of Events by Situation Awareness Interventions

The Table depicts the identification of events by watcher status and PEWS. All events were included for watcher identification, and events with a documented score in the 12 hours prior to transfer were included for PEWS. While half or less of the events were captured by watcher or PEWS separately, over 85% of events were captured by either one or both of the situation awareness interventions. The situation awareness interventions identified CDEs and ETs similarly.

This study is the first to classify and compare two proximal measures of clinical deterioration in children. Given that children with escalating respiratory symptoms are often treated successfully outside of the ICU, the findings that most events are respiratory in nature and occur within 1 hour of transfer are not unexpected. The analysis of situation awareness interventions suggests that neither watcher identification nor PEWS is independently sufficient to predict future deterioration. These findings support the necessity of both a clinician “gut feeling” and objective vital sign and physical exam findings to indicate a patient’s clinical status.⁹ Initiatives to improve the early recognition and mitigation of patient deterioration should focus on both tools to initiate an escalation of care, and work to understand gaps in these identification systems, which currently miss approximately 15% of acutely deteriorating patients. Although most patients had watcher identification or elevated PEWS prior to the event, they still required emergent life-sustaining care, which suggests that opportunities exist to improve mitigation and escalation pathways as a critical prevention effort.^7,10

It is likely that CDEs and ETs are important outcome metrics in the evaluation of pediatric escalation systems, including rapid response systems.¹⁵ ETs are less common and more specific for unrecognized deterioration, which makes them a more feasible early metric for assessment. CDEs, which are likely more sensitive, may be useful in settings in which deterioration is rare or a more common outcome enhances power to detect the effect of interventions.¹⁰

This study has limitations and lends itself to future work. While CDEs and ETs are more common than cardiopulmonary arrest, they remain relatively uncommon. This was a single-site study at a large, tertiary care, free-standing children’s hospital, so generalizability to centers with different characteristics and patient populations may be limited. Future work should focus on comparing patient-level outcomes of CDEs and ETs, including length of stay and mortality. The determination of specific diagnoses and conditions associated with CDEs and ETs may inform targeted preventive improvement science interventions.

CDEs were roughly fourfold more common than ETs, with most CDEs occurring within 1 hour of ICU transfer. Most patients were identified by either watcher status or elevated PEWS, suggesting that these tools, when utilized as complementary situation awareness interventions, are important for identifying patients at risk for deterioration. Opportunities exist for improved escalation plans for patients identified as high-risk to prevent the need for emergent life-sustaining intervention.

As part of improvement collaboratives that aimed to reduce overuse of continuous pulse oximetry in children hospitalized with bronchiolitis, researchers used the presence of an active order for it as a proxy for the actual use of such monitoring.^1,2 With use of this proxy, investigators on a national study documented a high burden of continuous oximetry overuse (86.5% before quality improvement interventions and 45.5% after),¹ but the validity of orders in representing actual monitoring practice is unknown. If the presence of an active pulse oximetry order accurately identifies infants on monitors, electronic health record data could inform epidemiologic estimates of monitoring overuse and measure the success of quality improvement and deimplementation interventions. Alternatively, if nurses commonly begin and/or discontinue pulse oximetry without updated orders, a pulse oximetry order would not be an accurate proxy, and additional data capture methods (eg, bedside observation or data capture from bedside monitors) would be needed.

Understanding the validity of orders for detection of actual use is critical because continuous pulse oximetry monitoring is considered an overused practice in pediatric acute viral bronchiolitis,³ and national guidelines recommend against its use in low-risk hospitalized children.^4,5 Continuous monitoring may identify trivial, self-resolving oxygen desaturation and its use is not associated with improved outcomes.^6-9 When self-resolving desaturations are treated with additional supplemental oxygen, hospital stays may be unnecessarily prolonged.¹⁰ In order to reduce unnecessary continuous pulse oximetry use, measurement of the extent of the overused practice is necessary. In this 56-hospital study,¹¹ we aimed to determine the validity of using active continuous pulse oximetry orders instead of bedside observation of actual monitor use.

Design

In this multicenter, repeated cross-sectional study, investigators used direct bedside observation to determine continuous pulse oximetry monitor use and then assessed whether an active continuous monitoring order was present in the electronic health record. The study took place during one bronchiolitis season, December 1, 2018, through March 31, 2019.

Setting and Patients

Investigators at 56 freestanding children’s hospitals, children’s hospitals within general hospitals, and community hospitals in the Pediatric Research in Inpatient Settings (PRIS) Network collected data on infants aged 8 weeks to 23 months who were hospitalized with bronchiolitis. As this work was a substudy of the larger Eliminating Monitor Overuse study, only infants not currently receiving supplemental oxygen were included.¹¹ Investigators observed eligible infants outside of the intensive care unit on general hospital medicine units. We excluded infants born premature (documented prematurity of <28 weeks’ gestation or documented “premature” without a gestational age listed), as well as those with a home oxygen requirement, cyanotic congenital heart disease, pulmonary hypertension, tracheostomy, primary neuromuscular disease, immunodeficiency, or cancer.

Data Collection

Investigators used the electronic health record to identify eligible infants. Investigators entered patient rooms to confirm the infant was not on supplemental oxygen (hence confirming eligibility for the study) and determine if continuous pulse oximetry was actively in use by examining the monitor display for a pulse oximetry waveform. Investigators then confirmed if active orders for pulse oximetry were present in the patient’s chart. Per study design, site investigators aimed to observe approximately half of eligible infants during the day (10 am to 5 pm) and the other half during the night (11 pm to 7 am).

Analysis

We excluded patients with conditional orders (eg, monitored only when certain conditions exist, such as when asleep) because of the time-varying and wide range of conditions that could be specified. Furthermore, conditional orders would not be useful as proxies to measure oximetry use because investigators would still need additional data (eg, bedside observation) to determine current monitoring status.

We calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of active orders using the reference standard of direct bedside observation, as well as corresponding 95% CIs that accounted for within-hospital clustering. We calculated these test characteristics overall and as stratified across four age groups: 8 weeks to 5 months, 6 months to 11 months, 12 months to 17 months, and 18 months to 23 months. We also calculated the test characteristics for each hospital. We decided a priori that a PPV and NPV of 80% would represent a reasonable threshold to use active orders as a proxy in multicenter research. For hospital-level analyses we included only hospitals with 60 or more total observations and more than 15 observations with active orders for PPV and more than 15 observations without active orders for NPV. We used Stata (StataCorp LLC, College Station, Texas) version 15.1 for analysis.

For US sites, the Institutional Review Board (IRB) at Children’s Hospital of Philadelphia approved the study as the single reviewing IRB, and the remaining US sites established reliance agreements with the reviewing IRB. Research Ethics Boards at the Canadian sites (University of Calgary and The Hospital for Sick Children) also reviewed and approved the study. All sites granted waivers of consent, assent, parental permission, and HIPAA authorization.

Investigators completed 3,612 observations in 56 hospitals. This included 33 freestanding children’s hospitals, 14 hospitals within large general hospitals, and 9 community hospitals. Of 3,612 completed observations, on 631 occasions (17%) patients had conditional orders (eg, continuous monitoring only when sleeping) and were excluded from further analysis.

Most pulse oximetry–monitored infants did not have an active monitoring order (670 out of 1,309; sensitivity of 49%). Test characteristics, stratified by age group, are presented in the Table. Across all observations, the overall PPV was 77% (95% CI, 72-82), and the overall NPV was 69% (95% CI, 61-77). Variation of all test characteristics across age group was small (eg, the sensitivity ranged from 43% to 51%).

With inclusion of only those hospitals with sufficient observations, hospital-level variation in the PPV and NPV of using active orders was substantial (PPV range of 48% to 96% and NPV range of 30% to 98%). Only two hospitals had both a PPV and NPV for using monitor orders that exceeded the 80% threshold.

Active continuous pulse oximetry orders did not accurately represent actual monitoring status in this study. Monitoring orders alone frequently misrepresent true monitoring status and, as such, should be interpreted with caution in research or quality improvement activities. If more valid estimates of monitoring use and overuse are needed, potential measurement options include direct observation, as used in our study, as well as the use of more complex data streams such as the output of monitoring devices or pulse oximetry data in the electronic health record. In only two of the hospitals, using active continuous monitoring orders was a reasonable proxy for detecting actual monitor use. Monitoring orders could potentially be validly used for deimplementation efforts at those centers; other hospitals could consider targeted improvement efforts (eg, morning huddles examining the discordance between monitoring orders and monitoring status) to improve the accuracy of using continuous pulse oximetry orders.

We acknowledge several limitations of this study. Site investigators employed a convenience sampling approach, so it is possible that some investigators observed sicker or less sick infants. Although the PRIS network includes a geographically diverse group of North American hospitals, community hospitals were underrepresented in this study. Our results hence generalize more precisely to freestanding children’s hospitals than to community hospitals. We did not observe infants currently on supplemental oxygen, so we do not know to what degree using orders is valid in that context. We did not collect data on why actual monitoring status differed from monitoring orders and hence cannot quantify to what extent different factors (eg, nurse belief that monitors are a safety net or infants inadvertently left on monitors after a spot check pulse oximetry reading) contributed to this discordance. Finally, our study only examined one electronic health record variable—the presence of an active order. It may be that other variables in the health record (eg, minute-by-minute pulse oximetry values in a vital sign flowsheet) are much better proxies of actual continuous monitor use.

Using an active order for continuous pulse oximetry has poor sensitivity, PPV, and NPV for detecting true monitoring status at the bedside. Teams intending to measure the actual use of pulse oximetry should be aware of the limitations of using active orders alone as an accurate measure of pulse oximetry monitoring.

We thank the NHLBI scientists who contributed to this project as part of the U01 Cooperative Agreement funding mechanism: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD.

We thank the Executive Council of the PRIS Network for their contributions to the early scientific development of this project. We thank the PRIS site investigators for their major contributions to the Eliminating Monitor Overuse (EMO) Study data collection. Each listed collaborator is a group author for the PRIS Network in this manuscript. Their names can be found in the online supplemental information.

As part of improvement collaboratives that aimed to reduce overuse of continuous pulse oximetry in children hospitalized with bronchiolitis, researchers used the presence of an active order for it as a proxy for the actual use of such monitoring.^1,2 With use of this proxy, investigators on a national study documented a high burden of continuous oximetry overuse (86.5% before quality improvement interventions and 45.5% after),¹ but the validity of orders in representing actual monitoring practice is unknown. If the presence of an active pulse oximetry order accurately identifies infants on monitors, electronic health record data could inform epidemiologic estimates of monitoring overuse and measure the success of quality improvement and deimplementation interventions. Alternatively, if nurses commonly begin and/or discontinue pulse oximetry without updated orders, a pulse oximetry order would not be an accurate proxy, and additional data capture methods (eg, bedside observation or data capture from bedside monitors) would be needed.

Understanding the validity of orders for detection of actual use is critical because continuous pulse oximetry monitoring is considered an overused practice in pediatric acute viral bronchiolitis,³ and national guidelines recommend against its use in low-risk hospitalized children.^4,5 Continuous monitoring may identify trivial, self-resolving oxygen desaturation and its use is not associated with improved outcomes.^6-9 When self-resolving desaturations are treated with additional supplemental oxygen, hospital stays may be unnecessarily prolonged.¹⁰ In order to reduce unnecessary continuous pulse oximetry use, measurement of the extent of the overused practice is necessary. In this 56-hospital study,¹¹ we aimed to determine the validity of using active continuous pulse oximetry orders instead of bedside observation of actual monitor use.

Design

In this multicenter, repeated cross-sectional study, investigators used direct bedside observation to determine continuous pulse oximetry monitor use and then assessed whether an active continuous monitoring order was present in the electronic health record. The study took place during one bronchiolitis season, December 1, 2018, through March 31, 2019.

Setting and Patients

Investigators at 56 freestanding children’s hospitals, children’s hospitals within general hospitals, and community hospitals in the Pediatric Research in Inpatient Settings (PRIS) Network collected data on infants aged 8 weeks to 23 months who were hospitalized with bronchiolitis. As this work was a substudy of the larger Eliminating Monitor Overuse study, only infants not currently receiving supplemental oxygen were included.¹¹ Investigators observed eligible infants outside of the intensive care unit on general hospital medicine units. We excluded infants born premature (documented prematurity of <28 weeks’ gestation or documented “premature” without a gestational age listed), as well as those with a home oxygen requirement, cyanotic congenital heart disease, pulmonary hypertension, tracheostomy, primary neuromuscular disease, immunodeficiency, or cancer.

Data Collection

Investigators used the electronic health record to identify eligible infants. Investigators entered patient rooms to confirm the infant was not on supplemental oxygen (hence confirming eligibility for the study) and determine if continuous pulse oximetry was actively in use by examining the monitor display for a pulse oximetry waveform. Investigators then confirmed if active orders for pulse oximetry were present in the patient’s chart. Per study design, site investigators aimed to observe approximately half of eligible infants during the day (10 am to 5 pm) and the other half during the night (11 pm to 7 am).

Analysis

We excluded patients with conditional orders (eg, monitored only when certain conditions exist, such as when asleep) because of the time-varying and wide range of conditions that could be specified. Furthermore, conditional orders would not be useful as proxies to measure oximetry use because investigators would still need additional data (eg, bedside observation) to determine current monitoring status.

We calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of active orders using the reference standard of direct bedside observation, as well as corresponding 95% CIs that accounted for within-hospital clustering. We calculated these test characteristics overall and as stratified across four age groups: 8 weeks to 5 months, 6 months to 11 months, 12 months to 17 months, and 18 months to 23 months. We also calculated the test characteristics for each hospital. We decided a priori that a PPV and NPV of 80% would represent a reasonable threshold to use active orders as a proxy in multicenter research. For hospital-level analyses we included only hospitals with 60 or more total observations and more than 15 observations with active orders for PPV and more than 15 observations without active orders for NPV. We used Stata (StataCorp LLC, College Station, Texas) version 15.1 for analysis.

For US sites, the Institutional Review Board (IRB) at Children’s Hospital of Philadelphia approved the study as the single reviewing IRB, and the remaining US sites established reliance agreements with the reviewing IRB. Research Ethics Boards at the Canadian sites (University of Calgary and The Hospital for Sick Children) also reviewed and approved the study. All sites granted waivers of consent, assent, parental permission, and HIPAA authorization.

Investigators completed 3,612 observations in 56 hospitals. This included 33 freestanding children’s hospitals, 14 hospitals within large general hospitals, and 9 community hospitals. Of 3,612 completed observations, on 631 occasions (17%) patients had conditional orders (eg, continuous monitoring only when sleeping) and were excluded from further analysis.

Most pulse oximetry–monitored infants did not have an active monitoring order (670 out of 1,309; sensitivity of 49%). Test characteristics, stratified by age group, are presented in the Table. Across all observations, the overall PPV was 77% (95% CI, 72-82), and the overall NPV was 69% (95% CI, 61-77). Variation of all test characteristics across age group was small (eg, the sensitivity ranged from 43% to 51%).

With inclusion of only those hospitals with sufficient observations, hospital-level variation in the PPV and NPV of using active orders was substantial (PPV range of 48% to 96% and NPV range of 30% to 98%). Only two hospitals had both a PPV and NPV for using monitor orders that exceeded the 80% threshold.

Active continuous pulse oximetry orders did not accurately represent actual monitoring status in this study. Monitoring orders alone frequently misrepresent true monitoring status and, as such, should be interpreted with caution in research or quality improvement activities. If more valid estimates of monitoring use and overuse are needed, potential measurement options include direct observation, as used in our study, as well as the use of more complex data streams such as the output of monitoring devices or pulse oximetry data in the electronic health record. In only two of the hospitals, using active continuous monitoring orders was a reasonable proxy for detecting actual monitor use. Monitoring orders could potentially be validly used for deimplementation efforts at those centers; other hospitals could consider targeted improvement efforts (eg, morning huddles examining the discordance between monitoring orders and monitoring status) to improve the accuracy of using continuous pulse oximetry orders.

We acknowledge several limitations of this study. Site investigators employed a convenience sampling approach, so it is possible that some investigators observed sicker or less sick infants. Although the PRIS network includes a geographically diverse group of North American hospitals, community hospitals were underrepresented in this study. Our results hence generalize more precisely to freestanding children’s hospitals than to community hospitals. We did not observe infants currently on supplemental oxygen, so we do not know to what degree using orders is valid in that context. We did not collect data on why actual monitoring status differed from monitoring orders and hence cannot quantify to what extent different factors (eg, nurse belief that monitors are a safety net or infants inadvertently left on monitors after a spot check pulse oximetry reading) contributed to this discordance. Finally, our study only examined one electronic health record variable—the presence of an active order. It may be that other variables in the health record (eg, minute-by-minute pulse oximetry values in a vital sign flowsheet) are much better proxies of actual continuous monitor use.

Using an active order for continuous pulse oximetry has poor sensitivity, PPV, and NPV for detecting true monitoring status at the bedside. Teams intending to measure the actual use of pulse oximetry should be aware of the limitations of using active orders alone as an accurate measure of pulse oximetry monitoring.

We thank the NHLBI scientists who contributed to this project as part of the U01 Cooperative Agreement funding mechanism: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD.

We thank the Executive Council of the PRIS Network for their contributions to the early scientific development of this project. We thank the PRIS site investigators for their major contributions to the Eliminating Monitor Overuse (EMO) Study data collection. Each listed collaborator is a group author for the PRIS Network in this manuscript. Their names can be found in the online supplemental information.

As part of improvement collaboratives that aimed to reduce overuse of continuous pulse oximetry in children hospitalized with bronchiolitis, researchers used the presence of an active order for it as a proxy for the actual use of such monitoring.^1,2 With use of this proxy, investigators on a national study documented a high burden of continuous oximetry overuse (86.5% before quality improvement interventions and 45.5% after),¹ but the validity of orders in representing actual monitoring practice is unknown. If the presence of an active pulse oximetry order accurately identifies infants on monitors, electronic health record data could inform epidemiologic estimates of monitoring overuse and measure the success of quality improvement and deimplementation interventions. Alternatively, if nurses commonly begin and/or discontinue pulse oximetry without updated orders, a pulse oximetry order would not be an accurate proxy, and additional data capture methods (eg, bedside observation or data capture from bedside monitors) would be needed.

Understanding the validity of orders for detection of actual use is critical because continuous pulse oximetry monitoring is considered an overused practice in pediatric acute viral bronchiolitis,³ and national guidelines recommend against its use in low-risk hospitalized children.^4,5 Continuous monitoring may identify trivial, self-resolving oxygen desaturation and its use is not associated with improved outcomes.^6-9 When self-resolving desaturations are treated with additional supplemental oxygen, hospital stays may be unnecessarily prolonged.¹⁰ In order to reduce unnecessary continuous pulse oximetry use, measurement of the extent of the overused practice is necessary. In this 56-hospital study,¹¹ we aimed to determine the validity of using active continuous pulse oximetry orders instead of bedside observation of actual monitor use.

Design

In this multicenter, repeated cross-sectional study, investigators used direct bedside observation to determine continuous pulse oximetry monitor use and then assessed whether an active continuous monitoring order was present in the electronic health record. The study took place during one bronchiolitis season, December 1, 2018, through March 31, 2019.

Setting and Patients

Investigators at 56 freestanding children’s hospitals, children’s hospitals within general hospitals, and community hospitals in the Pediatric Research in Inpatient Settings (PRIS) Network collected data on infants aged 8 weeks to 23 months who were hospitalized with bronchiolitis. As this work was a substudy of the larger Eliminating Monitor Overuse study, only infants not currently receiving supplemental oxygen were included.¹¹ Investigators observed eligible infants outside of the intensive care unit on general hospital medicine units. We excluded infants born premature (documented prematurity of <28 weeks’ gestation or documented “premature” without a gestational age listed), as well as those with a home oxygen requirement, cyanotic congenital heart disease, pulmonary hypertension, tracheostomy, primary neuromuscular disease, immunodeficiency, or cancer.

Data Collection

Investigators used the electronic health record to identify eligible infants. Investigators entered patient rooms to confirm the infant was not on supplemental oxygen (hence confirming eligibility for the study) and determine if continuous pulse oximetry was actively in use by examining the monitor display for a pulse oximetry waveform. Investigators then confirmed if active orders for pulse oximetry were present in the patient’s chart. Per study design, site investigators aimed to observe approximately half of eligible infants during the day (10 am to 5 pm) and the other half during the night (11 pm to 7 am).

Analysis

We excluded patients with conditional orders (eg, monitored only when certain conditions exist, such as when asleep) because of the time-varying and wide range of conditions that could be specified. Furthermore, conditional orders would not be useful as proxies to measure oximetry use because investigators would still need additional data (eg, bedside observation) to determine current monitoring status.

We calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of active orders using the reference standard of direct bedside observation, as well as corresponding 95% CIs that accounted for within-hospital clustering. We calculated these test characteristics overall and as stratified across four age groups: 8 weeks to 5 months, 6 months to 11 months, 12 months to 17 months, and 18 months to 23 months. We also calculated the test characteristics for each hospital. We decided a priori that a PPV and NPV of 80% would represent a reasonable threshold to use active orders as a proxy in multicenter research. For hospital-level analyses we included only hospitals with 60 or more total observations and more than 15 observations with active orders for PPV and more than 15 observations without active orders for NPV. We used Stata (StataCorp LLC, College Station, Texas) version 15.1 for analysis.

For US sites, the Institutional Review Board (IRB) at Children’s Hospital of Philadelphia approved the study as the single reviewing IRB, and the remaining US sites established reliance agreements with the reviewing IRB. Research Ethics Boards at the Canadian sites (University of Calgary and The Hospital for Sick Children) also reviewed and approved the study. All sites granted waivers of consent, assent, parental permission, and HIPAA authorization.

Investigators completed 3,612 observations in 56 hospitals. This included 33 freestanding children’s hospitals, 14 hospitals within large general hospitals, and 9 community hospitals. Of 3,612 completed observations, on 631 occasions (17%) patients had conditional orders (eg, continuous monitoring only when sleeping) and were excluded from further analysis.

Most pulse oximetry–monitored infants did not have an active monitoring order (670 out of 1,309; sensitivity of 49%). Test characteristics, stratified by age group, are presented in the Table. Across all observations, the overall PPV was 77% (95% CI, 72-82), and the overall NPV was 69% (95% CI, 61-77). Variation of all test characteristics across age group was small (eg, the sensitivity ranged from 43% to 51%).

With inclusion of only those hospitals with sufficient observations, hospital-level variation in the PPV and NPV of using active orders was substantial (PPV range of 48% to 96% and NPV range of 30% to 98%). Only two hospitals had both a PPV and NPV for using monitor orders that exceeded the 80% threshold.

Active continuous pulse oximetry orders did not accurately represent actual monitoring status in this study. Monitoring orders alone frequently misrepresent true monitoring status and, as such, should be interpreted with caution in research or quality improvement activities. If more valid estimates of monitoring use and overuse are needed, potential measurement options include direct observation, as used in our study, as well as the use of more complex data streams such as the output of monitoring devices or pulse oximetry data in the electronic health record. In only two of the hospitals, using active continuous monitoring orders was a reasonable proxy for detecting actual monitor use. Monitoring orders could potentially be validly used for deimplementation efforts at those centers; other hospitals could consider targeted improvement efforts (eg, morning huddles examining the discordance between monitoring orders and monitoring status) to improve the accuracy of using continuous pulse oximetry orders.

We acknowledge several limitations of this study. Site investigators employed a convenience sampling approach, so it is possible that some investigators observed sicker or less sick infants. Although the PRIS network includes a geographically diverse group of North American hospitals, community hospitals were underrepresented in this study. Our results hence generalize more precisely to freestanding children’s hospitals than to community hospitals. We did not observe infants currently on supplemental oxygen, so we do not know to what degree using orders is valid in that context. We did not collect data on why actual monitoring status differed from monitoring orders and hence cannot quantify to what extent different factors (eg, nurse belief that monitors are a safety net or infants inadvertently left on monitors after a spot check pulse oximetry reading) contributed to this discordance. Finally, our study only examined one electronic health record variable—the presence of an active order. It may be that other variables in the health record (eg, minute-by-minute pulse oximetry values in a vital sign flowsheet) are much better proxies of actual continuous monitor use.

Using an active order for continuous pulse oximetry has poor sensitivity, PPV, and NPV for detecting true monitoring status at the bedside. Teams intending to measure the actual use of pulse oximetry should be aware of the limitations of using active orders alone as an accurate measure of pulse oximetry monitoring.

We thank the NHLBI scientists who contributed to this project as part of the U01 Cooperative Agreement funding mechanism: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD.

We thank the Executive Council of the PRIS Network for their contributions to the early scientific development of this project. We thank the PRIS site investigators for their major contributions to the Eliminating Monitor Overuse (EMO) Study data collection. Each listed collaborator is a group author for the PRIS Network in this manuscript. Their names can be found in the online supplemental information.

Polypharmacy, the practice of taking multiple medications to manage health conditions, is common for children. Many children today have a higher burden chronic illness and an increasing number of pharmaceuticals—often delivered in various doses throughout the day. Polypharmacy has been linked to a variety of pediatric and adult outcomes, including medication errors and readmission.^1-3 Consequently, the Society of Hospital Medicine recognizes polypharmacy as a risk factor for readmission for adult populations.⁴ These adverse outcomes are related to both the human elements of polypharmacy (eg, cognitive burden, adherence) and the pharmacologic elements, including drug–drug interactions. For many children, the safety implications of polypharmacy may be more consequential due to the reliance of multiple caregivers to administer medications, which requires additional coordination to ensure that medications are administered and not duplicated. Dual administration of the same medication by both parents is the most common reason for pediatric calls to Poison Control Centers.⁵ Yet, there is a paucity of research in this area, with most of the pediatric literature focusing on the outpatient setting and specific populations, including epilepsy and mental health.^6-8

How providers, patients, and families translate medication lists to counts of medications—and hence the burden of polypharmacy—is not clearly or consistently described. Often in studies of polypharmacy, researchers utilize medication claims data to count the number of medications a patient has filled from the pharmacy. However, in routine clinical practice, clinicians rarely have access to medication claims and thus rely on patient or family report, which may or may not match the list of medications in the patients’ medical records.

Therefore, linking polypharmacy research to the pragmatic complexities of clinical care requires greater clarity and consistent application of concepts. At hospital discharge, families receive a list of medications to take, including home medications to resume as well as newly prescribed medications. However, not all medications are equally essential to patients’ care regarding importance of administration (eg, hydrocortisone ointment versus an anticonvulsant medication). Patients, parents, and caregivers are ultimately responsible for determining which medications to prioritize and administer.

Although there is no standard numerical definition for how to identify polypharmacy, five medications is commonly considered the threshold for polypharmacy.⁹ A recent review of the pediatric polypharmacy literature suggested a lower threshold, with any two concurrent medications for at least a day.⁷ Yet, the best approach to “count” medications at hospital discharge is unclear. The simplest method is to tally the number of medications listed in the discharge summary. However, medications are sometimes listed twice due to different dosages administered at different times. Frequently, medications are prescribed on an as-needed basis; these medications could be administered routinely or very infrequently (eg, epinephrine for anaphylaxis). Over-the-counter medications are also sometimes included in discharge summaries and consideration should be given as to whether these medications count toward measures of polypharmacy. Over-the-counter medications would not be counted by a polypharmacy measure that relies on claims data if those medications are not paid by the insurer.

We sought consensus on how to count discharge medications through a series of informal interviews with hospitalists, nurses, and parents. We asked the seemingly simple question, “How many medications is this child on?” across a variety of scenarios (Figure). For panel A, all stakeholders agreed that this medication list includes two medications. All other scenarios elicited disagreement. For panel B, many people responded three medications, but others (often physicians) counted only clindamycin and therefore responded one medication.

Disclosures

Dr. Auger has nothing to disclose. Dr. Shah is the Editor-in-Chief of the Journal of Hospital Medicine. Dr. Davis has nothing to disclose. Dr. Brady reports grants from Agency for Healthcare Research and Quality, outside the submitted work.

Funding

This project is supported by a grant from the Agency for Healthcare Research and Quality (1K08HS204735-01A1).

Polypharmacy, the practice of taking multiple medications to manage health conditions, is common for children. Many children today have a higher burden chronic illness and an increasing number of pharmaceuticals—often delivered in various doses throughout the day. Polypharmacy has been linked to a variety of pediatric and adult outcomes, including medication errors and readmission.^1-3 Consequently, the Society of Hospital Medicine recognizes polypharmacy as a risk factor for readmission for adult populations.⁴ These adverse outcomes are related to both the human elements of polypharmacy (eg, cognitive burden, adherence) and the pharmacologic elements, including drug–drug interactions. For many children, the safety implications of polypharmacy may be more consequential due to the reliance of multiple caregivers to administer medications, which requires additional coordination to ensure that medications are administered and not duplicated. Dual administration of the same medication by both parents is the most common reason for pediatric calls to Poison Control Centers.⁵ Yet, there is a paucity of research in this area, with most of the pediatric literature focusing on the outpatient setting and specific populations, including epilepsy and mental health.^6-8

How providers, patients, and families translate medication lists to counts of medications—and hence the burden of polypharmacy—is not clearly or consistently described. Often in studies of polypharmacy, researchers utilize medication claims data to count the number of medications a patient has filled from the pharmacy. However, in routine clinical practice, clinicians rarely have access to medication claims and thus rely on patient or family report, which may or may not match the list of medications in the patients’ medical records.

Therefore, linking polypharmacy research to the pragmatic complexities of clinical care requires greater clarity and consistent application of concepts. At hospital discharge, families receive a list of medications to take, including home medications to resume as well as newly prescribed medications. However, not all medications are equally essential to patients’ care regarding importance of administration (eg, hydrocortisone ointment versus an anticonvulsant medication). Patients, parents, and caregivers are ultimately responsible for determining which medications to prioritize and administer.

Although there is no standard numerical definition for how to identify polypharmacy, five medications is commonly considered the threshold for polypharmacy.⁹ A recent review of the pediatric polypharmacy literature suggested a lower threshold, with any two concurrent medications for at least a day.⁷ Yet, the best approach to “count” medications at hospital discharge is unclear. The simplest method is to tally the number of medications listed in the discharge summary. However, medications are sometimes listed twice due to different dosages administered at different times. Frequently, medications are prescribed on an as-needed basis; these medications could be administered routinely or very infrequently (eg, epinephrine for anaphylaxis). Over-the-counter medications are also sometimes included in discharge summaries and consideration should be given as to whether these medications count toward measures of polypharmacy. Over-the-counter medications would not be counted by a polypharmacy measure that relies on claims data if those medications are not paid by the insurer.

We sought consensus on how to count discharge medications through a series of informal interviews with hospitalists, nurses, and parents. We asked the seemingly simple question, “How many medications is this child on?” across a variety of scenarios (Figure). For panel A, all stakeholders agreed that this medication list includes two medications. All other scenarios elicited disagreement. For panel B, many people responded three medications, but others (often physicians) counted only clindamycin and therefore responded one medication.

Disclosures

Dr. Auger has nothing to disclose. Dr. Shah is the Editor-in-Chief of the Journal of Hospital Medicine. Dr. Davis has nothing to disclose. Dr. Brady reports grants from Agency for Healthcare Research and Quality, outside the submitted work.

Funding

This project is supported by a grant from the Agency for Healthcare Research and Quality (1K08HS204735-01A1).

Polypharmacy, the practice of taking multiple medications to manage health conditions, is common for children. Many children today have a higher burden chronic illness and an increasing number of pharmaceuticals—often delivered in various doses throughout the day. Polypharmacy has been linked to a variety of pediatric and adult outcomes, including medication errors and readmission.^1-3 Consequently, the Society of Hospital Medicine recognizes polypharmacy as a risk factor for readmission for adult populations.⁴ These adverse outcomes are related to both the human elements of polypharmacy (eg, cognitive burden, adherence) and the pharmacologic elements, including drug–drug interactions. For many children, the safety implications of polypharmacy may be more consequential due to the reliance of multiple caregivers to administer medications, which requires additional coordination to ensure that medications are administered and not duplicated. Dual administration of the same medication by both parents is the most common reason for pediatric calls to Poison Control Centers.⁵ Yet, there is a paucity of research in this area, with most of the pediatric literature focusing on the outpatient setting and specific populations, including epilepsy and mental health.^6-8

How providers, patients, and families translate medication lists to counts of medications—and hence the burden of polypharmacy—is not clearly or consistently described. Often in studies of polypharmacy, researchers utilize medication claims data to count the number of medications a patient has filled from the pharmacy. However, in routine clinical practice, clinicians rarely have access to medication claims and thus rely on patient or family report, which may or may not match the list of medications in the patients’ medical records.

Therefore, linking polypharmacy research to the pragmatic complexities of clinical care requires greater clarity and consistent application of concepts. At hospital discharge, families receive a list of medications to take, including home medications to resume as well as newly prescribed medications. However, not all medications are equally essential to patients’ care regarding importance of administration (eg, hydrocortisone ointment versus an anticonvulsant medication). Patients, parents, and caregivers are ultimately responsible for determining which medications to prioritize and administer.

Although there is no standard numerical definition for how to identify polypharmacy, five medications is commonly considered the threshold for polypharmacy.⁹ A recent review of the pediatric polypharmacy literature suggested a lower threshold, with any two concurrent medications for at least a day.⁷ Yet, the best approach to “count” medications at hospital discharge is unclear. The simplest method is to tally the number of medications listed in the discharge summary. However, medications are sometimes listed twice due to different dosages administered at different times. Frequently, medications are prescribed on an as-needed basis; these medications could be administered routinely or very infrequently (eg, epinephrine for anaphylaxis). Over-the-counter medications are also sometimes included in discharge summaries and consideration should be given as to whether these medications count toward measures of polypharmacy. Over-the-counter medications would not be counted by a polypharmacy measure that relies on claims data if those medications are not paid by the insurer.

We sought consensus on how to count discharge medications through a series of informal interviews with hospitalists, nurses, and parents. We asked the seemingly simple question, “How many medications is this child on?” across a variety of scenarios (Figure). For panel A, all stakeholders agreed that this medication list includes two medications. All other scenarios elicited disagreement. For panel B, many people responded three medications, but others (often physicians) counted only clindamycin and therefore responded one medication.

Disclosures

Dr. Auger has nothing to disclose. Dr. Shah is the Editor-in-Chief of the Journal of Hospital Medicine. Dr. Davis has nothing to disclose. Dr. Brady reports grants from Agency for Healthcare Research and Quality, outside the submitted work.

Funding

This project is supported by a grant from the Agency for Healthcare Research and Quality (1K08HS204735-01A1).

Hospitalizations are disruptive, stressful, and costly for patients and families.^1-5 Hospital readmissions subject families to the additional morbidity inherent to hospitalization and place patients at additional risk of hospital-acquired conditions or other harm.^6-9 In pediatrics, hospital readmissions are common for specific conditions;¹⁰ with rates varying across institutions;^10,11 and as many as one-third of unplanned pediatric readmissions are potentially preventable.¹²

Reducing pediatric readmissions requires a deeper understanding of the mechanisms through which readmissions occur. Medical complexity—specifically chronic conditions and use of medical technology—is associated with increased risk of readmission.^13,14 Polypharmacy at discharge has also been associated with readmission.^15,16 However, prior studies on polypharmacy and readmission risk examined the count of total medications and did not consider the nuances of scheduled versus as-needed medications, or the frequency of doses. These nuances may be critical to caregivers as discharge medical complexity can be overwhelming, even in diagnoses which are not traditionally considered complex.¹⁷ Finally, of potentially greater importance than medical complexity at discharge is a change in medical complexity during a hospitalization—for example, new diagnoses or new technologies that require additional education in hospital and management at home.

We sought to further understand the relationship between discharge medical complexity and readmission risk with regards to polypharmacy and home healthcare referrals at discharge. Specifically, we hypothesized that a change in medical complexity during an admission—ie, a new chronic diagnosis or new technology—would be a more prominent risk factor for readmission than discharge complexity alone. We examined these factors in the context of length of stay (LOS) since this is a marker of in-hospital severity of illness and a potentially modifiable function of time allowed for in-hospital teaching and discharge preparation.

We conducted a retrospective, case-control study of pediatric hospitalizations at one tertiary care children’s hospital. Children <18 years were eligible for inclusion. Normal birth hospitalizations were excluded. We randomly selected one hospitalization from each child as the index visit. We identified cases, hospitalizations at C.S. Mott Children’s Hospital between 2008 and 2012 with a subsequent unplanned 30-day readmission,¹⁸ and matched them one to one with hospitalizations at the same hospital during the same period without subsequent readmission. We matched cases to controls based on the month of admission to account for seasonality of certain illnesses. We also matched on distance and direction from the hospital to the patient’s home to account for the potential to have readmissions to other institutions. We utilized both distance and direction recognizing that a family living 30 miles in one direction would be closer to an urban area with access to more facilities, as opposed to 30 miles in another direction in a rural area without additional access. We subsequently performed medical record review to abstract relevant covariates.

Primary Predictors

Medical Complexity Models (Models 1 and 2):

We evaluated three attributes of discharge medical complexity abstracted by medical record review—discharge medications, technology assistance (ie, tracheostomy, cerebral spinal fluid ventricular shunt, enteral feeding tube, central line), and the need for home healthcare after discharge. We counted discharge medications based on the number of medications listed on the discharge summary separated into scheduled or as needed.¹⁹ We also considered the number of scheduled doses to be administered in a 24-hour period (see Appendix methods for more information on counting discharge medications). For assistance by technology, we considered the presence of tracheostomy, cerebral spinal fluid ventricular shunt, enteral feeding tube, and central lines. While we describe these technologies separately, for multivariable analyses we considered the presence of any of the four types of technology.

Change in Medical Complexity Models (Models 3 and 4)

We examined two aspects of change in medical complexity—the presence of a new complex chronic condition (CCC)²⁰ diagnosed during the hospitalization, and a new reliance on medical technology. The presence of new CCC was determined by comparing discharge diagnoses to past medical history abstracted by medical record review. A new CCC was defined as any complex chronic condition that was captured in the discharge diagnoses but was not evident in the past medical history. By definition, all CCCs coded during birth hospitalization (eg, at discharge from the neonatal intensive care unit) were assigned to “new” CCC. We calculated a kappa statistic to determine interrater reliability in determining the designation of new CCC. A sensitivity analysis examining these birth CCCs was also performed comparing no new CCC, new CCC, and new CCC after birth hospitalization. The methods appendix provides additional information on considering new CCCs. New technology, abstracted from chart review, was defined as technology placed during hospitalization that remained in place at discharge. If a child with existing technology had additional technology placed during the hospitalization (eg, a new tracheostomy in a child with a previously placed enteral feeding tube), the encounter was considered as having new technology placed.

Covariates

We created different sets of multivariable models to account for patient/hospitalization characteristics. In Models 1 and 3, we examined the primary predictors adjusting for patient characteristics (age, race/ethnicity, sex, and insurance). In Models 2 and 4, we added the index hospitalization LOS into the multivariable models adjusting for patient characteristics. We chose to add LOS in a second set of models because it is a potentially important confounder in readmission risk: discharge timing is a modifiable factor dependent on both physiologic recovery and the medical team’s perception of caregiver’s readiness for discharge. We elected to present models with and without LOS since LOS is also a marker of illness severity while in the hospital and is linked to discharge complexity.

Statistical Analysis

A review of 600 cases and 600 controls yields 89% power to detect statistical significance for covariates with an odds ratio of 1.25 (β = 0.22) if the candidate covariate has low to moderate correlation with other covariates (<0.3). If a candidate covariate has a moderate correlation with other covariates (0.6), we have 89% power to detect an odds ratio of 1.35 (β = 0.30).²¹ We calculated odds of 30-days unplanned readmission using conditional logistic regression to account for matched case-control design. All the analyses were performed using STATA 13 (Stata Corp., College Station, Texas).

Of the 41,422 eligible index hospitalizations during the study period, 9.4% resulted in a 30-day unplanned readmission. After randomly selecting one hospitalization per child, there were 781 eligible cases. We subsequent matched all but one eligible case to a control. We randomly selected encounters for medical record review, reviewing a total of 1,212 encounters. After excluding pairs with incomplete records, we included 595 cases and 595 controls in this analysis (Figure). Patient/hospitalization characteristics are displayed in Table 1. The most frequent primary discharge diagnoses are displayed in Appendix Table 1.

Models of Medical Complexity at Discharge

Polypharmacy after discharge was common for both readmitted and nonreadmitted patients. Children who experienced unplanned readmission in 30 days were discharged with a median of four different scheduled medications (interquartile range [IQR] 2,7) which translated into a median of six (IQR 3,12) scheduled doses in a 24-hour period. In comparison, children without an unplanned readmission had a median of two different scheduled medications (IQR 1,3) with a median of three (IQR 0,7) scheduled doses in a 24-hour period. Medical technology was more common in case children (42%) than in control children (14%). Central lines and enteral tubes were the most common forms of medical technology in both cases and controls. Home health referral was common in both cases (44%) and controls (23%; Table 1).

Change in Medical Complexity Models

The adjudication of new CCCs had good reliability (Κ = 0.72). New CCCs occurred in 18% and new technologies occurred in 17% of cases. Comparatively, new CCCs occurred in 10% and new technologies in 7% of hospitalizations in control children (Table 1). In bivariate analyses, both aspects of change in medical complexity were associated with higher odds of readmission (Table 2). In multivariate analysis with patient characteristics (Model 3; Table 3), all aspects of change in complexity were associated with elevated odds of readmission. A new CCC was associated with higher odds of readmission (adjusted OR (AOR) 1.75, 95% CI: 1.11-2.75) as was new technology during admission (AOR 1.84, 95%CI: 1.09-3.10). Furthermore, the odds of readmission for medical complexity variables (polypharmacy and home healthcare need) remained largely unchanged when adding the change in medical complexity variables (ie, comparing Model 1 and Model 3). However, when accounting for LOS (Model 4), neither the acquisition of a new CCC nor the addition of new technology was associated with readmission. The most common form of new technology was central line followed by nonsurgically placed enteral tube (Appendix Table 2). Finally, in sensitivity analyses (results not detailed), separating new CCC acquired at birth and new CCCs in nonbirth hospitalizations, compared to hospitalizations with no new CCC, yielded similar results as the primary analyses.

We examined multiple attributes of polypharmacy—the number of scheduled medications, number of as-needed medications, and number of scheduled doses per 24 hours. Interestingly, only the scheduled medications (count of medication and number of doses) were associated with elevated readmission risk. As-needed medications have heterogeneity in the level of importance from critical (eg, seizure rescue) to discretionary (eg, antipyretics, creams). The burden of managing these types of medications may still be high (ie, parents must decide when to administer a critical medication); however, this burden does not translate into increased readmission risk in this population.

Not surprisingly, greater medical complexity—as defined by higher numbers of scheduled discharge medications and technology assistance—is associated with 30-day readmission risk. Our analyses do not allow us to determine how much of the increased risk is due to additional care burden and risks of polypharmacy versus the inherent increase in complexity and severity of illness for which polypharmacy is a marker. Tailoring discharge regimens to the realities of daily life, with the goal of “minimally disruptive medicine”^22,23 (eg, integrating manageable discharge medication routines into school and work schedules), is not a common feature of pediatric discharge planning. For adult patients with complex medical conditions, tailoring medication regimens in a minimally disruptive way is known to improve outcomes.²⁴ Similarly, adopting minimally disruptive techniques to integrate the polypharmacy inherent in discharge could potentially mitigate some of the readmission risks for children and adolescents.

Contrary to our hypothesis, new technologies and new diagnoses did not confer additional readmission risk when accounting for LOS and patient characteristics. One potential explanation is varying risks conveyed by different types of new technologies placed during hospitalization. Central lines, the most common form of new technology, is associated with higher odds of reutilization in unadjusted analyses. However, the second most common form of new technology, nonsurgically placed enteral feeding tube, was not. Further analyses of the differential effects of new technology should be further examined in larger datasets. Additionally, the lack of additional readmission risk from new technology may relate to additional teaching and support provided to families of patients undergoing unfamiliar procedures offsets the risks inherent of greater complexity. If so, it may be that the more intensive teaching and postdischarge support provided to families with new technology or a new diagnosis could be replicated through refresher teaching during hospitalizations, when a patient’s state of health is status quo for the family (ie, the child was admitted and discharged with the same technology and diagnoses). This notion is supported by prior work that demonstrated successful readmission reduction interventions for children with chronic conditions often rely on enhanced education or coaching.^25,26

We elected to present models both with and without LOS as a confounder because it is a potentially modifiable attribute of hospitalization. Change in medical complexity aspects were significantly associated with readmission in multivariable models without LOS. However, with the addition of LOS, they were no longer significant. Thus, the readmission risk of new complexity is accounted for by the readmission risk inherent in a longer LOS. This finding prompts additional questions that merit further study: is it that LOS is a general marker for heightened complexity, or is it that a longer LOS can modify readmission risk through additional in-hospital care and time for enhanced education?

Our study has several strengths. We were able to discern true complexity at the time of discharge through medical record review. For example, if a child had a peripherally inserted central catheter placed during hospitalization, it cannot be ascertained through administrative data without medical record review if the technology was removed or in place at discharge. Likewise, medical record review allows for identification of medical technology which is not surgically implanted (eg, nasogastric feeding tubes). Given the “fog” families report as part of their in-hospital experience and its threats to education and postdischarge contingency planning,¹⁷ we felt it important to evaluate medical technology regardless of whether or not it was surgically placed. Additionally, the more detailed and nuanced understanding gained of polypharmacy burden can better inform both risk prediction models and interventions to improve the transition from hospital to home.

This study should also be considered in the context of several limitations. First, the data was from a single children’s hospital, so the generalizability of our findings is uncertain. Second, we utilized a novel method for counting new CCCs which compared information collected for clinical purposes (eg, obtaining a past medical history) with data collected for billing purposes (ie, discharge diagnoses). This comparison of information collected for different purposes potentially introduced uncertainty in the classification of diagnoses as new or not new; however, the interrater reliability for adjudicating new diagnoses suggests that the process was reasonably reliable. Third, we did not have access to other hospitals where readmissions could have occurred. While this is a common limitation for readmission studies,^{10,12,14,15,18,27-29} we attempted to mitigate any differential risk of being readmitted to other institutions by matching on distance and direction from the hospital. Of note, it is possible that children with medical complexity may be more willing to travel further to the hospital of their choice; thus our matching may be imperfect. However, there is no established method available to identify preadmission medical complexity through administrative data. Finally, the case-control method of the study makes estimating the true incidence of a variety of elements of medical complexity challenging. For example, it is difficult to tell how often children are discharged on five or more medications from a population standpoint when this practice was quite common for cases. Likewise, the true incidence of new technologies and new CCCs is challenging to estimate.

Medical complexity at discharge is associated with pediatric readmission risk. Contrary to our hypothesis, the addition of new technologies and new CCC diagnoses are not associated with pediatric readmission, after accounting for patient and hospitalization factors including LOS. The dynamics of LOS as a risk factor for readmission for children with medical complexity are likely multifaceted and merit further investigation in a multi-institutional study.

Disclosures

The authors report no potential conflicts of interest.

Funding

This work was supported by a grant from the Agency for Healthcare Research and Quality (1K08HS204735-01A1) and a grant from the Blue Cross Blue Shield of Michigan Foundation.

Hospitalizations are disruptive, stressful, and costly for patients and families.^1-5 Hospital readmissions subject families to the additional morbidity inherent to hospitalization and place patients at additional risk of hospital-acquired conditions or other harm.^6-9 In pediatrics, hospital readmissions are common for specific conditions;¹⁰ with rates varying across institutions;^10,11 and as many as one-third of unplanned pediatric readmissions are potentially preventable.¹²

Reducing pediatric readmissions requires a deeper understanding of the mechanisms through which readmissions occur. Medical complexity—specifically chronic conditions and use of medical technology—is associated with increased risk of readmission.^13,14 Polypharmacy at discharge has also been associated with readmission.^15,16 However, prior studies on polypharmacy and readmission risk examined the count of total medications and did not consider the nuances of scheduled versus as-needed medications, or the frequency of doses. These nuances may be critical to caregivers as discharge medical complexity can be overwhelming, even in diagnoses which are not traditionally considered complex.¹⁷ Finally, of potentially greater importance than medical complexity at discharge is a change in medical complexity during a hospitalization—for example, new diagnoses or new technologies that require additional education in hospital and management at home.

We sought to further understand the relationship between discharge medical complexity and readmission risk with regards to polypharmacy and home healthcare referrals at discharge. Specifically, we hypothesized that a change in medical complexity during an admission—ie, a new chronic diagnosis or new technology—would be a more prominent risk factor for readmission than discharge complexity alone. We examined these factors in the context of length of stay (LOS) since this is a marker of in-hospital severity of illness and a potentially modifiable function of time allowed for in-hospital teaching and discharge preparation.

We conducted a retrospective, case-control study of pediatric hospitalizations at one tertiary care children’s hospital. Children <18 years were eligible for inclusion. Normal birth hospitalizations were excluded. We randomly selected one hospitalization from each child as the index visit. We identified cases, hospitalizations at C.S. Mott Children’s Hospital between 2008 and 2012 with a subsequent unplanned 30-day readmission,¹⁸ and matched them one to one with hospitalizations at the same hospital during the same period without subsequent readmission. We matched cases to controls based on the month of admission to account for seasonality of certain illnesses. We also matched on distance and direction from the hospital to the patient’s home to account for the potential to have readmissions to other institutions. We utilized both distance and direction recognizing that a family living 30 miles in one direction would be closer to an urban area with access to more facilities, as opposed to 30 miles in another direction in a rural area without additional access. We subsequently performed medical record review to abstract relevant covariates.

Primary Predictors

Medical Complexity Models (Models 1 and 2):

We evaluated three attributes of discharge medical complexity abstracted by medical record review—discharge medications, technology assistance (ie, tracheostomy, cerebral spinal fluid ventricular shunt, enteral feeding tube, central line), and the need for home healthcare after discharge. We counted discharge medications based on the number of medications listed on the discharge summary separated into scheduled or as needed.¹⁹ We also considered the number of scheduled doses to be administered in a 24-hour period (see Appendix methods for more information on counting discharge medications). For assistance by technology, we considered the presence of tracheostomy, cerebral spinal fluid ventricular shunt, enteral feeding tube, and central lines. While we describe these technologies separately, for multivariable analyses we considered the presence of any of the four types of technology.

Change in Medical Complexity Models (Models 3 and 4)

We examined two aspects of change in medical complexity—the presence of a new complex chronic condition (CCC)²⁰ diagnosed during the hospitalization, and a new reliance on medical technology. The presence of new CCC was determined by comparing discharge diagnoses to past medical history abstracted by medical record review. A new CCC was defined as any complex chronic condition that was captured in the discharge diagnoses but was not evident in the past medical history. By definition, all CCCs coded during birth hospitalization (eg, at discharge from the neonatal intensive care unit) were assigned to “new” CCC. We calculated a kappa statistic to determine interrater reliability in determining the designation of new CCC. A sensitivity analysis examining these birth CCCs was also performed comparing no new CCC, new CCC, and new CCC after birth hospitalization. The methods appendix provides additional information on considering new CCCs. New technology, abstracted from chart review, was defined as technology placed during hospitalization that remained in place at discharge. If a child with existing technology had additional technology placed during the hospitalization (eg, a new tracheostomy in a child with a previously placed enteral feeding tube), the encounter was considered as having new technology placed.

Covariates

We created different sets of multivariable models to account for patient/hospitalization characteristics. In Models 1 and 3, we examined the primary predictors adjusting for patient characteristics (age, race/ethnicity, sex, and insurance). In Models 2 and 4, we added the index hospitalization LOS into the multivariable models adjusting for patient characteristics. We chose to add LOS in a second set of models because it is a potentially important confounder in readmission risk: discharge timing is a modifiable factor dependent on both physiologic recovery and the medical team’s perception of caregiver’s readiness for discharge. We elected to present models with and without LOS since LOS is also a marker of illness severity while in the hospital and is linked to discharge complexity.

Statistical Analysis

A review of 600 cases and 600 controls yields 89% power to detect statistical significance for covariates with an odds ratio of 1.25 (β = 0.22) if the candidate covariate has low to moderate correlation with other covariates (<0.3). If a candidate covariate has a moderate correlation with other covariates (0.6), we have 89% power to detect an odds ratio of 1.35 (β = 0.30).²¹ We calculated odds of 30-days unplanned readmission using conditional logistic regression to account for matched case-control design. All the analyses were performed using STATA 13 (Stata Corp., College Station, Texas).

Of the 41,422 eligible index hospitalizations during the study period, 9.4% resulted in a 30-day unplanned readmission. After randomly selecting one hospitalization per child, there were 781 eligible cases. We subsequent matched all but one eligible case to a control. We randomly selected encounters for medical record review, reviewing a total of 1,212 encounters. After excluding pairs with incomplete records, we included 595 cases and 595 controls in this analysis (Figure). Patient/hospitalization characteristics are displayed in Table 1. The most frequent primary discharge diagnoses are displayed in Appendix Table 1.

Models of Medical Complexity at Discharge

Polypharmacy after discharge was common for both readmitted and nonreadmitted patients. Children who experienced unplanned readmission in 30 days were discharged with a median of four different scheduled medications (interquartile range [IQR] 2,7) which translated into a median of six (IQR 3,12) scheduled doses in a 24-hour period. In comparison, children without an unplanned readmission had a median of two different scheduled medications (IQR 1,3) with a median of three (IQR 0,7) scheduled doses in a 24-hour period. Medical technology was more common in case children (42%) than in control children (14%). Central lines and enteral tubes were the most common forms of medical technology in both cases and controls. Home health referral was common in both cases (44%) and controls (23%; Table 1).

Change in Medical Complexity Models

The adjudication of new CCCs had good reliability (Κ = 0.72). New CCCs occurred in 18% and new technologies occurred in 17% of cases. Comparatively, new CCCs occurred in 10% and new technologies in 7% of hospitalizations in control children (Table 1). In bivariate analyses, both aspects of change in medical complexity were associated with higher odds of readmission (Table 2). In multivariate analysis with patient characteristics (Model 3; Table 3), all aspects of change in complexity were associated with elevated odds of readmission. A new CCC was associated with higher odds of readmission (adjusted OR (AOR) 1.75, 95% CI: 1.11-2.75) as was new technology during admission (AOR 1.84, 95%CI: 1.09-3.10). Furthermore, the odds of readmission for medical complexity variables (polypharmacy and home healthcare need) remained largely unchanged when adding the change in medical complexity variables (ie, comparing Model 1 and Model 3). However, when accounting for LOS (Model 4), neither the acquisition of a new CCC nor the addition of new technology was associated with readmission. The most common form of new technology was central line followed by nonsurgically placed enteral tube (Appendix Table 2). Finally, in sensitivity analyses (results not detailed), separating new CCC acquired at birth and new CCCs in nonbirth hospitalizations, compared to hospitalizations with no new CCC, yielded similar results as the primary analyses.

We examined multiple attributes of polypharmacy—the number of scheduled medications, number of as-needed medications, and number of scheduled doses per 24 hours. Interestingly, only the scheduled medications (count of medication and number of doses) were associated with elevated readmission risk. As-needed medications have heterogeneity in the level of importance from critical (eg, seizure rescue) to discretionary (eg, antipyretics, creams). The burden of managing these types of medications may still be high (ie, parents must decide when to administer a critical medication); however, this burden does not translate into increased readmission risk in this population.

Not surprisingly, greater medical complexity—as defined by higher numbers of scheduled discharge medications and technology assistance—is associated with 30-day readmission risk. Our analyses do not allow us to determine how much of the increased risk is due to additional care burden and risks of polypharmacy versus the inherent increase in complexity and severity of illness for which polypharmacy is a marker. Tailoring discharge regimens to the realities of daily life, with the goal of “minimally disruptive medicine”^22,23 (eg, integrating manageable discharge medication routines into school and work schedules), is not a common feature of pediatric discharge planning. For adult patients with complex medical conditions, tailoring medication regimens in a minimally disruptive way is known to improve outcomes.²⁴ Similarly, adopting minimally disruptive techniques to integrate the polypharmacy inherent in discharge could potentially mitigate some of the readmission risks for children and adolescents.

Contrary to our hypothesis, new technologies and new diagnoses did not confer additional readmission risk when accounting for LOS and patient characteristics. One potential explanation is varying risks conveyed by different types of new technologies placed during hospitalization. Central lines, the most common form of new technology, is associated with higher odds of reutilization in unadjusted analyses. However, the second most common form of new technology, nonsurgically placed enteral feeding tube, was not. Further analyses of the differential effects of new technology should be further examined in larger datasets. Additionally, the lack of additional readmission risk from new technology may relate to additional teaching and support provided to families of patients undergoing unfamiliar procedures offsets the risks inherent of greater complexity. If so, it may be that the more intensive teaching and postdischarge support provided to families with new technology or a new diagnosis could be replicated through refresher teaching during hospitalizations, when a patient’s state of health is status quo for the family (ie, the child was admitted and discharged with the same technology and diagnoses). This notion is supported by prior work that demonstrated successful readmission reduction interventions for children with chronic conditions often rely on enhanced education or coaching.^25,26

We elected to present models both with and without LOS as a confounder because it is a potentially modifiable attribute of hospitalization. Change in medical complexity aspects were significantly associated with readmission in multivariable models without LOS. However, with the addition of LOS, they were no longer significant. Thus, the readmission risk of new complexity is accounted for by the readmission risk inherent in a longer LOS. This finding prompts additional questions that merit further study: is it that LOS is a general marker for heightened complexity, or is it that a longer LOS can modify readmission risk through additional in-hospital care and time for enhanced education?

Our study has several strengths. We were able to discern true complexity at the time of discharge through medical record review. For example, if a child had a peripherally inserted central catheter placed during hospitalization, it cannot be ascertained through administrative data without medical record review if the technology was removed or in place at discharge. Likewise, medical record review allows for identification of medical technology which is not surgically implanted (eg, nasogastric feeding tubes). Given the “fog” families report as part of their in-hospital experience and its threats to education and postdischarge contingency planning,¹⁷ we felt it important to evaluate medical technology regardless of whether or not it was surgically placed. Additionally, the more detailed and nuanced understanding gained of polypharmacy burden can better inform both risk prediction models and interventions to improve the transition from hospital to home.

This study should also be considered in the context of several limitations. First, the data was from a single children’s hospital, so the generalizability of our findings is uncertain. Second, we utilized a novel method for counting new CCCs which compared information collected for clinical purposes (eg, obtaining a past medical history) with data collected for billing purposes (ie, discharge diagnoses). This comparison of information collected for different purposes potentially introduced uncertainty in the classification of diagnoses as new or not new; however, the interrater reliability for adjudicating new diagnoses suggests that the process was reasonably reliable. Third, we did not have access to other hospitals where readmissions could have occurred. While this is a common limitation for readmission studies,^{10,12,14,15,18,27-29} we attempted to mitigate any differential risk of being readmitted to other institutions by matching on distance and direction from the hospital. Of note, it is possible that children with medical complexity may be more willing to travel further to the hospital of their choice; thus our matching may be imperfect. However, there is no established method available to identify preadmission medical complexity through administrative data. Finally, the case-control method of the study makes estimating the true incidence of a variety of elements of medical complexity challenging. For example, it is difficult to tell how often children are discharged on five or more medications from a population standpoint when this practice was quite common for cases. Likewise, the true incidence of new technologies and new CCCs is challenging to estimate.

Medical complexity at discharge is associated with pediatric readmission risk. Contrary to our hypothesis, the addition of new technologies and new CCC diagnoses are not associated with pediatric readmission, after accounting for patient and hospitalization factors including LOS. The dynamics of LOS as a risk factor for readmission for children with medical complexity are likely multifaceted and merit further investigation in a multi-institutional study.

Disclosures

The authors report no potential conflicts of interest.

Funding

This work was supported by a grant from the Agency for Healthcare Research and Quality (1K08HS204735-01A1) and a grant from the Blue Cross Blue Shield of Michigan Foundation.

Hospitalizations are disruptive, stressful, and costly for patients and families.^1-5 Hospital readmissions subject families to the additional morbidity inherent to hospitalization and place patients at additional risk of hospital-acquired conditions or other harm.^6-9 In pediatrics, hospital readmissions are common for specific conditions;¹⁰ with rates varying across institutions;^10,11 and as many as one-third of unplanned pediatric readmissions are potentially preventable.¹²

Reducing pediatric readmissions requires a deeper understanding of the mechanisms through which readmissions occur. Medical complexity—specifically chronic conditions and use of medical technology—is associated with increased risk of readmission.^13,14 Polypharmacy at discharge has also been associated with readmission.^15,16 However, prior studies on polypharmacy and readmission risk examined the count of total medications and did not consider the nuances of scheduled versus as-needed medications, or the frequency of doses. These nuances may be critical to caregivers as discharge medical complexity can be overwhelming, even in diagnoses which are not traditionally considered complex.¹⁷ Finally, of potentially greater importance than medical complexity at discharge is a change in medical complexity during a hospitalization—for example, new diagnoses or new technologies that require additional education in hospital and management at home.

We sought to further understand the relationship between discharge medical complexity and readmission risk with regards to polypharmacy and home healthcare referrals at discharge. Specifically, we hypothesized that a change in medical complexity during an admission—ie, a new chronic diagnosis or new technology—would be a more prominent risk factor for readmission than discharge complexity alone. We examined these factors in the context of length of stay (LOS) since this is a marker of in-hospital severity of illness and a potentially modifiable function of time allowed for in-hospital teaching and discharge preparation.

We conducted a retrospective, case-control study of pediatric hospitalizations at one tertiary care children’s hospital. Children <18 years were eligible for inclusion. Normal birth hospitalizations were excluded. We randomly selected one hospitalization from each child as the index visit. We identified cases, hospitalizations at C.S. Mott Children’s Hospital between 2008 and 2012 with a subsequent unplanned 30-day readmission,¹⁸ and matched them one to one with hospitalizations at the same hospital during the same period without subsequent readmission. We matched cases to controls based on the month of admission to account for seasonality of certain illnesses. We also matched on distance and direction from the hospital to the patient’s home to account for the potential to have readmissions to other institutions. We utilized both distance and direction recognizing that a family living 30 miles in one direction would be closer to an urban area with access to more facilities, as opposed to 30 miles in another direction in a rural area without additional access. We subsequently performed medical record review to abstract relevant covariates.

Primary Predictors

Medical Complexity Models (Models 1 and 2):

We evaluated three attributes of discharge medical complexity abstracted by medical record review—discharge medications, technology assistance (ie, tracheostomy, cerebral spinal fluid ventricular shunt, enteral feeding tube, central line), and the need for home healthcare after discharge. We counted discharge medications based on the number of medications listed on the discharge summary separated into scheduled or as needed.¹⁹ We also considered the number of scheduled doses to be administered in a 24-hour period (see Appendix methods for more information on counting discharge medications). For assistance by technology, we considered the presence of tracheostomy, cerebral spinal fluid ventricular shunt, enteral feeding tube, and central lines. While we describe these technologies separately, for multivariable analyses we considered the presence of any of the four types of technology.

Change in Medical Complexity Models (Models 3 and 4)

We examined two aspects of change in medical complexity—the presence of a new complex chronic condition (CCC)²⁰ diagnosed during the hospitalization, and a new reliance on medical technology. The presence of new CCC was determined by comparing discharge diagnoses to past medical history abstracted by medical record review. A new CCC was defined as any complex chronic condition that was captured in the discharge diagnoses but was not evident in the past medical history. By definition, all CCCs coded during birth hospitalization (eg, at discharge from the neonatal intensive care unit) were assigned to “new” CCC. We calculated a kappa statistic to determine interrater reliability in determining the designation of new CCC. A sensitivity analysis examining these birth CCCs was also performed comparing no new CCC, new CCC, and new CCC after birth hospitalization. The methods appendix provides additional information on considering new CCCs. New technology, abstracted from chart review, was defined as technology placed during hospitalization that remained in place at discharge. If a child with existing technology had additional technology placed during the hospitalization (eg, a new tracheostomy in a child with a previously placed enteral feeding tube), the encounter was considered as having new technology placed.

Covariates

We created different sets of multivariable models to account for patient/hospitalization characteristics. In Models 1 and 3, we examined the primary predictors adjusting for patient characteristics (age, race/ethnicity, sex, and insurance). In Models 2 and 4, we added the index hospitalization LOS into the multivariable models adjusting for patient characteristics. We chose to add LOS in a second set of models because it is a potentially important confounder in readmission risk: discharge timing is a modifiable factor dependent on both physiologic recovery and the medical team’s perception of caregiver’s readiness for discharge. We elected to present models with and without LOS since LOS is also a marker of illness severity while in the hospital and is linked to discharge complexity.

Statistical Analysis

A review of 600 cases and 600 controls yields 89% power to detect statistical significance for covariates with an odds ratio of 1.25 (β = 0.22) if the candidate covariate has low to moderate correlation with other covariates (<0.3). If a candidate covariate has a moderate correlation with other covariates (0.6), we have 89% power to detect an odds ratio of 1.35 (β = 0.30).²¹ We calculated odds of 30-days unplanned readmission using conditional logistic regression to account for matched case-control design. All the analyses were performed using STATA 13 (Stata Corp., College Station, Texas).

Of the 41,422 eligible index hospitalizations during the study period, 9.4% resulted in a 30-day unplanned readmission. After randomly selecting one hospitalization per child, there were 781 eligible cases. We subsequent matched all but one eligible case to a control. We randomly selected encounters for medical record review, reviewing a total of 1,212 encounters. After excluding pairs with incomplete records, we included 595 cases and 595 controls in this analysis (Figure). Patient/hospitalization characteristics are displayed in Table 1. The most frequent primary discharge diagnoses are displayed in Appendix Table 1.

Models of Medical Complexity at Discharge

Polypharmacy after discharge was common for both readmitted and nonreadmitted patients. Children who experienced unplanned readmission in 30 days were discharged with a median of four different scheduled medications (interquartile range [IQR] 2,7) which translated into a median of six (IQR 3,12) scheduled doses in a 24-hour period. In comparison, children without an unplanned readmission had a median of two different scheduled medications (IQR 1,3) with a median of three (IQR 0,7) scheduled doses in a 24-hour period. Medical technology was more common in case children (42%) than in control children (14%). Central lines and enteral tubes were the most common forms of medical technology in both cases and controls. Home health referral was common in both cases (44%) and controls (23%; Table 1).

Change in Medical Complexity Models

The adjudication of new CCCs had good reliability (Κ = 0.72). New CCCs occurred in 18% and new technologies occurred in 17% of cases. Comparatively, new CCCs occurred in 10% and new technologies in 7% of hospitalizations in control children (Table 1). In bivariate analyses, both aspects of change in medical complexity were associated with higher odds of readmission (Table 2). In multivariate analysis with patient characteristics (Model 3; Table 3), all aspects of change in complexity were associated with elevated odds of readmission. A new CCC was associated with higher odds of readmission (adjusted OR (AOR) 1.75, 95% CI: 1.11-2.75) as was new technology during admission (AOR 1.84, 95%CI: 1.09-3.10). Furthermore, the odds of readmission for medical complexity variables (polypharmacy and home healthcare need) remained largely unchanged when adding the change in medical complexity variables (ie, comparing Model 1 and Model 3). However, when accounting for LOS (Model 4), neither the acquisition of a new CCC nor the addition of new technology was associated with readmission. The most common form of new technology was central line followed by nonsurgically placed enteral tube (Appendix Table 2). Finally, in sensitivity analyses (results not detailed), separating new CCC acquired at birth and new CCCs in nonbirth hospitalizations, compared to hospitalizations with no new CCC, yielded similar results as the primary analyses.

We examined multiple attributes of polypharmacy—the number of scheduled medications, number of as-needed medications, and number of scheduled doses per 24 hours. Interestingly, only the scheduled medications (count of medication and number of doses) were associated with elevated readmission risk. As-needed medications have heterogeneity in the level of importance from critical (eg, seizure rescue) to discretionary (eg, antipyretics, creams). The burden of managing these types of medications may still be high (ie, parents must decide when to administer a critical medication); however, this burden does not translate into increased readmission risk in this population.

Not surprisingly, greater medical complexity—as defined by higher numbers of scheduled discharge medications and technology assistance—is associated with 30-day readmission risk. Our analyses do not allow us to determine how much of the increased risk is due to additional care burden and risks of polypharmacy versus the inherent increase in complexity and severity of illness for which polypharmacy is a marker. Tailoring discharge regimens to the realities of daily life, with the goal of “minimally disruptive medicine”^22,23 (eg, integrating manageable discharge medication routines into school and work schedules), is not a common feature of pediatric discharge planning. For adult patients with complex medical conditions, tailoring medication regimens in a minimally disruptive way is known to improve outcomes.²⁴ Similarly, adopting minimally disruptive techniques to integrate the polypharmacy inherent in discharge could potentially mitigate some of the readmission risks for children and adolescents.

Contrary to our hypothesis, new technologies and new diagnoses did not confer additional readmission risk when accounting for LOS and patient characteristics. One potential explanation is varying risks conveyed by different types of new technologies placed during hospitalization. Central lines, the most common form of new technology, is associated with higher odds of reutilization in unadjusted analyses. However, the second most common form of new technology, nonsurgically placed enteral feeding tube, was not. Further analyses of the differential effects of new technology should be further examined in larger datasets. Additionally, the lack of additional readmission risk from new technology may relate to additional teaching and support provided to families of patients undergoing unfamiliar procedures offsets the risks inherent of greater complexity. If so, it may be that the more intensive teaching and postdischarge support provided to families with new technology or a new diagnosis could be replicated through refresher teaching during hospitalizations, when a patient’s state of health is status quo for the family (ie, the child was admitted and discharged with the same technology and diagnoses). This notion is supported by prior work that demonstrated successful readmission reduction interventions for children with chronic conditions often rely on enhanced education or coaching.^25,26

We elected to present models both with and without LOS as a confounder because it is a potentially modifiable attribute of hospitalization. Change in medical complexity aspects were significantly associated with readmission in multivariable models without LOS. However, with the addition of LOS, they were no longer significant. Thus, the readmission risk of new complexity is accounted for by the readmission risk inherent in a longer LOS. This finding prompts additional questions that merit further study: is it that LOS is a general marker for heightened complexity, or is it that a longer LOS can modify readmission risk through additional in-hospital care and time for enhanced education?

Our study has several strengths. We were able to discern true complexity at the time of discharge through medical record review. For example, if a child had a peripherally inserted central catheter placed during hospitalization, it cannot be ascertained through administrative data without medical record review if the technology was removed or in place at discharge. Likewise, medical record review allows for identification of medical technology which is not surgically implanted (eg, nasogastric feeding tubes). Given the “fog” families report as part of their in-hospital experience and its threats to education and postdischarge contingency planning,¹⁷ we felt it important to evaluate medical technology regardless of whether or not it was surgically placed. Additionally, the more detailed and nuanced understanding gained of polypharmacy burden can better inform both risk prediction models and interventions to improve the transition from hospital to home.

This study should also be considered in the context of several limitations. First, the data was from a single children’s hospital, so the generalizability of our findings is uncertain. Second, we utilized a novel method for counting new CCCs which compared information collected for clinical purposes (eg, obtaining a past medical history) with data collected for billing purposes (ie, discharge diagnoses). This comparison of information collected for different purposes potentially introduced uncertainty in the classification of diagnoses as new or not new; however, the interrater reliability for adjudicating new diagnoses suggests that the process was reasonably reliable. Third, we did not have access to other hospitals where readmissions could have occurred. While this is a common limitation for readmission studies,^{10,12,14,15,18,27-29} we attempted to mitigate any differential risk of being readmitted to other institutions by matching on distance and direction from the hospital. Of note, it is possible that children with medical complexity may be more willing to travel further to the hospital of their choice; thus our matching may be imperfect. However, there is no established method available to identify preadmission medical complexity through administrative data. Finally, the case-control method of the study makes estimating the true incidence of a variety of elements of medical complexity challenging. For example, it is difficult to tell how often children are discharged on five or more medications from a population standpoint when this practice was quite common for cases. Likewise, the true incidence of new technologies and new CCCs is challenging to estimate.

Medical complexity at discharge is associated with pediatric readmission risk. Contrary to our hypothesis, the addition of new technologies and new CCC diagnoses are not associated with pediatric readmission, after accounting for patient and hospitalization factors including LOS. The dynamics of LOS as a risk factor for readmission for children with medical complexity are likely multifaceted and merit further investigation in a multi-institutional study.

Disclosures

The authors report no potential conflicts of interest.

Funding

This work was supported by a grant from the Agency for Healthcare Research and Quality (1K08HS204735-01A1) and a grant from the Blue Cross Blue Shield of Michigan Foundation.

Unrecognized in-hospital deterioration can result in tragic consequences for pediatric patients. The majority of deterioration events have antecedents such as increasingly abnormal vital signs and new concerns from nurses.¹ Recent meta-analyses have shown that rapid response systems (RRSs), which include trigger mechanisms such as a pediatric early warning score (PEWS), are associated with a reduced rate of arrests and in-hospital mortality.^2,3 Cardiopulmonary arrest rates are useful metrics to judge the effectiveness of the system to identify and respond to deteriorating adult patients; however, there are important challenges to their use as an outcome measure in pediatrics. Arrests, which have been relatively uncommon in pediatric patients, are now even less frequent since the adoption of a RRS in the majority of children’s hospitals.^4,5 Several innovations in these systems will be context-dependent and hence best first evaluated in a single center, where arrests outside of the intensive care unit (ICU) may occur rarely. Identification of valid, more frequent proximal measures to arrests may better identify the risk factors for deterioration. This could potentially inform quality improvement efforts to mitigate clinical deterioration.

Bonafide et al. at the Children’s Hospital of Philadelphia developed and validated the critical deterioration event (CDE) metric, demonstrating that children who were transferred to the ICU and who received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of transfer had a >13-fold increased risk of in-hospital mortality.⁶ At Cincinnati Children’s Hospital Medical Center, an additional proximal outcome measure was developed for unrecognized clinical deterioration, now termed emergency transfers (ETs).^7-9 An ET is defined as any patient transferred to the ICU where the patient received intubation, inotropes, or three or more fluid boluses in the first hour after arrival or before transfer.⁹ Improvement science work that aimed at increasing clinician situation awareness was associated with a reduction in ETs,⁸ but the association of ETs with mortality or other healthcare utilization outcomes is unknown. The objective of this study was to determine the predictive validity of an ET on in-hospital mortality, ICU length of stay (LOS), and overall hospital LOS.

We conducted a case–control study at Cincinnati Children’s Hospital, a free-standing tertiary care children’s hospital. Our center has had an ICU-based RRS in place since 2005. In 2009, we eliminated the ICU consult such that each floor-to-ICU transfer is evaluated by the RRS. Nurses calculate a Monaghan PEWS every four hours on the majority of nursing units.

Patients of all ages cared for outside of the ICU at any point in their hospitalization from January 1, 2013, to July 31, 2017, were eligible for inclusion. There were no other exclusion criteria. The ICU included both the pediatric ICU and the cardiac ICU.

Cases

We identified all ET cases from an existing situation awareness database in which each RRS call is entered by the hospital nursing supervisor, whose role includes responding to each RRS activation. If the patient transfer meets the ET criteria, the nurse indicates this in the database. Each ET entry is later confirmed for assurance purposes by the nurse leader of the RRS committee (RG). For the purposes of this study, all records were again reviewed and validated using manual chart review in the electronic health record (Epic Systems, Verona, Wisconsin).

Controls

We identified nonemergent ICU transfers to serve as controls and matched those to ET in cases to limit the impact of confounders that may increase the likelihood of both an ET and a negative outcome such as ICU mortality. We identified up to three controls for each case from our database and matched in terms of age group (within five years of age), hospital unit before transfer, and time of year (within three months of ET). These variables were chosen to adjust for the impact of age, diversity of disease (as hospital units are generally organized by organ system of illness), and seasonality on outcomes.

Outcome Measures

Posttransfer LOS in the ICU, posttransfer hospital LOS, and in-hospital mortality were the primary outcome measures. Patient demographics, specific diagnoses, and number of medical conditions were a priori defined as covariates of interest. Data for each case and control were entered into a secure, web-based Research Electronic Data Capture (REDCap) database.

Analysis

Descriptive data were summarized using counts and percentages for categorical variables and medians and ranges for continuous variables due to nonnormal distributions. Chi-square test was used to compare in-hospital mortality between the ETs and the controls. The Wilcoxon rank-sum test was used to compare LOS between ETs and controls. All data analyses were performed using SAS version 9.4 (SAS Institute Inc., Cary, North Carolina).

A total of 45 ETs were identified, and 110 controls were matched. Patient demographics were similar among all cases and controls (P > .05). Patients with ETs had a median age of seven years (interquartile range: 3-18 years), and 51% of them were males. The majority of patients among our examined cases were white (68%) and non-Hispanic (93%). There was no statistical difference in insurance between the ETs and the controls. When evaluating the hospital unit before the transfer, ETs occurred most commonly in the Cardiology (22%), Hematology/Oncology (22%), and Neuroscience (16%) units.

ETs stayed longer in the ICU than non-ETs [median of 4.9 days vs 2.2 days, P = .001; Figure (A)]. Similarly, ET cases had a significantly longer posttransfer hospital LOS [median of 35 days vs 21 days, P = .001; Figure (B)]. ETs had a 22% in-hospital mortality rate, compared with 9% in-hospital mortality in the matched controls (P = .02; Table).

Children who experienced an ET had a significantly longer ICU LOS, a longer posttransfer LOS, and a higher in-hospital mortality than the matched controls who were also transferred to the ICU. Researchers and improvement science teams at multiple hospitals have demonstrated that interventions targeting improved situation awareness can reduce ETs; we have demonstrated that reducing ETs may reduce subsequent adverse outcomes.^8,10

These findings provide additional support for the use of the ET metric in children’s hospitals as a proximal measure for significant clinical deterioration. We found mortality rates that were overall high for a children’s hospital (22% in ET cases and 9% among controls) compared with a national average mortality rate of 2.3% in pediatric ICUs.¹¹ This is likely due to the study sample containing a significant proportion of children with medical complexity.

Aoki et al. recently demonstrated that ETs, compared with non-ETs, were associated with longer LOS and higher mortality in a bivariate analysis.¹² In our study, we found similar results with the important addition that these findings were robust when ETs were compared with matched controls who were likely at a higher risk of poor outcomes than ICU transfers in general. In addition, we demonstrated that ETs were associated with adverse outcomes in a United States children’s hospital with a mature, long-standing RRS process. As ETs are considerably more common than cardiac and respiratory arrests, use of the ET metric in children’s hospitals may enable more rapid learning and systems improvement implementations. We also found that most of the children with ETs present from units that care for children with substantial medical complexity, including Cardiology, Hematology/Oncology, and Neurosciences. Future work should continue to examine the relationship between medical complexity and ET risk.

The ET metric is complementary to the CDE measure developed by Bonafide et al. Both metrics capture potential events of unrecognized clinical deterioration, and both offer researchers the opportunity to better understand and improve their RRSs. Both ETs and CDEs are more common than arrests, and CDEs are more common than ETs. ETs, which by definition occur in the first hour of ICU care, are likely a more specific measure of unrecognized clinical deterioration. CDEs will capture therapies that may have been started up to 12 hours after transfer and thus are possibly more sensitive to identify unrecognized clinical deterioration. However, CDEs also may encompass some patients who arrived at the ICU after prompt recognition and then had a subacute deterioration in the ICU.

The maturity of the RRS and the bandwidth of teams to collect data may inform which metric(s) are best for individual centers. As ETs are less common and likely more specific to unrecognized clinical deterioration, they might be the first tracked as a center improves its RRS through QI methods. Alternatively, CDEs may be a useful metric for centers where unrecognized clinical deterioration is less common or in research studies where this more common outcome would lead to more power to detect the effect of interventions to improve care.

Our study had several limitations. Data collection was confined to one tertiary care children’s hospital with a high burden of complex cardiac and oncology care. The results may not generalize well to children hospitalized in smaller or community hospitals or in hospitals without a mature RRS. There is also the possibility of misclassification of covariates and outcomes, but any misclassification would likely be nondifferential and bias toward the null. Matching was not possible based on exact diagnosis, and the unit is a good but imperfect proxy for diagnosis grouping. At our center, overflow of patients into the Cardiology and Hematology/Oncology units is uncommon, mitigating this partially, although residual confounding may remain. The finding that ETs are associated with adverse outcomes does not necessarily mean that these events were preventable; however, it is important and encouraging that the rate of ETs has been reduced at two centers using improvement science interventions.^8,10

Patients who experienced an ET had a significantly higher likelihood of in-hospital mortality, spent more time in the ICU, and had a longer hospital LOS posttransfer than matched controls. The use of the ET metric in children’s hospitals would allow for further analysis of such patients in hopes of identifying clinical characteristics that serve as predictors of deterioration. This may facilitate better risk stratification in the clinical system as well as enable more rapid learning and systems improvements targeted toward preventing unrecognized clinical deterioration.

Disclosures

Dr. Hussain, Dr. Sosa, Dr. Ambroggio, and Mrs. Gallagher have nothing to disclose. Patrick Brady reports grants from the Agency for Healthcare Research and Quality, outside the submitted work. The authors certify that this submission is not under review by any other publication. The author team has no conflicts of interest to disclose.

Funding

Ms. Hussain was supported by the Society of Hospital Medicine’s Student Hospitalist Scholar Grant Program in 2017. Dr. Brady receives career development support from AHRQ K08-HS023827. The project described was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health, under Award Number 5UL1TR001425-04. The content is solely the responsibility of the authors and does not necessarily represent the official views of the SHM, AHRQ, or NIH.

Unrecognized in-hospital deterioration can result in tragic consequences for pediatric patients. The majority of deterioration events have antecedents such as increasingly abnormal vital signs and new concerns from nurses.¹ Recent meta-analyses have shown that rapid response systems (RRSs), which include trigger mechanisms such as a pediatric early warning score (PEWS), are associated with a reduced rate of arrests and in-hospital mortality.^2,3 Cardiopulmonary arrest rates are useful metrics to judge the effectiveness of the system to identify and respond to deteriorating adult patients; however, there are important challenges to their use as an outcome measure in pediatrics. Arrests, which have been relatively uncommon in pediatric patients, are now even less frequent since the adoption of a RRS in the majority of children’s hospitals.^4,5 Several innovations in these systems will be context-dependent and hence best first evaluated in a single center, where arrests outside of the intensive care unit (ICU) may occur rarely. Identification of valid, more frequent proximal measures to arrests may better identify the risk factors for deterioration. This could potentially inform quality improvement efforts to mitigate clinical deterioration.

Bonafide et al. at the Children’s Hospital of Philadelphia developed and validated the critical deterioration event (CDE) metric, demonstrating that children who were transferred to the ICU and who received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of transfer had a >13-fold increased risk of in-hospital mortality.⁶ At Cincinnati Children’s Hospital Medical Center, an additional proximal outcome measure was developed for unrecognized clinical deterioration, now termed emergency transfers (ETs).^7-9 An ET is defined as any patient transferred to the ICU where the patient received intubation, inotropes, or three or more fluid boluses in the first hour after arrival or before transfer.⁹ Improvement science work that aimed at increasing clinician situation awareness was associated with a reduction in ETs,⁸ but the association of ETs with mortality or other healthcare utilization outcomes is unknown. The objective of this study was to determine the predictive validity of an ET on in-hospital mortality, ICU length of stay (LOS), and overall hospital LOS.

We conducted a case–control study at Cincinnati Children’s Hospital, a free-standing tertiary care children’s hospital. Our center has had an ICU-based RRS in place since 2005. In 2009, we eliminated the ICU consult such that each floor-to-ICU transfer is evaluated by the RRS. Nurses calculate a Monaghan PEWS every four hours on the majority of nursing units.

Patients of all ages cared for outside of the ICU at any point in their hospitalization from January 1, 2013, to July 31, 2017, were eligible for inclusion. There were no other exclusion criteria. The ICU included both the pediatric ICU and the cardiac ICU.

Cases

We identified all ET cases from an existing situation awareness database in which each RRS call is entered by the hospital nursing supervisor, whose role includes responding to each RRS activation. If the patient transfer meets the ET criteria, the nurse indicates this in the database. Each ET entry is later confirmed for assurance purposes by the nurse leader of the RRS committee (RG). For the purposes of this study, all records were again reviewed and validated using manual chart review in the electronic health record (Epic Systems, Verona, Wisconsin).

Controls

We identified nonemergent ICU transfers to serve as controls and matched those to ET in cases to limit the impact of confounders that may increase the likelihood of both an ET and a negative outcome such as ICU mortality. We identified up to three controls for each case from our database and matched in terms of age group (within five years of age), hospital unit before transfer, and time of year (within three months of ET). These variables were chosen to adjust for the impact of age, diversity of disease (as hospital units are generally organized by organ system of illness), and seasonality on outcomes.

Outcome Measures

Posttransfer LOS in the ICU, posttransfer hospital LOS, and in-hospital mortality were the primary outcome measures. Patient demographics, specific diagnoses, and number of medical conditions were a priori defined as covariates of interest. Data for each case and control were entered into a secure, web-based Research Electronic Data Capture (REDCap) database.

Analysis

Descriptive data were summarized using counts and percentages for categorical variables and medians and ranges for continuous variables due to nonnormal distributions. Chi-square test was used to compare in-hospital mortality between the ETs and the controls. The Wilcoxon rank-sum test was used to compare LOS between ETs and controls. All data analyses were performed using SAS version 9.4 (SAS Institute Inc., Cary, North Carolina).

A total of 45 ETs were identified, and 110 controls were matched. Patient demographics were similar among all cases and controls (P > .05). Patients with ETs had a median age of seven years (interquartile range: 3-18 years), and 51% of them were males. The majority of patients among our examined cases were white (68%) and non-Hispanic (93%). There was no statistical difference in insurance between the ETs and the controls. When evaluating the hospital unit before the transfer, ETs occurred most commonly in the Cardiology (22%), Hematology/Oncology (22%), and Neuroscience (16%) units.

ETs stayed longer in the ICU than non-ETs [median of 4.9 days vs 2.2 days, P = .001; Figure (A)]. Similarly, ET cases had a significantly longer posttransfer hospital LOS [median of 35 days vs 21 days, P = .001; Figure (B)]. ETs had a 22% in-hospital mortality rate, compared with 9% in-hospital mortality in the matched controls (P = .02; Table).

Children who experienced an ET had a significantly longer ICU LOS, a longer posttransfer LOS, and a higher in-hospital mortality than the matched controls who were also transferred to the ICU. Researchers and improvement science teams at multiple hospitals have demonstrated that interventions targeting improved situation awareness can reduce ETs; we have demonstrated that reducing ETs may reduce subsequent adverse outcomes.^8,10

These findings provide additional support for the use of the ET metric in children’s hospitals as a proximal measure for significant clinical deterioration. We found mortality rates that were overall high for a children’s hospital (22% in ET cases and 9% among controls) compared with a national average mortality rate of 2.3% in pediatric ICUs.¹¹ This is likely due to the study sample containing a significant proportion of children with medical complexity.

Aoki et al. recently demonstrated that ETs, compared with non-ETs, were associated with longer LOS and higher mortality in a bivariate analysis.¹² In our study, we found similar results with the important addition that these findings were robust when ETs were compared with matched controls who were likely at a higher risk of poor outcomes than ICU transfers in general. In addition, we demonstrated that ETs were associated with adverse outcomes in a United States children’s hospital with a mature, long-standing RRS process. As ETs are considerably more common than cardiac and respiratory arrests, use of the ET metric in children’s hospitals may enable more rapid learning and systems improvement implementations. We also found that most of the children with ETs present from units that care for children with substantial medical complexity, including Cardiology, Hematology/Oncology, and Neurosciences. Future work should continue to examine the relationship between medical complexity and ET risk.

The ET metric is complementary to the CDE measure developed by Bonafide et al. Both metrics capture potential events of unrecognized clinical deterioration, and both offer researchers the opportunity to better understand and improve their RRSs. Both ETs and CDEs are more common than arrests, and CDEs are more common than ETs. ETs, which by definition occur in the first hour of ICU care, are likely a more specific measure of unrecognized clinical deterioration. CDEs will capture therapies that may have been started up to 12 hours after transfer and thus are possibly more sensitive to identify unrecognized clinical deterioration. However, CDEs also may encompass some patients who arrived at the ICU after prompt recognition and then had a subacute deterioration in the ICU.

The maturity of the RRS and the bandwidth of teams to collect data may inform which metric(s) are best for individual centers. As ETs are less common and likely more specific to unrecognized clinical deterioration, they might be the first tracked as a center improves its RRS through QI methods. Alternatively, CDEs may be a useful metric for centers where unrecognized clinical deterioration is less common or in research studies where this more common outcome would lead to more power to detect the effect of interventions to improve care.

Our study had several limitations. Data collection was confined to one tertiary care children’s hospital with a high burden of complex cardiac and oncology care. The results may not generalize well to children hospitalized in smaller or community hospitals or in hospitals without a mature RRS. There is also the possibility of misclassification of covariates and outcomes, but any misclassification would likely be nondifferential and bias toward the null. Matching was not possible based on exact diagnosis, and the unit is a good but imperfect proxy for diagnosis grouping. At our center, overflow of patients into the Cardiology and Hematology/Oncology units is uncommon, mitigating this partially, although residual confounding may remain. The finding that ETs are associated with adverse outcomes does not necessarily mean that these events were preventable; however, it is important and encouraging that the rate of ETs has been reduced at two centers using improvement science interventions.^8,10

Patients who experienced an ET had a significantly higher likelihood of in-hospital mortality, spent more time in the ICU, and had a longer hospital LOS posttransfer than matched controls. The use of the ET metric in children’s hospitals would allow for further analysis of such patients in hopes of identifying clinical characteristics that serve as predictors of deterioration. This may facilitate better risk stratification in the clinical system as well as enable more rapid learning and systems improvements targeted toward preventing unrecognized clinical deterioration.

Disclosures

Dr. Hussain, Dr. Sosa, Dr. Ambroggio, and Mrs. Gallagher have nothing to disclose. Patrick Brady reports grants from the Agency for Healthcare Research and Quality, outside the submitted work. The authors certify that this submission is not under review by any other publication. The author team has no conflicts of interest to disclose.

Funding

Ms. Hussain was supported by the Society of Hospital Medicine’s Student Hospitalist Scholar Grant Program in 2017. Dr. Brady receives career development support from AHRQ K08-HS023827. The project described was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health, under Award Number 5UL1TR001425-04. The content is solely the responsibility of the authors and does not necessarily represent the official views of the SHM, AHRQ, or NIH.

Unrecognized in-hospital deterioration can result in tragic consequences for pediatric patients. The majority of deterioration events have antecedents such as increasingly abnormal vital signs and new concerns from nurses.¹ Recent meta-analyses have shown that rapid response systems (RRSs), which include trigger mechanisms such as a pediatric early warning score (PEWS), are associated with a reduced rate of arrests and in-hospital mortality.^2,3 Cardiopulmonary arrest rates are useful metrics to judge the effectiveness of the system to identify and respond to deteriorating adult patients; however, there are important challenges to their use as an outcome measure in pediatrics. Arrests, which have been relatively uncommon in pediatric patients, are now even less frequent since the adoption of a RRS in the majority of children’s hospitals.^4,5 Several innovations in these systems will be context-dependent and hence best first evaluated in a single center, where arrests outside of the intensive care unit (ICU) may occur rarely. Identification of valid, more frequent proximal measures to arrests may better identify the risk factors for deterioration. This could potentially inform quality improvement efforts to mitigate clinical deterioration.

Bonafide et al. at the Children’s Hospital of Philadelphia developed and validated the critical deterioration event (CDE) metric, demonstrating that children who were transferred to the ICU and who received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of transfer had a >13-fold increased risk of in-hospital mortality.⁶ At Cincinnati Children’s Hospital Medical Center, an additional proximal outcome measure was developed for unrecognized clinical deterioration, now termed emergency transfers (ETs).^7-9 An ET is defined as any patient transferred to the ICU where the patient received intubation, inotropes, or three or more fluid boluses in the first hour after arrival or before transfer.⁹ Improvement science work that aimed at increasing clinician situation awareness was associated with a reduction in ETs,⁸ but the association of ETs with mortality or other healthcare utilization outcomes is unknown. The objective of this study was to determine the predictive validity of an ET on in-hospital mortality, ICU length of stay (LOS), and overall hospital LOS.

We conducted a case–control study at Cincinnati Children’s Hospital, a free-standing tertiary care children’s hospital. Our center has had an ICU-based RRS in place since 2005. In 2009, we eliminated the ICU consult such that each floor-to-ICU transfer is evaluated by the RRS. Nurses calculate a Monaghan PEWS every four hours on the majority of nursing units.

Patients of all ages cared for outside of the ICU at any point in their hospitalization from January 1, 2013, to July 31, 2017, were eligible for inclusion. There were no other exclusion criteria. The ICU included both the pediatric ICU and the cardiac ICU.

Cases

We identified all ET cases from an existing situation awareness database in which each RRS call is entered by the hospital nursing supervisor, whose role includes responding to each RRS activation. If the patient transfer meets the ET criteria, the nurse indicates this in the database. Each ET entry is later confirmed for assurance purposes by the nurse leader of the RRS committee (RG). For the purposes of this study, all records were again reviewed and validated using manual chart review in the electronic health record (Epic Systems, Verona, Wisconsin).

Controls

We identified nonemergent ICU transfers to serve as controls and matched those to ET in cases to limit the impact of confounders that may increase the likelihood of both an ET and a negative outcome such as ICU mortality. We identified up to three controls for each case from our database and matched in terms of age group (within five years of age), hospital unit before transfer, and time of year (within three months of ET). These variables were chosen to adjust for the impact of age, diversity of disease (as hospital units are generally organized by organ system of illness), and seasonality on outcomes.

Outcome Measures

Posttransfer LOS in the ICU, posttransfer hospital LOS, and in-hospital mortality were the primary outcome measures. Patient demographics, specific diagnoses, and number of medical conditions were a priori defined as covariates of interest. Data for each case and control were entered into a secure, web-based Research Electronic Data Capture (REDCap) database.

Analysis

Descriptive data were summarized using counts and percentages for categorical variables and medians and ranges for continuous variables due to nonnormal distributions. Chi-square test was used to compare in-hospital mortality between the ETs and the controls. The Wilcoxon rank-sum test was used to compare LOS between ETs and controls. All data analyses were performed using SAS version 9.4 (SAS Institute Inc., Cary, North Carolina).

A total of 45 ETs were identified, and 110 controls were matched. Patient demographics were similar among all cases and controls (P > .05). Patients with ETs had a median age of seven years (interquartile range: 3-18 years), and 51% of them were males. The majority of patients among our examined cases were white (68%) and non-Hispanic (93%). There was no statistical difference in insurance between the ETs and the controls. When evaluating the hospital unit before the transfer, ETs occurred most commonly in the Cardiology (22%), Hematology/Oncology (22%), and Neuroscience (16%) units.

ETs stayed longer in the ICU than non-ETs [median of 4.9 days vs 2.2 days, P = .001; Figure (A)]. Similarly, ET cases had a significantly longer posttransfer hospital LOS [median of 35 days vs 21 days, P = .001; Figure (B)]. ETs had a 22% in-hospital mortality rate, compared with 9% in-hospital mortality in the matched controls (P = .02; Table).

Children who experienced an ET had a significantly longer ICU LOS, a longer posttransfer LOS, and a higher in-hospital mortality than the matched controls who were also transferred to the ICU. Researchers and improvement science teams at multiple hospitals have demonstrated that interventions targeting improved situation awareness can reduce ETs; we have demonstrated that reducing ETs may reduce subsequent adverse outcomes.^8,10

These findings provide additional support for the use of the ET metric in children’s hospitals as a proximal measure for significant clinical deterioration. We found mortality rates that were overall high for a children’s hospital (22% in ET cases and 9% among controls) compared with a national average mortality rate of 2.3% in pediatric ICUs.¹¹ This is likely due to the study sample containing a significant proportion of children with medical complexity.

Aoki et al. recently demonstrated that ETs, compared with non-ETs, were associated with longer LOS and higher mortality in a bivariate analysis.¹² In our study, we found similar results with the important addition that these findings were robust when ETs were compared with matched controls who were likely at a higher risk of poor outcomes than ICU transfers in general. In addition, we demonstrated that ETs were associated with adverse outcomes in a United States children’s hospital with a mature, long-standing RRS process. As ETs are considerably more common than cardiac and respiratory arrests, use of the ET metric in children’s hospitals may enable more rapid learning and systems improvement implementations. We also found that most of the children with ETs present from units that care for children with substantial medical complexity, including Cardiology, Hematology/Oncology, and Neurosciences. Future work should continue to examine the relationship between medical complexity and ET risk.

The ET metric is complementary to the CDE measure developed by Bonafide et al. Both metrics capture potential events of unrecognized clinical deterioration, and both offer researchers the opportunity to better understand and improve their RRSs. Both ETs and CDEs are more common than arrests, and CDEs are more common than ETs. ETs, which by definition occur in the first hour of ICU care, are likely a more specific measure of unrecognized clinical deterioration. CDEs will capture therapies that may have been started up to 12 hours after transfer and thus are possibly more sensitive to identify unrecognized clinical deterioration. However, CDEs also may encompass some patients who arrived at the ICU after prompt recognition and then had a subacute deterioration in the ICU.

The maturity of the RRS and the bandwidth of teams to collect data may inform which metric(s) are best for individual centers. As ETs are less common and likely more specific to unrecognized clinical deterioration, they might be the first tracked as a center improves its RRS through QI methods. Alternatively, CDEs may be a useful metric for centers where unrecognized clinical deterioration is less common or in research studies where this more common outcome would lead to more power to detect the effect of interventions to improve care.

Our study had several limitations. Data collection was confined to one tertiary care children’s hospital with a high burden of complex cardiac and oncology care. The results may not generalize well to children hospitalized in smaller or community hospitals or in hospitals without a mature RRS. There is also the possibility of misclassification of covariates and outcomes, but any misclassification would likely be nondifferential and bias toward the null. Matching was not possible based on exact diagnosis, and the unit is a good but imperfect proxy for diagnosis grouping. At our center, overflow of patients into the Cardiology and Hematology/Oncology units is uncommon, mitigating this partially, although residual confounding may remain. The finding that ETs are associated with adverse outcomes does not necessarily mean that these events were preventable; however, it is important and encouraging that the rate of ETs has been reduced at two centers using improvement science interventions.^8,10

Patients who experienced an ET had a significantly higher likelihood of in-hospital mortality, spent more time in the ICU, and had a longer hospital LOS posttransfer than matched controls. The use of the ET metric in children’s hospitals would allow for further analysis of such patients in hopes of identifying clinical characteristics that serve as predictors of deterioration. This may facilitate better risk stratification in the clinical system as well as enable more rapid learning and systems improvements targeted toward preventing unrecognized clinical deterioration.

Disclosures

Dr. Hussain, Dr. Sosa, Dr. Ambroggio, and Mrs. Gallagher have nothing to disclose. Patrick Brady reports grants from the Agency for Healthcare Research and Quality, outside the submitted work. The authors certify that this submission is not under review by any other publication. The author team has no conflicts of interest to disclose.

Funding

Ms. Hussain was supported by the Society of Hospital Medicine’s Student Hospitalist Scholar Grant Program in 2017. Dr. Brady receives career development support from AHRQ K08-HS023827. The project described was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health, under Award Number 5UL1TR001425-04. The content is solely the responsibility of the authors and does not necessarily represent the official views of the SHM, AHRQ, or NIH.