Allison M Loechtenfeldt, BS

Readmission rates are used as metrics for care quality and reimbursement, with penalties applied to hospitals with higher than expected rates¹ and up to 30% of pediatric readmissions deemed potentially preventable.² There is a paucity of information on how to prevent pediatric readmissions,³ yet pediatric hospitals are tasked with implementing interventions for readmission reduction.

The Hospital to Home Outcomes (H2O) trial was a 2-arm, randomized controlled trial in which patients discharged from hospital medicine and neuroscience services at a single institution were randomized to receive a single home visit from a registered nurse (RN) within 96 hours of discharge.⁴ RNs completed a structured nurse visit designed specifically for the trial. Lists of “red flags” or warning signs associated with common diagnoses were provided to assist RNs in standardizing education about when to seek additional care. The hypothesis was that the postdischarge visits would result in lower reutilization rates (unplanned readmissions, emergency department [ED] visits, and urgent care visits).⁵

Unexpectedly, children randomized to receive the postdischarge nurse visit had higher rates of 30-day unplanned healthcare reutilization, with children randomly assigned to the intervention demonstrating higher odds of 30-day healthcare use (OR 1.33; 95% CI 1.003-1.76).⁴ We sought to understand perspectives on these unanticipated findings by obtaining input from relevant stakeholders. There were 2 goals for the qualitative analysis: first, to understand possible explanations of the increased reutilization finding; second, to elicit suggestions for improving the nurse visit intervention.

 

METHODS

We selected an in-depth qualitative approach, using interviews and focus groups to explore underlying explanations for the increase in 30-day unplanned healthcare reutilization among those randomized to receive the postdischarge nurse visit during the H2O trial.⁴ Input was sought from 4 stakeholder groups—parents, primary care physicians (PCPs), hospital medicine physicians, and home care RNs—in an effort to triangulate data sources and elicit rich and diverse opinions. Approval was obtained from the Institutional Review Board prior to conducting the study.

Recruitment
Parents

Because we conducted interviews approximately 1 year after the trial’s conclusion, we purposefully selected families who were enrolled in the latter portion of the H2O trial in order to enhance recall. Beginning with the last families in the study, we sequentially contacted families in reverse order. We contacted 10 families in each of 4 categories (intervention/reutilization, intervention/no reutilization, control/reutilization, control/no reutilization). A total of 3 attempts were made by telephone to contact each family. Participants received a grocery store gift card for participating in the study.

Primary Care Physicians

We conducted focus groups with a purposive sample of physicians recruited from 2 community practices and 1 hospital-­owned practice.

Hospital Medicine Physicians

We conducted focus groups with a purposive sample of physicians from our Division of Hospital Medicine. There was a varying level of knowledge of the original trial; however, none of the participants were collaborators in the trial.

Home Care RNs

We conducted focus groups with a subset of RNs who were involved with trial visits. All RNs were members of the pediatric home care division associated with the hospital with specific training in caring for patients at home.

Data Collection

The study team designed question guides for each stakeholder group (Appendix 1). While questions were tailored for specific stakeholders, all guides included the following topics: benefits and challenges of nurse visits, suggestions for improving the intervention in future trials, and reactions to the trial results (once presented to participants). Only the results of the intention-to-treat (ITT) analysis were shared with stakeholders because ITT is considered the gold standard for trial analysis and allows easy understanding of the results.

A single investigator (A.L.) conducted parental interviews by telephone. Focus groups for PCPs, hospital medicine physicians, and RN groups were held at practice locations in private conference rooms and were conducted by trained moderators (S.N.S., A.L., and H.T.C.). Moderators probed responses to the open-ended questions to delve deeply into issues. The question guides were modified in an iterative fashion to include new concepts raised during interviews or focus groups. All interviews and focus groups were recorded and transcribed verbatim with all identifiable information redacted.

Data Analysis

During multiple cycles of inductive thematic analysis,⁶ we examined, discussed, interpreted, and organized responses to the open-ended questions,^6,7 analyzing each stakeholder group separately. First, transcripts were shared with and reviewed by the entire multidisciplinary team (12 members) which included hospital medicine physicians, PCPs, home care nursing leaders, a nurse scientist, a parent representative, research coordinators, and a qualitative research methodologist. Second, team members convened to discuss overall concepts and ideas and created the preliminary coding frameworks. Third, a smaller subgroup (research coordinator [A.L]., hospital medicine physician [S.R.], parent representative [M.M.], and qualitative research methodologist [S.N.S.]), refined the unique coding framework for each stakeholder group and then independently applied codes to participant comments. This subgroup met regularly to reach consensus about the assigned codes and to further refine the codebooks. The codes were organized into major and minor themes based on recurring patterns in the data and the salience or emphasis given by participants. The subgroup’s work was reviewed and discussed on an ongoing basis by the entire multidisciplinary team. Triangulation of the data was achieved in multiple ways. The preliminary results were shared in several forums, and feedback was solicited and incorporated. Two of 4 members of the subgroup analytic team were not part of the trial planning or data collection, providing a potentially broader perspective. All coding decisions were maintained in an electronic database, and an audit trail was created to document codebook revisions.

 

RESULTS

A total of 33 parents participated in the interviews (intervention/readmit [8], intervention/no readmit [8], control/readmit [8], and control/no readmit [9]). Although we selected families from all 4 categories, we were not able to explore qualitative differences between these groups because of the relatively low numbers of participants. Parent data was very limited as interviews were brief and “control” parents had not received the intervention. Three focus groups were held with PCPs (7 participants in total), 2 focus groups were held with hospital medicine physicians (12 participants), and 2 focus groups were held with RNs (10 participants).

Goal 1: Explanation of Reutilization Rates

During interviews and focus groups, the results of the H2O trial were discussed, and stakeholders were asked to comment on potential explanations of the findings. 4 major themes and 5 minor themes emerged from analysis of the transcripts (summarized in Table 1).

Theme 1: Appropriateness of Patient Reutilization

Hospital medicine physicians and home care RNs questioned whether the reutilization events were clinically indicated. RNs wondered whether children who reutilized the ED were also readmitted to the hospital; many perceived that if the child was ill enough to be readmitted, then the ED revisit was warranted (Table 2). Parents commented on parental decision-making and changes in clinical status of the child leading to reutilization (Table 2).

Theme 2: Impact of Red Flags/Warning Sign Instructions on Family’s Reutilization Decisions

Hospital medicine physicians and RNs discussed the potential concern that the “red flags” lacked sufficient context for appropriate family decision making. They hypothesized that, as a result, parents might have returned to the ED rather than accessing other avenues of care such as their primary care office. For example, 1 participant noted that, if a fever recurred days after it had resolved, then perhaps that would require different action steps than if a child had a persistent fever. RNs also mentioned that the discussion of red flags may have made families “more diligent” (Table 2).

Theme 3: Hospital-Affiliated RNs “Directing Traffic” Back to Hospital

Both physician groups were concerned that, because the study was conducted by hospital-employed nurses, families might have been more likely to reaccess care at the hospital. Thus, the connection with the hospital was strengthened in the H2O model, potentially at the expense of the connection with PCPs. Physicians hypothesized that families might “still feel part of the medical system,” so families would return to the hospital if there was a problem. PCPs emphasized that there may have been straightforward situations that could have been handled appropriately in the outpatient office (Table 2).

Theme 4: Home Visit RNs Had a Low Threshold for Escalating Care

Parents and PCPs hypothesized that RNs are more conservative and, therefore, would have had a low threshold to refer back to the hospital if there were concerns in the home. One parent commented: “I guess, nurses are just by trade accustomed to erring on the side of caution and medical intervention instead of letting time take its course. … They’re more apt to say it’s better off to go to the hospital and have everything be fine” (Table 2).

 

Minor Themes

Participants also explained reutilization in ways that coalesced into 5 minor themes: (1) families receiving a visit might perceive that their child was sicker; (2) patients in the control group did not reutilize enough; (3) receiving more education on a child’s illness drives reutilization; (4) provider access issues; and (5) variability of RN experience may determine whether escalated care. Supportive quotations found in Appendix 2.

We directly asked parents if they would want a nurse home visit in the future after discussing the results of the study. Almost all of the parents in the intervention group and most of the parents in the control group were in favor of receiving a visit, even knowing that patients who had received a visit were more likely to reutilize care.

Goal 2: Suggestions for Improving Intervention Design

Three major themes and 3 minor themes were related to improving the design of the intervention (Table 1).

Theme 1: Need for Improved Postdischarge Communication

All stakeholder groups highlighted postdischarge communication as an area that could be improved. Parents were frustrated with regard to attempts to connect with inpatient physicians after discharge. PCPs suggested developing pathways for the RN to connect with the primary care office as opposed to the hospital. Hospital medicine physicians discussed a lack of consensus regarding patient ownership following discharge and were uncertain about what types of postdischarge symptoms PCPs would be comfortable managing. RNs described specific situations when they had difficulty contacting a physician to escalate care (Table 3).

Theme 2: Individualizing Home Visits—One Size Does Not Fit All

All stakeholder groups also encouraged “individualization” of home visits according to patient and family characteristics, diagnosis, and both timing and severity of illness. PCPs recommended visits only for certain diagnoses. Hospital medicine physicians voiced similar sentiments as the PCPs and added that worrisome family dynamics during a hospitalization, such as a lack of engagement with the medical team, might also warrant a visit. RNs suggested visits for those families with more concerns, for example, those with young children or children recovering from an acute respiratory illness (Table 3).

Theme 3: Providing Context for and Framing of Red Flags

Physicians and nurses suggested providing more context to “red flag” instructions and education. RNs emphasized that some families seemed to benefit from the opportunity to discuss their postdischarge concerns with a medical professional. Others appreciated concrete written instructions that spelled out how to respond in certain situations (Table 3).

Minor Themes

Three minor themes were revealed regarding intervention design improvement (Table 1): (1) streamlining the discharge process; (2) improving the definition of the scope and goal of intervention; and (3) extending inpatient team expertise post discharge. Supportive quotations can be found in Appendix 3.

DISCUSSION

When stakeholders were asked about why postdischarge RN visits led to increased postdischarge urgent healthcare visits, they questioned the appropriateness of the reutilization events, wondered about the lack of context for the warning signs that nurses provided families as part of the intervention, worried that families were encouraged to return to the hospital because of the ties of the trial to the hospital, and suggested that RNs had a low threshold to refer patients back to the hospital. When asked about how to design an improved nurse visit to better support families, stakeholders emphasized improving communication, individualizing the visit, and providing context around the red-flag discussion, enabling more nuanced instructions about how to respond to specific events.

 

A synthesis of themes suggests that potential drivers for increased utilization rates may lie in the design and goals of the initial project. The intervention was designed to support families and enhance education after discharge, with components derived from pretrial focus groups with families after a hospital discharge.⁸ The intervention was not designed to divert patients from the ED nor did it enhance access to the PCP. A second trial of the intervention adapted to a phone call also failed to decrease reutilization rates.⁹ Both physician stakeholder groups perceived that the intervention directed traffic back to the hospital because of the intervention design. Coupled with the perception that the red flags may have changed a family’s threshold for seeking care and/or that an RN may be more apt to refer back to care, this failure to push utilization to the primary care office may explain the unexpected trial results. Despite the stakeholders’ perception of enhanced connection back to the hospital as a result of the nurse visit, in analysis of visit referral patterns, a referral was made directly back to the ED in only 4 of the 651 trial visits (Tubbs-Cooley H, Riddle SR, Gold JM, et al.; under review. Pediatric clinical and social concerns identified by home visit nurses in the immediate postdischarge period 2020).

Both H2O trials demonstrated improved recall of red flags by parents who received the intervention, which may be important given the stakeholders’ perspectives that the red flags may not have been contextualized well enough. Yet neither trial demonstrated any differences in postdischarge coping or time to return to normal routine. In interviews with parents, despite the clearly stated results of increased reutilization, intervention parents endorsed a desire for a home visit in the future, raising the possibility that our outcome measures did not capture parents’ priorities adequately.

When asked to recommend design improvements of the intervention, 2 major themes (improvement in communication and individualization of visits) were discussed by all stakeholder groups, providing actionable information to modify or create new interventions. Focus groups with clinicians suggested that communication challenges may have influenced reutilization likelihood during the postdischarge period. RNs expressed uncertainty about who to call with problems or questions at the time of a home visit. This was compounded by difficulty reaching physicians. Both hospital medicine physicians and PCPs identified system challenges including questions of patient ownership, variable PCP practice communication preferences, and difficulty in identifying a partnered staff member (on either end of the inpatient-outpatient continuum) who was familiar with a specific patient. While the communication issues raised may reflect difficulties in our local healthcare system, there is broad evidence of postdischarge communication challenges. In adults, postdischarge communication failures between home health staff and physicians are associated with an increased risk of readmission.¹⁰ The real or perceived lack of communication between inpatient and outpatient providers can add to parental confusion post discharge.¹¹ Although there have been efforts to improve the reliability of communication across this gulf,^12,13 it is not clear whether changes to discharge communication could help to avoid pediatric reutilization events.¹⁴

The theme of individualization of the home nurse visit is consistent with evidence regarding the impact of focusing the intervention on patients with specific diagnoses or demographics. In adults, reduced reutilization associated with postdischarge home nurse visits has been described in specific populations such as patients with heart failure and chronic obstructive pulmonary disease.¹⁵ Impact of home nurse visits on patients within diagnosis-specific populations with certain demographics (such as advanced age) has also been described.¹⁶ In the pediatric population, readmission rates vary widely by diagnosis.¹⁷ A systematic review of interventions to reduce pediatric readmissions found increased impact of discharge interventions in specific populations (asthma, oncology, and NICU).³

Next steps may lie in interventions in targeted populations that function as part of a care continuum bridging the patient from the inpatient to the outpatient setting. A home nurse visit as part of this discharge structure may prove to have more impact on reducing reutilization. One population which accounts for a large proportion of readmissions and where there has been recent focus on discharge transition of care has been children with medical complexity.¹⁸ This group was largely excluded from the H2O trial. Postdischarge home nurse visits in this population have been found to be feasible and address many questions and problems, but the effect on readmission is less clear.¹⁹ Family priorities and preferences related to preparation for discharge, including family engagement, respect for discharge readiness, and goal of returning to normal routines, may be areas on which to focus with future interventions in this population.²⁰ In summary, although widespread postdischarge interventions (home nurse visit⁴ and nurse telephone call⁹) have not been found to be effective, targeting interventions to specific populations by diagnosis or demographic factors may prove to be more effective in reducing pediatric reutilization.

There were several strengths to this study. This qualitative approach allowed us to elucidate potential explanations for the H2O trial results from multiple perspectives. The multidisciplinary composition of our analytic team and the use of an iterative process sparked diverse contributions in a dynamic, ongoing discussion and interpretation of our data.

This study should be considered in the context of several limitations. For families and RNs, there was a time lag between participation in the trial and participation in the qualitative study call or focus group which could lead to difficulty recalling details. Only families who received the intervention could give opinions on their experience of the nurse visit, while families in the control group were asked to hypothesize. Focus groups with hospital medicine physicians and PCPs were purposive samples, and complete demographic information of participants was not collected.

 

CONCLUSION

Key stakeholders reflecting on a postdischarge RN visit trial suggested multiple potential explanations for the unexpected increase in reutilization in children randomized to the intervention. Certain participants questioned whether all reutilization events were appropriate or necessary. Others expressed concerns that the H2O intervention lacked context and directed children back to the hospital instead of the PCP. Parents, PCPs, hospital medicine physicians, and RNs all suggested that future transition-focused interventions should enhance postdischarge communication, strengthen connection to the PCP, and be more effectively tailored to the needs of the individual patient and family.

Acknowledgments

Collaborators: H2O Trial Study Group: Joanne Bachus, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Monica L Borell, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Lenisa V Chang, MA, PhD; Patricia Crawford, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Sarah A Ferris, BA, Division of Hospital Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Judy A Heilman BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Jane C Khoury, PhD, Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Karen Lawley, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Lynne O’Donnell, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Hadley S Sauers-Ford, MPH, Department of Pediatrics, UC Davis Health, Sacramento, California; Anita N Shah, DO, MPH, Division of Hospital Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Lauren G Solan, MD, Med, University of Rochester, Rochester, New York; Heidi J Sucharew, PhD, Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Karen P Sullivan, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Christine M White, MD, MAT, Division of Hospital Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio.

Readmission rates are used as metrics for care quality and reimbursement, with penalties applied to hospitals with higher than expected rates¹ and up to 30% of pediatric readmissions deemed potentially preventable.² There is a paucity of information on how to prevent pediatric readmissions,³ yet pediatric hospitals are tasked with implementing interventions for readmission reduction.

The Hospital to Home Outcomes (H2O) trial was a 2-arm, randomized controlled trial in which patients discharged from hospital medicine and neuroscience services at a single institution were randomized to receive a single home visit from a registered nurse (RN) within 96 hours of discharge.⁴ RNs completed a structured nurse visit designed specifically for the trial. Lists of “red flags” or warning signs associated with common diagnoses were provided to assist RNs in standardizing education about when to seek additional care. The hypothesis was that the postdischarge visits would result in lower reutilization rates (unplanned readmissions, emergency department [ED] visits, and urgent care visits).⁵

Unexpectedly, children randomized to receive the postdischarge nurse visit had higher rates of 30-day unplanned healthcare reutilization, with children randomly assigned to the intervention demonstrating higher odds of 30-day healthcare use (OR 1.33; 95% CI 1.003-1.76).⁴ We sought to understand perspectives on these unanticipated findings by obtaining input from relevant stakeholders. There were 2 goals for the qualitative analysis: first, to understand possible explanations of the increased reutilization finding; second, to elicit suggestions for improving the nurse visit intervention.

 

METHODS

We selected an in-depth qualitative approach, using interviews and focus groups to explore underlying explanations for the increase in 30-day unplanned healthcare reutilization among those randomized to receive the postdischarge nurse visit during the H2O trial.⁴ Input was sought from 4 stakeholder groups—parents, primary care physicians (PCPs), hospital medicine physicians, and home care RNs—in an effort to triangulate data sources and elicit rich and diverse opinions. Approval was obtained from the Institutional Review Board prior to conducting the study.

Recruitment
Parents

Because we conducted interviews approximately 1 year after the trial’s conclusion, we purposefully selected families who were enrolled in the latter portion of the H2O trial in order to enhance recall. Beginning with the last families in the study, we sequentially contacted families in reverse order. We contacted 10 families in each of 4 categories (intervention/reutilization, intervention/no reutilization, control/reutilization, control/no reutilization). A total of 3 attempts were made by telephone to contact each family. Participants received a grocery store gift card for participating in the study.

Primary Care Physicians

We conducted focus groups with a purposive sample of physicians recruited from 2 community practices and 1 hospital-­owned practice.

Hospital Medicine Physicians

We conducted focus groups with a purposive sample of physicians from our Division of Hospital Medicine. There was a varying level of knowledge of the original trial; however, none of the participants were collaborators in the trial.

Home Care RNs

We conducted focus groups with a subset of RNs who were involved with trial visits. All RNs were members of the pediatric home care division associated with the hospital with specific training in caring for patients at home.

Data Collection

The study team designed question guides for each stakeholder group (Appendix 1). While questions were tailored for specific stakeholders, all guides included the following topics: benefits and challenges of nurse visits, suggestions for improving the intervention in future trials, and reactions to the trial results (once presented to participants). Only the results of the intention-to-treat (ITT) analysis were shared with stakeholders because ITT is considered the gold standard for trial analysis and allows easy understanding of the results.

A single investigator (A.L.) conducted parental interviews by telephone. Focus groups for PCPs, hospital medicine physicians, and RN groups were held at practice locations in private conference rooms and were conducted by trained moderators (S.N.S., A.L., and H.T.C.). Moderators probed responses to the open-ended questions to delve deeply into issues. The question guides were modified in an iterative fashion to include new concepts raised during interviews or focus groups. All interviews and focus groups were recorded and transcribed verbatim with all identifiable information redacted.

Data Analysis

During multiple cycles of inductive thematic analysis,⁶ we examined, discussed, interpreted, and organized responses to the open-ended questions,^6,7 analyzing each stakeholder group separately. First, transcripts were shared with and reviewed by the entire multidisciplinary team (12 members) which included hospital medicine physicians, PCPs, home care nursing leaders, a nurse scientist, a parent representative, research coordinators, and a qualitative research methodologist. Second, team members convened to discuss overall concepts and ideas and created the preliminary coding frameworks. Third, a smaller subgroup (research coordinator [A.L]., hospital medicine physician [S.R.], parent representative [M.M.], and qualitative research methodologist [S.N.S.]), refined the unique coding framework for each stakeholder group and then independently applied codes to participant comments. This subgroup met regularly to reach consensus about the assigned codes and to further refine the codebooks. The codes were organized into major and minor themes based on recurring patterns in the data and the salience or emphasis given by participants. The subgroup’s work was reviewed and discussed on an ongoing basis by the entire multidisciplinary team. Triangulation of the data was achieved in multiple ways. The preliminary results were shared in several forums, and feedback was solicited and incorporated. Two of 4 members of the subgroup analytic team were not part of the trial planning or data collection, providing a potentially broader perspective. All coding decisions were maintained in an electronic database, and an audit trail was created to document codebook revisions.

 

RESULTS

A total of 33 parents participated in the interviews (intervention/readmit [8], intervention/no readmit [8], control/readmit [8], and control/no readmit [9]). Although we selected families from all 4 categories, we were not able to explore qualitative differences between these groups because of the relatively low numbers of participants. Parent data was very limited as interviews were brief and “control” parents had not received the intervention. Three focus groups were held with PCPs (7 participants in total), 2 focus groups were held with hospital medicine physicians (12 participants), and 2 focus groups were held with RNs (10 participants).

Goal 1: Explanation of Reutilization Rates

During interviews and focus groups, the results of the H2O trial were discussed, and stakeholders were asked to comment on potential explanations of the findings. 4 major themes and 5 minor themes emerged from analysis of the transcripts (summarized in Table 1).

Theme 1: Appropriateness of Patient Reutilization

Hospital medicine physicians and home care RNs questioned whether the reutilization events were clinically indicated. RNs wondered whether children who reutilized the ED were also readmitted to the hospital; many perceived that if the child was ill enough to be readmitted, then the ED revisit was warranted (Table 2). Parents commented on parental decision-making and changes in clinical status of the child leading to reutilization (Table 2).

Theme 2: Impact of Red Flags/Warning Sign Instructions on Family’s Reutilization Decisions

Hospital medicine physicians and RNs discussed the potential concern that the “red flags” lacked sufficient context for appropriate family decision making. They hypothesized that, as a result, parents might have returned to the ED rather than accessing other avenues of care such as their primary care office. For example, 1 participant noted that, if a fever recurred days after it had resolved, then perhaps that would require different action steps than if a child had a persistent fever. RNs also mentioned that the discussion of red flags may have made families “more diligent” (Table 2).

Theme 3: Hospital-Affiliated RNs “Directing Traffic” Back to Hospital

Both physician groups were concerned that, because the study was conducted by hospital-employed nurses, families might have been more likely to reaccess care at the hospital. Thus, the connection with the hospital was strengthened in the H2O model, potentially at the expense of the connection with PCPs. Physicians hypothesized that families might “still feel part of the medical system,” so families would return to the hospital if there was a problem. PCPs emphasized that there may have been straightforward situations that could have been handled appropriately in the outpatient office (Table 2).

Theme 4: Home Visit RNs Had a Low Threshold for Escalating Care

Parents and PCPs hypothesized that RNs are more conservative and, therefore, would have had a low threshold to refer back to the hospital if there were concerns in the home. One parent commented: “I guess, nurses are just by trade accustomed to erring on the side of caution and medical intervention instead of letting time take its course. … They’re more apt to say it’s better off to go to the hospital and have everything be fine” (Table 2).

 

Minor Themes

Participants also explained reutilization in ways that coalesced into 5 minor themes: (1) families receiving a visit might perceive that their child was sicker; (2) patients in the control group did not reutilize enough; (3) receiving more education on a child’s illness drives reutilization; (4) provider access issues; and (5) variability of RN experience may determine whether escalated care. Supportive quotations found in Appendix 2.

We directly asked parents if they would want a nurse home visit in the future after discussing the results of the study. Almost all of the parents in the intervention group and most of the parents in the control group were in favor of receiving a visit, even knowing that patients who had received a visit were more likely to reutilize care.

Goal 2: Suggestions for Improving Intervention Design

Three major themes and 3 minor themes were related to improving the design of the intervention (Table 1).

Theme 1: Need for Improved Postdischarge Communication

All stakeholder groups highlighted postdischarge communication as an area that could be improved. Parents were frustrated with regard to attempts to connect with inpatient physicians after discharge. PCPs suggested developing pathways for the RN to connect with the primary care office as opposed to the hospital. Hospital medicine physicians discussed a lack of consensus regarding patient ownership following discharge and were uncertain about what types of postdischarge symptoms PCPs would be comfortable managing. RNs described specific situations when they had difficulty contacting a physician to escalate care (Table 3).

Theme 2: Individualizing Home Visits—One Size Does Not Fit All

All stakeholder groups also encouraged “individualization” of home visits according to patient and family characteristics, diagnosis, and both timing and severity of illness. PCPs recommended visits only for certain diagnoses. Hospital medicine physicians voiced similar sentiments as the PCPs and added that worrisome family dynamics during a hospitalization, such as a lack of engagement with the medical team, might also warrant a visit. RNs suggested visits for those families with more concerns, for example, those with young children or children recovering from an acute respiratory illness (Table 3).

Theme 3: Providing Context for and Framing of Red Flags

Physicians and nurses suggested providing more context to “red flag” instructions and education. RNs emphasized that some families seemed to benefit from the opportunity to discuss their postdischarge concerns with a medical professional. Others appreciated concrete written instructions that spelled out how to respond in certain situations (Table 3).

Minor Themes

Three minor themes were revealed regarding intervention design improvement (Table 1): (1) streamlining the discharge process; (2) improving the definition of the scope and goal of intervention; and (3) extending inpatient team expertise post discharge. Supportive quotations can be found in Appendix 3.

DISCUSSION

When stakeholders were asked about why postdischarge RN visits led to increased postdischarge urgent healthcare visits, they questioned the appropriateness of the reutilization events, wondered about the lack of context for the warning signs that nurses provided families as part of the intervention, worried that families were encouraged to return to the hospital because of the ties of the trial to the hospital, and suggested that RNs had a low threshold to refer patients back to the hospital. When asked about how to design an improved nurse visit to better support families, stakeholders emphasized improving communication, individualizing the visit, and providing context around the red-flag discussion, enabling more nuanced instructions about how to respond to specific events.

 

A synthesis of themes suggests that potential drivers for increased utilization rates may lie in the design and goals of the initial project. The intervention was designed to support families and enhance education after discharge, with components derived from pretrial focus groups with families after a hospital discharge.⁸ The intervention was not designed to divert patients from the ED nor did it enhance access to the PCP. A second trial of the intervention adapted to a phone call also failed to decrease reutilization rates.⁹ Both physician stakeholder groups perceived that the intervention directed traffic back to the hospital because of the intervention design. Coupled with the perception that the red flags may have changed a family’s threshold for seeking care and/or that an RN may be more apt to refer back to care, this failure to push utilization to the primary care office may explain the unexpected trial results. Despite the stakeholders’ perception of enhanced connection back to the hospital as a result of the nurse visit, in analysis of visit referral patterns, a referral was made directly back to the ED in only 4 of the 651 trial visits (Tubbs-Cooley H, Riddle SR, Gold JM, et al.; under review. Pediatric clinical and social concerns identified by home visit nurses in the immediate postdischarge period 2020).

Both H2O trials demonstrated improved recall of red flags by parents who received the intervention, which may be important given the stakeholders’ perspectives that the red flags may not have been contextualized well enough. Yet neither trial demonstrated any differences in postdischarge coping or time to return to normal routine. In interviews with parents, despite the clearly stated results of increased reutilization, intervention parents endorsed a desire for a home visit in the future, raising the possibility that our outcome measures did not capture parents’ priorities adequately.

When asked to recommend design improvements of the intervention, 2 major themes (improvement in communication and individualization of visits) were discussed by all stakeholder groups, providing actionable information to modify or create new interventions. Focus groups with clinicians suggested that communication challenges may have influenced reutilization likelihood during the postdischarge period. RNs expressed uncertainty about who to call with problems or questions at the time of a home visit. This was compounded by difficulty reaching physicians. Both hospital medicine physicians and PCPs identified system challenges including questions of patient ownership, variable PCP practice communication preferences, and difficulty in identifying a partnered staff member (on either end of the inpatient-outpatient continuum) who was familiar with a specific patient. While the communication issues raised may reflect difficulties in our local healthcare system, there is broad evidence of postdischarge communication challenges. In adults, postdischarge communication failures between home health staff and physicians are associated with an increased risk of readmission.¹⁰ The real or perceived lack of communication between inpatient and outpatient providers can add to parental confusion post discharge.¹¹ Although there have been efforts to improve the reliability of communication across this gulf,^12,13 it is not clear whether changes to discharge communication could help to avoid pediatric reutilization events.¹⁴

The theme of individualization of the home nurse visit is consistent with evidence regarding the impact of focusing the intervention on patients with specific diagnoses or demographics. In adults, reduced reutilization associated with postdischarge home nurse visits has been described in specific populations such as patients with heart failure and chronic obstructive pulmonary disease.¹⁵ Impact of home nurse visits on patients within diagnosis-specific populations with certain demographics (such as advanced age) has also been described.¹⁶ In the pediatric population, readmission rates vary widely by diagnosis.¹⁷ A systematic review of interventions to reduce pediatric readmissions found increased impact of discharge interventions in specific populations (asthma, oncology, and NICU).³

Next steps may lie in interventions in targeted populations that function as part of a care continuum bridging the patient from the inpatient to the outpatient setting. A home nurse visit as part of this discharge structure may prove to have more impact on reducing reutilization. One population which accounts for a large proportion of readmissions and where there has been recent focus on discharge transition of care has been children with medical complexity.¹⁸ This group was largely excluded from the H2O trial. Postdischarge home nurse visits in this population have been found to be feasible and address many questions and problems, but the effect on readmission is less clear.¹⁹ Family priorities and preferences related to preparation for discharge, including family engagement, respect for discharge readiness, and goal of returning to normal routines, may be areas on which to focus with future interventions in this population.²⁰ In summary, although widespread postdischarge interventions (home nurse visit⁴ and nurse telephone call⁹) have not been found to be effective, targeting interventions to specific populations by diagnosis or demographic factors may prove to be more effective in reducing pediatric reutilization.

There were several strengths to this study. This qualitative approach allowed us to elucidate potential explanations for the H2O trial results from multiple perspectives. The multidisciplinary composition of our analytic team and the use of an iterative process sparked diverse contributions in a dynamic, ongoing discussion and interpretation of our data.

This study should be considered in the context of several limitations. For families and RNs, there was a time lag between participation in the trial and participation in the qualitative study call or focus group which could lead to difficulty recalling details. Only families who received the intervention could give opinions on their experience of the nurse visit, while families in the control group were asked to hypothesize. Focus groups with hospital medicine physicians and PCPs were purposive samples, and complete demographic information of participants was not collected.

 

CONCLUSION

Key stakeholders reflecting on a postdischarge RN visit trial suggested multiple potential explanations for the unexpected increase in reutilization in children randomized to the intervention. Certain participants questioned whether all reutilization events were appropriate or necessary. Others expressed concerns that the H2O intervention lacked context and directed children back to the hospital instead of the PCP. Parents, PCPs, hospital medicine physicians, and RNs all suggested that future transition-focused interventions should enhance postdischarge communication, strengthen connection to the PCP, and be more effectively tailored to the needs of the individual patient and family.

Acknowledgments

Collaborators: H2O Trial Study Group: Joanne Bachus, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Monica L Borell, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Lenisa V Chang, MA, PhD; Patricia Crawford, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Sarah A Ferris, BA, Division of Hospital Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Judy A Heilman BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Jane C Khoury, PhD, Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Karen Lawley, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Lynne O’Donnell, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Hadley S Sauers-Ford, MPH, Department of Pediatrics, UC Davis Health, Sacramento, California; Anita N Shah, DO, MPH, Division of Hospital Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Lauren G Solan, MD, Med, University of Rochester, Rochester, New York; Heidi J Sucharew, PhD, Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Karen P Sullivan, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Christine M White, MD, MAT, Division of Hospital Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio.

Readmission rates are used as metrics for care quality and reimbursement, with penalties applied to hospitals with higher than expected rates¹ and up to 30% of pediatric readmissions deemed potentially preventable.² There is a paucity of information on how to prevent pediatric readmissions,³ yet pediatric hospitals are tasked with implementing interventions for readmission reduction.

The Hospital to Home Outcomes (H2O) trial was a 2-arm, randomized controlled trial in which patients discharged from hospital medicine and neuroscience services at a single institution were randomized to receive a single home visit from a registered nurse (RN) within 96 hours of discharge.⁴ RNs completed a structured nurse visit designed specifically for the trial. Lists of “red flags” or warning signs associated with common diagnoses were provided to assist RNs in standardizing education about when to seek additional care. The hypothesis was that the postdischarge visits would result in lower reutilization rates (unplanned readmissions, emergency department [ED] visits, and urgent care visits).⁵

Unexpectedly, children randomized to receive the postdischarge nurse visit had higher rates of 30-day unplanned healthcare reutilization, with children randomly assigned to the intervention demonstrating higher odds of 30-day healthcare use (OR 1.33; 95% CI 1.003-1.76).⁴ We sought to understand perspectives on these unanticipated findings by obtaining input from relevant stakeholders. There were 2 goals for the qualitative analysis: first, to understand possible explanations of the increased reutilization finding; second, to elicit suggestions for improving the nurse visit intervention.

 

METHODS

We selected an in-depth qualitative approach, using interviews and focus groups to explore underlying explanations for the increase in 30-day unplanned healthcare reutilization among those randomized to receive the postdischarge nurse visit during the H2O trial.⁴ Input was sought from 4 stakeholder groups—parents, primary care physicians (PCPs), hospital medicine physicians, and home care RNs—in an effort to triangulate data sources and elicit rich and diverse opinions. Approval was obtained from the Institutional Review Board prior to conducting the study.

Recruitment
Parents

Because we conducted interviews approximately 1 year after the trial’s conclusion, we purposefully selected families who were enrolled in the latter portion of the H2O trial in order to enhance recall. Beginning with the last families in the study, we sequentially contacted families in reverse order. We contacted 10 families in each of 4 categories (intervention/reutilization, intervention/no reutilization, control/reutilization, control/no reutilization). A total of 3 attempts were made by telephone to contact each family. Participants received a grocery store gift card for participating in the study.

Primary Care Physicians

We conducted focus groups with a purposive sample of physicians recruited from 2 community practices and 1 hospital-­owned practice.

Hospital Medicine Physicians

We conducted focus groups with a purposive sample of physicians from our Division of Hospital Medicine. There was a varying level of knowledge of the original trial; however, none of the participants were collaborators in the trial.

Home Care RNs

We conducted focus groups with a subset of RNs who were involved with trial visits. All RNs were members of the pediatric home care division associated with the hospital with specific training in caring for patients at home.

Data Collection

The study team designed question guides for each stakeholder group (Appendix 1). While questions were tailored for specific stakeholders, all guides included the following topics: benefits and challenges of nurse visits, suggestions for improving the intervention in future trials, and reactions to the trial results (once presented to participants). Only the results of the intention-to-treat (ITT) analysis were shared with stakeholders because ITT is considered the gold standard for trial analysis and allows easy understanding of the results.

A single investigator (A.L.) conducted parental interviews by telephone. Focus groups for PCPs, hospital medicine physicians, and RN groups were held at practice locations in private conference rooms and were conducted by trained moderators (S.N.S., A.L., and H.T.C.). Moderators probed responses to the open-ended questions to delve deeply into issues. The question guides were modified in an iterative fashion to include new concepts raised during interviews or focus groups. All interviews and focus groups were recorded and transcribed verbatim with all identifiable information redacted.

Data Analysis

During multiple cycles of inductive thematic analysis,⁶ we examined, discussed, interpreted, and organized responses to the open-ended questions,^6,7 analyzing each stakeholder group separately. First, transcripts were shared with and reviewed by the entire multidisciplinary team (12 members) which included hospital medicine physicians, PCPs, home care nursing leaders, a nurse scientist, a parent representative, research coordinators, and a qualitative research methodologist. Second, team members convened to discuss overall concepts and ideas and created the preliminary coding frameworks. Third, a smaller subgroup (research coordinator [A.L]., hospital medicine physician [S.R.], parent representative [M.M.], and qualitative research methodologist [S.N.S.]), refined the unique coding framework for each stakeholder group and then independently applied codes to participant comments. This subgroup met regularly to reach consensus about the assigned codes and to further refine the codebooks. The codes were organized into major and minor themes based on recurring patterns in the data and the salience or emphasis given by participants. The subgroup’s work was reviewed and discussed on an ongoing basis by the entire multidisciplinary team. Triangulation of the data was achieved in multiple ways. The preliminary results were shared in several forums, and feedback was solicited and incorporated. Two of 4 members of the subgroup analytic team were not part of the trial planning or data collection, providing a potentially broader perspective. All coding decisions were maintained in an electronic database, and an audit trail was created to document codebook revisions.

 

RESULTS

A total of 33 parents participated in the interviews (intervention/readmit [8], intervention/no readmit [8], control/readmit [8], and control/no readmit [9]). Although we selected families from all 4 categories, we were not able to explore qualitative differences between these groups because of the relatively low numbers of participants. Parent data was very limited as interviews were brief and “control” parents had not received the intervention. Three focus groups were held with PCPs (7 participants in total), 2 focus groups were held with hospital medicine physicians (12 participants), and 2 focus groups were held with RNs (10 participants).

Goal 1: Explanation of Reutilization Rates

During interviews and focus groups, the results of the H2O trial were discussed, and stakeholders were asked to comment on potential explanations of the findings. 4 major themes and 5 minor themes emerged from analysis of the transcripts (summarized in Table 1).

Theme 1: Appropriateness of Patient Reutilization

Hospital medicine physicians and home care RNs questioned whether the reutilization events were clinically indicated. RNs wondered whether children who reutilized the ED were also readmitted to the hospital; many perceived that if the child was ill enough to be readmitted, then the ED revisit was warranted (Table 2). Parents commented on parental decision-making and changes in clinical status of the child leading to reutilization (Table 2).

Theme 2: Impact of Red Flags/Warning Sign Instructions on Family’s Reutilization Decisions

Hospital medicine physicians and RNs discussed the potential concern that the “red flags” lacked sufficient context for appropriate family decision making. They hypothesized that, as a result, parents might have returned to the ED rather than accessing other avenues of care such as their primary care office. For example, 1 participant noted that, if a fever recurred days after it had resolved, then perhaps that would require different action steps than if a child had a persistent fever. RNs also mentioned that the discussion of red flags may have made families “more diligent” (Table 2).

Theme 3: Hospital-Affiliated RNs “Directing Traffic” Back to Hospital

Both physician groups were concerned that, because the study was conducted by hospital-employed nurses, families might have been more likely to reaccess care at the hospital. Thus, the connection with the hospital was strengthened in the H2O model, potentially at the expense of the connection with PCPs. Physicians hypothesized that families might “still feel part of the medical system,” so families would return to the hospital if there was a problem. PCPs emphasized that there may have been straightforward situations that could have been handled appropriately in the outpatient office (Table 2).

Theme 4: Home Visit RNs Had a Low Threshold for Escalating Care

Parents and PCPs hypothesized that RNs are more conservative and, therefore, would have had a low threshold to refer back to the hospital if there were concerns in the home. One parent commented: “I guess, nurses are just by trade accustomed to erring on the side of caution and medical intervention instead of letting time take its course. … They’re more apt to say it’s better off to go to the hospital and have everything be fine” (Table 2).

 

Minor Themes

Participants also explained reutilization in ways that coalesced into 5 minor themes: (1) families receiving a visit might perceive that their child was sicker; (2) patients in the control group did not reutilize enough; (3) receiving more education on a child’s illness drives reutilization; (4) provider access issues; and (5) variability of RN experience may determine whether escalated care. Supportive quotations found in Appendix 2.

We directly asked parents if they would want a nurse home visit in the future after discussing the results of the study. Almost all of the parents in the intervention group and most of the parents in the control group were in favor of receiving a visit, even knowing that patients who had received a visit were more likely to reutilize care.

Goal 2: Suggestions for Improving Intervention Design

Three major themes and 3 minor themes were related to improving the design of the intervention (Table 1).

Theme 1: Need for Improved Postdischarge Communication

All stakeholder groups highlighted postdischarge communication as an area that could be improved. Parents were frustrated with regard to attempts to connect with inpatient physicians after discharge. PCPs suggested developing pathways for the RN to connect with the primary care office as opposed to the hospital. Hospital medicine physicians discussed a lack of consensus regarding patient ownership following discharge and were uncertain about what types of postdischarge symptoms PCPs would be comfortable managing. RNs described specific situations when they had difficulty contacting a physician to escalate care (Table 3).

Theme 2: Individualizing Home Visits—One Size Does Not Fit All

All stakeholder groups also encouraged “individualization” of home visits according to patient and family characteristics, diagnosis, and both timing and severity of illness. PCPs recommended visits only for certain diagnoses. Hospital medicine physicians voiced similar sentiments as the PCPs and added that worrisome family dynamics during a hospitalization, such as a lack of engagement with the medical team, might also warrant a visit. RNs suggested visits for those families with more concerns, for example, those with young children or children recovering from an acute respiratory illness (Table 3).

Theme 3: Providing Context for and Framing of Red Flags

Physicians and nurses suggested providing more context to “red flag” instructions and education. RNs emphasized that some families seemed to benefit from the opportunity to discuss their postdischarge concerns with a medical professional. Others appreciated concrete written instructions that spelled out how to respond in certain situations (Table 3).

Minor Themes

Three minor themes were revealed regarding intervention design improvement (Table 1): (1) streamlining the discharge process; (2) improving the definition of the scope and goal of intervention; and (3) extending inpatient team expertise post discharge. Supportive quotations can be found in Appendix 3.

DISCUSSION

When stakeholders were asked about why postdischarge RN visits led to increased postdischarge urgent healthcare visits, they questioned the appropriateness of the reutilization events, wondered about the lack of context for the warning signs that nurses provided families as part of the intervention, worried that families were encouraged to return to the hospital because of the ties of the trial to the hospital, and suggested that RNs had a low threshold to refer patients back to the hospital. When asked about how to design an improved nurse visit to better support families, stakeholders emphasized improving communication, individualizing the visit, and providing context around the red-flag discussion, enabling more nuanced instructions about how to respond to specific events.

 

A synthesis of themes suggests that potential drivers for increased utilization rates may lie in the design and goals of the initial project. The intervention was designed to support families and enhance education after discharge, with components derived from pretrial focus groups with families after a hospital discharge.⁸ The intervention was not designed to divert patients from the ED nor did it enhance access to the PCP. A second trial of the intervention adapted to a phone call also failed to decrease reutilization rates.⁹ Both physician stakeholder groups perceived that the intervention directed traffic back to the hospital because of the intervention design. Coupled with the perception that the red flags may have changed a family’s threshold for seeking care and/or that an RN may be more apt to refer back to care, this failure to push utilization to the primary care office may explain the unexpected trial results. Despite the stakeholders’ perception of enhanced connection back to the hospital as a result of the nurse visit, in analysis of visit referral patterns, a referral was made directly back to the ED in only 4 of the 651 trial visits (Tubbs-Cooley H, Riddle SR, Gold JM, et al.; under review. Pediatric clinical and social concerns identified by home visit nurses in the immediate postdischarge period 2020).

Both H2O trials demonstrated improved recall of red flags by parents who received the intervention, which may be important given the stakeholders’ perspectives that the red flags may not have been contextualized well enough. Yet neither trial demonstrated any differences in postdischarge coping or time to return to normal routine. In interviews with parents, despite the clearly stated results of increased reutilization, intervention parents endorsed a desire for a home visit in the future, raising the possibility that our outcome measures did not capture parents’ priorities adequately.

When asked to recommend design improvements of the intervention, 2 major themes (improvement in communication and individualization of visits) were discussed by all stakeholder groups, providing actionable information to modify or create new interventions. Focus groups with clinicians suggested that communication challenges may have influenced reutilization likelihood during the postdischarge period. RNs expressed uncertainty about who to call with problems or questions at the time of a home visit. This was compounded by difficulty reaching physicians. Both hospital medicine physicians and PCPs identified system challenges including questions of patient ownership, variable PCP practice communication preferences, and difficulty in identifying a partnered staff member (on either end of the inpatient-outpatient continuum) who was familiar with a specific patient. While the communication issues raised may reflect difficulties in our local healthcare system, there is broad evidence of postdischarge communication challenges. In adults, postdischarge communication failures between home health staff and physicians are associated with an increased risk of readmission.¹⁰ The real or perceived lack of communication between inpatient and outpatient providers can add to parental confusion post discharge.¹¹ Although there have been efforts to improve the reliability of communication across this gulf,^12,13 it is not clear whether changes to discharge communication could help to avoid pediatric reutilization events.¹⁴

The theme of individualization of the home nurse visit is consistent with evidence regarding the impact of focusing the intervention on patients with specific diagnoses or demographics. In adults, reduced reutilization associated with postdischarge home nurse visits has been described in specific populations such as patients with heart failure and chronic obstructive pulmonary disease.¹⁵ Impact of home nurse visits on patients within diagnosis-specific populations with certain demographics (such as advanced age) has also been described.¹⁶ In the pediatric population, readmission rates vary widely by diagnosis.¹⁷ A systematic review of interventions to reduce pediatric readmissions found increased impact of discharge interventions in specific populations (asthma, oncology, and NICU).³

Next steps may lie in interventions in targeted populations that function as part of a care continuum bridging the patient from the inpatient to the outpatient setting. A home nurse visit as part of this discharge structure may prove to have more impact on reducing reutilization. One population which accounts for a large proportion of readmissions and where there has been recent focus on discharge transition of care has been children with medical complexity.¹⁸ This group was largely excluded from the H2O trial. Postdischarge home nurse visits in this population have been found to be feasible and address many questions and problems, but the effect on readmission is less clear.¹⁹ Family priorities and preferences related to preparation for discharge, including family engagement, respect for discharge readiness, and goal of returning to normal routines, may be areas on which to focus with future interventions in this population.²⁰ In summary, although widespread postdischarge interventions (home nurse visit⁴ and nurse telephone call⁹) have not been found to be effective, targeting interventions to specific populations by diagnosis or demographic factors may prove to be more effective in reducing pediatric reutilization.

There were several strengths to this study. This qualitative approach allowed us to elucidate potential explanations for the H2O trial results from multiple perspectives. The multidisciplinary composition of our analytic team and the use of an iterative process sparked diverse contributions in a dynamic, ongoing discussion and interpretation of our data.

This study should be considered in the context of several limitations. For families and RNs, there was a time lag between participation in the trial and participation in the qualitative study call or focus group which could lead to difficulty recalling details. Only families who received the intervention could give opinions on their experience of the nurse visit, while families in the control group were asked to hypothesize. Focus groups with hospital medicine physicians and PCPs were purposive samples, and complete demographic information of participants was not collected.

 

CONCLUSION

Key stakeholders reflecting on a postdischarge RN visit trial suggested multiple potential explanations for the unexpected increase in reutilization in children randomized to the intervention. Certain participants questioned whether all reutilization events were appropriate or necessary. Others expressed concerns that the H2O intervention lacked context and directed children back to the hospital instead of the PCP. Parents, PCPs, hospital medicine physicians, and RNs all suggested that future transition-focused interventions should enhance postdischarge communication, strengthen connection to the PCP, and be more effectively tailored to the needs of the individual patient and family.

Acknowledgments

Collaborators: H2O Trial Study Group: Joanne Bachus, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Monica L Borell, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Lenisa V Chang, MA, PhD; Patricia Crawford, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Sarah A Ferris, BA, Division of Hospital Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Judy A Heilman BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Jane C Khoury, PhD, Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Karen Lawley, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Lynne O’Donnell, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Hadley S Sauers-Ford, MPH, Department of Pediatrics, UC Davis Health, Sacramento, California; Anita N Shah, DO, MPH, Division of Hospital Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Lauren G Solan, MD, Med, University of Rochester, Rochester, New York; Heidi J Sucharew, PhD, Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Karen P Sullivan, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Christine M White, MD, MAT, Division of Hospital Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio.

Alarms from bedside continuous physiologic monitors (CPMs) occur frequently in children’s hospitals and can lead to harm. Recent studies conducted in children’s hospitals have identified alarm rates of up to 152 alarms per patient per day outside of the intensive care unit,^1-3 with as few as 1% of alarms being considered clinically important.⁴ Excessive alarms have been linked to alarm fatigue, when providers become desensitized to and may miss alarms indicating impending patient deterioration. Alarm fatigue has been identified by national patient safety organizations as a patient safety concern given the risk of patient harm.^5-7 Despite these concerns, CPMs are routinely used: up to 48% of pediatric patients in nonintensive care units at children’s hospitals are monitored.²

Although the low number of alarms that receive responses has been well-described,^8,9 the reasons why clinicians do or do not respond to alarms are unclear. A study conducted in an adult perioperative unit noted prolonged nurse response times for patients with high alarm rates.¹⁰ A second study conducted in the pediatric inpatient setting demonstrated a dose-response effect and noted progressively prolonged nurse response times with increased rates of nonactionable alarms.^4,11 Findings from another study suggested that underlying factors are highly complex and may be a result of excessive alarms, clinician characteristics, and working conditions (eg, workload and unit noise level).¹² Evidence also suggests that humans have difficulty distinguishing the importance of alarms in situations where multiple alarm tones are used, a common scenario in hospitals.^13,14 Understanding the factors that contribute to clinicians responding or not responding to CPM alarms will be crucial for addressing this serious patient safety issue.

An enhanced understanding of why nurses respond to alarms in daily practice will inform intervention development and improvement work. In the long term, this information could help improve systems for monitoring pediatric inpatients that are less prone to issues with alarm fatigue. The objective of this qualitative study, which employed structured observation, was to describe how bedside nurses think about and act upon bedside monitor alarms in a general pediatric inpatient unit.

METHODS

Study Design and Setting

This prospective observational study took place on a 48-bed hospital medicine unit at a large, freestanding children’s hospital with >650 beds and >19,000 annual admissions. General Electric (Little Chalfont, United Kingdom) physiologic monitors (models Dash 3000, 4000, and 5000) were used at the time of the study, and nurses could be notified of monitor alarms in four ways: First, an in-room auditory alarm sounds. Second, a light positioned above the door outside of each patient room blinks for alarms that are at a “warning” or “critical level” (eg ventricular tachycardia or low oxygen saturation). Third, audible alarms occur at the unit’s central monitoring station. Lastly, another staff member can notify the patient’s nurse via in-person conversion or secure smart phone communication. On the study unit, CPMs are initiated and discontinued through a physician order.

 

This study was reviewed and approved by the hospital’s institutional review board.

Study Population

We used a purposive recruitment strategy to enroll bedside nurses working on general hospital medicine units, stratified to ensure varying levels of experience and primary shifts (eg, day vs night). We planned to conduct approximately two observations with each participating nurse and to continue collecting data until we could no longer identify new insights in terms of responses to alarms (ie, thematic saturation¹⁵). Observations were targeted to cover times of day that coincided with increased rates of distraction. These times included just prior to and after the morning and evening change of shifts (7:00 am and 7:00 pm), during morning rounds (8:00 am-12:00 pm), and heavy admission times (12:00 pm-10:00 pm). After written informed consent, a nurse was eligible for observation during his/her shift if he/she was caring for at least one monitored patient. Enrolled nurses were made aware of the general study topic but were blinded to the study team’s hypotheses.

Data Sources

Prior to data collection, the research team, which consisted of physicians, bedside nurses, research coordinators, and a human factors expert, created a system for categorizing alarm responses. Categories for observed responses were based on the location and corresponding action taken. Initial categories were developed a priori from existing literature and expanded through input from the multidisciplinary study team, then vetted with bedside staff, and finally pilot tested through >4 hours of observations, thus producing the final categories. These categories were entered into a work-sampling program (WorkStudy by Quetech Ltd., Waterloo, Ontario, Canada) to facilitate quick data recording during observations.

The hospital uses a central alarm collection software (BedMasterEx by Anandic Medical Systems, Feuerthalen, Switzerland), which permitted the collection of date, time, trigger (eg, high heart rate), and level (eg, crisis, warning) of the generated CPM alarms. Alarms collected are based on thresholds preset at the bedside monitor. The central collection software does not differentiate between accurate (eg, correctly representing the physiologic state of the patient) and inaccurate alarms.

Observation Procedure

At the time of observation, nurse demographic information (eg, primary shift worked and years working as a nurse) was obtained. A brief preobservation questionnaire was administered to collect patient information (eg, age and diagnosis) and the nurses’ perspectives on the necessity of monitors for each monitored patient in his/her care.

The observer shadowed the nurse for a two-hour block of his/her shift. During this time, nurses were instructed to “think aloud” as they responded to alarms (eg, “I notice the oxygen saturation monitor alarming off, but the probe has fallen off”). A trained observer (AML or KMT) recorded responses verbalized by the nurse and his/her reaction by selecting the appropriate category using the work-sampling software. Data were also collected on the vital sign associated with the alarm (eg, heart rate). Moreover, the observer kept written notes to provide context for electronically recorded data. Alarms that were not verbalized by the nurse were not counted. Similarly, alarms that were noted outside of the room by the nurse were not classified by vital sign unless the nurse confirmed with the bedside monitor. Observers did not adjudicate the accuracy of the alarms. The session was stopped if monitors were discontinued during the observation period. Alarm data generated by the bedside monitor were pulled for each patient room after observations were completed.

 

Analysis

Descriptive statistics were used to assess the percentage of each nurse response category and each alarm type (eg, heart rate and respiratory rate). The observed alarm rate was calculated by taking the total number of observed alarms (ie, alarms noted by the nurse) divided by the total number of patient-hours observed. The monitor-generated alarm rate was calculated by taking the total number of alarms from the bedside-alarm generated data divided by the number of patient-hours observed.

Electronically recorded observations using the work-sampling program were cross-referenced with hand-written field notes to assess for any discrepancies or identify relevant events not captured by the program. Three study team members (AML, KMT, and ACS) reviewed each observation independently and compared field notes to ensure accurate categorization. Discrepancies were referred to the larger study group in cases of uncertainty.

RESULTS

Nine nurses had monitored patients during the available observations and participated in 19 observation sessions, which included 35 monitored patients for a total of 61.3 patient-hours of observation. Nurses were observed for a median of two times each (range 1-4). The median number of monitored patients during a single observation session was two (range 1-3). Observed nurses were female with a median of eight years of experience (range 0.5-26 years). Patients represented a broad range of age categories and were hospitalized with a variety of diagnoses (Table). Nurses, when queried at the start of the observation, felt that monitors were necessary for 29 (82.9%) of the observed patients given either patient condition or unit policy.

A total of 207 observed nurse responses to alarms occurred during the study period for a rate of 3.4 responses per patient per hour. Of the total number of responses, 45 (21.7%) were noted outside of a patient room, and in 15 (33.3%) the nurse chose to go to the room. The other 162 were recorded when the nurse was present in the room when the alarm activated. Of the 177 in-person nurse responses, 50 were related to a pulse oximetry alarm, 66 were related to a heart rate alarm, and 61 were related to a respiratory rate alarm. The most common observed in-person response to an alarm involved the nurse judging that no intervention was necessary (n = 152, 73.1%). Only 14 (7% of total responses) observed in-person responses involved a clinical intervention, such as suctioning or titrating supplemental oxygen. Findings are summarized in the Figure and describe nurse-verbalized reasons to further assess (or not) and then whether the nurse chose to take action (or not) after an alarm.

Alarm data were available for 17 of the 19 observation periods during the study. Technical issues with the central alarm collection software precluded alarm data collection for two of the observation sessions. A total of 483 alarms were recorded on bedside monitors during those 17 observation periods or 8.8 alarms per patient per hour, which was equivalent to 211.2 alarms per patient-day. A total of 175 observed responses were collected during these 17 observation periods. This number of responses was 36% of the number we would have expected on the basis of the alarm count from the central alarm software.

There were no patients transferred to the intensive care unit during the observation period. Nurses who chose not to respond to alarms outside the room most often cited the brevity of the alarm or other reassuring contextual details, such as that a family member was in the room to notify them if anything was truly wrong, that another member of the medical team was with the patient, or that they had recently assessed the patient and thought likely the alarm did not require any action. During three observations, the observed nurse cited the presence of family in the patient’s room in their decision not to conduct further assessment in response to the alarm, noting that the parent would be able to notify the nurse if something required attention. On two occasions in which a nurse had multiple monitored patients, the observed nurse noted that if the other monitored patients were alarming and she happened to be in another patient’s room, she would not be able to hear them. Four nurses cited policy as the reason a patient was on monitors (eg, patient was on respiratory support at night for obstructive sleep apnea).

 

DISCUSSION

We characterized responses to physiologic monitor alarms by a group of nurses with a range of experience levels. We found that most nurse responses to alarms in continuously monitored general pediatric patients involved no intervention, and further assessment was often not conducted for alarms that occurred outside of the room if the nurse noted otherwise reassuring clinical context. Observed responses occurred for 36% of alarms during the study period when compared with bedside monitor-alarm generated data. Overall, only 14 clinical interventions were noted among the observed responses. Nurses noted that they felt the monitors were necessary for 82.9% of monitored patients because of the clinical context or because of unit policy.

Our study findings highlight some potential contradictions in the current widespread use of CPMs in general pediatric units and how clinicians respond to them in practice.² First, while nurses reported that monitors were necessary for most of their patients, participating nurses deemed few alarms clinically actionable and often chose not to further assess when they noted alarms outside of the room. This is in line with findings from prior studies suggesting that clinicians overvalue the contribution of monitoring systems to patient safety.^16,17 Second, while this finding occurred in a minority of the observations, the presence of family members at the patient’s bedside was cited by nurses as a rationale for whether they responded to alarms. While family members are capable of identifying safety issues,¹⁸ formal systems to engage them in patient safety and physiologic monitoring are lacking. Finally, clinical interventions or responses to the alerts of deteriorating patients, which best represented the original intent of CPMs, were rare and accounted for just 7% of the responses. Further work elucidating why physicians and nurses choose to use CPMs may be helpful to identify interventions to reduce inappropriate monitor use and highlight gaps in frontline staff knowledge about the benefits and risks of CPM use.

Our findings provide a novel understanding of previously observed phenomena, such as long response times or nonresponses in settings with high alarm rates.^4,10 Similar to that in a prior study conducted in the pediatric setting,¹¹ alarms with an observed response constituted a minority of the total alarms that occurred in our study. This finding has previously been attributed to mental fatigue, caregiver apathy, and desensitization.⁸ However, even though a minority of observed responses in our study included an intervention, the nurse had a rationale for why the alarm did or did not need a response. This behavior and the verbalized rationale indicate that in his/her opinion, not responding to the alarm was clinically appropriate. Study participants also reflected on the difficulties of responding to alarms given the monitor system setup, in which they may not always be capable of hearing alarms for their patients. Without data from nurses regarding the alarms that had no observed response, we can only speculate; however, based on our findings, each of these factors could contribute to nonresponse. Finally, while high numbers of false alarms have been posited as an underlying cause of alarm fatigue, we noted that a majority of nonresponse was reported to be related to other clinical factors. This relationship suggests that from the nurse’s perspective, a more applicable framework for understanding alarms would be based on clinical actionability⁴ over physiologic accuracy.

In total, our findings suggest that a multifaceted approach will be necessary to improve alarm response rates. These interventions should include adjusting parameters such that alarms are highly likely to indicate a need for intervention coupled with educational interventions addressing clinician knowledge of the alarm system and bias about the actionability of alarms may improve response rates. Changes in the monitoring system setup such that nurses can easily be notified when alarms occur may also be indicated, in addition to formally engaging patients and families around response to alarms. Although secondary notification systems (eg, alarms transmitted to individual clinician’s devices) are one solution, the utilization of these systems needs to be balanced with the risks of contributing to existing alarm fatigue and the need to appropriately tailor monitoring thresholds and strategies to patients.

Our study has several limitations. First, nurses may have responded in a way they perceive to be socially desirable, and studies using in-person observers are also prone to a Hawthorne-like effect,^19-21 where the nurse may have tried to respond more frequently to alarms than usual during observations. However, given that the majority of bedside alarms did not receive a response and a substantial number of responses involved no action, these effects were likely weak. Second, we were unable to assess which alarms were accurately reflecting the patient’s physiologic status and which were not; we were also unable to link observed alarm response to monitor-recorded alarms. Third, despite the use of silent observers and an actual, rather than a simulated, clinical setting, by virtue of the data collection method we likely captured a more deliberate thought process (so-called System 2 thinking)²² rather than the subconscious processes that may predominate when nurses respond to alarms in the course of clinical care (System 1 thinking).²² Despite this limitation, our study findings, which reflect a nurse’s in-the-moment thinking, remain relevant to guiding the improvement of monitoring systems, and the development of nurse-facing interventions and education. Finally, we studied a small, purposive sample of nurses at a single hospital. Our study sample impacts the generalizability of our results and precluded a detailed analysis of the effect of nurse- and patient-level variables.

 

CONCLUSION

We found that nurses often deemed that no response was necessary for CPM alarms. Nurses cited contextual factors, including the duration of alarms and the presence of other providers or parents in their decision-making. Few (7%) of the alarm responses in our study included a clinical intervention. The number of observed alarm responses constituted roughly a third of the alarms recorded by bedside CPMs during the study. This result supports concerns about the nurse’s capacity to hear and process all CPM alarms given system limitations and a heavy clinical workload. Subsequent steps should include staff education, reducing overall alarm rates with appropriate monitor use and actionable alarm thresholds, and ensuring that patient alarms are easily recognizable for frontline staff.

Disclosures

The authors have no conflicts of interest to disclose.

Funding

This work was supported by the Place Outcomes Research Award from the Cincinnati Children’s Research Foundation. Dr. Brady is supported by the Agency for Healthcare Research and Quality under Award Number K08HS23827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

Alarms from bedside continuous physiologic monitors (CPMs) occur frequently in children’s hospitals and can lead to harm. Recent studies conducted in children’s hospitals have identified alarm rates of up to 152 alarms per patient per day outside of the intensive care unit,^1-3 with as few as 1% of alarms being considered clinically important.⁴ Excessive alarms have been linked to alarm fatigue, when providers become desensitized to and may miss alarms indicating impending patient deterioration. Alarm fatigue has been identified by national patient safety organizations as a patient safety concern given the risk of patient harm.^5-7 Despite these concerns, CPMs are routinely used: up to 48% of pediatric patients in nonintensive care units at children’s hospitals are monitored.²

Although the low number of alarms that receive responses has been well-described,^8,9 the reasons why clinicians do or do not respond to alarms are unclear. A study conducted in an adult perioperative unit noted prolonged nurse response times for patients with high alarm rates.¹⁰ A second study conducted in the pediatric inpatient setting demonstrated a dose-response effect and noted progressively prolonged nurse response times with increased rates of nonactionable alarms.^4,11 Findings from another study suggested that underlying factors are highly complex and may be a result of excessive alarms, clinician characteristics, and working conditions (eg, workload and unit noise level).¹² Evidence also suggests that humans have difficulty distinguishing the importance of alarms in situations where multiple alarm tones are used, a common scenario in hospitals.^13,14 Understanding the factors that contribute to clinicians responding or not responding to CPM alarms will be crucial for addressing this serious patient safety issue.

An enhanced understanding of why nurses respond to alarms in daily practice will inform intervention development and improvement work. In the long term, this information could help improve systems for monitoring pediatric inpatients that are less prone to issues with alarm fatigue. The objective of this qualitative study, which employed structured observation, was to describe how bedside nurses think about and act upon bedside monitor alarms in a general pediatric inpatient unit.

METHODS

Study Design and Setting

This prospective observational study took place on a 48-bed hospital medicine unit at a large, freestanding children’s hospital with >650 beds and >19,000 annual admissions. General Electric (Little Chalfont, United Kingdom) physiologic monitors (models Dash 3000, 4000, and 5000) were used at the time of the study, and nurses could be notified of monitor alarms in four ways: First, an in-room auditory alarm sounds. Second, a light positioned above the door outside of each patient room blinks for alarms that are at a “warning” or “critical level” (eg ventricular tachycardia or low oxygen saturation). Third, audible alarms occur at the unit’s central monitoring station. Lastly, another staff member can notify the patient’s nurse via in-person conversion or secure smart phone communication. On the study unit, CPMs are initiated and discontinued through a physician order.

 

This study was reviewed and approved by the hospital’s institutional review board.

Study Population

We used a purposive recruitment strategy to enroll bedside nurses working on general hospital medicine units, stratified to ensure varying levels of experience and primary shifts (eg, day vs night). We planned to conduct approximately two observations with each participating nurse and to continue collecting data until we could no longer identify new insights in terms of responses to alarms (ie, thematic saturation¹⁵). Observations were targeted to cover times of day that coincided with increased rates of distraction. These times included just prior to and after the morning and evening change of shifts (7:00 am and 7:00 pm), during morning rounds (8:00 am-12:00 pm), and heavy admission times (12:00 pm-10:00 pm). After written informed consent, a nurse was eligible for observation during his/her shift if he/she was caring for at least one monitored patient. Enrolled nurses were made aware of the general study topic but were blinded to the study team’s hypotheses.

Data Sources

Prior to data collection, the research team, which consisted of physicians, bedside nurses, research coordinators, and a human factors expert, created a system for categorizing alarm responses. Categories for observed responses were based on the location and corresponding action taken. Initial categories were developed a priori from existing literature and expanded through input from the multidisciplinary study team, then vetted with bedside staff, and finally pilot tested through >4 hours of observations, thus producing the final categories. These categories were entered into a work-sampling program (WorkStudy by Quetech Ltd., Waterloo, Ontario, Canada) to facilitate quick data recording during observations.

The hospital uses a central alarm collection software (BedMasterEx by Anandic Medical Systems, Feuerthalen, Switzerland), which permitted the collection of date, time, trigger (eg, high heart rate), and level (eg, crisis, warning) of the generated CPM alarms. Alarms collected are based on thresholds preset at the bedside monitor. The central collection software does not differentiate between accurate (eg, correctly representing the physiologic state of the patient) and inaccurate alarms.

Observation Procedure

At the time of observation, nurse demographic information (eg, primary shift worked and years working as a nurse) was obtained. A brief preobservation questionnaire was administered to collect patient information (eg, age and diagnosis) and the nurses’ perspectives on the necessity of monitors for each monitored patient in his/her care.

The observer shadowed the nurse for a two-hour block of his/her shift. During this time, nurses were instructed to “think aloud” as they responded to alarms (eg, “I notice the oxygen saturation monitor alarming off, but the probe has fallen off”). A trained observer (AML or KMT) recorded responses verbalized by the nurse and his/her reaction by selecting the appropriate category using the work-sampling software. Data were also collected on the vital sign associated with the alarm (eg, heart rate). Moreover, the observer kept written notes to provide context for electronically recorded data. Alarms that were not verbalized by the nurse were not counted. Similarly, alarms that were noted outside of the room by the nurse were not classified by vital sign unless the nurse confirmed with the bedside monitor. Observers did not adjudicate the accuracy of the alarms. The session was stopped if monitors were discontinued during the observation period. Alarm data generated by the bedside monitor were pulled for each patient room after observations were completed.

 

Analysis

Descriptive statistics were used to assess the percentage of each nurse response category and each alarm type (eg, heart rate and respiratory rate). The observed alarm rate was calculated by taking the total number of observed alarms (ie, alarms noted by the nurse) divided by the total number of patient-hours observed. The monitor-generated alarm rate was calculated by taking the total number of alarms from the bedside-alarm generated data divided by the number of patient-hours observed.

Electronically recorded observations using the work-sampling program were cross-referenced with hand-written field notes to assess for any discrepancies or identify relevant events not captured by the program. Three study team members (AML, KMT, and ACS) reviewed each observation independently and compared field notes to ensure accurate categorization. Discrepancies were referred to the larger study group in cases of uncertainty.

RESULTS

Nine nurses had monitored patients during the available observations and participated in 19 observation sessions, which included 35 monitored patients for a total of 61.3 patient-hours of observation. Nurses were observed for a median of two times each (range 1-4). The median number of monitored patients during a single observation session was two (range 1-3). Observed nurses were female with a median of eight years of experience (range 0.5-26 years). Patients represented a broad range of age categories and were hospitalized with a variety of diagnoses (Table). Nurses, when queried at the start of the observation, felt that monitors were necessary for 29 (82.9%) of the observed patients given either patient condition or unit policy.

A total of 207 observed nurse responses to alarms occurred during the study period for a rate of 3.4 responses per patient per hour. Of the total number of responses, 45 (21.7%) were noted outside of a patient room, and in 15 (33.3%) the nurse chose to go to the room. The other 162 were recorded when the nurse was present in the room when the alarm activated. Of the 177 in-person nurse responses, 50 were related to a pulse oximetry alarm, 66 were related to a heart rate alarm, and 61 were related to a respiratory rate alarm. The most common observed in-person response to an alarm involved the nurse judging that no intervention was necessary (n = 152, 73.1%). Only 14 (7% of total responses) observed in-person responses involved a clinical intervention, such as suctioning or titrating supplemental oxygen. Findings are summarized in the Figure and describe nurse-verbalized reasons to further assess (or not) and then whether the nurse chose to take action (or not) after an alarm.

Alarm data were available for 17 of the 19 observation periods during the study. Technical issues with the central alarm collection software precluded alarm data collection for two of the observation sessions. A total of 483 alarms were recorded on bedside monitors during those 17 observation periods or 8.8 alarms per patient per hour, which was equivalent to 211.2 alarms per patient-day. A total of 175 observed responses were collected during these 17 observation periods. This number of responses was 36% of the number we would have expected on the basis of the alarm count from the central alarm software.

There were no patients transferred to the intensive care unit during the observation period. Nurses who chose not to respond to alarms outside the room most often cited the brevity of the alarm or other reassuring contextual details, such as that a family member was in the room to notify them if anything was truly wrong, that another member of the medical team was with the patient, or that they had recently assessed the patient and thought likely the alarm did not require any action. During three observations, the observed nurse cited the presence of family in the patient’s room in their decision not to conduct further assessment in response to the alarm, noting that the parent would be able to notify the nurse if something required attention. On two occasions in which a nurse had multiple monitored patients, the observed nurse noted that if the other monitored patients were alarming and she happened to be in another patient’s room, she would not be able to hear them. Four nurses cited policy as the reason a patient was on monitors (eg, patient was on respiratory support at night for obstructive sleep apnea).

 

DISCUSSION

We characterized responses to physiologic monitor alarms by a group of nurses with a range of experience levels. We found that most nurse responses to alarms in continuously monitored general pediatric patients involved no intervention, and further assessment was often not conducted for alarms that occurred outside of the room if the nurse noted otherwise reassuring clinical context. Observed responses occurred for 36% of alarms during the study period when compared with bedside monitor-alarm generated data. Overall, only 14 clinical interventions were noted among the observed responses. Nurses noted that they felt the monitors were necessary for 82.9% of monitored patients because of the clinical context or because of unit policy.

Our study findings highlight some potential contradictions in the current widespread use of CPMs in general pediatric units and how clinicians respond to them in practice.² First, while nurses reported that monitors were necessary for most of their patients, participating nurses deemed few alarms clinically actionable and often chose not to further assess when they noted alarms outside of the room. This is in line with findings from prior studies suggesting that clinicians overvalue the contribution of monitoring systems to patient safety.^16,17 Second, while this finding occurred in a minority of the observations, the presence of family members at the patient’s bedside was cited by nurses as a rationale for whether they responded to alarms. While family members are capable of identifying safety issues,¹⁸ formal systems to engage them in patient safety and physiologic monitoring are lacking. Finally, clinical interventions or responses to the alerts of deteriorating patients, which best represented the original intent of CPMs, were rare and accounted for just 7% of the responses. Further work elucidating why physicians and nurses choose to use CPMs may be helpful to identify interventions to reduce inappropriate monitor use and highlight gaps in frontline staff knowledge about the benefits and risks of CPM use.

Our findings provide a novel understanding of previously observed phenomena, such as long response times or nonresponses in settings with high alarm rates.^4,10 Similar to that in a prior study conducted in the pediatric setting,¹¹ alarms with an observed response constituted a minority of the total alarms that occurred in our study. This finding has previously been attributed to mental fatigue, caregiver apathy, and desensitization.⁸ However, even though a minority of observed responses in our study included an intervention, the nurse had a rationale for why the alarm did or did not need a response. This behavior and the verbalized rationale indicate that in his/her opinion, not responding to the alarm was clinically appropriate. Study participants also reflected on the difficulties of responding to alarms given the monitor system setup, in which they may not always be capable of hearing alarms for their patients. Without data from nurses regarding the alarms that had no observed response, we can only speculate; however, based on our findings, each of these factors could contribute to nonresponse. Finally, while high numbers of false alarms have been posited as an underlying cause of alarm fatigue, we noted that a majority of nonresponse was reported to be related to other clinical factors. This relationship suggests that from the nurse’s perspective, a more applicable framework for understanding alarms would be based on clinical actionability⁴ over physiologic accuracy.

In total, our findings suggest that a multifaceted approach will be necessary to improve alarm response rates. These interventions should include adjusting parameters such that alarms are highly likely to indicate a need for intervention coupled with educational interventions addressing clinician knowledge of the alarm system and bias about the actionability of alarms may improve response rates. Changes in the monitoring system setup such that nurses can easily be notified when alarms occur may also be indicated, in addition to formally engaging patients and families around response to alarms. Although secondary notification systems (eg, alarms transmitted to individual clinician’s devices) are one solution, the utilization of these systems needs to be balanced with the risks of contributing to existing alarm fatigue and the need to appropriately tailor monitoring thresholds and strategies to patients.

Our study has several limitations. First, nurses may have responded in a way they perceive to be socially desirable, and studies using in-person observers are also prone to a Hawthorne-like effect,^19-21 where the nurse may have tried to respond more frequently to alarms than usual during observations. However, given that the majority of bedside alarms did not receive a response and a substantial number of responses involved no action, these effects were likely weak. Second, we were unable to assess which alarms were accurately reflecting the patient’s physiologic status and which were not; we were also unable to link observed alarm response to monitor-recorded alarms. Third, despite the use of silent observers and an actual, rather than a simulated, clinical setting, by virtue of the data collection method we likely captured a more deliberate thought process (so-called System 2 thinking)²² rather than the subconscious processes that may predominate when nurses respond to alarms in the course of clinical care (System 1 thinking).²² Despite this limitation, our study findings, which reflect a nurse’s in-the-moment thinking, remain relevant to guiding the improvement of monitoring systems, and the development of nurse-facing interventions and education. Finally, we studied a small, purposive sample of nurses at a single hospital. Our study sample impacts the generalizability of our results and precluded a detailed analysis of the effect of nurse- and patient-level variables.

 

CONCLUSION

We found that nurses often deemed that no response was necessary for CPM alarms. Nurses cited contextual factors, including the duration of alarms and the presence of other providers or parents in their decision-making. Few (7%) of the alarm responses in our study included a clinical intervention. The number of observed alarm responses constituted roughly a third of the alarms recorded by bedside CPMs during the study. This result supports concerns about the nurse’s capacity to hear and process all CPM alarms given system limitations and a heavy clinical workload. Subsequent steps should include staff education, reducing overall alarm rates with appropriate monitor use and actionable alarm thresholds, and ensuring that patient alarms are easily recognizable for frontline staff.

Disclosures

The authors have no conflicts of interest to disclose.

Funding

This work was supported by the Place Outcomes Research Award from the Cincinnati Children’s Research Foundation. Dr. Brady is supported by the Agency for Healthcare Research and Quality under Award Number K08HS23827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

Alarms from bedside continuous physiologic monitors (CPMs) occur frequently in children’s hospitals and can lead to harm. Recent studies conducted in children’s hospitals have identified alarm rates of up to 152 alarms per patient per day outside of the intensive care unit,^1-3 with as few as 1% of alarms being considered clinically important.⁴ Excessive alarms have been linked to alarm fatigue, when providers become desensitized to and may miss alarms indicating impending patient deterioration. Alarm fatigue has been identified by national patient safety organizations as a patient safety concern given the risk of patient harm.^5-7 Despite these concerns, CPMs are routinely used: up to 48% of pediatric patients in nonintensive care units at children’s hospitals are monitored.²

Although the low number of alarms that receive responses has been well-described,^8,9 the reasons why clinicians do or do not respond to alarms are unclear. A study conducted in an adult perioperative unit noted prolonged nurse response times for patients with high alarm rates.¹⁰ A second study conducted in the pediatric inpatient setting demonstrated a dose-response effect and noted progressively prolonged nurse response times with increased rates of nonactionable alarms.^4,11 Findings from another study suggested that underlying factors are highly complex and may be a result of excessive alarms, clinician characteristics, and working conditions (eg, workload and unit noise level).¹² Evidence also suggests that humans have difficulty distinguishing the importance of alarms in situations where multiple alarm tones are used, a common scenario in hospitals.^13,14 Understanding the factors that contribute to clinicians responding or not responding to CPM alarms will be crucial for addressing this serious patient safety issue.

An enhanced understanding of why nurses respond to alarms in daily practice will inform intervention development and improvement work. In the long term, this information could help improve systems for monitoring pediatric inpatients that are less prone to issues with alarm fatigue. The objective of this qualitative study, which employed structured observation, was to describe how bedside nurses think about and act upon bedside monitor alarms in a general pediatric inpatient unit.

METHODS

Study Design and Setting

This prospective observational study took place on a 48-bed hospital medicine unit at a large, freestanding children’s hospital with >650 beds and >19,000 annual admissions. General Electric (Little Chalfont, United Kingdom) physiologic monitors (models Dash 3000, 4000, and 5000) were used at the time of the study, and nurses could be notified of monitor alarms in four ways: First, an in-room auditory alarm sounds. Second, a light positioned above the door outside of each patient room blinks for alarms that are at a “warning” or “critical level” (eg ventricular tachycardia or low oxygen saturation). Third, audible alarms occur at the unit’s central monitoring station. Lastly, another staff member can notify the patient’s nurse via in-person conversion or secure smart phone communication. On the study unit, CPMs are initiated and discontinued through a physician order.

 

This study was reviewed and approved by the hospital’s institutional review board.

Study Population

We used a purposive recruitment strategy to enroll bedside nurses working on general hospital medicine units, stratified to ensure varying levels of experience and primary shifts (eg, day vs night). We planned to conduct approximately two observations with each participating nurse and to continue collecting data until we could no longer identify new insights in terms of responses to alarms (ie, thematic saturation¹⁵). Observations were targeted to cover times of day that coincided with increased rates of distraction. These times included just prior to and after the morning and evening change of shifts (7:00 am and 7:00 pm), during morning rounds (8:00 am-12:00 pm), and heavy admission times (12:00 pm-10:00 pm). After written informed consent, a nurse was eligible for observation during his/her shift if he/she was caring for at least one monitored patient. Enrolled nurses were made aware of the general study topic but were blinded to the study team’s hypotheses.

Data Sources

Prior to data collection, the research team, which consisted of physicians, bedside nurses, research coordinators, and a human factors expert, created a system for categorizing alarm responses. Categories for observed responses were based on the location and corresponding action taken. Initial categories were developed a priori from existing literature and expanded through input from the multidisciplinary study team, then vetted with bedside staff, and finally pilot tested through >4 hours of observations, thus producing the final categories. These categories were entered into a work-sampling program (WorkStudy by Quetech Ltd., Waterloo, Ontario, Canada) to facilitate quick data recording during observations.

The hospital uses a central alarm collection software (BedMasterEx by Anandic Medical Systems, Feuerthalen, Switzerland), which permitted the collection of date, time, trigger (eg, high heart rate), and level (eg, crisis, warning) of the generated CPM alarms. Alarms collected are based on thresholds preset at the bedside monitor. The central collection software does not differentiate between accurate (eg, correctly representing the physiologic state of the patient) and inaccurate alarms.

Observation Procedure

At the time of observation, nurse demographic information (eg, primary shift worked and years working as a nurse) was obtained. A brief preobservation questionnaire was administered to collect patient information (eg, age and diagnosis) and the nurses’ perspectives on the necessity of monitors for each monitored patient in his/her care.

The observer shadowed the nurse for a two-hour block of his/her shift. During this time, nurses were instructed to “think aloud” as they responded to alarms (eg, “I notice the oxygen saturation monitor alarming off, but the probe has fallen off”). A trained observer (AML or KMT) recorded responses verbalized by the nurse and his/her reaction by selecting the appropriate category using the work-sampling software. Data were also collected on the vital sign associated with the alarm (eg, heart rate). Moreover, the observer kept written notes to provide context for electronically recorded data. Alarms that were not verbalized by the nurse were not counted. Similarly, alarms that were noted outside of the room by the nurse were not classified by vital sign unless the nurse confirmed with the bedside monitor. Observers did not adjudicate the accuracy of the alarms. The session was stopped if monitors were discontinued during the observation period. Alarm data generated by the bedside monitor were pulled for each patient room after observations were completed.

 

Analysis

Descriptive statistics were used to assess the percentage of each nurse response category and each alarm type (eg, heart rate and respiratory rate). The observed alarm rate was calculated by taking the total number of observed alarms (ie, alarms noted by the nurse) divided by the total number of patient-hours observed. The monitor-generated alarm rate was calculated by taking the total number of alarms from the bedside-alarm generated data divided by the number of patient-hours observed.

Electronically recorded observations using the work-sampling program were cross-referenced with hand-written field notes to assess for any discrepancies or identify relevant events not captured by the program. Three study team members (AML, KMT, and ACS) reviewed each observation independently and compared field notes to ensure accurate categorization. Discrepancies were referred to the larger study group in cases of uncertainty.

RESULTS

Nine nurses had monitored patients during the available observations and participated in 19 observation sessions, which included 35 monitored patients for a total of 61.3 patient-hours of observation. Nurses were observed for a median of two times each (range 1-4). The median number of monitored patients during a single observation session was two (range 1-3). Observed nurses were female with a median of eight years of experience (range 0.5-26 years). Patients represented a broad range of age categories and were hospitalized with a variety of diagnoses (Table). Nurses, when queried at the start of the observation, felt that monitors were necessary for 29 (82.9%) of the observed patients given either patient condition or unit policy.

A total of 207 observed nurse responses to alarms occurred during the study period for a rate of 3.4 responses per patient per hour. Of the total number of responses, 45 (21.7%) were noted outside of a patient room, and in 15 (33.3%) the nurse chose to go to the room. The other 162 were recorded when the nurse was present in the room when the alarm activated. Of the 177 in-person nurse responses, 50 were related to a pulse oximetry alarm, 66 were related to a heart rate alarm, and 61 were related to a respiratory rate alarm. The most common observed in-person response to an alarm involved the nurse judging that no intervention was necessary (n = 152, 73.1%). Only 14 (7% of total responses) observed in-person responses involved a clinical intervention, such as suctioning or titrating supplemental oxygen. Findings are summarized in the Figure and describe nurse-verbalized reasons to further assess (or not) and then whether the nurse chose to take action (or not) after an alarm.

Alarm data were available for 17 of the 19 observation periods during the study. Technical issues with the central alarm collection software precluded alarm data collection for two of the observation sessions. A total of 483 alarms were recorded on bedside monitors during those 17 observation periods or 8.8 alarms per patient per hour, which was equivalent to 211.2 alarms per patient-day. A total of 175 observed responses were collected during these 17 observation periods. This number of responses was 36% of the number we would have expected on the basis of the alarm count from the central alarm software.

There were no patients transferred to the intensive care unit during the observation period. Nurses who chose not to respond to alarms outside the room most often cited the brevity of the alarm or other reassuring contextual details, such as that a family member was in the room to notify them if anything was truly wrong, that another member of the medical team was with the patient, or that they had recently assessed the patient and thought likely the alarm did not require any action. During three observations, the observed nurse cited the presence of family in the patient’s room in their decision not to conduct further assessment in response to the alarm, noting that the parent would be able to notify the nurse if something required attention. On two occasions in which a nurse had multiple monitored patients, the observed nurse noted that if the other monitored patients were alarming and she happened to be in another patient’s room, she would not be able to hear them. Four nurses cited policy as the reason a patient was on monitors (eg, patient was on respiratory support at night for obstructive sleep apnea).

 

DISCUSSION

We characterized responses to physiologic monitor alarms by a group of nurses with a range of experience levels. We found that most nurse responses to alarms in continuously monitored general pediatric patients involved no intervention, and further assessment was often not conducted for alarms that occurred outside of the room if the nurse noted otherwise reassuring clinical context. Observed responses occurred for 36% of alarms during the study period when compared with bedside monitor-alarm generated data. Overall, only 14 clinical interventions were noted among the observed responses. Nurses noted that they felt the monitors were necessary for 82.9% of monitored patients because of the clinical context or because of unit policy.

Our study findings highlight some potential contradictions in the current widespread use of CPMs in general pediatric units and how clinicians respond to them in practice.² First, while nurses reported that monitors were necessary for most of their patients, participating nurses deemed few alarms clinically actionable and often chose not to further assess when they noted alarms outside of the room. This is in line with findings from prior studies suggesting that clinicians overvalue the contribution of monitoring systems to patient safety.^16,17 Second, while this finding occurred in a minority of the observations, the presence of family members at the patient’s bedside was cited by nurses as a rationale for whether they responded to alarms. While family members are capable of identifying safety issues,¹⁸ formal systems to engage them in patient safety and physiologic monitoring are lacking. Finally, clinical interventions or responses to the alerts of deteriorating patients, which best represented the original intent of CPMs, were rare and accounted for just 7% of the responses. Further work elucidating why physicians and nurses choose to use CPMs may be helpful to identify interventions to reduce inappropriate monitor use and highlight gaps in frontline staff knowledge about the benefits and risks of CPM use.

Our findings provide a novel understanding of previously observed phenomena, such as long response times or nonresponses in settings with high alarm rates.^4,10 Similar to that in a prior study conducted in the pediatric setting,¹¹ alarms with an observed response constituted a minority of the total alarms that occurred in our study. This finding has previously been attributed to mental fatigue, caregiver apathy, and desensitization.⁸ However, even though a minority of observed responses in our study included an intervention, the nurse had a rationale for why the alarm did or did not need a response. This behavior and the verbalized rationale indicate that in his/her opinion, not responding to the alarm was clinically appropriate. Study participants also reflected on the difficulties of responding to alarms given the monitor system setup, in which they may not always be capable of hearing alarms for their patients. Without data from nurses regarding the alarms that had no observed response, we can only speculate; however, based on our findings, each of these factors could contribute to nonresponse. Finally, while high numbers of false alarms have been posited as an underlying cause of alarm fatigue, we noted that a majority of nonresponse was reported to be related to other clinical factors. This relationship suggests that from the nurse’s perspective, a more applicable framework for understanding alarms would be based on clinical actionability⁴ over physiologic accuracy.

In total, our findings suggest that a multifaceted approach will be necessary to improve alarm response rates. These interventions should include adjusting parameters such that alarms are highly likely to indicate a need for intervention coupled with educational interventions addressing clinician knowledge of the alarm system and bias about the actionability of alarms may improve response rates. Changes in the monitoring system setup such that nurses can easily be notified when alarms occur may also be indicated, in addition to formally engaging patients and families around response to alarms. Although secondary notification systems (eg, alarms transmitted to individual clinician’s devices) are one solution, the utilization of these systems needs to be balanced with the risks of contributing to existing alarm fatigue and the need to appropriately tailor monitoring thresholds and strategies to patients.

Our study has several limitations. First, nurses may have responded in a way they perceive to be socially desirable, and studies using in-person observers are also prone to a Hawthorne-like effect,^19-21 where the nurse may have tried to respond more frequently to alarms than usual during observations. However, given that the majority of bedside alarms did not receive a response and a substantial number of responses involved no action, these effects were likely weak. Second, we were unable to assess which alarms were accurately reflecting the patient’s physiologic status and which were not; we were also unable to link observed alarm response to monitor-recorded alarms. Third, despite the use of silent observers and an actual, rather than a simulated, clinical setting, by virtue of the data collection method we likely captured a more deliberate thought process (so-called System 2 thinking)²² rather than the subconscious processes that may predominate when nurses respond to alarms in the course of clinical care (System 1 thinking).²² Despite this limitation, our study findings, which reflect a nurse’s in-the-moment thinking, remain relevant to guiding the improvement of monitoring systems, and the development of nurse-facing interventions and education. Finally, we studied a small, purposive sample of nurses at a single hospital. Our study sample impacts the generalizability of our results and precluded a detailed analysis of the effect of nurse- and patient-level variables.

 

CONCLUSION

We found that nurses often deemed that no response was necessary for CPM alarms. Nurses cited contextual factors, including the duration of alarms and the presence of other providers or parents in their decision-making. Few (7%) of the alarm responses in our study included a clinical intervention. The number of observed alarm responses constituted roughly a third of the alarms recorded by bedside CPMs during the study. This result supports concerns about the nurse’s capacity to hear and process all CPM alarms given system limitations and a heavy clinical workload. Subsequent steps should include staff education, reducing overall alarm rates with appropriate monitor use and actionable alarm thresholds, and ensuring that patient alarms are easily recognizable for frontline staff.

Disclosures

The authors have no conflicts of interest to disclose.

Funding

This work was supported by the Place Outcomes Research Award from the Cincinnati Children’s Research Foundation. Dr. Brady is supported by the Agency for Healthcare Research and Quality under Award Number K08HS23827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.