NEW YORK – Seven years after the formation of the first pulmonary embolism response team (PERT), more than 100 institutions have joined the PERT Consortium, which was created to guide care and research for this thrombotic complication, according to a status report at a symposium on vascular and endovascular issues sponsored by the Cleveland Clinic Foundation.

Ted Bosworth/MDedge News

“Why are PERTs needed? Pulmonary embolism patients are like snowflakes. No two are the same,” explained Richard Channick, MD, director of the pulmonary vascular disease program, University of California, Los Angeles.

Patient variability is an issue because algorithms for pulmonary embolism (PE) often differ at the point of diagnosis, such as the emergency department or intensive are unit, according to Dr. Channick, who was present when the first PERT was created in 2012 at Massachusetts General Hospital (MGH) in Boston. In addition, treatment algorithms can seem complex at a time when patients are deteriorating quickly.

“The treatment algorithms always say consider this or consider that, and then you get a recommendation with a 2B grade of evidence. So what do you do?” Dr. Channick asked, “This has really been crying for an organized approach.”

PERTs were created to fill this need. In most centers, PERTs are organized to respond to a diagnosis of PE wherever it occurs in the hospital. The goal is rapid activation of a team of experts who can reach a single consensus recommendation.

At MGH and UCLA, a similar relatively simple scheme has been created to guide physicians on how to activate the PERT and which situations make this appropriate.

“A big part of the PERT value has been our ability to conduct a real-time virtual consultation where we leverage online technology to look at images together in order to agree on a strategy,” Dr. Channick explained.

Although frequently asked what specialists are needed for an effective PERT, Dr. Channick said it depends on institutional structures, the types of specialists available, and, in some cases, the specific characteristics of the patient. In many situations, a pulmonary vascular specialist and an interventional radiologist might be sufficient. In others, team members might include some combination of an interventional cardiologist, a cardiac surgeon, and a hematologist.

It is also appropriate to include clinicians likely to participate in care following acute treatment of the PE. “One of the most critical values to PERT is the ability to systematically follow patients” after the PE is treated, Dr. Channick said.

So far, there are no data to confirm patients managed with PERT achieve better outcomes than those who are not. Reductions in mortality, length of stay, and costs are reasonably anticipated and might eventually be demonstrated, but Dr. Channick said that PERTs already have value.

“I think the efficiency of care is important,”he said. He called PERT a “one-stop shopping” approach to ensuring that multiple strategies are considered systematically.

There are many anecdotal examples of the benefits of shared decision-making for PE treatment. In one, a pulmonary specialist in a PERT team narrowly averted a planned thrombolysis in a patient diagnosed with PE who was actually found to have severe pulmonary fibrosis, according to Dr. Channick.

Not least important, the shared decision-making of a PERT could relieve the burden of difficult choices in complex situations. Bad outcomes in PE can be unavoidable even with optimal therapy.

“To me personally, a very important benefit of being part of a PERT is the feeling that we are all in it together,” Dr. Channick said. “Patients can go from being pretty stable to being dead very quickly.”

The PERT Consortium has sponsored an annual meeting on PE since 2015. It also maintains an ongoing registry for PE data from member institutions. These data are expected to have increasing value for comparing the impact of patient characteristics, treatment strategies, and other variables on outcomes.

For clinicians who are uncertain whether the PE incidence at their institution justifies a PERT, Dr. Channick had some advice. “If you build it, they will clot,” he said, meaning that due to the frequency of PE, a PERT will generally have plenty of work once created.

SOURCE: VEITHSYMPOSIUM

NEW YORK – Seven years after the formation of the first pulmonary embolism response team (PERT), more than 100 institutions have joined the PERT Consortium, which was created to guide care and research for this thrombotic complication, according to a status report at a symposium on vascular and endovascular issues sponsored by the Cleveland Clinic Foundation.

Ted Bosworth/MDedge News

“Why are PERTs needed? Pulmonary embolism patients are like snowflakes. No two are the same,” explained Richard Channick, MD, director of the pulmonary vascular disease program, University of California, Los Angeles.

Patient variability is an issue because algorithms for pulmonary embolism (PE) often differ at the point of diagnosis, such as the emergency department or intensive are unit, according to Dr. Channick, who was present when the first PERT was created in 2012 at Massachusetts General Hospital (MGH) in Boston. In addition, treatment algorithms can seem complex at a time when patients are deteriorating quickly.

“The treatment algorithms always say consider this or consider that, and then you get a recommendation with a 2B grade of evidence. So what do you do?” Dr. Channick asked, “This has really been crying for an organized approach.”

PERTs were created to fill this need. In most centers, PERTs are organized to respond to a diagnosis of PE wherever it occurs in the hospital. The goal is rapid activation of a team of experts who can reach a single consensus recommendation.

At MGH and UCLA, a similar relatively simple scheme has been created to guide physicians on how to activate the PERT and which situations make this appropriate.

“A big part of the PERT value has been our ability to conduct a real-time virtual consultation where we leverage online technology to look at images together in order to agree on a strategy,” Dr. Channick explained.

Although frequently asked what specialists are needed for an effective PERT, Dr. Channick said it depends on institutional structures, the types of specialists available, and, in some cases, the specific characteristics of the patient. In many situations, a pulmonary vascular specialist and an interventional radiologist might be sufficient. In others, team members might include some combination of an interventional cardiologist, a cardiac surgeon, and a hematologist.

It is also appropriate to include clinicians likely to participate in care following acute treatment of the PE. “One of the most critical values to PERT is the ability to systematically follow patients” after the PE is treated, Dr. Channick said.

So far, there are no data to confirm patients managed with PERT achieve better outcomes than those who are not. Reductions in mortality, length of stay, and costs are reasonably anticipated and might eventually be demonstrated, but Dr. Channick said that PERTs already have value.

“I think the efficiency of care is important,”he said. He called PERT a “one-stop shopping” approach to ensuring that multiple strategies are considered systematically.

There are many anecdotal examples of the benefits of shared decision-making for PE treatment. In one, a pulmonary specialist in a PERT team narrowly averted a planned thrombolysis in a patient diagnosed with PE who was actually found to have severe pulmonary fibrosis, according to Dr. Channick.

Not least important, the shared decision-making of a PERT could relieve the burden of difficult choices in complex situations. Bad outcomes in PE can be unavoidable even with optimal therapy.

“To me personally, a very important benefit of being part of a PERT is the feeling that we are all in it together,” Dr. Channick said. “Patients can go from being pretty stable to being dead very quickly.”

The PERT Consortium has sponsored an annual meeting on PE since 2015. It also maintains an ongoing registry for PE data from member institutions. These data are expected to have increasing value for comparing the impact of patient characteristics, treatment strategies, and other variables on outcomes.

For clinicians who are uncertain whether the PE incidence at their institution justifies a PERT, Dr. Channick had some advice. “If you build it, they will clot,” he said, meaning that due to the frequency of PE, a PERT will generally have plenty of work once created.

SOURCE: VEITHSYMPOSIUM

NEW YORK – Seven years after the formation of the first pulmonary embolism response team (PERT), more than 100 institutions have joined the PERT Consortium, which was created to guide care and research for this thrombotic complication, according to a status report at a symposium on vascular and endovascular issues sponsored by the Cleveland Clinic Foundation.

Ted Bosworth/MDedge News

“Why are PERTs needed? Pulmonary embolism patients are like snowflakes. No two are the same,” explained Richard Channick, MD, director of the pulmonary vascular disease program, University of California, Los Angeles.

Patient variability is an issue because algorithms for pulmonary embolism (PE) often differ at the point of diagnosis, such as the emergency department or intensive are unit, according to Dr. Channick, who was present when the first PERT was created in 2012 at Massachusetts General Hospital (MGH) in Boston. In addition, treatment algorithms can seem complex at a time when patients are deteriorating quickly.

“The treatment algorithms always say consider this or consider that, and then you get a recommendation with a 2B grade of evidence. So what do you do?” Dr. Channick asked, “This has really been crying for an organized approach.”

PERTs were created to fill this need. In most centers, PERTs are organized to respond to a diagnosis of PE wherever it occurs in the hospital. The goal is rapid activation of a team of experts who can reach a single consensus recommendation.

At MGH and UCLA, a similar relatively simple scheme has been created to guide physicians on how to activate the PERT and which situations make this appropriate.

“A big part of the PERT value has been our ability to conduct a real-time virtual consultation where we leverage online technology to look at images together in order to agree on a strategy,” Dr. Channick explained.

Although frequently asked what specialists are needed for an effective PERT, Dr. Channick said it depends on institutional structures, the types of specialists available, and, in some cases, the specific characteristics of the patient. In many situations, a pulmonary vascular specialist and an interventional radiologist might be sufficient. In others, team members might include some combination of an interventional cardiologist, a cardiac surgeon, and a hematologist.

It is also appropriate to include clinicians likely to participate in care following acute treatment of the PE. “One of the most critical values to PERT is the ability to systematically follow patients” after the PE is treated, Dr. Channick said.

So far, there are no data to confirm patients managed with PERT achieve better outcomes than those who are not. Reductions in mortality, length of stay, and costs are reasonably anticipated and might eventually be demonstrated, but Dr. Channick said that PERTs already have value.

“I think the efficiency of care is important,”he said. He called PERT a “one-stop shopping” approach to ensuring that multiple strategies are considered systematically.

There are many anecdotal examples of the benefits of shared decision-making for PE treatment. In one, a pulmonary specialist in a PERT team narrowly averted a planned thrombolysis in a patient diagnosed with PE who was actually found to have severe pulmonary fibrosis, according to Dr. Channick.

Not least important, the shared decision-making of a PERT could relieve the burden of difficult choices in complex situations. Bad outcomes in PE can be unavoidable even with optimal therapy.

“To me personally, a very important benefit of being part of a PERT is the feeling that we are all in it together,” Dr. Channick said. “Patients can go from being pretty stable to being dead very quickly.”

The PERT Consortium has sponsored an annual meeting on PE since 2015. It also maintains an ongoing registry for PE data from member institutions. These data are expected to have increasing value for comparing the impact of patient characteristics, treatment strategies, and other variables on outcomes.

For clinicians who are uncertain whether the PE incidence at their institution justifies a PERT, Dr. Channick had some advice. “If you build it, they will clot,” he said, meaning that due to the frequency of PE, a PERT will generally have plenty of work once created.

SOURCE: VEITHSYMPOSIUM

Viral bronchiolitis is the most common indication for infant hospitalization in the United States.¹ The treatment mainstay remains supportive care, including supplemental oxygen when indicated.¹ High flow nasal cannula (HFNC) therapy delivers humidified, heated air blended with oxygen, allowing much higher flow rates than standard nasal cannula therapy and is being used more frequently in inpatient settings.

Infants and toddlers with bronchiolitis develop increased work of breathing to preserve oxygenation and ventilation in the setting of altered airway resistance and lung compliance.^2,3 In addition to oxygen supplementation, HFNC is used to reduce work of breathing through several mechanisms:^2-6 (1) Nasopharyngeal dead space washout clears oxygen-depleted gas at the end of expiration, facilitating alveolar ventilation (ie, carbon dioxide retention improves); (2) High flow rates match increased inspiratory flow demands of acutely ill patients, reducing nasopharyngeal inspiratory resistance and optimizing dead space washout, thus decreasing work of breathing; (3) Adequate flow rates generate distending pressure, which prevents pharyngeal collapse, supports lung recruitment, and reduces respiratory effort (demonstrated in younger infants); and (4) HFNC systems heat and humidify the breathing gas, reducing the metabolic work required to condition cool, dry gas and improving conductance and pulmonary compliance.^2-5

HFNC therapy is used more commonly in acute care units despite limited literature on its effectiveness outside the intensive care unit (ICU).^7,8 We asked the question, “Does use of HFNC therapy for infants with bronchiolitis hospitalized in acute care units result in improved outcomes when compared with standard nasal cannula oxygen therapy, including length of stay (LOS), oxygen therapy duration, and preventing escalations of care such as ICU transfer, positive pressure ventilation, and intubation?” Also, do published studies provide guidance for the initiation and management of HFNC? We focused our search on studies published in the last five years that included patients with bronchiolitis treated with HFNC outside the ICU; here, we review those studies most relevant to pediatric hospitalists.

No guideline exists for initiating flow or fraction of inspired oxygen (FiO₂). HFNC may be initiated for hypoxia, increased work of breathing, or both in patients with bronchiolitis. To achieve optimal dead space washout, inspiratory flow, and distending pressure, initial flow rates should be 1.5 to 2 L/kg/min, particularly for infants and young children.^2-5 Weiler et al.³ evaluated the breathing effort of ICU patients at 0.5, 1, 1.5, and 2 L/kg/min and found optimal flow rates for improved work of breathing were 1.5-2 L/kg/min. The smallest patients, ≤8 kg, saw the greatest benefit, a finding likely explained by larger anatomic dead space in infants/small children compared with older children.³ For older/larger children (>20 kg), an initial flow closer to 1 L/kg/min is often appropriate.⁵ When used for hypoxia, initiating flow without supplemental FiO₂ may improve oxygenation by flushing nasopharyngeal dead space. FiO₂ should be titrated to achieve the goal set by the treatment team, often ≥90%. Improvement in heart rate and peripheral oxygen saturation (SpO₂) can be observed within 60 minutes of initiating HFNC in patients responsive to therapy.⁶

HFNC therapy is safe when used correctly.^6,9,10Potential adverse effects include pneumothorax, pressure injury, mucosal injury/bleeding, and delayed escalation to invasive ventilation. While difficult to quantify, recent studies report low rates or no serious HFNC complications. For example, only 2 of 1,127 patients supported with HFNC developed a pneumothorax and neither required evacuation.^2,9-12

Inclusion criteria and HFNC protocols vary among published studies. Most HFNC protocols reviewed may not have optimally supported all of the patients in their HFNC groups, often by limiting flow to <2 L/kg/min.^6-9,11,12 These variables may explain the disparate results, with some studies demonstrating apparent benefits and others no difference.^7,9,10,12

Two studies of infants with bronchiolitis showed HFNC therapy may prevent ICU transfer, but this benefit may be limited to rescue when standard oxygen therapy fails, rather than as a superior initial support modality.^7,9 Kepreotes et al.⁹ reported a single-center, randomized controlled trial comparing HFNC with standard oxygen therapy with 101 patients in each treatment arm. The primary outcome, median time to wean off oxygen, was not significantly different between the two groups: 24 hours (95% CI: 18-28) in the HFNC group versus 20 hours in the standard therapy group (95% CI: 17-34). The HFNC group had fewer treatment failures (abnormal heart rate, respiratory rate, SpO₂ <90%, or severe respiratory distress score while on maximum therapy) than the standard therapy group, and 20 (63%) of the 33 patients who failed standard therapy were rescued with HFNC, avoiding transfer to the ICU. Fourteen patients from the HFNC group and 12 from the standard oxygen group required transfer to the ICU for support escalation. Although this study did not show a significant difference in oxygen weaning time between groups, it appears to support HFNC use as a rescue modality to reduce or prevent ICU transfer.⁹ Franklin et al.¹⁰ conducted a multicenter, randomized, controlled trial to compare standard nasal cannula oxygen therapy with HFNC (2 L/kg/min) in 1,472 patients. Patients receiving HFNC had lower care escalation rates due to treatment failure, defined as the presence of at least three of four clinical criteria and the clinician determining escalation was indicated. Oxygen therapy duration, ICU admission rates, and LOS were not significantly different between groups. Similar to the previous study, a large portion of the standard therapy patients who failed treatment (102 of 167) crossed over to the HFNC arm in an attempt to avoid ICU transfer. Twelve patients required intubation: 8 (1%) receiving HFNC and 4 (0.5%) receiving the standard therapy.¹⁰

Two additional studies, both with study design limitations, did not demonstrate differences in ICU transfer rates and had variable differences in outcomes. Riese et al.⁷ retrospectively assessed HFNC use outside the ICU at one institution and included 936 patients admitted before and 1,001 patients admitted after HFNC guideline implementation on the wards. Flow rates were based on age and not weight. They found no difference in LOS, ICU transfer rate, ICU LOS, intubation rates, or 30-day readmission rates, though HFNC use increased over time. The HFNC guideline is a potentially significant limitation as it may not have provided optimal flow rates to all subjects given it was based on age rather than weight. Milani et al.¹² performed a single-center observational study of 36 infants aged <12 months, treated for bronchiolitis on the ward, who were informally assigned to HFNC or standard therapy based upon HFNC device availability. HFNC flow rate was determined by the equation: L/min = 8 mL/kg × respiratory rate × 0.3. Using mean weight and respiratory rate for patients in this group, it appears patients in the HFNC group were treated with flow rates less than the 1.5-2 L/kg/min recommended to be effective.^2,3,12 Despite this, clinical improvement was faster in the HFNC group, including respiratory rate and effort, ability to feed, days on oxygen supplementation, and hospital LOS. ICU admission was not different between the two groups.¹² The Table compares the four studies discussed above.

Many children’s hospitals have extended the use of HFNC outside the ICU for children with bronchiolitis despite the paucity of evidence demonstrating its benefit over standard flow oxygen. Given variation in protocols, study designs, outcomes, and number of patients studied, it is difficult to assess its efficacy outside the ICU. However, based on the studies reviewed herein, HFNC therapy does not appear to decrease LOS, time on oxygen, or escalations of care, such as ICU transfers, positive pressure ventilation, or intubation, when used as a primary therapy.^7,9,11,12 Future research will ideally use optimal flow rates to determine the effectiveness of HFNC on acute care units. Although not addressed in the above studies, additional benefits to be considered in future studies include: (1) increased critical care capacity by allowing patients to be supported on the floor and (2) the ability for patients to remain closer to home when HFNC is used in the community hospital setting.

In each of the large, randomized studies reviewed, most (66%-75%) patients treated with standard low flow oxygen were supported successfully and did not require escalation to HFNC.^9,10 Hospitalists should continue to use standard low flow oxygen as first-line respiratory support for patients with bronchiolitis.¹ No evidence supports the use of HFNC therapy early in a child’s inpatient course; rather, it should be used when standard oxygen therapy fails. Future research should focus on better elucidating which patients will benefit most from HFNC to prevent overuse.

Viral bronchiolitis is the most common indication for infant hospitalization in the United States.¹ The treatment mainstay remains supportive care, including supplemental oxygen when indicated.¹ High flow nasal cannula (HFNC) therapy delivers humidified, heated air blended with oxygen, allowing much higher flow rates than standard nasal cannula therapy and is being used more frequently in inpatient settings.

Infants and toddlers with bronchiolitis develop increased work of breathing to preserve oxygenation and ventilation in the setting of altered airway resistance and lung compliance.^2,3 In addition to oxygen supplementation, HFNC is used to reduce work of breathing through several mechanisms:^2-6 (1) Nasopharyngeal dead space washout clears oxygen-depleted gas at the end of expiration, facilitating alveolar ventilation (ie, carbon dioxide retention improves); (2) High flow rates match increased inspiratory flow demands of acutely ill patients, reducing nasopharyngeal inspiratory resistance and optimizing dead space washout, thus decreasing work of breathing; (3) Adequate flow rates generate distending pressure, which prevents pharyngeal collapse, supports lung recruitment, and reduces respiratory effort (demonstrated in younger infants); and (4) HFNC systems heat and humidify the breathing gas, reducing the metabolic work required to condition cool, dry gas and improving conductance and pulmonary compliance.^2-5

HFNC therapy is used more commonly in acute care units despite limited literature on its effectiveness outside the intensive care unit (ICU).^7,8 We asked the question, “Does use of HFNC therapy for infants with bronchiolitis hospitalized in acute care units result in improved outcomes when compared with standard nasal cannula oxygen therapy, including length of stay (LOS), oxygen therapy duration, and preventing escalations of care such as ICU transfer, positive pressure ventilation, and intubation?” Also, do published studies provide guidance for the initiation and management of HFNC? We focused our search on studies published in the last five years that included patients with bronchiolitis treated with HFNC outside the ICU; here, we review those studies most relevant to pediatric hospitalists.

No guideline exists for initiating flow or fraction of inspired oxygen (FiO₂). HFNC may be initiated for hypoxia, increased work of breathing, or both in patients with bronchiolitis. To achieve optimal dead space washout, inspiratory flow, and distending pressure, initial flow rates should be 1.5 to 2 L/kg/min, particularly for infants and young children.^2-5 Weiler et al.³ evaluated the breathing effort of ICU patients at 0.5, 1, 1.5, and 2 L/kg/min and found optimal flow rates for improved work of breathing were 1.5-2 L/kg/min. The smallest patients, ≤8 kg, saw the greatest benefit, a finding likely explained by larger anatomic dead space in infants/small children compared with older children.³ For older/larger children (>20 kg), an initial flow closer to 1 L/kg/min is often appropriate.⁵ When used for hypoxia, initiating flow without supplemental FiO₂ may improve oxygenation by flushing nasopharyngeal dead space. FiO₂ should be titrated to achieve the goal set by the treatment team, often ≥90%. Improvement in heart rate and peripheral oxygen saturation (SpO₂) can be observed within 60 minutes of initiating HFNC in patients responsive to therapy.⁶

HFNC therapy is safe when used correctly.^6,9,10Potential adverse effects include pneumothorax, pressure injury, mucosal injury/bleeding, and delayed escalation to invasive ventilation. While difficult to quantify, recent studies report low rates or no serious HFNC complications. For example, only 2 of 1,127 patients supported with HFNC developed a pneumothorax and neither required evacuation.^2,9-12

Inclusion criteria and HFNC protocols vary among published studies. Most HFNC protocols reviewed may not have optimally supported all of the patients in their HFNC groups, often by limiting flow to <2 L/kg/min.^6-9,11,12 These variables may explain the disparate results, with some studies demonstrating apparent benefits and others no difference.^7,9,10,12

Two studies of infants with bronchiolitis showed HFNC therapy may prevent ICU transfer, but this benefit may be limited to rescue when standard oxygen therapy fails, rather than as a superior initial support modality.^7,9 Kepreotes et al.⁹ reported a single-center, randomized controlled trial comparing HFNC with standard oxygen therapy with 101 patients in each treatment arm. The primary outcome, median time to wean off oxygen, was not significantly different between the two groups: 24 hours (95% CI: 18-28) in the HFNC group versus 20 hours in the standard therapy group (95% CI: 17-34). The HFNC group had fewer treatment failures (abnormal heart rate, respiratory rate, SpO₂ <90%, or severe respiratory distress score while on maximum therapy) than the standard therapy group, and 20 (63%) of the 33 patients who failed standard therapy were rescued with HFNC, avoiding transfer to the ICU. Fourteen patients from the HFNC group and 12 from the standard oxygen group required transfer to the ICU for support escalation. Although this study did not show a significant difference in oxygen weaning time between groups, it appears to support HFNC use as a rescue modality to reduce or prevent ICU transfer.⁹ Franklin et al.¹⁰ conducted a multicenter, randomized, controlled trial to compare standard nasal cannula oxygen therapy with HFNC (2 L/kg/min) in 1,472 patients. Patients receiving HFNC had lower care escalation rates due to treatment failure, defined as the presence of at least three of four clinical criteria and the clinician determining escalation was indicated. Oxygen therapy duration, ICU admission rates, and LOS were not significantly different between groups. Similar to the previous study, a large portion of the standard therapy patients who failed treatment (102 of 167) crossed over to the HFNC arm in an attempt to avoid ICU transfer. Twelve patients required intubation: 8 (1%) receiving HFNC and 4 (0.5%) receiving the standard therapy.¹⁰

Two additional studies, both with study design limitations, did not demonstrate differences in ICU transfer rates and had variable differences in outcomes. Riese et al.⁷ retrospectively assessed HFNC use outside the ICU at one institution and included 936 patients admitted before and 1,001 patients admitted after HFNC guideline implementation on the wards. Flow rates were based on age and not weight. They found no difference in LOS, ICU transfer rate, ICU LOS, intubation rates, or 30-day readmission rates, though HFNC use increased over time. The HFNC guideline is a potentially significant limitation as it may not have provided optimal flow rates to all subjects given it was based on age rather than weight. Milani et al.¹² performed a single-center observational study of 36 infants aged <12 months, treated for bronchiolitis on the ward, who were informally assigned to HFNC or standard therapy based upon HFNC device availability. HFNC flow rate was determined by the equation: L/min = 8 mL/kg × respiratory rate × 0.3. Using mean weight and respiratory rate for patients in this group, it appears patients in the HFNC group were treated with flow rates less than the 1.5-2 L/kg/min recommended to be effective.^2,3,12 Despite this, clinical improvement was faster in the HFNC group, including respiratory rate and effort, ability to feed, days on oxygen supplementation, and hospital LOS. ICU admission was not different between the two groups.¹² The Table compares the four studies discussed above.

Many children’s hospitals have extended the use of HFNC outside the ICU for children with bronchiolitis despite the paucity of evidence demonstrating its benefit over standard flow oxygen. Given variation in protocols, study designs, outcomes, and number of patients studied, it is difficult to assess its efficacy outside the ICU. However, based on the studies reviewed herein, HFNC therapy does not appear to decrease LOS, time on oxygen, or escalations of care, such as ICU transfers, positive pressure ventilation, or intubation, when used as a primary therapy.^7,9,11,12 Future research will ideally use optimal flow rates to determine the effectiveness of HFNC on acute care units. Although not addressed in the above studies, additional benefits to be considered in future studies include: (1) increased critical care capacity by allowing patients to be supported on the floor and (2) the ability for patients to remain closer to home when HFNC is used in the community hospital setting.

In each of the large, randomized studies reviewed, most (66%-75%) patients treated with standard low flow oxygen were supported successfully and did not require escalation to HFNC.^9,10 Hospitalists should continue to use standard low flow oxygen as first-line respiratory support for patients with bronchiolitis.¹ No evidence supports the use of HFNC therapy early in a child’s inpatient course; rather, it should be used when standard oxygen therapy fails. Future research should focus on better elucidating which patients will benefit most from HFNC to prevent overuse.

Viral bronchiolitis is the most common indication for infant hospitalization in the United States.¹ The treatment mainstay remains supportive care, including supplemental oxygen when indicated.¹ High flow nasal cannula (HFNC) therapy delivers humidified, heated air blended with oxygen, allowing much higher flow rates than standard nasal cannula therapy and is being used more frequently in inpatient settings.

Infants and toddlers with bronchiolitis develop increased work of breathing to preserve oxygenation and ventilation in the setting of altered airway resistance and lung compliance.^2,3 In addition to oxygen supplementation, HFNC is used to reduce work of breathing through several mechanisms:^2-6 (1) Nasopharyngeal dead space washout clears oxygen-depleted gas at the end of expiration, facilitating alveolar ventilation (ie, carbon dioxide retention improves); (2) High flow rates match increased inspiratory flow demands of acutely ill patients, reducing nasopharyngeal inspiratory resistance and optimizing dead space washout, thus decreasing work of breathing; (3) Adequate flow rates generate distending pressure, which prevents pharyngeal collapse, supports lung recruitment, and reduces respiratory effort (demonstrated in younger infants); and (4) HFNC systems heat and humidify the breathing gas, reducing the metabolic work required to condition cool, dry gas and improving conductance and pulmonary compliance.^2-5

HFNC therapy is used more commonly in acute care units despite limited literature on its effectiveness outside the intensive care unit (ICU).^7,8 We asked the question, “Does use of HFNC therapy for infants with bronchiolitis hospitalized in acute care units result in improved outcomes when compared with standard nasal cannula oxygen therapy, including length of stay (LOS), oxygen therapy duration, and preventing escalations of care such as ICU transfer, positive pressure ventilation, and intubation?” Also, do published studies provide guidance for the initiation and management of HFNC? We focused our search on studies published in the last five years that included patients with bronchiolitis treated with HFNC outside the ICU; here, we review those studies most relevant to pediatric hospitalists.

No guideline exists for initiating flow or fraction of inspired oxygen (FiO₂). HFNC may be initiated for hypoxia, increased work of breathing, or both in patients with bronchiolitis. To achieve optimal dead space washout, inspiratory flow, and distending pressure, initial flow rates should be 1.5 to 2 L/kg/min, particularly for infants and young children.^2-5 Weiler et al.³ evaluated the breathing effort of ICU patients at 0.5, 1, 1.5, and 2 L/kg/min and found optimal flow rates for improved work of breathing were 1.5-2 L/kg/min. The smallest patients, ≤8 kg, saw the greatest benefit, a finding likely explained by larger anatomic dead space in infants/small children compared with older children.³ For older/larger children (>20 kg), an initial flow closer to 1 L/kg/min is often appropriate.⁵ When used for hypoxia, initiating flow without supplemental FiO₂ may improve oxygenation by flushing nasopharyngeal dead space. FiO₂ should be titrated to achieve the goal set by the treatment team, often ≥90%. Improvement in heart rate and peripheral oxygen saturation (SpO₂) can be observed within 60 minutes of initiating HFNC in patients responsive to therapy.⁶

HFNC therapy is safe when used correctly.^6,9,10Potential adverse effects include pneumothorax, pressure injury, mucosal injury/bleeding, and delayed escalation to invasive ventilation. While difficult to quantify, recent studies report low rates or no serious HFNC complications. For example, only 2 of 1,127 patients supported with HFNC developed a pneumothorax and neither required evacuation.^2,9-12

Inclusion criteria and HFNC protocols vary among published studies. Most HFNC protocols reviewed may not have optimally supported all of the patients in their HFNC groups, often by limiting flow to <2 L/kg/min.^6-9,11,12 These variables may explain the disparate results, with some studies demonstrating apparent benefits and others no difference.^7,9,10,12

Two studies of infants with bronchiolitis showed HFNC therapy may prevent ICU transfer, but this benefit may be limited to rescue when standard oxygen therapy fails, rather than as a superior initial support modality.^7,9 Kepreotes et al.⁹ reported a single-center, randomized controlled trial comparing HFNC with standard oxygen therapy with 101 patients in each treatment arm. The primary outcome, median time to wean off oxygen, was not significantly different between the two groups: 24 hours (95% CI: 18-28) in the HFNC group versus 20 hours in the standard therapy group (95% CI: 17-34). The HFNC group had fewer treatment failures (abnormal heart rate, respiratory rate, SpO₂ <90%, or severe respiratory distress score while on maximum therapy) than the standard therapy group, and 20 (63%) of the 33 patients who failed standard therapy were rescued with HFNC, avoiding transfer to the ICU. Fourteen patients from the HFNC group and 12 from the standard oxygen group required transfer to the ICU for support escalation. Although this study did not show a significant difference in oxygen weaning time between groups, it appears to support HFNC use as a rescue modality to reduce or prevent ICU transfer.⁹ Franklin et al.¹⁰ conducted a multicenter, randomized, controlled trial to compare standard nasal cannula oxygen therapy with HFNC (2 L/kg/min) in 1,472 patients. Patients receiving HFNC had lower care escalation rates due to treatment failure, defined as the presence of at least three of four clinical criteria and the clinician determining escalation was indicated. Oxygen therapy duration, ICU admission rates, and LOS were not significantly different between groups. Similar to the previous study, a large portion of the standard therapy patients who failed treatment (102 of 167) crossed over to the HFNC arm in an attempt to avoid ICU transfer. Twelve patients required intubation: 8 (1%) receiving HFNC and 4 (0.5%) receiving the standard therapy.¹⁰

Two additional studies, both with study design limitations, did not demonstrate differences in ICU transfer rates and had variable differences in outcomes. Riese et al.⁷ retrospectively assessed HFNC use outside the ICU at one institution and included 936 patients admitted before and 1,001 patients admitted after HFNC guideline implementation on the wards. Flow rates were based on age and not weight. They found no difference in LOS, ICU transfer rate, ICU LOS, intubation rates, or 30-day readmission rates, though HFNC use increased over time. The HFNC guideline is a potentially significant limitation as it may not have provided optimal flow rates to all subjects given it was based on age rather than weight. Milani et al.¹² performed a single-center observational study of 36 infants aged <12 months, treated for bronchiolitis on the ward, who were informally assigned to HFNC or standard therapy based upon HFNC device availability. HFNC flow rate was determined by the equation: L/min = 8 mL/kg × respiratory rate × 0.3. Using mean weight and respiratory rate for patients in this group, it appears patients in the HFNC group were treated with flow rates less than the 1.5-2 L/kg/min recommended to be effective.^2,3,12 Despite this, clinical improvement was faster in the HFNC group, including respiratory rate and effort, ability to feed, days on oxygen supplementation, and hospital LOS. ICU admission was not different between the two groups.¹² The Table compares the four studies discussed above.

Many children’s hospitals have extended the use of HFNC outside the ICU for children with bronchiolitis despite the paucity of evidence demonstrating its benefit over standard flow oxygen. Given variation in protocols, study designs, outcomes, and number of patients studied, it is difficult to assess its efficacy outside the ICU. However, based on the studies reviewed herein, HFNC therapy does not appear to decrease LOS, time on oxygen, or escalations of care, such as ICU transfers, positive pressure ventilation, or intubation, when used as a primary therapy.^7,9,11,12 Future research will ideally use optimal flow rates to determine the effectiveness of HFNC on acute care units. Although not addressed in the above studies, additional benefits to be considered in future studies include: (1) increased critical care capacity by allowing patients to be supported on the floor and (2) the ability for patients to remain closer to home when HFNC is used in the community hospital setting.

In each of the large, randomized studies reviewed, most (66%-75%) patients treated with standard low flow oxygen were supported successfully and did not require escalation to HFNC.^9,10 Hospitalists should continue to use standard low flow oxygen as first-line respiratory support for patients with bronchiolitis.¹ No evidence supports the use of HFNC therapy early in a child’s inpatient course; rather, it should be used when standard oxygen therapy fails. Future research should focus on better elucidating which patients will benefit most from HFNC to prevent overuse.

High-flow nasal cannula (HFNC) oxygen therapy is effective in treating adults with acute hypoxemic respiratory failure, and to a lesser extent acute hypercapnic respiratory failure.^1-3 HFNC oxygen is capable of delivering oxygen with flows of 30-60 liters/minute, and can provide a high fraction of inspired oxygen, flush anatomic dead space, augment respiratory efforts, and provide mild continuous positive airway pressure effects. Several systematic reviews and meta-analyses have evaluated the effectiveness of HFNC oxygen and have shown modestly lower rates of intubation compared with conventional oxygen^4,5 and similar intubation rates compared with noninvasive positive pressure ventilation.^4-9 Although one randomized trial showed a lower risk of 90-day mortality for HFNC oxygen compared with either conventional oxygen or noninvasive positive pressure ventilation, several meta-analyses have shown no difference in intensive care unit (ICU) mortality.^4,6,8,10 The majority of studies have shown improvements in oxygenation, comfort, dyspnea scores, and breathing pattern with the initiation of HFNC oxygen.⁶

While the evidence to support the use of HFNC oxygen in patients with nonhypercapnic acute hypoxemic respiratory failure is growing, this evidence is based on patients enrolled in clinical trials who have no treatment limitations and consent to intubation if necessary. Indeed, several, if not all, randomized trials evaluating HFNC oxygen excluded patients who had do-not-intubate (DNI) or do-not-resuscitate (DNR) orders.^1,2,11 For patients with acute respiratory failure whose primary goal is not to extend life or utilize life support interventions such as invasive mechanical ventilation, HFNC oxygen may offer several benefits compared with other treatment options such as noninvasive positive pressure ventilation, conventional oxygen therapy, or palliative opioid therapy (Appendix Table 1). Determining which treatment options to use depends on the goals of care of the individual patient and the reasonable ability of a particular treatment to help the patient achieve those goals.

While a recent systematic review evaluated the existing evidence regarding the utility and outcomes of noninvasive positive pressure ventilation in adult patients with DNI orders,¹² a systematic review evaluating the evidence and rationale for HFNC oxygen in patients with DNI and/or DNR orders is lacking. Assessing such evidence is necessary to help clinicians and patients determine appropriate treatment choices and establish research priorities. Therefore, our primary objective was to determine what were the following outcomes: mortality, dyspnea, work of breathing, opioid doses, and quality of life in patients who received HFNC oxygen for acute respiratory failure and had a DNI and/or DNR order.

We conducted a systematic review of studies that evaluated patients who used HFNC oxygen for acute respiratory failure and had a DNI and/or DNR order. We reported the results using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statements.¹³ This review was registered with the PROSPERO registry, CRD42017059914.

We included studies that enrolled patients who were (1) hospitalized, (2) >18 years old, (3) had an acute respiratory failure of any cause, (4) received HFNC oxygen, and (5) had a DNI or DNR or comfort measures only order. We included publications of all study designs (interventional, observational, and posthoc analyses) and all languages. We excluded studies that enrolled <5 patients. If necessary, we contacted the authors of the included studies for additional information.

Our search strategy included the following databases from inception to October 14, 2018: PubMed, MEDLINE, CINAHL, MICROMEDEX, EMBASE, Web of Science, and Scopus. The database-specific search strategy was developed using an experienced librarian (Appendix Table 2). In addition, we screened the reference lists of systematic reviews as well as the included studies to find additional relevant articles. Two authors (AM, MEW) independently assessed the inclusion criteria of the titles and abstracts that were identified in the search. In addition, these two authors abstracted relevant data of the included studies.

Using the search strategy, we identified 2,757 citations and included 301 of these in the full-text review (Figure). We included six studies, which enrolled 293 patients in the final systematic review. Table 1 summarizes the characteristics of the included investigations, all of which were observational studies.^15-20 The studies were conducted in the United States of America (n = 3), Europe (n = 2), and Asia (n = 1). Two studies were conducted in the general ICU populations and included patients with hypoxemic respiratory failure only. Four studies were conducted in cancer populations in the hospital wards or ICU and did not specify the type of respiratory failure (hypoxemic versus hypercapnic). Two studies included patients with DNI orders only.^15,20 One study included patients with DNR orders only (DNI orders were excluded).¹⁷ Three studies included patients with both DNR and DNI orders.^16,18,19 The numbers of enrolled patients with treatment limitations were generally low, with the two largest studies including 101 patients each on HFNC oxygen.^18,19

Risk of Bias

All included studies had a high risk of bias (Table 2). A high risk of bias was suggested because the investigations were single-center studies with unclear patient selection methods, did not explicitly report how decisions to limit treatments were made, and did not explicitly differentiate and separately analyze patients with “comfort measures only” goals of care.

Mortality

The hospital mortality rates of patients with DNI and/or DNR orders receiving HFNC were variable and ranged from 40% to 87%. In the two studies enrolling general ICU patient populations, the hospital mortality rates ranged from 40% to 60%. In the four studies enrolling patients with active malignancy, the hospital mortality rates ranged from 75% to 87%. No studies compared mortality rates with and without DNI and/or DNR orders.

Dyspnea, Work of Breathing, and Reduction in Opioid Doses

The impact of HFNC oxygen on symptom relief was reported in one retrospective observational study (published as a conference abstract only to date), which compared the effect of HFNC oxygen (n = 101) with conventional oxygen (n = 110).¹⁸ At first evaluation after hospital admission to a palliative care unit (after the patients had previously been started on either conventional oxygen or high-flow oxygen), patients in the HFNC oxygen group had worse (higher) dyspnea scores compared with patients who used conventional oxygen (Edmonton Symptom Assessment Scale score of 7.5 versus 5, P < .001). At follow-up, approximately 24 hours after admission to the hospital palliative care unit, there was no difference in the change of dyspnea between the HFNC oxygen group (dyspnea score change of 0) and the conventional oxygen group (dyspnea score change of −1, P = .18. In the same study, there was also no significant difference in the morphine dose requirement in each group, and exact doses were not reported.

Two studies reported improvement in oxygen saturation and respiratory rate after HFNC oxygen initiation (compared with before HFNC initiation).^16,20 Oxygen saturation increased from 89% to 95%, P < .01, in one study and 92% to 97%, P < .01, in a second study. The respiratory rate decreased from 31 to 25 breaths/minute in one study, and from 28 to 25 breaths/minute in a second study (both P < .01).

Quality of Life

No studies evaluated the quality of life of survivors.

Secondary Outcomes

Transition to Noninvasive Positive Pressure Ventilation

The proportion of patients who transitioned from HFNC oxygen to NPPV was relatively low in the two studies that reported this outcome, ranging from 0%²⁰ to 18%.¹⁶ In one observational study of a general ICU population, 9/50 (18%) of patients transitioned from HFNC oxygen to NPPV. There was no statistically significant difference in hospital mortality rates among those who progressed to NPPV (67%) versus those who did not progress to NPPV (58%), P = .72.

Tolerance of HFNC and Adverse Events

HFNC oxygen was generally well tolerated based on the assessment of three studies (Table 1). One study reported no adverse events,¹⁶ one study reported that HFNC oxygen had to be discontinued because of nasal discomfort in 1% of patients,¹⁹ and a second study reported that HFNC oxygen had to be discontinued because of agitation in 4% of patients.²⁰

Quality of Death in Nonsurvivors

No studies evaluated the quality of death in those patients who died.

In this systematic review of six studies, all with a high risk of bias, a significant proportion of patients with a DNI and/or DNR order who used HFNC oxygen survived to hospital discharge. Oxygen saturation and respiratory rate consistently improved in the three studies that reported these outcomes. Only one study (published as a conference abstract only to date),¹⁸ however, measured patient-important outcomes related to symptom management and found no significant difference in dyspnea or morphine dose requirements in patients on HFNC oxygen compared with patients on conventional oxygen. HFNC oxygen was generally well tolerated and only had to be stopped in <5% of patients due to intolerance. We found no studies that assessed the quality of life in survivors or the quality of death in nonsurvivors.

Based on the limited evidence in the included studies, HFNC may be a viable treatment option for patients with preset treatment limitations who have acute respiratory failure—with potential benefits of improved oxygenation, decreased respiratory rates, and hospital survival in a proportion of patients. Nevertheless, this systematic review highlights the vast paucity of data available to guide the use of HFNC oxygen in patients with treatment limitations and acute respiratory failure. Only a few studies, which were at high risk of bias, have been conducted on this topic to date. There is an inadequate evidence base to evaluate the comparative effectiveness of HFNC oxygen (versus NPPV versus conventional oxygen versus palliative opioids) in patients with DNI orders or comfort measures only orders.

Our review included two studies that evaluated the comparative effectiveness of HFNC oxygen in patients with DNI and/or DNR orders. The first retrospective observational study compared HFNC oxygen with conventional oxygen in patients with DNR and DNI orders and malignancy—and found no change in dyspnea—but did note an increase in mortality with HFNC oxygen (76% versus 51%).¹⁸ The second observational study compared HFNC oxygen with NPPV in patients with DNR orders with malignancy noted no difference in mortality.¹⁷ In patients with full-code orders, systematic reviews have shown that HFNC oxygen (compared with conventional oxygen) was associated with possible reductions in intubation rates, respiratory rates, and improvements in oxygenation—with no difference in mortality, dyspnea, patient comfort, or ICU/hospital length of stay. Compared with NPPV, HFNC oxygen was associated with similar rates of intubation and mortality.^4-6,21

Future studies in patients with acute respiratory failure and DNI and/or DNR orders should identify which treatment modality (HFNC oxygen compared with other modalities, such as NPPV, conventional oxygen, with or without palliative opioids) impacts outcomes, such as dyspnea reduction while maintaining an alert mental status, short- and long-term quality of life in survivors, and quality of death in nonsurvivors. Future studies should also identify the optimal treatment pathway to utilize when patients using HFNC oxygen fail this therapy (eg, transition to NPPV versus intensifying palliative opioids) as well as the optimal process to withdraw palliative HFNC oxygen.²² Identifying which patient populations may benefit from different treatment pathways should also be considered as different treatment strategies may be more beneficial in different patient populations (eg, based on cause and severity of acute respiratory failure). In addition, it should be noted that the primary goal of care might affect which outcomes are the most important to measure. While patients with comfort measures only, orders usually have a primary goal to prepare for a high-quality death, patients with DNI and/or DNR orders (but without comfort measures only orders) may have a primary goal to survive—but with the desire not to endure the high burden of intubation and mechanical ventilation if it became necessary. Finally, future studies should utilize high-quality study designs (eg, randomized controlled trials) that enable robust evaluation of comparative effectiveness of clinically relevant treatment strategies.

While several previous systematic reviews have evaluated the efficacy of HFNC in patients with acute respiratory failure without preset limitations on life support; to our knowledge, this is the first systematic review to assess outcomes in patients rigorously with preset treatment limitations. Our review is, however, limited by the high risk of bias of the studies that were included (single-center nature, retrospective observational study designs, small sample sizes, and lack of a description of how DNI and/or DNR statuses were determined) as well as the small number of studies available to be included.

This systematic review points to a significant evidence gap in our understanding of the role for HFNC oxygen (compared with other acceptable alternative treatment strategies) in adult patients with acute respiratory failure who have DNI and/or DNR orders. Further high-quality research is needed to explore these unanswered questions in an effort to best treat, guide, and engage in optimal end-of-life decision making among patients with acute respiratory failure.

High-flow nasal cannula (HFNC) oxygen therapy is effective in treating adults with acute hypoxemic respiratory failure, and to a lesser extent acute hypercapnic respiratory failure.^1-3 HFNC oxygen is capable of delivering oxygen with flows of 30-60 liters/minute, and can provide a high fraction of inspired oxygen, flush anatomic dead space, augment respiratory efforts, and provide mild continuous positive airway pressure effects. Several systematic reviews and meta-analyses have evaluated the effectiveness of HFNC oxygen and have shown modestly lower rates of intubation compared with conventional oxygen^4,5 and similar intubation rates compared with noninvasive positive pressure ventilation.^4-9 Although one randomized trial showed a lower risk of 90-day mortality for HFNC oxygen compared with either conventional oxygen or noninvasive positive pressure ventilation, several meta-analyses have shown no difference in intensive care unit (ICU) mortality.^4,6,8,10 The majority of studies have shown improvements in oxygenation, comfort, dyspnea scores, and breathing pattern with the initiation of HFNC oxygen.⁶

While the evidence to support the use of HFNC oxygen in patients with nonhypercapnic acute hypoxemic respiratory failure is growing, this evidence is based on patients enrolled in clinical trials who have no treatment limitations and consent to intubation if necessary. Indeed, several, if not all, randomized trials evaluating HFNC oxygen excluded patients who had do-not-intubate (DNI) or do-not-resuscitate (DNR) orders.^1,2,11 For patients with acute respiratory failure whose primary goal is not to extend life or utilize life support interventions such as invasive mechanical ventilation, HFNC oxygen may offer several benefits compared with other treatment options such as noninvasive positive pressure ventilation, conventional oxygen therapy, or palliative opioid therapy (Appendix Table 1). Determining which treatment options to use depends on the goals of care of the individual patient and the reasonable ability of a particular treatment to help the patient achieve those goals.

While a recent systematic review evaluated the existing evidence regarding the utility and outcomes of noninvasive positive pressure ventilation in adult patients with DNI orders,¹² a systematic review evaluating the evidence and rationale for HFNC oxygen in patients with DNI and/or DNR orders is lacking. Assessing such evidence is necessary to help clinicians and patients determine appropriate treatment choices and establish research priorities. Therefore, our primary objective was to determine what were the following outcomes: mortality, dyspnea, work of breathing, opioid doses, and quality of life in patients who received HFNC oxygen for acute respiratory failure and had a DNI and/or DNR order.

We conducted a systematic review of studies that evaluated patients who used HFNC oxygen for acute respiratory failure and had a DNI and/or DNR order. We reported the results using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statements.¹³ This review was registered with the PROSPERO registry, CRD42017059914.

We included studies that enrolled patients who were (1) hospitalized, (2) >18 years old, (3) had an acute respiratory failure of any cause, (4) received HFNC oxygen, and (5) had a DNI or DNR or comfort measures only order. We included publications of all study designs (interventional, observational, and posthoc analyses) and all languages. We excluded studies that enrolled <5 patients. If necessary, we contacted the authors of the included studies for additional information.

Our search strategy included the following databases from inception to October 14, 2018: PubMed, MEDLINE, CINAHL, MICROMEDEX, EMBASE, Web of Science, and Scopus. The database-specific search strategy was developed using an experienced librarian (Appendix Table 2). In addition, we screened the reference lists of systematic reviews as well as the included studies to find additional relevant articles. Two authors (AM, MEW) independently assessed the inclusion criteria of the titles and abstracts that were identified in the search. In addition, these two authors abstracted relevant data of the included studies.

Using the search strategy, we identified 2,757 citations and included 301 of these in the full-text review (Figure). We included six studies, which enrolled 293 patients in the final systematic review. Table 1 summarizes the characteristics of the included investigations, all of which were observational studies.^15-20 The studies were conducted in the United States of America (n = 3), Europe (n = 2), and Asia (n = 1). Two studies were conducted in the general ICU populations and included patients with hypoxemic respiratory failure only. Four studies were conducted in cancer populations in the hospital wards or ICU and did not specify the type of respiratory failure (hypoxemic versus hypercapnic). Two studies included patients with DNI orders only.^15,20 One study included patients with DNR orders only (DNI orders were excluded).¹⁷ Three studies included patients with both DNR and DNI orders.^16,18,19 The numbers of enrolled patients with treatment limitations were generally low, with the two largest studies including 101 patients each on HFNC oxygen.^18,19

Risk of Bias

All included studies had a high risk of bias (Table 2). A high risk of bias was suggested because the investigations were single-center studies with unclear patient selection methods, did not explicitly report how decisions to limit treatments were made, and did not explicitly differentiate and separately analyze patients with “comfort measures only” goals of care.

Mortality

The hospital mortality rates of patients with DNI and/or DNR orders receiving HFNC were variable and ranged from 40% to 87%. In the two studies enrolling general ICU patient populations, the hospital mortality rates ranged from 40% to 60%. In the four studies enrolling patients with active malignancy, the hospital mortality rates ranged from 75% to 87%. No studies compared mortality rates with and without DNI and/or DNR orders.

Dyspnea, Work of Breathing, and Reduction in Opioid Doses

The impact of HFNC oxygen on symptom relief was reported in one retrospective observational study (published as a conference abstract only to date), which compared the effect of HFNC oxygen (n = 101) with conventional oxygen (n = 110).¹⁸ At first evaluation after hospital admission to a palliative care unit (after the patients had previously been started on either conventional oxygen or high-flow oxygen), patients in the HFNC oxygen group had worse (higher) dyspnea scores compared with patients who used conventional oxygen (Edmonton Symptom Assessment Scale score of 7.5 versus 5, P < .001). At follow-up, approximately 24 hours after admission to the hospital palliative care unit, there was no difference in the change of dyspnea between the HFNC oxygen group (dyspnea score change of 0) and the conventional oxygen group (dyspnea score change of −1, P = .18. In the same study, there was also no significant difference in the morphine dose requirement in each group, and exact doses were not reported.

Two studies reported improvement in oxygen saturation and respiratory rate after HFNC oxygen initiation (compared with before HFNC initiation).^16,20 Oxygen saturation increased from 89% to 95%, P < .01, in one study and 92% to 97%, P < .01, in a second study. The respiratory rate decreased from 31 to 25 breaths/minute in one study, and from 28 to 25 breaths/minute in a second study (both P < .01).

Quality of Life

No studies evaluated the quality of life of survivors.

Secondary Outcomes

Transition to Noninvasive Positive Pressure Ventilation

The proportion of patients who transitioned from HFNC oxygen to NPPV was relatively low in the two studies that reported this outcome, ranging from 0%²⁰ to 18%.¹⁶ In one observational study of a general ICU population, 9/50 (18%) of patients transitioned from HFNC oxygen to NPPV. There was no statistically significant difference in hospital mortality rates among those who progressed to NPPV (67%) versus those who did not progress to NPPV (58%), P = .72.

Tolerance of HFNC and Adverse Events

HFNC oxygen was generally well tolerated based on the assessment of three studies (Table 1). One study reported no adverse events,¹⁶ one study reported that HFNC oxygen had to be discontinued because of nasal discomfort in 1% of patients,¹⁹ and a second study reported that HFNC oxygen had to be discontinued because of agitation in 4% of patients.²⁰

Quality of Death in Nonsurvivors

No studies evaluated the quality of death in those patients who died.

In this systematic review of six studies, all with a high risk of bias, a significant proportion of patients with a DNI and/or DNR order who used HFNC oxygen survived to hospital discharge. Oxygen saturation and respiratory rate consistently improved in the three studies that reported these outcomes. Only one study (published as a conference abstract only to date),¹⁸ however, measured patient-important outcomes related to symptom management and found no significant difference in dyspnea or morphine dose requirements in patients on HFNC oxygen compared with patients on conventional oxygen. HFNC oxygen was generally well tolerated and only had to be stopped in <5% of patients due to intolerance. We found no studies that assessed the quality of life in survivors or the quality of death in nonsurvivors.

Based on the limited evidence in the included studies, HFNC may be a viable treatment option for patients with preset treatment limitations who have acute respiratory failure—with potential benefits of improved oxygenation, decreased respiratory rates, and hospital survival in a proportion of patients. Nevertheless, this systematic review highlights the vast paucity of data available to guide the use of HFNC oxygen in patients with treatment limitations and acute respiratory failure. Only a few studies, which were at high risk of bias, have been conducted on this topic to date. There is an inadequate evidence base to evaluate the comparative effectiveness of HFNC oxygen (versus NPPV versus conventional oxygen versus palliative opioids) in patients with DNI orders or comfort measures only orders.

Our review included two studies that evaluated the comparative effectiveness of HFNC oxygen in patients with DNI and/or DNR orders. The first retrospective observational study compared HFNC oxygen with conventional oxygen in patients with DNR and DNI orders and malignancy—and found no change in dyspnea—but did note an increase in mortality with HFNC oxygen (76% versus 51%).¹⁸ The second observational study compared HFNC oxygen with NPPV in patients with DNR orders with malignancy noted no difference in mortality.¹⁷ In patients with full-code orders, systematic reviews have shown that HFNC oxygen (compared with conventional oxygen) was associated with possible reductions in intubation rates, respiratory rates, and improvements in oxygenation—with no difference in mortality, dyspnea, patient comfort, or ICU/hospital length of stay. Compared with NPPV, HFNC oxygen was associated with similar rates of intubation and mortality.^4-6,21

Future studies in patients with acute respiratory failure and DNI and/or DNR orders should identify which treatment modality (HFNC oxygen compared with other modalities, such as NPPV, conventional oxygen, with or without palliative opioids) impacts outcomes, such as dyspnea reduction while maintaining an alert mental status, short- and long-term quality of life in survivors, and quality of death in nonsurvivors. Future studies should also identify the optimal treatment pathway to utilize when patients using HFNC oxygen fail this therapy (eg, transition to NPPV versus intensifying palliative opioids) as well as the optimal process to withdraw palliative HFNC oxygen.²² Identifying which patient populations may benefit from different treatment pathways should also be considered as different treatment strategies may be more beneficial in different patient populations (eg, based on cause and severity of acute respiratory failure). In addition, it should be noted that the primary goal of care might affect which outcomes are the most important to measure. While patients with comfort measures only, orders usually have a primary goal to prepare for a high-quality death, patients with DNI and/or DNR orders (but without comfort measures only orders) may have a primary goal to survive—but with the desire not to endure the high burden of intubation and mechanical ventilation if it became necessary. Finally, future studies should utilize high-quality study designs (eg, randomized controlled trials) that enable robust evaluation of comparative effectiveness of clinically relevant treatment strategies.

While several previous systematic reviews have evaluated the efficacy of HFNC in patients with acute respiratory failure without preset limitations on life support; to our knowledge, this is the first systematic review to assess outcomes in patients rigorously with preset treatment limitations. Our review is, however, limited by the high risk of bias of the studies that were included (single-center nature, retrospective observational study designs, small sample sizes, and lack of a description of how DNI and/or DNR statuses were determined) as well as the small number of studies available to be included.

This systematic review points to a significant evidence gap in our understanding of the role for HFNC oxygen (compared with other acceptable alternative treatment strategies) in adult patients with acute respiratory failure who have DNI and/or DNR orders. Further high-quality research is needed to explore these unanswered questions in an effort to best treat, guide, and engage in optimal end-of-life decision making among patients with acute respiratory failure.

High-flow nasal cannula (HFNC) oxygen therapy is effective in treating adults with acute hypoxemic respiratory failure, and to a lesser extent acute hypercapnic respiratory failure.^1-3 HFNC oxygen is capable of delivering oxygen with flows of 30-60 liters/minute, and can provide a high fraction of inspired oxygen, flush anatomic dead space, augment respiratory efforts, and provide mild continuous positive airway pressure effects. Several systematic reviews and meta-analyses have evaluated the effectiveness of HFNC oxygen and have shown modestly lower rates of intubation compared with conventional oxygen^4,5 and similar intubation rates compared with noninvasive positive pressure ventilation.^4-9 Although one randomized trial showed a lower risk of 90-day mortality for HFNC oxygen compared with either conventional oxygen or noninvasive positive pressure ventilation, several meta-analyses have shown no difference in intensive care unit (ICU) mortality.^4,6,8,10 The majority of studies have shown improvements in oxygenation, comfort, dyspnea scores, and breathing pattern with the initiation of HFNC oxygen.⁶

While the evidence to support the use of HFNC oxygen in patients with nonhypercapnic acute hypoxemic respiratory failure is growing, this evidence is based on patients enrolled in clinical trials who have no treatment limitations and consent to intubation if necessary. Indeed, several, if not all, randomized trials evaluating HFNC oxygen excluded patients who had do-not-intubate (DNI) or do-not-resuscitate (DNR) orders.^1,2,11 For patients with acute respiratory failure whose primary goal is not to extend life or utilize life support interventions such as invasive mechanical ventilation, HFNC oxygen may offer several benefits compared with other treatment options such as noninvasive positive pressure ventilation, conventional oxygen therapy, or palliative opioid therapy (Appendix Table 1). Determining which treatment options to use depends on the goals of care of the individual patient and the reasonable ability of a particular treatment to help the patient achieve those goals.

While a recent systematic review evaluated the existing evidence regarding the utility and outcomes of noninvasive positive pressure ventilation in adult patients with DNI orders,¹² a systematic review evaluating the evidence and rationale for HFNC oxygen in patients with DNI and/or DNR orders is lacking. Assessing such evidence is necessary to help clinicians and patients determine appropriate treatment choices and establish research priorities. Therefore, our primary objective was to determine what were the following outcomes: mortality, dyspnea, work of breathing, opioid doses, and quality of life in patients who received HFNC oxygen for acute respiratory failure and had a DNI and/or DNR order.

We conducted a systematic review of studies that evaluated patients who used HFNC oxygen for acute respiratory failure and had a DNI and/or DNR order. We reported the results using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statements.¹³ This review was registered with the PROSPERO registry, CRD42017059914.

We included studies that enrolled patients who were (1) hospitalized, (2) >18 years old, (3) had an acute respiratory failure of any cause, (4) received HFNC oxygen, and (5) had a DNI or DNR or comfort measures only order. We included publications of all study designs (interventional, observational, and posthoc analyses) and all languages. We excluded studies that enrolled <5 patients. If necessary, we contacted the authors of the included studies for additional information.

Our search strategy included the following databases from inception to October 14, 2018: PubMed, MEDLINE, CINAHL, MICROMEDEX, EMBASE, Web of Science, and Scopus. The database-specific search strategy was developed using an experienced librarian (Appendix Table 2). In addition, we screened the reference lists of systematic reviews as well as the included studies to find additional relevant articles. Two authors (AM, MEW) independently assessed the inclusion criteria of the titles and abstracts that were identified in the search. In addition, these two authors abstracted relevant data of the included studies.

Using the search strategy, we identified 2,757 citations and included 301 of these in the full-text review (Figure). We included six studies, which enrolled 293 patients in the final systematic review. Table 1 summarizes the characteristics of the included investigations, all of which were observational studies.^15-20 The studies were conducted in the United States of America (n = 3), Europe (n = 2), and Asia (n = 1). Two studies were conducted in the general ICU populations and included patients with hypoxemic respiratory failure only. Four studies were conducted in cancer populations in the hospital wards or ICU and did not specify the type of respiratory failure (hypoxemic versus hypercapnic). Two studies included patients with DNI orders only.^15,20 One study included patients with DNR orders only (DNI orders were excluded).¹⁷ Three studies included patients with both DNR and DNI orders.^16,18,19 The numbers of enrolled patients with treatment limitations were generally low, with the two largest studies including 101 patients each on HFNC oxygen.^18,19

Risk of Bias

All included studies had a high risk of bias (Table 2). A high risk of bias was suggested because the investigations were single-center studies with unclear patient selection methods, did not explicitly report how decisions to limit treatments were made, and did not explicitly differentiate and separately analyze patients with “comfort measures only” goals of care.

Mortality

The hospital mortality rates of patients with DNI and/or DNR orders receiving HFNC were variable and ranged from 40% to 87%. In the two studies enrolling general ICU patient populations, the hospital mortality rates ranged from 40% to 60%. In the four studies enrolling patients with active malignancy, the hospital mortality rates ranged from 75% to 87%. No studies compared mortality rates with and without DNI and/or DNR orders.

Dyspnea, Work of Breathing, and Reduction in Opioid Doses

The impact of HFNC oxygen on symptom relief was reported in one retrospective observational study (published as a conference abstract only to date), which compared the effect of HFNC oxygen (n = 101) with conventional oxygen (n = 110).¹⁸ At first evaluation after hospital admission to a palliative care unit (after the patients had previously been started on either conventional oxygen or high-flow oxygen), patients in the HFNC oxygen group had worse (higher) dyspnea scores compared with patients who used conventional oxygen (Edmonton Symptom Assessment Scale score of 7.5 versus 5, P < .001). At follow-up, approximately 24 hours after admission to the hospital palliative care unit, there was no difference in the change of dyspnea between the HFNC oxygen group (dyspnea score change of 0) and the conventional oxygen group (dyspnea score change of −1, P = .18. In the same study, there was also no significant difference in the morphine dose requirement in each group, and exact doses were not reported.

Two studies reported improvement in oxygen saturation and respiratory rate after HFNC oxygen initiation (compared with before HFNC initiation).^16,20 Oxygen saturation increased from 89% to 95%, P < .01, in one study and 92% to 97%, P < .01, in a second study. The respiratory rate decreased from 31 to 25 breaths/minute in one study, and from 28 to 25 breaths/minute in a second study (both P < .01).

Quality of Life

No studies evaluated the quality of life of survivors.

Secondary Outcomes

Transition to Noninvasive Positive Pressure Ventilation

The proportion of patients who transitioned from HFNC oxygen to NPPV was relatively low in the two studies that reported this outcome, ranging from 0%²⁰ to 18%.¹⁶ In one observational study of a general ICU population, 9/50 (18%) of patients transitioned from HFNC oxygen to NPPV. There was no statistically significant difference in hospital mortality rates among those who progressed to NPPV (67%) versus those who did not progress to NPPV (58%), P = .72.

Tolerance of HFNC and Adverse Events

HFNC oxygen was generally well tolerated based on the assessment of three studies (Table 1). One study reported no adverse events,¹⁶ one study reported that HFNC oxygen had to be discontinued because of nasal discomfort in 1% of patients,¹⁹ and a second study reported that HFNC oxygen had to be discontinued because of agitation in 4% of patients.²⁰

Quality of Death in Nonsurvivors

No studies evaluated the quality of death in those patients who died.

In this systematic review of six studies, all with a high risk of bias, a significant proportion of patients with a DNI and/or DNR order who used HFNC oxygen survived to hospital discharge. Oxygen saturation and respiratory rate consistently improved in the three studies that reported these outcomes. Only one study (published as a conference abstract only to date),¹⁸ however, measured patient-important outcomes related to symptom management and found no significant difference in dyspnea or morphine dose requirements in patients on HFNC oxygen compared with patients on conventional oxygen. HFNC oxygen was generally well tolerated and only had to be stopped in <5% of patients due to intolerance. We found no studies that assessed the quality of life in survivors or the quality of death in nonsurvivors.

Based on the limited evidence in the included studies, HFNC may be a viable treatment option for patients with preset treatment limitations who have acute respiratory failure—with potential benefits of improved oxygenation, decreased respiratory rates, and hospital survival in a proportion of patients. Nevertheless, this systematic review highlights the vast paucity of data available to guide the use of HFNC oxygen in patients with treatment limitations and acute respiratory failure. Only a few studies, which were at high risk of bias, have been conducted on this topic to date. There is an inadequate evidence base to evaluate the comparative effectiveness of HFNC oxygen (versus NPPV versus conventional oxygen versus palliative opioids) in patients with DNI orders or comfort measures only orders.

Our review included two studies that evaluated the comparative effectiveness of HFNC oxygen in patients with DNI and/or DNR orders. The first retrospective observational study compared HFNC oxygen with conventional oxygen in patients with DNR and DNI orders and malignancy—and found no change in dyspnea—but did note an increase in mortality with HFNC oxygen (76% versus 51%).¹⁸ The second observational study compared HFNC oxygen with NPPV in patients with DNR orders with malignancy noted no difference in mortality.¹⁷ In patients with full-code orders, systematic reviews have shown that HFNC oxygen (compared with conventional oxygen) was associated with possible reductions in intubation rates, respiratory rates, and improvements in oxygenation—with no difference in mortality, dyspnea, patient comfort, or ICU/hospital length of stay. Compared with NPPV, HFNC oxygen was associated with similar rates of intubation and mortality.^4-6,21

Future studies in patients with acute respiratory failure and DNI and/or DNR orders should identify which treatment modality (HFNC oxygen compared with other modalities, such as NPPV, conventional oxygen, with or without palliative opioids) impacts outcomes, such as dyspnea reduction while maintaining an alert mental status, short- and long-term quality of life in survivors, and quality of death in nonsurvivors. Future studies should also identify the optimal treatment pathway to utilize when patients using HFNC oxygen fail this therapy (eg, transition to NPPV versus intensifying palliative opioids) as well as the optimal process to withdraw palliative HFNC oxygen.²² Identifying which patient populations may benefit from different treatment pathways should also be considered as different treatment strategies may be more beneficial in different patient populations (eg, based on cause and severity of acute respiratory failure). In addition, it should be noted that the primary goal of care might affect which outcomes are the most important to measure. While patients with comfort measures only, orders usually have a primary goal to prepare for a high-quality death, patients with DNI and/or DNR orders (but without comfort measures only orders) may have a primary goal to survive—but with the desire not to endure the high burden of intubation and mechanical ventilation if it became necessary. Finally, future studies should utilize high-quality study designs (eg, randomized controlled trials) that enable robust evaluation of comparative effectiveness of clinically relevant treatment strategies.

While several previous systematic reviews have evaluated the efficacy of HFNC in patients with acute respiratory failure without preset limitations on life support; to our knowledge, this is the first systematic review to assess outcomes in patients rigorously with preset treatment limitations. Our review is, however, limited by the high risk of bias of the studies that were included (single-center nature, retrospective observational study designs, small sample sizes, and lack of a description of how DNI and/or DNR statuses were determined) as well as the small number of studies available to be included.

This systematic review points to a significant evidence gap in our understanding of the role for HFNC oxygen (compared with other acceptable alternative treatment strategies) in adult patients with acute respiratory failure who have DNI and/or DNR orders. Further high-quality research is needed to explore these unanswered questions in an effort to best treat, guide, and engage in optimal end-of-life decision making among patients with acute respiratory failure.

The recent designation of Pediatric Hospital Medicine (PHM) as a board-certified subspecialty has provided the opportunity to define which skills are core to hospitalist practice. One skill that is novel to the field and gaining traction is point-of-care ultrasonography (POCUS). POCUS differs from traditional ultrasonography in that it is performed at the bedside by the primary clinician and aims to answer a focused clinical question (eg, does this patient have a skin abscess?) rather than to provide a comprehensive evaluation of the anatomy and physiology. The proposed advantages of POCUS include real-time image interpretation, cost savings, procedural guidance to minimize complications, and reduction of ionizing radiation. Although specialties such as Critical Care (CC) and Emergency Medicine (EM) have integrated POCUS into their practice and training, best practices in PHM have not been defined. This Progress Note is a summary of recent evidence to update past reviews and set the stage for future PHM POCUS research and education.

We met with an academic librarian in March 2019 and performed a search of PubMed using Medical Subject Headings (MESH) terms associated with POCUS as well as Pediatrics. We limited our search to studies published within the past five years. The search was originally focused to the field of PHM before expanding to a broader search since very few studies were found that focused on Hospital Medicine or general pediatric ward populations. This initial search generated 274 publications. We then performed a supplemental literature search using references from studies found in our initial search, as well as further ad hoc searching in Embase and Google Scholar.

After our literature search, we reviewed the PHM core competencies and identified the common clinical diagnoses and core skills for which there is POCUS literature published in the past five years. These included acute abdominal pain, bronchiolitis, pneumonia, skin and soft tissue infection, newborn care/delivery room management, bladder catheterization, fluid management, intravenous access, and lumbar puncture (LP). We chose to focus on one skill and two diagnoses that were generalizable to pediatric hospitalists across different settings and for which there was compelling evidence for POCUS use, such as pneumonia, skin abscess, and LP. We found few studies that included general pediatric ward patients, but we considered EM and CC studies to be relevant as several pediatric hospitalists practice in these clinical settings and with these patient populations.

POCUS can be useful for diagnosing pneumonia by direct visualization of lung consolidation or by identification of various sonographic artifacts that suggest pathology. For example, “B-lines” are vertical artifacts that extend from the pleura and suggest interstitial fluid or pneumonia when they are present in abnormally high numbers or density. POCUS can also be used to diagnose parapneumonic effusions by scanning dependent areas of the lung (eg, the diaphragm in children sitting upright) and looking for anechoic or hypoechoic areas.

Three recent meta-analyses found favorable operating characteristics when using POCUS for the diagnosis of pneumonia in children, with summary sensitivities of 93%-94% and specificities of 92%-96%.^1-3 However, these meta-analyses were limited by high heterogeneity due to the inclusion of multiple different care settings and the use of variable reference standards and sonographic criteria for diagnosing pneumonia. POCUS is superior to chest radiography for evaluating parapneumonic pleural effusions,⁴ allowing for rapid identification of loculations, fibrin strands, and proteinaceous material, and for serial bedside evaluation of effusion size and characteristics.

Additional advantages of POCUS include avoidance of ionizing radiation and the potential for cost and time savings. Two studies demonstrated reductions in radiography use and improved cost, although they were not conducted on hospitalized patients. One randomized controlled trial (RCT) conducted in a pediatric emergency department (ED) demonstrated a 38.8% reduction in chest radiography use without increasing the ED length of stay (EDLOS), antibiotic use, or unscheduled follow-up visits.⁵ A retrospective matched cohort study conducted in another pediatric ED reported that when compared with patients evaluated by chest radiography, those evaluated by POCUS had significantly shorter EDLOS (−60.9 min) and mean health systems savings ($187 per patient).⁶ We believe that POCUS has value in the evaluation and management of pneumonia and parapneumonic effusions, although further studies investigating patient outcomes and involving inpatient populations are required.

POCUS can augment the physical examination, helping to both avoid unnecessary incision and drainage (I+D) procedures and detect drainable fluid collections. Abscess is suggested when hypoechoic material without vascular flow is detected, and although other structures such as vessels, cysts, and lymph nodes can mimic skin abscesses, this is a relatively straightforward examination for clinicians to learn.

Two meta-analyses found that POCUS had high sensitivity for diagnosing skin abscesses in the ED.^7,8 A pediatric subgroup analysis conducted in a study by Barbic et al. found a sensitivity and a specificity of 94% (95% C: 88%-98%) and 83% (95% C: 47%-97%), respectively.⁷ Subramaniam et al. included six studies (four pediatric) with 800 patients (653 ≤ 18 years old) and found an overall pooled sensitivity of 97% (95% C: 94%-98%) and a specificity of 83% (95% C: 75%-88%).⁸ No subgroup analysis was performed, but the included pediatric studies reported sensitivities and specificities between 90%-98% and 68%-87%, respectively.

Although POCUS performs better than physical examination for the diagnosis of drainable abscesses, evidence regarding patient outcomes is mixed. A retrospective review from four pediatric EDs found that integration of POCUS lowered treatment failure rates, defined as any incision and drainage (I+D) or surgical manipulation after discharge from the initial ED visit (4.4% vs 15.6%, P < .005).⁹ A single-center retrospective cohort study found that POCUS reduced EDLOS by a median of 73 minutes (95% C: 52-94 min) when compared with radiology-performed studies.¹⁰ The aforementioned study conducted by Barbic et al. found that in pediatric studies, POCUS led to a change in management (eg, whether or not to attempt I+D) in 14%-27% of patients.⁷ However, a multicenter prospective observational cohort study involving seven pediatric EDs found that despite changing the management in 22.9% of cases, POCUS was not associated with any statistically significant differences in treatment failure rates, EDLOS, discharge rates, use of sedation, or use of alternative imaging.¹¹ These studies were limited by a lack of randomization or control group and emphasize the need for RCTs that measure patient outcomes. Future studies should investigate how POCUS can be used in inpatient settings both for initial diagnosis of drainable abscesses and for serial evaluation of evolving phlegmon or incompletely drained collections.

LP is commonly performed by pediatric hospitalists, although success can be influenced by numerous factors, including provider and staff expertise, patient anatomy, and body habitus. Requiring multiple attempts can increase patient discomfort and parental anxiety. Failure to obtain cerebrospinal fluid can delay diagnosis or leave providers in uncertain clinical situations that may commit patients to prolonged antibiotic courses. POCUS can be used to identify anatomic markers such as interspinous processes, anatomic midline, and depth of the ligamentum flavum.¹² It can also be used to identify epidural hematomas after failed LPs to avoid additional unsuccessful attempts.¹³ POCUS guidance for LP has been described using both static (preprocedural marking) and dynamic (scanning during the procedure) techniques, although most of the studies use the static approach. The Society for Hospital Medicine POCUS Task Force has recently released a position statement recommending that POCUS should be used for site selection before performing LP in adult patients when providers are adequately trained.¹² Although this position statement was for adult patients, recent evidence suggests that there is also benefit in Pediatrics.

Two recent meta-analyses have investigated POCUS use for pediatric LPs.^14,15 Olowoyeye et al. included four studies with a total of 277 patients and found that POCUS use was associated with a reduction in traumatic taps (risk ratio [RR] = 0.53, 95% C: 0.13-0.82) when compared with landmark approaches.¹⁴ However, there was no statistically significant reduction in LP failure, number of needle insertion attempts, or procedure length. A more recent meta-analysis performed a pediatric subgroup analysis of six studies including 452 patients and found a statistically significant reduction in traumatic taps (13.7% vs 31.8%, risk difference = −21.3%, 95% C: −38.2% to −4.3%) and number of needle insertion attempts (1.53 vs 2.07, mean difference = −0.47, 95% C: −0.73 to −0.21).¹⁵ The primary outcome of LP success trended toward favoring POCUS, but it was not statistically significant (88.4% vs 74.0%, OR = 2.55, 95% C: 0.99-6.52). We believe that recent evidence suggests that there is benefit in using POCUS when hospitalists attempt pediatric LPs, particularly when physical landmarks are difficult to identify or after failed attempts. However, adequate training with simulation and supervised practice should be undertaken before integrating this into clinical practice.

Evidence accumulated in the past five years has built on previous work suggesting that POCUS has a role in the diagnosis of pneumonia and skin abscess and in the performance of LPs. However, gaps in the literature remain when applying POCUS in PHM. Only a few studies to date were conducted in non-CC inpatient settings, and although several pediatric hospitalists work in EDs or care for critically ill children, our largest population comprises general pediatric ward patients. Studies have also used ultrasonographers with variable POCUS training and clinical experience, which makes comparing or combining studies challenging since POCUS is dependent on provider skills. Studies involving PHM providers and inpatient populations are needed. Additional studies evaluating the process and outcome measures are also needed to understand whether the theoretical advantages are consistently realized in real-world PHM practice. Finally, PHM-specific curricula should be designed in collaboration with various PHM stakeholders and with specialties who already have robust POCUS training pathways. There is opportunity within PHM for multi institutional research collaboration, identification of best practices, and development of PHM-specific training for fellowship and faculty development programs.

The recent designation of Pediatric Hospital Medicine (PHM) as a board-certified subspecialty has provided the opportunity to define which skills are core to hospitalist practice. One skill that is novel to the field and gaining traction is point-of-care ultrasonography (POCUS). POCUS differs from traditional ultrasonography in that it is performed at the bedside by the primary clinician and aims to answer a focused clinical question (eg, does this patient have a skin abscess?) rather than to provide a comprehensive evaluation of the anatomy and physiology. The proposed advantages of POCUS include real-time image interpretation, cost savings, procedural guidance to minimize complications, and reduction of ionizing radiation. Although specialties such as Critical Care (CC) and Emergency Medicine (EM) have integrated POCUS into their practice and training, best practices in PHM have not been defined. This Progress Note is a summary of recent evidence to update past reviews and set the stage for future PHM POCUS research and education.

We met with an academic librarian in March 2019 and performed a search of PubMed using Medical Subject Headings (MESH) terms associated with POCUS as well as Pediatrics. We limited our search to studies published within the past five years. The search was originally focused to the field of PHM before expanding to a broader search since very few studies were found that focused on Hospital Medicine or general pediatric ward populations. This initial search generated 274 publications. We then performed a supplemental literature search using references from studies found in our initial search, as well as further ad hoc searching in Embase and Google Scholar.

After our literature search, we reviewed the PHM core competencies and identified the common clinical diagnoses and core skills for which there is POCUS literature published in the past five years. These included acute abdominal pain, bronchiolitis, pneumonia, skin and soft tissue infection, newborn care/delivery room management, bladder catheterization, fluid management, intravenous access, and lumbar puncture (LP). We chose to focus on one skill and two diagnoses that were generalizable to pediatric hospitalists across different settings and for which there was compelling evidence for POCUS use, such as pneumonia, skin abscess, and LP. We found few studies that included general pediatric ward patients, but we considered EM and CC studies to be relevant as several pediatric hospitalists practice in these clinical settings and with these patient populations.

POCUS can be useful for diagnosing pneumonia by direct visualization of lung consolidation or by identification of various sonographic artifacts that suggest pathology. For example, “B-lines” are vertical artifacts that extend from the pleura and suggest interstitial fluid or pneumonia when they are present in abnormally high numbers or density. POCUS can also be used to diagnose parapneumonic effusions by scanning dependent areas of the lung (eg, the diaphragm in children sitting upright) and looking for anechoic or hypoechoic areas.

Three recent meta-analyses found favorable operating characteristics when using POCUS for the diagnosis of pneumonia in children, with summary sensitivities of 93%-94% and specificities of 92%-96%.^1-3 However, these meta-analyses were limited by high heterogeneity due to the inclusion of multiple different care settings and the use of variable reference standards and sonographic criteria for diagnosing pneumonia. POCUS is superior to chest radiography for evaluating parapneumonic pleural effusions,⁴ allowing for rapid identification of loculations, fibrin strands, and proteinaceous material, and for serial bedside evaluation of effusion size and characteristics.

Additional advantages of POCUS include avoidance of ionizing radiation and the potential for cost and time savings. Two studies demonstrated reductions in radiography use and improved cost, although they were not conducted on hospitalized patients. One randomized controlled trial (RCT) conducted in a pediatric emergency department (ED) demonstrated a 38.8% reduction in chest radiography use without increasing the ED length of stay (EDLOS), antibiotic use, or unscheduled follow-up visits.⁵ A retrospective matched cohort study conducted in another pediatric ED reported that when compared with patients evaluated by chest radiography, those evaluated by POCUS had significantly shorter EDLOS (−60.9 min) and mean health systems savings ($187 per patient).⁶ We believe that POCUS has value in the evaluation and management of pneumonia and parapneumonic effusions, although further studies investigating patient outcomes and involving inpatient populations are required.

POCUS can augment the physical examination, helping to both avoid unnecessary incision and drainage (I+D) procedures and detect drainable fluid collections. Abscess is suggested when hypoechoic material without vascular flow is detected, and although other structures such as vessels, cysts, and lymph nodes can mimic skin abscesses, this is a relatively straightforward examination for clinicians to learn.

Two meta-analyses found that POCUS had high sensitivity for diagnosing skin abscesses in the ED.^7,8 A pediatric subgroup analysis conducted in a study by Barbic et al. found a sensitivity and a specificity of 94% (95% C: 88%-98%) and 83% (95% C: 47%-97%), respectively.⁷ Subramaniam et al. included six studies (four pediatric) with 800 patients (653 ≤ 18 years old) and found an overall pooled sensitivity of 97% (95% C: 94%-98%) and a specificity of 83% (95% C: 75%-88%).⁸ No subgroup analysis was performed, but the included pediatric studies reported sensitivities and specificities between 90%-98% and 68%-87%, respectively.

Although POCUS performs better than physical examination for the diagnosis of drainable abscesses, evidence regarding patient outcomes is mixed. A retrospective review from four pediatric EDs found that integration of POCUS lowered treatment failure rates, defined as any incision and drainage (I+D) or surgical manipulation after discharge from the initial ED visit (4.4% vs 15.6%, P < .005).⁹ A single-center retrospective cohort study found that POCUS reduced EDLOS by a median of 73 minutes (95% C: 52-94 min) when compared with radiology-performed studies.¹⁰ The aforementioned study conducted by Barbic et al. found that in pediatric studies, POCUS led to a change in management (eg, whether or not to attempt I+D) in 14%-27% of patients.⁷ However, a multicenter prospective observational cohort study involving seven pediatric EDs found that despite changing the management in 22.9% of cases, POCUS was not associated with any statistically significant differences in treatment failure rates, EDLOS, discharge rates, use of sedation, or use of alternative imaging.¹¹ These studies were limited by a lack of randomization or control group and emphasize the need for RCTs that measure patient outcomes. Future studies should investigate how POCUS can be used in inpatient settings both for initial diagnosis of drainable abscesses and for serial evaluation of evolving phlegmon or incompletely drained collections.

LP is commonly performed by pediatric hospitalists, although success can be influenced by numerous factors, including provider and staff expertise, patient anatomy, and body habitus. Requiring multiple attempts can increase patient discomfort and parental anxiety. Failure to obtain cerebrospinal fluid can delay diagnosis or leave providers in uncertain clinical situations that may commit patients to prolonged antibiotic courses. POCUS can be used to identify anatomic markers such as interspinous processes, anatomic midline, and depth of the ligamentum flavum.¹² It can also be used to identify epidural hematomas after failed LPs to avoid additional unsuccessful attempts.¹³ POCUS guidance for LP has been described using both static (preprocedural marking) and dynamic (scanning during the procedure) techniques, although most of the studies use the static approach. The Society for Hospital Medicine POCUS Task Force has recently released a position statement recommending that POCUS should be used for site selection before performing LP in adult patients when providers are adequately trained.¹² Although this position statement was for adult patients, recent evidence suggests that there is also benefit in Pediatrics.

Two recent meta-analyses have investigated POCUS use for pediatric LPs.^14,15 Olowoyeye et al. included four studies with a total of 277 patients and found that POCUS use was associated with a reduction in traumatic taps (risk ratio [RR] = 0.53, 95% C: 0.13-0.82) when compared with landmark approaches.¹⁴ However, there was no statistically significant reduction in LP failure, number of needle insertion attempts, or procedure length. A more recent meta-analysis performed a pediatric subgroup analysis of six studies including 452 patients and found a statistically significant reduction in traumatic taps (13.7% vs 31.8%, risk difference = −21.3%, 95% C: −38.2% to −4.3%) and number of needle insertion attempts (1.53 vs 2.07, mean difference = −0.47, 95% C: −0.73 to −0.21).¹⁵ The primary outcome of LP success trended toward favoring POCUS, but it was not statistically significant (88.4% vs 74.0%, OR = 2.55, 95% C: 0.99-6.52). We believe that recent evidence suggests that there is benefit in using POCUS when hospitalists attempt pediatric LPs, particularly when physical landmarks are difficult to identify or after failed attempts. However, adequate training with simulation and supervised practice should be undertaken before integrating this into clinical practice.

Evidence accumulated in the past five years has built on previous work suggesting that POCUS has a role in the diagnosis of pneumonia and skin abscess and in the performance of LPs. However, gaps in the literature remain when applying POCUS in PHM. Only a few studies to date were conducted in non-CC inpatient settings, and although several pediatric hospitalists work in EDs or care for critically ill children, our largest population comprises general pediatric ward patients. Studies have also used ultrasonographers with variable POCUS training and clinical experience, which makes comparing or combining studies challenging since POCUS is dependent on provider skills. Studies involving PHM providers and inpatient populations are needed. Additional studies evaluating the process and outcome measures are also needed to understand whether the theoretical advantages are consistently realized in real-world PHM practice. Finally, PHM-specific curricula should be designed in collaboration with various PHM stakeholders and with specialties who already have robust POCUS training pathways. There is opportunity within PHM for multi institutional research collaboration, identification of best practices, and development of PHM-specific training for fellowship and faculty development programs.

The recent designation of Pediatric Hospital Medicine (PHM) as a board-certified subspecialty has provided the opportunity to define which skills are core to hospitalist practice. One skill that is novel to the field and gaining traction is point-of-care ultrasonography (POCUS). POCUS differs from traditional ultrasonography in that it is performed at the bedside by the primary clinician and aims to answer a focused clinical question (eg, does this patient have a skin abscess?) rather than to provide a comprehensive evaluation of the anatomy and physiology. The proposed advantages of POCUS include real-time image interpretation, cost savings, procedural guidance to minimize complications, and reduction of ionizing radiation. Although specialties such as Critical Care (CC) and Emergency Medicine (EM) have integrated POCUS into their practice and training, best practices in PHM have not been defined. This Progress Note is a summary of recent evidence to update past reviews and set the stage for future PHM POCUS research and education.

We met with an academic librarian in March 2019 and performed a search of PubMed using Medical Subject Headings (MESH) terms associated with POCUS as well as Pediatrics. We limited our search to studies published within the past five years. The search was originally focused to the field of PHM before expanding to a broader search since very few studies were found that focused on Hospital Medicine or general pediatric ward populations. This initial search generated 274 publications. We then performed a supplemental literature search using references from studies found in our initial search, as well as further ad hoc searching in Embase and Google Scholar.

After our literature search, we reviewed the PHM core competencies and identified the common clinical diagnoses and core skills for which there is POCUS literature published in the past five years. These included acute abdominal pain, bronchiolitis, pneumonia, skin and soft tissue infection, newborn care/delivery room management, bladder catheterization, fluid management, intravenous access, and lumbar puncture (LP). We chose to focus on one skill and two diagnoses that were generalizable to pediatric hospitalists across different settings and for which there was compelling evidence for POCUS use, such as pneumonia, skin abscess, and LP. We found few studies that included general pediatric ward patients, but we considered EM and CC studies to be relevant as several pediatric hospitalists practice in these clinical settings and with these patient populations.

POCUS can be useful for diagnosing pneumonia by direct visualization of lung consolidation or by identification of various sonographic artifacts that suggest pathology. For example, “B-lines” are vertical artifacts that extend from the pleura and suggest interstitial fluid or pneumonia when they are present in abnormally high numbers or density. POCUS can also be used to diagnose parapneumonic effusions by scanning dependent areas of the lung (eg, the diaphragm in children sitting upright) and looking for anechoic or hypoechoic areas.

Three recent meta-analyses found favorable operating characteristics when using POCUS for the diagnosis of pneumonia in children, with summary sensitivities of 93%-94% and specificities of 92%-96%.^1-3 However, these meta-analyses were limited by high heterogeneity due to the inclusion of multiple different care settings and the use of variable reference standards and sonographic criteria for diagnosing pneumonia. POCUS is superior to chest radiography for evaluating parapneumonic pleural effusions,⁴ allowing for rapid identification of loculations, fibrin strands, and proteinaceous material, and for serial bedside evaluation of effusion size and characteristics.

Additional advantages of POCUS include avoidance of ionizing radiation and the potential for cost and time savings. Two studies demonstrated reductions in radiography use and improved cost, although they were not conducted on hospitalized patients. One randomized controlled trial (RCT) conducted in a pediatric emergency department (ED) demonstrated a 38.8% reduction in chest radiography use without increasing the ED length of stay (EDLOS), antibiotic use, or unscheduled follow-up visits.⁵ A retrospective matched cohort study conducted in another pediatric ED reported that when compared with patients evaluated by chest radiography, those evaluated by POCUS had significantly shorter EDLOS (−60.9 min) and mean health systems savings ($187 per patient).⁶ We believe that POCUS has value in the evaluation and management of pneumonia and parapneumonic effusions, although further studies investigating patient outcomes and involving inpatient populations are required.

POCUS can augment the physical examination, helping to both avoid unnecessary incision and drainage (I+D) procedures and detect drainable fluid collections. Abscess is suggested when hypoechoic material without vascular flow is detected, and although other structures such as vessels, cysts, and lymph nodes can mimic skin abscesses, this is a relatively straightforward examination for clinicians to learn.

Two meta-analyses found that POCUS had high sensitivity for diagnosing skin abscesses in the ED.^7,8 A pediatric subgroup analysis conducted in a study by Barbic et al. found a sensitivity and a specificity of 94% (95% C: 88%-98%) and 83% (95% C: 47%-97%), respectively.⁷ Subramaniam et al. included six studies (four pediatric) with 800 patients (653 ≤ 18 years old) and found an overall pooled sensitivity of 97% (95% C: 94%-98%) and a specificity of 83% (95% C: 75%-88%).⁸ No subgroup analysis was performed, but the included pediatric studies reported sensitivities and specificities between 90%-98% and 68%-87%, respectively.

Although POCUS performs better than physical examination for the diagnosis of drainable abscesses, evidence regarding patient outcomes is mixed. A retrospective review from four pediatric EDs found that integration of POCUS lowered treatment failure rates, defined as any incision and drainage (I+D) or surgical manipulation after discharge from the initial ED visit (4.4% vs 15.6%, P < .005).⁹ A single-center retrospective cohort study found that POCUS reduced EDLOS by a median of 73 minutes (95% C: 52-94 min) when compared with radiology-performed studies.¹⁰ The aforementioned study conducted by Barbic et al. found that in pediatric studies, POCUS led to a change in management (eg, whether or not to attempt I+D) in 14%-27% of patients.⁷ However, a multicenter prospective observational cohort study involving seven pediatric EDs found that despite changing the management in 22.9% of cases, POCUS was not associated with any statistically significant differences in treatment failure rates, EDLOS, discharge rates, use of sedation, or use of alternative imaging.¹¹ These studies were limited by a lack of randomization or control group and emphasize the need for RCTs that measure patient outcomes. Future studies should investigate how POCUS can be used in inpatient settings both for initial diagnosis of drainable abscesses and for serial evaluation of evolving phlegmon or incompletely drained collections.

LP is commonly performed by pediatric hospitalists, although success can be influenced by numerous factors, including provider and staff expertise, patient anatomy, and body habitus. Requiring multiple attempts can increase patient discomfort and parental anxiety. Failure to obtain cerebrospinal fluid can delay diagnosis or leave providers in uncertain clinical situations that may commit patients to prolonged antibiotic courses. POCUS can be used to identify anatomic markers such as interspinous processes, anatomic midline, and depth of the ligamentum flavum.¹² It can also be used to identify epidural hematomas after failed LPs to avoid additional unsuccessful attempts.¹³ POCUS guidance for LP has been described using both static (preprocedural marking) and dynamic (scanning during the procedure) techniques, although most of the studies use the static approach. The Society for Hospital Medicine POCUS Task Force has recently released a position statement recommending that POCUS should be used for site selection before performing LP in adult patients when providers are adequately trained.¹² Although this position statement was for adult patients, recent evidence suggests that there is also benefit in Pediatrics.

Two recent meta-analyses have investigated POCUS use for pediatric LPs.^14,15 Olowoyeye et al. included four studies with a total of 277 patients and found that POCUS use was associated with a reduction in traumatic taps (risk ratio [RR] = 0.53, 95% C: 0.13-0.82) when compared with landmark approaches.¹⁴ However, there was no statistically significant reduction in LP failure, number of needle insertion attempts, or procedure length. A more recent meta-analysis performed a pediatric subgroup analysis of six studies including 452 patients and found a statistically significant reduction in traumatic taps (13.7% vs 31.8%, risk difference = −21.3%, 95% C: −38.2% to −4.3%) and number of needle insertion attempts (1.53 vs 2.07, mean difference = −0.47, 95% C: −0.73 to −0.21).¹⁵ The primary outcome of LP success trended toward favoring POCUS, but it was not statistically significant (88.4% vs 74.0%, OR = 2.55, 95% C: 0.99-6.52). We believe that recent evidence suggests that there is benefit in using POCUS when hospitalists attempt pediatric LPs, particularly when physical landmarks are difficult to identify or after failed attempts. However, adequate training with simulation and supervised practice should be undertaken before integrating this into clinical practice.

Evidence accumulated in the past five years has built on previous work suggesting that POCUS has a role in the diagnosis of pneumonia and skin abscess and in the performance of LPs. However, gaps in the literature remain when applying POCUS in PHM. Only a few studies to date were conducted in non-CC inpatient settings, and although several pediatric hospitalists work in EDs or care for critically ill children, our largest population comprises general pediatric ward patients. Studies have also used ultrasonographers with variable POCUS training and clinical experience, which makes comparing or combining studies challenging since POCUS is dependent on provider skills. Studies involving PHM providers and inpatient populations are needed. Additional studies evaluating the process and outcome measures are also needed to understand whether the theoretical advantages are consistently realized in real-world PHM practice. Finally, PHM-specific curricula should be designed in collaboration with various PHM stakeholders and with specialties who already have robust POCUS training pathways. There is opportunity within PHM for multi institutional research collaboration, identification of best practices, and development of PHM-specific training for fellowship and faculty development programs.

Research, in the field of Hospital Medicine, often leverages data collected for reasons other than research. For example, electronic medical record data or patient satisfaction survey results can be used to answer questions that are relevant to the practice of hospital medicine. In these types of datasets, data will inevitably be missing. Such missing data can compromise our ability to draw definitive conclusions from our research study. This review introduces the concept of missing data, describes patterns and mechanisms of missing data, and discusses common approaches for the handling of missing data, including sensitivity analyses for determining how robust the results are despite assumptions made about the missing data.

Missing data create a host of problems for researchers. First, missing data result in a loss of information and can diminish the power of the proposed study. Second, the irregular data complicate the analysis because many of the standard software procedures used have been developed for fully observed or “complete” data (ie, each subject has a value for all measures of interest). Finally, missing data may introduce bias due to the systematic difference between the observed and the unobserved data. For example, if men are less likely than women to complete all questions in a patient satisfaction survey when they are not satisfied, then hospital satisfaction analyses that rely on completed surveys would tend to provide biased estimates of the satisfaction males have with their care.

The ideal approach to mitigating problems caused by missing data is to anticipate and incorporate strategies to minimize missing data into the study design (ie, when planning data collection protocols for prospective studies). This plan should provide strategies for minimizing nonresponse and estimating the magnitude of anticipated missing data to ensure that the study achieves sufficient strength despite the missing data.

Strategies for minimizing nonresponse include (1) informing potential study participants, at initial contact, about the implications of missing data on the ability to answer the research question; (2) collecting several phone numbers, addresses, preferred method of contact and calling times, as well as an alternative contact, in case the primary study contact is unable to be reached; (3) specifying the number of call backs, as well as the time of contact; and (4) piloting data capture questions for phrasing, clarity, and sensitivity, in order to resolve problems before initiating the study. One approach that can be used to mitigate the impact of missing data in surveys is to contact a sample of the initial nonrespondents using a more intensive follow-up approach (eg, a nonresponse to a mailed survey is followed up by a telephone call in order to conduct the survey again over the phone), and this is referred to as “nonresponse two-phase sampling.” The additional data, captured in the second phase, not only reduces the nonresponse rate but can also provide important information on the missing data mechanism.^1,2 In longitudinal studies with dropouts, one can measure participants’ intent to drop out in order to evaluate how much the probability of dropping out depends on missing responses.³ One may also choose to determine the power and implications of sample size under different missing data assumptions.⁴

Different data sources are likely to have unique reasons for missing values due to the workflows involved in how the data are collected. In research involving the use of data from electronic medical records, missing data on specific diagnoses involving patients who are regularly engaged in care are often considered to be “not present” or “normal”, since clinical documentation workflows are largely governed by the concept of “documentation by exception” in which diagnoses are documented only when there is an exception to the expectation that these are not present. For example, “diabetes mellitus” is commonly documented, but “diabetes mellitus not present” is rarely documented in electronic medical records which are used for clinical care. Thus, lack of explicit documentation is likely to indicate that diabetes mellitus is, in fact, not present.

Certain variables may be missing simply because there is no quantifiable value­—ie, the data do not exist. Structural missingness refers to a value that does not exist for a logical reason (eg, “What is the gender of your first child?” for those who do not have a child). Censoring, which occurs during “time to event” analysis, refers to a situation where information about a subject stops before the event of interest happens, for example, when a subject in a study involving a 30-day outcome dies at day 14. The term “limit of detection” refers to the lowest or highest level at which two distinct values can reasonably be distinguished (eg, the lower limit of detection of a C-reactive protein assay may be 1 mg/dL, so lower values might simply be reported by the lab as <1 mg/dL).⁵ These types of missing data require specific methods that are not discussed in this review.

These examples illustrate that approaches to dealing with missing data vary depending on what data sources are used and how data are collected. Understanding the reasons missing data are present is a necessary step in formulating a robust analytic approach to handling missing data.

Missing Data Patterns

Evaluating missing data patterns provides information on the degree and complexity of the missing data problem and can aid in choosing an appropriate missing data handling method. This is because some analytic methods work well for a general pattern (nonmonotone) and other methods work for special patterns (eg, monotone, file matching). In longitudinal studies, missing data is commonly missing in a monotone pattern, where once one variable is missing then all subsequent variables are also missing for a particular subject. This occurs when a study participant is lost to follow-up. For example, a monotone missing data pattern may occur in a study that requires a series of follow-up visits for laboratory blood tests. If a patient drops out, it results in a monotone missing data pattern, as no data on blood test results are available once the patient drops out. If the patient just skips an intermediate visit but returns for the final blood test, this would show a nonmonotone missing data pattern. A file-matching pattern occurs when variables are never observed together. This pattern can occur when data from several studies are merged and some variables are not collected in all studies. For example, three studies are merged and all three collect blood pressure, but only one study collects age and only one study collects sex.

Missing Data Mechanisms

The missing data mechanism relates to the underlying reasons for missing values and the relationships between variables with and without missing data. In general, missing data can be either random or nonrandom with distinctions in randomness made by three types: (1) data missing completely at random (MCAR); (2) data missing at random (MAR); and (3) data missing not at random (MNAR).⁶ As with the missing data pattern, understanding the missing data mechanism can aid in selecting an appropriate approach to handling the missing data.

Data are MCAR if the missingness does not depend on any study variables, meaning that all subjects are equally likely to be missing certain data elements. When the data are MCAR, those with missing values can be viewed as a simple random sample from the complete (but never actually observed) data and can be dropped from analysis without causing bias in the results. If the values of some diagnostic tests were missing for some patients due to equipment malfunction or electricity outage, for example, then the missingness may be considered MCAR.

Data are MAR if the missingness depends on the observed characteristics but not the unobserved characteristics, meaning that the relationships observed in the data can be used to predict the occurrence of missing values. Because the “randomness” of MAR is conditional on observed characteristics, which distinguishes it from the “completely at random” type of MCAR, dropping or omitting those cases with missing values from the analysis may lead to biased results.⁷ In a study of quality of life (QOL) for patients with mild to moderate traumatic brain injury, if health-related QOL questions were not answered by some patients with high pain levels (even though the pain levels were recorded), the missingness of QOL may be considered as MAR. This is due to the fact that within subjects grouped by the observed characteristic of pain (that is, conditional on similar levels of pain) the missingness of QOL is the result of chance and does not depend on the values (observed or unobserved) of QOL. It follows then, that once grouped into a high (or low) pain stratum, if QOL is considered MAR, then, whether or not it is observed, is random.

Data are considered MNAR if their missingness depends on characteristics that are not observed and cannot be fully explained by the observed characteristics. Systematic differences between missing and nonmissing data exist for data that is MNAR. For example, if a survey of household income had an increased probability of missing incomes from the low-income families then the data would be considered as MNAR.

Randomness in the missing data mechanism may be ignored without affecting the inference in some circumstances.⁸ Both MCAR and MAR can be considered as “ignorable” in the sense that a proper method (eg, multiple imputation) may recover the missing information without modeling (ie, accounting for) the random process of the missing data mechanism (Table).⁹ In contrast, the MNAR mechanism requires a method that takes into account the missing data mechanism in order to make inferences about the complete (and partially unobserved) data; or in other words, a model for the missing data mechanism cannot be ignored. It is for this reason that the MNAR mechanism is often called “nonignorable”. Nonignorable missing data present a challenge to researchers because the mechanism underlying the missingness must be included in the analysis. Yet researchers rarely know what the missingness mechanism is, and the data needed to validate any putative mechanism is, in fact, missing. In cases when more than one variable is subject to missingness, researchers need to assess the missingness mechanism for each variable and tailor their approach to the specific missing data problems.⁹

There is no universally accepted standard to guide when statistical methods should be applied to account for missing data. The amount of missing data alone cannot fully assess the missing data problem; missing data patterns and mechanisms can have greater impact on research results than the proportion of missing data alone. A good statistical method for handling missing data should provide an unbiased estimate of the quantity that the investigators intend to estimate; make use of the partial information in the incomplete cases to improve efficiency (and in most cases also to reduce bias); and provide valid estimates of the standard errors, confidence intervals, and P values for statistical tests. There are generally four broadly defined classes of methods for handling missing data in clinical research: (1) the complete-case analysis, (2) single imputation methods, (3) the weighted estimating-equation approach, and (4) the model-based approach including maximum likelihood (ML) and multiple imputation (Table and Appendix).¹⁰

Since missing data mechanisms cannot be conclusively verified, it is good practice to conduct some sensitivity analyses to test the robustness of the primary results. For this purpose, pattern-mixture models provide a flexible framework for implementing sensitivity analyses to missing data assumptions and can be used to evaluate the possibility of the data being MNAR. In this framework, the missing data distribution is modeled and then incorporated into the outcome model of interest. Tipping-point analysis is a sensitivity analysis where the missing data is replaced with a range of values to determine how much the values must change for the results of the study to tip from significant to not significant. If the same general conclusions remain valid over a range of assumptions about the missing data values, then one can have greater confidence in the study conclusions.

In dealing with missing data from clinical research, clinicians and statisticians need to work together to minimize missingness at the data collection stage, document the reasons for missingness, use substantive knowledge, if possible, to assess the missing data mechanism, perform primary analysis based on a defensible missing data mechanism, and conduct a sensitivity analysis to assess whether the primary result is robust despite departure from the assumed missing data mechanism.

Acknowledgments

The following members of the Journal of Hospital Medicine Leadership team contributed to this review: Mel L. Anderson, MD; Peter Cram, MD, MBA; JoAnna K. Leyenaar, MD, PhD, MPH; Brian P. Lucas, MD, MS; Oanh Nguyen, MD, MAS; Samir S. Shah, MD, MSCE; Erin E. Shaughnessy, MD, MSHCM; and Heidi J. Sucharew, PhD.

Research, in the field of Hospital Medicine, often leverages data collected for reasons other than research. For example, electronic medical record data or patient satisfaction survey results can be used to answer questions that are relevant to the practice of hospital medicine. In these types of datasets, data will inevitably be missing. Such missing data can compromise our ability to draw definitive conclusions from our research study. This review introduces the concept of missing data, describes patterns and mechanisms of missing data, and discusses common approaches for the handling of missing data, including sensitivity analyses for determining how robust the results are despite assumptions made about the missing data.

Missing data create a host of problems for researchers. First, missing data result in a loss of information and can diminish the power of the proposed study. Second, the irregular data complicate the analysis because many of the standard software procedures used have been developed for fully observed or “complete” data (ie, each subject has a value for all measures of interest). Finally, missing data may introduce bias due to the systematic difference between the observed and the unobserved data. For example, if men are less likely than women to complete all questions in a patient satisfaction survey when they are not satisfied, then hospital satisfaction analyses that rely on completed surveys would tend to provide biased estimates of the satisfaction males have with their care.

The ideal approach to mitigating problems caused by missing data is to anticipate and incorporate strategies to minimize missing data into the study design (ie, when planning data collection protocols for prospective studies). This plan should provide strategies for minimizing nonresponse and estimating the magnitude of anticipated missing data to ensure that the study achieves sufficient strength despite the missing data.

Strategies for minimizing nonresponse include (1) informing potential study participants, at initial contact, about the implications of missing data on the ability to answer the research question; (2) collecting several phone numbers, addresses, preferred method of contact and calling times, as well as an alternative contact, in case the primary study contact is unable to be reached; (3) specifying the number of call backs, as well as the time of contact; and (4) piloting data capture questions for phrasing, clarity, and sensitivity, in order to resolve problems before initiating the study. One approach that can be used to mitigate the impact of missing data in surveys is to contact a sample of the initial nonrespondents using a more intensive follow-up approach (eg, a nonresponse to a mailed survey is followed up by a telephone call in order to conduct the survey again over the phone), and this is referred to as “nonresponse two-phase sampling.” The additional data, captured in the second phase, not only reduces the nonresponse rate but can also provide important information on the missing data mechanism.^1,2 In longitudinal studies with dropouts, one can measure participants’ intent to drop out in order to evaluate how much the probability of dropping out depends on missing responses.³ One may also choose to determine the power and implications of sample size under different missing data assumptions.⁴

Different data sources are likely to have unique reasons for missing values due to the workflows involved in how the data are collected. In research involving the use of data from electronic medical records, missing data on specific diagnoses involving patients who are regularly engaged in care are often considered to be “not present” or “normal”, since clinical documentation workflows are largely governed by the concept of “documentation by exception” in which diagnoses are documented only when there is an exception to the expectation that these are not present. For example, “diabetes mellitus” is commonly documented, but “diabetes mellitus not present” is rarely documented in electronic medical records which are used for clinical care. Thus, lack of explicit documentation is likely to indicate that diabetes mellitus is, in fact, not present.

Certain variables may be missing simply because there is no quantifiable value­—ie, the data do not exist. Structural missingness refers to a value that does not exist for a logical reason (eg, “What is the gender of your first child?” for those who do not have a child). Censoring, which occurs during “time to event” analysis, refers to a situation where information about a subject stops before the event of interest happens, for example, when a subject in a study involving a 30-day outcome dies at day 14. The term “limit of detection” refers to the lowest or highest level at which two distinct values can reasonably be distinguished (eg, the lower limit of detection of a C-reactive protein assay may be 1 mg/dL, so lower values might simply be reported by the lab as <1 mg/dL).⁵ These types of missing data require specific methods that are not discussed in this review.

These examples illustrate that approaches to dealing with missing data vary depending on what data sources are used and how data are collected. Understanding the reasons missing data are present is a necessary step in formulating a robust analytic approach to handling missing data.

Missing Data Patterns

Evaluating missing data patterns provides information on the degree and complexity of the missing data problem and can aid in choosing an appropriate missing data handling method. This is because some analytic methods work well for a general pattern (nonmonotone) and other methods work for special patterns (eg, monotone, file matching). In longitudinal studies, missing data is commonly missing in a monotone pattern, where once one variable is missing then all subsequent variables are also missing for a particular subject. This occurs when a study participant is lost to follow-up. For example, a monotone missing data pattern may occur in a study that requires a series of follow-up visits for laboratory blood tests. If a patient drops out, it results in a monotone missing data pattern, as no data on blood test results are available once the patient drops out. If the patient just skips an intermediate visit but returns for the final blood test, this would show a nonmonotone missing data pattern. A file-matching pattern occurs when variables are never observed together. This pattern can occur when data from several studies are merged and some variables are not collected in all studies. For example, three studies are merged and all three collect blood pressure, but only one study collects age and only one study collects sex.

Missing Data Mechanisms

The missing data mechanism relates to the underlying reasons for missing values and the relationships between variables with and without missing data. In general, missing data can be either random or nonrandom with distinctions in randomness made by three types: (1) data missing completely at random (MCAR); (2) data missing at random (MAR); and (3) data missing not at random (MNAR).⁶ As with the missing data pattern, understanding the missing data mechanism can aid in selecting an appropriate approach to handling the missing data.

Data are MCAR if the missingness does not depend on any study variables, meaning that all subjects are equally likely to be missing certain data elements. When the data are MCAR, those with missing values can be viewed as a simple random sample from the complete (but never actually observed) data and can be dropped from analysis without causing bias in the results. If the values of some diagnostic tests were missing for some patients due to equipment malfunction or electricity outage, for example, then the missingness may be considered MCAR.

Data are MAR if the missingness depends on the observed characteristics but not the unobserved characteristics, meaning that the relationships observed in the data can be used to predict the occurrence of missing values. Because the “randomness” of MAR is conditional on observed characteristics, which distinguishes it from the “completely at random” type of MCAR, dropping or omitting those cases with missing values from the analysis may lead to biased results.⁷ In a study of quality of life (QOL) for patients with mild to moderate traumatic brain injury, if health-related QOL questions were not answered by some patients with high pain levels (even though the pain levels were recorded), the missingness of QOL may be considered as MAR. This is due to the fact that within subjects grouped by the observed characteristic of pain (that is, conditional on similar levels of pain) the missingness of QOL is the result of chance and does not depend on the values (observed or unobserved) of QOL. It follows then, that once grouped into a high (or low) pain stratum, if QOL is considered MAR, then, whether or not it is observed, is random.

Data are considered MNAR if their missingness depends on characteristics that are not observed and cannot be fully explained by the observed characteristics. Systematic differences between missing and nonmissing data exist for data that is MNAR. For example, if a survey of household income had an increased probability of missing incomes from the low-income families then the data would be considered as MNAR.

Randomness in the missing data mechanism may be ignored without affecting the inference in some circumstances.⁸ Both MCAR and MAR can be considered as “ignorable” in the sense that a proper method (eg, multiple imputation) may recover the missing information without modeling (ie, accounting for) the random process of the missing data mechanism (Table).⁹ In contrast, the MNAR mechanism requires a method that takes into account the missing data mechanism in order to make inferences about the complete (and partially unobserved) data; or in other words, a model for the missing data mechanism cannot be ignored. It is for this reason that the MNAR mechanism is often called “nonignorable”. Nonignorable missing data present a challenge to researchers because the mechanism underlying the missingness must be included in the analysis. Yet researchers rarely know what the missingness mechanism is, and the data needed to validate any putative mechanism is, in fact, missing. In cases when more than one variable is subject to missingness, researchers need to assess the missingness mechanism for each variable and tailor their approach to the specific missing data problems.⁹

There is no universally accepted standard to guide when statistical methods should be applied to account for missing data. The amount of missing data alone cannot fully assess the missing data problem; missing data patterns and mechanisms can have greater impact on research results than the proportion of missing data alone. A good statistical method for handling missing data should provide an unbiased estimate of the quantity that the investigators intend to estimate; make use of the partial information in the incomplete cases to improve efficiency (and in most cases also to reduce bias); and provide valid estimates of the standard errors, confidence intervals, and P values for statistical tests. There are generally four broadly defined classes of methods for handling missing data in clinical research: (1) the complete-case analysis, (2) single imputation methods, (3) the weighted estimating-equation approach, and (4) the model-based approach including maximum likelihood (ML) and multiple imputation (Table and Appendix).¹⁰

Since missing data mechanisms cannot be conclusively verified, it is good practice to conduct some sensitivity analyses to test the robustness of the primary results. For this purpose, pattern-mixture models provide a flexible framework for implementing sensitivity analyses to missing data assumptions and can be used to evaluate the possibility of the data being MNAR. In this framework, the missing data distribution is modeled and then incorporated into the outcome model of interest. Tipping-point analysis is a sensitivity analysis where the missing data is replaced with a range of values to determine how much the values must change for the results of the study to tip from significant to not significant. If the same general conclusions remain valid over a range of assumptions about the missing data values, then one can have greater confidence in the study conclusions.

In dealing with missing data from clinical research, clinicians and statisticians need to work together to minimize missingness at the data collection stage, document the reasons for missingness, use substantive knowledge, if possible, to assess the missing data mechanism, perform primary analysis based on a defensible missing data mechanism, and conduct a sensitivity analysis to assess whether the primary result is robust despite departure from the assumed missing data mechanism.

Acknowledgments

The following members of the Journal of Hospital Medicine Leadership team contributed to this review: Mel L. Anderson, MD; Peter Cram, MD, MBA; JoAnna K. Leyenaar, MD, PhD, MPH; Brian P. Lucas, MD, MS; Oanh Nguyen, MD, MAS; Samir S. Shah, MD, MSCE; Erin E. Shaughnessy, MD, MSHCM; and Heidi J. Sucharew, PhD.

Research, in the field of Hospital Medicine, often leverages data collected for reasons other than research. For example, electronic medical record data or patient satisfaction survey results can be used to answer questions that are relevant to the practice of hospital medicine. In these types of datasets, data will inevitably be missing. Such missing data can compromise our ability to draw definitive conclusions from our research study. This review introduces the concept of missing data, describes patterns and mechanisms of missing data, and discusses common approaches for the handling of missing data, including sensitivity analyses for determining how robust the results are despite assumptions made about the missing data.

Missing data create a host of problems for researchers. First, missing data result in a loss of information and can diminish the power of the proposed study. Second, the irregular data complicate the analysis because many of the standard software procedures used have been developed for fully observed or “complete” data (ie, each subject has a value for all measures of interest). Finally, missing data may introduce bias due to the systematic difference between the observed and the unobserved data. For example, if men are less likely than women to complete all questions in a patient satisfaction survey when they are not satisfied, then hospital satisfaction analyses that rely on completed surveys would tend to provide biased estimates of the satisfaction males have with their care.

The ideal approach to mitigating problems caused by missing data is to anticipate and incorporate strategies to minimize missing data into the study design (ie, when planning data collection protocols for prospective studies). This plan should provide strategies for minimizing nonresponse and estimating the magnitude of anticipated missing data to ensure that the study achieves sufficient strength despite the missing data.

Strategies for minimizing nonresponse include (1) informing potential study participants, at initial contact, about the implications of missing data on the ability to answer the research question; (2) collecting several phone numbers, addresses, preferred method of contact and calling times, as well as an alternative contact, in case the primary study contact is unable to be reached; (3) specifying the number of call backs, as well as the time of contact; and (4) piloting data capture questions for phrasing, clarity, and sensitivity, in order to resolve problems before initiating the study. One approach that can be used to mitigate the impact of missing data in surveys is to contact a sample of the initial nonrespondents using a more intensive follow-up approach (eg, a nonresponse to a mailed survey is followed up by a telephone call in order to conduct the survey again over the phone), and this is referred to as “nonresponse two-phase sampling.” The additional data, captured in the second phase, not only reduces the nonresponse rate but can also provide important information on the missing data mechanism.^1,2 In longitudinal studies with dropouts, one can measure participants’ intent to drop out in order to evaluate how much the probability of dropping out depends on missing responses.³ One may also choose to determine the power and implications of sample size under different missing data assumptions.⁴

Different data sources are likely to have unique reasons for missing values due to the workflows involved in how the data are collected. In research involving the use of data from electronic medical records, missing data on specific diagnoses involving patients who are regularly engaged in care are often considered to be “not present” or “normal”, since clinical documentation workflows are largely governed by the concept of “documentation by exception” in which diagnoses are documented only when there is an exception to the expectation that these are not present. For example, “diabetes mellitus” is commonly documented, but “diabetes mellitus not present” is rarely documented in electronic medical records which are used for clinical care. Thus, lack of explicit documentation is likely to indicate that diabetes mellitus is, in fact, not present.

Certain variables may be missing simply because there is no quantifiable value­—ie, the data do not exist. Structural missingness refers to a value that does not exist for a logical reason (eg, “What is the gender of your first child?” for those who do not have a child). Censoring, which occurs during “time to event” analysis, refers to a situation where information about a subject stops before the event of interest happens, for example, when a subject in a study involving a 30-day outcome dies at day 14. The term “limit of detection” refers to the lowest or highest level at which two distinct values can reasonably be distinguished (eg, the lower limit of detection of a C-reactive protein assay may be 1 mg/dL, so lower values might simply be reported by the lab as <1 mg/dL).⁵ These types of missing data require specific methods that are not discussed in this review.

These examples illustrate that approaches to dealing with missing data vary depending on what data sources are used and how data are collected. Understanding the reasons missing data are present is a necessary step in formulating a robust analytic approach to handling missing data.

Missing Data Patterns

Evaluating missing data patterns provides information on the degree and complexity of the missing data problem and can aid in choosing an appropriate missing data handling method. This is because some analytic methods work well for a general pattern (nonmonotone) and other methods work for special patterns (eg, monotone, file matching). In longitudinal studies, missing data is commonly missing in a monotone pattern, where once one variable is missing then all subsequent variables are also missing for a particular subject. This occurs when a study participant is lost to follow-up. For example, a monotone missing data pattern may occur in a study that requires a series of follow-up visits for laboratory blood tests. If a patient drops out, it results in a monotone missing data pattern, as no data on blood test results are available once the patient drops out. If the patient just skips an intermediate visit but returns for the final blood test, this would show a nonmonotone missing data pattern. A file-matching pattern occurs when variables are never observed together. This pattern can occur when data from several studies are merged and some variables are not collected in all studies. For example, three studies are merged and all three collect blood pressure, but only one study collects age and only one study collects sex.

Missing Data Mechanisms

The missing data mechanism relates to the underlying reasons for missing values and the relationships between variables with and without missing data. In general, missing data can be either random or nonrandom with distinctions in randomness made by three types: (1) data missing completely at random (MCAR); (2) data missing at random (MAR); and (3) data missing not at random (MNAR).⁶ As with the missing data pattern, understanding the missing data mechanism can aid in selecting an appropriate approach to handling the missing data.

Data are MCAR if the missingness does not depend on any study variables, meaning that all subjects are equally likely to be missing certain data elements. When the data are MCAR, those with missing values can be viewed as a simple random sample from the complete (but never actually observed) data and can be dropped from analysis without causing bias in the results. If the values of some diagnostic tests were missing for some patients due to equipment malfunction or electricity outage, for example, then the missingness may be considered MCAR.

Data are MAR if the missingness depends on the observed characteristics but not the unobserved characteristics, meaning that the relationships observed in the data can be used to predict the occurrence of missing values. Because the “randomness” of MAR is conditional on observed characteristics, which distinguishes it from the “completely at random” type of MCAR, dropping or omitting those cases with missing values from the analysis may lead to biased results.⁷ In a study of quality of life (QOL) for patients with mild to moderate traumatic brain injury, if health-related QOL questions were not answered by some patients with high pain levels (even though the pain levels were recorded), the missingness of QOL may be considered as MAR. This is due to the fact that within subjects grouped by the observed characteristic of pain (that is, conditional on similar levels of pain) the missingness of QOL is the result of chance and does not depend on the values (observed or unobserved) of QOL. It follows then, that once grouped into a high (or low) pain stratum, if QOL is considered MAR, then, whether or not it is observed, is random.

Data are considered MNAR if their missingness depends on characteristics that are not observed and cannot be fully explained by the observed characteristics. Systematic differences between missing and nonmissing data exist for data that is MNAR. For example, if a survey of household income had an increased probability of missing incomes from the low-income families then the data would be considered as MNAR.

Randomness in the missing data mechanism may be ignored without affecting the inference in some circumstances.⁸ Both MCAR and MAR can be considered as “ignorable” in the sense that a proper method (eg, multiple imputation) may recover the missing information without modeling (ie, accounting for) the random process of the missing data mechanism (Table).⁹ In contrast, the MNAR mechanism requires a method that takes into account the missing data mechanism in order to make inferences about the complete (and partially unobserved) data; or in other words, a model for the missing data mechanism cannot be ignored. It is for this reason that the MNAR mechanism is often called “nonignorable”. Nonignorable missing data present a challenge to researchers because the mechanism underlying the missingness must be included in the analysis. Yet researchers rarely know what the missingness mechanism is, and the data needed to validate any putative mechanism is, in fact, missing. In cases when more than one variable is subject to missingness, researchers need to assess the missingness mechanism for each variable and tailor their approach to the specific missing data problems.⁹

There is no universally accepted standard to guide when statistical methods should be applied to account for missing data. The amount of missing data alone cannot fully assess the missing data problem; missing data patterns and mechanisms can have greater impact on research results than the proportion of missing data alone. A good statistical method for handling missing data should provide an unbiased estimate of the quantity that the investigators intend to estimate; make use of the partial information in the incomplete cases to improve efficiency (and in most cases also to reduce bias); and provide valid estimates of the standard errors, confidence intervals, and P values for statistical tests. There are generally four broadly defined classes of methods for handling missing data in clinical research: (1) the complete-case analysis, (2) single imputation methods, (3) the weighted estimating-equation approach, and (4) the model-based approach including maximum likelihood (ML) and multiple imputation (Table and Appendix).¹⁰

Since missing data mechanisms cannot be conclusively verified, it is good practice to conduct some sensitivity analyses to test the robustness of the primary results. For this purpose, pattern-mixture models provide a flexible framework for implementing sensitivity analyses to missing data assumptions and can be used to evaluate the possibility of the data being MNAR. In this framework, the missing data distribution is modeled and then incorporated into the outcome model of interest. Tipping-point analysis is a sensitivity analysis where the missing data is replaced with a range of values to determine how much the values must change for the results of the study to tip from significant to not significant. If the same general conclusions remain valid over a range of assumptions about the missing data values, then one can have greater confidence in the study conclusions.

In dealing with missing data from clinical research, clinicians and statisticians need to work together to minimize missingness at the data collection stage, document the reasons for missingness, use substantive knowledge, if possible, to assess the missing data mechanism, perform primary analysis based on a defensible missing data mechanism, and conduct a sensitivity analysis to assess whether the primary result is robust despite departure from the assumed missing data mechanism.

Acknowledgments

The following members of the Journal of Hospital Medicine Leadership team contributed to this review: Mel L. Anderson, MD; Peter Cram, MD, MBA; JoAnna K. Leyenaar, MD, PhD, MPH; Brian P. Lucas, MD, MS; Oanh Nguyen, MD, MAS; Samir S. Shah, MD, MSCE; Erin E. Shaughnessy, MD, MSHCM; and Heidi J. Sucharew, PhD.

Hilda and I shared childhood stories while we enjoyed one of her favorite Mexican dishes, grilled nopalitos (cactus). Hilda loved nopalitos, but she rarely ate them because they are high in potassium. Hilda had end-stage kidney disease (ESKD), and as an undocumented Mexican immigrant in Denver, CO, she relied on emergency-only hemodialysis. Instead of receiving standard hemodialysis three times per week as required, Hilda would arrive critically ill to the hospital after her nausea, vomiting, and shortness of breath became unbearable. After three cardiac arrests from high potassium levels, she fervently avoided foods high in it. This time, however, she was not worried about potassium. This was our last meal together. She would fly to Mexico a few days later to die.

Our hospital medicine team knew Hilda well. We had continuity because we had been admitting her to the intensive care unit or medicine floor one night each week to receive two hemodialysis sessions when she was critically ill. I immediately connected with Hilda because our lives were parallel in many ways. Hilda and I were both in our early 30s, English was our second language, we both grew up in poverty, and we now had children in elementary school. I, however, was documented. My United States citizenship allowed me the privilege of pursuing a medical degree and gaining access to quality healthcare. In contrast, Hilda had been forced to end her education prematurely, marry her mother’s friend for financial stability at the age of 14, and eventually flee to the US to escape poverty. She survived by cleaning homes until her kidneys failed. Initially, Hilda was my patient. Over time, she became a dear friend.

The first two years of emergency-only hemodialysis devastated Hilda. Too sick to work, she became homeless, staying with a nurse until we found a shelter for single mothers. Multiple cardiac arrests and resuscitations traumatized her young sons, who called 911 each time she collapsed and witnessed the resuscitations. Her boys did not understand the cycle of separation from their mother for her emergent, weekly dialysis hospital admissions and wondered if she would survive to the following week. After two years of emergency-only dialysis, Hilda’s deep love for her boys and concern about the possibility that her sudden death could leave them alone led her to pre-emptively decide to stop emergency-only dialysis. Had Hilda’s treatment costs been covered by emergency Medicaid, as undocumented immigrants with ESKD are in some other states, she may not have been forced into this terrible decision. Moving to a state where standard dialysis is covered was not an option for Hilda because she wanted her boys to stay in Colorado where they had family and friends. With no other options, she first sought a loving adoptive family in the US so that her boys could grow up and have the opportunity to pursue an education. After carefully finding the right adoptive parents, Hilda wanted to celebrate her life with the people she loved. To show her gratitude, she organized a large Mexican Christmas party and invited all of the healthcare providers and friends that had supported her. She generously gave everyone a small gift to remember her by from the few things she owned. I received the wooden rosary her father had left her. A short while later, Hilda flew home to Mexico and passed away on Mother’s Day in 2014.

Two years of caring for Hilda as an internal medicine hospitalist changed me. Grief gave way to anger, anger to determination. I found it morally distressing to continue to provide this type of care. Something had to change and there was little research in this area. One small study had demonstrated that emergency-only hemodialysis was nearly four-fold more expensive due to additional visits to the emergency department and admissions to the hospital, compared to standard outpatient hemodialysis.¹ After much soul-searching and advice seeking, I scaled down my clinical hospitalist shifts and gathered a team to do research. For four years, we worked on illuminating the suffering of undocumented immigrants with ESKD that rely on emergency-only hemodialysis. We conducted 20 individual face-to-face qualitative interviews with undocumented immigrants with ESKD and heard first-hand about the emotional and physical burdens and the existential anxiety associated with weekly threats to life.² We published a retrospective cohort study looking at differences in mortality and found that immigrants who relied on emergency-only hemodialysis had a 14-fold greater mortality rate than those on standard hemodialysis five years after initiating hemodialysis.³ In another retrospective study, we described the circumstances among undocumented immigrants with ESKD who died in the hospital after presenting with ESKD complications, and found that the majority presented with high potassium and a recorded rhythm disturbance.⁴ I discovered that as a hospitalist physician, I was not the only one distressed. We conducted 50 qualitative interviews to determine the perspectives of interdisciplinary clinicians on providing emergency dialysis and found that there are more clinicians experiencing moral distress. They described several important drivers of burnout,⁵ including emotional exhaustion from witnessing needless suffering and high mortality, as well as physical exhaustion from overextending themselves to bridge their patient’s care. Together, we discovered that the research told the larger narrative behind Hilda’s struggles. These publications caught the attention of the media and enabled us to speak to a wider audience of clinicians, health policy makers, and the general public.^6-10 They also became a catalyst to engaging and enlisting the good will and interest of a number of key stakeholders to look for solutions.

In the US, undocumented immigrants do not qualify for insurance through traditional Medicaid, Medicare, or the provisions from the Patient Protection and Affordable Care Act. Instead, emergency Medicaid provides reimbursements for care of undocumented immigrants. According to the 1986 Emergency Medicaid Treatment and Active Labor Act, federal Medicaid payments can only be made for the care of undocumented immigrants if care is necessary for the treatment of an emergency medical condition.¹¹ However, the Centers for Medicare and Medicaid (CMS) has outlined certain conditions that cannot qualify for matching federal funds under emergency Medicaid (ie, organ transplant and routine prenatal or postpartum care). Beyond these requirements, federal CMS and the Office of the Inspector General defer to states to define what constitutes a medical emergency. A few states include ESKD in the definition of “emergency medical condition,” thereby expanding access to standard hemodialysis to undocumented immigrants. We wanted Colorado to join that list.

On August 2018, after four years of research and months of dialog, everything changed: Colorado Medicaid announced that ESKD was now an “emergency medical condition.” As simple as that, undocumented immigrants would receive standard maintenance hemodialysis. Tears streamed down my face as I read a message from a policy specialist from the Colorado Medicaid: Your team “played a big role in bringing awareness to this issue, and your advocacy for these patients is impressive … thank you for fighting for such an important cause.” I reread her message, imagining what this would have meant to Hilda and her boys.

Our work to enhance care in this community is not over. To better understand the provision of dialysis care for undocumented immigrants in the United States, our team reviewed the Medicaid language for each of the 50 US states in addition to connecting with clinicians and organizations (eg, National Kidney Foundation and ESKD Networks). We found that only 12 states provide Medicaid reimbursement for standard dialysis and that a majority of the US states do not currently define need for dialysis as an emergency medical condition.¹² As our Colorado team works with stakeholders in other states interested in similarly redefining their state’s emergency Medicaid definition, our most important advice is that advocacy is a team-based effort. There may be resistance and some may argue that expanding access to care would be an economic burden on taxpayers; however, research demonstrates that undocumented immigrants contribute more to the US Medicare Trust Fund than they actually withdraw toward healthcare.¹³ Furthermore, a new study has demonstrated that a net savings of nearly $6,000 per person per month is realized when patients are transitioned from emergency-only hemodialysis to standard hemodialysis.¹⁴

Internal medicine hospitalists on the front-line of healthcare systems are regular witnesses to its horrible injustices. We rarely share our perspectives and do not expect change to follow. With Hilda, we saw how a powerful combination of research and coalition building could lift one patient’s tragic story to a level where it could produce change. Augmenting Hilda’s experience of tragically poor access to care with evidence-based research gave her story validity far beyond our immediate circle of friends and colleagues, making a singular tragedy, policy relevant. Each time we shared our research to community advocacy groups, health policy stakeholders, state legislators, nurses, and staff; we began with Hilda’s story, not just because it inspired us, but because its truth was undeniable. Our patients’ stories matter, and it is our responsibility to tell them.

Each time I prepare nopalitos for my family, I think of my last meal with Hilda. No matter how painful or difficult her struggle with ESKD, Hilda persisted. She protected her boys. They were her purpose. When she knew she could no longer give them the life she wanted for them, she found a family who would. Hilda’s sons now live with a loving adoptive family, are thriving in school, and her oldest is interested in becoming a physician. Nopal, or cactus, symbolizes such endurance—a plant with unique adaptations and strength that can flourish under extreme environmental stress. Like a cactus storing precious water, Hilda treasured her children, and her resolve to provide for them was unstoppable, right to the edge of death. When our team first took up Hilda’s cause, change seemed impossible. We discovered the opposite. As I clench the wooden rosary she left me that Christmas, I thank her for giving our team the courage to adapt and persist, for in doing so we found a path, first to research and then to broader partnerships and more meaningful policy changes.

Acknowledgments

The author would like to thank Hilda, her family, and the patients at Denver Health. She would also like to acknowledge Hilda’s family, Drs. Mark Earnest, John F. Steiner, Romana Hasnain-Wynia, Rudolph Rodriguez, Judy Regensteiner, and Michel Chonchol for reading and providing feedback on earlier drafts of this narrative.

Hilda and I shared childhood stories while we enjoyed one of her favorite Mexican dishes, grilled nopalitos (cactus). Hilda loved nopalitos, but she rarely ate them because they are high in potassium. Hilda had end-stage kidney disease (ESKD), and as an undocumented Mexican immigrant in Denver, CO, she relied on emergency-only hemodialysis. Instead of receiving standard hemodialysis three times per week as required, Hilda would arrive critically ill to the hospital after her nausea, vomiting, and shortness of breath became unbearable. After three cardiac arrests from high potassium levels, she fervently avoided foods high in it. This time, however, she was not worried about potassium. This was our last meal together. She would fly to Mexico a few days later to die.

Our hospital medicine team knew Hilda well. We had continuity because we had been admitting her to the intensive care unit or medicine floor one night each week to receive two hemodialysis sessions when she was critically ill. I immediately connected with Hilda because our lives were parallel in many ways. Hilda and I were both in our early 30s, English was our second language, we both grew up in poverty, and we now had children in elementary school. I, however, was documented. My United States citizenship allowed me the privilege of pursuing a medical degree and gaining access to quality healthcare. In contrast, Hilda had been forced to end her education prematurely, marry her mother’s friend for financial stability at the age of 14, and eventually flee to the US to escape poverty. She survived by cleaning homes until her kidneys failed. Initially, Hilda was my patient. Over time, she became a dear friend.

The first two years of emergency-only hemodialysis devastated Hilda. Too sick to work, she became homeless, staying with a nurse until we found a shelter for single mothers. Multiple cardiac arrests and resuscitations traumatized her young sons, who called 911 each time she collapsed and witnessed the resuscitations. Her boys did not understand the cycle of separation from their mother for her emergent, weekly dialysis hospital admissions and wondered if she would survive to the following week. After two years of emergency-only dialysis, Hilda’s deep love for her boys and concern about the possibility that her sudden death could leave them alone led her to pre-emptively decide to stop emergency-only dialysis. Had Hilda’s treatment costs been covered by emergency Medicaid, as undocumented immigrants with ESKD are in some other states, she may not have been forced into this terrible decision. Moving to a state where standard dialysis is covered was not an option for Hilda because she wanted her boys to stay in Colorado where they had family and friends. With no other options, she first sought a loving adoptive family in the US so that her boys could grow up and have the opportunity to pursue an education. After carefully finding the right adoptive parents, Hilda wanted to celebrate her life with the people she loved. To show her gratitude, she organized a large Mexican Christmas party and invited all of the healthcare providers and friends that had supported her. She generously gave everyone a small gift to remember her by from the few things she owned. I received the wooden rosary her father had left her. A short while later, Hilda flew home to Mexico and passed away on Mother’s Day in 2014.

Two years of caring for Hilda as an internal medicine hospitalist changed me. Grief gave way to anger, anger to determination. I found it morally distressing to continue to provide this type of care. Something had to change and there was little research in this area. One small study had demonstrated that emergency-only hemodialysis was nearly four-fold more expensive due to additional visits to the emergency department and admissions to the hospital, compared to standard outpatient hemodialysis.¹ After much soul-searching and advice seeking, I scaled down my clinical hospitalist shifts and gathered a team to do research. For four years, we worked on illuminating the suffering of undocumented immigrants with ESKD that rely on emergency-only hemodialysis. We conducted 20 individual face-to-face qualitative interviews with undocumented immigrants with ESKD and heard first-hand about the emotional and physical burdens and the existential anxiety associated with weekly threats to life.² We published a retrospective cohort study looking at differences in mortality and found that immigrants who relied on emergency-only hemodialysis had a 14-fold greater mortality rate than those on standard hemodialysis five years after initiating hemodialysis.³ In another retrospective study, we described the circumstances among undocumented immigrants with ESKD who died in the hospital after presenting with ESKD complications, and found that the majority presented with high potassium and a recorded rhythm disturbance.⁴ I discovered that as a hospitalist physician, I was not the only one distressed. We conducted 50 qualitative interviews to determine the perspectives of interdisciplinary clinicians on providing emergency dialysis and found that there are more clinicians experiencing moral distress. They described several important drivers of burnout,⁵ including emotional exhaustion from witnessing needless suffering and high mortality, as well as physical exhaustion from overextending themselves to bridge their patient’s care. Together, we discovered that the research told the larger narrative behind Hilda’s struggles. These publications caught the attention of the media and enabled us to speak to a wider audience of clinicians, health policy makers, and the general public.^6-10 They also became a catalyst to engaging and enlisting the good will and interest of a number of key stakeholders to look for solutions.

In the US, undocumented immigrants do not qualify for insurance through traditional Medicaid, Medicare, or the provisions from the Patient Protection and Affordable Care Act. Instead, emergency Medicaid provides reimbursements for care of undocumented immigrants. According to the 1986 Emergency Medicaid Treatment and Active Labor Act, federal Medicaid payments can only be made for the care of undocumented immigrants if care is necessary for the treatment of an emergency medical condition.¹¹ However, the Centers for Medicare and Medicaid (CMS) has outlined certain conditions that cannot qualify for matching federal funds under emergency Medicaid (ie, organ transplant and routine prenatal or postpartum care). Beyond these requirements, federal CMS and the Office of the Inspector General defer to states to define what constitutes a medical emergency. A few states include ESKD in the definition of “emergency medical condition,” thereby expanding access to standard hemodialysis to undocumented immigrants. We wanted Colorado to join that list.

On August 2018, after four years of research and months of dialog, everything changed: Colorado Medicaid announced that ESKD was now an “emergency medical condition.” As simple as that, undocumented immigrants would receive standard maintenance hemodialysis. Tears streamed down my face as I read a message from a policy specialist from the Colorado Medicaid: Your team “played a big role in bringing awareness to this issue, and your advocacy for these patients is impressive … thank you for fighting for such an important cause.” I reread her message, imagining what this would have meant to Hilda and her boys.

Our work to enhance care in this community is not over. To better understand the provision of dialysis care for undocumented immigrants in the United States, our team reviewed the Medicaid language for each of the 50 US states in addition to connecting with clinicians and organizations (eg, National Kidney Foundation and ESKD Networks). We found that only 12 states provide Medicaid reimbursement for standard dialysis and that a majority of the US states do not currently define need for dialysis as an emergency medical condition.¹² As our Colorado team works with stakeholders in other states interested in similarly redefining their state’s emergency Medicaid definition, our most important advice is that advocacy is a team-based effort. There may be resistance and some may argue that expanding access to care would be an economic burden on taxpayers; however, research demonstrates that undocumented immigrants contribute more to the US Medicare Trust Fund than they actually withdraw toward healthcare.¹³ Furthermore, a new study has demonstrated that a net savings of nearly $6,000 per person per month is realized when patients are transitioned from emergency-only hemodialysis to standard hemodialysis.¹⁴

Internal medicine hospitalists on the front-line of healthcare systems are regular witnesses to its horrible injustices. We rarely share our perspectives and do not expect change to follow. With Hilda, we saw how a powerful combination of research and coalition building could lift one patient’s tragic story to a level where it could produce change. Augmenting Hilda’s experience of tragically poor access to care with evidence-based research gave her story validity far beyond our immediate circle of friends and colleagues, making a singular tragedy, policy relevant. Each time we shared our research to community advocacy groups, health policy stakeholders, state legislators, nurses, and staff; we began with Hilda’s story, not just because it inspired us, but because its truth was undeniable. Our patients’ stories matter, and it is our responsibility to tell them.

Each time I prepare nopalitos for my family, I think of my last meal with Hilda. No matter how painful or difficult her struggle with ESKD, Hilda persisted. She protected her boys. They were her purpose. When she knew she could no longer give them the life she wanted for them, she found a family who would. Hilda’s sons now live with a loving adoptive family, are thriving in school, and her oldest is interested in becoming a physician. Nopal, or cactus, symbolizes such endurance—a plant with unique adaptations and strength that can flourish under extreme environmental stress. Like a cactus storing precious water, Hilda treasured her children, and her resolve to provide for them was unstoppable, right to the edge of death. When our team first took up Hilda’s cause, change seemed impossible. We discovered the opposite. As I clench the wooden rosary she left me that Christmas, I thank her for giving our team the courage to adapt and persist, for in doing so we found a path, first to research and then to broader partnerships and more meaningful policy changes.

Acknowledgments

The author would like to thank Hilda, her family, and the patients at Denver Health. She would also like to acknowledge Hilda’s family, Drs. Mark Earnest, John F. Steiner, Romana Hasnain-Wynia, Rudolph Rodriguez, Judy Regensteiner, and Michel Chonchol for reading and providing feedback on earlier drafts of this narrative.

Hilda and I shared childhood stories while we enjoyed one of her favorite Mexican dishes, grilled nopalitos (cactus). Hilda loved nopalitos, but she rarely ate them because they are high in potassium. Hilda had end-stage kidney disease (ESKD), and as an undocumented Mexican immigrant in Denver, CO, she relied on emergency-only hemodialysis. Instead of receiving standard hemodialysis three times per week as required, Hilda would arrive critically ill to the hospital after her nausea, vomiting, and shortness of breath became unbearable. After three cardiac arrests from high potassium levels, she fervently avoided foods high in it. This time, however, she was not worried about potassium. This was our last meal together. She would fly to Mexico a few days later to die.

Our hospital medicine team knew Hilda well. We had continuity because we had been admitting her to the intensive care unit or medicine floor one night each week to receive two hemodialysis sessions when she was critically ill. I immediately connected with Hilda because our lives were parallel in many ways. Hilda and I were both in our early 30s, English was our second language, we both grew up in poverty, and we now had children in elementary school. I, however, was documented. My United States citizenship allowed me the privilege of pursuing a medical degree and gaining access to quality healthcare. In contrast, Hilda had been forced to end her education prematurely, marry her mother’s friend for financial stability at the age of 14, and eventually flee to the US to escape poverty. She survived by cleaning homes until her kidneys failed. Initially, Hilda was my patient. Over time, she became a dear friend.

The first two years of emergency-only hemodialysis devastated Hilda. Too sick to work, she became homeless, staying with a nurse until we found a shelter for single mothers. Multiple cardiac arrests and resuscitations traumatized her young sons, who called 911 each time she collapsed and witnessed the resuscitations. Her boys did not understand the cycle of separation from their mother for her emergent, weekly dialysis hospital admissions and wondered if she would survive to the following week. After two years of emergency-only dialysis, Hilda’s deep love for her boys and concern about the possibility that her sudden death could leave them alone led her to pre-emptively decide to stop emergency-only dialysis. Had Hilda’s treatment costs been covered by emergency Medicaid, as undocumented immigrants with ESKD are in some other states, she may not have been forced into this terrible decision. Moving to a state where standard dialysis is covered was not an option for Hilda because she wanted her boys to stay in Colorado where they had family and friends. With no other options, she first sought a loving adoptive family in the US so that her boys could grow up and have the opportunity to pursue an education. After carefully finding the right adoptive parents, Hilda wanted to celebrate her life with the people she loved. To show her gratitude, she organized a large Mexican Christmas party and invited all of the healthcare providers and friends that had supported her. She generously gave everyone a small gift to remember her by from the few things she owned. I received the wooden rosary her father had left her. A short while later, Hilda flew home to Mexico and passed away on Mother’s Day in 2014.

Two years of caring for Hilda as an internal medicine hospitalist changed me. Grief gave way to anger, anger to determination. I found it morally distressing to continue to provide this type of care. Something had to change and there was little research in this area. One small study had demonstrated that emergency-only hemodialysis was nearly four-fold more expensive due to additional visits to the emergency department and admissions to the hospital, compared to standard outpatient hemodialysis.¹ After much soul-searching and advice seeking, I scaled down my clinical hospitalist shifts and gathered a team to do research. For four years, we worked on illuminating the suffering of undocumented immigrants with ESKD that rely on emergency-only hemodialysis. We conducted 20 individual face-to-face qualitative interviews with undocumented immigrants with ESKD and heard first-hand about the emotional and physical burdens and the existential anxiety associated with weekly threats to life.² We published a retrospective cohort study looking at differences in mortality and found that immigrants who relied on emergency-only hemodialysis had a 14-fold greater mortality rate than those on standard hemodialysis five years after initiating hemodialysis.³ In another retrospective study, we described the circumstances among undocumented immigrants with ESKD who died in the hospital after presenting with ESKD complications, and found that the majority presented with high potassium and a recorded rhythm disturbance.⁴ I discovered that as a hospitalist physician, I was not the only one distressed. We conducted 50 qualitative interviews to determine the perspectives of interdisciplinary clinicians on providing emergency dialysis and found that there are more clinicians experiencing moral distress. They described several important drivers of burnout,⁵ including emotional exhaustion from witnessing needless suffering and high mortality, as well as physical exhaustion from overextending themselves to bridge their patient’s care. Together, we discovered that the research told the larger narrative behind Hilda’s struggles. These publications caught the attention of the media and enabled us to speak to a wider audience of clinicians, health policy makers, and the general public.^6-10 They also became a catalyst to engaging and enlisting the good will and interest of a number of key stakeholders to look for solutions.

In the US, undocumented immigrants do not qualify for insurance through traditional Medicaid, Medicare, or the provisions from the Patient Protection and Affordable Care Act. Instead, emergency Medicaid provides reimbursements for care of undocumented immigrants. According to the 1986 Emergency Medicaid Treatment and Active Labor Act, federal Medicaid payments can only be made for the care of undocumented immigrants if care is necessary for the treatment of an emergency medical condition.¹¹ However, the Centers for Medicare and Medicaid (CMS) has outlined certain conditions that cannot qualify for matching federal funds under emergency Medicaid (ie, organ transplant and routine prenatal or postpartum care). Beyond these requirements, federal CMS and the Office of the Inspector General defer to states to define what constitutes a medical emergency. A few states include ESKD in the definition of “emergency medical condition,” thereby expanding access to standard hemodialysis to undocumented immigrants. We wanted Colorado to join that list.

On August 2018, after four years of research and months of dialog, everything changed: Colorado Medicaid announced that ESKD was now an “emergency medical condition.” As simple as that, undocumented immigrants would receive standard maintenance hemodialysis. Tears streamed down my face as I read a message from a policy specialist from the Colorado Medicaid: Your team “played a big role in bringing awareness to this issue, and your advocacy for these patients is impressive … thank you for fighting for such an important cause.” I reread her message, imagining what this would have meant to Hilda and her boys.

Our work to enhance care in this community is not over. To better understand the provision of dialysis care for undocumented immigrants in the United States, our team reviewed the Medicaid language for each of the 50 US states in addition to connecting with clinicians and organizations (eg, National Kidney Foundation and ESKD Networks). We found that only 12 states provide Medicaid reimbursement for standard dialysis and that a majority of the US states do not currently define need for dialysis as an emergency medical condition.¹² As our Colorado team works with stakeholders in other states interested in similarly redefining their state’s emergency Medicaid definition, our most important advice is that advocacy is a team-based effort. There may be resistance and some may argue that expanding access to care would be an economic burden on taxpayers; however, research demonstrates that undocumented immigrants contribute more to the US Medicare Trust Fund than they actually withdraw toward healthcare.¹³ Furthermore, a new study has demonstrated that a net savings of nearly $6,000 per person per month is realized when patients are transitioned from emergency-only hemodialysis to standard hemodialysis.¹⁴

Internal medicine hospitalists on the front-line of healthcare systems are regular witnesses to its horrible injustices. We rarely share our perspectives and do not expect change to follow. With Hilda, we saw how a powerful combination of research and coalition building could lift one patient’s tragic story to a level where it could produce change. Augmenting Hilda’s experience of tragically poor access to care with evidence-based research gave her story validity far beyond our immediate circle of friends and colleagues, making a singular tragedy, policy relevant. Each time we shared our research to community advocacy groups, health policy stakeholders, state legislators, nurses, and staff; we began with Hilda’s story, not just because it inspired us, but because its truth was undeniable. Our patients’ stories matter, and it is our responsibility to tell them.

Each time I prepare nopalitos for my family, I think of my last meal with Hilda. No matter how painful or difficult her struggle with ESKD, Hilda persisted. She protected her boys. They were her purpose. When she knew she could no longer give them the life she wanted for them, she found a family who would. Hilda’s sons now live with a loving adoptive family, are thriving in school, and her oldest is interested in becoming a physician. Nopal, or cactus, symbolizes such endurance—a plant with unique adaptations and strength that can flourish under extreme environmental stress. Like a cactus storing precious water, Hilda treasured her children, and her resolve to provide for them was unstoppable, right to the edge of death. When our team first took up Hilda’s cause, change seemed impossible. We discovered the opposite. As I clench the wooden rosary she left me that Christmas, I thank her for giving our team the courage to adapt and persist, for in doing so we found a path, first to research and then to broader partnerships and more meaningful policy changes.

Acknowledgments

The author would like to thank Hilda, her family, and the patients at Denver Health. She would also like to acknowledge Hilda’s family, Drs. Mark Earnest, John F. Steiner, Romana Hasnain-Wynia, Rudolph Rodriguez, Judy Regensteiner, and Michel Chonchol for reading and providing feedback on earlier drafts of this narrative.

Internists are experts in general medicine, skilled at mapping the few hundred ways the human body can go awry onto thousands of diagnoses, and managing the uncertainty inherent in that process. Generalists, almost by definition, consult specialists with their specialty-focused questions; but who does one call for a general consultation about diagnosis if a specific diagnosis remains elusive and the pathology does not fit cleanly into the purview of a consultant? Outside of sage advice from colleagues (usually senior), most medical centers lack a consultation service focused on diagnosis. There is no oracle to seek. In this perspective, we describe our institution’s answer to this problem: the creation of a service for difficult diagnosis based on Socratic principles, particularly the role of iterative hypothesis testing in the process of diagnosis.¹

In 2015, Northwestern Medicine began the Socrates Project, a physician-to-physician consultation service that assists doctors working to diagnose conditions that have so far eluded detection. Our service’s goal is to improve patient care by providing an opinion to the referring physician on diagnostic possibilities for a particular case and ideas to reduce—or at least manage—diagnostic uncertainty.

Most patients referred to the Socrates Project have already undergone an extensive evaluation at top medical centers by experienced clinicians. It would be hubris to assume that we will find a definitive diagnosis in every case; indeed, because of the types of cases referred to our group, it is rare that we find a “Eureka!” diagnosis. When a colleague consults our group, we under-promise in hopes of over-delivering. Instead, we convey to referring physicians that we will conduct a thorough case review and explain our thinking in hopes of uncovering an additional diagnostic avenue, even if that avenue does not ultimately lead to a definitive diagnosis. In addition, the Socrates Project often serves as a broker between consulting services that are deadlocked because of differing diagnostic opinions. We also assist with cases in which a functional disorder is suspected, yet the referring physician is hesitant to diagnose a patient with such a disorder out of concern about missing an important (and possibly obscure) diagnosis.

The Socrates Project receives approximately two consult requests per week, usually from general internists but also from specialists in nearly all disciplines. Around 80% of the referrals are for current inpatients. Our service model is similar to a tumor board, which exists as an interdisciplinary group operating in parallel to the clinical services, to provide consensus-based recommendations. As a result, we act as doctors for doctors, formalizing the curbside consultation. Our usual turnaround time is a week but can be faster for urgent cases. Currently, Socrates Project members, including the faculty leader, volunteer their time and effort at no cost, and there are no charges to patients when physicians consult our group. An overview of the Socrates Project’s personnel and process are outlined in the Figure.

Northwestern’s Chief Medical Residents (CMRs) serve as the fellows for the service, and one of them assumes primary responsibility for each new consultation request the service receives. After obtaining the patient’s case history from the referring provider, the CMR then undertakes a thorough review of the electronic health record and any other available records from other institutions. In the inpatient setting, the CMR performs a new history and physical; phone calls or video conferencing permit history taking for outpatients. In contrast with the standard consultant note, we do not redocument the history, physical, and lab and imaging findings but instead construct a detailed problem list that synthesizes relevant findings into a useful working document.

The service’s faculty leader (BDS) then reviews the problem list with the CMRs to help refine the problem list and begin producing a differential diagnosis during a weekly hour-long meeting. As evidence supports team-based diagnostic collaborations,² the problem list and preliminary differential diagnosis then becomes a shareable document that the CMR or team leader presents to ad hoc general internists, specialists, and the other CMRs. The presentation can be in person, by phone, or e-mail. These ad hoc members, approximately 20 in number and spanning from junior attending physicians to senior clinicians, have volunteered to help the Socrates Project by adding their thoughts on differential diagnoses that explain the problem list and how to move forward with further testing. The ad hoc members have self-identified as clinicians with an interest in medical diagnosis—including surgeons, neurologists, psychiatrists, radiologists, and pathologists—and range in expertise from general internists to subspecialists. Finally, we document our problem list, differential diagnosis, and recommendations in the medical record and discuss the case with the referring team. The service limits its scope of clinical recommendation to diagnosis and avoids commenting on management decisions outside of the use of therapies as empiric diagnostic tests. A sample note is provided as an online Appendix.

Despite our process, we are often left without a satisfying diagnosis. We then are then faced with three possibilities: (1) The diagnosis is identifiable, just not by the physicians involved in the case—we did not think of the diagnosis in our deliberations; (2) The diagnosis is a described condition but without an available test—autoimmune limbic encephalitis associated with an unassayable or unknown auto-antibody, or the acuity of a critically ill patient makes diagnostic testing unreliable or not feasible; (3) The diagnosis has not yet been described by medical science—we are seeing a case of HIV infection in 1971.

With the personnel and process outlined above, we hope to provide recommendations that are useful in guiding a diagnostic workup regardless of which of these three scenarios is applicable. Our flexibility with involving the appropriate specialists in the Socrates Project should minimize the number of patients with a knowable diagnosis that is unknown to us. In the second scenario, our recommendations may rest upon the incorporation of a treatment as a diagnostic test. In the limbic encephalitis example above, a trial of steroids with rapid improvement in the patient’s condition may increase diagnostic certainty. The third scenario is the most difficult to identify. Pattern recognition of similarly presenting patients, keeping ourselves updated on pertinent primary literature, and consideration of advanced diagnostic testing such as exome sequencing and other next-generation sequencing strategies are essential in hoping to characterize a specific clinical syndrome that has yet to be described.

For situations in which our recommendations do not yield a diagnosis, we recognize the role for protocols such as genomic or metagenomic sequencing that assess multiple diagnostic possibilities in parallel without an a priori hypothesis.^3,4 The utility of multi-omics testing in diagnostic workups has been detailed by the Undiagnosed Diseases Network (UDN), which has created a systematic approach to describing new syndromes with the aid of metabolomic and genomic profiling.⁵ It is important to note that even with the resources available to the UDN, the diagnosis rate is 35%, emphasizing that in the majority of diagnosis-refractory cases, a diagnosis will not be found. This low diagnosis rate underscores the need for continued inquiry and cataloging of cases and data for further review or synthesis as the body of medical knowledge continues to expand. For these reasons, we have a follow-up system in place, which involves the assigned CMR regularly reviewing the chart and reporting during our weekly meetings. We make phone calls to patients and providers for cases that appear to be lost to follow-up.

We recognize several important limitations to our care model that may represent barriers to establishing, maintaining, and evaluating a similar service at other institutions. For example, there are limitations and benefits of the CMR as point person for managing our consultations. While they are admittedly junior colleagues with limited experience, CMRs tend to be among the best-read and up-to-date clinicians in the hospital by virtue of their recent general-medicine training and identification as a top clinician and leader. Moreover, in their role with the Socrates Project, CMRs have more time to think, talk with patients, and review the medical record than other clinicians, who may be under pressure to see an increasing number of patients while billing at higher levels. Indeed, the Socrates Project CMRs have, on a number of occasions, been the team members who find the piece of data that no one else thought relevant.

Another factor that may limit establishment of a similar team at other institutions is our volunteer-based model. The Socrates Project members volunteer because they love clinical medicine and serve on the team without remuneration for professional effort. With the CMR role as a notable exception, pressure from achieving relative value unit targets, obtaining grant funding, and publishing primary research publications in their field may limit this care model, particularly when shifting from a clinical-only activity to one that also formally investigates the service’s process and outcomes.

Beyond our clinical objective, we hope that the Socrates Project will further the discovery and description of previously unrecognized disease processes. To that end, we are pursuing an institutional review board-approved protocol to perform a rigorous assessment of the Socrates Project’s process and outcomes, including a cataloging of case archetypes and the time to definitive diagnosis if a diagnosis is established. As we continue to collect data, increasing our referral network may also lead to refinement and improvement in diagnostic processes and outcomes. Over time, we expect that the diagnostic resources available to us will evolve. Utilizing collective intelligence has been shown to improve diagnostic accuracy,⁶ and emerging artificial intelligence technologies may improve diagnostic performance as well.^7,8 Most importantly, through this endeavor, we hope to serve less as an oracle and more as a humble Socratic consultant for clinicians working to reduce diagnostic uncertainty for their patients.

Acknowledgments

The authors wish to thank the Northwestern University Chief Medical Residents, 2015-present, for their tireless efforts in support of the Socrates Project.

Internists are experts in general medicine, skilled at mapping the few hundred ways the human body can go awry onto thousands of diagnoses, and managing the uncertainty inherent in that process. Generalists, almost by definition, consult specialists with their specialty-focused questions; but who does one call for a general consultation about diagnosis if a specific diagnosis remains elusive and the pathology does not fit cleanly into the purview of a consultant? Outside of sage advice from colleagues (usually senior), most medical centers lack a consultation service focused on diagnosis. There is no oracle to seek. In this perspective, we describe our institution’s answer to this problem: the creation of a service for difficult diagnosis based on Socratic principles, particularly the role of iterative hypothesis testing in the process of diagnosis.¹

In 2015, Northwestern Medicine began the Socrates Project, a physician-to-physician consultation service that assists doctors working to diagnose conditions that have so far eluded detection. Our service’s goal is to improve patient care by providing an opinion to the referring physician on diagnostic possibilities for a particular case and ideas to reduce—or at least manage—diagnostic uncertainty.

Most patients referred to the Socrates Project have already undergone an extensive evaluation at top medical centers by experienced clinicians. It would be hubris to assume that we will find a definitive diagnosis in every case; indeed, because of the types of cases referred to our group, it is rare that we find a “Eureka!” diagnosis. When a colleague consults our group, we under-promise in hopes of over-delivering. Instead, we convey to referring physicians that we will conduct a thorough case review and explain our thinking in hopes of uncovering an additional diagnostic avenue, even if that avenue does not ultimately lead to a definitive diagnosis. In addition, the Socrates Project often serves as a broker between consulting services that are deadlocked because of differing diagnostic opinions. We also assist with cases in which a functional disorder is suspected, yet the referring physician is hesitant to diagnose a patient with such a disorder out of concern about missing an important (and possibly obscure) diagnosis.

The Socrates Project receives approximately two consult requests per week, usually from general internists but also from specialists in nearly all disciplines. Around 80% of the referrals are for current inpatients. Our service model is similar to a tumor board, which exists as an interdisciplinary group operating in parallel to the clinical services, to provide consensus-based recommendations. As a result, we act as doctors for doctors, formalizing the curbside consultation. Our usual turnaround time is a week but can be faster for urgent cases. Currently, Socrates Project members, including the faculty leader, volunteer their time and effort at no cost, and there are no charges to patients when physicians consult our group. An overview of the Socrates Project’s personnel and process are outlined in the Figure.

Northwestern’s Chief Medical Residents (CMRs) serve as the fellows for the service, and one of them assumes primary responsibility for each new consultation request the service receives. After obtaining the patient’s case history from the referring provider, the CMR then undertakes a thorough review of the electronic health record and any other available records from other institutions. In the inpatient setting, the CMR performs a new history and physical; phone calls or video conferencing permit history taking for outpatients. In contrast with the standard consultant note, we do not redocument the history, physical, and lab and imaging findings but instead construct a detailed problem list that synthesizes relevant findings into a useful working document.

The service’s faculty leader (BDS) then reviews the problem list with the CMRs to help refine the problem list and begin producing a differential diagnosis during a weekly hour-long meeting. As evidence supports team-based diagnostic collaborations,² the problem list and preliminary differential diagnosis then becomes a shareable document that the CMR or team leader presents to ad hoc general internists, specialists, and the other CMRs. The presentation can be in person, by phone, or e-mail. These ad hoc members, approximately 20 in number and spanning from junior attending physicians to senior clinicians, have volunteered to help the Socrates Project by adding their thoughts on differential diagnoses that explain the problem list and how to move forward with further testing. The ad hoc members have self-identified as clinicians with an interest in medical diagnosis—including surgeons, neurologists, psychiatrists, radiologists, and pathologists—and range in expertise from general internists to subspecialists. Finally, we document our problem list, differential diagnosis, and recommendations in the medical record and discuss the case with the referring team. The service limits its scope of clinical recommendation to diagnosis and avoids commenting on management decisions outside of the use of therapies as empiric diagnostic tests. A sample note is provided as an online Appendix.

Despite our process, we are often left without a satisfying diagnosis. We then are then faced with three possibilities: (1) The diagnosis is identifiable, just not by the physicians involved in the case—we did not think of the diagnosis in our deliberations; (2) The diagnosis is a described condition but without an available test—autoimmune limbic encephalitis associated with an unassayable or unknown auto-antibody, or the acuity of a critically ill patient makes diagnostic testing unreliable or not feasible; (3) The diagnosis has not yet been described by medical science—we are seeing a case of HIV infection in 1971.

With the personnel and process outlined above, we hope to provide recommendations that are useful in guiding a diagnostic workup regardless of which of these three scenarios is applicable. Our flexibility with involving the appropriate specialists in the Socrates Project should minimize the number of patients with a knowable diagnosis that is unknown to us. In the second scenario, our recommendations may rest upon the incorporation of a treatment as a diagnostic test. In the limbic encephalitis example above, a trial of steroids with rapid improvement in the patient’s condition may increase diagnostic certainty. The third scenario is the most difficult to identify. Pattern recognition of similarly presenting patients, keeping ourselves updated on pertinent primary literature, and consideration of advanced diagnostic testing such as exome sequencing and other next-generation sequencing strategies are essential in hoping to characterize a specific clinical syndrome that has yet to be described.

For situations in which our recommendations do not yield a diagnosis, we recognize the role for protocols such as genomic or metagenomic sequencing that assess multiple diagnostic possibilities in parallel without an a priori hypothesis.^3,4 The utility of multi-omics testing in diagnostic workups has been detailed by the Undiagnosed Diseases Network (UDN), which has created a systematic approach to describing new syndromes with the aid of metabolomic and genomic profiling.⁵ It is important to note that even with the resources available to the UDN, the diagnosis rate is 35%, emphasizing that in the majority of diagnosis-refractory cases, a diagnosis will not be found. This low diagnosis rate underscores the need for continued inquiry and cataloging of cases and data for further review or synthesis as the body of medical knowledge continues to expand. For these reasons, we have a follow-up system in place, which involves the assigned CMR regularly reviewing the chart and reporting during our weekly meetings. We make phone calls to patients and providers for cases that appear to be lost to follow-up.

We recognize several important limitations to our care model that may represent barriers to establishing, maintaining, and evaluating a similar service at other institutions. For example, there are limitations and benefits of the CMR as point person for managing our consultations. While they are admittedly junior colleagues with limited experience, CMRs tend to be among the best-read and up-to-date clinicians in the hospital by virtue of their recent general-medicine training and identification as a top clinician and leader. Moreover, in their role with the Socrates Project, CMRs have more time to think, talk with patients, and review the medical record than other clinicians, who may be under pressure to see an increasing number of patients while billing at higher levels. Indeed, the Socrates Project CMRs have, on a number of occasions, been the team members who find the piece of data that no one else thought relevant.

Another factor that may limit establishment of a similar team at other institutions is our volunteer-based model. The Socrates Project members volunteer because they love clinical medicine and serve on the team without remuneration for professional effort. With the CMR role as a notable exception, pressure from achieving relative value unit targets, obtaining grant funding, and publishing primary research publications in their field may limit this care model, particularly when shifting from a clinical-only activity to one that also formally investigates the service’s process and outcomes.

Beyond our clinical objective, we hope that the Socrates Project will further the discovery and description of previously unrecognized disease processes. To that end, we are pursuing an institutional review board-approved protocol to perform a rigorous assessment of the Socrates Project’s process and outcomes, including a cataloging of case archetypes and the time to definitive diagnosis if a diagnosis is established. As we continue to collect data, increasing our referral network may also lead to refinement and improvement in diagnostic processes and outcomes. Over time, we expect that the diagnostic resources available to us will evolve. Utilizing collective intelligence has been shown to improve diagnostic accuracy,⁶ and emerging artificial intelligence technologies may improve diagnostic performance as well.^7,8 Most importantly, through this endeavor, we hope to serve less as an oracle and more as a humble Socratic consultant for clinicians working to reduce diagnostic uncertainty for their patients.

Acknowledgments

The authors wish to thank the Northwestern University Chief Medical Residents, 2015-present, for their tireless efforts in support of the Socrates Project.

Internists are experts in general medicine, skilled at mapping the few hundred ways the human body can go awry onto thousands of diagnoses, and managing the uncertainty inherent in that process. Generalists, almost by definition, consult specialists with their specialty-focused questions; but who does one call for a general consultation about diagnosis if a specific diagnosis remains elusive and the pathology does not fit cleanly into the purview of a consultant? Outside of sage advice from colleagues (usually senior), most medical centers lack a consultation service focused on diagnosis. There is no oracle to seek. In this perspective, we describe our institution’s answer to this problem: the creation of a service for difficult diagnosis based on Socratic principles, particularly the role of iterative hypothesis testing in the process of diagnosis.¹

In 2015, Northwestern Medicine began the Socrates Project, a physician-to-physician consultation service that assists doctors working to diagnose conditions that have so far eluded detection. Our service’s goal is to improve patient care by providing an opinion to the referring physician on diagnostic possibilities for a particular case and ideas to reduce—or at least manage—diagnostic uncertainty.

Most patients referred to the Socrates Project have already undergone an extensive evaluation at top medical centers by experienced clinicians. It would be hubris to assume that we will find a definitive diagnosis in every case; indeed, because of the types of cases referred to our group, it is rare that we find a “Eureka!” diagnosis. When a colleague consults our group, we under-promise in hopes of over-delivering. Instead, we convey to referring physicians that we will conduct a thorough case review and explain our thinking in hopes of uncovering an additional diagnostic avenue, even if that avenue does not ultimately lead to a definitive diagnosis. In addition, the Socrates Project often serves as a broker between consulting services that are deadlocked because of differing diagnostic opinions. We also assist with cases in which a functional disorder is suspected, yet the referring physician is hesitant to diagnose a patient with such a disorder out of concern about missing an important (and possibly obscure) diagnosis.

The Socrates Project receives approximately two consult requests per week, usually from general internists but also from specialists in nearly all disciplines. Around 80% of the referrals are for current inpatients. Our service model is similar to a tumor board, which exists as an interdisciplinary group operating in parallel to the clinical services, to provide consensus-based recommendations. As a result, we act as doctors for doctors, formalizing the curbside consultation. Our usual turnaround time is a week but can be faster for urgent cases. Currently, Socrates Project members, including the faculty leader, volunteer their time and effort at no cost, and there are no charges to patients when physicians consult our group. An overview of the Socrates Project’s personnel and process are outlined in the Figure.

Northwestern’s Chief Medical Residents (CMRs) serve as the fellows for the service, and one of them assumes primary responsibility for each new consultation request the service receives. After obtaining the patient’s case history from the referring provider, the CMR then undertakes a thorough review of the electronic health record and any other available records from other institutions. In the inpatient setting, the CMR performs a new history and physical; phone calls or video conferencing permit history taking for outpatients. In contrast with the standard consultant note, we do not redocument the history, physical, and lab and imaging findings but instead construct a detailed problem list that synthesizes relevant findings into a useful working document.

The service’s faculty leader (BDS) then reviews the problem list with the CMRs to help refine the problem list and begin producing a differential diagnosis during a weekly hour-long meeting. As evidence supports team-based diagnostic collaborations,² the problem list and preliminary differential diagnosis then becomes a shareable document that the CMR or team leader presents to ad hoc general internists, specialists, and the other CMRs. The presentation can be in person, by phone, or e-mail. These ad hoc members, approximately 20 in number and spanning from junior attending physicians to senior clinicians, have volunteered to help the Socrates Project by adding their thoughts on differential diagnoses that explain the problem list and how to move forward with further testing. The ad hoc members have self-identified as clinicians with an interest in medical diagnosis—including surgeons, neurologists, psychiatrists, radiologists, and pathologists—and range in expertise from general internists to subspecialists. Finally, we document our problem list, differential diagnosis, and recommendations in the medical record and discuss the case with the referring team. The service limits its scope of clinical recommendation to diagnosis and avoids commenting on management decisions outside of the use of therapies as empiric diagnostic tests. A sample note is provided as an online Appendix.

Despite our process, we are often left without a satisfying diagnosis. We then are then faced with three possibilities: (1) The diagnosis is identifiable, just not by the physicians involved in the case—we did not think of the diagnosis in our deliberations; (2) The diagnosis is a described condition but without an available test—autoimmune limbic encephalitis associated with an unassayable or unknown auto-antibody, or the acuity of a critically ill patient makes diagnostic testing unreliable or not feasible; (3) The diagnosis has not yet been described by medical science—we are seeing a case of HIV infection in 1971.

With the personnel and process outlined above, we hope to provide recommendations that are useful in guiding a diagnostic workup regardless of which of these three scenarios is applicable. Our flexibility with involving the appropriate specialists in the Socrates Project should minimize the number of patients with a knowable diagnosis that is unknown to us. In the second scenario, our recommendations may rest upon the incorporation of a treatment as a diagnostic test. In the limbic encephalitis example above, a trial of steroids with rapid improvement in the patient’s condition may increase diagnostic certainty. The third scenario is the most difficult to identify. Pattern recognition of similarly presenting patients, keeping ourselves updated on pertinent primary literature, and consideration of advanced diagnostic testing such as exome sequencing and other next-generation sequencing strategies are essential in hoping to characterize a specific clinical syndrome that has yet to be described.

For situations in which our recommendations do not yield a diagnosis, we recognize the role for protocols such as genomic or metagenomic sequencing that assess multiple diagnostic possibilities in parallel without an a priori hypothesis.^3,4 The utility of multi-omics testing in diagnostic workups has been detailed by the Undiagnosed Diseases Network (UDN), which has created a systematic approach to describing new syndromes with the aid of metabolomic and genomic profiling.⁵ It is important to note that even with the resources available to the UDN, the diagnosis rate is 35%, emphasizing that in the majority of diagnosis-refractory cases, a diagnosis will not be found. This low diagnosis rate underscores the need for continued inquiry and cataloging of cases and data for further review or synthesis as the body of medical knowledge continues to expand. For these reasons, we have a follow-up system in place, which involves the assigned CMR regularly reviewing the chart and reporting during our weekly meetings. We make phone calls to patients and providers for cases that appear to be lost to follow-up.

We recognize several important limitations to our care model that may represent barriers to establishing, maintaining, and evaluating a similar service at other institutions. For example, there are limitations and benefits of the CMR as point person for managing our consultations. While they are admittedly junior colleagues with limited experience, CMRs tend to be among the best-read and up-to-date clinicians in the hospital by virtue of their recent general-medicine training and identification as a top clinician and leader. Moreover, in their role with the Socrates Project, CMRs have more time to think, talk with patients, and review the medical record than other clinicians, who may be under pressure to see an increasing number of patients while billing at higher levels. Indeed, the Socrates Project CMRs have, on a number of occasions, been the team members who find the piece of data that no one else thought relevant.

Another factor that may limit establishment of a similar team at other institutions is our volunteer-based model. The Socrates Project members volunteer because they love clinical medicine and serve on the team without remuneration for professional effort. With the CMR role as a notable exception, pressure from achieving relative value unit targets, obtaining grant funding, and publishing primary research publications in their field may limit this care model, particularly when shifting from a clinical-only activity to one that also formally investigates the service’s process and outcomes.

Beyond our clinical objective, we hope that the Socrates Project will further the discovery and description of previously unrecognized disease processes. To that end, we are pursuing an institutional review board-approved protocol to perform a rigorous assessment of the Socrates Project’s process and outcomes, including a cataloging of case archetypes and the time to definitive diagnosis if a diagnosis is established. As we continue to collect data, increasing our referral network may also lead to refinement and improvement in diagnostic processes and outcomes. Over time, we expect that the diagnostic resources available to us will evolve. Utilizing collective intelligence has been shown to improve diagnostic accuracy,⁶ and emerging artificial intelligence technologies may improve diagnostic performance as well.^7,8 Most importantly, through this endeavor, we hope to serve less as an oracle and more as a humble Socratic consultant for clinicians working to reduce diagnostic uncertainty for their patients.

Acknowledgments

The authors wish to thank the Northwestern University Chief Medical Residents, 2015-present, for their tireless efforts in support of the Socrates Project.

Perioperative venous thromboembolism (VTE) is a major contributor to the morbidity and mortality of hospitalized patients. The historical incidence of postoperative VTE varied between 15%-80% depending on the type of procedure and monitoring strategies. Higher incidences of VTE occurred with major surgery (15%-40%), knee or hip arthroplasty (40%-60%), and trauma (60%-80%).¹ The use of VTE prophylaxis with subcutaneous heparin reduced DVTs by 70% and PEs by 50%.² A recent study from Olmstead County showed improved adherence to inpatient VTE prophylaxis from 2005 to 2010 but no difference in VTE incidence. However, 52% of VTE events were associated with hospitalization.³ As such, VTE continues to be a healthcare-associated adverse event, and surgery remains a significant risk factor for thrombosis.⁴

The Agency for Healthcare Research and Quality (AHRQ) released Patient Safety Indicators (PSI) in 2003 to provide a means of screening for adverse events.⁵ Over time, PSIs have been adopted as a measure of hospital performance and are utilized in several pay-for-performance programs. PSI-90 is a composite measure of several other PSIs and is a core metric in the Centers for Medicare and Medicaid Services (CMS) Hospital-Acquired Condition Reduction Program and the Hospital Value-Based Purchasing Program which impacts up to 2% of a hospital’s Medicare payments.^6,7 One component of PSI-90 is PSI-12, which captures perioperative VTE. PSI-12 events are identified using software that screens medical records based on International Classification of Diseases (ICD)-9/10 codes for thrombosis and procedure codes at time of discharge.⁸

Over the last several years, there has been concern regarding the validity of including PSI-12 in pay-for-performance metrics. Common areas for concern include PSI-12’s accuracy in detecting true postoperative VTE⁹ in addition to surveillance bias.^10,11 However, some note that PSI-12 is useful when applied with its original intent: as a screening tool for hospitals to identify specific areas to implement improvements.^9,12The aim of our study was to review all PSI-12 events at our institution to evaluate the accuracy of PSI-12 and identify areas for improvement to prevent VTE events in surgical patients. While several other studies have looked at the positive predictive values, accuracy, or surveillance bias of PSI-12, to the best of our knowledge, few, if any, previous studies have reported PSI-12 events in relation to their timing, type of prophylaxis used, and mitigating factors to identify areas for quality improvement.

PSI-12 events were identified between June 2015 and June 2017 using AHRQ software (version 5). Cases were also identified through Vizient and reviewed to ensure congruence between the methods. Patients’ electronic medical records were reviewed for patient demographics, type of VTE event, platelet count at VTE diagnosis, procedure type, and both the timing and type of VTE prophylaxis. Summary statistics were calculated.

We considered perioperative VTE pharmacologic prophylaxis appropriate if started within 24 hours of a low bleeding risk procedure or 72 hours of a high-bleeding risk procedure.¹³ Mechanical prophylaxis was considered appropriate if pharmacologic prophylaxis was not used because of procedure risk, thrombocytopenia, or active bleeding. The medication administration record was reviewed to determine if prophylaxis was ordered, given, and/or refused.

During the two-year period, 18,084 surgeries were performed, and 161 cases of VTE events were identified. A detailed chart review and correction of documentation led to the exclusion of seven cases (4%) because the VTE event occurred prior to admission (n = 5) or were incidental findings that did not meet the Uniform Hospital Discharge Data Set definition for reporting (n = 2). In total, 154 (0.9% of all surgeries) cases were considered PSI-12 events. Pulmonary embolism (PE) occurred in most cases (n = 97, 62.9%), followed by deep vein thrombosis (DVT) (n = 37, 24.0%). Twenty cases (12.9%) experienced concurrent PE and DVT. Within the PE group, 16 cases (14%) were subsegmental PE only. Eight patients (14% of DVT cases) had only a distal DVT. The mean age of patients was 56 years (+/− 16 years), and the majority (59%) were male. The clinical specialties with the most events included neurosurgery (21%), orthopedics (14%), general surgery (13%), and trauma (11%). Fourteen patients (9%) died during the hospitalization, and of these, six (43%) had either sudden death or death attributed to PE (Table 1).

Cases were also reviewed for the type of VTE prophylactic strategy administered at the time of the event. The top three prophylactic strategies were subcutaneous unfractionated heparin (61%), mechanical prophylaxis only (51%), and enoxaparin (31%). Nine cases of VTE occurred during therapeutic anticoagulation (6%; Table 1).

Utilizing AHRQ version 5 software for PSI-12, our institution identified 154 cases of perioperative VTE. Most of these cases were a pulmonary embolism, occurred within a week of admission, and were associated with surgical specialties that portend a higher risk of VTE. Very few of these cases were deficient in guideline-directed VTE prophylaxis, and several cases had associated factors such as trauma and thrombocytopenia that may have appropriately influenced the decision to use mechanical only prophylaxis. Sixteen percent of pharmacologic VTE prophylactic doses were refused or held for a procedure, a known but rarely quantified influence on rates of pharmacologic prophylaxis. The use of patient-level data and adjudication of the clinical decision that affected the administration of VTE prophylaxis is a major strength of this work. In all, our data raise several questions about the accuracy of PSI-12 in identifying preventable postoperative VTE, especially as it is utilized as a marker for pay-for-performance measures in addition to identifying further areas for research and improvement.

Our results align with previous studies that suggest that PSI-12 is an inaccurate measure of performance quality. A study by Bilimoria et al. in 2013 noted that surveillance biases associated with PSI-12, showing that hospitals with higher compliance to appropriate VTE prophylaxis paradoxically had worse outcomes.¹⁰ Furthermore, Blay et al. recently published a study evaluating PSI-90 scores for hospitals with and without the VTE measure included. Their results indicated that larger hospitals (teaching hospitals, level I trauma centers, etc) caring for sicker patients were noted to improve by 8%-25% when the VTE measure was removed.¹¹ Similarly, our data indicate that the PSI-12 may not be an accurate measure of quality performance, as only 6% of cases were noted to be deficient in appropriate guideline-directed prophylaxis. Even when accounting for the refusal of doses of pharmacologic prophylaxis, this figure only increased to 11%.

Procedures included in the current PSI-12 algorithm also vary in the risk they pose to developing VTE. For example, the current version of PSI-12 includes surgical Medicare Severity Diagnosis Related Groups (MS-DRGs) for procedures with a high risk of VTE such as orthopedic, abdominal, or thoracic procedures.^14,15, However, it is worth noting that procedures such as tracheostomy and ocular surgery are also included.¹⁴ The variation in risk for development of VTE is reflected in the Caprini score, a perioperative risk stratification tool. These latter procedures only contribute one point, whereas trauma would add five points each to the total score, and make the patient a high VTE risk from the procedure alone.¹⁶ With regard to PSI-12, in theory, scores could vary significantly between centers even if the quality of care is the same, based on the volume and risk of procedures performed.

Perioperative venous thromboembolism (VTE) is a major contributor to the morbidity and mortality of hospitalized patients. The historical incidence of postoperative VTE varied between 15%-80% depending on the type of procedure and monitoring strategies. Higher incidences of VTE occurred with major surgery (15%-40%), knee or hip arthroplasty (40%-60%), and trauma (60%-80%).¹ The use of VTE prophylaxis with subcutaneous heparin reduced DVTs by 70% and PEs by 50%.² A recent study from Olmstead County showed improved adherence to inpatient VTE prophylaxis from 2005 to 2010 but no difference in VTE incidence. However, 52% of VTE events were associated with hospitalization.³ As such, VTE continues to be a healthcare-associated adverse event, and surgery remains a significant risk factor for thrombosis.⁴

The Agency for Healthcare Research and Quality (AHRQ) released Patient Safety Indicators (PSI) in 2003 to provide a means of screening for adverse events.⁵ Over time, PSIs have been adopted as a measure of hospital performance and are utilized in several pay-for-performance programs. PSI-90 is a composite measure of several other PSIs and is a core metric in the Centers for Medicare and Medicaid Services (CMS) Hospital-Acquired Condition Reduction Program and the Hospital Value-Based Purchasing Program which impacts up to 2% of a hospital’s Medicare payments.^6,7 One component of PSI-90 is PSI-12, which captures perioperative VTE. PSI-12 events are identified using software that screens medical records based on International Classification of Diseases (ICD)-9/10 codes for thrombosis and procedure codes at time of discharge.⁸

Over the last several years, there has been concern regarding the validity of including PSI-12 in pay-for-performance metrics. Common areas for concern include PSI-12’s accuracy in detecting true postoperative VTE⁹ in addition to surveillance bias.^10,11 However, some note that PSI-12 is useful when applied with its original intent: as a screening tool for hospitals to identify specific areas to implement improvements.^9,12The aim of our study was to review all PSI-12 events at our institution to evaluate the accuracy of PSI-12 and identify areas for improvement to prevent VTE events in surgical patients. While several other studies have looked at the positive predictive values, accuracy, or surveillance bias of PSI-12, to the best of our knowledge, few, if any, previous studies have reported PSI-12 events in relation to their timing, type of prophylaxis used, and mitigating factors to identify areas for quality improvement.

PSI-12 events were identified between June 2015 and June 2017 using AHRQ software (version 5). Cases were also identified through Vizient and reviewed to ensure congruence between the methods. Patients’ electronic medical records were reviewed for patient demographics, type of VTE event, platelet count at VTE diagnosis, procedure type, and both the timing and type of VTE prophylaxis. Summary statistics were calculated.

We considered perioperative VTE pharmacologic prophylaxis appropriate if started within 24 hours of a low bleeding risk procedure or 72 hours of a high-bleeding risk procedure.¹³ Mechanical prophylaxis was considered appropriate if pharmacologic prophylaxis was not used because of procedure risk, thrombocytopenia, or active bleeding. The medication administration record was reviewed to determine if prophylaxis was ordered, given, and/or refused.

During the two-year period, 18,084 surgeries were performed, and 161 cases of VTE events were identified. A detailed chart review and correction of documentation led to the exclusion of seven cases (4%) because the VTE event occurred prior to admission (n = 5) or were incidental findings that did not meet the Uniform Hospital Discharge Data Set definition for reporting (n = 2). In total, 154 (0.9% of all surgeries) cases were considered PSI-12 events. Pulmonary embolism (PE) occurred in most cases (n = 97, 62.9%), followed by deep vein thrombosis (DVT) (n = 37, 24.0%). Twenty cases (12.9%) experienced concurrent PE and DVT. Within the PE group, 16 cases (14%) were subsegmental PE only. Eight patients (14% of DVT cases) had only a distal DVT. The mean age of patients was 56 years (+/− 16 years), and the majority (59%) were male. The clinical specialties with the most events included neurosurgery (21%), orthopedics (14%), general surgery (13%), and trauma (11%). Fourteen patients (9%) died during the hospitalization, and of these, six (43%) had either sudden death or death attributed to PE (Table 1).

Cases were also reviewed for the type of VTE prophylactic strategy administered at the time of the event. The top three prophylactic strategies were subcutaneous unfractionated heparin (61%), mechanical prophylaxis only (51%), and enoxaparin (31%). Nine cases of VTE occurred during therapeutic anticoagulation (6%; Table 1).

Utilizing AHRQ version 5 software for PSI-12, our institution identified 154 cases of perioperative VTE. Most of these cases were a pulmonary embolism, occurred within a week of admission, and were associated with surgical specialties that portend a higher risk of VTE. Very few of these cases were deficient in guideline-directed VTE prophylaxis, and several cases had associated factors such as trauma and thrombocytopenia that may have appropriately influenced the decision to use mechanical only prophylaxis. Sixteen percent of pharmacologic VTE prophylactic doses were refused or held for a procedure, a known but rarely quantified influence on rates of pharmacologic prophylaxis. The use of patient-level data and adjudication of the clinical decision that affected the administration of VTE prophylaxis is a major strength of this work. In all, our data raise several questions about the accuracy of PSI-12 in identifying preventable postoperative VTE, especially as it is utilized as a marker for pay-for-performance measures in addition to identifying further areas for research and improvement.

Our results align with previous studies that suggest that PSI-12 is an inaccurate measure of performance quality. A study by Bilimoria et al. in 2013 noted that surveillance biases associated with PSI-12, showing that hospitals with higher compliance to appropriate VTE prophylaxis paradoxically had worse outcomes.¹⁰ Furthermore, Blay et al. recently published a study evaluating PSI-90 scores for hospitals with and without the VTE measure included. Their results indicated that larger hospitals (teaching hospitals, level I trauma centers, etc) caring for sicker patients were noted to improve by 8%-25% when the VTE measure was removed.¹¹ Similarly, our data indicate that the PSI-12 may not be an accurate measure of quality performance, as only 6% of cases were noted to be deficient in appropriate guideline-directed prophylaxis. Even when accounting for the refusal of doses of pharmacologic prophylaxis, this figure only increased to 11%.

Procedures included in the current PSI-12 algorithm also vary in the risk they pose to developing VTE. For example, the current version of PSI-12 includes surgical Medicare Severity Diagnosis Related Groups (MS-DRGs) for procedures with a high risk of VTE such as orthopedic, abdominal, or thoracic procedures.^14,15, However, it is worth noting that procedures such as tracheostomy and ocular surgery are also included.¹⁴ The variation in risk for development of VTE is reflected in the Caprini score, a perioperative risk stratification tool. These latter procedures only contribute one point, whereas trauma would add five points each to the total score, and make the patient a high VTE risk from the procedure alone.¹⁶ With regard to PSI-12, in theory, scores could vary significantly between centers even if the quality of care is the same, based on the volume and risk of procedures performed.

Perioperative venous thromboembolism (VTE) is a major contributor to the morbidity and mortality of hospitalized patients. The historical incidence of postoperative VTE varied between 15%-80% depending on the type of procedure and monitoring strategies. Higher incidences of VTE occurred with major surgery (15%-40%), knee or hip arthroplasty (40%-60%), and trauma (60%-80%).¹ The use of VTE prophylaxis with subcutaneous heparin reduced DVTs by 70% and PEs by 50%.² A recent study from Olmstead County showed improved adherence to inpatient VTE prophylaxis from 2005 to 2010 but no difference in VTE incidence. However, 52% of VTE events were associated with hospitalization.³ As such, VTE continues to be a healthcare-associated adverse event, and surgery remains a significant risk factor for thrombosis.⁴

The Agency for Healthcare Research and Quality (AHRQ) released Patient Safety Indicators (PSI) in 2003 to provide a means of screening for adverse events.⁵ Over time, PSIs have been adopted as a measure of hospital performance and are utilized in several pay-for-performance programs. PSI-90 is a composite measure of several other PSIs and is a core metric in the Centers for Medicare and Medicaid Services (CMS) Hospital-Acquired Condition Reduction Program and the Hospital Value-Based Purchasing Program which impacts up to 2% of a hospital’s Medicare payments.^6,7 One component of PSI-90 is PSI-12, which captures perioperative VTE. PSI-12 events are identified using software that screens medical records based on International Classification of Diseases (ICD)-9/10 codes for thrombosis and procedure codes at time of discharge.⁸

Over the last several years, there has been concern regarding the validity of including PSI-12 in pay-for-performance metrics. Common areas for concern include PSI-12’s accuracy in detecting true postoperative VTE⁹ in addition to surveillance bias.^10,11 However, some note that PSI-12 is useful when applied with its original intent: as a screening tool for hospitals to identify specific areas to implement improvements.^9,12The aim of our study was to review all PSI-12 events at our institution to evaluate the accuracy of PSI-12 and identify areas for improvement to prevent VTE events in surgical patients. While several other studies have looked at the positive predictive values, accuracy, or surveillance bias of PSI-12, to the best of our knowledge, few, if any, previous studies have reported PSI-12 events in relation to their timing, type of prophylaxis used, and mitigating factors to identify areas for quality improvement.

PSI-12 events were identified between June 2015 and June 2017 using AHRQ software (version 5). Cases were also identified through Vizient and reviewed to ensure congruence between the methods. Patients’ electronic medical records were reviewed for patient demographics, type of VTE event, platelet count at VTE diagnosis, procedure type, and both the timing and type of VTE prophylaxis. Summary statistics were calculated.

We considered perioperative VTE pharmacologic prophylaxis appropriate if started within 24 hours of a low bleeding risk procedure or 72 hours of a high-bleeding risk procedure.¹³ Mechanical prophylaxis was considered appropriate if pharmacologic prophylaxis was not used because of procedure risk, thrombocytopenia, or active bleeding. The medication administration record was reviewed to determine if prophylaxis was ordered, given, and/or refused.

During the two-year period, 18,084 surgeries were performed, and 161 cases of VTE events were identified. A detailed chart review and correction of documentation led to the exclusion of seven cases (4%) because the VTE event occurred prior to admission (n = 5) or were incidental findings that did not meet the Uniform Hospital Discharge Data Set definition for reporting (n = 2). In total, 154 (0.9% of all surgeries) cases were considered PSI-12 events. Pulmonary embolism (PE) occurred in most cases (n = 97, 62.9%), followed by deep vein thrombosis (DVT) (n = 37, 24.0%). Twenty cases (12.9%) experienced concurrent PE and DVT. Within the PE group, 16 cases (14%) were subsegmental PE only. Eight patients (14% of DVT cases) had only a distal DVT. The mean age of patients was 56 years (+/− 16 years), and the majority (59%) were male. The clinical specialties with the most events included neurosurgery (21%), orthopedics (14%), general surgery (13%), and trauma (11%). Fourteen patients (9%) died during the hospitalization, and of these, six (43%) had either sudden death or death attributed to PE (Table 1).

Cases were also reviewed for the type of VTE prophylactic strategy administered at the time of the event. The top three prophylactic strategies were subcutaneous unfractionated heparin (61%), mechanical prophylaxis only (51%), and enoxaparin (31%). Nine cases of VTE occurred during therapeutic anticoagulation (6%; Table 1).

Utilizing AHRQ version 5 software for PSI-12, our institution identified 154 cases of perioperative VTE. Most of these cases were a pulmonary embolism, occurred within a week of admission, and were associated with surgical specialties that portend a higher risk of VTE. Very few of these cases were deficient in guideline-directed VTE prophylaxis, and several cases had associated factors such as trauma and thrombocytopenia that may have appropriately influenced the decision to use mechanical only prophylaxis. Sixteen percent of pharmacologic VTE prophylactic doses were refused or held for a procedure, a known but rarely quantified influence on rates of pharmacologic prophylaxis. The use of patient-level data and adjudication of the clinical decision that affected the administration of VTE prophylaxis is a major strength of this work. In all, our data raise several questions about the accuracy of PSI-12 in identifying preventable postoperative VTE, especially as it is utilized as a marker for pay-for-performance measures in addition to identifying further areas for research and improvement.

Our results align with previous studies that suggest that PSI-12 is an inaccurate measure of performance quality. A study by Bilimoria et al. in 2013 noted that surveillance biases associated with PSI-12, showing that hospitals with higher compliance to appropriate VTE prophylaxis paradoxically had worse outcomes.¹⁰ Furthermore, Blay et al. recently published a study evaluating PSI-90 scores for hospitals with and without the VTE measure included. Their results indicated that larger hospitals (teaching hospitals, level I trauma centers, etc) caring for sicker patients were noted to improve by 8%-25% when the VTE measure was removed.¹¹ Similarly, our data indicate that the PSI-12 may not be an accurate measure of quality performance, as only 6% of cases were noted to be deficient in appropriate guideline-directed prophylaxis. Even when accounting for the refusal of doses of pharmacologic prophylaxis, this figure only increased to 11%.

Procedures included in the current PSI-12 algorithm also vary in the risk they pose to developing VTE. For example, the current version of PSI-12 includes surgical Medicare Severity Diagnosis Related Groups (MS-DRGs) for procedures with a high risk of VTE such as orthopedic, abdominal, or thoracic procedures.^14,15, However, it is worth noting that procedures such as tracheostomy and ocular surgery are also included.¹⁴ The variation in risk for development of VTE is reflected in the Caprini score, a perioperative risk stratification tool. These latter procedures only contribute one point, whereas trauma would add five points each to the total score, and make the patient a high VTE risk from the procedure alone.¹⁶ With regard to PSI-12, in theory, scores could vary significantly between centers even if the quality of care is the same, based on the volume and risk of procedures performed.

Cervical lymphadenitis is a common superficial neck infection in childhood. While most children with cervical lymphadenitis recover with antibiotic therapy, a subset can develop an abscess that may require surgical drainage. Radiologic imaging, most commonly ultrasound or computed tomography (CT), is often performed to identify such an abscess.^1-3 However, no national standards exist to guide clinician decision making around imaging in this population. In the absence of evidence-based guidelines, variability in frequency, timing, and modality of imaging likely exists in children hospitalized with cervical lymphadenitis.

As demonstrated for several other common pediatric conditions,^4,5 variability in imaging practices may contribute to overutilization of resources in children with cervical lymphadenitis. In particular, routinely conducting imaging on presentation may constitute overuse, as children with cervical lymphadenitis who present with less than 72 hours of neck swelling rarely undergo surgical drainage within the first 24 hours of hospitalization.^1,6,7 Imaging performed on presentation is often repeated later during hospitalization, particularly if the patient has not improved with antibiotic therapy. The net result may be unnecessary, redundant radiologic studies. Furthermore, serious complications such as bacteremia, extension of infection into the retropharyngeal space, or involvement of the airway or vasculature rarely occur in children with cervical lymphadenitis.^6,8 In this context, deferring initial imaging in this population is unlikely to lead to adverse outcomes and may reduce radiation exposure.

The overall objectives of this study are to describe hospital-level variation in imaging practices for pediatric cervical lymphadenitis and to examine the association between early imaging and outcomes in this population.

Study Design and Data Source

We conducted a multicenter, cross-sectional study using the Pediatric Health Information Systems (PHIS) database, which contains administrative and billing data from 49 geographically diverse children’s hospitals across the United States (US) affiliated with the Children’s Hospital Association (Lenexa, Kansas). PHIS includes data on patient demographics, discharge diagnoses, and procedures using the International Classification of Diseases, 9th (ICD-9) and 10th Revision (ICD-10) diagnosis codes, as well as daily billed resource utilization for laboratory tests, imaging studies, and medications. Encrypted medical record numbers permit longitudinal identification of children across multiple visits to the same hospital. Use of de-identified PHIS data was deemed to be nonhuman subjects research; our approach to validation of ICD codes using local electronic medical record review was reviewed and approved by the Cincinnati Children’s Hospital Medical Center Institutional Review Board.

Study Population

Our study team developed an algorithm to identify children with cervical lymphadenitis and minimize misclassification using PHIS (Appendix A). All children with lymphadenitis-related ICD-9 and ICD-10 discharge diagnosis codes were eligible for inclusion. Codes were validated at a single center via electronic medical record review; clinician-documented discharge diagnosis of cervical lymphadenitis or the presence of fever and unilateral or asymmetrical neck swelling with overlying skin changes was used as the reference standard. We then excluded children who did not receive antibiotics, children who received radiologic imaging not involving the head or neck (which suggested noncervical lymphadenitis or other illness), and children who had discharge diagnosis codes for other specified conditions that are sometimes associated with enlarged cervical lymph nodes but warrant different evaluation or treatment (eg, Kawasaki disease, retropharyngeal abscess, and dental abscess; Appendix A). Our final algorithm yielded a positive predictive value of 87.5% (95% CI: 79.2%-93.4%) when ICD-9 codes were considered, and 95.1% (95% CI: 88.9%-98.4%) when ICD-10 codes were considered (Appendix A).

This algorithm was subsequently applied to the PHIS database. Children ages two months to 18 years hospitalized at participating PHIS institutions between July 2013 and December 2017 with a diagnosis of cervical lymphadenitis as per the algorithm (Appendix A) were eligible for inclusion. For children with multiple eligible admissions during the study period, we only included the first hospitalization. Children with complex chronic condition diagnosis codes⁹ were excluded as their clinical complexity could influence decisions around timing and modality of diagnostic imaging. In addition, we excluded children who did not have an emergency department (ED) visit associated with their hospitalization. This step was intended to exclude children who were transferred from another institution, as imaging performed at outside institutions prior to transfer is not available in PHIS. To avoid overinflating hospital-level variation in the setting of a small sample size, we also excluded all children admitted to the five hospitals with fewer than 50 cases of cervical lymphadenitis during the study period. Our final cohort consisted of 44 PHIS hospitals.

Measures of Interest

To examine hospital-level variation in imaging practices, we measured the proportion of children at each hospital who underwent any neck imaging study, CT or ultrasound imaging, early imaging, and multiple imaging studies within a single hospitalization. Neck imaging was defined as the presence of a billing code for ultrasound, CT, or magnetic resonance imaging (MRI) study of the neck (Appendix B). Early imaging was defined as neck imaging conducted on day 0 of hospitalization (ie, calendar day of admission and ending at midnight). Multiple imaging studies were defined as the receipt of more than one imaging study, regardless of timing or modality. We also measured the proportion of children by hospital who received surgical drainage, defined by the presence of procedure codes for incision and drainage of abscess of the neck (Appendix B).

In examining patient-level association between early imaging and clinical outcomes, our primary outcome of interest was the receipt of multiple imaging studies. Secondary outcomes included rates of surgical drainage, length of stay (in hospital days), and rates of lymphadenitis-related hospital readmission within 30 days of index discharge.

Covariates

Baseline demographic characteristics included age, gender, race/ethnicity, and insurance type. We measured ED visits associated with lymphadenitis-related diagnosis codes in the 30 days prior to admission as a proxy measure for illness duration prior to presentation. To approximate illness severity, we included the following covariates: rates of intensive care unit admission on presentation, rates of receipt of intravenous (IV) analgesia (Appendix B) on hospital days prior to surgical drainage, and rates of receipt of broad-spectrum antibiotics on day 0 or 1 of hospitalization. Broad-spectrum antibiotics (Appendix B) were defined by an independent three-person review of available antibiotic codes (SD, SSS, and JT); differences were resolved by group consensus.

Analysis

Categorical variables were described using frequencies and percentages, while continuous data were described using median and interquartile range. We described hospital-level variation in imaging practices by calculating and comparing the proportion of children at each hospital who underwent any neck imaging study, CT imaging, ultrasound imaging, early imaging, multiple imaging studies, and surgical drainage.

Patient-level demographics and clinical characteristics were compared across groups using chi-square test. To examine the association between early imaging and outcomes, we used generalized linear or logistic mixed effects models to control for patient demographic characteristics and clinical markers of illness duration and severity, with a random effect for hospital to account for clustering. Patient demographics in the model defined a priori included age, race/ethnicity, and insurance type; clinical characteristics included prior ED visit for lymphadenitis, initial intensive care unit (ICU) admission, use of IV analgesia, and use of broad-spectrum antibiotics on day 0 or 1 of hospitalization. To assess the potential for misclassification related to the availability of calendar day but not time of imaging in PHIS, we conducted a secondary analysis to examine the patient-level association between early imaging and outcomes using an alternative definition for early imaging (defined as imaging conducted on day 0 or day 1 of hospitalization).

We identified 19,785 PHIS hospitalizations with lymphadenitis-related discharge diagnosis codes between July 1, 2013 and December 31, 2017. Applying our algorithm and exclusion criteria, we assembled a cohort of 10,014 children hospitalized with cervical lymphadenitis (Figure 1). Two-thirds of the children in our cohort were <4 years old, 42% were non-Hispanic white, and 63% had a government payor (Table 1). Neck imaging (ultrasound, CT, or MRI) was conducted in 8,103 (81%) children. CT imaging was performed in 4,097 (41%) of children, and early imaging was conducted in 6,111 (61%) of children with cervical lymphadenitis.

We noted hospital-level variation in rates of any neck imaging (median: 82.1%, interquartile range [IQR]: 77.7%-85.5%, full range: 68.7%-93.1%), CT imaging (median: 42.3%, IQR: 26.7%-55.2%, full range: 12.0%-81.5%), early imaging (median: 64.4%, IQR: 59.8%-68.4%, full range: 13.8%-76.9%), and multiple imaging studies (median: 23.7%, IQR: 18.6%-28.9%, full range: 1.2%-40.7%; Figure 2). Rates of surgical drainage also varied by hospital (median: 35.1%, IQR: 31.3%-42.0%, full range: 17.1%-54.5%).

In this large multicenter study of children with cervical lymphadenitis, we found variation in imaging practices across 44 US children’s hospitals. Children with cervical lymphadenitis who underwent early imaging were more likely to receive multiple imaging studies during a single hospitalization than those who did not receive early imaging. At the patient level, early imaging was also associated with higher rates of surgical drainage, more frequent 30-day readmission, and longer lengths of stay.

To our knowledge, imaging practices in the population of children hospitalized with cervical lymphadenitis have not been previously characterized in the US; one study from Atlanta, Georgia, describes imaging practices in all children evaluated in the ED.¹ Single-center studies of children hospitalized with cervical lymphadenitis have been previously conducted in Canada⁶ and New Zealand,⁸ in which 42%-51% of children received imaging. In our study, most (81%) children hospitalized with lymphadenitis received some form of imaging, with 61% of all children receiving early imaging. Furthermore, 41% received CT imaging, as compared with 8%-10% of children in the aforementioned studies from Canada and New Zealand.^6,8 This finding is consistent with a pattern of imaging overuse in the US, which has amongst the highest utilization rates globally for advanced imaging such as CT and MRI.^10,11 Identifying opportunities to safely reduce routine imaging, particularly CT imaging, in this population could decrease unnecessary radiation exposure without compromising outcomes.

We also noted variability in imaging practices across PHIS hospitals. Some of this variability may be partially explained by differences in the patient population or illness severity across hospitals. However, given the absence of evidence-based best practices for children with cervical lymphadenitis, clinicians may rely on anecdotal experience or local practice culture to guide their decision making,¹² leading to variability in frequency, timing, and modality of imaging.

At the patient level, we found that children who received early imaging were more likely to receive multiple imaging studies. This finding supports our hypothesis that clinicians often order a second imaging study when the initial imaging study does not clearly demonstrate an abscess, and the child subsequently fails to demonstrate clear improvement after 24-48 hours of antibiotics.

Furthermore, early imaging was associated with overall increased utilization in our cohort, including increased likelihood of surgical drainage, 30-day readmission for lymphadenitis, as well as longer lengths of stay. Confounding may be one explanation for this finding. For instance, clinicians may pursue early imaging in children who present with longer duration of symptoms or more severe illness on presentation, as these factors may be associated with abscess formation.^1,6,7 These clinical covariates are not available in PHIS. Thus, we used prior ED visits for lymphadenitis to approximate illness duration, and initial admission to ICU, receipt of IV analgesia, and receipt of broad-spectrum antibiotics to approximate illness severity in an attempt to mitigate confounding. However, our proxy measures may not appropriately estimate illness duration and severity. For instance, children who had urgent care or outpatient visits for lymphadenitis would not be captured using the proxy of prior ED visit for lymphadenitis. Similarly, use of broad-spectrum antibiotics and IV analgesia may be influenced by provider or institutional preference rather than illness severity. Thus, residual confounding may exist despite adjusting for these measures.

On the other hand, it is also possible that a proportion of children with a small fluid collection on imaging may have improved with antibiotics alone. There is a growing body of evidence in children with other head and neck infections (eg, retropharyngeal abscess and orbital cellulitis with periosteal abscess)^13-15 that suggests that children with small abscesses often improve with antibiotic therapy alone. In children with cervical lymphadenitis who have small or developing abscesses identified via routine imaging on presentation, clinicians may be driven to pursue a surgical intervention with uncertain benefit. Deferring routine imaging in this population may provide an opportunity to improve the value of care in children with lymphadenitis without adversely affecting outcomes.

Upon closer examination of readmissions, children who received early imaging during index hospitalization were more likely to have a 30-day readmission when only evaluating the subset of patients who did not receive surgical drainage during the index admission. This suggests that readmissions are less likely attributable to surgical complications and more likely a reflection of the natural history of lymphadenitis in which a subset of patients eventually develop an abscess. Further supporting this, 61% of children who had a 30-day readmission for lymphadenitis underwent surgical drainage during readmission. Given that lymphadenitis is a slow-brewing infection in which serious complications are rare, patients who demonstrate gradual clinical improvement do not need to remain hospitalized and serially imaged to identify a possible abscess. Outpatient expectant management and readmission as needed for drainage may be an acceptable approach.

This study has several limitations given our use of an administrative database. Children with lymphadenitis may have been misclassified as these patients were identified using discharge diagnosis codes. To mitigate this potential misclassification, we conducted a structured validation process and found that the included codes had high positive predictive values (Appendix A). This validation process was conducted at a single hospital, and coding may vary across hospitals. To approximate sensitivity, we also sampled children without our included codes but with neck imaging and antibiotic use, and found that rates of cervical lymphadenitis were very low among children without our included diagnosis codes.

Furthermore, we were unable to measure the exact time of imaging study in PHIS; we used imaging conducted on hospital day 0 as a proxy measure for imaging conducted within the first 24 hours of presentation. With this definition, some children who had early imaging were likely misclassified as not having received early imaging. For example, a patient who arrived in the ED at 9 pm on day 0 of admission and had a neck ultrasound performed at 1 am would be classified as having had an imaging study on day 1 of hospitalization even though the imaging study was conducted within 4 hours of presentation. Using an alternative definition of early imaging as imaging conducted on hospital day 0 and day 1, we found a much higher adjusted OR for multiple imaging studies, with similar associations for secondary outcomes. As such, our definition of early imaging as day 0 likely biases the results toward the null; the true increase in likelihood of multiple imaging for those who receive early imaging is probably greater than our conservative estimation.

Additionally, there may be a subset of children who underwent imaging prior to presentation at the PHIS hospital ED for further workup and admission. Imaging conducted outside a PHIS hospital was not captured in this database. Similarly, children who had a readmission at a different hospital than their index admission would not be captured using PHIS. Finally, PHIS captures data from children’s hospitals; practices at these hospitals may not be generalizable to practices in the community hospital setting.

In conclusion, we found that imaging practices in children hospitalized with cervical lymphadenitis were widely variable across hospitals. Children receiving early imaging had more resource utilization and intervention when compared with children who did not receive early imaging. Our findings may represent a cascade effect, in which routinely conducted early imaging prompts clinicians to pursue more testing and interventions in this population. Future studies should obtain more detailed patient level covariates to further characterize clinical factors that may impact decisions around imaging and clinical outcomes for children with cervical lymphadenitis.

Acknowledgments

The authors would like to acknowledge the following investigators for their contributions to data interpretation and review of the final manuscript: Angela Choe MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Margaret Rush MD, Children’s National Medical Center, Washington, DC; Ryosuke Takei MD, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania; Wallis Molchen DO, Texas Children’s Hospital, Houston, Texas; Stephanie Royer Moss MD, Cleveland Clinic, Cleveland, Ohio; Rebecca Dang, MD, Lucile Packard Children’s Hospital Stanford, Palo Alto, California; Joy Solano MD, Children’s Mercy Hospital Kansas, Overland Park, Kansas; Nathaniel P. Goodrich MD, Children’s Hospital & Medical Center, Omaha, Nebraska; Ngozi Eboh MD, Texas Tech University Health Sciences Center, Dallas, Texas; Ashley Jenkins MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Rebecca Steuart MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Sonya Tang Girdwood MD, PhD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Alissa McInerney MD, Maria Fareri Children’s Hospital at Westchester Medical Center, Valhalla, New York; Sumeet Banker MD, MPH, New York Presbyterian Morgan Stanley Children’s Hospital, New York, New York; Corrie McDaniel DO, Seattle Children’s Hospital, Seattle, Washington; Christiane Lenzen MD, Rady Children’s Hospital, San Diego, California; Aleisha Nabower MD, Children’s Hospital & Medical Center, Omaha, Nebraska; Waheeda Samady MD, Ann & Robert H. Lurie Children’s Hospital, Chicago, Illinois; Jennifer Chen MD, Rady Children’s Hospital, San Diego, California; Marquita Genies MD, MPH, John’s Hopkins Children’s Center, Baltimore, Maryland; Justin Lockwood MD, Children’s Hospital Colorado, Aurora, Colorado; David Synhorst MD, Children’s Mercy Hospital Kansas, Overland Park, Kansas.

Cervical lymphadenitis is a common superficial neck infection in childhood. While most children with cervical lymphadenitis recover with antibiotic therapy, a subset can develop an abscess that may require surgical drainage. Radiologic imaging, most commonly ultrasound or computed tomography (CT), is often performed to identify such an abscess.^1-3 However, no national standards exist to guide clinician decision making around imaging in this population. In the absence of evidence-based guidelines, variability in frequency, timing, and modality of imaging likely exists in children hospitalized with cervical lymphadenitis.

As demonstrated for several other common pediatric conditions,^4,5 variability in imaging practices may contribute to overutilization of resources in children with cervical lymphadenitis. In particular, routinely conducting imaging on presentation may constitute overuse, as children with cervical lymphadenitis who present with less than 72 hours of neck swelling rarely undergo surgical drainage within the first 24 hours of hospitalization.^1,6,7 Imaging performed on presentation is often repeated later during hospitalization, particularly if the patient has not improved with antibiotic therapy. The net result may be unnecessary, redundant radiologic studies. Furthermore, serious complications such as bacteremia, extension of infection into the retropharyngeal space, or involvement of the airway or vasculature rarely occur in children with cervical lymphadenitis.^6,8 In this context, deferring initial imaging in this population is unlikely to lead to adverse outcomes and may reduce radiation exposure.

The overall objectives of this study are to describe hospital-level variation in imaging practices for pediatric cervical lymphadenitis and to examine the association between early imaging and outcomes in this population.

Study Design and Data Source

We conducted a multicenter, cross-sectional study using the Pediatric Health Information Systems (PHIS) database, which contains administrative and billing data from 49 geographically diverse children’s hospitals across the United States (US) affiliated with the Children’s Hospital Association (Lenexa, Kansas). PHIS includes data on patient demographics, discharge diagnoses, and procedures using the International Classification of Diseases, 9th (ICD-9) and 10th Revision (ICD-10) diagnosis codes, as well as daily billed resource utilization for laboratory tests, imaging studies, and medications. Encrypted medical record numbers permit longitudinal identification of children across multiple visits to the same hospital. Use of de-identified PHIS data was deemed to be nonhuman subjects research; our approach to validation of ICD codes using local electronic medical record review was reviewed and approved by the Cincinnati Children’s Hospital Medical Center Institutional Review Board.

Study Population

Our study team developed an algorithm to identify children with cervical lymphadenitis and minimize misclassification using PHIS (Appendix A). All children with lymphadenitis-related ICD-9 and ICD-10 discharge diagnosis codes were eligible for inclusion. Codes were validated at a single center via electronic medical record review; clinician-documented discharge diagnosis of cervical lymphadenitis or the presence of fever and unilateral or asymmetrical neck swelling with overlying skin changes was used as the reference standard. We then excluded children who did not receive antibiotics, children who received radiologic imaging not involving the head or neck (which suggested noncervical lymphadenitis or other illness), and children who had discharge diagnosis codes for other specified conditions that are sometimes associated with enlarged cervical lymph nodes but warrant different evaluation or treatment (eg, Kawasaki disease, retropharyngeal abscess, and dental abscess; Appendix A). Our final algorithm yielded a positive predictive value of 87.5% (95% CI: 79.2%-93.4%) when ICD-9 codes were considered, and 95.1% (95% CI: 88.9%-98.4%) when ICD-10 codes were considered (Appendix A).

This algorithm was subsequently applied to the PHIS database. Children ages two months to 18 years hospitalized at participating PHIS institutions between July 2013 and December 2017 with a diagnosis of cervical lymphadenitis as per the algorithm (Appendix A) were eligible for inclusion. For children with multiple eligible admissions during the study period, we only included the first hospitalization. Children with complex chronic condition diagnosis codes⁹ were excluded as their clinical complexity could influence decisions around timing and modality of diagnostic imaging. In addition, we excluded children who did not have an emergency department (ED) visit associated with their hospitalization. This step was intended to exclude children who were transferred from another institution, as imaging performed at outside institutions prior to transfer is not available in PHIS. To avoid overinflating hospital-level variation in the setting of a small sample size, we also excluded all children admitted to the five hospitals with fewer than 50 cases of cervical lymphadenitis during the study period. Our final cohort consisted of 44 PHIS hospitals.

Measures of Interest

To examine hospital-level variation in imaging practices, we measured the proportion of children at each hospital who underwent any neck imaging study, CT or ultrasound imaging, early imaging, and multiple imaging studies within a single hospitalization. Neck imaging was defined as the presence of a billing code for ultrasound, CT, or magnetic resonance imaging (MRI) study of the neck (Appendix B). Early imaging was defined as neck imaging conducted on day 0 of hospitalization (ie, calendar day of admission and ending at midnight). Multiple imaging studies were defined as the receipt of more than one imaging study, regardless of timing or modality. We also measured the proportion of children by hospital who received surgical drainage, defined by the presence of procedure codes for incision and drainage of abscess of the neck (Appendix B).

In examining patient-level association between early imaging and clinical outcomes, our primary outcome of interest was the receipt of multiple imaging studies. Secondary outcomes included rates of surgical drainage, length of stay (in hospital days), and rates of lymphadenitis-related hospital readmission within 30 days of index discharge.

Covariates

Baseline demographic characteristics included age, gender, race/ethnicity, and insurance type. We measured ED visits associated with lymphadenitis-related diagnosis codes in the 30 days prior to admission as a proxy measure for illness duration prior to presentation. To approximate illness severity, we included the following covariates: rates of intensive care unit admission on presentation, rates of receipt of intravenous (IV) analgesia (Appendix B) on hospital days prior to surgical drainage, and rates of receipt of broad-spectrum antibiotics on day 0 or 1 of hospitalization. Broad-spectrum antibiotics (Appendix B) were defined by an independent three-person review of available antibiotic codes (SD, SSS, and JT); differences were resolved by group consensus.

Analysis

Categorical variables were described using frequencies and percentages, while continuous data were described using median and interquartile range. We described hospital-level variation in imaging practices by calculating and comparing the proportion of children at each hospital who underwent any neck imaging study, CT imaging, ultrasound imaging, early imaging, multiple imaging studies, and surgical drainage.

Patient-level demographics and clinical characteristics were compared across groups using chi-square test. To examine the association between early imaging and outcomes, we used generalized linear or logistic mixed effects models to control for patient demographic characteristics and clinical markers of illness duration and severity, with a random effect for hospital to account for clustering. Patient demographics in the model defined a priori included age, race/ethnicity, and insurance type; clinical characteristics included prior ED visit for lymphadenitis, initial intensive care unit (ICU) admission, use of IV analgesia, and use of broad-spectrum antibiotics on day 0 or 1 of hospitalization. To assess the potential for misclassification related to the availability of calendar day but not time of imaging in PHIS, we conducted a secondary analysis to examine the patient-level association between early imaging and outcomes using an alternative definition for early imaging (defined as imaging conducted on day 0 or day 1 of hospitalization).

We identified 19,785 PHIS hospitalizations with lymphadenitis-related discharge diagnosis codes between July 1, 2013 and December 31, 2017. Applying our algorithm and exclusion criteria, we assembled a cohort of 10,014 children hospitalized with cervical lymphadenitis (Figure 1). Two-thirds of the children in our cohort were <4 years old, 42% were non-Hispanic white, and 63% had a government payor (Table 1). Neck imaging (ultrasound, CT, or MRI) was conducted in 8,103 (81%) children. CT imaging was performed in 4,097 (41%) of children, and early imaging was conducted in 6,111 (61%) of children with cervical lymphadenitis.

We noted hospital-level variation in rates of any neck imaging (median: 82.1%, interquartile range [IQR]: 77.7%-85.5%, full range: 68.7%-93.1%), CT imaging (median: 42.3%, IQR: 26.7%-55.2%, full range: 12.0%-81.5%), early imaging (median: 64.4%, IQR: 59.8%-68.4%, full range: 13.8%-76.9%), and multiple imaging studies (median: 23.7%, IQR: 18.6%-28.9%, full range: 1.2%-40.7%; Figure 2). Rates of surgical drainage also varied by hospital (median: 35.1%, IQR: 31.3%-42.0%, full range: 17.1%-54.5%).

In this large multicenter study of children with cervical lymphadenitis, we found variation in imaging practices across 44 US children’s hospitals. Children with cervical lymphadenitis who underwent early imaging were more likely to receive multiple imaging studies during a single hospitalization than those who did not receive early imaging. At the patient level, early imaging was also associated with higher rates of surgical drainage, more frequent 30-day readmission, and longer lengths of stay.

To our knowledge, imaging practices in the population of children hospitalized with cervical lymphadenitis have not been previously characterized in the US; one study from Atlanta, Georgia, describes imaging practices in all children evaluated in the ED.¹ Single-center studies of children hospitalized with cervical lymphadenitis have been previously conducted in Canada⁶ and New Zealand,⁸ in which 42%-51% of children received imaging. In our study, most (81%) children hospitalized with lymphadenitis received some form of imaging, with 61% of all children receiving early imaging. Furthermore, 41% received CT imaging, as compared with 8%-10% of children in the aforementioned studies from Canada and New Zealand.^6,8 This finding is consistent with a pattern of imaging overuse in the US, which has amongst the highest utilization rates globally for advanced imaging such as CT and MRI.^10,11 Identifying opportunities to safely reduce routine imaging, particularly CT imaging, in this population could decrease unnecessary radiation exposure without compromising outcomes.

We also noted variability in imaging practices across PHIS hospitals. Some of this variability may be partially explained by differences in the patient population or illness severity across hospitals. However, given the absence of evidence-based best practices for children with cervical lymphadenitis, clinicians may rely on anecdotal experience or local practice culture to guide their decision making,¹² leading to variability in frequency, timing, and modality of imaging.

At the patient level, we found that children who received early imaging were more likely to receive multiple imaging studies. This finding supports our hypothesis that clinicians often order a second imaging study when the initial imaging study does not clearly demonstrate an abscess, and the child subsequently fails to demonstrate clear improvement after 24-48 hours of antibiotics.

Furthermore, early imaging was associated with overall increased utilization in our cohort, including increased likelihood of surgical drainage, 30-day readmission for lymphadenitis, as well as longer lengths of stay. Confounding may be one explanation for this finding. For instance, clinicians may pursue early imaging in children who present with longer duration of symptoms or more severe illness on presentation, as these factors may be associated with abscess formation.^1,6,7 These clinical covariates are not available in PHIS. Thus, we used prior ED visits for lymphadenitis to approximate illness duration, and initial admission to ICU, receipt of IV analgesia, and receipt of broad-spectrum antibiotics to approximate illness severity in an attempt to mitigate confounding. However, our proxy measures may not appropriately estimate illness duration and severity. For instance, children who had urgent care or outpatient visits for lymphadenitis would not be captured using the proxy of prior ED visit for lymphadenitis. Similarly, use of broad-spectrum antibiotics and IV analgesia may be influenced by provider or institutional preference rather than illness severity. Thus, residual confounding may exist despite adjusting for these measures.

On the other hand, it is also possible that a proportion of children with a small fluid collection on imaging may have improved with antibiotics alone. There is a growing body of evidence in children with other head and neck infections (eg, retropharyngeal abscess and orbital cellulitis with periosteal abscess)^13-15 that suggests that children with small abscesses often improve with antibiotic therapy alone. In children with cervical lymphadenitis who have small or developing abscesses identified via routine imaging on presentation, clinicians may be driven to pursue a surgical intervention with uncertain benefit. Deferring routine imaging in this population may provide an opportunity to improve the value of care in children with lymphadenitis without adversely affecting outcomes.

Upon closer examination of readmissions, children who received early imaging during index hospitalization were more likely to have a 30-day readmission when only evaluating the subset of patients who did not receive surgical drainage during the index admission. This suggests that readmissions are less likely attributable to surgical complications and more likely a reflection of the natural history of lymphadenitis in which a subset of patients eventually develop an abscess. Further supporting this, 61% of children who had a 30-day readmission for lymphadenitis underwent surgical drainage during readmission. Given that lymphadenitis is a slow-brewing infection in which serious complications are rare, patients who demonstrate gradual clinical improvement do not need to remain hospitalized and serially imaged to identify a possible abscess. Outpatient expectant management and readmission as needed for drainage may be an acceptable approach.

This study has several limitations given our use of an administrative database. Children with lymphadenitis may have been misclassified as these patients were identified using discharge diagnosis codes. To mitigate this potential misclassification, we conducted a structured validation process and found that the included codes had high positive predictive values (Appendix A). This validation process was conducted at a single hospital, and coding may vary across hospitals. To approximate sensitivity, we also sampled children without our included codes but with neck imaging and antibiotic use, and found that rates of cervical lymphadenitis were very low among children without our included diagnosis codes.

Furthermore, we were unable to measure the exact time of imaging study in PHIS; we used imaging conducted on hospital day 0 as a proxy measure for imaging conducted within the first 24 hours of presentation. With this definition, some children who had early imaging were likely misclassified as not having received early imaging. For example, a patient who arrived in the ED at 9 pm on day 0 of admission and had a neck ultrasound performed at 1 am would be classified as having had an imaging study on day 1 of hospitalization even though the imaging study was conducted within 4 hours of presentation. Using an alternative definition of early imaging as imaging conducted on hospital day 0 and day 1, we found a much higher adjusted OR for multiple imaging studies, with similar associations for secondary outcomes. As such, our definition of early imaging as day 0 likely biases the results toward the null; the true increase in likelihood of multiple imaging for those who receive early imaging is probably greater than our conservative estimation.

Additionally, there may be a subset of children who underwent imaging prior to presentation at the PHIS hospital ED for further workup and admission. Imaging conducted outside a PHIS hospital was not captured in this database. Similarly, children who had a readmission at a different hospital than their index admission would not be captured using PHIS. Finally, PHIS captures data from children’s hospitals; practices at these hospitals may not be generalizable to practices in the community hospital setting.

In conclusion, we found that imaging practices in children hospitalized with cervical lymphadenitis were widely variable across hospitals. Children receiving early imaging had more resource utilization and intervention when compared with children who did not receive early imaging. Our findings may represent a cascade effect, in which routinely conducted early imaging prompts clinicians to pursue more testing and interventions in this population. Future studies should obtain more detailed patient level covariates to further characterize clinical factors that may impact decisions around imaging and clinical outcomes for children with cervical lymphadenitis.

Acknowledgments

The authors would like to acknowledge the following investigators for their contributions to data interpretation and review of the final manuscript: Angela Choe MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Margaret Rush MD, Children’s National Medical Center, Washington, DC; Ryosuke Takei MD, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania; Wallis Molchen DO, Texas Children’s Hospital, Houston, Texas; Stephanie Royer Moss MD, Cleveland Clinic, Cleveland, Ohio; Rebecca Dang, MD, Lucile Packard Children’s Hospital Stanford, Palo Alto, California; Joy Solano MD, Children’s Mercy Hospital Kansas, Overland Park, Kansas; Nathaniel P. Goodrich MD, Children’s Hospital & Medical Center, Omaha, Nebraska; Ngozi Eboh MD, Texas Tech University Health Sciences Center, Dallas, Texas; Ashley Jenkins MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Rebecca Steuart MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Sonya Tang Girdwood MD, PhD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Alissa McInerney MD, Maria Fareri Children’s Hospital at Westchester Medical Center, Valhalla, New York; Sumeet Banker MD, MPH, New York Presbyterian Morgan Stanley Children’s Hospital, New York, New York; Corrie McDaniel DO, Seattle Children’s Hospital, Seattle, Washington; Christiane Lenzen MD, Rady Children’s Hospital, San Diego, California; Aleisha Nabower MD, Children’s Hospital & Medical Center, Omaha, Nebraska; Waheeda Samady MD, Ann & Robert H. Lurie Children’s Hospital, Chicago, Illinois; Jennifer Chen MD, Rady Children’s Hospital, San Diego, California; Marquita Genies MD, MPH, John’s Hopkins Children’s Center, Baltimore, Maryland; Justin Lockwood MD, Children’s Hospital Colorado, Aurora, Colorado; David Synhorst MD, Children’s Mercy Hospital Kansas, Overland Park, Kansas.

Cervical lymphadenitis is a common superficial neck infection in childhood. While most children with cervical lymphadenitis recover with antibiotic therapy, a subset can develop an abscess that may require surgical drainage. Radiologic imaging, most commonly ultrasound or computed tomography (CT), is often performed to identify such an abscess.^1-3 However, no national standards exist to guide clinician decision making around imaging in this population. In the absence of evidence-based guidelines, variability in frequency, timing, and modality of imaging likely exists in children hospitalized with cervical lymphadenitis.

As demonstrated for several other common pediatric conditions,^4,5 variability in imaging practices may contribute to overutilization of resources in children with cervical lymphadenitis. In particular, routinely conducting imaging on presentation may constitute overuse, as children with cervical lymphadenitis who present with less than 72 hours of neck swelling rarely undergo surgical drainage within the first 24 hours of hospitalization.^1,6,7 Imaging performed on presentation is often repeated later during hospitalization, particularly if the patient has not improved with antibiotic therapy. The net result may be unnecessary, redundant radiologic studies. Furthermore, serious complications such as bacteremia, extension of infection into the retropharyngeal space, or involvement of the airway or vasculature rarely occur in children with cervical lymphadenitis.^6,8 In this context, deferring initial imaging in this population is unlikely to lead to adverse outcomes and may reduce radiation exposure.

The overall objectives of this study are to describe hospital-level variation in imaging practices for pediatric cervical lymphadenitis and to examine the association between early imaging and outcomes in this population.

Study Design and Data Source

We conducted a multicenter, cross-sectional study using the Pediatric Health Information Systems (PHIS) database, which contains administrative and billing data from 49 geographically diverse children’s hospitals across the United States (US) affiliated with the Children’s Hospital Association (Lenexa, Kansas). PHIS includes data on patient demographics, discharge diagnoses, and procedures using the International Classification of Diseases, 9th (ICD-9) and 10th Revision (ICD-10) diagnosis codes, as well as daily billed resource utilization for laboratory tests, imaging studies, and medications. Encrypted medical record numbers permit longitudinal identification of children across multiple visits to the same hospital. Use of de-identified PHIS data was deemed to be nonhuman subjects research; our approach to validation of ICD codes using local electronic medical record review was reviewed and approved by the Cincinnati Children’s Hospital Medical Center Institutional Review Board.

Study Population

Our study team developed an algorithm to identify children with cervical lymphadenitis and minimize misclassification using PHIS (Appendix A). All children with lymphadenitis-related ICD-9 and ICD-10 discharge diagnosis codes were eligible for inclusion. Codes were validated at a single center via electronic medical record review; clinician-documented discharge diagnosis of cervical lymphadenitis or the presence of fever and unilateral or asymmetrical neck swelling with overlying skin changes was used as the reference standard. We then excluded children who did not receive antibiotics, children who received radiologic imaging not involving the head or neck (which suggested noncervical lymphadenitis or other illness), and children who had discharge diagnosis codes for other specified conditions that are sometimes associated with enlarged cervical lymph nodes but warrant different evaluation or treatment (eg, Kawasaki disease, retropharyngeal abscess, and dental abscess; Appendix A). Our final algorithm yielded a positive predictive value of 87.5% (95% CI: 79.2%-93.4%) when ICD-9 codes were considered, and 95.1% (95% CI: 88.9%-98.4%) when ICD-10 codes were considered (Appendix A).

This algorithm was subsequently applied to the PHIS database. Children ages two months to 18 years hospitalized at participating PHIS institutions between July 2013 and December 2017 with a diagnosis of cervical lymphadenitis as per the algorithm (Appendix A) were eligible for inclusion. For children with multiple eligible admissions during the study period, we only included the first hospitalization. Children with complex chronic condition diagnosis codes⁹ were excluded as their clinical complexity could influence decisions around timing and modality of diagnostic imaging. In addition, we excluded children who did not have an emergency department (ED) visit associated with their hospitalization. This step was intended to exclude children who were transferred from another institution, as imaging performed at outside institutions prior to transfer is not available in PHIS. To avoid overinflating hospital-level variation in the setting of a small sample size, we also excluded all children admitted to the five hospitals with fewer than 50 cases of cervical lymphadenitis during the study period. Our final cohort consisted of 44 PHIS hospitals.

Measures of Interest

To examine hospital-level variation in imaging practices, we measured the proportion of children at each hospital who underwent any neck imaging study, CT or ultrasound imaging, early imaging, and multiple imaging studies within a single hospitalization. Neck imaging was defined as the presence of a billing code for ultrasound, CT, or magnetic resonance imaging (MRI) study of the neck (Appendix B). Early imaging was defined as neck imaging conducted on day 0 of hospitalization (ie, calendar day of admission and ending at midnight). Multiple imaging studies were defined as the receipt of more than one imaging study, regardless of timing or modality. We also measured the proportion of children by hospital who received surgical drainage, defined by the presence of procedure codes for incision and drainage of abscess of the neck (Appendix B).

In examining patient-level association between early imaging and clinical outcomes, our primary outcome of interest was the receipt of multiple imaging studies. Secondary outcomes included rates of surgical drainage, length of stay (in hospital days), and rates of lymphadenitis-related hospital readmission within 30 days of index discharge.

Covariates

Baseline demographic characteristics included age, gender, race/ethnicity, and insurance type. We measured ED visits associated with lymphadenitis-related diagnosis codes in the 30 days prior to admission as a proxy measure for illness duration prior to presentation. To approximate illness severity, we included the following covariates: rates of intensive care unit admission on presentation, rates of receipt of intravenous (IV) analgesia (Appendix B) on hospital days prior to surgical drainage, and rates of receipt of broad-spectrum antibiotics on day 0 or 1 of hospitalization. Broad-spectrum antibiotics (Appendix B) were defined by an independent three-person review of available antibiotic codes (SD, SSS, and JT); differences were resolved by group consensus.

Analysis

Categorical variables were described using frequencies and percentages, while continuous data were described using median and interquartile range. We described hospital-level variation in imaging practices by calculating and comparing the proportion of children at each hospital who underwent any neck imaging study, CT imaging, ultrasound imaging, early imaging, multiple imaging studies, and surgical drainage.

Patient-level demographics and clinical characteristics were compared across groups using chi-square test. To examine the association between early imaging and outcomes, we used generalized linear or logistic mixed effects models to control for patient demographic characteristics and clinical markers of illness duration and severity, with a random effect for hospital to account for clustering. Patient demographics in the model defined a priori included age, race/ethnicity, and insurance type; clinical characteristics included prior ED visit for lymphadenitis, initial intensive care unit (ICU) admission, use of IV analgesia, and use of broad-spectrum antibiotics on day 0 or 1 of hospitalization. To assess the potential for misclassification related to the availability of calendar day but not time of imaging in PHIS, we conducted a secondary analysis to examine the patient-level association between early imaging and outcomes using an alternative definition for early imaging (defined as imaging conducted on day 0 or day 1 of hospitalization).

We identified 19,785 PHIS hospitalizations with lymphadenitis-related discharge diagnosis codes between July 1, 2013 and December 31, 2017. Applying our algorithm and exclusion criteria, we assembled a cohort of 10,014 children hospitalized with cervical lymphadenitis (Figure 1). Two-thirds of the children in our cohort were <4 years old, 42% were non-Hispanic white, and 63% had a government payor (Table 1). Neck imaging (ultrasound, CT, or MRI) was conducted in 8,103 (81%) children. CT imaging was performed in 4,097 (41%) of children, and early imaging was conducted in 6,111 (61%) of children with cervical lymphadenitis.

We noted hospital-level variation in rates of any neck imaging (median: 82.1%, interquartile range [IQR]: 77.7%-85.5%, full range: 68.7%-93.1%), CT imaging (median: 42.3%, IQR: 26.7%-55.2%, full range: 12.0%-81.5%), early imaging (median: 64.4%, IQR: 59.8%-68.4%, full range: 13.8%-76.9%), and multiple imaging studies (median: 23.7%, IQR: 18.6%-28.9%, full range: 1.2%-40.7%; Figure 2). Rates of surgical drainage also varied by hospital (median: 35.1%, IQR: 31.3%-42.0%, full range: 17.1%-54.5%).

In this large multicenter study of children with cervical lymphadenitis, we found variation in imaging practices across 44 US children’s hospitals. Children with cervical lymphadenitis who underwent early imaging were more likely to receive multiple imaging studies during a single hospitalization than those who did not receive early imaging. At the patient level, early imaging was also associated with higher rates of surgical drainage, more frequent 30-day readmission, and longer lengths of stay.

To our knowledge, imaging practices in the population of children hospitalized with cervical lymphadenitis have not been previously characterized in the US; one study from Atlanta, Georgia, describes imaging practices in all children evaluated in the ED.¹ Single-center studies of children hospitalized with cervical lymphadenitis have been previously conducted in Canada⁶ and New Zealand,⁸ in which 42%-51% of children received imaging. In our study, most (81%) children hospitalized with lymphadenitis received some form of imaging, with 61% of all children receiving early imaging. Furthermore, 41% received CT imaging, as compared with 8%-10% of children in the aforementioned studies from Canada and New Zealand.^6,8 This finding is consistent with a pattern of imaging overuse in the US, which has amongst the highest utilization rates globally for advanced imaging such as CT and MRI.^10,11 Identifying opportunities to safely reduce routine imaging, particularly CT imaging, in this population could decrease unnecessary radiation exposure without compromising outcomes.

We also noted variability in imaging practices across PHIS hospitals. Some of this variability may be partially explained by differences in the patient population or illness severity across hospitals. However, given the absence of evidence-based best practices for children with cervical lymphadenitis, clinicians may rely on anecdotal experience or local practice culture to guide their decision making,¹² leading to variability in frequency, timing, and modality of imaging.

At the patient level, we found that children who received early imaging were more likely to receive multiple imaging studies. This finding supports our hypothesis that clinicians often order a second imaging study when the initial imaging study does not clearly demonstrate an abscess, and the child subsequently fails to demonstrate clear improvement after 24-48 hours of antibiotics.

Furthermore, early imaging was associated with overall increased utilization in our cohort, including increased likelihood of surgical drainage, 30-day readmission for lymphadenitis, as well as longer lengths of stay. Confounding may be one explanation for this finding. For instance, clinicians may pursue early imaging in children who present with longer duration of symptoms or more severe illness on presentation, as these factors may be associated with abscess formation.^1,6,7 These clinical covariates are not available in PHIS. Thus, we used prior ED visits for lymphadenitis to approximate illness duration, and initial admission to ICU, receipt of IV analgesia, and receipt of broad-spectrum antibiotics to approximate illness severity in an attempt to mitigate confounding. However, our proxy measures may not appropriately estimate illness duration and severity. For instance, children who had urgent care or outpatient visits for lymphadenitis would not be captured using the proxy of prior ED visit for lymphadenitis. Similarly, use of broad-spectrum antibiotics and IV analgesia may be influenced by provider or institutional preference rather than illness severity. Thus, residual confounding may exist despite adjusting for these measures.

On the other hand, it is also possible that a proportion of children with a small fluid collection on imaging may have improved with antibiotics alone. There is a growing body of evidence in children with other head and neck infections (eg, retropharyngeal abscess and orbital cellulitis with periosteal abscess)^13-15 that suggests that children with small abscesses often improve with antibiotic therapy alone. In children with cervical lymphadenitis who have small or developing abscesses identified via routine imaging on presentation, clinicians may be driven to pursue a surgical intervention with uncertain benefit. Deferring routine imaging in this population may provide an opportunity to improve the value of care in children with lymphadenitis without adversely affecting outcomes.

Upon closer examination of readmissions, children who received early imaging during index hospitalization were more likely to have a 30-day readmission when only evaluating the subset of patients who did not receive surgical drainage during the index admission. This suggests that readmissions are less likely attributable to surgical complications and more likely a reflection of the natural history of lymphadenitis in which a subset of patients eventually develop an abscess. Further supporting this, 61% of children who had a 30-day readmission for lymphadenitis underwent surgical drainage during readmission. Given that lymphadenitis is a slow-brewing infection in which serious complications are rare, patients who demonstrate gradual clinical improvement do not need to remain hospitalized and serially imaged to identify a possible abscess. Outpatient expectant management and readmission as needed for drainage may be an acceptable approach.

This study has several limitations given our use of an administrative database. Children with lymphadenitis may have been misclassified as these patients were identified using discharge diagnosis codes. To mitigate this potential misclassification, we conducted a structured validation process and found that the included codes had high positive predictive values (Appendix A). This validation process was conducted at a single hospital, and coding may vary across hospitals. To approximate sensitivity, we also sampled children without our included codes but with neck imaging and antibiotic use, and found that rates of cervical lymphadenitis were very low among children without our included diagnosis codes.

Furthermore, we were unable to measure the exact time of imaging study in PHIS; we used imaging conducted on hospital day 0 as a proxy measure for imaging conducted within the first 24 hours of presentation. With this definition, some children who had early imaging were likely misclassified as not having received early imaging. For example, a patient who arrived in the ED at 9 pm on day 0 of admission and had a neck ultrasound performed at 1 am would be classified as having had an imaging study on day 1 of hospitalization even though the imaging study was conducted within 4 hours of presentation. Using an alternative definition of early imaging as imaging conducted on hospital day 0 and day 1, we found a much higher adjusted OR for multiple imaging studies, with similar associations for secondary outcomes. As such, our definition of early imaging as day 0 likely biases the results toward the null; the true increase in likelihood of multiple imaging for those who receive early imaging is probably greater than our conservative estimation.

Additionally, there may be a subset of children who underwent imaging prior to presentation at the PHIS hospital ED for further workup and admission. Imaging conducted outside a PHIS hospital was not captured in this database. Similarly, children who had a readmission at a different hospital than their index admission would not be captured using PHIS. Finally, PHIS captures data from children’s hospitals; practices at these hospitals may not be generalizable to practices in the community hospital setting.

In conclusion, we found that imaging practices in children hospitalized with cervical lymphadenitis were widely variable across hospitals. Children receiving early imaging had more resource utilization and intervention when compared with children who did not receive early imaging. Our findings may represent a cascade effect, in which routinely conducted early imaging prompts clinicians to pursue more testing and interventions in this population. Future studies should obtain more detailed patient level covariates to further characterize clinical factors that may impact decisions around imaging and clinical outcomes for children with cervical lymphadenitis.

Acknowledgments

The authors would like to acknowledge the following investigators for their contributions to data interpretation and review of the final manuscript: Angela Choe MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Margaret Rush MD, Children’s National Medical Center, Washington, DC; Ryosuke Takei MD, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania; Wallis Molchen DO, Texas Children’s Hospital, Houston, Texas; Stephanie Royer Moss MD, Cleveland Clinic, Cleveland, Ohio; Rebecca Dang, MD, Lucile Packard Children’s Hospital Stanford, Palo Alto, California; Joy Solano MD, Children’s Mercy Hospital Kansas, Overland Park, Kansas; Nathaniel P. Goodrich MD, Children’s Hospital & Medical Center, Omaha, Nebraska; Ngozi Eboh MD, Texas Tech University Health Sciences Center, Dallas, Texas; Ashley Jenkins MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Rebecca Steuart MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Sonya Tang Girdwood MD, PhD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Alissa McInerney MD, Maria Fareri Children’s Hospital at Westchester Medical Center, Valhalla, New York; Sumeet Banker MD, MPH, New York Presbyterian Morgan Stanley Children’s Hospital, New York, New York; Corrie McDaniel DO, Seattle Children’s Hospital, Seattle, Washington; Christiane Lenzen MD, Rady Children’s Hospital, San Diego, California; Aleisha Nabower MD, Children’s Hospital & Medical Center, Omaha, Nebraska; Waheeda Samady MD, Ann & Robert H. Lurie Children’s Hospital, Chicago, Illinois; Jennifer Chen MD, Rady Children’s Hospital, San Diego, California; Marquita Genies MD, MPH, John’s Hopkins Children’s Center, Baltimore, Maryland; Justin Lockwood MD, Children’s Hospital Colorado, Aurora, Colorado; David Synhorst MD, Children’s Mercy Hospital Kansas, Overland Park, Kansas.

Sepsis is defined as a life-threatening organ dysfunction that occurs in response to systemic infection.^1,2 It is frequently fatal, common in hospital medicine, and a leading contributor to critical illness, morbidity, and healthcare expenditures.^2-5 While sepsis care and outcomes have improved in the past decade,^6,7 inpatient mortality remains high.⁸

A number of studies have sought to determine whether race plays a role in sepsis mortality. While Black patients with sepsis have frequently been identified as having the highest rates of death,^9-14 similar observations have been made for most non-White races/ethnicities.^13-15 Studies have also demonstrated higher rates of hospital-acquired infections among Asian and Latino patients.¹⁶

There are several possible explanations for why racial minorities experience disparate outcomes in sepsis, including access to care, comorbidities, implicit biases, and biological or environmental factors,^17-20 as well as characteristics of hospitals most likely to care for racial minorities.^13,15,21 One explanation that has not been explored is that racial disparities in sepsis are mediated by language. Limited English proficiency (LEP) has previously been associated with increased rates of adverse hospital events,²² longer length of stay,²³ and greater likelihood of readmission.²⁴ LEP has also been shown to represent a significant barrier to accessing healthcare and preventive screening.²⁵ The role of LEP in sepsis mortality, however, has yet to be examined.

The diverse patient population at the University of California, San Francisco (UCSF) provides a unique opportunity to build upon existing literature by further exploring racial differences in sepsis, specifically by investigating the role of LEP. The objective of this study was to determine the association between LEP and inpatient mortality among adults hospitalized with sepsis.

Setting

The study was conducted at the University of California, San Francisco, California (UCSF), an 800-bed tertiary care, academic medical center. It was approved by the UCSF Institutional Review Board with waiver of informed consent. UCSF cares for a population of patients who are racially and linguistically diverse, with high proportions of patients of East Asian descent and with LEP. According to recent United States census estimates, more than half of San Francisco County residents identify as non-White (35% Asians, 15% Hispanic/Latino, 6% Black), and 44% report speaking a language other than English at home.²⁶

Study Population and Data Collection

The UCSF Medical Center uses the electronic health record (EHR) Epic (Epic 2017, Epic Systems Corporation, Verona, Wisconsin). We obtained computerized EHR data from Clarity, the relational database that stores Epic’s inpatient data in thousands of tables. We identified all patients ≥18 years of age presenting to the emergency department (ED) between June 1, 2012 and December 31, 2016 with suspected serious infection, defined as having blood cultures ordered within 72 hours of ED presentation (N = 25,441). Patients who did not receive at least two doses of intravenous (IV) antibiotics within 48 hours were excluded, as they were unlikely to have serious infections.

We defined sepsis based on Sepsis-3 consensus guidelines² as a change in sequential [sepsis-related] organ failure assessment (SOFA) score ≥2 within the first 48 hours of ED presentation. The SOFA score is comprised of six variables representing different organ systems, each rated 0-4 based on the degree of dysfunction.² Patient vital signs, laboratory data, vasopressor medication doses, and ventilator settings were used to determine the exact timestamp at which each patient attained a change in SOFA score ≥2. Missing values were considered to be normal. To adjust for baseline organ dysfunction, SOFA elements associated with elevated bilirubin and/or creatinine were excluded for patients with chronic liver/kidney disease based on Elixhauser comorbidities.²⁷ We chose to focus on the first 48 hours in an attempt to capture patients with the most severe illnesses and the highest probability of true sepsis.

All primary and secondary International Classification of Diseases (ICD)-9/10 diagnosis codes were extracted from Clarity coding tables at the time of hospital discharge. Diagnosis codes signifying bacterial infection were grouped into the following categories based on type/location: pneumonia; bacteremia; urinary tract infection; and skin and soft tissue infection. All remaining diagnostic codes indicating bacterial infections at other sites were categorized as “Other”. If no codes indicating infection were present, patients were categorized as “None coded”. Patients with discharge diagnosis codes of “sepsis” were also identified. Dates and times of antibiotic administrations were obtained from the medications table. Time to first antibiotic was defined as the time in minutes from ED presentation to initiation of the first IV antibacterial medication. This variable was transformed using a natural log transformation based on best fit for normal distribution.

We limited our analyses to 8,974 patients who were diagnosed with sepsis as defined above and had either (1) ≥4 qualifying antibiotic days (QADs) or (2) an ICD-9/10 discharge diagnosis code of “sepsis” (Figure). QADs were defined based on the recent publication by Rhee et al. as having received four or more consecutive days of antibiotics, with the first dose given IV within 48 hours of presentation.²⁸ Patients who died or were discharged to hospice prior to the 4th QAD were also included. These additional parameters were added to increase specificity of the study sample for patients with true sepsis. Patients admitted to all levels of care (acute care, transitional care unit [TCU], intensive care unit [ICU]) and under all hospital services were included. There were no missing data for mortality, race, or language. We chose to focus on patients with sepsis in this initial study as this is a common diagnosis in hospital medicine that is enriched for high mortality.

Primary Outcome

The primary outcome of the study was inpatient mortality, which was obtained from the hospital encounters table in Clarity.

Primary Predictors

The primary predictor of interest was LEP. The encounter numbers from the dataset were used to link to self-reported demographic data, including “preferred language” and need for interpreter services. A manual chart review of 60 patients speaking the top six languages was conducted to verify the accuracy of the data on language and interpreter use (KNK). Defining the gold standard for LEP as having any chart note indicating non-English language and/or that an interpreter was used, the “interpreter needed” variable in Epic was found to have a positive predictive value for LEP of 100%. Therefore, patients in the study cohort were defined as having LEP if they met both of the following criteria: (1) a self-reported “preferred language” other than English and (2) having the “interpreter needed” variable indicating “yes”.

Covariate Data Collection

Additional data were obtained from the demographics tables, including age, race, sex, and insurance status. Race and ethnicity were combined into a single five-category variable including White, Asian, Black, Latino, and Other. This approach has been suggested as the best way to operationalize these variables²⁹ and has been utilized by similar studies in the literature.^9,14,15 We considered the Asian race to include all people of East Asian, Southeast Asian, or South Asian descent, which is consistent with the United States Census Bureau definition.³⁰ Patients identifying as Native Hawaiians/Pacific Islanders, Native Americans/Alaskan Natives, as well as those with unspecified race or ethnicity, were categorized as Other. Insurance status was categorized as Commercial, Medicare, Medicaid, or Other.

We estimated illness severity in several ways. First, the total qualifying SOFA score was calculated for each patient, which was defined as the total score achieved at the time that SOFA criteria were first met (≥2, within 48 hours). Second, we dichotomized patients based on whether they had received mechanical ventilation at any point during hospitalization. Finally, we used admission location as a surrogate marker for severity at the time of initial hospitalization.

To estimate the burden of baseline comorbidities, we calculated the van Walraven score (VWS),³¹ a validated modification of the Elixhauser Comorbidity Index.²⁷ This score conveys an estimated risk of in hospital death based on ICD-9/10 diagnosis codes for preexisting conditions, which ranges from <1% for the minimum score of –19 to >99% for the maximum score of 89.

Statistical Analyses

All statistical analyses were performed using Stata software version 15 (StataCorp LLC, College Station, Texas). Baseline demographics and patient characteristics were stratified by LEP. These were compared using two-sample t-tests or chi-squared tests of significance. Wilcoxon rank-sum tests were used for non-normally distributed variables. Inpatient mortality was compared across all races stratified by LEP using chi-squared tests of significance.

We fit a series of multivariable logistic regression models to examine the association between race and inpatient mortality adjusting for LEP and other patient/clinical characteristics. We first examined the unadjusted association between mortality and race; then adjusted for LEP alone; and finally adjusted for all covariates of interest, including LEP, age, sex, insurance status, year, admission level of care, VWS, total qualifying SOFA score, need for mechanical ventilation, site of infection, and time to first IV antibiotic. A subgroup analysis was also performed using the fully adjusted model restricted to patients who were mechanically ventilated. This population was selected because the patients (1) have among the highest severity of illness and (2) share a common barrier to communication, regardless of English proficiency.

Several potential interactions between LEP with other covariates were explored, including age, race, ICU admission level of care, and need for mechanical ventilation. Lastly, a mediation analysis was performed based on Baron & Kenny’s four-step model³² in order to calculate the proportion of the association between race and mortality explained by the proposed mediator (LEP).

To evaluate for the likelihood of residual confounding, we calculated an E-value, which is defined as the minimum strength of association that an unmeasured confounder would need to have with both the predictor and outcome variables, above and beyond the measured covariates, in order to fully explain away an observed predictor-outcome association.^33,34

We identified 8,974 patients hospitalized with sepsis based on the above inclusion criteria. This represented a medically complex, racially and linguistically diverse population (Table 1). The cohort was comprised of 24% Asian, 12% Black, and 11% Latino patients. Among those categorized as Other race, Native Americans/Alaskan Natives and Native Hawaiians/Pacific Islanders accounted for 4% (n = 31) and 21% (n = 159), respectively. A fifth of all patients had LEP (n = 1,716), 62% of whom were Asian (n = 1,064). Patients with LEP tended to be older, female, and to have a greater number of comorbid conditions (Table 1). The total qualifying SOFA score was also higher among patients with LEP (median 5; interquartile range [IQR]: 4-8 vs 5; IQR: 3-7; P <.001), though there was no association between LEP and mechanical ventilation (P = .22). The prevalence of LEP differed significantly across races, with 50% LEP among Asians, 32% among Latinos, 5% among White patients (P < .001). Only eight Black patients had LEP. More than 40 unique languages were represented in the cohort, with English, Cantonese, Spanish, Russian, and Mandarin accounting for ~95% (Appendix Table 1). Among Latino patients, 63% spoke English and 36% spoke Spanish.

In-hospital mortality was significantly higher among patients who had LEP (n = 268/1,716, 16%) compared to non-LEP patients (n = 678/7,258, 9%), with 80% greater unadjusted odds of mortality (OR 1.80; 95% CI: 1.54-2.09; P < .001). Notably we also found that Asian race was associated with a 1.57 unadjusted odds of mortality compared to White race (95% CI: 1.34-1.85; P < .001). Age, VWS, total qualifying SOFA score, mechanical ventilation, and admission level of care all exhibited a positive dose-response association with mortality (Appendix Table 2). In unadjusted analyses, there was no evidence of interaction between LEP and age (P = .38), LEP and race (P = .45), LEP and ICU admission level of care (P = .31), or LEP and mechanical ventilation (P = .19). Asian patients had the highest overall mortality (14% total, 17% with LEP). LEP was associated with increased unadjusted mortality among White, Asian, and Other races compared to their non-LEP counterparts (Appendix Figure 1). There was no significant difference in mortality between Latino patients with and without LEP. The sample size for Black patients with LEP (n = 8) was too small to draw conclusions about mortality.

Following multivariable logistic regression modeling for the association between race and mortality, we found that the increased odds of death among Asian patients was partially attenuated after adjusting for LEP (odds ratio [OR] 1.23, 95% CI: 1.02-1.48; P = .03; Table 2). Meanwhile, LEP was associated with a 1.66 odds of mortality (95% CI: 1.38-1.99; P < .001) after adjustment for race. In the full multivariable model adjusting for demographics and clinical characteristics, illness severity, and comorbidities, LEP was associated with a 31% increase in the odds of mortality compared to non-LEP (95% CI: 1.06-1.63; P = .02). In this model, the association between Asian race and mortality was now fully attenuated, with a point estimate near 1.0 (OR 0.98; 95% CI: 0.79-1.22; P = .87). Markers of illness severity, including total qualifying SOFA score (OR 1.23; 95% CI: 1.20-1.27; P < .001) and need for mechanical ventilation (OR 1.88; 95% CI: 1.52-2.33; P < .001), were both associated with greater odds of death. Based on a four-step mediation analysis, LEP was found to be a partial mediator to the association between Asian race and mortality (76% proportion explained). The E-value for the association between LEP and mortality was 1.95, with an E-value for the corresponding confidence interval of 1.29.

At a single US academic medical center serving a diverse population, we found that LEP was associated with sepsis mortality across all races except Black and Latino, conveying a 31% increase in the odds of death after adjusting for illness severity, comorbidities, and baseline characteristics. The higher mortality among Asian patients was largely mediated by LEP (76% proportion explained). While previous studies have variably found Black, Asian, Latino, and other non-White races/ethnicities to be at an increased risk of death from sepsis,^9-15 LEP has not been previously evaluated as a mediator of sepsis mortality. We were uniquely suited to uncover such an association due to the racial and linguistic diversity of our patient population. LEP has previously been implicated in poor health outcomes among hospitalized patients in general.^22-24 Future studies will be necessary to determine whether similar associations between LEP and mortality are observed among broader patient populations outside of sepsis.

There are a number of possible explanations for how LEP could mediate the association between race and mortality. First, LEP is known to be associated with greater difficulties in accessing medical care,²⁵ which could result in poorer baseline control of chronic comorbid conditions, fewer opportunities for preventive screening, and greater reluctance to seek medical attention when ill, theoretically leading to more severe presentations and worse outcomes. Indeed, LEP patients in our cohort had both a shorter median time to receiving their first antibiotic, as well as a higher total qualifying SOFA score, both of which may suggest more severe initial presentations. LEP is also known to contribute to, or exacerbate, the impact of low health literacy, which is itself associated with poor health.³⁵ Second, implicit biases may also have been present, as they are known to be common among healthcare providers and have been shown to negatively impact patient care.³⁶

Finally,it is possible that the association is related to the language barrier itself, which impacts providers’ ability to take an appropriate clinical history, and can lead to clinical errors or delays in care.³⁷ The fact that the association between LEP and mortality was eliminated when the analysis was restricted to mechanically ventilated patients seems to support this, since differences in language proficiency become irrelevant in this subgroup. While we are unable to comment on causality based on this observational study, we included a directed acyclic graph (DAG) in the supplemental materials, which shows one proposed model for describing these associations (Appendix Figure 2).

Assuming that the language barrier itself does, at least in part, drive the observed association, LEP represents a potentially modifiable risk factor that could be a target for quality improvement interventions. There is evidence that the use of medical interpreters among patients with LEP leads to greater satisfaction, fewer errors, and improved clinical outcomes;³⁸ however, several recent studies have documented underutilization of professional interpreter services, even when readily available.^39,40 At our institution, phone and video interpreter services are available 24/7 for approximately 150 languages. Due to limitations inherent to the EHR, we were unable to ascertain the extent to which these services were used in the present study. Heavy clinical workloads, connectivity issues, and missing or faulty equipment represent theoretical barriers to utilization of these services.

There are some limitations to our study. First, by utilizing a large database of electronic data, the quality of our analyses was reliant on the accuracy of the EHR. Demographic data such as language may have been subject to misclassification due to self-reporting. We attempted to minimize this by also including the need for interpreter services within the definition of LEP, which was validated by manual chart review. Second, generalizability is limited in this single-center study conducted at an institution with unique demographics, wherein nearly two-thirds of the LEP patients were Asian, and the Chinese-speaking population outnumbered those who speak Spanish.

Finally, the most important limitation to our study is the potential for residual confounding. While we attempted to mitigate this by adjusting for as many clinically relevant covariates as possible, there may still be unmeasured confounders to the association between LEP and mortality, such as access to outpatient care, functional status, interpreter use, and other markers of illness severity like the number and type of supportive therapies received. Based on our E-value calculations, with an observed OR of 1.31 for the association between LEP and mortality, an unmeasured confounder with an OR of 1.95 would fully explain away this association, while an OR of 1.29 would shift the confidence interval to include the null. These values suggest at least some risk of residual confounding. The fact that our fully adjusted model included multiple covariates, including several markers of illness severity, does somewhat lessen the likelihood of a confounder achieving these values, since they represent the minimum strength of an unmeasured confounder above and beyond the measured covariates. Regardless, the finding that patients with LEP are more likely to die from sepsis remains an important one, recognizing the need for further studies including multicenter investigations.

In this study, we showed that LEP was associated with sepsis mortality across nearly all races in our cohort. While Asian race was associated with a higher unadjusted odds of death compared to White race, this was attenuated after adjusting for LEP. This may suggest that some of the racial disparities in sepsis identified in prior studies were in fact mediated by language proficiency. Further studies will be required to explore the causal nature of this novel association. If modifiable factors are identified, this could represent a potential target for future quality improvement initiatives aimed at improving sepsis outcomes.

Disclaimer

The contents are solely the responsibility of the authors and do not necessarily represent the official views of the University of California, San Francisco or the National Institutes of Health.

Sepsis is defined as a life-threatening organ dysfunction that occurs in response to systemic infection.^1,2 It is frequently fatal, common in hospital medicine, and a leading contributor to critical illness, morbidity, and healthcare expenditures.^2-5 While sepsis care and outcomes have improved in the past decade,^6,7 inpatient mortality remains high.⁸

A number of studies have sought to determine whether race plays a role in sepsis mortality. While Black patients with sepsis have frequently been identified as having the highest rates of death,^9-14 similar observations have been made for most non-White races/ethnicities.^13-15 Studies have also demonstrated higher rates of hospital-acquired infections among Asian and Latino patients.¹⁶

There are several possible explanations for why racial minorities experience disparate outcomes in sepsis, including access to care, comorbidities, implicit biases, and biological or environmental factors,^17-20 as well as characteristics of hospitals most likely to care for racial minorities.^13,15,21 One explanation that has not been explored is that racial disparities in sepsis are mediated by language. Limited English proficiency (LEP) has previously been associated with increased rates of adverse hospital events,²² longer length of stay,²³ and greater likelihood of readmission.²⁴ LEP has also been shown to represent a significant barrier to accessing healthcare and preventive screening.²⁵ The role of LEP in sepsis mortality, however, has yet to be examined.

The diverse patient population at the University of California, San Francisco (UCSF) provides a unique opportunity to build upon existing literature by further exploring racial differences in sepsis, specifically by investigating the role of LEP. The objective of this study was to determine the association between LEP and inpatient mortality among adults hospitalized with sepsis.

Setting

The study was conducted at the University of California, San Francisco, California (UCSF), an 800-bed tertiary care, academic medical center. It was approved by the UCSF Institutional Review Board with waiver of informed consent. UCSF cares for a population of patients who are racially and linguistically diverse, with high proportions of patients of East Asian descent and with LEP. According to recent United States census estimates, more than half of San Francisco County residents identify as non-White (35% Asians, 15% Hispanic/Latino, 6% Black), and 44% report speaking a language other than English at home.²⁶

Study Population and Data Collection

The UCSF Medical Center uses the electronic health record (EHR) Epic (Epic 2017, Epic Systems Corporation, Verona, Wisconsin). We obtained computerized EHR data from Clarity, the relational database that stores Epic’s inpatient data in thousands of tables. We identified all patients ≥18 years of age presenting to the emergency department (ED) between June 1, 2012 and December 31, 2016 with suspected serious infection, defined as having blood cultures ordered within 72 hours of ED presentation (N = 25,441). Patients who did not receive at least two doses of intravenous (IV) antibiotics within 48 hours were excluded, as they were unlikely to have serious infections.

We defined sepsis based on Sepsis-3 consensus guidelines² as a change in sequential [sepsis-related] organ failure assessment (SOFA) score ≥2 within the first 48 hours of ED presentation. The SOFA score is comprised of six variables representing different organ systems, each rated 0-4 based on the degree of dysfunction.² Patient vital signs, laboratory data, vasopressor medication doses, and ventilator settings were used to determine the exact timestamp at which each patient attained a change in SOFA score ≥2. Missing values were considered to be normal. To adjust for baseline organ dysfunction, SOFA elements associated with elevated bilirubin and/or creatinine were excluded for patients with chronic liver/kidney disease based on Elixhauser comorbidities.²⁷ We chose to focus on the first 48 hours in an attempt to capture patients with the most severe illnesses and the highest probability of true sepsis.

All primary and secondary International Classification of Diseases (ICD)-9/10 diagnosis codes were extracted from Clarity coding tables at the time of hospital discharge. Diagnosis codes signifying bacterial infection were grouped into the following categories based on type/location: pneumonia; bacteremia; urinary tract infection; and skin and soft tissue infection. All remaining diagnostic codes indicating bacterial infections at other sites were categorized as “Other”. If no codes indicating infection were present, patients were categorized as “None coded”. Patients with discharge diagnosis codes of “sepsis” were also identified. Dates and times of antibiotic administrations were obtained from the medications table. Time to first antibiotic was defined as the time in minutes from ED presentation to initiation of the first IV antibacterial medication. This variable was transformed using a natural log transformation based on best fit for normal distribution.

We limited our analyses to 8,974 patients who were diagnosed with sepsis as defined above and had either (1) ≥4 qualifying antibiotic days (QADs) or (2) an ICD-9/10 discharge diagnosis code of “sepsis” (Figure). QADs were defined based on the recent publication by Rhee et al. as having received four or more consecutive days of antibiotics, with the first dose given IV within 48 hours of presentation.²⁸ Patients who died or were discharged to hospice prior to the 4th QAD were also included. These additional parameters were added to increase specificity of the study sample for patients with true sepsis. Patients admitted to all levels of care (acute care, transitional care unit [TCU], intensive care unit [ICU]) and under all hospital services were included. There were no missing data for mortality, race, or language. We chose to focus on patients with sepsis in this initial study as this is a common diagnosis in hospital medicine that is enriched for high mortality.

Primary Outcome

The primary outcome of the study was inpatient mortality, which was obtained from the hospital encounters table in Clarity.

Primary Predictors

The primary predictor of interest was LEP. The encounter numbers from the dataset were used to link to self-reported demographic data, including “preferred language” and need for interpreter services. A manual chart review of 60 patients speaking the top six languages was conducted to verify the accuracy of the data on language and interpreter use (KNK). Defining the gold standard for LEP as having any chart note indicating non-English language and/or that an interpreter was used, the “interpreter needed” variable in Epic was found to have a positive predictive value for LEP of 100%. Therefore, patients in the study cohort were defined as having LEP if they met both of the following criteria: (1) a self-reported “preferred language” other than English and (2) having the “interpreter needed” variable indicating “yes”.

Covariate Data Collection

Additional data were obtained from the demographics tables, including age, race, sex, and insurance status. Race and ethnicity were combined into a single five-category variable including White, Asian, Black, Latino, and Other. This approach has been suggested as the best way to operationalize these variables²⁹ and has been utilized by similar studies in the literature.^9,14,15 We considered the Asian race to include all people of East Asian, Southeast Asian, or South Asian descent, which is consistent with the United States Census Bureau definition.³⁰ Patients identifying as Native Hawaiians/Pacific Islanders, Native Americans/Alaskan Natives, as well as those with unspecified race or ethnicity, were categorized as Other. Insurance status was categorized as Commercial, Medicare, Medicaid, or Other.

We estimated illness severity in several ways. First, the total qualifying SOFA score was calculated for each patient, which was defined as the total score achieved at the time that SOFA criteria were first met (≥2, within 48 hours). Second, we dichotomized patients based on whether they had received mechanical ventilation at any point during hospitalization. Finally, we used admission location as a surrogate marker for severity at the time of initial hospitalization.

To estimate the burden of baseline comorbidities, we calculated the van Walraven score (VWS),³¹ a validated modification of the Elixhauser Comorbidity Index.²⁷ This score conveys an estimated risk of in hospital death based on ICD-9/10 diagnosis codes for preexisting conditions, which ranges from <1% for the minimum score of –19 to >99% for the maximum score of 89.

Statistical Analyses

All statistical analyses were performed using Stata software version 15 (StataCorp LLC, College Station, Texas). Baseline demographics and patient characteristics were stratified by LEP. These were compared using two-sample t-tests or chi-squared tests of significance. Wilcoxon rank-sum tests were used for non-normally distributed variables. Inpatient mortality was compared across all races stratified by LEP using chi-squared tests of significance.

We fit a series of multivariable logistic regression models to examine the association between race and inpatient mortality adjusting for LEP and other patient/clinical characteristics. We first examined the unadjusted association between mortality and race; then adjusted for LEP alone; and finally adjusted for all covariates of interest, including LEP, age, sex, insurance status, year, admission level of care, VWS, total qualifying SOFA score, need for mechanical ventilation, site of infection, and time to first IV antibiotic. A subgroup analysis was also performed using the fully adjusted model restricted to patients who were mechanically ventilated. This population was selected because the patients (1) have among the highest severity of illness and (2) share a common barrier to communication, regardless of English proficiency.

Several potential interactions between LEP with other covariates were explored, including age, race, ICU admission level of care, and need for mechanical ventilation. Lastly, a mediation analysis was performed based on Baron & Kenny’s four-step model³² in order to calculate the proportion of the association between race and mortality explained by the proposed mediator (LEP).

To evaluate for the likelihood of residual confounding, we calculated an E-value, which is defined as the minimum strength of association that an unmeasured confounder would need to have with both the predictor and outcome variables, above and beyond the measured covariates, in order to fully explain away an observed predictor-outcome association.^33,34

We identified 8,974 patients hospitalized with sepsis based on the above inclusion criteria. This represented a medically complex, racially and linguistically diverse population (Table 1). The cohort was comprised of 24% Asian, 12% Black, and 11% Latino patients. Among those categorized as Other race, Native Americans/Alaskan Natives and Native Hawaiians/Pacific Islanders accounted for 4% (n = 31) and 21% (n = 159), respectively. A fifth of all patients had LEP (n = 1,716), 62% of whom were Asian (n = 1,064). Patients with LEP tended to be older, female, and to have a greater number of comorbid conditions (Table 1). The total qualifying SOFA score was also higher among patients with LEP (median 5; interquartile range [IQR]: 4-8 vs 5; IQR: 3-7; P <.001), though there was no association between LEP and mechanical ventilation (P = .22). The prevalence of LEP differed significantly across races, with 50% LEP among Asians, 32% among Latinos, 5% among White patients (P < .001). Only eight Black patients had LEP. More than 40 unique languages were represented in the cohort, with English, Cantonese, Spanish, Russian, and Mandarin accounting for ~95% (Appendix Table 1). Among Latino patients, 63% spoke English and 36% spoke Spanish.

In-hospital mortality was significantly higher among patients who had LEP (n = 268/1,716, 16%) compared to non-LEP patients (n = 678/7,258, 9%), with 80% greater unadjusted odds of mortality (OR 1.80; 95% CI: 1.54-2.09; P < .001). Notably we also found that Asian race was associated with a 1.57 unadjusted odds of mortality compared to White race (95% CI: 1.34-1.85; P < .001). Age, VWS, total qualifying SOFA score, mechanical ventilation, and admission level of care all exhibited a positive dose-response association with mortality (Appendix Table 2). In unadjusted analyses, there was no evidence of interaction between LEP and age (P = .38), LEP and race (P = .45), LEP and ICU admission level of care (P = .31), or LEP and mechanical ventilation (P = .19). Asian patients had the highest overall mortality (14% total, 17% with LEP). LEP was associated with increased unadjusted mortality among White, Asian, and Other races compared to their non-LEP counterparts (Appendix Figure 1). There was no significant difference in mortality between Latino patients with and without LEP. The sample size for Black patients with LEP (n = 8) was too small to draw conclusions about mortality.

Following multivariable logistic regression modeling for the association between race and mortality, we found that the increased odds of death among Asian patients was partially attenuated after adjusting for LEP (odds ratio [OR] 1.23, 95% CI: 1.02-1.48; P = .03; Table 2). Meanwhile, LEP was associated with a 1.66 odds of mortality (95% CI: 1.38-1.99; P < .001) after adjustment for race. In the full multivariable model adjusting for demographics and clinical characteristics, illness severity, and comorbidities, LEP was associated with a 31% increase in the odds of mortality compared to non-LEP (95% CI: 1.06-1.63; P = .02). In this model, the association between Asian race and mortality was now fully attenuated, with a point estimate near 1.0 (OR 0.98; 95% CI: 0.79-1.22; P = .87). Markers of illness severity, including total qualifying SOFA score (OR 1.23; 95% CI: 1.20-1.27; P < .001) and need for mechanical ventilation (OR 1.88; 95% CI: 1.52-2.33; P < .001), were both associated with greater odds of death. Based on a four-step mediation analysis, LEP was found to be a partial mediator to the association between Asian race and mortality (76% proportion explained). The E-value for the association between LEP and mortality was 1.95, with an E-value for the corresponding confidence interval of 1.29.

At a single US academic medical center serving a diverse population, we found that LEP was associated with sepsis mortality across all races except Black and Latino, conveying a 31% increase in the odds of death after adjusting for illness severity, comorbidities, and baseline characteristics. The higher mortality among Asian patients was largely mediated by LEP (76% proportion explained). While previous studies have variably found Black, Asian, Latino, and other non-White races/ethnicities to be at an increased risk of death from sepsis,^9-15 LEP has not been previously evaluated as a mediator of sepsis mortality. We were uniquely suited to uncover such an association due to the racial and linguistic diversity of our patient population. LEP has previously been implicated in poor health outcomes among hospitalized patients in general.^22-24 Future studies will be necessary to determine whether similar associations between LEP and mortality are observed among broader patient populations outside of sepsis.

There are a number of possible explanations for how LEP could mediate the association between race and mortality. First, LEP is known to be associated with greater difficulties in accessing medical care,²⁵ which could result in poorer baseline control of chronic comorbid conditions, fewer opportunities for preventive screening, and greater reluctance to seek medical attention when ill, theoretically leading to more severe presentations and worse outcomes. Indeed, LEP patients in our cohort had both a shorter median time to receiving their first antibiotic, as well as a higher total qualifying SOFA score, both of which may suggest more severe initial presentations. LEP is also known to contribute to, or exacerbate, the impact of low health literacy, which is itself associated with poor health.³⁵ Second, implicit biases may also have been present, as they are known to be common among healthcare providers and have been shown to negatively impact patient care.³⁶

Finally,it is possible that the association is related to the language barrier itself, which impacts providers’ ability to take an appropriate clinical history, and can lead to clinical errors or delays in care.³⁷ The fact that the association between LEP and mortality was eliminated when the analysis was restricted to mechanically ventilated patients seems to support this, since differences in language proficiency become irrelevant in this subgroup. While we are unable to comment on causality based on this observational study, we included a directed acyclic graph (DAG) in the supplemental materials, which shows one proposed model for describing these associations (Appendix Figure 2).

Assuming that the language barrier itself does, at least in part, drive the observed association, LEP represents a potentially modifiable risk factor that could be a target for quality improvement interventions. There is evidence that the use of medical interpreters among patients with LEP leads to greater satisfaction, fewer errors, and improved clinical outcomes;³⁸ however, several recent studies have documented underutilization of professional interpreter services, even when readily available.^39,40 At our institution, phone and video interpreter services are available 24/7 for approximately 150 languages. Due to limitations inherent to the EHR, we were unable to ascertain the extent to which these services were used in the present study. Heavy clinical workloads, connectivity issues, and missing or faulty equipment represent theoretical barriers to utilization of these services.

There are some limitations to our study. First, by utilizing a large database of electronic data, the quality of our analyses was reliant on the accuracy of the EHR. Demographic data such as language may have been subject to misclassification due to self-reporting. We attempted to minimize this by also including the need for interpreter services within the definition of LEP, which was validated by manual chart review. Second, generalizability is limited in this single-center study conducted at an institution with unique demographics, wherein nearly two-thirds of the LEP patients were Asian, and the Chinese-speaking population outnumbered those who speak Spanish.

Finally, the most important limitation to our study is the potential for residual confounding. While we attempted to mitigate this by adjusting for as many clinically relevant covariates as possible, there may still be unmeasured confounders to the association between LEP and mortality, such as access to outpatient care, functional status, interpreter use, and other markers of illness severity like the number and type of supportive therapies received. Based on our E-value calculations, with an observed OR of 1.31 for the association between LEP and mortality, an unmeasured confounder with an OR of 1.95 would fully explain away this association, while an OR of 1.29 would shift the confidence interval to include the null. These values suggest at least some risk of residual confounding. The fact that our fully adjusted model included multiple covariates, including several markers of illness severity, does somewhat lessen the likelihood of a confounder achieving these values, since they represent the minimum strength of an unmeasured confounder above and beyond the measured covariates. Regardless, the finding that patients with LEP are more likely to die from sepsis remains an important one, recognizing the need for further studies including multicenter investigations.

In this study, we showed that LEP was associated with sepsis mortality across nearly all races in our cohort. While Asian race was associated with a higher unadjusted odds of death compared to White race, this was attenuated after adjusting for LEP. This may suggest that some of the racial disparities in sepsis identified in prior studies were in fact mediated by language proficiency. Further studies will be required to explore the causal nature of this novel association. If modifiable factors are identified, this could represent a potential target for future quality improvement initiatives aimed at improving sepsis outcomes.

Disclaimer

The contents are solely the responsibility of the authors and do not necessarily represent the official views of the University of California, San Francisco or the National Institutes of Health.

Sepsis is defined as a life-threatening organ dysfunction that occurs in response to systemic infection.^1,2 It is frequently fatal, common in hospital medicine, and a leading contributor to critical illness, morbidity, and healthcare expenditures.^2-5 While sepsis care and outcomes have improved in the past decade,^6,7 inpatient mortality remains high.⁸

A number of studies have sought to determine whether race plays a role in sepsis mortality. While Black patients with sepsis have frequently been identified as having the highest rates of death,^9-14 similar observations have been made for most non-White races/ethnicities.^13-15 Studies have also demonstrated higher rates of hospital-acquired infections among Asian and Latino patients.¹⁶

There are several possible explanations for why racial minorities experience disparate outcomes in sepsis, including access to care, comorbidities, implicit biases, and biological or environmental factors,^17-20 as well as characteristics of hospitals most likely to care for racial minorities.^13,15,21 One explanation that has not been explored is that racial disparities in sepsis are mediated by language. Limited English proficiency (LEP) has previously been associated with increased rates of adverse hospital events,²² longer length of stay,²³ and greater likelihood of readmission.²⁴ LEP has also been shown to represent a significant barrier to accessing healthcare and preventive screening.²⁵ The role of LEP in sepsis mortality, however, has yet to be examined.

The diverse patient population at the University of California, San Francisco (UCSF) provides a unique opportunity to build upon existing literature by further exploring racial differences in sepsis, specifically by investigating the role of LEP. The objective of this study was to determine the association between LEP and inpatient mortality among adults hospitalized with sepsis.

Setting

The study was conducted at the University of California, San Francisco, California (UCSF), an 800-bed tertiary care, academic medical center. It was approved by the UCSF Institutional Review Board with waiver of informed consent. UCSF cares for a population of patients who are racially and linguistically diverse, with high proportions of patients of East Asian descent and with LEP. According to recent United States census estimates, more than half of San Francisco County residents identify as non-White (35% Asians, 15% Hispanic/Latino, 6% Black), and 44% report speaking a language other than English at home.²⁶

Study Population and Data Collection

The UCSF Medical Center uses the electronic health record (EHR) Epic (Epic 2017, Epic Systems Corporation, Verona, Wisconsin). We obtained computerized EHR data from Clarity, the relational database that stores Epic’s inpatient data in thousands of tables. We identified all patients ≥18 years of age presenting to the emergency department (ED) between June 1, 2012 and December 31, 2016 with suspected serious infection, defined as having blood cultures ordered within 72 hours of ED presentation (N = 25,441). Patients who did not receive at least two doses of intravenous (IV) antibiotics within 48 hours were excluded, as they were unlikely to have serious infections.

We defined sepsis based on Sepsis-3 consensus guidelines² as a change in sequential [sepsis-related] organ failure assessment (SOFA) score ≥2 within the first 48 hours of ED presentation. The SOFA score is comprised of six variables representing different organ systems, each rated 0-4 based on the degree of dysfunction.² Patient vital signs, laboratory data, vasopressor medication doses, and ventilator settings were used to determine the exact timestamp at which each patient attained a change in SOFA score ≥2. Missing values were considered to be normal. To adjust for baseline organ dysfunction, SOFA elements associated with elevated bilirubin and/or creatinine were excluded for patients with chronic liver/kidney disease based on Elixhauser comorbidities.²⁷ We chose to focus on the first 48 hours in an attempt to capture patients with the most severe illnesses and the highest probability of true sepsis.

All primary and secondary International Classification of Diseases (ICD)-9/10 diagnosis codes were extracted from Clarity coding tables at the time of hospital discharge. Diagnosis codes signifying bacterial infection were grouped into the following categories based on type/location: pneumonia; bacteremia; urinary tract infection; and skin and soft tissue infection. All remaining diagnostic codes indicating bacterial infections at other sites were categorized as “Other”. If no codes indicating infection were present, patients were categorized as “None coded”. Patients with discharge diagnosis codes of “sepsis” were also identified. Dates and times of antibiotic administrations were obtained from the medications table. Time to first antibiotic was defined as the time in minutes from ED presentation to initiation of the first IV antibacterial medication. This variable was transformed using a natural log transformation based on best fit for normal distribution.

We limited our analyses to 8,974 patients who were diagnosed with sepsis as defined above and had either (1) ≥4 qualifying antibiotic days (QADs) or (2) an ICD-9/10 discharge diagnosis code of “sepsis” (Figure). QADs were defined based on the recent publication by Rhee et al. as having received four or more consecutive days of antibiotics, with the first dose given IV within 48 hours of presentation.²⁸ Patients who died or were discharged to hospice prior to the 4th QAD were also included. These additional parameters were added to increase specificity of the study sample for patients with true sepsis. Patients admitted to all levels of care (acute care, transitional care unit [TCU], intensive care unit [ICU]) and under all hospital services were included. There were no missing data for mortality, race, or language. We chose to focus on patients with sepsis in this initial study as this is a common diagnosis in hospital medicine that is enriched for high mortality.

Primary Outcome

The primary outcome of the study was inpatient mortality, which was obtained from the hospital encounters table in Clarity.

Primary Predictors

The primary predictor of interest was LEP. The encounter numbers from the dataset were used to link to self-reported demographic data, including “preferred language” and need for interpreter services. A manual chart review of 60 patients speaking the top six languages was conducted to verify the accuracy of the data on language and interpreter use (KNK). Defining the gold standard for LEP as having any chart note indicating non-English language and/or that an interpreter was used, the “interpreter needed” variable in Epic was found to have a positive predictive value for LEP of 100%. Therefore, patients in the study cohort were defined as having LEP if they met both of the following criteria: (1) a self-reported “preferred language” other than English and (2) having the “interpreter needed” variable indicating “yes”.

Covariate Data Collection

Additional data were obtained from the demographics tables, including age, race, sex, and insurance status. Race and ethnicity were combined into a single five-category variable including White, Asian, Black, Latino, and Other. This approach has been suggested as the best way to operationalize these variables²⁹ and has been utilized by similar studies in the literature.^9,14,15 We considered the Asian race to include all people of East Asian, Southeast Asian, or South Asian descent, which is consistent with the United States Census Bureau definition.³⁰ Patients identifying as Native Hawaiians/Pacific Islanders, Native Americans/Alaskan Natives, as well as those with unspecified race or ethnicity, were categorized as Other. Insurance status was categorized as Commercial, Medicare, Medicaid, or Other.

We estimated illness severity in several ways. First, the total qualifying SOFA score was calculated for each patient, which was defined as the total score achieved at the time that SOFA criteria were first met (≥2, within 48 hours). Second, we dichotomized patients based on whether they had received mechanical ventilation at any point during hospitalization. Finally, we used admission location as a surrogate marker for severity at the time of initial hospitalization.

To estimate the burden of baseline comorbidities, we calculated the van Walraven score (VWS),³¹ a validated modification of the Elixhauser Comorbidity Index.²⁷ This score conveys an estimated risk of in hospital death based on ICD-9/10 diagnosis codes for preexisting conditions, which ranges from <1% for the minimum score of –19 to >99% for the maximum score of 89.

Statistical Analyses

All statistical analyses were performed using Stata software version 15 (StataCorp LLC, College Station, Texas). Baseline demographics and patient characteristics were stratified by LEP. These were compared using two-sample t-tests or chi-squared tests of significance. Wilcoxon rank-sum tests were used for non-normally distributed variables. Inpatient mortality was compared across all races stratified by LEP using chi-squared tests of significance.

We fit a series of multivariable logistic regression models to examine the association between race and inpatient mortality adjusting for LEP and other patient/clinical characteristics. We first examined the unadjusted association between mortality and race; then adjusted for LEP alone; and finally adjusted for all covariates of interest, including LEP, age, sex, insurance status, year, admission level of care, VWS, total qualifying SOFA score, need for mechanical ventilation, site of infection, and time to first IV antibiotic. A subgroup analysis was also performed using the fully adjusted model restricted to patients who were mechanically ventilated. This population was selected because the patients (1) have among the highest severity of illness and (2) share a common barrier to communication, regardless of English proficiency.

Several potential interactions between LEP with other covariates were explored, including age, race, ICU admission level of care, and need for mechanical ventilation. Lastly, a mediation analysis was performed based on Baron & Kenny’s four-step model³² in order to calculate the proportion of the association between race and mortality explained by the proposed mediator (LEP).

To evaluate for the likelihood of residual confounding, we calculated an E-value, which is defined as the minimum strength of association that an unmeasured confounder would need to have with both the predictor and outcome variables, above and beyond the measured covariates, in order to fully explain away an observed predictor-outcome association.^33,34

We identified 8,974 patients hospitalized with sepsis based on the above inclusion criteria. This represented a medically complex, racially and linguistically diverse population (Table 1). The cohort was comprised of 24% Asian, 12% Black, and 11% Latino patients. Among those categorized as Other race, Native Americans/Alaskan Natives and Native Hawaiians/Pacific Islanders accounted for 4% (n = 31) and 21% (n = 159), respectively. A fifth of all patients had LEP (n = 1,716), 62% of whom were Asian (n = 1,064). Patients with LEP tended to be older, female, and to have a greater number of comorbid conditions (Table 1). The total qualifying SOFA score was also higher among patients with LEP (median 5; interquartile range [IQR]: 4-8 vs 5; IQR: 3-7; P <.001), though there was no association between LEP and mechanical ventilation (P = .22). The prevalence of LEP differed significantly across races, with 50% LEP among Asians, 32% among Latinos, 5% among White patients (P < .001). Only eight Black patients had LEP. More than 40 unique languages were represented in the cohort, with English, Cantonese, Spanish, Russian, and Mandarin accounting for ~95% (Appendix Table 1). Among Latino patients, 63% spoke English and 36% spoke Spanish.

In-hospital mortality was significantly higher among patients who had LEP (n = 268/1,716, 16%) compared to non-LEP patients (n = 678/7,258, 9%), with 80% greater unadjusted odds of mortality (OR 1.80; 95% CI: 1.54-2.09; P < .001). Notably we also found that Asian race was associated with a 1.57 unadjusted odds of mortality compared to White race (95% CI: 1.34-1.85; P < .001). Age, VWS, total qualifying SOFA score, mechanical ventilation, and admission level of care all exhibited a positive dose-response association with mortality (Appendix Table 2). In unadjusted analyses, there was no evidence of interaction between LEP and age (P = .38), LEP and race (P = .45), LEP and ICU admission level of care (P = .31), or LEP and mechanical ventilation (P = .19). Asian patients had the highest overall mortality (14% total, 17% with LEP). LEP was associated with increased unadjusted mortality among White, Asian, and Other races compared to their non-LEP counterparts (Appendix Figure 1). There was no significant difference in mortality between Latino patients with and without LEP. The sample size for Black patients with LEP (n = 8) was too small to draw conclusions about mortality.

Following multivariable logistic regression modeling for the association between race and mortality, we found that the increased odds of death among Asian patients was partially attenuated after adjusting for LEP (odds ratio [OR] 1.23, 95% CI: 1.02-1.48; P = .03; Table 2). Meanwhile, LEP was associated with a 1.66 odds of mortality (95% CI: 1.38-1.99; P < .001) after adjustment for race. In the full multivariable model adjusting for demographics and clinical characteristics, illness severity, and comorbidities, LEP was associated with a 31% increase in the odds of mortality compared to non-LEP (95% CI: 1.06-1.63; P = .02). In this model, the association between Asian race and mortality was now fully attenuated, with a point estimate near 1.0 (OR 0.98; 95% CI: 0.79-1.22; P = .87). Markers of illness severity, including total qualifying SOFA score (OR 1.23; 95% CI: 1.20-1.27; P < .001) and need for mechanical ventilation (OR 1.88; 95% CI: 1.52-2.33; P < .001), were both associated with greater odds of death. Based on a four-step mediation analysis, LEP was found to be a partial mediator to the association between Asian race and mortality (76% proportion explained). The E-value for the association between LEP and mortality was 1.95, with an E-value for the corresponding confidence interval of 1.29.

At a single US academic medical center serving a diverse population, we found that LEP was associated with sepsis mortality across all races except Black and Latino, conveying a 31% increase in the odds of death after adjusting for illness severity, comorbidities, and baseline characteristics. The higher mortality among Asian patients was largely mediated by LEP (76% proportion explained). While previous studies have variably found Black, Asian, Latino, and other non-White races/ethnicities to be at an increased risk of death from sepsis,^9-15 LEP has not been previously evaluated as a mediator of sepsis mortality. We were uniquely suited to uncover such an association due to the racial and linguistic diversity of our patient population. LEP has previously been implicated in poor health outcomes among hospitalized patients in general.^22-24 Future studies will be necessary to determine whether similar associations between LEP and mortality are observed among broader patient populations outside of sepsis.

There are a number of possible explanations for how LEP could mediate the association between race and mortality. First, LEP is known to be associated with greater difficulties in accessing medical care,²⁵ which could result in poorer baseline control of chronic comorbid conditions, fewer opportunities for preventive screening, and greater reluctance to seek medical attention when ill, theoretically leading to more severe presentations and worse outcomes. Indeed, LEP patients in our cohort had both a shorter median time to receiving their first antibiotic, as well as a higher total qualifying SOFA score, both of which may suggest more severe initial presentations. LEP is also known to contribute to, or exacerbate, the impact of low health literacy, which is itself associated with poor health.³⁵ Second, implicit biases may also have been present, as they are known to be common among healthcare providers and have been shown to negatively impact patient care.³⁶

Finally,it is possible that the association is related to the language barrier itself, which impacts providers’ ability to take an appropriate clinical history, and can lead to clinical errors or delays in care.³⁷ The fact that the association between LEP and mortality was eliminated when the analysis was restricted to mechanically ventilated patients seems to support this, since differences in language proficiency become irrelevant in this subgroup. While we are unable to comment on causality based on this observational study, we included a directed acyclic graph (DAG) in the supplemental materials, which shows one proposed model for describing these associations (Appendix Figure 2).

Assuming that the language barrier itself does, at least in part, drive the observed association, LEP represents a potentially modifiable risk factor that could be a target for quality improvement interventions. There is evidence that the use of medical interpreters among patients with LEP leads to greater satisfaction, fewer errors, and improved clinical outcomes;³⁸ however, several recent studies have documented underutilization of professional interpreter services, even when readily available.^39,40 At our institution, phone and video interpreter services are available 24/7 for approximately 150 languages. Due to limitations inherent to the EHR, we were unable to ascertain the extent to which these services were used in the present study. Heavy clinical workloads, connectivity issues, and missing or faulty equipment represent theoretical barriers to utilization of these services.

There are some limitations to our study. First, by utilizing a large database of electronic data, the quality of our analyses was reliant on the accuracy of the EHR. Demographic data such as language may have been subject to misclassification due to self-reporting. We attempted to minimize this by also including the need for interpreter services within the definition of LEP, which was validated by manual chart review. Second, generalizability is limited in this single-center study conducted at an institution with unique demographics, wherein nearly two-thirds of the LEP patients were Asian, and the Chinese-speaking population outnumbered those who speak Spanish.

Finally, the most important limitation to our study is the potential for residual confounding. While we attempted to mitigate this by adjusting for as many clinically relevant covariates as possible, there may still be unmeasured confounders to the association between LEP and mortality, such as access to outpatient care, functional status, interpreter use, and other markers of illness severity like the number and type of supportive therapies received. Based on our E-value calculations, with an observed OR of 1.31 for the association between LEP and mortality, an unmeasured confounder with an OR of 1.95 would fully explain away this association, while an OR of 1.29 would shift the confidence interval to include the null. These values suggest at least some risk of residual confounding. The fact that our fully adjusted model included multiple covariates, including several markers of illness severity, does somewhat lessen the likelihood of a confounder achieving these values, since they represent the minimum strength of an unmeasured confounder above and beyond the measured covariates. Regardless, the finding that patients with LEP are more likely to die from sepsis remains an important one, recognizing the need for further studies including multicenter investigations.

In this study, we showed that LEP was associated with sepsis mortality across nearly all races in our cohort. While Asian race was associated with a higher unadjusted odds of death compared to White race, this was attenuated after adjusting for LEP. This may suggest that some of the racial disparities in sepsis identified in prior studies were in fact mediated by language proficiency. Further studies will be required to explore the causal nature of this novel association. If modifiable factors are identified, this could represent a potential target for future quality improvement initiatives aimed at improving sepsis outcomes.

Disclaimer

The contents are solely the responsibility of the authors and do not necessarily represent the official views of the University of California, San Francisco or the National Institutes of Health.