Amanda C. Schondelmeyer, MD

Fever is one of the most common reasons for emergency department (ED) evaluation of infants under 90 days of age.[1] Up to 10% to 20% of febrile young infants will have a serious bacterial infection (SBI),[2, 3, 4] but infants with SBI are difficult to distinguish from those without SBI based upon symptoms and physical examination findings alone.[5] Previously developed clinical prediction algorithms can help to identify febrile infants at low risk for SBI, but differ in age range as well as recommendations for testing and empiric treatment.[6, 7, 8] Consequently, there is widespread variation in management of febrile young infants at US children's hospitals,[9, 10, 11] and defining optimal management strategies remains an important issue in pediatric healthcare.[12] Administrative datasets are convenient and inexpensive, and can be used to evaluate practice variation, trends, and outcomes of a large, diverse group of patients within and across institutions.[9, 10] Accurately identifying febrile infants evaluated for suspected SBI in administrative databases would facilitate comparative effectiveness research, quality improvement initiatives, and institutional benchmarking.

Prior studies have validated the accuracy of administrative billing codes for identification of other common childhood illnesses, including urinary tract infection (UTI)[13] and pneumonia.[14] The accuracy of International Classification of Diseases, Ninth Revision (ICD‐9) diagnosis codes in identifying febrile young infants evaluated for SBI is not known. Reliance on administrative ICD‐9 diagnosis codes for patient identification can lead to misclassification of patients due to variable database quality, the validity of the diagnosis codes being utilized, and hospital coding practices.[15] Additionally, fever is a symptom and not a specific diagnosis. If a particular bacterial or viral diagnosis is established (eg, enterovirus meningitis), a discharge diagnosis of fever may not be attributed to the patient encounter. Thus, evaluating the performance characteristics and capture of clinical outcomes of different combinations of ICD‐9 diagnosis codes for identifying febrile infants is necessary for both the conduct and interpretation of studies that utilize administrative databases. The primary objective of this investigation was to identify the most accurate ICD‐9 coding strategies for the identification of febrile infants aged <90 days using administrative data. We also sought to evaluate capture of clinically important outcomes across identification strategies.

METHODS

ICD‐9 Diagnosis Code Groups

In the PHIS database, all patients discharged from the hospital (including hospitalized patients as well as patients discharged from the ED) receive 1 or more ICD‐9 discharge diagnosis codes. These diagnosis codes are ascribed after discharge from the hospital, or for ED patients, after ED discharge. Additionally, patients may receive an admission diagnosis, which reflects the diagnosis ascribed at the time of ED discharge or transfer to the inpatient unit.

We reviewed medical records of infants selected from the following ICD‐9 diagnosis code groups (Figure 1): (1) discharge diagnosis code of fever (780.6 [fever and other physiologic disturbances of temperature regulation], 778.4 [other disturbances of temperature regulation of newborn], 780.60 [fever, unspecified], or 780.61 [fever presenting with conditions classified elsewhere])[9, 10] regardless of the presence of admission diagnosis of fever or diagnosis of serious infection, (2) admission diagnosis code of fever without associated discharge diagnosis code of fever,[10] (3) discharge diagnosis code of serious infection determined a priori (see Supporting Information, Appendix, in the online version of this article) without discharge or admission diagnosis code of fever, and (4) infants without any diagnosis code of fever or serious infection.

Figure 1

Medical records reviewed in each of the 4 ICD‐9 diagnosis code groups were randomly selected from the overall set of ED encounters in the population of infants <90 days of age evaluated during the study period. Twenty‐five percent population sampling was used for 3 of the ICD‐9 diagnosis code groups, whereas 5% sampling was used for the no fever/no serious infection code group. The number of medical records reviewed in each ICD‐9 diagnosis code group was proportional to the distribution of ICD‐9 codes across the entire population of infants <90 days of age. These records were distributed equally across sites (228 records per site), except for 1 site that does not assign admission diagnoses (201 records).

Investigators were blinded to ICD‐9 diagnosis code groups during medical record review. Infants with multiple visits during the study period were eligible to be included more than once if the visits occurred more than 3 days apart. For infants with more than 1 ED visit on a particular calendar day, investigators were instructed to review the initial visit.

For each encounter, we also abstracted demographic characteristics (gender, race/ethnicity), insurance status, hospital region (using US Census categories[22]), and season from the PHIS database.

RESULTS

ICD‐9 Code Case‐Identification Algorithm Performance

Compared with the reference standard, the 4 case‐identification algorithms demonstrated specificity of 96.2% to 98.2% but lower sensitivity overall (Figure 2). Discharge diagnosis of fever alone demonstrated the lowest sensitivity. The algorithm of discharge or admission diagnosis of fever resulted in increased sensitivity and the highest PPV of all 4 algorithms (86.9%, 95% CI: 84.5‐89.3). Addition of serious infection codes to this algorithm resulted in a marginal increase in sensitivity and a similar decrease in PPV (Table 1). When limited to hospitalized infants, specificity was highest for the case‐identification algorithm of discharge diagnosis of fever and similarly high for discharge or admission diagnosis of fever; sensitivity was highest for the algorithm of discharge or admission diagnosis of fever or diagnosis of serious infection. For infants discharged from the ED, algorithm specificity was 97.7% to 98.4%, with lower sensitivity for all 4 algorithms (Table 1). Inclusion of the 47 infants with abnormal temperature as fever did not materially change algorithm performance (data not shown).

Figure 2

Across all 3 age groups (28 days, 2956 days, and 5789 days), the 4 case‐identification algorithms demonstrated specificity >96%, whereas algorithm sensitivity was highest in the 29‐ to 56‐days‐old age group and lowest among infants 57 to 89 days old across all 4 algorithms (Figure 2). Similar to the overall cohort, an algorithm of discharge or admission diagnosis of fever demonstrated specificity of nearly 98% in all age groups; addition of serious infection codes to this algorithm increased sensitivity, highest in the 29‐ to 56‐days‐old age group (Figure 2; see also Supporting Information, Table 2, in the online version of this article).

Table 2.Performance Characteristics of ICD‐9 Diagnosis Code Case‐Identification Algorithms Across the Eight Sites According to Reference Standard.*

ICD‐9 Diagnosis Code Algorithm	Sensitivity, Median % (Range)	Specificity, Median % (Range)	Negative Predictive Value, Median % (Range)	Positive Predictive Value, Median % (Range)
NOTE: Abbreviations: ICD‐9, International Classification of Diseases, Ninth Revision. *Reference standard of fever was defined by documented temperature 100.4F (38.0 C) on review of electronic medical record.
Discharge diagnosis of fever	56.2 (34.681.0)	98.3 (96.499.1)	92.1 (83.297.4)	87.7 (74.093.2)
Discharge or Admission diagnosis of Fever	76.7 (51.385.0)	97.8 (96.298.7)	95.6 (86.997.4)	87.4 (80.092.9)
Discharge diagnosis of fever or serious infection	68.3 (44.287.3)	96.5 (95.498.0)	93.6 (85.298.2)	78.3 (74.289.0)
Discharge or admission diagnosis of fever or serious infection	83.1 (58.390.7)	95.8 (95.498.0)	96.5 (88.598.2)	79.1 (77.490.4)

Across the 8 study sites, median specificity was 95.8% to 98.3% for the 4 algorithms, with little interhospital variability; however, algorithm sensitivity varied widely by site. Median PPV was highest for discharge diagnosis of fever alone at 87.7% but ranged from 74.0% to 93.2% across sites. Median PPV for an algorithm of discharge or admission diagnosis of fever was similar (87.4%) but with less variation by site (range 80.0%92.9%) (Table 2).

DISCUSSION

We described the performance of 4 ICD‐9 diagnosis code case‐identification algorithms for the identification of febrile young infants <90 days of age at US children's hospitals. Although the specificity was high across algorithms and institutions, the sensitivity was relatively low, particularly for discharge diagnosis of fever, and varied by institution. Given the high specificity, ICD‐9 diagnosis code case‐identification algorithms for fever reliably identify febrile infants using administrative data with low rates of inclusion of infants without fever. However, underidentification of patients, particularly those more prone to SBIs and severe outcomes depending on the algorithm utilized, can impact interpretation of comparative effectiveness studies or the quality of care delivered by an institution.

ICD‐9 discharge diagnosis codes are frequently used to identify pediatric patients across a variety of administrative databases, diseases, and symptoms.[19, 27, 28, 29, 30, 31] Although discharge diagnosis of fever is highly specific, sensitivity is substantially lower than other case‐identification algorithms we studied, particularly for hospitalized infants. This may be due to a fever code sometimes being omitted in favor of a more specific diagnosis (eg, bacteremia) prior to hospital discharge. Therefore, case identification relying only on ICD‐9 discharge diagnosis codes for fever may under‐report clinically important SBI or severe outcomes as demonstrated in our study. This is in contrast to ICD‐9 diagnosis code identification strategies for childhood UTI and pneumonia, which largely have higher sensitivity but lower specificity than fever codes.[13, 14]

Admission diagnosis of fever is important for febrile infants as they may not have an explicit diagnosis at the time of disposition from the ED. Addition of admission diagnosis of fever to an algorithm relying on discharge diagnosis code alone increased sensitivity without a demonstrable reduction in specificity and PPV, likely due to capture of infants with a fever diagnosis at presentation before a specific infection was identified. Although using an algorithm of discharge or admission diagnosis of fever captured a higher percentage of hospitalized febrile infants with SBIs, sensitivity was only 71% overall with this algorithm, and 43% of febrile infants with SBI would still have been missed. Importantly, though, addition of various ICD‐9 codes for serious infection to this algorithm resulted in capture of all febrile infants with SBI and should be used as a sensitivity analysis.

The test characteristics of diagnosis codes were highest in the 29‐ to 56‐days‐old age group. Given the differing low‐risk criteria[6, 7, 8] and lack of best practice guidelines[16] in this age group, the use of administrative data may allow for the comparison of testing and treatment strategies across a large cohort of febrile infants aged 29 to 56 days. However, individual hospital coding practices may affect algorithm performance, in particular sensitivity, which varied substantially by hospital. This variation in algorithm sensitivity may impact comparisons of outcomes across institutions. Therefore, when conducting studies of febrile infants using administrative data, sensitivity analyses or use of chart review should be considered to augment the use of ICD‐9 code‐based identification strategies, particularly for comparative benchmarking and outcomes studies. These additional analyses are particularly important for studies of febrile infants >56 days of age, in whom the sensitivity of diagnosis codes is particularly low. We speculate that the lower sensitivity in older febrile infants may relate to a lack of consensus on the clinical significance of fever in this age group and the varying management strategies employed.[10]

Strengths of this study include the assessment of ICD‐9 code algorithms across multiple institutions for identification of fever in young infants, and the patterns of our findings remained robust when comparing median performance characteristics of the algorithms across hospitals to our overall findings. We were also able to accurately estimate PPV and NPV using a case‐identification strategy weighted to the actual population sizes. Although sensitivity and specificity are the primary measures of test performance, predictive values are highly informative for investigators using administrative data. Additionally, our findings may inform public health efforts including disease surveillance, assessment of seasonal variation, and identification and monitoring of healthcare‐associated infections among febrile infants.

Our study has limitations. We did not review all identified records, which raises the possibility that our evaluated cohort may not be representative of the entire febrile infant population. We attempted to mitigate this possibility by using a random sampling strategy for our population selection that was weighted to the actual population sizes. Second, we identified serious infections using ICD‐9 diagnosis codes determined by group consensus, which may not capture all serious infection codes that identify febrile infants whose fever code was omitted. Third, 47 infants had abnormal temperature that did not meet our reference standard criteria for fever and were included in the no fever group. Although there may be disagreement regarding what constitutes a fever, we used a widely accepted reference standard to define fever.[16] Further, inclusion of these 47 infants as fever did not materially change algorithm performance. Last, our study was conducted at 8 large tertiary‐care children's hospitals, and our results may not be generalizable to other children's hospitals and community‐based hospitals.

CONCLUSIONS

Studies of febrile young infants that rely on ICD‐9 discharge diagnosis code of fever for case ascertainment have high specificity but low sensitivity for the identification of febrile infants, particularly among hospitalized patients. A case‐identification strategy that includes discharge or admission diagnosis of fever demonstrated higher sensitivity, and should be considered for studies of febrile infants using administrative data. However, additional strategies such as incorporation of ICD‐9 codes for serious infection should be used when comparing outcomes across institutions.

Fever is one of the most common reasons for emergency department (ED) evaluation of infants under 90 days of age.[1] Up to 10% to 20% of febrile young infants will have a serious bacterial infection (SBI),[2, 3, 4] but infants with SBI are difficult to distinguish from those without SBI based upon symptoms and physical examination findings alone.[5] Previously developed clinical prediction algorithms can help to identify febrile infants at low risk for SBI, but differ in age range as well as recommendations for testing and empiric treatment.[6, 7, 8] Consequently, there is widespread variation in management of febrile young infants at US children's hospitals,[9, 10, 11] and defining optimal management strategies remains an important issue in pediatric healthcare.[12] Administrative datasets are convenient and inexpensive, and can be used to evaluate practice variation, trends, and outcomes of a large, diverse group of patients within and across institutions.[9, 10] Accurately identifying febrile infants evaluated for suspected SBI in administrative databases would facilitate comparative effectiveness research, quality improvement initiatives, and institutional benchmarking.

Prior studies have validated the accuracy of administrative billing codes for identification of other common childhood illnesses, including urinary tract infection (UTI)[13] and pneumonia.[14] The accuracy of International Classification of Diseases, Ninth Revision (ICD‐9) diagnosis codes in identifying febrile young infants evaluated for SBI is not known. Reliance on administrative ICD‐9 diagnosis codes for patient identification can lead to misclassification of patients due to variable database quality, the validity of the diagnosis codes being utilized, and hospital coding practices.[15] Additionally, fever is a symptom and not a specific diagnosis. If a particular bacterial or viral diagnosis is established (eg, enterovirus meningitis), a discharge diagnosis of fever may not be attributed to the patient encounter. Thus, evaluating the performance characteristics and capture of clinical outcomes of different combinations of ICD‐9 diagnosis codes for identifying febrile infants is necessary for both the conduct and interpretation of studies that utilize administrative databases. The primary objective of this investigation was to identify the most accurate ICD‐9 coding strategies for the identification of febrile infants aged <90 days using administrative data. We also sought to evaluate capture of clinically important outcomes across identification strategies.

METHODS

ICD‐9 Diagnosis Code Groups

In the PHIS database, all patients discharged from the hospital (including hospitalized patients as well as patients discharged from the ED) receive 1 or more ICD‐9 discharge diagnosis codes. These diagnosis codes are ascribed after discharge from the hospital, or for ED patients, after ED discharge. Additionally, patients may receive an admission diagnosis, which reflects the diagnosis ascribed at the time of ED discharge or transfer to the inpatient unit.

We reviewed medical records of infants selected from the following ICD‐9 diagnosis code groups (Figure 1): (1) discharge diagnosis code of fever (780.6 [fever and other physiologic disturbances of temperature regulation], 778.4 [other disturbances of temperature regulation of newborn], 780.60 [fever, unspecified], or 780.61 [fever presenting with conditions classified elsewhere])[9, 10] regardless of the presence of admission diagnosis of fever or diagnosis of serious infection, (2) admission diagnosis code of fever without associated discharge diagnosis code of fever,[10] (3) discharge diagnosis code of serious infection determined a priori (see Supporting Information, Appendix, in the online version of this article) without discharge or admission diagnosis code of fever, and (4) infants without any diagnosis code of fever or serious infection.

Figure 1

Medical records reviewed in each of the 4 ICD‐9 diagnosis code groups were randomly selected from the overall set of ED encounters in the population of infants <90 days of age evaluated during the study period. Twenty‐five percent population sampling was used for 3 of the ICD‐9 diagnosis code groups, whereas 5% sampling was used for the no fever/no serious infection code group. The number of medical records reviewed in each ICD‐9 diagnosis code group was proportional to the distribution of ICD‐9 codes across the entire population of infants <90 days of age. These records were distributed equally across sites (228 records per site), except for 1 site that does not assign admission diagnoses (201 records).

Investigators were blinded to ICD‐9 diagnosis code groups during medical record review. Infants with multiple visits during the study period were eligible to be included more than once if the visits occurred more than 3 days apart. For infants with more than 1 ED visit on a particular calendar day, investigators were instructed to review the initial visit.

For each encounter, we also abstracted demographic characteristics (gender, race/ethnicity), insurance status, hospital region (using US Census categories[22]), and season from the PHIS database.

RESULTS

ICD‐9 Code Case‐Identification Algorithm Performance

Compared with the reference standard, the 4 case‐identification algorithms demonstrated specificity of 96.2% to 98.2% but lower sensitivity overall (Figure 2). Discharge diagnosis of fever alone demonstrated the lowest sensitivity. The algorithm of discharge or admission diagnosis of fever resulted in increased sensitivity and the highest PPV of all 4 algorithms (86.9%, 95% CI: 84.5‐89.3). Addition of serious infection codes to this algorithm resulted in a marginal increase in sensitivity and a similar decrease in PPV (Table 1). When limited to hospitalized infants, specificity was highest for the case‐identification algorithm of discharge diagnosis of fever and similarly high for discharge or admission diagnosis of fever; sensitivity was highest for the algorithm of discharge or admission diagnosis of fever or diagnosis of serious infection. For infants discharged from the ED, algorithm specificity was 97.7% to 98.4%, with lower sensitivity for all 4 algorithms (Table 1). Inclusion of the 47 infants with abnormal temperature as fever did not materially change algorithm performance (data not shown).

Figure 2

Across all 3 age groups (28 days, 2956 days, and 5789 days), the 4 case‐identification algorithms demonstrated specificity >96%, whereas algorithm sensitivity was highest in the 29‐ to 56‐days‐old age group and lowest among infants 57 to 89 days old across all 4 algorithms (Figure 2). Similar to the overall cohort, an algorithm of discharge or admission diagnosis of fever demonstrated specificity of nearly 98% in all age groups; addition of serious infection codes to this algorithm increased sensitivity, highest in the 29‐ to 56‐days‐old age group (Figure 2; see also Supporting Information, Table 2, in the online version of this article).

Table 2.Performance Characteristics of ICD‐9 Diagnosis Code Case‐Identification Algorithms Across the Eight Sites According to Reference Standard.*

ICD‐9 Diagnosis Code Algorithm	Sensitivity, Median % (Range)	Specificity, Median % (Range)	Negative Predictive Value, Median % (Range)	Positive Predictive Value, Median % (Range)
NOTE: Abbreviations: ICD‐9, International Classification of Diseases, Ninth Revision. *Reference standard of fever was defined by documented temperature 100.4F (38.0 C) on review of electronic medical record.
Discharge diagnosis of fever	56.2 (34.681.0)	98.3 (96.499.1)	92.1 (83.297.4)	87.7 (74.093.2)
Discharge or Admission diagnosis of Fever	76.7 (51.385.0)	97.8 (96.298.7)	95.6 (86.997.4)	87.4 (80.092.9)
Discharge diagnosis of fever or serious infection	68.3 (44.287.3)	96.5 (95.498.0)	93.6 (85.298.2)	78.3 (74.289.0)
Discharge or admission diagnosis of fever or serious infection	83.1 (58.390.7)	95.8 (95.498.0)	96.5 (88.598.2)	79.1 (77.490.4)

Across the 8 study sites, median specificity was 95.8% to 98.3% for the 4 algorithms, with little interhospital variability; however, algorithm sensitivity varied widely by site. Median PPV was highest for discharge diagnosis of fever alone at 87.7% but ranged from 74.0% to 93.2% across sites. Median PPV for an algorithm of discharge or admission diagnosis of fever was similar (87.4%) but with less variation by site (range 80.0%92.9%) (Table 2).

DISCUSSION

We described the performance of 4 ICD‐9 diagnosis code case‐identification algorithms for the identification of febrile young infants <90 days of age at US children's hospitals. Although the specificity was high across algorithms and institutions, the sensitivity was relatively low, particularly for discharge diagnosis of fever, and varied by institution. Given the high specificity, ICD‐9 diagnosis code case‐identification algorithms for fever reliably identify febrile infants using administrative data with low rates of inclusion of infants without fever. However, underidentification of patients, particularly those more prone to SBIs and severe outcomes depending on the algorithm utilized, can impact interpretation of comparative effectiveness studies or the quality of care delivered by an institution.

ICD‐9 discharge diagnosis codes are frequently used to identify pediatric patients across a variety of administrative databases, diseases, and symptoms.[19, 27, 28, 29, 30, 31] Although discharge diagnosis of fever is highly specific, sensitivity is substantially lower than other case‐identification algorithms we studied, particularly for hospitalized infants. This may be due to a fever code sometimes being omitted in favor of a more specific diagnosis (eg, bacteremia) prior to hospital discharge. Therefore, case identification relying only on ICD‐9 discharge diagnosis codes for fever may under‐report clinically important SBI or severe outcomes as demonstrated in our study. This is in contrast to ICD‐9 diagnosis code identification strategies for childhood UTI and pneumonia, which largely have higher sensitivity but lower specificity than fever codes.[13, 14]

Admission diagnosis of fever is important for febrile infants as they may not have an explicit diagnosis at the time of disposition from the ED. Addition of admission diagnosis of fever to an algorithm relying on discharge diagnosis code alone increased sensitivity without a demonstrable reduction in specificity and PPV, likely due to capture of infants with a fever diagnosis at presentation before a specific infection was identified. Although using an algorithm of discharge or admission diagnosis of fever captured a higher percentage of hospitalized febrile infants with SBIs, sensitivity was only 71% overall with this algorithm, and 43% of febrile infants with SBI would still have been missed. Importantly, though, addition of various ICD‐9 codes for serious infection to this algorithm resulted in capture of all febrile infants with SBI and should be used as a sensitivity analysis.

The test characteristics of diagnosis codes were highest in the 29‐ to 56‐days‐old age group. Given the differing low‐risk criteria[6, 7, 8] and lack of best practice guidelines[16] in this age group, the use of administrative data may allow for the comparison of testing and treatment strategies across a large cohort of febrile infants aged 29 to 56 days. However, individual hospital coding practices may affect algorithm performance, in particular sensitivity, which varied substantially by hospital. This variation in algorithm sensitivity may impact comparisons of outcomes across institutions. Therefore, when conducting studies of febrile infants using administrative data, sensitivity analyses or use of chart review should be considered to augment the use of ICD‐9 code‐based identification strategies, particularly for comparative benchmarking and outcomes studies. These additional analyses are particularly important for studies of febrile infants >56 days of age, in whom the sensitivity of diagnosis codes is particularly low. We speculate that the lower sensitivity in older febrile infants may relate to a lack of consensus on the clinical significance of fever in this age group and the varying management strategies employed.[10]

Strengths of this study include the assessment of ICD‐9 code algorithms across multiple institutions for identification of fever in young infants, and the patterns of our findings remained robust when comparing median performance characteristics of the algorithms across hospitals to our overall findings. We were also able to accurately estimate PPV and NPV using a case‐identification strategy weighted to the actual population sizes. Although sensitivity and specificity are the primary measures of test performance, predictive values are highly informative for investigators using administrative data. Additionally, our findings may inform public health efforts including disease surveillance, assessment of seasonal variation, and identification and monitoring of healthcare‐associated infections among febrile infants.

Our study has limitations. We did not review all identified records, which raises the possibility that our evaluated cohort may not be representative of the entire febrile infant population. We attempted to mitigate this possibility by using a random sampling strategy for our population selection that was weighted to the actual population sizes. Second, we identified serious infections using ICD‐9 diagnosis codes determined by group consensus, which may not capture all serious infection codes that identify febrile infants whose fever code was omitted. Third, 47 infants had abnormal temperature that did not meet our reference standard criteria for fever and were included in the no fever group. Although there may be disagreement regarding what constitutes a fever, we used a widely accepted reference standard to define fever.[16] Further, inclusion of these 47 infants as fever did not materially change algorithm performance. Last, our study was conducted at 8 large tertiary‐care children's hospitals, and our results may not be generalizable to other children's hospitals and community‐based hospitals.

CONCLUSIONS

Studies of febrile young infants that rely on ICD‐9 discharge diagnosis code of fever for case ascertainment have high specificity but low sensitivity for the identification of febrile infants, particularly among hospitalized patients. A case‐identification strategy that includes discharge or admission diagnosis of fever demonstrated higher sensitivity, and should be considered for studies of febrile infants using administrative data. However, additional strategies such as incorporation of ICD‐9 codes for serious infection should be used when comparing outcomes across institutions.

Fever is one of the most common reasons for emergency department (ED) evaluation of infants under 90 days of age.[1] Up to 10% to 20% of febrile young infants will have a serious bacterial infection (SBI),[2, 3, 4] but infants with SBI are difficult to distinguish from those without SBI based upon symptoms and physical examination findings alone.[5] Previously developed clinical prediction algorithms can help to identify febrile infants at low risk for SBI, but differ in age range as well as recommendations for testing and empiric treatment.[6, 7, 8] Consequently, there is widespread variation in management of febrile young infants at US children's hospitals,[9, 10, 11] and defining optimal management strategies remains an important issue in pediatric healthcare.[12] Administrative datasets are convenient and inexpensive, and can be used to evaluate practice variation, trends, and outcomes of a large, diverse group of patients within and across institutions.[9, 10] Accurately identifying febrile infants evaluated for suspected SBI in administrative databases would facilitate comparative effectiveness research, quality improvement initiatives, and institutional benchmarking.

Prior studies have validated the accuracy of administrative billing codes for identification of other common childhood illnesses, including urinary tract infection (UTI)[13] and pneumonia.[14] The accuracy of International Classification of Diseases, Ninth Revision (ICD‐9) diagnosis codes in identifying febrile young infants evaluated for SBI is not known. Reliance on administrative ICD‐9 diagnosis codes for patient identification can lead to misclassification of patients due to variable database quality, the validity of the diagnosis codes being utilized, and hospital coding practices.[15] Additionally, fever is a symptom and not a specific diagnosis. If a particular bacterial or viral diagnosis is established (eg, enterovirus meningitis), a discharge diagnosis of fever may not be attributed to the patient encounter. Thus, evaluating the performance characteristics and capture of clinical outcomes of different combinations of ICD‐9 diagnosis codes for identifying febrile infants is necessary for both the conduct and interpretation of studies that utilize administrative databases. The primary objective of this investigation was to identify the most accurate ICD‐9 coding strategies for the identification of febrile infants aged <90 days using administrative data. We also sought to evaluate capture of clinically important outcomes across identification strategies.

METHODS

ICD‐9 Diagnosis Code Groups

In the PHIS database, all patients discharged from the hospital (including hospitalized patients as well as patients discharged from the ED) receive 1 or more ICD‐9 discharge diagnosis codes. These diagnosis codes are ascribed after discharge from the hospital, or for ED patients, after ED discharge. Additionally, patients may receive an admission diagnosis, which reflects the diagnosis ascribed at the time of ED discharge or transfer to the inpatient unit.

We reviewed medical records of infants selected from the following ICD‐9 diagnosis code groups (Figure 1): (1) discharge diagnosis code of fever (780.6 [fever and other physiologic disturbances of temperature regulation], 778.4 [other disturbances of temperature regulation of newborn], 780.60 [fever, unspecified], or 780.61 [fever presenting with conditions classified elsewhere])[9, 10] regardless of the presence of admission diagnosis of fever or diagnosis of serious infection, (2) admission diagnosis code of fever without associated discharge diagnosis code of fever,[10] (3) discharge diagnosis code of serious infection determined a priori (see Supporting Information, Appendix, in the online version of this article) without discharge or admission diagnosis code of fever, and (4) infants without any diagnosis code of fever or serious infection.

Figure 1

Medical records reviewed in each of the 4 ICD‐9 diagnosis code groups were randomly selected from the overall set of ED encounters in the population of infants <90 days of age evaluated during the study period. Twenty‐five percent population sampling was used for 3 of the ICD‐9 diagnosis code groups, whereas 5% sampling was used for the no fever/no serious infection code group. The number of medical records reviewed in each ICD‐9 diagnosis code group was proportional to the distribution of ICD‐9 codes across the entire population of infants <90 days of age. These records were distributed equally across sites (228 records per site), except for 1 site that does not assign admission diagnoses (201 records).

Investigators were blinded to ICD‐9 diagnosis code groups during medical record review. Infants with multiple visits during the study period were eligible to be included more than once if the visits occurred more than 3 days apart. For infants with more than 1 ED visit on a particular calendar day, investigators were instructed to review the initial visit.

For each encounter, we also abstracted demographic characteristics (gender, race/ethnicity), insurance status, hospital region (using US Census categories[22]), and season from the PHIS database.

RESULTS

ICD‐9 Code Case‐Identification Algorithm Performance

Compared with the reference standard, the 4 case‐identification algorithms demonstrated specificity of 96.2% to 98.2% but lower sensitivity overall (Figure 2). Discharge diagnosis of fever alone demonstrated the lowest sensitivity. The algorithm of discharge or admission diagnosis of fever resulted in increased sensitivity and the highest PPV of all 4 algorithms (86.9%, 95% CI: 84.5‐89.3). Addition of serious infection codes to this algorithm resulted in a marginal increase in sensitivity and a similar decrease in PPV (Table 1). When limited to hospitalized infants, specificity was highest for the case‐identification algorithm of discharge diagnosis of fever and similarly high for discharge or admission diagnosis of fever; sensitivity was highest for the algorithm of discharge or admission diagnosis of fever or diagnosis of serious infection. For infants discharged from the ED, algorithm specificity was 97.7% to 98.4%, with lower sensitivity for all 4 algorithms (Table 1). Inclusion of the 47 infants with abnormal temperature as fever did not materially change algorithm performance (data not shown).

Figure 2

Across all 3 age groups (28 days, 2956 days, and 5789 days), the 4 case‐identification algorithms demonstrated specificity >96%, whereas algorithm sensitivity was highest in the 29‐ to 56‐days‐old age group and lowest among infants 57 to 89 days old across all 4 algorithms (Figure 2). Similar to the overall cohort, an algorithm of discharge or admission diagnosis of fever demonstrated specificity of nearly 98% in all age groups; addition of serious infection codes to this algorithm increased sensitivity, highest in the 29‐ to 56‐days‐old age group (Figure 2; see also Supporting Information, Table 2, in the online version of this article).

Table 2.Performance Characteristics of ICD‐9 Diagnosis Code Case‐Identification Algorithms Across the Eight Sites According to Reference Standard.*

ICD‐9 Diagnosis Code Algorithm	Sensitivity, Median % (Range)	Specificity, Median % (Range)	Negative Predictive Value, Median % (Range)	Positive Predictive Value, Median % (Range)
NOTE: Abbreviations: ICD‐9, International Classification of Diseases, Ninth Revision. *Reference standard of fever was defined by documented temperature 100.4F (38.0 C) on review of electronic medical record.
Discharge diagnosis of fever	56.2 (34.681.0)	98.3 (96.499.1)	92.1 (83.297.4)	87.7 (74.093.2)
Discharge or Admission diagnosis of Fever	76.7 (51.385.0)	97.8 (96.298.7)	95.6 (86.997.4)	87.4 (80.092.9)
Discharge diagnosis of fever or serious infection	68.3 (44.287.3)	96.5 (95.498.0)	93.6 (85.298.2)	78.3 (74.289.0)
Discharge or admission diagnosis of fever or serious infection	83.1 (58.390.7)	95.8 (95.498.0)	96.5 (88.598.2)	79.1 (77.490.4)

Across the 8 study sites, median specificity was 95.8% to 98.3% for the 4 algorithms, with little interhospital variability; however, algorithm sensitivity varied widely by site. Median PPV was highest for discharge diagnosis of fever alone at 87.7% but ranged from 74.0% to 93.2% across sites. Median PPV for an algorithm of discharge or admission diagnosis of fever was similar (87.4%) but with less variation by site (range 80.0%92.9%) (Table 2).

DISCUSSION

We described the performance of 4 ICD‐9 diagnosis code case‐identification algorithms for the identification of febrile young infants <90 days of age at US children's hospitals. Although the specificity was high across algorithms and institutions, the sensitivity was relatively low, particularly for discharge diagnosis of fever, and varied by institution. Given the high specificity, ICD‐9 diagnosis code case‐identification algorithms for fever reliably identify febrile infants using administrative data with low rates of inclusion of infants without fever. However, underidentification of patients, particularly those more prone to SBIs and severe outcomes depending on the algorithm utilized, can impact interpretation of comparative effectiveness studies or the quality of care delivered by an institution.

ICD‐9 discharge diagnosis codes are frequently used to identify pediatric patients across a variety of administrative databases, diseases, and symptoms.[19, 27, 28, 29, 30, 31] Although discharge diagnosis of fever is highly specific, sensitivity is substantially lower than other case‐identification algorithms we studied, particularly for hospitalized infants. This may be due to a fever code sometimes being omitted in favor of a more specific diagnosis (eg, bacteremia) prior to hospital discharge. Therefore, case identification relying only on ICD‐9 discharge diagnosis codes for fever may under‐report clinically important SBI or severe outcomes as demonstrated in our study. This is in contrast to ICD‐9 diagnosis code identification strategies for childhood UTI and pneumonia, which largely have higher sensitivity but lower specificity than fever codes.[13, 14]

Admission diagnosis of fever is important for febrile infants as they may not have an explicit diagnosis at the time of disposition from the ED. Addition of admission diagnosis of fever to an algorithm relying on discharge diagnosis code alone increased sensitivity without a demonstrable reduction in specificity and PPV, likely due to capture of infants with a fever diagnosis at presentation before a specific infection was identified. Although using an algorithm of discharge or admission diagnosis of fever captured a higher percentage of hospitalized febrile infants with SBIs, sensitivity was only 71% overall with this algorithm, and 43% of febrile infants with SBI would still have been missed. Importantly, though, addition of various ICD‐9 codes for serious infection to this algorithm resulted in capture of all febrile infants with SBI and should be used as a sensitivity analysis.

The test characteristics of diagnosis codes were highest in the 29‐ to 56‐days‐old age group. Given the differing low‐risk criteria[6, 7, 8] and lack of best practice guidelines[16] in this age group, the use of administrative data may allow for the comparison of testing and treatment strategies across a large cohort of febrile infants aged 29 to 56 days. However, individual hospital coding practices may affect algorithm performance, in particular sensitivity, which varied substantially by hospital. This variation in algorithm sensitivity may impact comparisons of outcomes across institutions. Therefore, when conducting studies of febrile infants using administrative data, sensitivity analyses or use of chart review should be considered to augment the use of ICD‐9 code‐based identification strategies, particularly for comparative benchmarking and outcomes studies. These additional analyses are particularly important for studies of febrile infants >56 days of age, in whom the sensitivity of diagnosis codes is particularly low. We speculate that the lower sensitivity in older febrile infants may relate to a lack of consensus on the clinical significance of fever in this age group and the varying management strategies employed.[10]

Strengths of this study include the assessment of ICD‐9 code algorithms across multiple institutions for identification of fever in young infants, and the patterns of our findings remained robust when comparing median performance characteristics of the algorithms across hospitals to our overall findings. We were also able to accurately estimate PPV and NPV using a case‐identification strategy weighted to the actual population sizes. Although sensitivity and specificity are the primary measures of test performance, predictive values are highly informative for investigators using administrative data. Additionally, our findings may inform public health efforts including disease surveillance, assessment of seasonal variation, and identification and monitoring of healthcare‐associated infections among febrile infants.

Our study has limitations. We did not review all identified records, which raises the possibility that our evaluated cohort may not be representative of the entire febrile infant population. We attempted to mitigate this possibility by using a random sampling strategy for our population selection that was weighted to the actual population sizes. Second, we identified serious infections using ICD‐9 diagnosis codes determined by group consensus, which may not capture all serious infection codes that identify febrile infants whose fever code was omitted. Third, 47 infants had abnormal temperature that did not meet our reference standard criteria for fever and were included in the no fever group. Although there may be disagreement regarding what constitutes a fever, we used a widely accepted reference standard to define fever.[16] Further, inclusion of these 47 infants as fever did not materially change algorithm performance. Last, our study was conducted at 8 large tertiary‐care children's hospitals, and our results may not be generalizable to other children's hospitals and community‐based hospitals.

CONCLUSIONS

Studies of febrile young infants that rely on ICD‐9 discharge diagnosis code of fever for case ascertainment have high specificity but low sensitivity for the identification of febrile infants, particularly among hospitalized patients. A case‐identification strategy that includes discharge or admission diagnosis of fever demonstrated higher sensitivity, and should be considered for studies of febrile infants using administrative data. However, additional strategies such as incorporation of ICD‐9 codes for serious infection should be used when comparing outcomes across institutions.

Febrile young infants are at high risk for serious bacterial infection (SBI) with reported rates of 8.5% to 12%, even higher in neonates 28 days of age.[1, 2, 3] As a result, febrile infants often undergo extensive diagnostic evaluation consisting of a combination of urine, blood, and cerebrospinal fluid (CSF) testing.[4, 5, 6] Several clinical prediction algorithms use this diagnostic testing to identify febrile infants at low risk for SBI, but they differ with respect to age range, recommended testing, antibiotic administration, and threshold for hospitalization.[4, 5, 6] Additionally, the optimal management strategy for this population has not been defined.[7] Consequently, laboratory testing, antibiotic use, and hospitalization for febrile young infants vary widely among hospitals.[8, 9, 10]

Clinical practice guidelines (CPGs) are designed to implement evidence‐based care and reduce practice variability, with the goal of improving quality of care and optimizing costs.[11] Implementation of a CPG for management of febrile young infants in the Intermountain Healthcare System was associated with greater adherence to evidence‐based care and lower costs.[12] However, when strong evidence is lacking, different interpretations of febrile infant risk classification incorporated into local CPGs may be a major driver of the across‐hospital practice variation observed in prior studies.[8, 9] Understanding sources of variability as well as determining the association of CPGs with clinicians' practice patterns can help identify quality improvement opportunities, either through national benchmarking or local efforts.

Our primary objectives were to compare (1) recommendations of pediatric emergency departmentbased institutional CPGs for febrile young infants and (2) rates of urine, blood, CSF testing, hospitalization, and ceftriaxone use at emergency department (ED) discharge based upon CPG presence and the specific CPG recommendations. Our secondary objectives were to describe the association of CPGs with healthcare costs and return visits for SBI.

METHODS

RESULTS

CPG by Institution

Thirty‐three (89.2%) of the 37 EDs surveyed completed the questionnaire. Overall, 21 (63.6%) of the 33 EDs had a CPG; 15 (45.5%) had a CPG for all infants 56 days of age, 5 (15.2%) had a CPG for infants 28 days only, and 1 (3.0%) had a CPG for infants 29 to 56 days but not 28 days of age (Figure 1). Seventeen EDs had an established CPG prior to 2013, and 4 hospitals implemented a CPG during the 2013 calendar year, 2 with CPGs for neonates 28 days and 2 with CPGs for both 28 days and 29 to 56 days of age. Hospitals with CPGs were more likely to be located in the Northeast and West regions of the United States and provide care to a higher proportion of non‐Hispanic white patients, as well as those with commercial insurance (Table 1).

Figure 1

Table 1.Characteristics of Patients in Hospitals With and Without CPGs for the Febrile Young Infant 56 Days of Age

Characteristic	28 Days			2956 Days
	No CPG, n=996, N (%)	CPG, n=2,149, N (%)	P Value	No CPG, n=2,460, N (%)	CPG, n=3,772, N (%)	P Value
NOTE: Abbreviations: CPG, clinical practice guideline; IQR, interquartile range; UTI, urinary tract infection. *Includes UTI/pyelonephritis, bacteremia/sepsis, bacterial meningitis, pneumonia, and bacterial enteritis. Some infants had more than 1 site of infection.
Race
Non‐Hispanic white	325 (32.6)	996 (46.3)		867 (35.2)	1,728 (45.8)
Non‐Hispanic black	248 (24.9)	381 (17.7)		593 (24.1)	670 (17.8)
Hispanic	243 (24.4)	531 (24.7)		655 (26.6)	986 (26.1)
Asian	28 (2.8)	78 (3.6)		40 (1.6)	122 (3.2)
Other Race	152 (15.3)	163 (7.6)	<0.001	305 (12.4)	266 (7.1)	<0.001
Gender
Female	435 (43.7)	926 (43.1)	0.76	1,067 (43.4)	1,714 (45.4)	0.22
Payer
Commercial	243 (24.4)	738 (34.3)		554 (22.5)	1,202 (31.9)
Government	664 (66.7)	1,269 (59.1)		1,798 (73.1)	2,342 (62.1)
Other payer	89 (8.9)	142 (6.6)	<0.001	108 (4.4)	228 (6.0)	<0.001
Region
Northeast	39 (3.9)	245 (11.4)		77 (3.1)	572 (15.2)
South	648 (65.1)	915 (42.6)		1,662 (67.6)	1,462 (38.8)
Midwest	271 (27.2)	462 (21.5)		506 (20.6)	851 (22.6)
West	38 (3.8)	527 (24.5)	<0.001	215 (8.7)	887 (23.5)	<0.001
Serious bacterial infection
Overall*	131 (13.2)	242 (11.3)	0.14	191 (7.8)	237 (6.3)	0.03
UTI/pyelonephritis	73 (7.3)	153 (7.1)		103 (4.2)	154 (4.1)
Bacteremia/sepsis	56 (5.6)	91 (4.2)		78 (3.2)	61 (1.6)
Bacterial meningitis	15 (1.5)	15 (0.7)		4 (0.2)	14 (0.4)
Age, d, median (IQR)	18 (11, 24)	18 (11, 23)	0.67	46 (37, 53)	45 (37, 53)	0.11

All 20 CPGs for the febrile young infant 28 days of age recommended urine, blood, CSF testing, and hospitalization for all infants (Figure 1). Of the 16 hospitals with CPGs for febrile infants aged 29 to 56 days, all recommended urine and blood testing for all patients, except for 1 CPG, which recommended consideration of blood testing but not to obtain routinely. Hospitals varied in recommendations for CSF testing among infants aged 29 to 56 days: 8 (50%) recommended CSF testing in all patients and 8 (50%) recommended CSF testing only if the patient was high risk per defined criteria (based on history, physical examination, urine, and blood testing). In all 16 CPGs, hospitalization was recommended only for high‐risk infants. For low‐risk infants aged 2956 days being discharged from the ED, 3 hospitals recommended ceftriaxone for all, 9 recommended consideration of ceftriaxone, and 4 recommended against antibiotics (Figure 1).

DISCUSSION

We described the content and association of CPGs with management of the febrile infant 56 days of age across a large sample of children's hospitals. Nearly two‐thirds of included pediatric EDs have a CPG for the management of young febrile infants. Management of febrile infants 28 days was uniform, with a majority hospitalized after urine, blood, and CSF testing regardless of the presence of a CPG. In contrast, CPGs for infants 29 to 56 days of age varied in their recommendations for CSF testing as well as ceftriaxone use for infants discharged from the ED. Consequently, we observed considerable hospital variability in CSF testing and ceftriaxone use for discharged infants, which correlates with variation in the presence and content of CPGs. Institutional CPGs may be a source of the across‐hospital variation in care of febrile young infants observed in prior study.[9]

Febrile infants 28 days of age are at particularly high risk for SBI, with a prevalence of nearly 20% or higher.[2, 3, 29] The high prevalence of SBI, combined with the inherent difficulty in distinguishing neonates with and without SBI,[2, 30] has resulted in uniform CPG recommendations to perform the full‐sepsis workup in this young age group. Similar to prior studies,[8, 9] we observed that most febrile infants 28 days undergo the full sepsis evaluation, including CSF testing, and are hospitalized regardless of the presence of a CPG.

However, given the conflicting recommendations for febrile infants 29 to 56 days of age,[4, 5, 6] the optimal management strategy is less certain.[7] The Rochester, Philadelphia, and Boston criteria, 3 published models to identify infants at low risk for SBI, primarily differ in their recommendations for CSF testing and ceftriaxone use in this age group.[4, 5, 6] Half of the CPGs recommended CSF testing for all febrile infants, and half recommended CSF testing only if the infant was high risk. Institutional guidelines that recommended selective CSF testing for febrile infants aged 29 to 56 days were associated with lower rates of CSF testing. Furthermore, ceftriaxone use varied based on CPG recommendations for low‐risk infants discharged from the ED. Therefore, the influence of febrile infant CPGs mainly relates to the limiting of CSF testing and targeted ceftriaxone use in low‐risk infants. As the rate of return visits for SBI is low across hospitals, future study should assess outcomes at hospitals with CPGs recommending selective CSF testing. Of note, infants 29 to 56 days of age were less likely to be hospitalized when cared for at a hospital with an established CPG prior to 2013 without increase in 3‐day revisits for SBI. This finding may indicate that longer duration of CPG implementation is associated with lower rates of hospitalization for low‐risk infants; this finding merits further study.

The presence of a CPG was not associated with lower costs for febrile infants in either age group. Although individual healthcare systems have achieved lower costs with CPG implementation,[12] the mere presence of a CPG is not associated with lower costs when assessed across institutions. Higher costs for admitted and discharged infants 29 to 56 days of age in the presence of a CPG likely reflects the higher rate of CSF testing at hospitals whose CPGs recommend testing for all febrile infants, as well as inpatient management strategies for hospitalized infants not captured in our study. Future investigation should include an assessment of the cost‐effectiveness of the various testing and treatment strategies employed for the febrile young infant.

Our study has several limitations. First, the validity of ICD‐9 diagnosis codes for identifying young infants with fever is not well established, and thus our study is subject to misclassification bias. To minimize missed patients, we included infants with either an ICD‐9 admission or discharge diagnosis of fever; however, utilization of diagnosis codes for patient identification may have resulted in undercapture of infants with a measured temperature of 38.0C. It is also possible that some patients who did not undergo testing were misclassified as having a fever or had temperatures below standard thresholds to prompt diagnostic testing. This is a potential reason that testing was not performed in 100% of infants, even at hospitals with CPGs that recommended testing for all patients. Additionally, some febrile infants diagnosed with SBI may not have an associated ICD‐9 diagnosis code for fever. Although the overall SBI rate observed in our study was similar to prior studies,[4, 31] the rate in neonates 28 days of age was lower than reported in recent investigations,[2, 3] which may indicate inclusion of a higher proportion of low‐risk febrile infants. With the exception of bronchiolitis, we also did not assess diagnostic testing in the presence of other identified sources of infection such as herpes simplex virus.

Second, we were unable to assess the presence or absence of a CPG at the 4 excluded EDs that did not respond to the survey or the institutions excluded for data‐quality issues. However, included and excluded hospitals did not differ in region or annual ED volume (data not shown).

Third, although we classified hospitals based upon the presence and content of CPGs, we were unable to fully evaluate adherence to the CPG at each site.

Last, though PHIS hospitals represent 85% of freestanding children's hospitals, many febrile infants are hospitalized at non‐PHIS institutions; our results may not be generalizable to care provided at nonchildren's hospitals.

CONCLUSIONS

Management of febrile neonates 28 days of age does not vary based on CPG presence. However, CPGs for the febrile infant aged 29 to 56 days vary in recommendations for CSF testing as well as ceftriaxone use for low‐risk patients, which significantly contributes to practice variation and healthcare costs across institutions.

Disclosures

This project was funded in part by The Gerber Foundation Novice Researcher Award (Ref #18273835). Dr. Fran Balamuth received career development support from the National Institutes of Health (NHLBI K12‐HL109009). Funders were not involved in design or conduct of the study; collection, management, analysis, or interpretation of the data; preparation, review, or approval of the manuscript. The authors have no financial relationships relevant to this article to disclose. No payment was received for the production of this article. The authors have no conflicts of interest relevant to this article to disclose.

Febrile young infants are at high risk for serious bacterial infection (SBI) with reported rates of 8.5% to 12%, even higher in neonates 28 days of age.[1, 2, 3] As a result, febrile infants often undergo extensive diagnostic evaluation consisting of a combination of urine, blood, and cerebrospinal fluid (CSF) testing.[4, 5, 6] Several clinical prediction algorithms use this diagnostic testing to identify febrile infants at low risk for SBI, but they differ with respect to age range, recommended testing, antibiotic administration, and threshold for hospitalization.[4, 5, 6] Additionally, the optimal management strategy for this population has not been defined.[7] Consequently, laboratory testing, antibiotic use, and hospitalization for febrile young infants vary widely among hospitals.[8, 9, 10]

Clinical practice guidelines (CPGs) are designed to implement evidence‐based care and reduce practice variability, with the goal of improving quality of care and optimizing costs.[11] Implementation of a CPG for management of febrile young infants in the Intermountain Healthcare System was associated with greater adherence to evidence‐based care and lower costs.[12] However, when strong evidence is lacking, different interpretations of febrile infant risk classification incorporated into local CPGs may be a major driver of the across‐hospital practice variation observed in prior studies.[8, 9] Understanding sources of variability as well as determining the association of CPGs with clinicians' practice patterns can help identify quality improvement opportunities, either through national benchmarking or local efforts.

Our primary objectives were to compare (1) recommendations of pediatric emergency departmentbased institutional CPGs for febrile young infants and (2) rates of urine, blood, CSF testing, hospitalization, and ceftriaxone use at emergency department (ED) discharge based upon CPG presence and the specific CPG recommendations. Our secondary objectives were to describe the association of CPGs with healthcare costs and return visits for SBI.

METHODS

RESULTS

CPG by Institution

Thirty‐three (89.2%) of the 37 EDs surveyed completed the questionnaire. Overall, 21 (63.6%) of the 33 EDs had a CPG; 15 (45.5%) had a CPG for all infants 56 days of age, 5 (15.2%) had a CPG for infants 28 days only, and 1 (3.0%) had a CPG for infants 29 to 56 days but not 28 days of age (Figure 1). Seventeen EDs had an established CPG prior to 2013, and 4 hospitals implemented a CPG during the 2013 calendar year, 2 with CPGs for neonates 28 days and 2 with CPGs for both 28 days and 29 to 56 days of age. Hospitals with CPGs were more likely to be located in the Northeast and West regions of the United States and provide care to a higher proportion of non‐Hispanic white patients, as well as those with commercial insurance (Table 1).

Figure 1

Table 1.Characteristics of Patients in Hospitals With and Without CPGs for the Febrile Young Infant 56 Days of Age

Characteristic	28 Days			2956 Days
	No CPG, n=996, N (%)	CPG, n=2,149, N (%)	P Value	No CPG, n=2,460, N (%)	CPG, n=3,772, N (%)	P Value
NOTE: Abbreviations: CPG, clinical practice guideline; IQR, interquartile range; UTI, urinary tract infection. *Includes UTI/pyelonephritis, bacteremia/sepsis, bacterial meningitis, pneumonia, and bacterial enteritis. Some infants had more than 1 site of infection.
Race
Non‐Hispanic white	325 (32.6)	996 (46.3)		867 (35.2)	1,728 (45.8)
Non‐Hispanic black	248 (24.9)	381 (17.7)		593 (24.1)	670 (17.8)
Hispanic	243 (24.4)	531 (24.7)		655 (26.6)	986 (26.1)
Asian	28 (2.8)	78 (3.6)		40 (1.6)	122 (3.2)
Other Race	152 (15.3)	163 (7.6)	<0.001	305 (12.4)	266 (7.1)	<0.001
Gender
Female	435 (43.7)	926 (43.1)	0.76	1,067 (43.4)	1,714 (45.4)	0.22
Payer
Commercial	243 (24.4)	738 (34.3)		554 (22.5)	1,202 (31.9)
Government	664 (66.7)	1,269 (59.1)		1,798 (73.1)	2,342 (62.1)
Other payer	89 (8.9)	142 (6.6)	<0.001	108 (4.4)	228 (6.0)	<0.001
Region
Northeast	39 (3.9)	245 (11.4)		77 (3.1)	572 (15.2)
South	648 (65.1)	915 (42.6)		1,662 (67.6)	1,462 (38.8)
Midwest	271 (27.2)	462 (21.5)		506 (20.6)	851 (22.6)
West	38 (3.8)	527 (24.5)	<0.001	215 (8.7)	887 (23.5)	<0.001
Serious bacterial infection
Overall*	131 (13.2)	242 (11.3)	0.14	191 (7.8)	237 (6.3)	0.03
UTI/pyelonephritis	73 (7.3)	153 (7.1)		103 (4.2)	154 (4.1)
Bacteremia/sepsis	56 (5.6)	91 (4.2)		78 (3.2)	61 (1.6)
Bacterial meningitis	15 (1.5)	15 (0.7)		4 (0.2)	14 (0.4)
Age, d, median (IQR)	18 (11, 24)	18 (11, 23)	0.67	46 (37, 53)	45 (37, 53)	0.11

All 20 CPGs for the febrile young infant 28 days of age recommended urine, blood, CSF testing, and hospitalization for all infants (Figure 1). Of the 16 hospitals with CPGs for febrile infants aged 29 to 56 days, all recommended urine and blood testing for all patients, except for 1 CPG, which recommended consideration of blood testing but not to obtain routinely. Hospitals varied in recommendations for CSF testing among infants aged 29 to 56 days: 8 (50%) recommended CSF testing in all patients and 8 (50%) recommended CSF testing only if the patient was high risk per defined criteria (based on history, physical examination, urine, and blood testing). In all 16 CPGs, hospitalization was recommended only for high‐risk infants. For low‐risk infants aged 2956 days being discharged from the ED, 3 hospitals recommended ceftriaxone for all, 9 recommended consideration of ceftriaxone, and 4 recommended against antibiotics (Figure 1).

DISCUSSION

We described the content and association of CPGs with management of the febrile infant 56 days of age across a large sample of children's hospitals. Nearly two‐thirds of included pediatric EDs have a CPG for the management of young febrile infants. Management of febrile infants 28 days was uniform, with a majority hospitalized after urine, blood, and CSF testing regardless of the presence of a CPG. In contrast, CPGs for infants 29 to 56 days of age varied in their recommendations for CSF testing as well as ceftriaxone use for infants discharged from the ED. Consequently, we observed considerable hospital variability in CSF testing and ceftriaxone use for discharged infants, which correlates with variation in the presence and content of CPGs. Institutional CPGs may be a source of the across‐hospital variation in care of febrile young infants observed in prior study.[9]

Febrile infants 28 days of age are at particularly high risk for SBI, with a prevalence of nearly 20% or higher.[2, 3, 29] The high prevalence of SBI, combined with the inherent difficulty in distinguishing neonates with and without SBI,[2, 30] has resulted in uniform CPG recommendations to perform the full‐sepsis workup in this young age group. Similar to prior studies,[8, 9] we observed that most febrile infants 28 days undergo the full sepsis evaluation, including CSF testing, and are hospitalized regardless of the presence of a CPG.

However, given the conflicting recommendations for febrile infants 29 to 56 days of age,[4, 5, 6] the optimal management strategy is less certain.[7] The Rochester, Philadelphia, and Boston criteria, 3 published models to identify infants at low risk for SBI, primarily differ in their recommendations for CSF testing and ceftriaxone use in this age group.[4, 5, 6] Half of the CPGs recommended CSF testing for all febrile infants, and half recommended CSF testing only if the infant was high risk. Institutional guidelines that recommended selective CSF testing for febrile infants aged 29 to 56 days were associated with lower rates of CSF testing. Furthermore, ceftriaxone use varied based on CPG recommendations for low‐risk infants discharged from the ED. Therefore, the influence of febrile infant CPGs mainly relates to the limiting of CSF testing and targeted ceftriaxone use in low‐risk infants. As the rate of return visits for SBI is low across hospitals, future study should assess outcomes at hospitals with CPGs recommending selective CSF testing. Of note, infants 29 to 56 days of age were less likely to be hospitalized when cared for at a hospital with an established CPG prior to 2013 without increase in 3‐day revisits for SBI. This finding may indicate that longer duration of CPG implementation is associated with lower rates of hospitalization for low‐risk infants; this finding merits further study.

The presence of a CPG was not associated with lower costs for febrile infants in either age group. Although individual healthcare systems have achieved lower costs with CPG implementation,[12] the mere presence of a CPG is not associated with lower costs when assessed across institutions. Higher costs for admitted and discharged infants 29 to 56 days of age in the presence of a CPG likely reflects the higher rate of CSF testing at hospitals whose CPGs recommend testing for all febrile infants, as well as inpatient management strategies for hospitalized infants not captured in our study. Future investigation should include an assessment of the cost‐effectiveness of the various testing and treatment strategies employed for the febrile young infant.

Our study has several limitations. First, the validity of ICD‐9 diagnosis codes for identifying young infants with fever is not well established, and thus our study is subject to misclassification bias. To minimize missed patients, we included infants with either an ICD‐9 admission or discharge diagnosis of fever; however, utilization of diagnosis codes for patient identification may have resulted in undercapture of infants with a measured temperature of 38.0C. It is also possible that some patients who did not undergo testing were misclassified as having a fever or had temperatures below standard thresholds to prompt diagnostic testing. This is a potential reason that testing was not performed in 100% of infants, even at hospitals with CPGs that recommended testing for all patients. Additionally, some febrile infants diagnosed with SBI may not have an associated ICD‐9 diagnosis code for fever. Although the overall SBI rate observed in our study was similar to prior studies,[4, 31] the rate in neonates 28 days of age was lower than reported in recent investigations,[2, 3] which may indicate inclusion of a higher proportion of low‐risk febrile infants. With the exception of bronchiolitis, we also did not assess diagnostic testing in the presence of other identified sources of infection such as herpes simplex virus.

Second, we were unable to assess the presence or absence of a CPG at the 4 excluded EDs that did not respond to the survey or the institutions excluded for data‐quality issues. However, included and excluded hospitals did not differ in region or annual ED volume (data not shown).

Third, although we classified hospitals based upon the presence and content of CPGs, we were unable to fully evaluate adherence to the CPG at each site.

Last, though PHIS hospitals represent 85% of freestanding children's hospitals, many febrile infants are hospitalized at non‐PHIS institutions; our results may not be generalizable to care provided at nonchildren's hospitals.

CONCLUSIONS

Management of febrile neonates 28 days of age does not vary based on CPG presence. However, CPGs for the febrile infant aged 29 to 56 days vary in recommendations for CSF testing as well as ceftriaxone use for low‐risk patients, which significantly contributes to practice variation and healthcare costs across institutions.

Disclosures

This project was funded in part by The Gerber Foundation Novice Researcher Award (Ref #18273835). Dr. Fran Balamuth received career development support from the National Institutes of Health (NHLBI K12‐HL109009). Funders were not involved in design or conduct of the study; collection, management, analysis, or interpretation of the data; preparation, review, or approval of the manuscript. The authors have no financial relationships relevant to this article to disclose. No payment was received for the production of this article. The authors have no conflicts of interest relevant to this article to disclose.

Febrile young infants are at high risk for serious bacterial infection (SBI) with reported rates of 8.5% to 12%, even higher in neonates 28 days of age.[1, 2, 3] As a result, febrile infants often undergo extensive diagnostic evaluation consisting of a combination of urine, blood, and cerebrospinal fluid (CSF) testing.[4, 5, 6] Several clinical prediction algorithms use this diagnostic testing to identify febrile infants at low risk for SBI, but they differ with respect to age range, recommended testing, antibiotic administration, and threshold for hospitalization.[4, 5, 6] Additionally, the optimal management strategy for this population has not been defined.[7] Consequently, laboratory testing, antibiotic use, and hospitalization for febrile young infants vary widely among hospitals.[8, 9, 10]

Clinical practice guidelines (CPGs) are designed to implement evidence‐based care and reduce practice variability, with the goal of improving quality of care and optimizing costs.[11] Implementation of a CPG for management of febrile young infants in the Intermountain Healthcare System was associated with greater adherence to evidence‐based care and lower costs.[12] However, when strong evidence is lacking, different interpretations of febrile infant risk classification incorporated into local CPGs may be a major driver of the across‐hospital practice variation observed in prior studies.[8, 9] Understanding sources of variability as well as determining the association of CPGs with clinicians' practice patterns can help identify quality improvement opportunities, either through national benchmarking or local efforts.

Our primary objectives were to compare (1) recommendations of pediatric emergency departmentbased institutional CPGs for febrile young infants and (2) rates of urine, blood, CSF testing, hospitalization, and ceftriaxone use at emergency department (ED) discharge based upon CPG presence and the specific CPG recommendations. Our secondary objectives were to describe the association of CPGs with healthcare costs and return visits for SBI.

METHODS

RESULTS

CPG by Institution

Thirty‐three (89.2%) of the 37 EDs surveyed completed the questionnaire. Overall, 21 (63.6%) of the 33 EDs had a CPG; 15 (45.5%) had a CPG for all infants 56 days of age, 5 (15.2%) had a CPG for infants 28 days only, and 1 (3.0%) had a CPG for infants 29 to 56 days but not 28 days of age (Figure 1). Seventeen EDs had an established CPG prior to 2013, and 4 hospitals implemented a CPG during the 2013 calendar year, 2 with CPGs for neonates 28 days and 2 with CPGs for both 28 days and 29 to 56 days of age. Hospitals with CPGs were more likely to be located in the Northeast and West regions of the United States and provide care to a higher proportion of non‐Hispanic white patients, as well as those with commercial insurance (Table 1).

Figure 1

Table 1.Characteristics of Patients in Hospitals With and Without CPGs for the Febrile Young Infant 56 Days of Age

Characteristic	28 Days			2956 Days
	No CPG, n=996, N (%)	CPG, n=2,149, N (%)	P Value	No CPG, n=2,460, N (%)	CPG, n=3,772, N (%)	P Value
NOTE: Abbreviations: CPG, clinical practice guideline; IQR, interquartile range; UTI, urinary tract infection. *Includes UTI/pyelonephritis, bacteremia/sepsis, bacterial meningitis, pneumonia, and bacterial enteritis. Some infants had more than 1 site of infection.
Race
Non‐Hispanic white	325 (32.6)	996 (46.3)		867 (35.2)	1,728 (45.8)
Non‐Hispanic black	248 (24.9)	381 (17.7)		593 (24.1)	670 (17.8)
Hispanic	243 (24.4)	531 (24.7)		655 (26.6)	986 (26.1)
Asian	28 (2.8)	78 (3.6)		40 (1.6)	122 (3.2)
Other Race	152 (15.3)	163 (7.6)	<0.001	305 (12.4)	266 (7.1)	<0.001
Gender
Female	435 (43.7)	926 (43.1)	0.76	1,067 (43.4)	1,714 (45.4)	0.22
Payer
Commercial	243 (24.4)	738 (34.3)		554 (22.5)	1,202 (31.9)
Government	664 (66.7)	1,269 (59.1)		1,798 (73.1)	2,342 (62.1)
Other payer	89 (8.9)	142 (6.6)	<0.001	108 (4.4)	228 (6.0)	<0.001
Region
Northeast	39 (3.9)	245 (11.4)		77 (3.1)	572 (15.2)
South	648 (65.1)	915 (42.6)		1,662 (67.6)	1,462 (38.8)
Midwest	271 (27.2)	462 (21.5)		506 (20.6)	851 (22.6)
West	38 (3.8)	527 (24.5)	<0.001	215 (8.7)	887 (23.5)	<0.001
Serious bacterial infection
Overall*	131 (13.2)	242 (11.3)	0.14	191 (7.8)	237 (6.3)	0.03
UTI/pyelonephritis	73 (7.3)	153 (7.1)		103 (4.2)	154 (4.1)
Bacteremia/sepsis	56 (5.6)	91 (4.2)		78 (3.2)	61 (1.6)
Bacterial meningitis	15 (1.5)	15 (0.7)		4 (0.2)	14 (0.4)
Age, d, median (IQR)	18 (11, 24)	18 (11, 23)	0.67	46 (37, 53)	45 (37, 53)	0.11

All 20 CPGs for the febrile young infant 28 days of age recommended urine, blood, CSF testing, and hospitalization for all infants (Figure 1). Of the 16 hospitals with CPGs for febrile infants aged 29 to 56 days, all recommended urine and blood testing for all patients, except for 1 CPG, which recommended consideration of blood testing but not to obtain routinely. Hospitals varied in recommendations for CSF testing among infants aged 29 to 56 days: 8 (50%) recommended CSF testing in all patients and 8 (50%) recommended CSF testing only if the patient was high risk per defined criteria (based on history, physical examination, urine, and blood testing). In all 16 CPGs, hospitalization was recommended only for high‐risk infants. For low‐risk infants aged 2956 days being discharged from the ED, 3 hospitals recommended ceftriaxone for all, 9 recommended consideration of ceftriaxone, and 4 recommended against antibiotics (Figure 1).

DISCUSSION

We described the content and association of CPGs with management of the febrile infant 56 days of age across a large sample of children's hospitals. Nearly two‐thirds of included pediatric EDs have a CPG for the management of young febrile infants. Management of febrile infants 28 days was uniform, with a majority hospitalized after urine, blood, and CSF testing regardless of the presence of a CPG. In contrast, CPGs for infants 29 to 56 days of age varied in their recommendations for CSF testing as well as ceftriaxone use for infants discharged from the ED. Consequently, we observed considerable hospital variability in CSF testing and ceftriaxone use for discharged infants, which correlates with variation in the presence and content of CPGs. Institutional CPGs may be a source of the across‐hospital variation in care of febrile young infants observed in prior study.[9]

Febrile infants 28 days of age are at particularly high risk for SBI, with a prevalence of nearly 20% or higher.[2, 3, 29] The high prevalence of SBI, combined with the inherent difficulty in distinguishing neonates with and without SBI,[2, 30] has resulted in uniform CPG recommendations to perform the full‐sepsis workup in this young age group. Similar to prior studies,[8, 9] we observed that most febrile infants 28 days undergo the full sepsis evaluation, including CSF testing, and are hospitalized regardless of the presence of a CPG.

However, given the conflicting recommendations for febrile infants 29 to 56 days of age,[4, 5, 6] the optimal management strategy is less certain.[7] The Rochester, Philadelphia, and Boston criteria, 3 published models to identify infants at low risk for SBI, primarily differ in their recommendations for CSF testing and ceftriaxone use in this age group.[4, 5, 6] Half of the CPGs recommended CSF testing for all febrile infants, and half recommended CSF testing only if the infant was high risk. Institutional guidelines that recommended selective CSF testing for febrile infants aged 29 to 56 days were associated with lower rates of CSF testing. Furthermore, ceftriaxone use varied based on CPG recommendations for low‐risk infants discharged from the ED. Therefore, the influence of febrile infant CPGs mainly relates to the limiting of CSF testing and targeted ceftriaxone use in low‐risk infants. As the rate of return visits for SBI is low across hospitals, future study should assess outcomes at hospitals with CPGs recommending selective CSF testing. Of note, infants 29 to 56 days of age were less likely to be hospitalized when cared for at a hospital with an established CPG prior to 2013 without increase in 3‐day revisits for SBI. This finding may indicate that longer duration of CPG implementation is associated with lower rates of hospitalization for low‐risk infants; this finding merits further study.

The presence of a CPG was not associated with lower costs for febrile infants in either age group. Although individual healthcare systems have achieved lower costs with CPG implementation,[12] the mere presence of a CPG is not associated with lower costs when assessed across institutions. Higher costs for admitted and discharged infants 29 to 56 days of age in the presence of a CPG likely reflects the higher rate of CSF testing at hospitals whose CPGs recommend testing for all febrile infants, as well as inpatient management strategies for hospitalized infants not captured in our study. Future investigation should include an assessment of the cost‐effectiveness of the various testing and treatment strategies employed for the febrile young infant.

Our study has several limitations. First, the validity of ICD‐9 diagnosis codes for identifying young infants with fever is not well established, and thus our study is subject to misclassification bias. To minimize missed patients, we included infants with either an ICD‐9 admission or discharge diagnosis of fever; however, utilization of diagnosis codes for patient identification may have resulted in undercapture of infants with a measured temperature of 38.0C. It is also possible that some patients who did not undergo testing were misclassified as having a fever or had temperatures below standard thresholds to prompt diagnostic testing. This is a potential reason that testing was not performed in 100% of infants, even at hospitals with CPGs that recommended testing for all patients. Additionally, some febrile infants diagnosed with SBI may not have an associated ICD‐9 diagnosis code for fever. Although the overall SBI rate observed in our study was similar to prior studies,[4, 31] the rate in neonates 28 days of age was lower than reported in recent investigations,[2, 3] which may indicate inclusion of a higher proportion of low‐risk febrile infants. With the exception of bronchiolitis, we also did not assess diagnostic testing in the presence of other identified sources of infection such as herpes simplex virus.

Second, we were unable to assess the presence or absence of a CPG at the 4 excluded EDs that did not respond to the survey or the institutions excluded for data‐quality issues. However, included and excluded hospitals did not differ in region or annual ED volume (data not shown).

Third, although we classified hospitals based upon the presence and content of CPGs, we were unable to fully evaluate adherence to the CPG at each site.

Last, though PHIS hospitals represent 85% of freestanding children's hospitals, many febrile infants are hospitalized at non‐PHIS institutions; our results may not be generalizable to care provided at nonchildren's hospitals.

CONCLUSIONS

Management of febrile neonates 28 days of age does not vary based on CPG presence. However, CPGs for the febrile infant aged 29 to 56 days vary in recommendations for CSF testing as well as ceftriaxone use for low‐risk patients, which significantly contributes to practice variation and healthcare costs across institutions.

Disclosures

This project was funded in part by The Gerber Foundation Novice Researcher Award (Ref #18273835). Dr. Fran Balamuth received career development support from the National Institutes of Health (NHLBI K12‐HL109009). Funders were not involved in design or conduct of the study; collection, management, analysis, or interpretation of the data; preparation, review, or approval of the manuscript. The authors have no financial relationships relevant to this article to disclose. No payment was received for the production of this article. The authors have no conflicts of interest relevant to this article to disclose.

Bronchiolitis is the most common cause of hospitalization in infancy, with estimated annual US costs of over $1.7 billion.[1] The last 2 decades have seen numerous thoughtful and well‐designed research studies but little improvement in the value of care.[1, 2, 3, 4] The diagnosis and treatment section of the recently released 2014 American Academy of Pediatrics (AAP) Clinical Practice Guideline for bronchiolitis contains 7 should not's and 3 should's,[3] with the only clear affirmative recommendations related to the history and physical and to the use of supplemental fluids. As supported by several systematic reviews and randomized controlled trials, the use of respiratory treatments, including ‐agonists, racemic epinephrine, and hypertonic saline, was discouraged. There continues to be significant variation in care for patients with bronchiolitis[5, 6] and rigorous evidence was lacking on when a child could be safely discharged home.

Mansbach and colleagues in the Multicenter Airway Research Collaboration (MARC‐30) provide the best evidence to date on the clinical course of bronchiolitis and present multicenter data upon which to build evidence‐based discharge criteria.[7] In their prospective cohort study of 16 US children's hospitals, Mansbach et al. sought to answer 3 research questions: (1) In infants hospitalized with bronchiolitis, what is the time to clinical improvement? (2) What is the risk of clinical worsening after standardized improvement criteria are met? (3) What discharge criteria might balance both timely discharge and very low readmission risk? In an analytic cohort of 1916 children <2 years of age with a physician diagnosis of bronchiolitis, the time from onset of difficulty breathing until clinical improvement was a median of 4 days, with a 75th percentile of 7.5 days. Of the 1702 children who clinically improved before discharge, only 76 (4%) then worsened. Although there are some limitations to how these criteria were assessed, the authors' work supports discharge criteria of (1) no or mild and stable or improving retractions, (2) stable or improving respiratory rate that is below the 90th percentile for age, (3) estimated room air saturation of 90% without any points <88%, and (4) clinician assessment of the child maintaining adequate oral hydration, regardless of use of intravenous fluids.

Three limitations warrant consideration when interpreting the study results. First, the MARC‐30 investigators oversampled from the intensive care unit and excluded 109 children with a hospital length of stay (LOS) <1 day. Although it is uncertain what effect these decisions would have on worsening after improving, both would overestimate the LOS in the sampled population at study hospitals. It is likely that the median LOS and 75th percentile of 4 and 7.5 days, respectively, are higher than what hospital medicine physicians saw at these hospitals. Second, the study team did not use a scoring tool. The authors note that the holistic assessments clinicians used to estimate respiratory rate and oxygen saturation may be more similar to standard clinical practice more than a calculated mean. This raises an important question: If less numerous data might lead to more information and knowledge, might they also lead to reliability and validity concerns? Given an absence of a structured, validated assessment of these severity indicators, it seems possible clinicians worked backward from the holistic assessment of this child is ready to go home and then entered data to support their larger assessment. This would tend to bias toward lower proportions of worsening after clinical improvement. Third, the once‐daily review of the medical record led to less precise estimates of each event including time from difficulty breathing to improvement and LOS. In addition to the absence of a scoring tool, this likely adds a modest bias toward underdetection of clinical worsening after improvement, because observations from discharged children were effectively censored from analysis. Importantly the low readmission rates suggest neither of those biases is substantial.

Several of the findings in this article support recent changes to the recommendations in the 2014 AAP Bronchiolitis Clinical Practice Guideline.[3] Although there is no recommendation on discharge readiness, Mansbach and colleagues found that an operationalization of the core criteria outlined in the 2006 version of the AAP Bronchiolitis Clinical Practice Guideline would result in a low proportion of subsequent clinical worsening.[8] This study also informs and supports an additional change to the AAP's 2006 guideline recommendation on continuous pulse oximetry. Key Action Statement 6b in the 2014 guideline notes Clinicians may choose not to use continuous pulse oximetry for infants and children with a diagnosis of bronchiolitis, expanding the recommendation from the 2006 guideline discouraging continuous pulse oximetry as the child's clinical course improves.[3, 8] Mansbach and colleagues found that removing the lower desaturation threshold of 88% improved the percentage of children who met criteria, with no changes in proportion subsequently worsening. With an improvement criterion of average oxygen saturation threshold of 95%, less than half of the children met this criteria before discharge, and an increased percentage (5%) clinically worsened, presumably due to clinically inconsequential desaturations to 94%. The less stringent the pulse oximetry criteria, the better their improvement criteria performed. This study adds to the modest literature on how overuse of continuous pulse oximetry may prolong hospitalization, leading to nonvalue‐added care and potentially increasing the risk of iatrogenic harm.[9, 10, 11]

Another strength of this study is the extensive viral testing on nasal aspirates. The absence of an association between individual viral pathogen or coinfection on the risk of worsening after improving further supports the recommendation against viral testing. The authors also identified a large group of children with a very low risk of worsening after an improving course: children 2 months, born at term, and who did not present with severe retractions. This finding, which will resonate with clinicians who care for patients with bronchiolitis, provides additional data on a group likely to have short hospitalization and unlikely to benefit from therapies. It also identifies a group of children with increased risk of worsening, which could be targeted for future research efforts on therapies such as hypertonic saline and high‐flow nasal cannula, where the evidence is mixed and inconclusive.

Both the MARC‐30 study and the 2014 AAP guidelines are tremendous contributions to the scientific literature on this common, costly, and often frustrating disease for clinicians and families alike. More important, however, will be implementation and dissemination efforts to ensure children benefit from this new knowledge. After the 2006 AAP guidelines, there was some evidence of improved care[12] but remaining profound hospital‐level variation.[5] Immediate next steps to improve bronchiolitis care should include interventions to standardize evidence‐based discharge criteria and reduce the overuse of nonevidence‐based care. Local clinical practice guidelines aid in the early phases of standardization, but without work and willpower in the implementation and sustain phase, their effect may be modest.[13] This study and the new guideline raise several important T3[14] or how questions for pediatric hospital medicine clinicians, researchers, and improvers. First, how can evidence‐based discharge criteria, such as those presented here, be applied reliably and broadly at the point of care? White and colleagues at Cincinnati shared a strategy that will benefit from further testing and adaptation.[15] Second, how can continuous pulse oximetry be either greatly reduced or have its data put in a broader context to inform decision making? Relatedly, which strategy is more effective and for whom? Finally, what incentives at the hospital and policy level are most effective in helping physicians to choose wisely[16] and do less?

Answering these questions will be crucial to ensure the knowledge produced from Mansbach and colleagues benefits the hundreds of thousands of children hospitalized with bronchiolitis each year.

Disclosure

Nothing to report.

Bronchiolitis is the most common cause of hospitalization in infancy, with estimated annual US costs of over $1.7 billion.[1] The last 2 decades have seen numerous thoughtful and well‐designed research studies but little improvement in the value of care.[1, 2, 3, 4] The diagnosis and treatment section of the recently released 2014 American Academy of Pediatrics (AAP) Clinical Practice Guideline for bronchiolitis contains 7 should not's and 3 should's,[3] with the only clear affirmative recommendations related to the history and physical and to the use of supplemental fluids. As supported by several systematic reviews and randomized controlled trials, the use of respiratory treatments, including ‐agonists, racemic epinephrine, and hypertonic saline, was discouraged. There continues to be significant variation in care for patients with bronchiolitis[5, 6] and rigorous evidence was lacking on when a child could be safely discharged home.

Mansbach and colleagues in the Multicenter Airway Research Collaboration (MARC‐30) provide the best evidence to date on the clinical course of bronchiolitis and present multicenter data upon which to build evidence‐based discharge criteria.[7] In their prospective cohort study of 16 US children's hospitals, Mansbach et al. sought to answer 3 research questions: (1) In infants hospitalized with bronchiolitis, what is the time to clinical improvement? (2) What is the risk of clinical worsening after standardized improvement criteria are met? (3) What discharge criteria might balance both timely discharge and very low readmission risk? In an analytic cohort of 1916 children <2 years of age with a physician diagnosis of bronchiolitis, the time from onset of difficulty breathing until clinical improvement was a median of 4 days, with a 75th percentile of 7.5 days. Of the 1702 children who clinically improved before discharge, only 76 (4%) then worsened. Although there are some limitations to how these criteria were assessed, the authors' work supports discharge criteria of (1) no or mild and stable or improving retractions, (2) stable or improving respiratory rate that is below the 90th percentile for age, (3) estimated room air saturation of 90% without any points <88%, and (4) clinician assessment of the child maintaining adequate oral hydration, regardless of use of intravenous fluids.

Three limitations warrant consideration when interpreting the study results. First, the MARC‐30 investigators oversampled from the intensive care unit and excluded 109 children with a hospital length of stay (LOS) <1 day. Although it is uncertain what effect these decisions would have on worsening after improving, both would overestimate the LOS in the sampled population at study hospitals. It is likely that the median LOS and 75th percentile of 4 and 7.5 days, respectively, are higher than what hospital medicine physicians saw at these hospitals. Second, the study team did not use a scoring tool. The authors note that the holistic assessments clinicians used to estimate respiratory rate and oxygen saturation may be more similar to standard clinical practice more than a calculated mean. This raises an important question: If less numerous data might lead to more information and knowledge, might they also lead to reliability and validity concerns? Given an absence of a structured, validated assessment of these severity indicators, it seems possible clinicians worked backward from the holistic assessment of this child is ready to go home and then entered data to support their larger assessment. This would tend to bias toward lower proportions of worsening after clinical improvement. Third, the once‐daily review of the medical record led to less precise estimates of each event including time from difficulty breathing to improvement and LOS. In addition to the absence of a scoring tool, this likely adds a modest bias toward underdetection of clinical worsening after improvement, because observations from discharged children were effectively censored from analysis. Importantly the low readmission rates suggest neither of those biases is substantial.

Several of the findings in this article support recent changes to the recommendations in the 2014 AAP Bronchiolitis Clinical Practice Guideline.[3] Although there is no recommendation on discharge readiness, Mansbach and colleagues found that an operationalization of the core criteria outlined in the 2006 version of the AAP Bronchiolitis Clinical Practice Guideline would result in a low proportion of subsequent clinical worsening.[8] This study also informs and supports an additional change to the AAP's 2006 guideline recommendation on continuous pulse oximetry. Key Action Statement 6b in the 2014 guideline notes Clinicians may choose not to use continuous pulse oximetry for infants and children with a diagnosis of bronchiolitis, expanding the recommendation from the 2006 guideline discouraging continuous pulse oximetry as the child's clinical course improves.[3, 8] Mansbach and colleagues found that removing the lower desaturation threshold of 88% improved the percentage of children who met criteria, with no changes in proportion subsequently worsening. With an improvement criterion of average oxygen saturation threshold of 95%, less than half of the children met this criteria before discharge, and an increased percentage (5%) clinically worsened, presumably due to clinically inconsequential desaturations to 94%. The less stringent the pulse oximetry criteria, the better their improvement criteria performed. This study adds to the modest literature on how overuse of continuous pulse oximetry may prolong hospitalization, leading to nonvalue‐added care and potentially increasing the risk of iatrogenic harm.[9, 10, 11]

Another strength of this study is the extensive viral testing on nasal aspirates. The absence of an association between individual viral pathogen or coinfection on the risk of worsening after improving further supports the recommendation against viral testing. The authors also identified a large group of children with a very low risk of worsening after an improving course: children 2 months, born at term, and who did not present with severe retractions. This finding, which will resonate with clinicians who care for patients with bronchiolitis, provides additional data on a group likely to have short hospitalization and unlikely to benefit from therapies. It also identifies a group of children with increased risk of worsening, which could be targeted for future research efforts on therapies such as hypertonic saline and high‐flow nasal cannula, where the evidence is mixed and inconclusive.

Both the MARC‐30 study and the 2014 AAP guidelines are tremendous contributions to the scientific literature on this common, costly, and often frustrating disease for clinicians and families alike. More important, however, will be implementation and dissemination efforts to ensure children benefit from this new knowledge. After the 2006 AAP guidelines, there was some evidence of improved care[12] but remaining profound hospital‐level variation.[5] Immediate next steps to improve bronchiolitis care should include interventions to standardize evidence‐based discharge criteria and reduce the overuse of nonevidence‐based care. Local clinical practice guidelines aid in the early phases of standardization, but without work and willpower in the implementation and sustain phase, their effect may be modest.[13] This study and the new guideline raise several important T3[14] or how questions for pediatric hospital medicine clinicians, researchers, and improvers. First, how can evidence‐based discharge criteria, such as those presented here, be applied reliably and broadly at the point of care? White and colleagues at Cincinnati shared a strategy that will benefit from further testing and adaptation.[15] Second, how can continuous pulse oximetry be either greatly reduced or have its data put in a broader context to inform decision making? Relatedly, which strategy is more effective and for whom? Finally, what incentives at the hospital and policy level are most effective in helping physicians to choose wisely[16] and do less?

Answering these questions will be crucial to ensure the knowledge produced from Mansbach and colleagues benefits the hundreds of thousands of children hospitalized with bronchiolitis each year.

Disclosure

Nothing to report.

Bronchiolitis is the most common cause of hospitalization in infancy, with estimated annual US costs of over $1.7 billion.[1] The last 2 decades have seen numerous thoughtful and well‐designed research studies but little improvement in the value of care.[1, 2, 3, 4] The diagnosis and treatment section of the recently released 2014 American Academy of Pediatrics (AAP) Clinical Practice Guideline for bronchiolitis contains 7 should not's and 3 should's,[3] with the only clear affirmative recommendations related to the history and physical and to the use of supplemental fluids. As supported by several systematic reviews and randomized controlled trials, the use of respiratory treatments, including ‐agonists, racemic epinephrine, and hypertonic saline, was discouraged. There continues to be significant variation in care for patients with bronchiolitis[5, 6] and rigorous evidence was lacking on when a child could be safely discharged home.

Mansbach and colleagues in the Multicenter Airway Research Collaboration (MARC‐30) provide the best evidence to date on the clinical course of bronchiolitis and present multicenter data upon which to build evidence‐based discharge criteria.[7] In their prospective cohort study of 16 US children's hospitals, Mansbach et al. sought to answer 3 research questions: (1) In infants hospitalized with bronchiolitis, what is the time to clinical improvement? (2) What is the risk of clinical worsening after standardized improvement criteria are met? (3) What discharge criteria might balance both timely discharge and very low readmission risk? In an analytic cohort of 1916 children <2 years of age with a physician diagnosis of bronchiolitis, the time from onset of difficulty breathing until clinical improvement was a median of 4 days, with a 75th percentile of 7.5 days. Of the 1702 children who clinically improved before discharge, only 76 (4%) then worsened. Although there are some limitations to how these criteria were assessed, the authors' work supports discharge criteria of (1) no or mild and stable or improving retractions, (2) stable or improving respiratory rate that is below the 90th percentile for age, (3) estimated room air saturation of 90% without any points <88%, and (4) clinician assessment of the child maintaining adequate oral hydration, regardless of use of intravenous fluids.

Three limitations warrant consideration when interpreting the study results. First, the MARC‐30 investigators oversampled from the intensive care unit and excluded 109 children with a hospital length of stay (LOS) <1 day. Although it is uncertain what effect these decisions would have on worsening after improving, both would overestimate the LOS in the sampled population at study hospitals. It is likely that the median LOS and 75th percentile of 4 and 7.5 days, respectively, are higher than what hospital medicine physicians saw at these hospitals. Second, the study team did not use a scoring tool. The authors note that the holistic assessments clinicians used to estimate respiratory rate and oxygen saturation may be more similar to standard clinical practice more than a calculated mean. This raises an important question: If less numerous data might lead to more information and knowledge, might they also lead to reliability and validity concerns? Given an absence of a structured, validated assessment of these severity indicators, it seems possible clinicians worked backward from the holistic assessment of this child is ready to go home and then entered data to support their larger assessment. This would tend to bias toward lower proportions of worsening after clinical improvement. Third, the once‐daily review of the medical record led to less precise estimates of each event including time from difficulty breathing to improvement and LOS. In addition to the absence of a scoring tool, this likely adds a modest bias toward underdetection of clinical worsening after improvement, because observations from discharged children were effectively censored from analysis. Importantly the low readmission rates suggest neither of those biases is substantial.

Several of the findings in this article support recent changes to the recommendations in the 2014 AAP Bronchiolitis Clinical Practice Guideline.[3] Although there is no recommendation on discharge readiness, Mansbach and colleagues found that an operationalization of the core criteria outlined in the 2006 version of the AAP Bronchiolitis Clinical Practice Guideline would result in a low proportion of subsequent clinical worsening.[8] This study also informs and supports an additional change to the AAP's 2006 guideline recommendation on continuous pulse oximetry. Key Action Statement 6b in the 2014 guideline notes Clinicians may choose not to use continuous pulse oximetry for infants and children with a diagnosis of bronchiolitis, expanding the recommendation from the 2006 guideline discouraging continuous pulse oximetry as the child's clinical course improves.[3, 8] Mansbach and colleagues found that removing the lower desaturation threshold of 88% improved the percentage of children who met criteria, with no changes in proportion subsequently worsening. With an improvement criterion of average oxygen saturation threshold of 95%, less than half of the children met this criteria before discharge, and an increased percentage (5%) clinically worsened, presumably due to clinically inconsequential desaturations to 94%. The less stringent the pulse oximetry criteria, the better their improvement criteria performed. This study adds to the modest literature on how overuse of continuous pulse oximetry may prolong hospitalization, leading to nonvalue‐added care and potentially increasing the risk of iatrogenic harm.[9, 10, 11]

Another strength of this study is the extensive viral testing on nasal aspirates. The absence of an association between individual viral pathogen or coinfection on the risk of worsening after improving further supports the recommendation against viral testing. The authors also identified a large group of children with a very low risk of worsening after an improving course: children 2 months, born at term, and who did not present with severe retractions. This finding, which will resonate with clinicians who care for patients with bronchiolitis, provides additional data on a group likely to have short hospitalization and unlikely to benefit from therapies. It also identifies a group of children with increased risk of worsening, which could be targeted for future research efforts on therapies such as hypertonic saline and high‐flow nasal cannula, where the evidence is mixed and inconclusive.

Both the MARC‐30 study and the 2014 AAP guidelines are tremendous contributions to the scientific literature on this common, costly, and often frustrating disease for clinicians and families alike. More important, however, will be implementation and dissemination efforts to ensure children benefit from this new knowledge. After the 2006 AAP guidelines, there was some evidence of improved care[12] but remaining profound hospital‐level variation.[5] Immediate next steps to improve bronchiolitis care should include interventions to standardize evidence‐based discharge criteria and reduce the overuse of nonevidence‐based care. Local clinical practice guidelines aid in the early phases of standardization, but without work and willpower in the implementation and sustain phase, their effect may be modest.[13] This study and the new guideline raise several important T3[14] or how questions for pediatric hospital medicine clinicians, researchers, and improvers. First, how can evidence‐based discharge criteria, such as those presented here, be applied reliably and broadly at the point of care? White and colleagues at Cincinnati shared a strategy that will benefit from further testing and adaptation.[15] Second, how can continuous pulse oximetry be either greatly reduced or have its data put in a broader context to inform decision making? Relatedly, which strategy is more effective and for whom? Finally, what incentives at the hospital and policy level are most effective in helping physicians to choose wisely[16] and do less?

Answering these questions will be crucial to ensure the knowledge produced from Mansbach and colleagues benefits the hundreds of thousands of children hospitalized with bronchiolitis each year.

Disclosure

Nothing to report.