Erin Pete Devon, MD

As physicians strive to increase the value of healthcare delivery, there has been increased focus on improving the quality of care that patients receive while lowering per capita costs. A provision of the Affordable Care Act implemented in 2012 identified all‐cause 30‐day readmission rates as a measure of hospital quality, and as part of the Act's Hospital Readmission and Reduction Program, Medicare now penalizes hospitals with higher than expected all‐cause readmissions rates for adult patients with certain conditions by lowering reimbursements.[1] Although readmissions are not yet commonly used to determine reimbursements for pediatric hospitals, several states are penalizing higher than expected readmission rates for Medicaid enrollees,[2, 3] using an imprecise algorithm to determine which readmissions resulted from low‐quality care during the index admission.[4, 5, 6]

There is growing concern, however, that readmission rates are not an accurate gauge of the quality of care patients receive while in the hospital or during the discharge process to prepare them for their transition home.[7, 8, 9, 10] This is especially true in pediatric settings, where overall readmission rates are much lower than in adult settings, many readmissions are expected as part of a patient's planned course of care, and variation in readmission rates between hospitals is correlated with the percentage of patients with certain complex chronic conditions.[1, 7, 11] Thus, there is increasing agreement that hospitals and external evaluators need to shift the focus from all‐cause readmissions to a reliable, consistent, and fair measure of potentially preventable readmissions.[12, 13] In addition to being a more useful quality metric, analyzing preventable readmissions will help hospitals focus resources on patients with potentially modifiable risk factors and develop meaningful quality‐improvement initiatives to improve inpatient care as well as the discharge process to prepare families for their transition to home.[14]

Although previous studies have attempted to distinguish preventable from nonpreventable readmissions, many reported significant challenges in completing reviews efficiently, achieving consistency in how readmissions were classified, and attaining consensus on final determinations.[12, 13, 14] Studies have also demonstrated that the algorithms some states are using to streamline preventability reviews and determine reimbursements overestimate the rate of potentially preventable readmissions.[4, 5, 6]

To increase the efficiency of preventability reviews and reduce the subjectivity involved in reaching final determinations, while still accounting for the nuances necessary to conduct a fair review, a quality‐improvement team from the Division of General Pediatrics at The Children's Hospital of Philadelphia (CHOP) implemented a fault tree analysis tool based on a framework developed by Howard Parker at Intermountain Primary Children's Hospital. The CHOP team coded this framework into a secure Web‐based data‐collection tool in the form of a decision tree to guide reviewers through a logical progression of questions that result in 1 of 18 root causes of readmissions, 8 of which are considered potentially preventable. We hypothesized that this method would help reviewers efficiently reach consensus on the root causes of hospital readmissions, and thus help the division and the hospital focus efforts on developing relevant quality‐improvement initiatives.

METHODS

Fault Tree Analysis

Using the decision tree framework for analyzing readmissions that was developed at Intermountain Primary Children's Hospital, the REDCap tool prompted reviewers with a series of sequential questions, each with mutually exclusive options. Using imbedded branching logic to select follow‐up questions, the tool guided reviewers to 1 of 18 terminal nodes, each representing a potential root cause of the readmission. Of those 18 potential causes, 8 were considered potentially preventable. A diagram of the fault tree framework, color coded to indicate which nodes were considered potentially preventable, is shown in Figure 1.

jhm2555-fig-0001-m.png

RESULTS

In 2014, 7252 patients were discharged from the general pediatrics service at CHOP. Of those patients, 248 were readmitted within 15 days for an overall general pediatrics 15‐day readmission rate of 3.4%.

DISCUSSION

As part of an effort to direct quality‐improvement initiatives, this project used a Web‐based fault tree tool to identify root causes of general pediatrics readmissions at a freestanding children's hospital and classify them as either preventable or not preventable. The project also investigated the efficiency and inter‐rater reliability of the tool, which was designed to systematically guide physicians through the chart review process to a final determination about preventability. The project confirmed that using the tool helped reviewers reach final determinations about preventability efficiently with a high degree of consistency. It also confirmed that only a very small percentage of general pediatrics 15‐day readmissions are potentially preventable. Specifically, potentially preventable readmissions accounted for only 6.0% of total readmissions and 0.2% of general pediatrics discharges in 2014. Although our analysis focused on 15‐day readmissions, the fault tree methodology can be applied to any timeframe.

Previous studies attempting to distinguish preventable from nonpreventable readmissions, which used a range of methodologies to reach final determinations, reported that their review process was both time intensive and highly subjective. One study, which had 4 reviewers independently review charts and assign each case a preventability score on a 5‐point Likert scale, reported that reviewers disagreed on the final determination in 62.5% of cases.[12] Another study had 2 physicians independently review a selection of cases and assign a preventability score on a scale from 0 to 3. Scores for the 2 reviewers were added together, and cases above a certain composite threshold were classified as preventable. Despite being time‐intensive, this method resulted in only moderate agreement among physicians about the likelihood of preventability (weighted statistic of 0.44).[14] A more recent study, in which 2 physicians independently classified readmissions into 1 of 4 predefined categories, also reported only moderate agreement between reviewers ( = 0.44).[13] Other methods that have been reported include classifying readmissions as preventable only if multiple reviewers independently agreed, and using a third reviewer as a tie‐breaker.[14]

In an attempt to identify potentially preventable readmissions without using chart reviews, 3M (St. Paul, MN) developed its Potentially Preventable Readmissions software (3M‐PPR), which uses administrative data to identify which readmissions were potentially preventable. Although this automated approach is less time intensive, evidence suggests that due to a lack of nuance, the algorithm significantly overestimates the percentage of readmissions that are potentially preventable.[4, 5] A study that used 3M‐PPR to assess 1.7 million hospitalizations across 58 children's hospitals found that the algorithm classified 81% of sickle cell crisis and asthma readmissions, and 83% of bronchiolitis readmissions as potentially preventable.[10, 11] However, many readmissions for asthma and bronchiolitis are due to social factors that are outside of a hospital's direct control,[4, 5] and at many hospitals, readmissions for sickle cell crisis are part of a high‐value care model that weighs length of stay against potential readmissions. In addition, when assessing readmissions 7, 15, and 30 days after discharge, the algorithm classified almost the same percentage as potentially preventable, which is inconsistent with the notion that readmissions are more likely to have been preventable if they occurred closer to the initial discharge.[4, 13] Another study that assessed the performance of the software in the adult population reported that the algorithm performed with 85% sensitivity, but only 28% specificity.[5, 6]

The results of this quality‐improvement project indicate that using the fault tree tool to guide physicians through the chart review process helped address some of the shortcomings of methods reported in previous studies, by increasing the efficiency and reducing the subjectivity of final determinations, while still accounting for the nuances necessary to conduct a fair review. Because the tool provided a systematic framework for reviews, each case was completed in approximately 20 minutes, and because the process was the same for all reviewers, inter‐rater reliability was extremely high. In 86% of cross‐checked cases, the second reviewer ended at the same terminal node in the decision tree as the original reviewer, and in 98% of cross‐checked cases the second reviewer reached the same conclusion about preventability, even if they did not end at the same terminal node. Even accounting for agreement due to chance, the statistic of 0.79 confirmed that there was substantial agreement among reviewers about final determinations. Because the tool is easily adaptable, other hospitals can adopt this framework for their own preventability reviews and quality‐improvement initiatives.

Using the fault tree tool to access root causes of all 15‐day general pediatric readmissions helped the division focus quality‐improvement efforts on the most common causes of potentially preventable readmissions. Because 40% of potentially preventable readmissions were due to premature discharges, this prompted quality‐improvement teams to focus efforts on improving and clarifying the division's discharge criteria and clinical pathways. The division also initiated processes to improve discharge planning, including improved teaching of discharge instructions and having families pick up prescriptions prior to discharge.

Although these results did help the division identify a few areas of focus to potentially reduce readmissions, the fact that the overall 15‐day readmission rate for general pediatrics, as well as the percentage of readmissions and total discharges that were deemed potentially preventable, were so low (3.4%, 6.0%, and 0.2%, respectively), supports those who question whether prioritizing pediatric readmissions is the best place for hospitals to focus quality‐improvement efforts.[10, 12, 15, 16] As these results indicate, most pediatric readmissions are not preventable, and thus consistent with an efficient, effective, timely, patient‐centered, and equitable health system. Other studies have also shown that because overall and condition‐specific readmissions at pediatric hospitals are low, few pediatric hospitals are high or low performing for readmissions, and thus readmission rates are likely not a good measure of hospital quality.[8]

However, other condition‐specific studies of readmissions in pediatrics have indicated that there are some areas of opportunity to identify populations at high risk for readmission. One study found that although pneumonia‐specific 30‐day readmission rates in a national cohort of children hospitalized with pneumonia was only 3.1%, the chances of readmission were higher for children <1 year old, children with chronic comorbidities or complicated pneumonia, and children cared for in hospitals with lower volumes of pneumonia admissions.[17] Another study found that 17.1% of adolescents in a statewide database were readmitted post‐tonsillectomy for pain, nausea, and dehydration.[18] Thus, adapting the tool to identify root causes of condition‐specific or procedure‐specific readmissions, especially for surgical patients, may be an area of opportunity for future quality‐improvement efforts.[5] However, for general pediatrics, shifting the focus from reducing readmissions to improving the quality of care patients receive in the hospital, improving the discharge process, and adopting a population health approach to mitigate external risk factors, may be appropriate.

This project was subject to limitations. First, because it was conducted at a single site and only on general pediatrics patients, results may not be generalizable to other hospitals or other pediatric divisions. Thus, future studies might use the fault tree framework to assess preventability of pediatric readmissions in other divisions or specialties. Second, because readmissions to other hospitals were not included in the sample, the overall readmissions rate is likely underestimated.[19] However, it is unclear how this would affect the rate of potentially preventable readmissions. Third, although the fault tree framework reduced the subjectivity of the review process, there is still a degree of subjectivity inherent at each decision node. To minimize this, reviewers should try to discuss and come to consensus on how they are making determinations at each juncture in the decision tree. Similarly, because reviewers' answers to decision‐tree questions rely heavily on chart documentation, reviews may be compromised by unclear or incomplete documentation. For example, if information about steps the hospital team took to prepare a family for discharge were not properly documented, it would be difficult to determine whether appropriate steps were taken to minimize the likelihood of a complication. In the case of insufficient documentation of relevant social concerns, cases may be incorrectly classified as preventable, because addressing social issues is often not within a hospital's direct control. Finally, because reviewers were not blinded to the original discharging physician, there may have been some unconscious bias of unknown direction in the reviews.

CONCLUSION

Using the Web‐based fault tree tool helped physicians to identify the root causes of hospital readmissions and classify them as preventable or not preventable in a standardized, efficient, and consistent way, while still accounting for the nuances necessary to conduct a fair review. Thus, other hospitals should consider adopting this framework for their own preventability reviews and quality‐improvement initiatives. However, this project also confirmed that only a very small percentage of general pediatrics 15‐day readmissions are potentially preventable, suggesting that general pediatrics readmissions are not an appropriate measure of hospital quality. Instead, adapting the tool to identify root causes of condition‐specific or procedure‐specific readmission rates may be an area of opportunity for future quality‐improvement efforts.

Disclosures: This work was supported through internal funds from The Children's Hospital of Philadelphia. The authors have no financial interests, relationships or affiliations relevant to the subject matter or materials discussed in the article to disclose. The authors have no potential conflicts of interest relevant to the subject matter or materials discussed in the article to disclose.

As physicians strive to increase the value of healthcare delivery, there has been increased focus on improving the quality of care that patients receive while lowering per capita costs. A provision of the Affordable Care Act implemented in 2012 identified all‐cause 30‐day readmission rates as a measure of hospital quality, and as part of the Act's Hospital Readmission and Reduction Program, Medicare now penalizes hospitals with higher than expected all‐cause readmissions rates for adult patients with certain conditions by lowering reimbursements.[1] Although readmissions are not yet commonly used to determine reimbursements for pediatric hospitals, several states are penalizing higher than expected readmission rates for Medicaid enrollees,[2, 3] using an imprecise algorithm to determine which readmissions resulted from low‐quality care during the index admission.[4, 5, 6]

There is growing concern, however, that readmission rates are not an accurate gauge of the quality of care patients receive while in the hospital or during the discharge process to prepare them for their transition home.[7, 8, 9, 10] This is especially true in pediatric settings, where overall readmission rates are much lower than in adult settings, many readmissions are expected as part of a patient's planned course of care, and variation in readmission rates between hospitals is correlated with the percentage of patients with certain complex chronic conditions.[1, 7, 11] Thus, there is increasing agreement that hospitals and external evaluators need to shift the focus from all‐cause readmissions to a reliable, consistent, and fair measure of potentially preventable readmissions.[12, 13] In addition to being a more useful quality metric, analyzing preventable readmissions will help hospitals focus resources on patients with potentially modifiable risk factors and develop meaningful quality‐improvement initiatives to improve inpatient care as well as the discharge process to prepare families for their transition to home.[14]

Although previous studies have attempted to distinguish preventable from nonpreventable readmissions, many reported significant challenges in completing reviews efficiently, achieving consistency in how readmissions were classified, and attaining consensus on final determinations.[12, 13, 14] Studies have also demonstrated that the algorithms some states are using to streamline preventability reviews and determine reimbursements overestimate the rate of potentially preventable readmissions.[4, 5, 6]

To increase the efficiency of preventability reviews and reduce the subjectivity involved in reaching final determinations, while still accounting for the nuances necessary to conduct a fair review, a quality‐improvement team from the Division of General Pediatrics at The Children's Hospital of Philadelphia (CHOP) implemented a fault tree analysis tool based on a framework developed by Howard Parker at Intermountain Primary Children's Hospital. The CHOP team coded this framework into a secure Web‐based data‐collection tool in the form of a decision tree to guide reviewers through a logical progression of questions that result in 1 of 18 root causes of readmissions, 8 of which are considered potentially preventable. We hypothesized that this method would help reviewers efficiently reach consensus on the root causes of hospital readmissions, and thus help the division and the hospital focus efforts on developing relevant quality‐improvement initiatives.

METHODS

Fault Tree Analysis

Using the decision tree framework for analyzing readmissions that was developed at Intermountain Primary Children's Hospital, the REDCap tool prompted reviewers with a series of sequential questions, each with mutually exclusive options. Using imbedded branching logic to select follow‐up questions, the tool guided reviewers to 1 of 18 terminal nodes, each representing a potential root cause of the readmission. Of those 18 potential causes, 8 were considered potentially preventable. A diagram of the fault tree framework, color coded to indicate which nodes were considered potentially preventable, is shown in Figure 1.

jhm2555-fig-0001-m.png

RESULTS

In 2014, 7252 patients were discharged from the general pediatrics service at CHOP. Of those patients, 248 were readmitted within 15 days for an overall general pediatrics 15‐day readmission rate of 3.4%.

DISCUSSION

As part of an effort to direct quality‐improvement initiatives, this project used a Web‐based fault tree tool to identify root causes of general pediatrics readmissions at a freestanding children's hospital and classify them as either preventable or not preventable. The project also investigated the efficiency and inter‐rater reliability of the tool, which was designed to systematically guide physicians through the chart review process to a final determination about preventability. The project confirmed that using the tool helped reviewers reach final determinations about preventability efficiently with a high degree of consistency. It also confirmed that only a very small percentage of general pediatrics 15‐day readmissions are potentially preventable. Specifically, potentially preventable readmissions accounted for only 6.0% of total readmissions and 0.2% of general pediatrics discharges in 2014. Although our analysis focused on 15‐day readmissions, the fault tree methodology can be applied to any timeframe.

Previous studies attempting to distinguish preventable from nonpreventable readmissions, which used a range of methodologies to reach final determinations, reported that their review process was both time intensive and highly subjective. One study, which had 4 reviewers independently review charts and assign each case a preventability score on a 5‐point Likert scale, reported that reviewers disagreed on the final determination in 62.5% of cases.[12] Another study had 2 physicians independently review a selection of cases and assign a preventability score on a scale from 0 to 3. Scores for the 2 reviewers were added together, and cases above a certain composite threshold were classified as preventable. Despite being time‐intensive, this method resulted in only moderate agreement among physicians about the likelihood of preventability (weighted statistic of 0.44).[14] A more recent study, in which 2 physicians independently classified readmissions into 1 of 4 predefined categories, also reported only moderate agreement between reviewers ( = 0.44).[13] Other methods that have been reported include classifying readmissions as preventable only if multiple reviewers independently agreed, and using a third reviewer as a tie‐breaker.[14]

In an attempt to identify potentially preventable readmissions without using chart reviews, 3M (St. Paul, MN) developed its Potentially Preventable Readmissions software (3M‐PPR), which uses administrative data to identify which readmissions were potentially preventable. Although this automated approach is less time intensive, evidence suggests that due to a lack of nuance, the algorithm significantly overestimates the percentage of readmissions that are potentially preventable.[4, 5] A study that used 3M‐PPR to assess 1.7 million hospitalizations across 58 children's hospitals found that the algorithm classified 81% of sickle cell crisis and asthma readmissions, and 83% of bronchiolitis readmissions as potentially preventable.[10, 11] However, many readmissions for asthma and bronchiolitis are due to social factors that are outside of a hospital's direct control,[4, 5] and at many hospitals, readmissions for sickle cell crisis are part of a high‐value care model that weighs length of stay against potential readmissions. In addition, when assessing readmissions 7, 15, and 30 days after discharge, the algorithm classified almost the same percentage as potentially preventable, which is inconsistent with the notion that readmissions are more likely to have been preventable if they occurred closer to the initial discharge.[4, 13] Another study that assessed the performance of the software in the adult population reported that the algorithm performed with 85% sensitivity, but only 28% specificity.[5, 6]

The results of this quality‐improvement project indicate that using the fault tree tool to guide physicians through the chart review process helped address some of the shortcomings of methods reported in previous studies, by increasing the efficiency and reducing the subjectivity of final determinations, while still accounting for the nuances necessary to conduct a fair review. Because the tool provided a systematic framework for reviews, each case was completed in approximately 20 minutes, and because the process was the same for all reviewers, inter‐rater reliability was extremely high. In 86% of cross‐checked cases, the second reviewer ended at the same terminal node in the decision tree as the original reviewer, and in 98% of cross‐checked cases the second reviewer reached the same conclusion about preventability, even if they did not end at the same terminal node. Even accounting for agreement due to chance, the statistic of 0.79 confirmed that there was substantial agreement among reviewers about final determinations. Because the tool is easily adaptable, other hospitals can adopt this framework for their own preventability reviews and quality‐improvement initiatives.

Using the fault tree tool to access root causes of all 15‐day general pediatric readmissions helped the division focus quality‐improvement efforts on the most common causes of potentially preventable readmissions. Because 40% of potentially preventable readmissions were due to premature discharges, this prompted quality‐improvement teams to focus efforts on improving and clarifying the division's discharge criteria and clinical pathways. The division also initiated processes to improve discharge planning, including improved teaching of discharge instructions and having families pick up prescriptions prior to discharge.

Although these results did help the division identify a few areas of focus to potentially reduce readmissions, the fact that the overall 15‐day readmission rate for general pediatrics, as well as the percentage of readmissions and total discharges that were deemed potentially preventable, were so low (3.4%, 6.0%, and 0.2%, respectively), supports those who question whether prioritizing pediatric readmissions is the best place for hospitals to focus quality‐improvement efforts.[10, 12, 15, 16] As these results indicate, most pediatric readmissions are not preventable, and thus consistent with an efficient, effective, timely, patient‐centered, and equitable health system. Other studies have also shown that because overall and condition‐specific readmissions at pediatric hospitals are low, few pediatric hospitals are high or low performing for readmissions, and thus readmission rates are likely not a good measure of hospital quality.[8]

However, other condition‐specific studies of readmissions in pediatrics have indicated that there are some areas of opportunity to identify populations at high risk for readmission. One study found that although pneumonia‐specific 30‐day readmission rates in a national cohort of children hospitalized with pneumonia was only 3.1%, the chances of readmission were higher for children <1 year old, children with chronic comorbidities or complicated pneumonia, and children cared for in hospitals with lower volumes of pneumonia admissions.[17] Another study found that 17.1% of adolescents in a statewide database were readmitted post‐tonsillectomy for pain, nausea, and dehydration.[18] Thus, adapting the tool to identify root causes of condition‐specific or procedure‐specific readmissions, especially for surgical patients, may be an area of opportunity for future quality‐improvement efforts.[5] However, for general pediatrics, shifting the focus from reducing readmissions to improving the quality of care patients receive in the hospital, improving the discharge process, and adopting a population health approach to mitigate external risk factors, may be appropriate.

This project was subject to limitations. First, because it was conducted at a single site and only on general pediatrics patients, results may not be generalizable to other hospitals or other pediatric divisions. Thus, future studies might use the fault tree framework to assess preventability of pediatric readmissions in other divisions or specialties. Second, because readmissions to other hospitals were not included in the sample, the overall readmissions rate is likely underestimated.[19] However, it is unclear how this would affect the rate of potentially preventable readmissions. Third, although the fault tree framework reduced the subjectivity of the review process, there is still a degree of subjectivity inherent at each decision node. To minimize this, reviewers should try to discuss and come to consensus on how they are making determinations at each juncture in the decision tree. Similarly, because reviewers' answers to decision‐tree questions rely heavily on chart documentation, reviews may be compromised by unclear or incomplete documentation. For example, if information about steps the hospital team took to prepare a family for discharge were not properly documented, it would be difficult to determine whether appropriate steps were taken to minimize the likelihood of a complication. In the case of insufficient documentation of relevant social concerns, cases may be incorrectly classified as preventable, because addressing social issues is often not within a hospital's direct control. Finally, because reviewers were not blinded to the original discharging physician, there may have been some unconscious bias of unknown direction in the reviews.

CONCLUSION

Using the Web‐based fault tree tool helped physicians to identify the root causes of hospital readmissions and classify them as preventable or not preventable in a standardized, efficient, and consistent way, while still accounting for the nuances necessary to conduct a fair review. Thus, other hospitals should consider adopting this framework for their own preventability reviews and quality‐improvement initiatives. However, this project also confirmed that only a very small percentage of general pediatrics 15‐day readmissions are potentially preventable, suggesting that general pediatrics readmissions are not an appropriate measure of hospital quality. Instead, adapting the tool to identify root causes of condition‐specific or procedure‐specific readmission rates may be an area of opportunity for future quality‐improvement efforts.

Disclosures: This work was supported through internal funds from The Children's Hospital of Philadelphia. The authors have no financial interests, relationships or affiliations relevant to the subject matter or materials discussed in the article to disclose. The authors have no potential conflicts of interest relevant to the subject matter or materials discussed in the article to disclose.

As physicians strive to increase the value of healthcare delivery, there has been increased focus on improving the quality of care that patients receive while lowering per capita costs. A provision of the Affordable Care Act implemented in 2012 identified all‐cause 30‐day readmission rates as a measure of hospital quality, and as part of the Act's Hospital Readmission and Reduction Program, Medicare now penalizes hospitals with higher than expected all‐cause readmissions rates for adult patients with certain conditions by lowering reimbursements.[1] Although readmissions are not yet commonly used to determine reimbursements for pediatric hospitals, several states are penalizing higher than expected readmission rates for Medicaid enrollees,[2, 3] using an imprecise algorithm to determine which readmissions resulted from low‐quality care during the index admission.[4, 5, 6]

There is growing concern, however, that readmission rates are not an accurate gauge of the quality of care patients receive while in the hospital or during the discharge process to prepare them for their transition home.[7, 8, 9, 10] This is especially true in pediatric settings, where overall readmission rates are much lower than in adult settings, many readmissions are expected as part of a patient's planned course of care, and variation in readmission rates between hospitals is correlated with the percentage of patients with certain complex chronic conditions.[1, 7, 11] Thus, there is increasing agreement that hospitals and external evaluators need to shift the focus from all‐cause readmissions to a reliable, consistent, and fair measure of potentially preventable readmissions.[12, 13] In addition to being a more useful quality metric, analyzing preventable readmissions will help hospitals focus resources on patients with potentially modifiable risk factors and develop meaningful quality‐improvement initiatives to improve inpatient care as well as the discharge process to prepare families for their transition to home.[14]

Although previous studies have attempted to distinguish preventable from nonpreventable readmissions, many reported significant challenges in completing reviews efficiently, achieving consistency in how readmissions were classified, and attaining consensus on final determinations.[12, 13, 14] Studies have also demonstrated that the algorithms some states are using to streamline preventability reviews and determine reimbursements overestimate the rate of potentially preventable readmissions.[4, 5, 6]

To increase the efficiency of preventability reviews and reduce the subjectivity involved in reaching final determinations, while still accounting for the nuances necessary to conduct a fair review, a quality‐improvement team from the Division of General Pediatrics at The Children's Hospital of Philadelphia (CHOP) implemented a fault tree analysis tool based on a framework developed by Howard Parker at Intermountain Primary Children's Hospital. The CHOP team coded this framework into a secure Web‐based data‐collection tool in the form of a decision tree to guide reviewers through a logical progression of questions that result in 1 of 18 root causes of readmissions, 8 of which are considered potentially preventable. We hypothesized that this method would help reviewers efficiently reach consensus on the root causes of hospital readmissions, and thus help the division and the hospital focus efforts on developing relevant quality‐improvement initiatives.

METHODS

Fault Tree Analysis

Using the decision tree framework for analyzing readmissions that was developed at Intermountain Primary Children's Hospital, the REDCap tool prompted reviewers with a series of sequential questions, each with mutually exclusive options. Using imbedded branching logic to select follow‐up questions, the tool guided reviewers to 1 of 18 terminal nodes, each representing a potential root cause of the readmission. Of those 18 potential causes, 8 were considered potentially preventable. A diagram of the fault tree framework, color coded to indicate which nodes were considered potentially preventable, is shown in Figure 1.

jhm2555-fig-0001-m.png

RESULTS

In 2014, 7252 patients were discharged from the general pediatrics service at CHOP. Of those patients, 248 were readmitted within 15 days for an overall general pediatrics 15‐day readmission rate of 3.4%.

DISCUSSION

As part of an effort to direct quality‐improvement initiatives, this project used a Web‐based fault tree tool to identify root causes of general pediatrics readmissions at a freestanding children's hospital and classify them as either preventable or not preventable. The project also investigated the efficiency and inter‐rater reliability of the tool, which was designed to systematically guide physicians through the chart review process to a final determination about preventability. The project confirmed that using the tool helped reviewers reach final determinations about preventability efficiently with a high degree of consistency. It also confirmed that only a very small percentage of general pediatrics 15‐day readmissions are potentially preventable. Specifically, potentially preventable readmissions accounted for only 6.0% of total readmissions and 0.2% of general pediatrics discharges in 2014. Although our analysis focused on 15‐day readmissions, the fault tree methodology can be applied to any timeframe.

Previous studies attempting to distinguish preventable from nonpreventable readmissions, which used a range of methodologies to reach final determinations, reported that their review process was both time intensive and highly subjective. One study, which had 4 reviewers independently review charts and assign each case a preventability score on a 5‐point Likert scale, reported that reviewers disagreed on the final determination in 62.5% of cases.[12] Another study had 2 physicians independently review a selection of cases and assign a preventability score on a scale from 0 to 3. Scores for the 2 reviewers were added together, and cases above a certain composite threshold were classified as preventable. Despite being time‐intensive, this method resulted in only moderate agreement among physicians about the likelihood of preventability (weighted statistic of 0.44).[14] A more recent study, in which 2 physicians independently classified readmissions into 1 of 4 predefined categories, also reported only moderate agreement between reviewers ( = 0.44).[13] Other methods that have been reported include classifying readmissions as preventable only if multiple reviewers independently agreed, and using a third reviewer as a tie‐breaker.[14]

In an attempt to identify potentially preventable readmissions without using chart reviews, 3M (St. Paul, MN) developed its Potentially Preventable Readmissions software (3M‐PPR), which uses administrative data to identify which readmissions were potentially preventable. Although this automated approach is less time intensive, evidence suggests that due to a lack of nuance, the algorithm significantly overestimates the percentage of readmissions that are potentially preventable.[4, 5] A study that used 3M‐PPR to assess 1.7 million hospitalizations across 58 children's hospitals found that the algorithm classified 81% of sickle cell crisis and asthma readmissions, and 83% of bronchiolitis readmissions as potentially preventable.[10, 11] However, many readmissions for asthma and bronchiolitis are due to social factors that are outside of a hospital's direct control,[4, 5] and at many hospitals, readmissions for sickle cell crisis are part of a high‐value care model that weighs length of stay against potential readmissions. In addition, when assessing readmissions 7, 15, and 30 days after discharge, the algorithm classified almost the same percentage as potentially preventable, which is inconsistent with the notion that readmissions are more likely to have been preventable if they occurred closer to the initial discharge.[4, 13] Another study that assessed the performance of the software in the adult population reported that the algorithm performed with 85% sensitivity, but only 28% specificity.[5, 6]

The results of this quality‐improvement project indicate that using the fault tree tool to guide physicians through the chart review process helped address some of the shortcomings of methods reported in previous studies, by increasing the efficiency and reducing the subjectivity of final determinations, while still accounting for the nuances necessary to conduct a fair review. Because the tool provided a systematic framework for reviews, each case was completed in approximately 20 minutes, and because the process was the same for all reviewers, inter‐rater reliability was extremely high. In 86% of cross‐checked cases, the second reviewer ended at the same terminal node in the decision tree as the original reviewer, and in 98% of cross‐checked cases the second reviewer reached the same conclusion about preventability, even if they did not end at the same terminal node. Even accounting for agreement due to chance, the statistic of 0.79 confirmed that there was substantial agreement among reviewers about final determinations. Because the tool is easily adaptable, other hospitals can adopt this framework for their own preventability reviews and quality‐improvement initiatives.

Using the fault tree tool to access root causes of all 15‐day general pediatric readmissions helped the division focus quality‐improvement efforts on the most common causes of potentially preventable readmissions. Because 40% of potentially preventable readmissions were due to premature discharges, this prompted quality‐improvement teams to focus efforts on improving and clarifying the division's discharge criteria and clinical pathways. The division also initiated processes to improve discharge planning, including improved teaching of discharge instructions and having families pick up prescriptions prior to discharge.

Although these results did help the division identify a few areas of focus to potentially reduce readmissions, the fact that the overall 15‐day readmission rate for general pediatrics, as well as the percentage of readmissions and total discharges that were deemed potentially preventable, were so low (3.4%, 6.0%, and 0.2%, respectively), supports those who question whether prioritizing pediatric readmissions is the best place for hospitals to focus quality‐improvement efforts.[10, 12, 15, 16] As these results indicate, most pediatric readmissions are not preventable, and thus consistent with an efficient, effective, timely, patient‐centered, and equitable health system. Other studies have also shown that because overall and condition‐specific readmissions at pediatric hospitals are low, few pediatric hospitals are high or low performing for readmissions, and thus readmission rates are likely not a good measure of hospital quality.[8]

However, other condition‐specific studies of readmissions in pediatrics have indicated that there are some areas of opportunity to identify populations at high risk for readmission. One study found that although pneumonia‐specific 30‐day readmission rates in a national cohort of children hospitalized with pneumonia was only 3.1%, the chances of readmission were higher for children <1 year old, children with chronic comorbidities or complicated pneumonia, and children cared for in hospitals with lower volumes of pneumonia admissions.[17] Another study found that 17.1% of adolescents in a statewide database were readmitted post‐tonsillectomy for pain, nausea, and dehydration.[18] Thus, adapting the tool to identify root causes of condition‐specific or procedure‐specific readmissions, especially for surgical patients, may be an area of opportunity for future quality‐improvement efforts.[5] However, for general pediatrics, shifting the focus from reducing readmissions to improving the quality of care patients receive in the hospital, improving the discharge process, and adopting a population health approach to mitigate external risk factors, may be appropriate.

This project was subject to limitations. First, because it was conducted at a single site and only on general pediatrics patients, results may not be generalizable to other hospitals or other pediatric divisions. Thus, future studies might use the fault tree framework to assess preventability of pediatric readmissions in other divisions or specialties. Second, because readmissions to other hospitals were not included in the sample, the overall readmissions rate is likely underestimated.[19] However, it is unclear how this would affect the rate of potentially preventable readmissions. Third, although the fault tree framework reduced the subjectivity of the review process, there is still a degree of subjectivity inherent at each decision node. To minimize this, reviewers should try to discuss and come to consensus on how they are making determinations at each juncture in the decision tree. Similarly, because reviewers' answers to decision‐tree questions rely heavily on chart documentation, reviews may be compromised by unclear or incomplete documentation. For example, if information about steps the hospital team took to prepare a family for discharge were not properly documented, it would be difficult to determine whether appropriate steps were taken to minimize the likelihood of a complication. In the case of insufficient documentation of relevant social concerns, cases may be incorrectly classified as preventable, because addressing social issues is often not within a hospital's direct control. Finally, because reviewers were not blinded to the original discharging physician, there may have been some unconscious bias of unknown direction in the reviews.

CONCLUSION

Using the Web‐based fault tree tool helped physicians to identify the root causes of hospital readmissions and classify them as preventable or not preventable in a standardized, efficient, and consistent way, while still accounting for the nuances necessary to conduct a fair review. Thus, other hospitals should consider adopting this framework for their own preventability reviews and quality‐improvement initiatives. However, this project also confirmed that only a very small percentage of general pediatrics 15‐day readmissions are potentially preventable, suggesting that general pediatrics readmissions are not an appropriate measure of hospital quality. Instead, adapting the tool to identify root causes of condition‐specific or procedure‐specific readmission rates may be an area of opportunity for future quality‐improvement efforts.

Disclosures: This work was supported through internal funds from The Children's Hospital of Philadelphia. The authors have no financial interests, relationships or affiliations relevant to the subject matter or materials discussed in the article to disclose. The authors have no potential conflicts of interest relevant to the subject matter or materials discussed in the article to disclose.