User login
Acute kidney injury (AKI) is increasingly common in the hospitalized patient1,2 with recent adult and pediatric multinational studies reporting AKI rates of 57% and 27%, respectively.3,4 The development of AKI is associated with significant adverse outcomes including an increased risk of mortality.5-7 For those that survive, the history of AKI may contribute to a lifetime of impaired health with chronic kidney disease.8,9 This is particularly concerning for pediatric patients as AKI may impact morbidity for many decades, influence available therapies for these morbidities, and ultimately contribute to a shortened lifespan.10
AKI in the hospitalized patient is no longer accepted as an unfortunate and unavoidable consequence of illness or the indicated therapy. Currently, there is strong interest in this hospital-acquired condition with global initiatives aimed at increased prevention and early detection and treatment of AKI.11,12 To this objective, risk stratification tools or prediction models could assist clinicians in decision making. Numerous studies have tested AKI prediction models either in particular high-risk populations or based on associated comorbidities, biomarkers, and critical illness scores. These studies are predominantly in adult populations, and few have been externally validated.13 While associations between certain medications and AKI are well known, an AKI prediction model that is applicable to pediatric or adult populations and is based on medication exposure is difficult. However, there is a growing recognition of the potential to develop such a model using the electronic health record (EHR).14
In 2013, Seattle Children’s Hospital (SCH) implemented a nephrotoxin and AKI detection system to assist in clinical decision making within the EHR. This system instituted the automatic ordering of serum creatinines to screen for AKI when the provider ordered three or more medications that were suspected to be nephrotoxic. Other clinical factors such as the diagnoses or preexisting conditions were not considered in the decision-tool algorithm. This original algorithm (Algorithm 1) was later modified and the list of suspected nephrotoxins was expanded (Table 1) in order to align with a national pediatric AKI collaborative (Algorithm 2). However, it was unclear whether the algorithm modification would improve AKI detection.
The present study had two objectives. The first was to evaluate the impact of the modifications on the sensitivity and specificity of our system. The second objective, if either the sensitivity or specificity was determined to be suboptimal, was to develop an improved model for nephrotoxin-related AKI detection. Having either the sensitivity or the specificity under 50% would be equivalent to or worse than a random guess, which we would consider unacceptable.
METHODS
Context
SCH is a tertiary care academic teaching hospital affiliated with the University of Washington School of Medicine, Harborview Medical Center, and the Seattle Cancer Care Alliance. The hospital has 371 licensed beds and approximately 18 medical subspecialty services.
Study Population
This was a retrospective cohort study examining all patients ages 0-21 years admitted to SCH between December 1, 2013 and November 30, 2015. The detection system was modified to align with the national pediatric AKI collaborative, Nephrotoxic Injury Negated by Just-in-Time Action (NINJA) in November 2014. Both acute care and intensive care patients were included (data not separated by location). Patients who had end-stage kidney disease and were receiving dialysis and patients who were evaluated in the emergency department without being admitted or admitted as observation status were excluded from analysis. Patients were also excluded if they did not have a baseline serum creatinine as defined below.
Study Measures
AKI is defined at SCH using the Kidney Disease: Improving Global Outcomes Stage 1 criteria as a guideline. The diagnosis of AKI is based on an increase in the baseline serum creatinine by 0.3 mg/dL or an increase in the serum creatinine by >1.5 times the baseline assuming the incoming creatinine is 0.5 mg/dL or higher. For our definition, the increase in serum creatinine needs to have occurred within a one-week timeframe and urine output is not a diagnostic criterion.15 Baseline serum creatinine is defined as the lowest serum creatinine in the previous six months. Forty medications were classified as nephrotoxins based on previous analysis16 and adapted for our institutional formulary.
Statistical Analysis
To evaluate the efficacy of our systems in detecting nephrotoxin-related AKI, the sensitivity and the specificity using both our original algorithm (Algorithm 1) and the modified algorithm (Algorithm 2) were generated on our complete data set. To test sensitivity, the proportion of AKI patients who would trigger alert using Algorithm 1 and then with Algorithm 2 was identified. Similarly, to test specificity, the proportion of non-AKI patients who did not trigger an alert by the surveillance systems was identified. The differences in sensitivity and specificity between the two algorithms were evaluated using two-sample tests of proportion.
The statistical method of Combinatorial Inference has been utilized in studies of cancer biology17 and in genomics.18 A variation of this approach was used in this study to identify the specific medication combinations most associated with AKI. First, all of the nephrotoxic medications and medication combinations that were prescribed during our study period were identified from a data set (ie, a training set) containing 75% of all encounters selected at random without replacement. Using this training set, the prevalence of each medication combination and the rate of AKI associated with each combination were identified. The predicted overall AKI risk of an individual medication is the average of all the AKI rates associated with each combination containing that specific medication. Also incorporated into the determination of the predicted AKI risk was the prevalence of that medication combination.
To test our model’s predictive capability, the algorithm was applied to the remaining 25% of the total patient data (ie, the test set). The predicted AKI risk was compared with the actual AKI rate in the test data set. Our model’s predictive capability was represented in a receiver operator characteristic (ROC) analysis. The goal was to achieve an area under the ROC curve (AUC) approaching one as this would reflect 100% sensitivity and 100% specificity, whereas an AUC of 0.5 would represent a random guess (50% chance of being correct).
Lastly, our final step was to use our model’s ROC curve to determine an optimal threshold of AKI risk for which to trigger an alert. This predicted risk threshold was based on our goal to increase our surveillance system’s sensitivity balanced with maintaining an acceptable specificity.
An a priori threshold of P = .05 was used to determine statistical significance of all results. Analyses were conducted in Stata 12.1 (StataCorp LP, College Station, Texas) and R 3.3.2 (R Foundation for Statistical Computing, Vienna, Austria). A sample data set containing replication code for our model can be found in an online repository (https://dataverse.harvard.edu/dataverse/chuan). This study was approved by the Seattle Children’s Institutional Review Board.
RESULTS
Sensitivity and Specificity
Of the patient encounters, 14,779 were eligible during the study period. The sensitivity of the system’s ability to identify nephrotoxin-related AKI decreased from 46.9% using Algorithm 1 to 43.3% using Algorithm 2, a change of 3.6% (P = .22). The specificity increased from 73.6% to 89.3%, a change of 15.7% (P < .001; Table 2).
Improvement of Our Nephrotoxin-Related AKI Detection System Using a Novel AKI Prediction Strategy
A total of 838 medication combinations were identified in our training set and the predicted AKI risk for every medication combination was determined. By comparing the predicted risk of AKI to the actual AKI occurrence, an ROC curve with an AUC of 0.756 (Figure) was generated. An increase in system sensitivity was prioritized when determining the optimal AKI risk at which the model would trigger an alert. Setting an alert threshold at a predicted AKI risk of >8%, our model performed with a sensitivity of 74% while decreasing the specificity to 70%.
Identification of High-Risk Nephrotoxic Medications and Medication Combinations
Approximately 200 medication combinations were associated with >8% AKI risk, our new AKI prediction model’s alert threshold. Medication combinations consisting of up to 11 concomitantly prescribed medications were present in our data set. However, many of these combinations were infrequently prescribed. Further analysis, conducted in order to increase the clinical relevance of our findings, identified 10 medications or medication combinations that were both associated with a predicted AKI risk of >8% and that were prescribed on average greater than twice a month (Table 3).
DISCUSSION
The nephrotoxin-related AKI detection system at SCH automatically places orders for serum creatinines on patients who have met criteria for concomitant nephrotoxin exposure. This has given us a robust database from which to develop our clinical decision-making tool. Both our original and updated systems were based on the absolute number of concomitant nephrotoxic medications prescribed.16 This is a reasonable approach given the complexity of building a surveillance system19 and resource limitations. However, a system based on observed rather than theoretical or in vitro data, adaptable to the institution and designed for ongoing refinement, would be more valuable.
The interest in AKI prediction tools continues to be high. Bedford et al. employed numerous variables and diagnostic codes to predict the development of AKI in adults during hospitalization. They were able to produce a prediction model with a reasonable fit (AUC 0.72) to identify patients at higher risk for AKI but were less successful in their attempts to predict progression to severe AKI.20 Hodgson et al. recently developed an adult AKI prediction score (AUC 0.65-0.72) also based on numerous clinical factors that was able to positively impact inpatient mortality.21 To our knowledge, our model is unique in that it focuses on nephrotoxins using a predicted AKI risk algorithm based on observed AKI rates of previously ordered medications/medication combinations (two to 11 medications). Having a decision tool targeting medications gives the clinician guidance that can be used to make a specific intervention rather than identifying a patient at risk due to a diagnosis code or other difficult to modify factors.
There are abundant case studies and reports using logistic regression models identifying specific medications associated with AKI. Our choice of methodology was based on our assessment that logistic regression models would be inadequate for the development of a real-time clinical decision-making tool for several reasons. Using logistic regression to explore every medication combination based on our medication list would be challenging as there are approximately 5.5 × 1010 potential medication combinations. Additionally, logistic regression ignores any potential interactions between the medications. This is an important point as medication interactions can be synergistic, neutral, or antagonist. Consequently, the outcome generated from a set of combined variables may be different from one generated from the sum of each variable taken independently. Logistic regression also does not account for the potential prescribing trends among providers as it assumes that all medications or medication combinations are equally available at the same time. However, in practice, depending on numerous factors, such as hospital culture (eg, the presence of clinical standard work pathways), local bacterial resistance patterns, or medication shortages; certain medication combinations may occur more frequently while others not at all. Finally, logistic regression cannot account for the possibility of a medication combination occurring; therefore, logistic regression may identify a combination strongly associated with AKI that is rarely prescribed.
We theorized that AKI detection would improve with the Algorithm 2 modifications, including the expanded nephrotoxin list, which accompanied alignment with the national pediatric AKI collaborative, NINJA. The finding that our surveillance sensitivity did not improve with this system update supported our subsequent objective to develop a novel nephrotoxin-related AKI decision tool or detection system using our EHR data to identify which specific medications and/or medication combinations were associated with a higher rate of AKI. However, it should be noted that two factors related to measurement bias introduce limitations to our sensitivity and specificity analyses. First, regarding the presence of the alert system, our system will order serum creatinines on patients when they have been exposed to nephrotoxins. Consequently, the proportion of patients with creatinines measured will increase in the nephrotoxin-exposed patients. Unexposed patients may have AKI that is not detected because creatinines may not be ordered. Therefore, there is the potential for a relative increase in AKI detection among nephrotoxin-exposed patients as compared with unexposed patients, which would then affect the measured sensitivity and specificity of the alert. Second, the automated alerts require a baseline creatinine in order to trigger therefore are unable to identify AKI among patients who do not have a baseline serum creatinine measurement.
Our new nephrotoxin-related AKI detection model performed best when an alert was triggered for those medications or medication combinations with a predicted AKI risk of >8%. Forty-six medication combinations consisting of exactly two medications were determined to have a predicted AKI risk of >8% therefore would trigger an alert in our new model system. These medication combinations would not have triggered an alert using either of the previous system algorithms as both algorithms are based on the presence of three or more concomitant nephrotoxic medications.
From the list of suspected nephrotoxins, we identified 11 unique medications in 10 different combinations with a predicted AKI risk of >8% that were prescribed frequently (at least twice a month on average; Table 3). Notably, six out of 10 medication combinations involved vancomycin. Piperacillin-tazobactam was also represented in several combinations. These findings support the concern that others have reported regarding these two medications particularly when prescribed together.22,23
Interestingly, enalapril was identified as a higher-risk medication both alone and in combination with another medication. We do not suspect that enalapril carries a higher risk than other angiotensin-converting enzyme (ACE) inhibitors to increase a patient’s serum creatinine. Rather, we suspect that in our hospitalized patients, this relatively short-acting ACE inhibitor is commonly used in several of our vulnerable populations such as in cardiac and bone marrow transplant patients.
The alert threshold of our model can be adjusted to increase either the sensitivity or the specificity of AKI detection. Our detection sensitivity increased by >1.5-fold with the alert trigger threshold set at a predicted AKI risk of >8%. As a screening tool, our alert limits could be set such that our sensitivity would be greater; however, balancing the potential for alert fatigue is important in determining the acceptance and, ultimately, the success of a working surveillance system.24
A patient’s overall risk of AKI is influenced by many factors such as the presence of underlying chronic comorbidities and the nature or severity of the acute illness as this may affect the patient’s intravascular volume status, systemic blood pressures, or drug metabolism. Our study is limited as we are a children’s hospital and our patients may have fewer comorbidities than seen in the adult population. One could argue that this permits a perspective not clouded by the confounders of chronic disease and allows for the effect of the medications prescribed to be more apparent. However, our study includes critically ill patients and patients who may have been hemodynamically unstable. This may explain why the NINJA algorithm did not improve the sensitivity of our AKI detection as the NINJA collaborative excludes critically ill patients.
Dose and dosing frequency of the prescribed medications could not be taken into account, which could explain the finding that nonsteroidal anti-inflammatory drugs (NSAIDs) such as aspirin, ibuprofen, or ketorolac when used alone were associated with a low (<1%) rate of AKI despite being frequently prescribed. Additionally, as many providers are aware of the AKI risk of NSAIDs, these medications may have been used intermittently (as needed) or in select, perhaps healthier, patients or in patients that take these medications chronically who were admitted for reasons that did not alter their outpatient medication regimen.
Our study also reflects the prescribing habits of our institution and may not be directly applicable to nontertiary care hospitals or centers that do not have large cystic fibrosis, bone marrow, or solid organ transplant populations. Despite our study’s limitations, we feel that there are several findings that are relevant across centers and populations. Our data were derived from the systematic ordering of daily serum creatinines when a patient is at risk for nephrotoxin-related AKI. This is in step with the philosophy advocated by others that AKI identification can only occur if the providers are aware of this risk and are vigilant.25 In this vigilance, we also recognize that not all risks are of the same magnitude and may not deserve the same attention when resources are limited. Our identification of those medication combinations most associated with AKI at our institution has helped us narrow our focus and identify specific areas of potential education and intervention. The specific combinations identified may also be relevant to similar institutions serving similarly complex patients. Those with dissimilar populations could use this methodology to identify those medication combinations most relevant for their patient population and their prescriber’s habits. More studies of this type would be beneficial to the medical community as a whole as certain medication combinations may be found to be high risk regardless of the institution and the age or demographics of the populations they serve.
Acknowledgments
Dr. Karyn E. Yonekawa conceptualized and designed the study, directed the data analysis, interpreted the data, drafted, revised and gave final approval of the manuscript. Dr. Chuan Zhou contributed to the study design, acquired data, conducted the data analysis, critically reviewed, and gave final approval of the manuscript. Ms. Wren L. Haaland contributed to the study design, acquired data, conducted the data analysis, critically reviewed, and gave final approval of the manuscript. Dr. Davene R. Wright contributed to the study design, data analysis, critically reviewed, revised, and gave final approval of the manuscript.
The authors would like to thank Holly Clifton and Suzanne Spencer for their assistance with data acquisition and Drs. Derya Caglar, Corrie McDaniel, and Thida Ong for their writing support.
All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
Disclosures
The authors have no conflicts of interest to report.
1. Siew ED, Davenport A. The growth of acute kidney injury: a rising tide or just closer attention to detail? Kidney Int. 2015;87(1):46-61. https://doi.org/10.1038/ki.2014.293.
2. Matuszkiewicz-Rowinska J, Zebrowski P, Koscielska M, Malyszko J, Mazur A. The growth of acute kidney injury: Eastern European perspective. Kidney Int. 2015;87(6):1264. https://doi.org/10.1038/ki.2015.61.
3. Hoste EA, Bagshaw SM, Bellomo R, et al. Epidemiology of acute kidney injury in critically ill patients: the multinational AKI-EPI study. Intensive Care Med. 2015;41(8):1411-1423. https://doi.org/10.1007/s00134-015-3934-7.
4. Kaddourah A, Basu RK, Bagshaw SM, Goldstein SL, AWARE Investigators. Epidemiology of acute kidney injury in critically ill children and young adults. N Engl J Med. 2017;376(1):11-20. https://doi.org/10.1056/NEJMoa1611391.
5. Soler YA, Nieves-Plaza M, Prieto M, Garcia-De Jesus R, Suarez-Rivera M. Pediatric risk, injury, failure, loss, end-stage renal disease score identifies acute kidney injury and predicts mortality in critically ill children: a prospective study. Pediatr Crit Care Med. 2013;14(4):e189-e195. https://doi.org/10.1097/PCC.0b013e3182745675.
6. Case J, Khan S, Khalid R, Khan A. Epidemiology of acute kidney injury in the intensive care unit. Crit Care Res Pract. 2013;2013:479730. https://doi.org/10.1155/2013/479730.
7. Rewa O, Bagshaw SM. Acute kidney injury-epidemiology, outcomes and economics. Nat Rev Nephrol. 2014;10(4):193-207. https://doi.org/10.1038/nrneph.2013.282.
8. Hsu RK, Hsu CY. The role of acute kidney injury in chronic kidney disease. Semin Nephrol. 2016;36(4):283-292. https://doi.org/10.1016/j.semnephrol.2016.05.005.
9. Menon S, Kirkendall ES, Nguyen H, Goldstein SL. Acute kidney injury associated with high nephrotoxic medication exposure leads to chronic kidney disease after 6 months. J Pediatr. 2014;165(3):522-527.https://doi.org/10.1016/j.jpeds.2014.04.058.
10. Neild GH. Life expectancy with chronic kidney disease: an educational review. Pediatr Nephrol. 2017;32(2):243-248. https://doi.org/10.1007/s00467-016-3383-8.
11. Kellum JA. Acute kidney injury: AKI: the myth of inevitability is finally shattered. Nat Rev Nephrol. 2017;13(3):140-141. https://doi.org/10.1038/nrneph.2017.11.
12. Mehta RL, Cerda J, Burdmann EA, et al. International Society of Nephrology’s 0by25 initiative for acute kidney injury (zero preventable deaths by 2025): a human rights case for nephrology. Lancet. 2015;385(9987):2616-2643. https://doi.org/10.106/S0140-6736(15)60126-X.13.
13. Hodgson LE, Sarnowski A, Roderick PJ, Dimitrov BD, Venn RM, Forni LG. Systematic review of prognostic prediction models for acute kidney injury (AKI) in general hospital populations. BMJ Open. 2017;7(9):e016591. https://doi.org/10.1136/bmjopen-2017-016591.
14. Sutherland SM. Electronic health record-enabled big-data approaches to nephrotoxin-associated acute kidney injury risk prediction. Pharmacotherapy. 2018;38(8):804-812. https://doi.org/10.1002/phar.2150.
15. KDIGO Work Group. KDIGO clinical practice guidelines for acute kidney injury. Kidney Int Suppl. 2012;2(1):S1-138. PubMed
16. Moffett BS, Goldstein SL. Acute kidney injury and increasing nephrotoxic-medication exposure in noncritically-ill children. Clin J Am Soc Nephrol. 2011;6(4):856-863. https://doi.org/10.2215/CJN.08110910.
17. Mukherjee S, Pelech S, Neve RM, et al. Sparse combinatorial inference with an application in cancer biology. Bioinformatics. 2009;25(2):265-271. https://doi.org/10.1093/bioinformatics/btn611.
18. Bailly-Bechet M, Braunstein A, Pagnani A, Weigt M, Zecchina R. Inference of sparse combinatorial-control networks from gene-expression data: a message passing approach. BMC Bioinformatics. 2010;11:355. https://doi.org/10.1186/1471-2105-11-355.
19. Kirkendall ES, Spires WL, Mottes TA, et al. Development and performance of electronic acute kidney injury triggers to identify pediatric patients at risk for nephrotoxic medication-associated harm. Appl Clin Inform. 2014;5(2):313-333. https://doi.org/10.4338/ACI-2013-12-RA-0102.
20. Bedford M, Stevens P, Coulton S, et al. Development of Risk Models for the Prediction of New or Worsening Acute Kidney Injury on or During Hospital Admission: A Cohort and Nested Study. Southampton, UK: NIHR Journals Library; 2016. PubMed
21. Hodgson LE, Roderick PJ, Venn RM, Yao GL, Dimitrov BD, Forni LG. The ICE-AKI study: impact analysis of a clinical prediction rule and electronic AKI alert in general medical patients. PLoS One. 2018;13(8):e0200584. https://doi.org/10.1371/journal.pone.0200584.
22. Hammond DA, Smith MN, Li C, Hayes SM, Lusardi K, Bookstaver PB. Systematic review and meta-analysis of acute kidney injury associated with concomitant vancomycin and piperacillin/tazobactam. Clin Infect Dis. 2017;64(5):666-674. https://doi.org/10.1093/cid/ciw811.
23. Downes KJ, Cowden C, Laskin BL, et al. Association of acute kidney injury with concomitant vancomycin and piperacillin/tazobactam treatment among hospitalized children. JAMA Pediatr. 2017;171(12):e173219.https://doi.org/10.1001/jamapediatrics.2017.3219.
24. Agency for Heathcare Research and Quality. Alert Fatigue Web site. https://psnet.ahrq.gov/primers/primer/28/alert-fatigue. Updated July 2016. Accessed April 14, 2017.
25. Downes KJ, Rao MB, Kahill L, Nguyen H, Clancy JP, Goldstein SL. Daily serum creatinine monitoring promotes earlier detection of acute kidney injury in children and adolescents with cystic fibrosis. J Cyst Fibros. 2014;13(4):435-441. https://doi.org/10.1016/j.jcf.2014.03.005.
Acute kidney injury (AKI) is increasingly common in the hospitalized patient1,2 with recent adult and pediatric multinational studies reporting AKI rates of 57% and 27%, respectively.3,4 The development of AKI is associated with significant adverse outcomes including an increased risk of mortality.5-7 For those that survive, the history of AKI may contribute to a lifetime of impaired health with chronic kidney disease.8,9 This is particularly concerning for pediatric patients as AKI may impact morbidity for many decades, influence available therapies for these morbidities, and ultimately contribute to a shortened lifespan.10
AKI in the hospitalized patient is no longer accepted as an unfortunate and unavoidable consequence of illness or the indicated therapy. Currently, there is strong interest in this hospital-acquired condition with global initiatives aimed at increased prevention and early detection and treatment of AKI.11,12 To this objective, risk stratification tools or prediction models could assist clinicians in decision making. Numerous studies have tested AKI prediction models either in particular high-risk populations or based on associated comorbidities, biomarkers, and critical illness scores. These studies are predominantly in adult populations, and few have been externally validated.13 While associations between certain medications and AKI are well known, an AKI prediction model that is applicable to pediatric or adult populations and is based on medication exposure is difficult. However, there is a growing recognition of the potential to develop such a model using the electronic health record (EHR).14
In 2013, Seattle Children’s Hospital (SCH) implemented a nephrotoxin and AKI detection system to assist in clinical decision making within the EHR. This system instituted the automatic ordering of serum creatinines to screen for AKI when the provider ordered three or more medications that were suspected to be nephrotoxic. Other clinical factors such as the diagnoses or preexisting conditions were not considered in the decision-tool algorithm. This original algorithm (Algorithm 1) was later modified and the list of suspected nephrotoxins was expanded (Table 1) in order to align with a national pediatric AKI collaborative (Algorithm 2). However, it was unclear whether the algorithm modification would improve AKI detection.
The present study had two objectives. The first was to evaluate the impact of the modifications on the sensitivity and specificity of our system. The second objective, if either the sensitivity or specificity was determined to be suboptimal, was to develop an improved model for nephrotoxin-related AKI detection. Having either the sensitivity or the specificity under 50% would be equivalent to or worse than a random guess, which we would consider unacceptable.
METHODS
Context
SCH is a tertiary care academic teaching hospital affiliated with the University of Washington School of Medicine, Harborview Medical Center, and the Seattle Cancer Care Alliance. The hospital has 371 licensed beds and approximately 18 medical subspecialty services.
Study Population
This was a retrospective cohort study examining all patients ages 0-21 years admitted to SCH between December 1, 2013 and November 30, 2015. The detection system was modified to align with the national pediatric AKI collaborative, Nephrotoxic Injury Negated by Just-in-Time Action (NINJA) in November 2014. Both acute care and intensive care patients were included (data not separated by location). Patients who had end-stage kidney disease and were receiving dialysis and patients who were evaluated in the emergency department without being admitted or admitted as observation status were excluded from analysis. Patients were also excluded if they did not have a baseline serum creatinine as defined below.
Study Measures
AKI is defined at SCH using the Kidney Disease: Improving Global Outcomes Stage 1 criteria as a guideline. The diagnosis of AKI is based on an increase in the baseline serum creatinine by 0.3 mg/dL or an increase in the serum creatinine by >1.5 times the baseline assuming the incoming creatinine is 0.5 mg/dL or higher. For our definition, the increase in serum creatinine needs to have occurred within a one-week timeframe and urine output is not a diagnostic criterion.15 Baseline serum creatinine is defined as the lowest serum creatinine in the previous six months. Forty medications were classified as nephrotoxins based on previous analysis16 and adapted for our institutional formulary.
Statistical Analysis
To evaluate the efficacy of our systems in detecting nephrotoxin-related AKI, the sensitivity and the specificity using both our original algorithm (Algorithm 1) and the modified algorithm (Algorithm 2) were generated on our complete data set. To test sensitivity, the proportion of AKI patients who would trigger alert using Algorithm 1 and then with Algorithm 2 was identified. Similarly, to test specificity, the proportion of non-AKI patients who did not trigger an alert by the surveillance systems was identified. The differences in sensitivity and specificity between the two algorithms were evaluated using two-sample tests of proportion.
The statistical method of Combinatorial Inference has been utilized in studies of cancer biology17 and in genomics.18 A variation of this approach was used in this study to identify the specific medication combinations most associated with AKI. First, all of the nephrotoxic medications and medication combinations that were prescribed during our study period were identified from a data set (ie, a training set) containing 75% of all encounters selected at random without replacement. Using this training set, the prevalence of each medication combination and the rate of AKI associated with each combination were identified. The predicted overall AKI risk of an individual medication is the average of all the AKI rates associated with each combination containing that specific medication. Also incorporated into the determination of the predicted AKI risk was the prevalence of that medication combination.
To test our model’s predictive capability, the algorithm was applied to the remaining 25% of the total patient data (ie, the test set). The predicted AKI risk was compared with the actual AKI rate in the test data set. Our model’s predictive capability was represented in a receiver operator characteristic (ROC) analysis. The goal was to achieve an area under the ROC curve (AUC) approaching one as this would reflect 100% sensitivity and 100% specificity, whereas an AUC of 0.5 would represent a random guess (50% chance of being correct).
Lastly, our final step was to use our model’s ROC curve to determine an optimal threshold of AKI risk for which to trigger an alert. This predicted risk threshold was based on our goal to increase our surveillance system’s sensitivity balanced with maintaining an acceptable specificity.
An a priori threshold of P = .05 was used to determine statistical significance of all results. Analyses were conducted in Stata 12.1 (StataCorp LP, College Station, Texas) and R 3.3.2 (R Foundation for Statistical Computing, Vienna, Austria). A sample data set containing replication code for our model can be found in an online repository (https://dataverse.harvard.edu/dataverse/chuan). This study was approved by the Seattle Children’s Institutional Review Board.
RESULTS
Sensitivity and Specificity
Of the patient encounters, 14,779 were eligible during the study period. The sensitivity of the system’s ability to identify nephrotoxin-related AKI decreased from 46.9% using Algorithm 1 to 43.3% using Algorithm 2, a change of 3.6% (P = .22). The specificity increased from 73.6% to 89.3%, a change of 15.7% (P < .001; Table 2).
Improvement of Our Nephrotoxin-Related AKI Detection System Using a Novel AKI Prediction Strategy
A total of 838 medication combinations were identified in our training set and the predicted AKI risk for every medication combination was determined. By comparing the predicted risk of AKI to the actual AKI occurrence, an ROC curve with an AUC of 0.756 (Figure) was generated. An increase in system sensitivity was prioritized when determining the optimal AKI risk at which the model would trigger an alert. Setting an alert threshold at a predicted AKI risk of >8%, our model performed with a sensitivity of 74% while decreasing the specificity to 70%.
Identification of High-Risk Nephrotoxic Medications and Medication Combinations
Approximately 200 medication combinations were associated with >8% AKI risk, our new AKI prediction model’s alert threshold. Medication combinations consisting of up to 11 concomitantly prescribed medications were present in our data set. However, many of these combinations were infrequently prescribed. Further analysis, conducted in order to increase the clinical relevance of our findings, identified 10 medications or medication combinations that were both associated with a predicted AKI risk of >8% and that were prescribed on average greater than twice a month (Table 3).
DISCUSSION
The nephrotoxin-related AKI detection system at SCH automatically places orders for serum creatinines on patients who have met criteria for concomitant nephrotoxin exposure. This has given us a robust database from which to develop our clinical decision-making tool. Both our original and updated systems were based on the absolute number of concomitant nephrotoxic medications prescribed.16 This is a reasonable approach given the complexity of building a surveillance system19 and resource limitations. However, a system based on observed rather than theoretical or in vitro data, adaptable to the institution and designed for ongoing refinement, would be more valuable.
The interest in AKI prediction tools continues to be high. Bedford et al. employed numerous variables and diagnostic codes to predict the development of AKI in adults during hospitalization. They were able to produce a prediction model with a reasonable fit (AUC 0.72) to identify patients at higher risk for AKI but were less successful in their attempts to predict progression to severe AKI.20 Hodgson et al. recently developed an adult AKI prediction score (AUC 0.65-0.72) also based on numerous clinical factors that was able to positively impact inpatient mortality.21 To our knowledge, our model is unique in that it focuses on nephrotoxins using a predicted AKI risk algorithm based on observed AKI rates of previously ordered medications/medication combinations (two to 11 medications). Having a decision tool targeting medications gives the clinician guidance that can be used to make a specific intervention rather than identifying a patient at risk due to a diagnosis code or other difficult to modify factors.
There are abundant case studies and reports using logistic regression models identifying specific medications associated with AKI. Our choice of methodology was based on our assessment that logistic regression models would be inadequate for the development of a real-time clinical decision-making tool for several reasons. Using logistic regression to explore every medication combination based on our medication list would be challenging as there are approximately 5.5 × 1010 potential medication combinations. Additionally, logistic regression ignores any potential interactions between the medications. This is an important point as medication interactions can be synergistic, neutral, or antagonist. Consequently, the outcome generated from a set of combined variables may be different from one generated from the sum of each variable taken independently. Logistic regression also does not account for the potential prescribing trends among providers as it assumes that all medications or medication combinations are equally available at the same time. However, in practice, depending on numerous factors, such as hospital culture (eg, the presence of clinical standard work pathways), local bacterial resistance patterns, or medication shortages; certain medication combinations may occur more frequently while others not at all. Finally, logistic regression cannot account for the possibility of a medication combination occurring; therefore, logistic regression may identify a combination strongly associated with AKI that is rarely prescribed.
We theorized that AKI detection would improve with the Algorithm 2 modifications, including the expanded nephrotoxin list, which accompanied alignment with the national pediatric AKI collaborative, NINJA. The finding that our surveillance sensitivity did not improve with this system update supported our subsequent objective to develop a novel nephrotoxin-related AKI decision tool or detection system using our EHR data to identify which specific medications and/or medication combinations were associated with a higher rate of AKI. However, it should be noted that two factors related to measurement bias introduce limitations to our sensitivity and specificity analyses. First, regarding the presence of the alert system, our system will order serum creatinines on patients when they have been exposed to nephrotoxins. Consequently, the proportion of patients with creatinines measured will increase in the nephrotoxin-exposed patients. Unexposed patients may have AKI that is not detected because creatinines may not be ordered. Therefore, there is the potential for a relative increase in AKI detection among nephrotoxin-exposed patients as compared with unexposed patients, which would then affect the measured sensitivity and specificity of the alert. Second, the automated alerts require a baseline creatinine in order to trigger therefore are unable to identify AKI among patients who do not have a baseline serum creatinine measurement.
Our new nephrotoxin-related AKI detection model performed best when an alert was triggered for those medications or medication combinations with a predicted AKI risk of >8%. Forty-six medication combinations consisting of exactly two medications were determined to have a predicted AKI risk of >8% therefore would trigger an alert in our new model system. These medication combinations would not have triggered an alert using either of the previous system algorithms as both algorithms are based on the presence of three or more concomitant nephrotoxic medications.
From the list of suspected nephrotoxins, we identified 11 unique medications in 10 different combinations with a predicted AKI risk of >8% that were prescribed frequently (at least twice a month on average; Table 3). Notably, six out of 10 medication combinations involved vancomycin. Piperacillin-tazobactam was also represented in several combinations. These findings support the concern that others have reported regarding these two medications particularly when prescribed together.22,23
Interestingly, enalapril was identified as a higher-risk medication both alone and in combination with another medication. We do not suspect that enalapril carries a higher risk than other angiotensin-converting enzyme (ACE) inhibitors to increase a patient’s serum creatinine. Rather, we suspect that in our hospitalized patients, this relatively short-acting ACE inhibitor is commonly used in several of our vulnerable populations such as in cardiac and bone marrow transplant patients.
The alert threshold of our model can be adjusted to increase either the sensitivity or the specificity of AKI detection. Our detection sensitivity increased by >1.5-fold with the alert trigger threshold set at a predicted AKI risk of >8%. As a screening tool, our alert limits could be set such that our sensitivity would be greater; however, balancing the potential for alert fatigue is important in determining the acceptance and, ultimately, the success of a working surveillance system.24
A patient’s overall risk of AKI is influenced by many factors such as the presence of underlying chronic comorbidities and the nature or severity of the acute illness as this may affect the patient’s intravascular volume status, systemic blood pressures, or drug metabolism. Our study is limited as we are a children’s hospital and our patients may have fewer comorbidities than seen in the adult population. One could argue that this permits a perspective not clouded by the confounders of chronic disease and allows for the effect of the medications prescribed to be more apparent. However, our study includes critically ill patients and patients who may have been hemodynamically unstable. This may explain why the NINJA algorithm did not improve the sensitivity of our AKI detection as the NINJA collaborative excludes critically ill patients.
Dose and dosing frequency of the prescribed medications could not be taken into account, which could explain the finding that nonsteroidal anti-inflammatory drugs (NSAIDs) such as aspirin, ibuprofen, or ketorolac when used alone were associated with a low (<1%) rate of AKI despite being frequently prescribed. Additionally, as many providers are aware of the AKI risk of NSAIDs, these medications may have been used intermittently (as needed) or in select, perhaps healthier, patients or in patients that take these medications chronically who were admitted for reasons that did not alter their outpatient medication regimen.
Our study also reflects the prescribing habits of our institution and may not be directly applicable to nontertiary care hospitals or centers that do not have large cystic fibrosis, bone marrow, or solid organ transplant populations. Despite our study’s limitations, we feel that there are several findings that are relevant across centers and populations. Our data were derived from the systematic ordering of daily serum creatinines when a patient is at risk for nephrotoxin-related AKI. This is in step with the philosophy advocated by others that AKI identification can only occur if the providers are aware of this risk and are vigilant.25 In this vigilance, we also recognize that not all risks are of the same magnitude and may not deserve the same attention when resources are limited. Our identification of those medication combinations most associated with AKI at our institution has helped us narrow our focus and identify specific areas of potential education and intervention. The specific combinations identified may also be relevant to similar institutions serving similarly complex patients. Those with dissimilar populations could use this methodology to identify those medication combinations most relevant for their patient population and their prescriber’s habits. More studies of this type would be beneficial to the medical community as a whole as certain medication combinations may be found to be high risk regardless of the institution and the age or demographics of the populations they serve.
Acknowledgments
Dr. Karyn E. Yonekawa conceptualized and designed the study, directed the data analysis, interpreted the data, drafted, revised and gave final approval of the manuscript. Dr. Chuan Zhou contributed to the study design, acquired data, conducted the data analysis, critically reviewed, and gave final approval of the manuscript. Ms. Wren L. Haaland contributed to the study design, acquired data, conducted the data analysis, critically reviewed, and gave final approval of the manuscript. Dr. Davene R. Wright contributed to the study design, data analysis, critically reviewed, revised, and gave final approval of the manuscript.
The authors would like to thank Holly Clifton and Suzanne Spencer for their assistance with data acquisition and Drs. Derya Caglar, Corrie McDaniel, and Thida Ong for their writing support.
All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
Disclosures
The authors have no conflicts of interest to report.
Acute kidney injury (AKI) is increasingly common in the hospitalized patient1,2 with recent adult and pediatric multinational studies reporting AKI rates of 57% and 27%, respectively.3,4 The development of AKI is associated with significant adverse outcomes including an increased risk of mortality.5-7 For those that survive, the history of AKI may contribute to a lifetime of impaired health with chronic kidney disease.8,9 This is particularly concerning for pediatric patients as AKI may impact morbidity for many decades, influence available therapies for these morbidities, and ultimately contribute to a shortened lifespan.10
AKI in the hospitalized patient is no longer accepted as an unfortunate and unavoidable consequence of illness or the indicated therapy. Currently, there is strong interest in this hospital-acquired condition with global initiatives aimed at increased prevention and early detection and treatment of AKI.11,12 To this objective, risk stratification tools or prediction models could assist clinicians in decision making. Numerous studies have tested AKI prediction models either in particular high-risk populations or based on associated comorbidities, biomarkers, and critical illness scores. These studies are predominantly in adult populations, and few have been externally validated.13 While associations between certain medications and AKI are well known, an AKI prediction model that is applicable to pediatric or adult populations and is based on medication exposure is difficult. However, there is a growing recognition of the potential to develop such a model using the electronic health record (EHR).14
In 2013, Seattle Children’s Hospital (SCH) implemented a nephrotoxin and AKI detection system to assist in clinical decision making within the EHR. This system instituted the automatic ordering of serum creatinines to screen for AKI when the provider ordered three or more medications that were suspected to be nephrotoxic. Other clinical factors such as the diagnoses or preexisting conditions were not considered in the decision-tool algorithm. This original algorithm (Algorithm 1) was later modified and the list of suspected nephrotoxins was expanded (Table 1) in order to align with a national pediatric AKI collaborative (Algorithm 2). However, it was unclear whether the algorithm modification would improve AKI detection.
The present study had two objectives. The first was to evaluate the impact of the modifications on the sensitivity and specificity of our system. The second objective, if either the sensitivity or specificity was determined to be suboptimal, was to develop an improved model for nephrotoxin-related AKI detection. Having either the sensitivity or the specificity under 50% would be equivalent to or worse than a random guess, which we would consider unacceptable.
METHODS
Context
SCH is a tertiary care academic teaching hospital affiliated with the University of Washington School of Medicine, Harborview Medical Center, and the Seattle Cancer Care Alliance. The hospital has 371 licensed beds and approximately 18 medical subspecialty services.
Study Population
This was a retrospective cohort study examining all patients ages 0-21 years admitted to SCH between December 1, 2013 and November 30, 2015. The detection system was modified to align with the national pediatric AKI collaborative, Nephrotoxic Injury Negated by Just-in-Time Action (NINJA) in November 2014. Both acute care and intensive care patients were included (data not separated by location). Patients who had end-stage kidney disease and were receiving dialysis and patients who were evaluated in the emergency department without being admitted or admitted as observation status were excluded from analysis. Patients were also excluded if they did not have a baseline serum creatinine as defined below.
Study Measures
AKI is defined at SCH using the Kidney Disease: Improving Global Outcomes Stage 1 criteria as a guideline. The diagnosis of AKI is based on an increase in the baseline serum creatinine by 0.3 mg/dL or an increase in the serum creatinine by >1.5 times the baseline assuming the incoming creatinine is 0.5 mg/dL or higher. For our definition, the increase in serum creatinine needs to have occurred within a one-week timeframe and urine output is not a diagnostic criterion.15 Baseline serum creatinine is defined as the lowest serum creatinine in the previous six months. Forty medications were classified as nephrotoxins based on previous analysis16 and adapted for our institutional formulary.
Statistical Analysis
To evaluate the efficacy of our systems in detecting nephrotoxin-related AKI, the sensitivity and the specificity using both our original algorithm (Algorithm 1) and the modified algorithm (Algorithm 2) were generated on our complete data set. To test sensitivity, the proportion of AKI patients who would trigger alert using Algorithm 1 and then with Algorithm 2 was identified. Similarly, to test specificity, the proportion of non-AKI patients who did not trigger an alert by the surveillance systems was identified. The differences in sensitivity and specificity between the two algorithms were evaluated using two-sample tests of proportion.
The statistical method of Combinatorial Inference has been utilized in studies of cancer biology17 and in genomics.18 A variation of this approach was used in this study to identify the specific medication combinations most associated with AKI. First, all of the nephrotoxic medications and medication combinations that were prescribed during our study period were identified from a data set (ie, a training set) containing 75% of all encounters selected at random without replacement. Using this training set, the prevalence of each medication combination and the rate of AKI associated with each combination were identified. The predicted overall AKI risk of an individual medication is the average of all the AKI rates associated with each combination containing that specific medication. Also incorporated into the determination of the predicted AKI risk was the prevalence of that medication combination.
To test our model’s predictive capability, the algorithm was applied to the remaining 25% of the total patient data (ie, the test set). The predicted AKI risk was compared with the actual AKI rate in the test data set. Our model’s predictive capability was represented in a receiver operator characteristic (ROC) analysis. The goal was to achieve an area under the ROC curve (AUC) approaching one as this would reflect 100% sensitivity and 100% specificity, whereas an AUC of 0.5 would represent a random guess (50% chance of being correct).
Lastly, our final step was to use our model’s ROC curve to determine an optimal threshold of AKI risk for which to trigger an alert. This predicted risk threshold was based on our goal to increase our surveillance system’s sensitivity balanced with maintaining an acceptable specificity.
An a priori threshold of P = .05 was used to determine statistical significance of all results. Analyses were conducted in Stata 12.1 (StataCorp LP, College Station, Texas) and R 3.3.2 (R Foundation for Statistical Computing, Vienna, Austria). A sample data set containing replication code for our model can be found in an online repository (https://dataverse.harvard.edu/dataverse/chuan). This study was approved by the Seattle Children’s Institutional Review Board.
RESULTS
Sensitivity and Specificity
Of the patient encounters, 14,779 were eligible during the study period. The sensitivity of the system’s ability to identify nephrotoxin-related AKI decreased from 46.9% using Algorithm 1 to 43.3% using Algorithm 2, a change of 3.6% (P = .22). The specificity increased from 73.6% to 89.3%, a change of 15.7% (P < .001; Table 2).
Improvement of Our Nephrotoxin-Related AKI Detection System Using a Novel AKI Prediction Strategy
A total of 838 medication combinations were identified in our training set and the predicted AKI risk for every medication combination was determined. By comparing the predicted risk of AKI to the actual AKI occurrence, an ROC curve with an AUC of 0.756 (Figure) was generated. An increase in system sensitivity was prioritized when determining the optimal AKI risk at which the model would trigger an alert. Setting an alert threshold at a predicted AKI risk of >8%, our model performed with a sensitivity of 74% while decreasing the specificity to 70%.
Identification of High-Risk Nephrotoxic Medications and Medication Combinations
Approximately 200 medication combinations were associated with >8% AKI risk, our new AKI prediction model’s alert threshold. Medication combinations consisting of up to 11 concomitantly prescribed medications were present in our data set. However, many of these combinations were infrequently prescribed. Further analysis, conducted in order to increase the clinical relevance of our findings, identified 10 medications or medication combinations that were both associated with a predicted AKI risk of >8% and that were prescribed on average greater than twice a month (Table 3).
DISCUSSION
The nephrotoxin-related AKI detection system at SCH automatically places orders for serum creatinines on patients who have met criteria for concomitant nephrotoxin exposure. This has given us a robust database from which to develop our clinical decision-making tool. Both our original and updated systems were based on the absolute number of concomitant nephrotoxic medications prescribed.16 This is a reasonable approach given the complexity of building a surveillance system19 and resource limitations. However, a system based on observed rather than theoretical or in vitro data, adaptable to the institution and designed for ongoing refinement, would be more valuable.
The interest in AKI prediction tools continues to be high. Bedford et al. employed numerous variables and diagnostic codes to predict the development of AKI in adults during hospitalization. They were able to produce a prediction model with a reasonable fit (AUC 0.72) to identify patients at higher risk for AKI but were less successful in their attempts to predict progression to severe AKI.20 Hodgson et al. recently developed an adult AKI prediction score (AUC 0.65-0.72) also based on numerous clinical factors that was able to positively impact inpatient mortality.21 To our knowledge, our model is unique in that it focuses on nephrotoxins using a predicted AKI risk algorithm based on observed AKI rates of previously ordered medications/medication combinations (two to 11 medications). Having a decision tool targeting medications gives the clinician guidance that can be used to make a specific intervention rather than identifying a patient at risk due to a diagnosis code or other difficult to modify factors.
There are abundant case studies and reports using logistic regression models identifying specific medications associated with AKI. Our choice of methodology was based on our assessment that logistic regression models would be inadequate for the development of a real-time clinical decision-making tool for several reasons. Using logistic regression to explore every medication combination based on our medication list would be challenging as there are approximately 5.5 × 1010 potential medication combinations. Additionally, logistic regression ignores any potential interactions between the medications. This is an important point as medication interactions can be synergistic, neutral, or antagonist. Consequently, the outcome generated from a set of combined variables may be different from one generated from the sum of each variable taken independently. Logistic regression also does not account for the potential prescribing trends among providers as it assumes that all medications or medication combinations are equally available at the same time. However, in practice, depending on numerous factors, such as hospital culture (eg, the presence of clinical standard work pathways), local bacterial resistance patterns, or medication shortages; certain medication combinations may occur more frequently while others not at all. Finally, logistic regression cannot account for the possibility of a medication combination occurring; therefore, logistic regression may identify a combination strongly associated with AKI that is rarely prescribed.
We theorized that AKI detection would improve with the Algorithm 2 modifications, including the expanded nephrotoxin list, which accompanied alignment with the national pediatric AKI collaborative, NINJA. The finding that our surveillance sensitivity did not improve with this system update supported our subsequent objective to develop a novel nephrotoxin-related AKI decision tool or detection system using our EHR data to identify which specific medications and/or medication combinations were associated with a higher rate of AKI. However, it should be noted that two factors related to measurement bias introduce limitations to our sensitivity and specificity analyses. First, regarding the presence of the alert system, our system will order serum creatinines on patients when they have been exposed to nephrotoxins. Consequently, the proportion of patients with creatinines measured will increase in the nephrotoxin-exposed patients. Unexposed patients may have AKI that is not detected because creatinines may not be ordered. Therefore, there is the potential for a relative increase in AKI detection among nephrotoxin-exposed patients as compared with unexposed patients, which would then affect the measured sensitivity and specificity of the alert. Second, the automated alerts require a baseline creatinine in order to trigger therefore are unable to identify AKI among patients who do not have a baseline serum creatinine measurement.
Our new nephrotoxin-related AKI detection model performed best when an alert was triggered for those medications or medication combinations with a predicted AKI risk of >8%. Forty-six medication combinations consisting of exactly two medications were determined to have a predicted AKI risk of >8% therefore would trigger an alert in our new model system. These medication combinations would not have triggered an alert using either of the previous system algorithms as both algorithms are based on the presence of three or more concomitant nephrotoxic medications.
From the list of suspected nephrotoxins, we identified 11 unique medications in 10 different combinations with a predicted AKI risk of >8% that were prescribed frequently (at least twice a month on average; Table 3). Notably, six out of 10 medication combinations involved vancomycin. Piperacillin-tazobactam was also represented in several combinations. These findings support the concern that others have reported regarding these two medications particularly when prescribed together.22,23
Interestingly, enalapril was identified as a higher-risk medication both alone and in combination with another medication. We do not suspect that enalapril carries a higher risk than other angiotensin-converting enzyme (ACE) inhibitors to increase a patient’s serum creatinine. Rather, we suspect that in our hospitalized patients, this relatively short-acting ACE inhibitor is commonly used in several of our vulnerable populations such as in cardiac and bone marrow transplant patients.
The alert threshold of our model can be adjusted to increase either the sensitivity or the specificity of AKI detection. Our detection sensitivity increased by >1.5-fold with the alert trigger threshold set at a predicted AKI risk of >8%. As a screening tool, our alert limits could be set such that our sensitivity would be greater; however, balancing the potential for alert fatigue is important in determining the acceptance and, ultimately, the success of a working surveillance system.24
A patient’s overall risk of AKI is influenced by many factors such as the presence of underlying chronic comorbidities and the nature or severity of the acute illness as this may affect the patient’s intravascular volume status, systemic blood pressures, or drug metabolism. Our study is limited as we are a children’s hospital and our patients may have fewer comorbidities than seen in the adult population. One could argue that this permits a perspective not clouded by the confounders of chronic disease and allows for the effect of the medications prescribed to be more apparent. However, our study includes critically ill patients and patients who may have been hemodynamically unstable. This may explain why the NINJA algorithm did not improve the sensitivity of our AKI detection as the NINJA collaborative excludes critically ill patients.
Dose and dosing frequency of the prescribed medications could not be taken into account, which could explain the finding that nonsteroidal anti-inflammatory drugs (NSAIDs) such as aspirin, ibuprofen, or ketorolac when used alone were associated with a low (<1%) rate of AKI despite being frequently prescribed. Additionally, as many providers are aware of the AKI risk of NSAIDs, these medications may have been used intermittently (as needed) or in select, perhaps healthier, patients or in patients that take these medications chronically who were admitted for reasons that did not alter their outpatient medication regimen.
Our study also reflects the prescribing habits of our institution and may not be directly applicable to nontertiary care hospitals or centers that do not have large cystic fibrosis, bone marrow, or solid organ transplant populations. Despite our study’s limitations, we feel that there are several findings that are relevant across centers and populations. Our data were derived from the systematic ordering of daily serum creatinines when a patient is at risk for nephrotoxin-related AKI. This is in step with the philosophy advocated by others that AKI identification can only occur if the providers are aware of this risk and are vigilant.25 In this vigilance, we also recognize that not all risks are of the same magnitude and may not deserve the same attention when resources are limited. Our identification of those medication combinations most associated with AKI at our institution has helped us narrow our focus and identify specific areas of potential education and intervention. The specific combinations identified may also be relevant to similar institutions serving similarly complex patients. Those with dissimilar populations could use this methodology to identify those medication combinations most relevant for their patient population and their prescriber’s habits. More studies of this type would be beneficial to the medical community as a whole as certain medication combinations may be found to be high risk regardless of the institution and the age or demographics of the populations they serve.
Acknowledgments
Dr. Karyn E. Yonekawa conceptualized and designed the study, directed the data analysis, interpreted the data, drafted, revised and gave final approval of the manuscript. Dr. Chuan Zhou contributed to the study design, acquired data, conducted the data analysis, critically reviewed, and gave final approval of the manuscript. Ms. Wren L. Haaland contributed to the study design, acquired data, conducted the data analysis, critically reviewed, and gave final approval of the manuscript. Dr. Davene R. Wright contributed to the study design, data analysis, critically reviewed, revised, and gave final approval of the manuscript.
The authors would like to thank Holly Clifton and Suzanne Spencer for their assistance with data acquisition and Drs. Derya Caglar, Corrie McDaniel, and Thida Ong for their writing support.
All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
Disclosures
The authors have no conflicts of interest to report.
1. Siew ED, Davenport A. The growth of acute kidney injury: a rising tide or just closer attention to detail? Kidney Int. 2015;87(1):46-61. https://doi.org/10.1038/ki.2014.293.
2. Matuszkiewicz-Rowinska J, Zebrowski P, Koscielska M, Malyszko J, Mazur A. The growth of acute kidney injury: Eastern European perspective. Kidney Int. 2015;87(6):1264. https://doi.org/10.1038/ki.2015.61.
3. Hoste EA, Bagshaw SM, Bellomo R, et al. Epidemiology of acute kidney injury in critically ill patients: the multinational AKI-EPI study. Intensive Care Med. 2015;41(8):1411-1423. https://doi.org/10.1007/s00134-015-3934-7.
4. Kaddourah A, Basu RK, Bagshaw SM, Goldstein SL, AWARE Investigators. Epidemiology of acute kidney injury in critically ill children and young adults. N Engl J Med. 2017;376(1):11-20. https://doi.org/10.1056/NEJMoa1611391.
5. Soler YA, Nieves-Plaza M, Prieto M, Garcia-De Jesus R, Suarez-Rivera M. Pediatric risk, injury, failure, loss, end-stage renal disease score identifies acute kidney injury and predicts mortality in critically ill children: a prospective study. Pediatr Crit Care Med. 2013;14(4):e189-e195. https://doi.org/10.1097/PCC.0b013e3182745675.
6. Case J, Khan S, Khalid R, Khan A. Epidemiology of acute kidney injury in the intensive care unit. Crit Care Res Pract. 2013;2013:479730. https://doi.org/10.1155/2013/479730.
7. Rewa O, Bagshaw SM. Acute kidney injury-epidemiology, outcomes and economics. Nat Rev Nephrol. 2014;10(4):193-207. https://doi.org/10.1038/nrneph.2013.282.
8. Hsu RK, Hsu CY. The role of acute kidney injury in chronic kidney disease. Semin Nephrol. 2016;36(4):283-292. https://doi.org/10.1016/j.semnephrol.2016.05.005.
9. Menon S, Kirkendall ES, Nguyen H, Goldstein SL. Acute kidney injury associated with high nephrotoxic medication exposure leads to chronic kidney disease after 6 months. J Pediatr. 2014;165(3):522-527.https://doi.org/10.1016/j.jpeds.2014.04.058.
10. Neild GH. Life expectancy with chronic kidney disease: an educational review. Pediatr Nephrol. 2017;32(2):243-248. https://doi.org/10.1007/s00467-016-3383-8.
11. Kellum JA. Acute kidney injury: AKI: the myth of inevitability is finally shattered. Nat Rev Nephrol. 2017;13(3):140-141. https://doi.org/10.1038/nrneph.2017.11.
12. Mehta RL, Cerda J, Burdmann EA, et al. International Society of Nephrology’s 0by25 initiative for acute kidney injury (zero preventable deaths by 2025): a human rights case for nephrology. Lancet. 2015;385(9987):2616-2643. https://doi.org/10.106/S0140-6736(15)60126-X.13.
13. Hodgson LE, Sarnowski A, Roderick PJ, Dimitrov BD, Venn RM, Forni LG. Systematic review of prognostic prediction models for acute kidney injury (AKI) in general hospital populations. BMJ Open. 2017;7(9):e016591. https://doi.org/10.1136/bmjopen-2017-016591.
14. Sutherland SM. Electronic health record-enabled big-data approaches to nephrotoxin-associated acute kidney injury risk prediction. Pharmacotherapy. 2018;38(8):804-812. https://doi.org/10.1002/phar.2150.
15. KDIGO Work Group. KDIGO clinical practice guidelines for acute kidney injury. Kidney Int Suppl. 2012;2(1):S1-138. PubMed
16. Moffett BS, Goldstein SL. Acute kidney injury and increasing nephrotoxic-medication exposure in noncritically-ill children. Clin J Am Soc Nephrol. 2011;6(4):856-863. https://doi.org/10.2215/CJN.08110910.
17. Mukherjee S, Pelech S, Neve RM, et al. Sparse combinatorial inference with an application in cancer biology. Bioinformatics. 2009;25(2):265-271. https://doi.org/10.1093/bioinformatics/btn611.
18. Bailly-Bechet M, Braunstein A, Pagnani A, Weigt M, Zecchina R. Inference of sparse combinatorial-control networks from gene-expression data: a message passing approach. BMC Bioinformatics. 2010;11:355. https://doi.org/10.1186/1471-2105-11-355.
19. Kirkendall ES, Spires WL, Mottes TA, et al. Development and performance of electronic acute kidney injury triggers to identify pediatric patients at risk for nephrotoxic medication-associated harm. Appl Clin Inform. 2014;5(2):313-333. https://doi.org/10.4338/ACI-2013-12-RA-0102.
20. Bedford M, Stevens P, Coulton S, et al. Development of Risk Models for the Prediction of New or Worsening Acute Kidney Injury on or During Hospital Admission: A Cohort and Nested Study. Southampton, UK: NIHR Journals Library; 2016. PubMed
21. Hodgson LE, Roderick PJ, Venn RM, Yao GL, Dimitrov BD, Forni LG. The ICE-AKI study: impact analysis of a clinical prediction rule and electronic AKI alert in general medical patients. PLoS One. 2018;13(8):e0200584. https://doi.org/10.1371/journal.pone.0200584.
22. Hammond DA, Smith MN, Li C, Hayes SM, Lusardi K, Bookstaver PB. Systematic review and meta-analysis of acute kidney injury associated with concomitant vancomycin and piperacillin/tazobactam. Clin Infect Dis. 2017;64(5):666-674. https://doi.org/10.1093/cid/ciw811.
23. Downes KJ, Cowden C, Laskin BL, et al. Association of acute kidney injury with concomitant vancomycin and piperacillin/tazobactam treatment among hospitalized children. JAMA Pediatr. 2017;171(12):e173219.https://doi.org/10.1001/jamapediatrics.2017.3219.
24. Agency for Heathcare Research and Quality. Alert Fatigue Web site. https://psnet.ahrq.gov/primers/primer/28/alert-fatigue. Updated July 2016. Accessed April 14, 2017.
25. Downes KJ, Rao MB, Kahill L, Nguyen H, Clancy JP, Goldstein SL. Daily serum creatinine monitoring promotes earlier detection of acute kidney injury in children and adolescents with cystic fibrosis. J Cyst Fibros. 2014;13(4):435-441. https://doi.org/10.1016/j.jcf.2014.03.005.
1. Siew ED, Davenport A. The growth of acute kidney injury: a rising tide or just closer attention to detail? Kidney Int. 2015;87(1):46-61. https://doi.org/10.1038/ki.2014.293.
2. Matuszkiewicz-Rowinska J, Zebrowski P, Koscielska M, Malyszko J, Mazur A. The growth of acute kidney injury: Eastern European perspective. Kidney Int. 2015;87(6):1264. https://doi.org/10.1038/ki.2015.61.
3. Hoste EA, Bagshaw SM, Bellomo R, et al. Epidemiology of acute kidney injury in critically ill patients: the multinational AKI-EPI study. Intensive Care Med. 2015;41(8):1411-1423. https://doi.org/10.1007/s00134-015-3934-7.
4. Kaddourah A, Basu RK, Bagshaw SM, Goldstein SL, AWARE Investigators. Epidemiology of acute kidney injury in critically ill children and young adults. N Engl J Med. 2017;376(1):11-20. https://doi.org/10.1056/NEJMoa1611391.
5. Soler YA, Nieves-Plaza M, Prieto M, Garcia-De Jesus R, Suarez-Rivera M. Pediatric risk, injury, failure, loss, end-stage renal disease score identifies acute kidney injury and predicts mortality in critically ill children: a prospective study. Pediatr Crit Care Med. 2013;14(4):e189-e195. https://doi.org/10.1097/PCC.0b013e3182745675.
6. Case J, Khan S, Khalid R, Khan A. Epidemiology of acute kidney injury in the intensive care unit. Crit Care Res Pract. 2013;2013:479730. https://doi.org/10.1155/2013/479730.
7. Rewa O, Bagshaw SM. Acute kidney injury-epidemiology, outcomes and economics. Nat Rev Nephrol. 2014;10(4):193-207. https://doi.org/10.1038/nrneph.2013.282.
8. Hsu RK, Hsu CY. The role of acute kidney injury in chronic kidney disease. Semin Nephrol. 2016;36(4):283-292. https://doi.org/10.1016/j.semnephrol.2016.05.005.
9. Menon S, Kirkendall ES, Nguyen H, Goldstein SL. Acute kidney injury associated with high nephrotoxic medication exposure leads to chronic kidney disease after 6 months. J Pediatr. 2014;165(3):522-527.https://doi.org/10.1016/j.jpeds.2014.04.058.
10. Neild GH. Life expectancy with chronic kidney disease: an educational review. Pediatr Nephrol. 2017;32(2):243-248. https://doi.org/10.1007/s00467-016-3383-8.
11. Kellum JA. Acute kidney injury: AKI: the myth of inevitability is finally shattered. Nat Rev Nephrol. 2017;13(3):140-141. https://doi.org/10.1038/nrneph.2017.11.
12. Mehta RL, Cerda J, Burdmann EA, et al. International Society of Nephrology’s 0by25 initiative for acute kidney injury (zero preventable deaths by 2025): a human rights case for nephrology. Lancet. 2015;385(9987):2616-2643. https://doi.org/10.106/S0140-6736(15)60126-X.13.
13. Hodgson LE, Sarnowski A, Roderick PJ, Dimitrov BD, Venn RM, Forni LG. Systematic review of prognostic prediction models for acute kidney injury (AKI) in general hospital populations. BMJ Open. 2017;7(9):e016591. https://doi.org/10.1136/bmjopen-2017-016591.
14. Sutherland SM. Electronic health record-enabled big-data approaches to nephrotoxin-associated acute kidney injury risk prediction. Pharmacotherapy. 2018;38(8):804-812. https://doi.org/10.1002/phar.2150.
15. KDIGO Work Group. KDIGO clinical practice guidelines for acute kidney injury. Kidney Int Suppl. 2012;2(1):S1-138. PubMed
16. Moffett BS, Goldstein SL. Acute kidney injury and increasing nephrotoxic-medication exposure in noncritically-ill children. Clin J Am Soc Nephrol. 2011;6(4):856-863. https://doi.org/10.2215/CJN.08110910.
17. Mukherjee S, Pelech S, Neve RM, et al. Sparse combinatorial inference with an application in cancer biology. Bioinformatics. 2009;25(2):265-271. https://doi.org/10.1093/bioinformatics/btn611.
18. Bailly-Bechet M, Braunstein A, Pagnani A, Weigt M, Zecchina R. Inference of sparse combinatorial-control networks from gene-expression data: a message passing approach. BMC Bioinformatics. 2010;11:355. https://doi.org/10.1186/1471-2105-11-355.
19. Kirkendall ES, Spires WL, Mottes TA, et al. Development and performance of electronic acute kidney injury triggers to identify pediatric patients at risk for nephrotoxic medication-associated harm. Appl Clin Inform. 2014;5(2):313-333. https://doi.org/10.4338/ACI-2013-12-RA-0102.
20. Bedford M, Stevens P, Coulton S, et al. Development of Risk Models for the Prediction of New or Worsening Acute Kidney Injury on or During Hospital Admission: A Cohort and Nested Study. Southampton, UK: NIHR Journals Library; 2016. PubMed
21. Hodgson LE, Roderick PJ, Venn RM, Yao GL, Dimitrov BD, Forni LG. The ICE-AKI study: impact analysis of a clinical prediction rule and electronic AKI alert in general medical patients. PLoS One. 2018;13(8):e0200584. https://doi.org/10.1371/journal.pone.0200584.
22. Hammond DA, Smith MN, Li C, Hayes SM, Lusardi K, Bookstaver PB. Systematic review and meta-analysis of acute kidney injury associated with concomitant vancomycin and piperacillin/tazobactam. Clin Infect Dis. 2017;64(5):666-674. https://doi.org/10.1093/cid/ciw811.
23. Downes KJ, Cowden C, Laskin BL, et al. Association of acute kidney injury with concomitant vancomycin and piperacillin/tazobactam treatment among hospitalized children. JAMA Pediatr. 2017;171(12):e173219.https://doi.org/10.1001/jamapediatrics.2017.3219.
24. Agency for Heathcare Research and Quality. Alert Fatigue Web site. https://psnet.ahrq.gov/primers/primer/28/alert-fatigue. Updated July 2016. Accessed April 14, 2017.
25. Downes KJ, Rao MB, Kahill L, Nguyen H, Clancy JP, Goldstein SL. Daily serum creatinine monitoring promotes earlier detection of acute kidney injury in children and adolescents with cystic fibrosis. J Cyst Fibros. 2014;13(4):435-441. https://doi.org/10.1016/j.jcf.2014.03.005.
© 2019 Society of Hospital Medicine